CN110865939A - Application program quality monitoring method and device, computer equipment and storage medium - Google Patents
Application program quality monitoring method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110865939A CN110865939A CN201911092538.9A CN201911092538A CN110865939A CN 110865939 A CN110865939 A CN 110865939A CN 201911092538 A CN201911092538 A CN 201911092538A CN 110865939 A CN110865939 A CN 110865939A
- Authority
- CN
- China
- Prior art keywords
- application program
- test
- quality
- module
- test case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 94
- 238000012544 monitoring process Methods 0.000 title claims abstract description 32
- 238000012360 testing method Methods 0.000 claims abstract description 250
- 238000011156 evaluation Methods 0.000 claims abstract description 41
- 230000006870 function Effects 0.000 claims description 103
- 230000007547 defect Effects 0.000 claims description 59
- 230000008569 process Effects 0.000 claims description 46
- 238000004590 computer program Methods 0.000 claims description 20
- 230000008439 repair process Effects 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 description 8
- 238000011161 development Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000007635 classification algorithm Methods 0.000 description 3
- 238000012806 monitoring device Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to an application program quality monitoring method and device, computer equipment and a storage medium. Relates to the technical field of computers. The method comprises the following steps: firstly, acquiring a test case set, wherein the test case set comprises at least one test case; then, testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module; determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold; and finally, evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program. By adopting the method, the time for evaluating the quality of the application program can be shortened.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for monitoring application program quality, a computer device, and a storage medium.
Background
As computer technology has evolved, the development of applications has become commonplace. During the development of the application, the quality of the application needs to be monitored. Conventionally, the quality monitoring of applications can be achieved using behavior driven development (English: BDD) techniques.
In the related process of monitoring the quality of the application program by using the BDD technology, a test case needs to be formulated in advance, and then the application program is tested by using the test case to obtain the execution result of the test case. And then, manually analyzing the execution result, compiling a test report, and finally evaluating the quality and risk of the application program according to the test report.
However, in the related process of monitoring the quality of the application program by using the BDD technology, the quality and the risk of the application program can be known only by manually analyzing the execution result and writing a test report. Therefore, in the case where the number of test cases is large, there is a problem in that it takes a long time to evaluate the quality of the application.
Disclosure of Invention
In view of the above, it is necessary to provide an application quality monitoring method, apparatus, computer device and storage medium capable of shortening the time period.
In a first aspect, a method for monitoring application program quality is provided, the method comprising:
acquiring a test case set, wherein the test case set comprises at least one test case;
testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module;
determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold;
and evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program.
In one embodiment, the determining the target function module in the application program according to the test result includes:
obtaining the fault probability of each functional module output by the membership function according to the test result by using the membership function;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible fault larger than the preset probability threshold value as the target functional module.
In one embodiment, the membership function is constructed according to a Bayesian classification function or a hidden Markov computational model.
In one embodiment, the evaluating the quality index corresponding to the target function module to obtain the evaluation information of the application program includes:
and inputting the numerical value of the quality index corresponding to the target function module into a fuzzy inference model to obtain the evaluation information output by the fuzzy inference model.
In one embodiment, the quality indicator includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In one embodiment, the fuzzy inference model comprises: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In one embodiment, the obtaining of the test case set includes:
and acquiring labels of the test case and the functional module corresponding to the test case, wherein the labels are used for indicating the functional module to which the test case belongs.
In a second aspect, an apparatus for monitoring application quality is provided, the apparatus comprising:
the system comprises an acquisition module, a test case collection module and a test result generation module, wherein the acquisition module is used for acquiring the test case collection which comprises at least one test case;
the test module is used for testing the application program by using the test cases in the test case set to obtain a test result, and the application program comprises at least one functional module;
the determining module is used for determining a target function module in the application program according to the test result, and the probability of the target function module having a fault is greater than a preset probability threshold;
and the evaluation module is used for evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, and the evaluation information is used for indicating the quality and the risk of the application program.
In one embodiment, the determining module is specifically configured to obtain, by using a membership function, a probability of failure of each functional module output by the membership function according to the test result;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible fault larger than the preset probability threshold value as the target functional module.
In one embodiment, the membership function is constructed according to a Bayesian classification function or a hidden Markov computational model.
In one embodiment, the evaluation module is specifically configured to input the numerical value of the quality index corresponding to the target function module into a fuzzy inference model, so as to obtain the evaluation information output by the fuzzy inference model.
In one embodiment, the quality indicator includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In one embodiment, the fuzzy inference model comprises: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In one embodiment, the obtaining module is specifically configured to obtain the test case and a label of the functional module corresponding to the test case, where the label is used to indicate the functional module to which the test case belongs.
In a third aspect, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the first aspects when executing the computer program.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method of any of the first aspects described above.
According to the application program quality monitoring method, the application program quality monitoring device, the computer equipment and the storage medium, firstly, a test case set is obtained, wherein the test case set comprises at least one test case; then, testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module; determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold; and finally, evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program. The quality and risk of the application program can be evaluated without manually compiling a test report, so that the application program quality monitoring method provided by the application program shortens the time for evaluating the quality of the application program to a certain extent.
Drawings
FIG. 1 is a diagram illustrating an exemplary application scenario of a method for monitoring application quality;
FIG. 2 is a flow diagram illustrating a method for application quality monitoring in one embodiment;
FIG. 3 is a flow diagram that illustrates the determination of a target function module in an application, according to one embodiment;
FIG. 4 is a block diagram of an embodiment of an application quality monitoring apparatus;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
As computer technology has evolved, the development of applications has become commonplace. During the development of the application, the quality of the application needs to be monitored. Conventionally, the quality monitoring of applications can be achieved using behavior driven development (English: BDD) techniques. The BDD provides a universal, simple and structured description language which can be English or other local languages, and the BDD can conveniently enable developers and common people to communicate with the requirements of application programs very smoothly.
In the process of carrying out quality monitoring on the application program by using the related BDD technology, a test case needs to be formulated in advance, wherein the test case refers to the description of a test task for a specific application program and embodies a test scheme, a method, a technology and a strategy. The contents of the test object, the test environment, the input data, the test steps, the expected results, the test scripts and the like are included, and finally, a document is formed. It is simply considered that a test case is a set of test inputs, execution conditions, and expected results tailored for a particular purpose to verify that the requirements of a particular application are met. And then testing the application program by using the test case to obtain an execution result of the test case. And then, manually analyzing the execution result, compiling a test report, and finally evaluating the quality and risk of the application program according to the test report.
However, in the related process of monitoring the quality of the application program by using the BDD technology, the quality and the risk of the application program can be known only by manually analyzing the execution result and writing a test report. Therefore, in the case where the number of test cases is large, there is a problem in that it takes a long time to evaluate the quality of the application.
The application program quality monitoring method provided by the application program quality monitoring method can be applied to the application environment shown in fig. 1. Wherein the terminal 101 communicates with the server 102 via a network connection. The server 102 may receive a test case set sent by the terminal 101, then the server 102 may test the application program by using the test case, then the server 102 may determine a target function module (i.e., a function module having a probability of failure exceeding a preset probability threshold) according to a test result obtained by the test, then the server 102 may evaluate a quality index generated by the target function module in a test process to obtain evaluation information, and send the evaluation information to the terminal 102, and the terminal 102 may display the evaluation information, where the evaluation information may indicate quality and risk of the application program.
It should be noted that, in some possible implementations, the implementation environment related to the application quality monitoring method provided by the present application may only include the terminal 101 or only include the server 102.
The terminal 101 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 102 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In the embodiment of the present application, as shown in fig. 2, an application quality monitoring method is provided, which is described by taking an example that the method is applied to the server in fig. 1, and includes the following steps:
The test cases are a small amount of test data which are carefully designed for efficiently finding the defects of the application program, and are the basis for the application program test.
In this step, the test cases may be divided into different categories according to the functional modules of the application program, and then the corresponding test cases are selected to test the application program according to the requirements during actual testing. For example, the application program is divided into a user management function module, a role management function module, an authority control function module, and the like according to functions. The test cases of user creation, user query, user information maintenance, user account opening, user account unlocking and user association can be divided into the user management function module. Then, in the actual test, only the test case corresponding to the user management function module can be selected to test the application program.
The application program includes at least one functional module.
In this step, when the application program is tested by using the test case, corresponding test results can be obtained, and the test results have a very important role in analyzing the quality of the application program in subsequent operations.
Typically, a test case will contain at least one step, and an expected result. For example, there is a test case for logging in an application, and the test case includes three steps of inputting an account number, inputting a password, and clicking a login button. The expected result is that the application will switch to a login-successful interface if the entered account number and password are both correct and the login button is clicked correctly.
In an actual test, when the application program is tested by using the test case, two test results can be obtained, and login is successful or failed. If the login fails, the specific case of login failure can be analyzed. Specifically, the specific steps that have problems can be analyzed to find out which problems exist in the process of developing the application.
The probability of the target function module failing is greater than a preset probability threshold.
In this step, specifically, the analysis may be performed with emphasis on the portion in which the problem occurs in the test result, so as to infer the portion in which the failure may occur in the application program. Then in subsequent operations, these potentially malfunctioning parts are monitored with emphasis.
And 204, evaluating the quality index corresponding to the target function module by the server to obtain evaluation information of the application program.
The assessment information is used to indicate the quality and risk of the application.
In this step, after the target function module is determined, important monitoring needs to be performed for the target function module. Specifically, the quality index data generated in the test process of the test case corresponding to the target function module can be analyzed, so that the quality and risk of the whole application program can be known. The quality and risk of the application can be judged according to whether the application is reliable or not, the average time that the application can normally run or not, whether the application is easy to expand or not, the ratio of the number of code annotation lines of the application to the number of code lines and whether the format of the code annotation of the application is uniform and clear or not.
In the application program quality monitoring method, firstly, a test case set is obtained, wherein the test case set comprises at least one test case; then, testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module; determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold; and finally, evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program. The quality and risk of the application program can be evaluated without manually compiling a test report, so that the application program quality monitoring method provided by the application program shortens the time for evaluating the quality of the application program to a certain extent.
In an embodiment of the present application, please refer to fig. 3, which provides a method for determining a target function module in an application, the method includes:
Membership functions, also known as membership functions or fuzzy metafunctions, are functions that are used in fuzzy sets and indicate whether an element in a set belongs to a particular subset. The indicator function of an element may have a value of 0 or 1, and the membership function of an element may have a value between 0 and 1, indicating the "trueness" of the element to some fuzzy set.
In this step, the membership function may be used to determine the probability of failure of each functional module, or the membership function may be used to determine the true degree of each functional module belonging to a failed module.
In step 302, the server screens out the functional modules with the probability of possible failure greater than the preset probability threshold from the probabilities of possible failure of the functional modules.
In this step, for example, the preset probability threshold is 0.7, and when the membership function predicts that the true degree of each functional module belonging to the faulty module is 0.4, 0.5, and 0.8, respectively. Wherein 0.8 is greater than the preset probability threshold of 0.7, then it can be considered that the functional module corresponding to 0.8 has a fault.
In step 303, the server determines the functional module with the probability of possible failure greater than the preset probability threshold as the target functional module.
Based on the above steps, after the functional module corresponding to 0.8 is screened, at this time, it may be considered that the functional module corresponding to 0.8 has a fault, and then the functional module corresponding to 0.8 is taken as the target functional module. In subsequent monitoring, the quality index corresponding to the functional module can be analyzed with emphasis. Thereby assessing the quality and risk of the application.
In the embodiment of the present application, the membership function is constructed according to a bayesian classification function or a hidden markov calculation model.
The Bayes classification is an irregular classification method, and a Bayes classification function learns and induces a classification function by training a classified sample subset, and realizes classification of unclassified data by using a classifier obtained by training. The application effect of the method is better than that of a neural network classification algorithm and a decision tree classification algorithm, and particularly when the data volume to be classified is very large, the Bayesian classification method has high accuracy compared with other classification algorithms.
The hidden Markov model is a dynamic Bayesian network and is mainly used for time series data modeling.
In the embodiment of the application, the membership function is constructed by using a Bayesian classification function or a hidden Markov calculation model, so that the finally obtained membership function has better classification performance.
In an embodiment of the present application, a method for obtaining evaluation information of an application is provided, where the method includes: and the server inputs the numerical value of the quality index corresponding to the target function module into the fuzzy inference model to obtain the evaluation information output by the fuzzy inference model.
Fuzzy inference is a system that defines input, output and state variables on a fuzzy set, and is a generalization of deterministic systems. The fuzzy system is based on macroscopical view, grasps the fuzzy characteristic of human brain thinking, has the advantages in describing high-level knowledge, and can simulate the comprehensive inference of human to process the fuzzy information processing problem which is difficult to solve by the conventional mathematical method.
In this embodiment, optionally, the values of the defect repair rate, the repaired number, the unrepaired number, the secondary failure rate, and the test pass rate corresponding to the target function module may be input into the fuzzy inference model. Then the fuzzy reasoning model can calculate and cluster-analyze the fuzzy relation of the data, and then fuzzy mapping and comprehensive judgment are carried out on the basis, and the obtained result can be used for judging the quality and the risk of an application program.
In the embodiment of the application, because a fuzzy inference model is used, the problem of nonlinearity can be solved well, for example, the problem of how to evaluate the quality of an application program in the application can be solved.
In the embodiment of the present application, the quality index includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In the embodiment of the application, the quality index can be used for performing quality evaluation and risk evaluation of the application program. The quality index mainly selects data of faults and defects of the test case in the test process. The data and the fuzzy reasoning model can be used for better deducing the quality and the fault of the application program.
In the embodiment of the present application, the fuzzy inference model includes: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In the embodiment of the present application, optionally, the quality fuzzy inference model predicts a specific value, and the value can be used to measure the quality of the application program. The risk fuzzy inference model can predict the probability of the application program failure, and a user can judge whether the application program is reliable or not according to the quality value of the application program and the probability value of the application program failure.
For example, the values of the quality indexes may be taken as percentages, a radar map is drawn according to the specific percentage values, and finally the area value of the obtained radar map is calculated, where the area value represents the value obtained by fusing the values of the quality indexes together, and the value may be used as a standard for evaluating the quality of the application program.
In the embodiment of the application, a quality fuzzy inference model and a risk fuzzy inference model are constructed on the basis of the fuzzy inference model. The quality information and the risk information of the application program can be obtained simultaneously, so that the application program can be more comprehensively evaluated by utilizing the two models.
In an embodiment of the present application, a method for obtaining a test case set is provided, where the method includes: the server obtains the test case and the label of the functional module corresponding to the test case, wherein the label is used for indicating the functional module to which the test case belongs.
In the embodiment of the present application, an application developer may divide an application into different functional modules according to differences of respective functions in the application. And then, corresponding test cases are formulated according to different functional modules, so that the specific situation of the application program can be considered more comprehensively when the test cases are formulated, and the formulated test cases can test the application program more comprehensively.
Furthermore, each test case has its corresponding functional module. Therefore, when a certain test case is used for testing the application program, if a fault occurs in the test, the fault reason can be quickly positioned to the functional module corresponding to the test case. And when the membership function predicts the target function module, the range of the next required quality index can be definitely found out according to the corresponding relation between the function module and the test case.
It should be understood that, although the respective steps in the flowcharts in fig. 2 to 3 are sequentially shown as indicated by arrows, the steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
In an embodiment of the present application, as shown in fig. 4, there is provided an application quality monitoring apparatus 400, including: an obtaining module 401, a testing module 402, a determining module 403 and an evaluating module 404, wherein:
an obtaining module 401, configured to obtain a test case set, where the test case set includes at least one test case;
a test module 402, configured to test an application program by using a test case in the test case set to obtain a test result, where the application program includes at least one functional module;
a determining module 403, configured to determine, according to the test result, a target function module in the application program, where a probability of a failure occurring in the target function module is greater than a preset probability threshold;
an evaluation module 404, configured to evaluate the quality index corresponding to the target function module to obtain evaluation information of the application program, where the evaluation information is used to indicate quality and risk of the application program.
In this embodiment, the determining module 403 is specifically configured to, by using a membership function, obtain, according to the test result, a probability that each functional module output by the membership function fails;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible fault larger than the preset probability threshold value as the target functional module.
In the embodiment of the present application, the membership function is constructed according to a bayesian classification function or a hidden markov calculation model.
In this embodiment of the application, the evaluation module 404 is specifically configured to input the numerical value of the quality index corresponding to the target function module into a fuzzy inference model, so as to obtain the evaluation information output by the fuzzy inference model.
In the embodiment of the present application, the quality index includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In the embodiment of the present application, the fuzzy inference model includes: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In this embodiment of the application, the obtaining module 401 is specifically configured to obtain the test case and a label of the function module corresponding to the test case, where the label is used to indicate the function module to which the test case belongs.
For specific limitations of the application quality monitoring device, reference may be made to the above limitations of the application quality monitoring method, which are not described herein again. The modules in the application quality monitoring device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store application quality monitoring data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an application quality monitoring method.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment of the present application, there is provided a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring a test case set, wherein the test case set comprises at least one test case;
testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module;
determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold;
and evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program.
In the embodiment of the present application, the processor, when executing the computer program, further implements the following steps:
obtaining the fault probability of each functional module output by the membership function according to the test result by using the membership function;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible fault larger than the preset probability threshold value as the target functional module.
In the embodiment of the present application, the membership function is constructed according to a bayesian classification function or a hidden markov calculation model.
In the embodiment of the present application, the processor, when executing the computer program, further implements the following steps:
and inputting the numerical value of the quality index corresponding to the target function module into a fuzzy inference model to obtain the evaluation information output by the fuzzy inference model.
In the embodiment of the present application, the quality index includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In the embodiment of the present application, the fuzzy inference model includes: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In the embodiment of the present application, the processor, when executing the computer program, further implements the following steps:
and acquiring labels of the test case and the functional module corresponding to the test case, wherein the labels are used for indicating the functional module to which the test case belongs.
In an embodiment of the application, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, realizes the steps of:
acquiring a test case set, wherein the test case set comprises at least one test case;
testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module;
determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold;
and evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program.
In an embodiment of the application, the computer program when executed by the processor further performs the steps of:
obtaining the fault probability of each functional module output by the membership function according to the test result by using the membership function;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible fault larger than the preset probability threshold value as the target functional module.
In the embodiment of the present application, the membership function is constructed according to a bayesian classification function or a hidden markov calculation model.
In an embodiment of the application, the computer program when executed by the processor further performs the steps of:
and inputting the numerical value of the quality index corresponding to the target function module into a fuzzy inference model to obtain the evaluation information output by the fuzzy inference model.
In the embodiment of the present application, the quality index includes:
a defect repair rate, which is used to characterize the ratio of the number of repaired defects to the number of total defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, which is used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
In the embodiment of the present application, the fuzzy inference model includes: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program is possible to fail.
In an embodiment of the application, the computer program when executed by the processor further performs the steps of:
and acquiring labels of the test case and the functional module corresponding to the test case, wherein the labels are used for indicating the functional module to which the test case belongs.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for application quality monitoring, the method comprising:
acquiring a test case set, wherein the test case set comprises at least one test case;
testing an application program by using the test cases in the test case set to obtain a test result, wherein the application program comprises at least one functional module;
determining a target function module in the application program according to the test result, wherein the probability of the target function module having a fault is greater than a preset probability threshold;
and evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, wherein the evaluation information is used for indicating the quality and risk of the application program.
2. The method of claim 1, wherein determining the target function module in the application according to the test result comprises:
obtaining the fault probability of each functional module output by the membership function according to the test result by using the membership function;
screening out the functional modules with the probability of possible faults larger than the preset probability threshold from the probability of possible faults of each functional module;
and determining the functional module with the probability of possible failure greater than the preset probability threshold value as the target functional module.
3. The method of claim 2, wherein the membership functions are constructed according to a bayesian classification function or a hidden markov model.
4. The method according to claim 1, wherein the evaluating the quality indicator corresponding to the target function module to obtain the evaluation information of the application program comprises:
and inputting the numerical value of the quality index corresponding to the target function module into a fuzzy inference model to obtain the evaluation information output by the fuzzy inference model.
5. The method of claim 1, wherein the quality indicator comprises:
a defect repair rate, which is used for representing the ratio of the number of repaired defects to the number of all defects in the test process;
a repaired number, the repaired number being used to characterize the number of defects that have been repaired during the testing process;
an unrepaired number, the unrepaired number being used to characterize the number of defects that have not been repaired during the testing process;
the secondary failure rate is used for representing the ratio of the number of the defects which are repaired for more than one time to the number of all the defects in the testing process;
and the test passing rate is used for representing the ratio of the number of the test cases which normally run to the number of all the test cases in the test process.
6. The method of claim 1, wherein the fuzzy inference model comprises: the system comprises a quality fuzzy inference model and a risk fuzzy inference model, wherein the quality fuzzy inference model is used for inferring the defect rate of the application program, and the risk fuzzy inference model is used for inferring the probability that the application program possibly fails.
7. The method of claim 1, wherein obtaining the set of test cases comprises:
and acquiring the test case and a label of the functional module corresponding to the test case, wherein the label is used for indicating the functional module to which the test case belongs.
8. An application quality monitoring apparatus, the apparatus comprising:
the system comprises an acquisition module, a test case collection and a test result generation module, wherein the acquisition module is used for acquiring the test case collection which comprises at least one test case;
the test module is used for testing an application program by using the test cases in the test case set to obtain a test result, and the application program comprises at least one functional module;
the determining module is used for determining a target function module in the application program according to the test result, and the probability of the target function module having a fault is greater than a preset probability threshold;
and the evaluation module is used for evaluating the quality index corresponding to the target function module to obtain evaluation information of the application program, and the evaluation information is used for indicating the quality and risk of the application program.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911092538.9A CN110865939B (en) | 2019-11-11 | 2019-11-11 | Application program quality monitoring method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911092538.9A CN110865939B (en) | 2019-11-11 | 2019-11-11 | Application program quality monitoring method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110865939A true CN110865939A (en) | 2020-03-06 |
| CN110865939B CN110865939B (en) | 2023-11-03 |
Family
ID=69653701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911092538.9A Active CN110865939B (en) | 2019-11-11 | 2019-11-11 | Application program quality monitoring method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110865939B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297090A (en) * | 2021-06-11 | 2021-08-24 | 南方电网数字电网研究院有限公司 | System testing method, device, computer equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080104577A1 (en) * | 2006-10-27 | 2008-05-01 | Holden Ian G | Method, Apparatus and Computer Program Product for Determining a Relative Measure of Build Quality for a Built System |
| CN102622510A (en) * | 2012-01-31 | 2012-08-01 | 龚波 | System and method for quantitative management of software defects |
| CN104050377A (en) * | 2014-06-24 | 2014-09-17 | 国家电网公司 | Method for determining probability of time-varying equipment failures |
| CN106444703A (en) * | 2016-09-20 | 2017-02-22 | 西南石油大学 | Rotating equipment running state fuzzy evaluation and prediction methods based on occurrence probability of fault modes |
| CN109582579A (en) * | 2018-11-30 | 2019-04-05 | 腾讯音乐娱乐科技（深圳）有限公司 | Applied program testing method, device, electronic equipment and storage medium |
| US20190163617A1 (en) * | 2017-11-29 | 2019-05-30 | Toyota Jidosha Kabushiki Kaisha | Software quality determination apparatus, software quality determination method, and software quality determination program |
| CN109828919A (en) * | 2019-01-18 | 2019-05-31 | 深圳壹账通智能科技有限公司 | Test report automatic generation method, device, computer equipment and storage medium |
| CN110362484A (en) * | 2019-07-03 | 2019-10-22 | 北京航空航天大学 | Localization method, device, electronic equipment and the storage medium of software multiple faults |
-
2019
- 2019-11-11 CN CN201911092538.9A patent/CN110865939B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080104577A1 (en) * | 2006-10-27 | 2008-05-01 | Holden Ian G | Method, Apparatus and Computer Program Product for Determining a Relative Measure of Build Quality for a Built System |
| CN102622510A (en) * | 2012-01-31 | 2012-08-01 | 龚波 | System and method for quantitative management of software defects |
| CN104050377A (en) * | 2014-06-24 | 2014-09-17 | 国家电网公司 | Method for determining probability of time-varying equipment failures |
| CN106444703A (en) * | 2016-09-20 | 2017-02-22 | 西南石油大学 | Rotating equipment running state fuzzy evaluation and prediction methods based on occurrence probability of fault modes |
| US20190163617A1 (en) * | 2017-11-29 | 2019-05-30 | Toyota Jidosha Kabushiki Kaisha | Software quality determination apparatus, software quality determination method, and software quality determination program |
| CN109582579A (en) * | 2018-11-30 | 2019-04-05 | 腾讯音乐娱乐科技（深圳）有限公司 | Applied program testing method, device, electronic equipment and storage medium |
| CN109828919A (en) * | 2019-01-18 | 2019-05-31 | 深圳壹账通智能科技有限公司 | Test report automatic generation method, device, computer equipment and storage medium |
| CN110362484A (en) * | 2019-07-03 | 2019-10-22 | 北京航空航天大学 | Localization method, device, electronic equipment and the storage medium of software multiple faults |
Non-Patent Citations (1)
| Title |
|---|
| 李国栋;李庚银;杨晓东;周明;: "基于雷达图法的电能质量综合评估模型", 电力系统自动化, no. 14 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297090A (en) * | 2021-06-11 | 2021-08-24 | 南方电网数字电网研究院有限公司 | System testing method, device, computer equipment and storage medium |
| CN113297090B (en) * | 2021-06-11 | 2024-01-23 | 南方电网数字平台科技(广东)有限公司 | System test method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110865939B (en) | 2023-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111240975B (en) | Artificial intelligence system risk detection method, device, computer equipment and medium | |
| CN110929036A (en) | Electric power marketing inspection management method and device, computer equipment and storage medium | |
| CN110489314B (en) | Model anomaly detection method and device, computer equipment and storage medium | |
| CN112884092A (en) | AI model generation method, electronic device, and storage medium | |
| CN111177714A (en) | Abnormal behavior detection method and device, computer equipment and storage medium | |
| Punitha et al. | Software defect prediction using software metrics-A survey | |
| CN110083514B (en) | Software test defect evaluation method and device, computer equipment and storage medium | |
| CN110941555A (en) | Test case recommendation method and device, computer equipment and storage medium | |
| CN115841255B (en) | On-site early warning method and system for building engineering based on-line analysis | |
| CN110245085B (en) | Embedded real-time operating system verification method and system by using online model inspection | |
| CN111459796B (en) | Automated testing method, apparatus, computer device and storage medium | |
| CN115952081A (en) | Software testing method, device, storage medium and equipment | |
| Kapila et al. | Analysis of CK metrics to predict software fault-proneness using bayesian inference | |
| Zhang et al. | Anomaly detection of sensor faults and extreme events based on support vector data description | |
| CN111767192A (en) | Service data detection method, device, equipment and medium based on artificial intelligence | |
| Challagulla et al. | A unified framework for defect data analysis using the mbr technique | |
| CN110865939B (en) | Application program quality monitoring method, device, computer equipment and storage medium | |
| Schleiss et al. | Towards the quantitative verification of deep learning for safe perception | |
| CN112379913A (en) | Software optimization method, device, equipment and storage medium based on risk identification | |
| Oral et al. | Defect prediction for embedded software | |
| Awadid et al. | AI Systems Trustworthiness Assessment: State of the Art | |
| Farhad et al. | Keep your distance: determining sampling and distance thresholds in machine learning monitoring | |
| Kumari et al. | Comparing efficiency of software fault prediction models developed through binary and multinomial logistic regression techniques | |
| RU2699685C1 (en) | Method of analyzing and monitoring the state of a technical installation comprising a plurality of dynamic systems | |
| CN114091644A (en) | Technical risk assessment method and system for artificial intelligence product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |