CN111143222A - Software evaluation method based on defect prediction - Google Patents
Software evaluation method based on defect prediction Download PDFInfo
- Publication number
- CN111143222A CN111143222A CN201911389043.2A CN201911389043A CN111143222A CN 111143222 A CN111143222 A CN 111143222A CN 201911389043 A CN201911389043 A CN 201911389043A CN 111143222 A CN111143222 A CN 111143222A
- Authority
- CN
- China
- Prior art keywords
- software
- test
- defect prediction
- testing
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a software evaluation method based on defect prediction, which comprises the following steps: a defect prediction stage, a software testing stage, a test evaluation stage and a test improvement stage. In the implementation process, firstly, the defect prediction is carried out to obtain a defect prediction result, then, the software test is carried out to record a test result, then, the defect prediction result is compared with the software test result, and the test improvement is carried out according to the comparison result, so that a better test result is obtained. The technical scheme of the invention can fully utilize the result of software defect prediction to effectively improve the quality and efficiency of software evaluation. The method can be used for guiding software research and development organizations and third-party software evaluation organizations to efficiently implement the test and evaluation of each stage of software research and development on the basis of software defect prediction.
Description
Technical Field
The invention relates to the technical field of software evaluation, in particular to a software evaluation method based on defect prediction.
Background
The software defect prediction is to predict the number or probability of defects contained in software with certain characteristics by using various historical data collected at each stage of the software life cycle. Depending on the metrology data used by the prediction method, defect prediction may be performed at various stages of the software lifecycle. The earliest software defect prediction models were proposed by Akiyama in the seventies of the twentieth century. Software defect prediction has been an important issue in software engineering research since then. After the twenty-first century, this research has significantly increased the temperature. People use software measurement sample data and mathematical tools with different characteristics respectively to provide various prediction models, and some preliminary conclusions are obtained in the aspects of measurement selection, prediction methods, evaluation indexes and the like.
The software defect prediction technology is applied to the improvement of the software evaluation process, is a new development in the technical field of the current software evaluation, and is expected to further improve the quality and efficiency of the software evaluation.
Disclosure of Invention
In order to improve the software evaluation process by using the software defect prediction technology and further improve the quality and efficiency of the software evaluation, the invention provides a software evaluation method based on defect prediction, which realizes the application of the software defect prediction technology in the software evaluation process.
In order to solve the problem, the invention adopts the following technical scheme:
a software evaluation method based on defect prediction is characterized by comprising the following steps:
a defect prediction stage, a software testing stage, a testing evaluation stage and a testing improvement stage.
In particular, the defect prediction phase accomplishes the following tasks:
determining software testing requirements, and defining the degree of software testing sufficiency;
according to the software testing requirement, defining a software target level of defect prediction and defining a program basic unit covered by software testing; collecting measurement data according to the software target level of the defect prediction;
determining a defect prediction model; preprocessing the measurement data according to the requirement of the defect prediction model, and determining basic attributes and data for executing prediction;
establishing a defect prediction model by using the determined basic attributes and data as test data, and determining an evaluation criterion of the defect prediction model;
and executing a test by using the constructed defect prediction model, and adjusting parameters of the defect prediction model according to a prediction result until an optimal result determined by the evaluation criterion of the defect prediction model is reached.
In particular, the software testing phase comprises the following steps: test design, test execution, and test recording.
In particular, the test evaluation phase accomplishes the following tasks:
determining a software testing sufficiency criterion according to software testing requirements;
checking whether the testing range of the software testing stage completely covers all contents of the testing requirement;
and comparing the defect prediction result with the software test result according to the software test sufficiency criterion.
If the defect prediction result is larger than the software test result, analyzing the problem distribution condition, and carrying out test improvement on the positions with more defect prediction problems;
if the number of the software test results is far greater than that of the defect prediction results, the defect prediction model is improved and confirmed;
and if the defect prediction result is close to the software test result and the test content is comprehensive, the requirement of the test sufficiency criterion is considered to be met, and the test is terminated.
In particular, the test improvement stage utilizes the results of the defect prediction to improve the test design, the improvement comprising:
concentrating the test to the modules with possible problems according to the distribution condition of the defect prediction result, adding test cases to the modules with possible problems according to the defect prediction result, and improving the test design method;
designing a new test case, and after the test design is improved, re-entering the software test stage until the software test result meets the requirement of the test sufficiency criterion.
In particular, the software object level of the defect prediction is a program cladding level, a file level, a class level and a method level.
In particular, the software testing phase develops software tests in accordance with test maturity level requirements and management requirements.
The software evaluation method based on the defect prediction can apply the software defect prediction technology to the software evaluation process, and improves the quality and efficiency of software evaluation and the accuracy of software defect prediction.
Drawings
Fig. 1 is a flowchart of a software evaluation method based on defect prediction according to an embodiment of the present invention.
Detailed Description
The invention is further illustrated by the following figures and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It is also to be noted that, for the convenience of description, only a part of the contents, not all of the contents, which are related to the present invention, are shown in the drawings, and unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a software evaluation method based on defect prediction according to an embodiment of the present invention.
The software evaluation method based on the defect prediction in the embodiment specifically comprises the following steps:
s110: a defect prediction stage;
s120: a software testing stage;
s130: a test evaluation stage;
s140: and (5) testing an improvement stage.
Specifically, the step S110: the defect prediction stage comprises:
s111: determining software testing requirements, and defining the degree of software testing sufficiency;
s112: according to the software test requirements, defining a software target level of defect prediction and defining a program basic unit covered by software test; collecting measurement data according to the software target level of the defect prediction;
s113: determining a defect prediction model; preprocessing the measurement data according to the requirement of the defect prediction model, and determining basic attributes and data for executing prediction;
s114: establishing a defect prediction model by using the determined basic attributes and data as test data, and determining an evaluation criterion of the defect prediction model;
s115: and executing a test by using the constructed defect prediction model, and adjusting parameters of the defect prediction model according to a prediction result until an optimal result determined by the evaluation criterion of the defect prediction model is reached.
The software target level of the defect prediction is a program cladding level, a file level, a class level and a method level.
In order to ensure the accuracy of the defect prediction model, the open source engineering data, the open source project data and the actual measurement project data can be comprehensively used for model construction.
Open source engineering data includes:
the mdp (metrics Data program) dataset of NASA (this Data originates from an IV & V (quality independent verification and validation) project group of NASA, containing 13 projects of metric and defect Data).
The data set of PROMISE (an open source item metric data set, containing 4 data sets).
The open source project data includes:
eclipse dataset (containing metric and defect information for 3 versions of Eclipse as a data case for an enterprise level project).
The measured item data includes:
measured data (2-3 data sets are selected) of completed third party test items.
Specifically, the step S120: the software testing phase comprises the following steps:
s121: testing and designing;
s122: executing the test;
s123: and (6) testing and recording.
And S120: the software testing stage can carry out testing work according to the testing maturity level requirement and the management requirement.
During the test recording process, detailed records are provided, and relevant reaction information of the test process, the testers and the tested products is collected
If the defect prediction information is needed to be used for distributing the test resources, the actually available test resources can be comprehensively considered by referring to the prediction result in the defect prediction stage for reasonable distribution.
If test resource allocation is not done with defect prediction, defect prediction and software testing may be done simultaneously.
Specifically, the S130: the test evaluation phase comprises the following steps:
s131: determining a software testing sufficiency criterion according to software testing requirements;
s132: checking whether the testing range of the software testing stage completely covers all contents of the testing requirement;
s133: and comparing the defect prediction result with the software test result according to the software test sufficiency criterion.
If the defect prediction result is larger than the software test result, analyzing the problem distribution condition, and carrying out test improvement on the positions with more defect prediction problems;
if the number of the software test results is far greater than that of the defect prediction results, the defect prediction model is improved and confirmed;
and if the defect prediction result is close to the software test result and the test content is comprehensive, the requirement of the test sufficiency criterion is considered to be met, and the test is terminated.
Specifically, the step S140: the test improvement stage comprises:
s141, concentrating the test to the modules possibly having problems according to the distribution condition of the defect prediction result, adding test cases to the modules possibly having problems according to the defect prediction result, and improving the test design method;
s142, designing a new test case, and after the test design is improved, re-entering the software test stage until the software test result meets the requirement of the test sufficiency criterion.
The technical scheme of the invention comprises four steps of a defect prediction stage, a software testing stage, a test evaluation stage and a test improvement stage, and specifically realizes the software evaluation based on the defect prediction. The technical scheme of the invention can fully utilize the result of software defect prediction to effectively improve the quality and efficiency of software evaluation. The method can be used for guiding software research and development organizations to efficiently implement the test and evaluation of each stage of software research and development on the basis of the defect prediction of the researched software; meanwhile, the method can also be used for guiding a third-party software evaluation mechanism to evaluate the software product on the basis of software defect prediction, and providing an operation basis for the software product.
It will be appreciated by those skilled in the art that various modifications may be made to the above-described method without departing from the spirit of the invention, and it is within the scope of the invention.
Claims (7)
1. A software evaluation method based on defect prediction is characterized by comprising the following steps: a defect prediction stage, a software testing stage, a testing evaluation stage and a testing improvement stage.
2. The software evaluation method based on defect prediction according to claim 1, wherein the defect prediction stage completes the following tasks:
determining software testing requirements, and defining the degree of software testing sufficiency;
according to the software testing requirement, defining a software target level of defect prediction and defining a program basic unit covered by software testing; collecting measurement data according to the software target level of the defect prediction;
determining a defect prediction model; preprocessing the measurement data according to the requirement of the defect prediction model, and determining basic attributes and data for executing prediction;
establishing a defect prediction model by using the determined basic attributes and data as test data, and determining an evaluation criterion of the defect prediction model;
and executing a test by using the constructed defect prediction model, and adjusting parameters of the defect prediction model according to a prediction result until an optimal result determined by the evaluation criterion of the defect prediction model is reached.
3. The software evaluation method based on defect prediction according to claim 2, wherein the software object level of defect prediction is program package level, file level, class level, method level.
4. The software evaluation method based on defect prediction according to claim 1, wherein the software testing phase comprises the steps of: test design, test execution, and test recording.
5. The software evaluation method based on defect prediction according to claim 4, wherein the software testing stage performs software testing according to testing maturity level requirements and management requirements.
6. The software evaluation method based on defect prediction according to claim 1, characterized in that the test evaluation phase accomplishes the following tasks:
determining a software testing sufficiency criterion according to software testing requirements;
checking whether the testing range of the software testing stage completely covers all contents of the testing requirement;
comparing the defect prediction result with the software test result according to the software test sufficiency criterion, and if the defect prediction result is greater than the software test result, analyzing the problem distribution condition and carrying out test improvement on the positions with more defect prediction problems; if the number of the software test results is far greater than that of the defect prediction results, the defect prediction model is improved and confirmed; and if the defect prediction result is close to the software test result and the test content is comprehensive, the requirement of the test sufficiency criterion is considered to be met, and the test is terminated.
7. The software evaluation method based on defect prediction as claimed in claim 6, wherein the test improvement stage utilizes the result of the defect prediction to improve the test design, the improvement comprising: concentrating the test to the modules with possible problems according to the distribution condition of the defect prediction result, adding test cases to the modules with possible problems according to the defect prediction result, and improving the test design method; designing a new test case, and after the test design is improved, re-entering the software test stage until the software test result meets the requirement of the test sufficiency criterion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911389043.2A CN111143222A (en) | 2019-12-30 | 2019-12-30 | Software evaluation method based on defect prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911389043.2A CN111143222A (en) | 2019-12-30 | 2019-12-30 | Software evaluation method based on defect prediction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111143222A true CN111143222A (en) | 2020-05-12 |
Family
ID=70521496
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911389043.2A Pending CN111143222A (en) | 2019-12-30 | 2019-12-30 | Software evaluation method based on defect prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111143222A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131108A (en) * | 2020-09-18 | 2020-12-25 | 电信科学技术第十研究所有限公司 | Test strategy adjusting method and device based on characteristic attributes |
| CN112416783A (en) * | 2020-11-25 | 2021-02-26 | 武汉联影医疗科技有限公司 | Method, device, equipment and storage medium for determining software quality influence factors |
| CN112506784A (en) * | 2020-12-16 | 2021-03-16 | 上海嗨酷强供应链信息技术有限公司 | System and method for evaluating product performance with autonomous learning capability |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810101A (en) * | 2014-02-19 | 2014-05-21 | 北京理工大学 | Software defect prediction method and system |
| CN109242363A (en) * | 2018-11-01 | 2019-01-18 | 大连飞创信息技术有限公司 | Full life cycle test management platform based on multiple quality control models |
| CN109634833A (en) * | 2017-10-09 | 2019-04-16 | 北京京东尚科信息技术有限公司 | A kind of Software Defects Predict Methods and device |
| CN109976998A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of Software Defects Predict Methods, device and electronic equipment |
| CN110147321A (en) * | 2019-04-19 | 2019-08-20 | 北京航空航天大学 | A kind of recognition methods of the defect high risk module based on software network |
| CN110471856A (en) * | 2019-08-21 | 2019-11-19 | 大连海事大学 | A kind of Software Defects Predict Methods based on data nonbalance |
-
2019
- 2019-12-30 CN CN201911389043.2A patent/CN111143222A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810101A (en) * | 2014-02-19 | 2014-05-21 | 北京理工大学 | Software defect prediction method and system |
| CN109634833A (en) * | 2017-10-09 | 2019-04-16 | 北京京东尚科信息技术有限公司 | A kind of Software Defects Predict Methods and device |
| CN109976998A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of Software Defects Predict Methods, device and electronic equipment |
| CN109242363A (en) * | 2018-11-01 | 2019-01-18 | 大连飞创信息技术有限公司 | Full life cycle test management platform based on multiple quality control models |
| CN110147321A (en) * | 2019-04-19 | 2019-08-20 | 北京航空航天大学 | A kind of recognition methods of the defect high risk module based on software network |
| CN110471856A (en) * | 2019-08-21 | 2019-11-19 | 大连海事大学 | A kind of Software Defects Predict Methods based on data nonbalance |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131108A (en) * | 2020-09-18 | 2020-12-25 | 电信科学技术第十研究所有限公司 | Test strategy adjusting method and device based on characteristic attributes |
| CN112131108B (en) * | 2020-09-18 | 2024-04-02 | 电信科学技术第十研究所有限公司 | Feature attribute-based test strategy adjustment method and device |
| CN112416783A (en) * | 2020-11-25 | 2021-02-26 | 武汉联影医疗科技有限公司 | Method, device, equipment and storage medium for determining software quality influence factors |
| CN112416783B (en) * | 2020-11-25 | 2022-05-20 | 武汉联影医疗科技有限公司 | Method, device, equipment and storage medium for determining software quality influence factors |
| CN112506784A (en) * | 2020-12-16 | 2021-03-16 | 上海嗨酷强供应链信息技术有限公司 | System and method for evaluating product performance with autonomous learning capability |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111143222A (en) | Software evaluation method based on defect prediction | |
| Felderer et al. | Integrating risk-based testing in industrial test processes | |
| CN112817865A (en) | Coverage precision test method and system based on componentized distributed system | |
| US8667458B2 (en) | System and method to produce business case metrics based on code inspection service results | |
| Chandra et al. | Improving software quality using machine learning | |
| CN101082876A (en) | Software automatically evaluating tool bag | |
| Wang et al. | Measuring and improving software process in China | |
| Plöesch et al. | On the validity of the it-cisq quality model for automatic measurement of maintainability | |
| Yakovyna et al. | The relation between software development methodologies and factors affecting software reliability | |
| JP3883449B2 (en) | Software system test plan creation support method and test plan creation support program | |
| Ozakinci et al. | The role of process in early software defect prediction: Methods, attributes and metrics | |
| CN110377525B (en) | Parallel program performance prediction system based on runtime characteristics and machine learning | |
| Hryszko et al. | Cost effectiveness of software defect prediction in an industrial project | |
| KR20060063349A (en) | System and method for support of embedded software development methodology with quantitative process management | |
| CN114791883B (en) | Program automation error positioning method and system based on high-order variation | |
| Coba et al. | Simulation-based approach to apply uncertainty evaluation framework, for PSS economic models | |
| CN109344079A (en) | Placement-and-routing's regression testing method, system, equipment and storage medium | |
| Ho et al. | A prototype tool supporting when-to-release decisions in iterative development | |
| CN115629956A (en) | Software defect management method and system based on interface automatic test | |
| Tripathi et al. | Improving software quality based on relationship among the change proneness and object oriented metrics | |
| Caballero et al. | MMPRO: A Methodology Based on ISO/IEC 15939 to Draw Up Data Quality Measurement Processes. | |
| Liu et al. | Agent-based online quality measurement approach in cloud computing environment | |
| CN112463126A (en) | Software development method based on information technology consulting technology | |
| CN106855840B (en) | System CPU analysis method and device | |
| Kulovits et al. | Scalable preservation decisions: A controlled case study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200512 |