JPH0619699A - Software release time estimating device and method - Google Patents

Abstract

PURPOSE:To attain a release time estimation with a constant reliability. CONSTITUTION:This device is equipped with an executing means 3 which executes a software to be tested according to a test case, collecting means 4-9, 11, 12a, 12b, and 12c which data-collect test time information being the cumulative time of the execution, number of bug detected at that time, number of bug correction, and time number required for the bug correction, estimating means 13A-1 to 13A-n,-13C-1 to 13C-n which estimate the release time of the software from each kind of statistical model from a relation between the test time and the transition of time correcting time based on the test time information, based on the information of the number of the bug non-correction obtained from the relation between the number of the bug detection and the number of the bug correction, and means 14, 15, and 16 which correct the estimates according to a prescribed rule based on an experience, integrately operate an estimation processing, and obtain the estimated result of the release time.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software product release time estimating device and an estimating method for estimating the releaseable time of software under development.

[0002]

2. Description of the Related Art When software under development is tested, a failure (feria) or a defect (fault) is found. Therefore, after that, the failure or defect (hereinafter referred to as a bug) is corrected. Then, testing and correction are repeated again, and as a result, the time when all the bugs are resolved is the time when the software is completed.

By the way, it is very difficult to determine when the software under development can be released.
However, it is important to determine the releaseable time from the viewpoint of reviewing the development schedule, calculating the development cost, improving the service to users, and gaining credit.

In general, bug correction is performed by preparing various test cases (simulated information for testing) that actually assume various situations for developed debugged software (program under test). By checking the existence of bugs such as failures (feria) and defects (faults) by trying to execute them, and fixing each of the bugs,
Follow the test case again, run it to check for the existence of a bug, and then repeat the process of fixing it.

In the conventional technology, the number of found bugs and the number of test times are found, the remaining number of bugs is estimated by the software reliability model, and the time required for finding the remaining bugs is estimated. Estimate the release time. Judgment at the time of software release was made based only on the cumulative curve of the number of bugs found in this way.

[0006]

As described above, according to the conventional method, the cumulative number of bugs such as failures and defects found by executing the software under development according to the prepared test case and viewing the software is used, and the reliability is improved. The total number of latent bugs is estimated by the sex estimation model, the total number of remaining bugs is known from the difference, and the time required to find it is estimated with reference to the correction time required for the already found bug, and its development The method of estimating the release time of the software is used.

In this case, the estimated value of the total number of latent bugs varies depending on the reliability estimation model to be used, and therefore the estimated release point differs greatly from our sense and experience. However, it was difficult to make a reliable estimation of the release time.

This is because, out of the numerous reliability estimation models, it is not known which one is the best to use due to the characteristics of the software whose reliability is to be estimated, and even if a bug is found, The correction work does not always correct all the discoveries each time it is discovered, and it is also due to neglecting the current situation that it is often postponed depending on the situation.

As described above, despite knowing the time at which a failure has occurred and the time at which correction has been made, the number of failures minus the number of modifications = the number of delayed modifications (the number of uncorrected) is not so much underestimated, and is still unresolved. Since I did not monitor the delay time for correction, I could not grasp the cause of the delay in the progress of the software development project,
Furthermore, it can be said that it was useless to discuss the reliability itself in the estimation of the release time point that ignored these factors.

Therefore, the object of the present invention is to
It is an object of the present invention to provide an apparatus and method for estimating a software release time point, which makes it possible to estimate a release time point with a certain reliability.

[0011]

In order to achieve the above object, the present invention is constructed as follows. That is, first, execution means for executing software to be evaluated according to a test case, which is a group of various conditions prepared for testing, cumulative test time information as cumulative time of execution by this execution means, and the evaluation. Test data collection means for collecting data on the number of detected failures / defects when the target software is executed, the number of failures / fixed defects, and the number of hours required to correct the failures / defects, and the collected data. Of the above, based on the information of the uncorrected number obtained from the relationship of the number of corrections to the number of detected failures / defects, based on the relationship between the test time based on the accumulated test time information and the change in the correction time, various statistics Means for estimating the release time of the software to be evaluated for each model by a statistical model, and accumulated for each of these estimated values After correcting in accordance with a predetermined rule based on experience (the rule consists of conditions and weights), by evaluating process comprehensively, provided a determining means for obtaining an estimation result of the time of release.

Further, secondly, execution means for executing the software to be evaluated according to a test case, which is a group of various conditions prepared for the test, cumulative test time information of the execution by this execution means, and the number of tests. And a test data collecting means for collecting data of the number of failures / defects detected when the software to be evaluated is executed, the number of failures / defects fixed, and the number of hours required to correct the failures / defects. Of these collected data, based on the cumulative test time and the number of failures / defects, various software reliability growth models are used to estimate the release time of the software to be evaluated for each model, and these are estimated values. First estimation means obtained as a group, and based on the correction time and the number of corrections among the collected data, various software reliability Second estimating means for estimating the release time of the software to be evaluated for each model by the model and obtaining these as an estimated value group; and the number of test hours, the number of modified hours, and the number of unmodified among the collected data. On the basis of the above, a third estimating means for estimating the release time of the software to be evaluated for each model by various statistical models and obtaining them as an estimated value group, and each of these estimating means are used for estimation. Judgment for obtaining the final estimation result at the release point by processing the release point estimated value for each of the estimated value groups according to a predetermined rule (a rule composed of conditions and weights) based on accumulated experience And means.

Thirdly, execution means for executing the software to be evaluated according to a test case, which is a group of various conditions prepared for the test, and cumulative test time information of execution by this execution means and information on the number of tests. Test data collecting means for collecting data on the number of detected faults / defects, the number of faults / defects fixed, and the number of hours required for faults / fixed defects when the software to be evaluated is executed, and the collection. Based on the accumulated test time and the number of failures / defects among the data collected, various software reliability growth models are used to estimate the release time of the software to be evaluated for each model, and these are obtained as an estimated value group. Based on the first estimating means and various modification times and the number of modifications of the collected data, various software reliability growth models are used. The second estimating means for estimating the release time of the software to be evaluated for each model as an estimated value group and the number of the detected failures / defects in the data collected by the data collecting means Based on the information of the uncorrected number obtained from the relationship of the number of modifications, from the relationship with the transition of the test time and the modification time based on the cumulative test time information, various statistical models are used to determine the release time of the software to be evaluated. According to a predetermined rule based on accumulated experience, a third estimating means for estimating each model and obtaining them as an estimated value group and each estimated value group for each of the first estimating means to the third estimating means After the correction, the first judgment means for obtaining the estimation result at the time of release for each estimated value group by performing the comprehensive evaluation processing for each estimated value group, and each of these estimated values The second judgment means obtains the final estimation result at the release point by processing the estimation result at another release point according to a predetermined rule (a rule composed of a condition and weight) based on accumulated experience. .

Fourth, an execution step for executing the software to be evaluated according to a test case which is a group of various conditions prepared for the test, cumulative test time information of the execution by this execution step, and the number of tests A test data collecting step of collecting information and the number of faults / defects detected when the software to be evaluated is executed, the number of faults / defects fixed, and the number of hours required for faults / defects fixed, Of the data collected in this test data collection step, based on the cumulative test time and the number of failures / defects, various software reliability growth models are used to estimate the release time of the software to be evaluated for each model. A first estimation step of obtaining these as an estimated value group, and a correction time of the collected data A second estimating step of estimating the release time of the software to be evaluated for each model by various software reliability growth models based on a positive number to obtain these as an estimated value group; and the data collecting step. Of the collected data, based on the information of the uncorrected number obtained from the relationship of the number of corrections to the number of detected failures / defects, from the relationship with the transition of the test time and the correction time based on the cumulative test time information A third estimation step of estimating the release time of the software to be evaluated for each model by various statistical models to obtain these as an estimated value group, and each estimation in each of the first estimation step to the third estimation step After correcting the value group according to a predetermined rule based on accumulated experience,
The first judgment step for obtaining the estimation result at the release time point for each estimated value group by performing the comprehensive evaluation process for each estimated value group, and the estimation result at the release time point for each estimated value group are accumulated. The final estimation result at the time of release is obtained by processing according to a predetermined rule based on the experience.
The decision step of.

[0015]

In the above, in the case of the first configuration, the execution means executes the software to be evaluated according to the test cases which are various condition groups prepared for the test, and the test data collection means is executed by this execution means. Cumulative test time information, which is the cumulative time of execution, and the number of faults / defects detected when the software to be evaluated was executed, the number of faults / defects fixed, and the number of hours required to correct the faults / defects. Collect data. Then, the estimating means is based on the cumulative test time information based on the information of the uncorrected number obtained from the relationship of the corrected number to the detected number of failures / defects in the data collected by the test data collecting means. From the relationship between the transition of the test time and the modification time, the release time of the software to be evaluated is estimated for each model by various statistical models. Then, the comprehensive judgment means, for each of these estimated values estimated by the estimation means,
After the correction is performed according to a predetermined rule (a rule composed of conditions and weights) based on the accumulated experience, an overall evaluation process is performed to obtain an estimated result at the time of release.

Here, the number of failures and defects found as a result of the execution of the program under test using the test case is used, and various statistical models are obtained by using the time series data of the failures and the number of defects which are transitions thereof. The software release point is estimated by the method. This is because the number of failures and defects found as the test progresses approaches the total number of failures and defects inherent in the program under test. If this is done, it can be considered that all latent faults and defects originally due to the program under test have been fixed, and it can be said that this time point is a time point at which release is possible. ， This discovery failure against the time when the defect is considered exhausted ，
It is to estimate the release time by statistically judging the transition of the correction time to estimate the release time, which is the time required to correct all the defects, and to estimate the release time by a method that determines the limit based on the transition of time series data. Moreover, since the final release time is estimated in consideration of the knowledge of the experienced person, a more realistic value can be provided.

In the second configuration, the execution means for executing the software to be evaluated according to the test case, which is a group of various conditions prepared for the test, performs the test run and the test data collection means As a result, cumulative test time information, information on the number of tests, the number of failures / defects detected when the software to be evaluated is executed, the number of failures / defects fixed, and the number of hours required to correct the failures / defects. To collect data. Then, based on the cumulative test time and the number of failures / defects among the data collected by the test data collecting means, the first estimating means uses the various software reliability growth models to release the software to be evaluated. Is estimated for each model to obtain these as an estimated value group. The second estimating means estimates the release time of the software to be evaluated for each model by various software reliability growth models based on the modification time and the number of modifications of the collected data. As an estimated value group, and the third estimating means uses various statistical models of the software to be evaluated by various statistical models based on the number of test hours, the number of modified hours, and the number of unmodified among the collected data. The release time points are estimated for each model to obtain these as an estimated value group. Then, the judging means follows a predetermined rule (a rule composed of conditions and weights) based on accumulated experience with respect to the release time point estimated value for each estimated value group estimated by each of these estimating means. By processing, the final estimation result at the time of release is obtained.

In the third configuration, the executing means executes the software to be evaluated according to the test cases which are various condition groups prepared for the test, and the test data collecting means executes the accumulated test time of the execution by the executing means. Information and information on the number of tests and the number of detected failures / defects and failures when the software to be evaluated is executed.
Data is collected on the number of defect corrections and the number of hours required for failure / defect correction. Then, the first estimating means determines the release time of the software to be evaluated by various software reliability growth models based on the cumulative test time and the number of failures / defects in the data collected by the test data collecting means. Each model is estimated and obtained as an estimated value group, and the second estimating means uses the various software reliability growth models to evaluate the evaluation target based on the modification time and the modification number in the collected data. The software release time is estimated for each model to obtain these as an estimated value group, and the third estimating means is a correction number for the number of the detected failures / defects in the data collected by the data collecting means. Based on the information on the number of uncorrected data obtained from the relationship between The total model to estimate the time of release of the evaluation target software by the model obtaining them as estimate value group.

Then, the first judging means corrects each estimated value group for each of the first estimating means to the third estimating means in accordance with a predetermined rule based on accumulated experience, and then, each estimated value is corrected. An estimation result at each release time point for each estimated value group was obtained by comprehensively performing evaluation processing for each group, and the estimated result at the release time point for each estimated value group by the first determination means was accumulated. The final estimation result at the time of release is obtained by processing according to a predetermined rule (a rule composed of conditions and weights) based on experience.

In the present invention, in order to estimate the release time of the software, the program under test is executed using the test case, and as a result, the number of bugs found is used,
Using the time-series data of the number of bugs, which is the transition, the software release time is estimated by various software reliability growth models, and the need for correction due to the discovery of a bug causes the bug to be fixed. The required time is registered as a modification time, the software release time is estimated from the time series data showing the transition of the modification time, and the number of bugs found at the time of testing and the time series data of the number of modifications to that bug are also recorded. Since the number of uncorrected cases can be calculated based on, the time of software release is estimated by the time series data showing the transition of the number of uncorrected cases.

Then, changing various points of interest,
That is, by applying rules based on past experience to each release time estimation result by different estimation methods, the final release time estimation result is obtained. Since the release time of the software is estimated in a form that incorporates information and experience, it becomes possible to provide a more realistic value and to estimate the release time with a certain degree of reliability. A software release time point estimation device and method can be provided.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. The present invention makes it possible to reliably estimate the release time of software under development, and based on time-series data of software test information (test time, number of bugs, cumulative bug fix time, number of fixes). Based on the relationship between the test time and the number of bugs, the relationship between the cumulative bug fix time and the number of fixes, and the relationship between the test time and the cumulative bug fix time and the number of unfixed bugs, there are multiple reliability estimation models and multiple statistical models. Estimated release time points by each are calculated, weighted correction is applied to these calculation results according to rules based on accumulated experience and intuition of experienced software development engineers, and then added to use various models and parameters. The objective is to obtain the release point as an objective estimate from a comprehensive viewpoint.

Regarding the total number of bugs hidden in the program under test, the relationship between the number of bugs found by the test and the number of fixed and uncorrected numbers is as shown in FIG. That is, FIG.
In, the vertical axis represents the number, E (m) represents the total number of latent bugs, and the horizontal axis represents the time axis t. As the test progresses, the cumulative number of bugs found will gradually approach the total number of bugs latent in the program under test, and should eventually match.

On the other hand, since the found bugs will be fixed, the total number of bugs latent in the program under test will be reached if all the corrections are completed. The number of unfixed bugs indicates the difference between the number of fixed bugs and the transition of the found bugs. Therefore, if the ultimate arrival time is estimated from this relationship and the required time from the present time to this ultimate arrival time is estimated, this becomes the estimated value at the release time. By applying the above-mentioned various transitions used here to the model, the time when the limit is reached can be determined.

The characteristic concept of the device according to the present invention will be described. In this device, a program under test is executed by using a test case as a processing flow for estimating the release time of software, and as a result, it is found. The software release time is estimated using the number of bugs and the time series data of the number of bugs. Furthermore, when a bug is found and it is necessary to correct it, the failure is corrected, but the time required for the correction is registered as the correction time, and the software release time is estimated from the time series data showing the change in the correction time. To do. In addition, the number of unfixed cases can be calculated based on the number of bugs found at the time of testing and the time series data of the number of fixes for that bug, so software release with time series data showing the transition of this number of unfixed cases. Estimate the time point. By comparing the release time estimation results of the above three estimation methods by the comprehensive judgment means and applying the rule based on past experience,
Output the final result.

The details will be described below. FIG. 1 is a block diagram showing an embodiment of the present invention, and FIGS. 2 and 3 are partially enlarged views thereof. 1 to 3, 1 is a test case input unit, 2 is a storage device storing a program under test, 3 is a test case processing unit, 4 is a bug correction detection unit,
5 is a bug result accumulation unit, 6 is a test result expected value holding unit, 7
Is a bug detection unit, 8 is a corrected bug number detection unit, 9 is a first timer, 10 is a program correction unit, and 11 is a second timer.

Further, 12a to 12c are data storage units, 13A-1 to 13A-n, 13B-1 to 13B-, respectively.
n and 13C-1 to 13C-n are an estimation unit, 14 is a knowledge unit, 15 is a first determination unit, 16 is a second determination unit, and 1 is a determination unit.
Reference numeral 7 is a determination result extraction unit. The test case input means 1 is for inputting a test case (simulated information for testing). The test case has various parameters, specific data, commands, flags, etc., according to the purpose, and is prepared according to the purpose. In addition to the above-mentioned test case information from the test case input means 1,
As a result of execution according to the test case, the expected value, which is the result that should be originally obtained, should be input.

The storage device 2 stores a program under test (a program under development which is a test target). The test case processing unit 3 is the test case input means 1
Holds the information of the test case input from the device, provides the expected value to the test result expected value holding unit 6, reads the program under test from the storage device 2, and uses the information of the test case to load the program under test. The processing is executed such that it is executed and the result is given to the test result expected value holding unit 6.

The program correction section 10 is for correcting a bug in the program under test. The program correction unit 10 may be a processing system capable of automatically correcting a bug that can be automatically corrected. If it depends on human hands, an input device such as a keyboard and an editing device such as an editor may be used. The configuration etc. can be considered.
While the program correction unit 10 is performing the processing, the timer 11 measures the time and cumulatively holds the processing time required for the program correction.

The bug result storage unit 5 stores the bug information detected by the bug detection unit 7, and the bug correction detection unit 4 stores the test results and expected values held by the test result expected value holding unit 6 and the bug result storage. The presence / absence of bug correction is detected by referring to the bug result information accumulated by the unit 5, and the test result expected value holding unit 6 holds the test result and the expected value output from the test case processing unit 3. It is for.

The bug detection unit 7 detects the presence or absence of a bug by referring to the test result and the expected value held by the test result expected value holding unit 6, and the information on the number N of bugs obtained by this detection. Is output. The first timer 9 is a timer for accumulating the processing time (test time T) in the test case processing unit 3 for the program under test.

The fixed bug number detection unit 8 obtains information on the fixed bug number M output by the bug correction detection unit 4 and information on the bug number N output by the bug detection unit 7, and then outputs information on the unfixed bug number E. Is what you get.

The data storage unit 12a holds information on the cumulative test time T accumulated by the first timer 9 and information N on the number of bugs detected by the bug detection unit 7, and the data storage unit 12a. 12b is information on the integrated correction time MT output from the second timer 11 and the bug correction detection unit 4
The data storage unit 12c holds the correction time MT output from the second timer 11, the test time T output from the first timer 9, and the correction. This is for holding the uncorrected bug number E output from the bug number detection unit 8.

The estimation units 13A-1 to 13A-n respectively use the information of the cumulative test time T and the information of the number of bugs N stored in the data storage unit 12a, based on a predetermined arithmetic expression to be described later, This is an estimation unit that calculates and estimates the release time. The estimation units 13A-1 to 13A-n use the arithmetic expressions of the estimation models of different types.

The estimating units 13B-1 to 13B-n respectively use the information of the correction time MT and the information of the number of correction bugs M stored in the data storage unit 12b, based on a predetermined arithmetic expression described later, This is an estimation unit that calculates and estimates the release time. The estimation units 13B-1 to 13B-n use the arithmetic expressions of the estimation models of different types.

The estimating units 13C-1 to 13C-n are based on a predetermined arithmetic expression described later based on the correction time MT, the test time T, and the uncorrected bug number E stored in the data storage unit 12c. Is an estimation unit that calculates and estimates the release time. The estimation units 13C-1 to 13C-n also use the calculation formulas of different estimation models.

The knowledge section 14 is a section that collects empirical information of software development engineers. The determination unit 15 includes the estimation units 13A-1 to 13A-n, 13B-1 to 13B-n, 1
The estimation results of 3C-1 to 13C-n are comprehensively determined by using the knowledge stored in the knowledge unit 14 to determine the release point estimated value of the program under test.

Then, the judging section 15 is arranged to estimate sections 13A-1 to 13A-1.
Information about the cumulative test time T and the number of bugs N is obtained by weighting the release point estimated values respectively output from 13A-n using the knowledge of the knowledge unit 14 and adding the weighted corrected release point estimated values. And a release time estimation value based on and, and the estimation units 13B-1 to 13B-n
Estimated release time output from each
Weighted correction is made using the knowledge of No. 4, and each weighted and corrected release time point estimated value is added to determine the release time point estimated value based on the information of the correction time MT and the number of corrected bugs M. 13C-1 to 13C-n, the release time point estimated values are weighted and corrected by using the knowledge of the knowledge unit 14, and the weighted and corrected release time point estimated values are added to each other to obtain the correction time MT and the test time. It has a function of determining an estimated release time point value based on T and the number E of uncorrected bugs, and outputting each estimated value.

The second judging section 16 comprehensively judges the information at each release time point estimated by the judging section 15 to obtain an estimated value at the final release time point, and the judgment result extracting section 17 is the second. The information at the time of release obtained by the determination unit 16 is output.

In the present invention, the estimation units 13A-1 to 13-1
In 3A-n, the release time is estimated by calculating using the formula of the reliability model growth curve based on the information of the cumulative test time T and the information of the number of bugs N. For example, the following model is used as the expression.

[0041]

[Table 1]

Here, various reliability models as shown in Table 1 are used to estimate the release point in time series failure (bug) data. This is because the reliability estimation model growth curve shows the test time ( ti) and the total number of bugs found up to that point (m (ti)) and the number of latent failures (latent bug number) are applied as a curve. The total number of found bugs reaches the latent bug number. , It is based on the rationale that when the bug is fixed, it can be considered that the bug is exhausted, and from this, the arrival time in the model can be estimated.

That is, the release time can be estimated by estimating the time when the total number of modifications reaches the saturation point of the reliability estimation model growth curve from the trend of the curve of the total number of modifications. This is obtained from the formula (reliability model).

In the various reliability models shown in Table 1 above,
The test time (ti) corresponds to T of the first timer 9, and the total number of found failures (m (ti)) corresponds to N in the bug detection unit 7.

There are various reliability growth models such as an exponential reliability growth model and a logistic curve model.
Since it is not known which is the most suitable, the estimation result by each model is used to make a comprehensive judgment so that an estimation result closer to reality can be obtained. For that purpose, the estimation unit prepares a plurality of 13A-1 to 13A-n. For example, the estimation unit 13A-1 is caused to perform release point time estimation based on the exponential reliability growth model, and the estimation unit 13A-1
In A-3, the release time point is estimated based on the logistic curve model, and the reliability growth model to be applied differs for each estimation unit.

In the present invention, the estimating unit 13B-
In 1 to 13B-n, the release time is estimated by calculating using the formula of the reliability model growth curve based on the information of the correction time MT and the information of the number of fixed bugs M. Here, the reliability model growth is performed. For example, the following model is used as the curve formula.

[0047]

[Table 2]

Here, various reliability models as shown in Table 2 are used to estimate the release time point from the bug correction time data. In this case, the reliability model growth curve corresponds to the bug correction time (Si). It is a curve that fits the relationship between the total number of faults (bug) fixed (n (Si)) and the number of latent faults (the number of latent bugs) up to the time when it is reached. When the number (potential bug fix time) estimated value is reached, it can be considered that all the bugs hidden in the program under test have been fixed, and from this, it is possible to estimate the arrival time in the model. It is based on the rationale that can be done.

That is, the release time can be estimated by estimating the time when the total correction time reaches the saturation point of the reliability model growth curve from the trend of the curve of the total modification time. This is obtained from the formula (reliability model).

In the various reliability models shown in Table 2 above,
The correction time (Si) corresponds to MT of the second timer 11,
The total number (n (Si)) of faults corrected up to the stage of reaching the correction time (Si) corresponds to the number M of fixed bugs in the bug correction detection unit 4.

There are various reliability growth models such as an exponential reliability growth model and a logistic curve model.
Since it is not known which is the most suitable, the estimation result by each model is used to make a comprehensive judgment so that an estimation result closer to reality can be obtained. Therefore, the estimation unit prepares a plurality of 13B-1 to 13B-n. For example, the estimation unit 13B-1 is caused to perform release point time estimation based on the exponential reliability growth model, and the estimation unit 13B-1
In B-3, the release time point is estimated based on the logistic curve model, and the reliability growth model to be applied is different for each estimation unit.

In the present invention, the estimating unit 13C-
In 1 to 13C-n, calculation is performed using a statistical distribution model based on the information of the fixed time MT, the information of the test time T and the number of unfixed bugs E, and the release time is estimated. For example, the following model is used as a typical distribution model.

[0053]

[Table 3]

Here, various statistical distribution models as shown in Table 3 are used, and the release time point is determined by the number of time series failures (number of bugs), the number of unfixed bugs, and the total time required for fixing the bugs up to the present time. The number of failures (bugs) may increase as the test time increases, but this tendency can be statistically identified and the number of bugs can be calculated from the cumulative test time. The upper limit can be determined to some extent, and failures (bugs) will be fixed at any time during repeated testing, so if the number of corrections increases, the number of uncorrected items should eventually become zero. Release at this point Since it should be possible, it is based on the rationale that it is possible to statistically analyze these and estimate the release time.

That is, it can be considered that the total number of discovered bugs and the total number of latent bugs become approximately the same in terms of statistics when the test according to various test data reaches a certain number of hours. By the transition of, it becomes possible to infer the limit at which the total number of discovered bugs and the total number of latent bugs are equal, and after that point, it can be considered that release is possible if the uncorrected number becomes zero. The time when zero is reached is estimated, and that time is taken as the estimated release time.

Although the software release time point is represented by this time series correction curve, E is applied as α in each statistical distribution formula in Table 3 as time series data of the uncorrected total number, and the release time point (ri) of Find an approximate solution.

The number of test times is monitored by the output of the first timer 9, and the time required to correct the uncorrected bug is estimated based on the correction time MT which is the output of the second timer 11. From this, the uncorrected curve is estimated, and the time when this uncorrected curve reaches zero is estimated as the release time. As the statistical distribution used for this estimation, a Rayleigh distribution, a gamma distribution, an exponential distribution, or the like is used as shown in FIG.

As described above, since there are various statistical models and it is not known which one is the best, it is necessary to make a comprehensive judgment using the estimation results of each model.
Make it possible to obtain more realistic estimation results. Therefore, the estimation unit prepares a plurality of 13C-1 to 13C-n. For example, the estimation unit 13C-1 is caused to perform release point time estimation based on the Rayleigh distribution model, and the estimation unit 13C
In -3, the release time point is estimated based on the gamma distribution model, and the reliability growth model to be applied is different for each estimation unit.

The first judgment unit 15 is the estimation units 13A-1 to 13-1.
3A-n, 13B-1 to 13B-n, 13C-1 to 13
The final release time is calculated by processing the release time estimated value in each model obtained in Cn according to the comprehensive judgment rule based on the empirical rule.
The comprehensive judgment rule is as in the following formula (1).

[0060]

[Equation 1]

In the equation (1), the weight e11 to the release point estimated value for each model-specific attention parameter based on the feeling of an experienced software development engineer as an empirical rule
Adopting the form of preparing and assigning e35, using the number of bugs, the number of corrections, and the number of uncorrected as attention parameters,

[0062]

[Equation 2]

In formula (1), abbreviated names such as "Reili", "Wive", "Logis", "Gompe" are used as model names, but "Rayleigh", "Weibull", and "Logistic" are used respectively. , Means "Gompertz". Others are the same.

[0064]

[Equation 3] Here, tr (1), tr (2), tr (3) are weights determined based on past experience, respectively,

[0065]

[Equation 4] It has the values of 0.0 < tr (1), tr (2), tr (3) < 1.0. In other words, the relationship is such that all are added to become 1.

Next, the operation of the present apparatus having the above-described structure will be described with reference to FIG.
This will be described with reference to the flowchart of. First, a test case (simulated information for testing) is input from the test case input means 1 (S1), and a program under test, which is a program under development, which is a test target is stored in the storage device 2 (S2).

Then, when the system is activated, the system first executes the process in the test case processing unit 3 under the control of the control means (not shown). In the processing here, first, the test program is read from the storage device 2, and then it is determined whether or not the test is completed (S
3) If the test is not completed, the test case processing unit 3 executes the test program using one of the unfinished test cases input from the test case input means 1 (S4).

As a result, the first timer 9 counts and integrates this execution time, and measures the test time T (S
5). The measurement result is given and stored in the corresponding one of the data storage units 12a to 12c.

Since the test case processing unit 3 gives the test result to the test result expected value holding unit 6 as the process based on the used test case is executed, the test result expected value holding unit 6 holds this. When a test case is given, the test result expected value holding unit 6 gives and holds an expected value that is a scheduled result of the test case.
This expected value is prepared in advance with the test case, and when the test ends, the bug detection unit 7 compares the test result stored in the test result expected value holding unit 6 with the expected value, and the two match. The number of non-executable objects is set as the number N of bugs generated in this process, and counting is performed.

In this way, the bug detection unit 7 obtains the number N of bugs in the test case used for the implementation. Then, this bug number N is given to the fixed bug number detection unit 8, and the fixed bug number detection unit 8 accumulates and holds this.

If a bug is found, a control unit (not shown) displays information about this bug on a display or the like to inform it. Therefore, the software developer will either take corrective action or continue testing for this bug. Also, if there is no bug, it is necessary to decide whether to perform the test in the next test case. Therefore, at this stage, an instruction for the next treatment is given to the system by a keyboard (not shown) or the like.

As a result, the test is not completed, and
If there is an instruction to move to the next test without "fixing the bug" (S6, S3), the control unit moves to step S4 and executes the above-mentioned processing again.

On the other hand, if it is "fix a bug",
The program correction unit 10 corrects the bug (S
7). The program correction unit 10 can automatically correct bugs that can be automatically corrected according to the operator's instructions. For bugs that require individual review and consideration by the programmer, an editor is activated to edit by the programmer or operator. It has a function that enables Thus, the programmer or operator will make the correction accordingly. Then, during that period, the second timer 11 counts the time and accumulates the accumulated time to modify the program under test M.
It is output as T (S8). The correction time MT is stored in each of the data storage units 12b and 12c.

In correcting this bug, the number of corrections is counted as follows. That is, when the correction is completed, the test case used when the bug is found is selected, the test case processing unit 3 performs the test, and the test result expected value holding unit 6 inputs and holds the result. The bug correction detection unit 4 knows the resolved bug based on this data and the expected value, and accumulates it as the corrected bug number M. This corrected bug number M is input to the data storage unit 12b and is also given to the fixed bug number detection unit 8, and the fixed bug number detection unit 8 outputs the fixed bug number M and the cumulative number of bugs output by the bug detection unit 7. The difference from N is calculated to calculate the number of uncorrected bugs E.

The obtained uncorrected bug number E is input and stored in the data storage unit 12c. When such a process is repeated and an instruction to end the test is given at the end, the current bug number N, test time T, fixed bug number M, and unfixed bug number E stored in the data storage units 12a to 12c. Of these, the release time point estimation processing based on specific parameters is performed by each estimation unit 13A-
1 to 13A-n to 13C-1 to 13C-n.

That is, when the test is completed (S3),
The estimation units 13A-1 to 13A-n estimate the release time point for each of the k (k = 1, 2, 3, to k) different software reliability growth models from the relationship between the test time T and the number of bugs N (S10). ). Then, with respect to the respective k estimated release time points obtained as a result, the determination time point is estimated by the determination section 15 using the knowledge stored in the knowledge section 14 based on the relationship between the test time T and the number of bugs N. (S
11).

In parallel with this, the estimation unit 13B-1
.About.13B-n estimates the release time point for each of 1 (l = 1,2,3, ... L) different software reliability growth models from the relationship between the modification time MT and the number of bug modifications M (S = 1.
12). Then, for each of the l estimated release time points obtained as a result, the determination unit 15 uses the knowledge stored in the knowledge unit 14 to determine the correction time MT and the bug correction number M.
The release time is estimated based on (S13).

Further, the estimating units 13C-1 to 13C-n use m (m = 1, 2, 3, ... M) statistical models which differ from the relationship between the test time T, the correction time MT, and the number E of uncorrected bugs. The release time is estimated for each (S14). And
The determination unit 1 uses the knowledge stored in the knowledge unit 14 with respect to the m estimated release time points obtained as a result.
5, the release time point is estimated based on the test time T, the correction time MT, and the uncorrected bug number E (S15).

The respective release time point estimated values thus estimated are used to make a final release time point estimation in a comprehensive manner by using the knowledge stored in the knowledge section 14 by the judging section 16 ( S16), the result is output and notified by the judgment extraction means 17.

The present invention is not limited to the embodiments described above and shown in the drawings, but can be carried out by appropriately modifying it within a range not changing the gist of the invention. Estimate the release time from the relationship between the time and the number of bugs, from the relationship between the fix time and the number of fixes, and from the relationship between the test time and the number of unfixed bugs using various models. It is possible to estimate the release time in a form closer to a more realistic line by performing weighted calculation according to the know-how of experienced persons and integrating it, but only by the relationship between the test time and the number of bugs, or It is also possible to estimate only by the relationship between the modification time and the number of modifications, or only the relationship between the test time, the modification time and the number of unfixed bugs. Use either of these two to estimate the release time, weight them according to the rules established based on accumulated empirical knowledge information, and make a comprehensive judgment to estimate the final release time. May be.

As described above, the present invention enables reliable estimation of the release point of software under development, and software test information (test time, number of bugs, cumulative bug fix time, number of fixes). Multiple reliability estimates based on the relationship between the test time and the number of bugs, the relationship between the cumulative bug fix time and the number of fixes, the relationship between the test time and the cumulative bug fix time, and the number of unfixed bugs based on the time series data of Obtain release point estimates for each model and multiple statistical models, add weighted corrections to these calculation results according to rules based on the experience and intuition accumulated by experienced software development engineers. The objective is to obtain the release point as an objective estimate from a comprehensive viewpoint using various models and parameters.

Regarding the total number of bugs hidden in the program under test, the relationship between the number of bugs found by the test and the number of fixed and unfixed bugs is that the cumulative value of the number of bugs found as the test progresses is latent in the program under test. The number of total bugs will gradually approach, and they will match soon, while the bugs found will be fixed, so if this is all fixed, it was latent in the program under test. The line for the total number of bugs will be reached, and the number of unfixed bugs will represent the difference between the number of fixed bugs and the transition of found bugs.

Therefore, if the time limit is reached from this relationship to estimate the time required from the present time to this time limit, this is the release time, and the various transitions used here are applied to the model. By using the various reliability growth models and various statistical models as models, the estimated values for each are obtained, and these are comprehensively judged using the knowledge of experienced persons. The release point can be estimated in a shape close to a realistic line. It should be noted that the present invention is not limited to the embodiments described above and shown in the drawings, but can be appropriately modified and implemented within the scope of the invention.

[0084]

As described above in detail, according to the present invention, it becomes possible to estimate the release point in a manner closer to a realistic line, and therefore, to judge the progress of software development and It is possible to provide a software release time point estimation device and an estimation method that enable appropriate management and the like.

[Brief description of drawings]

FIG. 1 is a block diagram of an overall configuration showing an embodiment of the present invention.

FIG. 2 is a partially enlarged view of FIG.

FIG. 3 is a partially enlarged view of FIG.

FIG. 4 is a diagram for explaining the principle of the present invention.

FIG. 5 is a flowchart explaining the operation of the device of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Test case input means, 2 ... Storage device storing the program under test, 3 ... Test case processing unit, 4 ... Bug correction unit, 5 ... Bug result storage unit, 6 ... Test result expected value holding unit, 7 ... Bug Detection unit, 8 ... Fixed bug number detection unit, 9 ... First timer, 10 ... Program correction unit, 11 ... Second timer, 12a to 12c ... Data storage unit, 13A-1 to 13A-1
3A-n, 13B-1 to 13B-n, 13C-1 to 13
Cn ... estimation unit, 14 ... knowledge unit, 15 ... first judgment unit,
16 ... 2nd judgment part, 17 ... Judgment result extraction part.

Claims (8)

[Claims] 1. An execution unit that executes software to be evaluated according to a test case, which is a group of various conditions prepared for testing, cumulative test time information that is the cumulative time of execution by this execution unit, and the evaluation target. A test data collecting means for collecting data on the number of detected failures / defects when the software is executed, the number of failures / defects fixed, and the number of hours required to correct the failures / defects. , Based on the information of the uncorrected number obtained from the relationship of the number of modifications to the number of detected failures / defects, based on the relationship between the test time based on the cumulative test time information and the transition of the modification time, various statistical models Estimating means for estimating the release time of the software to be evaluated by model according to the above, and the accumulated experience for each of these estimated values. A software release time point estimation device comprising: a determination unit that obtains an estimation result at a release time point by performing a comprehensive evaluation process after correction according to a predetermined rule based on the software release time point. 2. Execution means for executing software to be evaluated in accordance with a test case, which is a group of various conditions prepared for testing, cumulative test time information of execution by this execution means, information on the number of tests, and the evaluation object. Test data collection means for collecting the number of detected failures / defects, the number of failures / fixed defects, and the number of hours required to correct failures / defects when the software of Of these, based on the cumulative test time and the number of failures / defects, various software reliability growth models are used to estimate the release time of the software to be evaluated for each model and obtain these as an estimated value group. Based on the estimation means and the correction time and the number of corrections of the collected data, the evaluation is performed by various software reliability growth models. The second estimation means for estimating the release time of the software to be evaluated for each model and obtaining these as an estimated value group; and the number of test hours, the number of modified hours, and the number of unmodified among the collected data. And a third estimating means for estimating the release time of the software to be evaluated for each model by various statistical models to obtain these as an estimated value group, and the respective estimating means respectively estimated by these estimating means. Release time by estimated value group Software release time point characterized by being composed of a judgment means for obtaining the final estimation result at the release time point by processing the estimated value according to a predetermined rule based on accumulated experience Estimator. 3. The determination means comprises the rule consisting of a condition and a weight, and obtains an estimated value at the release time point by weighting and adding to each release time point estimation result according to the condition. The software release time point estimation device according to claim 1 or 2, characterized in that: 4. Execution means for executing software to be evaluated according to a test case, which is a group of various conditions prepared for testing, cumulative test time information of execution by this execution means, information on the number of tests, and the evaluation object. The test data collection means for collecting the number of detected failures / defects, the number of failures / defects corrected, and the number of hours required to correct the failures / defects when the software of Of these, based on the cumulative test time and the number of failures / defects, various software reliability growth models are used to estimate the release time of the software to be evaluated for each model and obtain these as an estimated value group. Based on the estimation means and the correction time and the number of corrections of the collected data, the evaluation is performed by various software reliability growth models. Second estimating means for estimating the release time of the software targeted for valuation for each model to obtain these as an estimated value group; and correction to the number of detected failures / defects of the data collected by the data collecting means. Based on the information of the uncorrected number obtained from the relationship of the number, from the relationship with the transition of the test time and the modification time based on the cumulative test time information, various statistical models are used to model the release time of the software to be evaluated. Third estimation means for separately estimating and obtaining these as an estimated value group and each estimated value group for each of the first to third estimating means are corrected in accordance with a predetermined rule based on accumulated experience. After that, the estimated result at the time of release for each estimated value group is obtained by comprehensively performing an evaluation process for each estimated value group.
And a second determining means for obtaining the final estimation result at the release point by processing the estimation result at each release point for each estimated value group according to a predetermined rule based on accumulated experience. A software release point in time estimation device characterized by the above. 5. The release time point is determined by the first and second determination means, each rule being composed of a condition and a weight, and weighted and added to each release time point estimation result according to the condition. 5. The software release time point estimation device according to claim 4, wherein the estimated value is calculated. 6. An execution step for executing software to be evaluated according to a test case, which is a group of various conditions prepared for testing, cumulative test time information of execution by this execution step, information on the number of tests, and the evaluation object. And a test data collecting step for collecting data on the number of detected failures / defects and the number of failures / defects repaired and the number of hours required to repair the failures / defects when executing the software of Based on the cumulative test time and the number of failures / defects among the data collected in
A first estimation step of estimating the release time of the software to be evaluated for each model by various software reliability growth models and obtaining these as an estimated value group; and a correction time and a correction number of the collected data. A second estimation step of estimating the release time of the software to be evaluated for each model by various software reliability growth models to obtain these as an estimated value group, and a test of the collected data. A third estimating step of estimating the release time of the software to be evaluated for each model by various statistical models based on the number of hours, the number of modified hours, and the number of uncorrected, and obtaining these as an estimated value group; , The release point estimated value for each estimated value group estimated in each of these estimation steps To, by processing in accordance with a predetermined rule based on accumulated experience, software release time estimation method characterized by comprising further a second determination step of obtaining a final estimation result at the time of the release. 7. An execution step for executing software to be evaluated according to a test case, which is a group of various conditions prepared for testing, cumulative test time information of execution by this execution step, information on the number of tests, and the evaluation object. And a test data collecting step for collecting data on the number of detected failures / defects and the number of failures / defects repaired and the number of hours required to repair the failures / defects when executing the software of Based on the cumulative test time and the number of failures / defects among the data collected in
A first estimation step of estimating the release time of the software to be evaluated for each model by various software reliability growth models and obtaining these as an estimated value group; and a correction time and a correction number of the collected data. A second estimation step of estimating the release time of the software to be evaluated for each model by various software reliability growth models as an estimated value group, and the data collected in the data collection step. Of the above, based on the information of the uncorrected number obtained from the relationship of the number of corrections to the number of detected failures / defects, based on the relationship between the test time based on the accumulated test time information and the change in the correction time, various statistics Of the release time of the software to be evaluated by model based on a statistical model After the third estimation step obtained as a group and each estimated value group in each of the first to third estimated steps are corrected according to a predetermined rule based on accumulated experience, each estimated value group Based on the accumulated experience, the first judgment step for obtaining the estimation result at each release point for each estimated value group by performing the comprehensive evaluation processing for each A software release time point estimation method comprising a second determination step of obtaining a final estimation result at a release time point by processing according to a predetermined rule. 8. The first and second judgment steps are configured by the rule consisting of a condition and a weight, and by weighting and adding to each release time point estimation result according to the condition, a release time point is obtained. The software release time point estimation method according to claim 6 or 7, wherein the estimated value is calculated.