JPH09274576A - Method and device for evaluating reliability of system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more suitably evaluate the reliability of a software. SOLUTION: Error data expressing the generation time of an already generated and corrected error are inputted (500) so that the number of remaining errors can be estimated (501) and the distribution of meantime-to-failures(MTTF) of the generated error can be found (502), the distribution of MTTF of remaining errors is estimated from the distribution of MTTF of the generated error, and the number of distributed MTTF of remaining errors is found (503) so as to be coincident with the number of remaining errors. Then, based on the found number of distributed MTTF of remaining errors, the entire MTTF is calculated (504).

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a technique for evaluating the reliability of a system such as a software system.

[0002]

2. Description of the Related Art When an error occurs during the operation of software, the cause of the error is analyzed and the found software bug is eliminated. The elimination of this error reduces the number of potential errors in the software. And
As the number of errors decreases, the error occurrence probability decreases and the error occurrence interval increases. This software operating time and error occurrence interval (MTTF: Me
an Time to Failure or MTBF: Mean Time Between Fa
ilure) relationship is shown in FIG.

FIG. 9 shows the relationship between the operating time of software and the total number (cumulative number) of errors that occur and are removed.

Here, "Introduction to Software Reliability Model", Mitsuru Ohba, Information Processing, VOL.31, No12, PP1623-1630 (19
As described in 90)), there are two positions for evaluating the reliability of software. The first position is to evaluate the realization level of software reliability by the interval of occurrence as a phenomenon of failure, and the second position is to evaluate the realization level of software reliability by the number of residual errors that are the cause of the failure. It is a thing.

Therefore, FIG. 8 is related to the change in the reliability of the software from the first standpoint with time, and FIG. 9 is related to the change in the reliability of the software from the second standpoint. . Here, a change in reliability of software over time is called “reliability growth”.

Now, in the evaluation of the reliability of software, it is necessary to estimate the reliability in consideration of the reliability growth. As software reliability models used for such estimation, some mathematical models based on probability / statistics have been proposed to enable quantitative evaluation of reliability.

The model according to the first aspect described above is called a software failure occurrence time model, and for example, a model in which the distribution of software failure occurrence time intervals is approximated by a gamma distribution is known.

The model in the second position is called an error discovery number model. For example, the curve of FIG.
Models that are approximated by logistic curves and Gompertz curves are known.

[0009]

According to the conventional technique using the software reliability model described above, the reliability evaluation of software cannot be performed sufficiently correctly, and the actual software is evaluated as it is when it is actually operated. There was a case where the reliability of was not satisfied. Moreover, such a situation applies not only to software but also to general systems.

Therefore, an object of the present invention is to provide a system reliability evaluation method and a system reliability evaluation apparatus which can more accurately evaluate the system reliability.

[0011]

In order to achieve the above object, the present invention is, for example, a method for evaluating the reliability of a system, in which an error which has already occurred in the system and whose factor has been eliminated. Estimating the distribution of occurrence intervals and the number of errors caused by factors remaining in the system, based on the distribution of error occurrence intervals from which the factors have been eliminated, estimating the distribution of error occurrence intervals due to the remaining factors, Distribution of error occurrence intervals due to the estimated remaining factors,
A method of calculating the reliability of the entire system from the number of errors caused by the remaining factors is provided.

According to the method of the present invention, the reliability of the entire system is evaluated in consideration of the distribution of errors caused by the factors remaining in the system. Since the reliability of the entire system is calculated by evaluating the property relating to the occurrence of the error, it is possible to more appropriately evaluate the reliability of the entire system as compared with the conventional method.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below.

FIG. 1 shows the hardware configuration of the software reliability evaluation apparatus according to this embodiment.

In the figure, 1 is a memory for storing various programs, 2 is a processor for executing the programs stored in the memory 1, 3 is a display output device, 4 is an input device such as a keyboard or mouse, and 5 is an external storage. The devices 6, 7, and 8 are interface units that perform input / output with the display output device 3, the input device 4, and the external storage device 5. As shown in the figure, a general personal computer or other computers can be used as the hardware of the software reliability evaluation apparatus.

Next, FIG. 2 shows the configuration of each functional unit of the software reliability evaluation apparatus, which is realized as a process on the software reliability evaluation apparatus by the processor 2 executing the program stored in the memory 1. Indicates.

In the figure, 409 is an already-generated error data reading unit, 401 is a residual error number prediction unit, 402 is a distribution prediction unit, 403 is an individual error MTTF calculation unit, 404 is an MTTF calculation unit, 405 is a simulation unit, and 406 is 406. Judgment part, 4
Reference numeral 07 is an error data updating unit, 410 is an MTTF distribution database, and 411 is an error data file.

In the figure, the MTTF distribution database 410 and the error data file 411 are not processes but information stored in the memory 1 or the external storage device 5.

Next, the operation of the software evaluation apparatus will be described.

FIG. 3 shows a processing procedure of processing performed by the software evaluation apparatus.

As shown in the figure, in this process, first, the error data reading unit 409 takes in error data that has already occurred from the input device 4 and creates an error data file 411 (step 500). As the error data that has already occurred, the error occurrence time S obtained by debugging while actually running the software to be evaluated
And the data of the elapsed time X from the last occurrence of the error until the occurrence of the error is used. Now, at the time when the debugging is performed up to time Si in FIG. 4, i sets of already-generated error data (X1 to Xi, S1 to Si) are input. In addition,
A software bug that causes an existing error is corrected each time the error occurs.

Next, the residual error prediction unit 401 reads the error data file 411, predicts the residual error number, and displays the predicted residual error number on the display output device 3 (step 501). The prediction of the number of remaining errors is performed in the same manner as the conventional method described above. That is, for example, as shown in FIG. 5, the relationship between the number of generated errors obtained from already generated error data and the operating time of software is approximated by a curve (indicated by a finger line in FIG. 5), and the time represented by this curve is infinite. The difference between the total number of errors at the time of and the cumulative error up to the present obtained from the error data is predicted as the number of remaining errors.

Next, the distribution predictor 402 reads the error data file 411, calculates the average and variance of the occurrence time intervals of the already-occurred errors, and estimates the distribution of the time intervals of the already-occurred errors (step 502). . However, the error data and the estimated distribution may be corrected in consideration of the system load, the usage frequency of each function, the execution frequency of the software, and the like in the operating state of the software.

Next, the individual error MTTF calculator 403 estimates the distribution of the MTTF of the residual error from the time interval of the already-occurred error.

Generally, the distribution of MTTF is from the initial (t = 0) to t = S
Since the smaller the MTTF is up to i, the error is more likely to occur. Therefore, if the initial MTTF distribution is f (λ) as shown in FIG. 6, exp (-λSi) f (λ ) Distribution.
That is, it can be expected that i errors will occur during the Si time from f (λ) and the MTTF distribution will be exp (-λ · Si) · f (λ). Then, the distribution obtained by subtracting this distribution from the initial distribution (the shaded area in the figure) is the MT of the existing error up to time Si.
It becomes the distribution of TF. Then, the distribution of the occurrence time intervals of the already-occurred errors obtained in step 502 follows the MTTF distribution of the already-occurred errors up to time Si. Therefore, the MTTF distribution of the residual error can be estimated from the distribution of the occurrence time intervals of the already-occurred errors obtained in step 502.

In the present embodiment, such an estimation method is not the method of mathematically solving the error occurrence time interval distribution to estimate the MTTF distribution of the residual error, but the following method. That is, the distributions of various MTTFs (constant,
An error is generated by executing a simulation that simulates the operation of software according to (uniformity, exponential, Weibull, and gamma distribution). For each MTTF distribution, the distribution of the error occurrence time interval at each time point and the residual error MTT
The correspondence with the distribution of F is collected and stored in the MTTF distribution database 410. In addition, such a simulation is also performed under the set conditions (here, various MTTFs are used).
Distribution), a transaction simulator that randomly generates each error can be used.

Then, the MTTF distribution of the residual error corresponding to the distribution of the occurrence time intervals close to the distribution of the existing time intervals obtained in step 502 is calculated as the MTTF distribution database 4
The residual error MTTF is obtained from the above equation.

Further, if the individual error MTTF calculation unit 403 estimates the distribution of the residual error MTTF in this way, the residual error MTTF calculation unit 403 calculates the residual error MTTF distribution based on the residual error MTTF distribution and the residual error number obtained in step 401. The MTTF of each error is calculated (step 503).

That is, as shown in FIG. 7, the residual error distribution is quantized for MTTF. In the figure, MTTF appropriately
The values are quantized into seven values a to g within the range below the maximum value of. Then, the MTTF of each quantized value is adjusted so that it fits the distribution of residual errors and the total number matches the number of residual errors.
Determine the number of remaining errors with. In the figure, the MTTF of the value a
The number of remaining errors with a is A, the number of remaining errors with an MTTF of value b is B, the number of remaining errors with an MTTF of value c is C, and so on. The number of errors A to G was determined as the number of errors. However, A + B + C + D + E + F + G matches the number of remaining errors.

The number of such residual errors and MTTF
The combination of and depends on the maximum value, the quantization unit of MTTF, and the value to be quantized. Further, when the number of remaining errors is small, it may not be uniquely obtained. However, as will be described later, this combination is changed later so as to better match the actually measured value in accordance with the result of comparison with the actually measured value. Therefore, an appropriate combination may be selected here.

Next, the MTTF calculation unit 404 sets the MTTF of the entire software as the number of residual errors having n as the quantization number of MTTF, MTTFi as the i-th quantized MTTF, and ki as the number of residual errors having MTTFi. In accordance with the idea of (1), it is calculated by the following equation (step 504).

[0032]

[Equation 1]

Next, the simulation unit 405 executes a simulation for simulating the operating status of software according to the combination of the number of residual errors and MTTF obtained in step 503 (step 505). Then, the error occurrence status (for example, the time interval until the next error occurrence) obtained as a result of the simulation is output to the display output device. Note that such a simulation can also be performed using a transaction simulator that randomly generates each error under set conditions (a combination of the number of remaining errors and MTTF in this embodiment).

Next, after this, input of the actual error occurrence status (for example, the time interval until the first error occurrence) obtained by operating the actual software whose debugging has been completed up to time Si. Upon receiving from the user via the input device 4, the determination unit 406 compares the actual error occurrence status with the error occurrence status obtained as a result of the simulation in step 505, and determines whether or not they are closer than a predetermined level. The determination is made (step 507).

If the actual error occurrence status and the error occurrence status obtained as a result of the simulation in step 505 are close to each other by a predetermined level or more, step 5
The combination of the number of remaining errors and the MTTF obtained in 03 almost correctly represents the current state of software, and the MTTF of the entire software obtained in step 504 based on this combination also almost correctly represents the current state of software. it is conceivable that. Therefore, when the actual error occurrence status and the error occurrence status obtained as a result of the simulation in step 505 are close to each other by a predetermined level or more (step 508), the determination unit 406 determines that
It is determined whether or not the MTTF of the entire software obtained in 4 satisfies the MTTF specifications required for the software to be evaluated (step 509), and if so, it is output to the display output device 3, The whole process is ended.

On the other hand, when the actual error occurrence status and the error occurrence status obtained as a result of the simulation in step 505 are not close to each other by a predetermined level or more (step 508), the residual error found in step 503 Since the combination of the number and MTTF may not be appropriate, in this case, the determination unit 406 activates the individual error MTTF calculation unit 402 and causes the process of step 503 to be performed, and the number of different residual errors and MTTF be different. Ask for a combination with. However, when step 503 is repeated a prescribed number of times or more, or when it is not possible to expect to find an appropriate combination more than the combination of the number of residual errors and the MTTF obtained up to now in step 503,
The determination of step 509 is performed.

After re-execution of step 503 each time, as described above, the processing after step 503 is executed based on the combination of the number of different residual errors and MTTF newly obtained in step 503. Will be done.

Although the actual error occurrence status and the error occurrence status obtained as a result of the simulation in step 505 are close to each other by a predetermined level or more,
If the MTTF of the entire software obtained in step 504 does not meet the MTTF specifications required for the software to be evaluated (step 509), it is determined that debugging of the software is insufficient to meet the MTTF specifications. Since it is shown, that effect is output to the display output device 3. In this case, the simulation in step 505 can be considered to imitate the actual software, and the prediction of future reliability growth can be obtained from the simulation result.

Then, when the user subsequently inputs the error data obtained in the new debugging, the error data updating unit 407 adds this error data to the error data file 411 (step 51).
Along with 0), the remaining error number predicting unit 401 is activated to re-execute the processing of step 501 based on the updated error data file 411. After the execution of step 501, the processings of step 502 and above are executed as described above.

According to the present embodiment described above,
Software reliability, ie number of remaining errors and MTTF
Alternatively, MTBF (Mean Time Between Failure) can be appropriately evaluated, and the growth of reliability can also be appropriately predicted, so that it is possible to properly judge the termination of debug and grasp the quality of the product. In addition, it is possible to efficiently allocate personnel in a project where debugging is performed by a large number of people, and to predict the operating status of the program system after shipping, so that flexible execution and management of development plans can be realized.

The embodiments described above can be similarly applied to the evaluation of the reliability of various systems other than the evaluation of the reliability of software.

[0042]

As described above, according to the present invention, it is possible to provide a system reliability evaluation method and a system reliability evaluation apparatus which can more accurately evaluate a system.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration of a software reliability evaluation device.

FIG. 2 is a block diagram showing a functional configuration of a software reliability evaluation device.

FIG. 3 is a flowchart showing a processing procedure of processing performed by a software reliability evaluation apparatus.

FIG. 4 is a diagram showing an example of error data.

FIG. 5 is a diagram showing how the number of remaining errors changes with time.

FIG. 6 is a diagram showing how the MTTF distribution of errors changes with time.

FIG. 7 is a diagram showing how MTTF is quantized for errors.

FIG. 8 is a diagram showing the change over time of error MTTF.

FIG. 9 is a diagram showing how the number of discovered errors changes with time.

[Explanation of symbols]

 1 memory 2 processor 3 display output device 4 input device 5 external storage device 409 existing error data reading unit 401 residual error number prediction unit 402 distribution prediction unit 403 individual error MTTF calculation unit 404 MTTF calculation unit 405 simulation unit 406 determination unit 407 error Data update unit 410 MTTF distribution database 411 Error data file

Claims (7)

[Claims] 1. A method for evaluating the reliability of a system, wherein the distribution of the intervals of occurrence of errors that have already occurred in the system and whose factors have been eliminated, and the number of errors caused by the factors remaining in the system. Based on the distribution of the error occurrence intervals with the factors removed,
It is possible to estimate the distribution of error occurrence intervals due to the remaining factors, and calculate the reliability of the entire system from the estimated distribution of error occurrence intervals due to the remaining factors and the number of errors caused by the remaining factors. How to characterize. 2. The method according to claim 1, wherein the distribution of error occurrence intervals due to the obtained remaining factors,
A simulation is performed in which an error is artificially generated according to the number of errors generated by the remaining factor, the simulation result is compared with the error occurrence status in the actual system, and the calculation is performed according to the degree of similarity. How to evaluate the reliability of the entire system. 3. The method of claim 1 or 2, wherein the system is a software system that is a system that executes software. 4. A system reliability evaluation device for evaluating the reliability of a system, which comprises means for inputting information on the interval of occurrence of an error which has already occurred in the system and whose cause has been removed. Based on the information of the generated error interval, the distribution of the error interval that has already occurred in the system, and the factor has been removed, and means for estimating the number of errors caused by the factor remaining in the system, Based on the distribution of error occurrence intervals with factors removed,
A means for estimating the distribution of error occurrence intervals due to the remaining factors, the reliability of the entire system is calculated from the estimated distribution of error occurrence intervals due to the remaining factors, and the number of errors caused by the remaining factors A reliability evaluation device for a system, comprising: 5. A reliability evaluation apparatus for a system according to claim 4, wherein the distribution of error occurrence intervals due to the obtained remaining factors,
A simulation is performed in which an error is artificially generated according to the number of errors generated by the remaining factor, the simulation result is compared with the error occurrence status in the actual system, and the calculation is performed according to the degree of similarity. A reliability evaluation apparatus for a system, comprising: a determination unit that determines the reliability of the entire system. 6. The reliability evaluation apparatus for a system according to claim 5, wherein the reliability calculation means quantizes an error occurrence interval value and estimates an error occurrence interval distribution due to the remaining factor. According to the number of errors caused by the remaining factors, the number of errors is distributed to each of the quantized occurrence interval values, the reliability of the entire system is calculated according to the distribution, and the determination means calculates the reliability. When it is determined that the reliability of the entire system does not reach a predetermined level, the content of the quantization of the error occurrence interval and the distribution of the number of errors for each of the quantized occurrence interval values is changed. A system reliability evaluation device characterized in that 7. A reliability evaluation apparatus for a system according to claim 4, 5 or 6, wherein the system is a software system which is a system for executing software. apparatus.