WO2005109253A2

WO2005109253A2 - Method for the computer-assisted evaluation of the prognosis of characteristic values of a technical system, performed by means of a predictive model

Info

Publication number: WO2005109253A2
Application number: PCT/EP2005/052038
Authority: WO
Inventors: Benedikte Elbel; Veronika Dunkel; Michael Greiner; David Meintrup; Oliver MÄCKEL
Original assignee: Siemens Aktiengesellschaft
Priority date: 2004-05-05
Filing date: 2005-05-04
Publication date: 2005-11-17
Also published as: DE102004022142B4; DE102004022142A1; WO2005109253A3

Abstract

The invention relates to a method for the computer-assisted evaluation of the prognosis of characteristic values (A(t)) of a technical system, performed by means of a predictive model. According to said method, a total temporal interval (I) is first divided into a plurality of partial intervals (I1, I2, ...., I), a plurality of characteristic values (A(ti)) of the technical system having been determined in the total temporal interval (I). For each partial interval (I1, I2, ...., I), a partial adaptation (Pi) of the predictive model is performed on the characteristic values (A(ti)) determined in the partial interval (I1, I2, ...., I) in such a way that each partial interval (I1, I2, ...., I) is associated with a partial adaptation (Pi). A prognosis precision (I) is then determined for each partial adaptation (Pi). The partial adaptations fulfilling at least one pre-determined criterion are selected, and a temporal stability measure (L), depending on a stability interval extending essentially between the smallest and the largest partial interval end point of the partial intervals (I1, I2, ...., I) associated with the selected partial adaptations (Pi), is determined for the selected partial adaptations (Pi). A prognosis quality (M1; M2) is then determined by means of the temporal stability measure (L) and the prognosis precisions (i) of the selected partial adaptations (Pi), said prognosis quality representing an evaluation measure for the quality of the prognosis performed by means of the predictive model.

Description

description

Process for computer-aided evaluation of the prognosis of parameters of a technical system carried out by means of a prognosis model

The invention relates to a method for computer-aided evaluation of the prognosis of parameters of a technical system, which is carried out by means of a prognosis model, as well as a corresponding arrangement and a corresponding computer program dummy product.

Technical systems offer an ever increasing range of functions. The associated functions can often only be implemented by using processor means in a technical system on which program means are executed. In the case of a mobile phone, for example, processor means and the associated programming means are indispensable components of these technical systems. Errors in the programming resources generally lead to a partial or complete failure of the technical systems. Such failures can manifest themselves in the form of malfunctions or in the fact that certain technical functions of the technical system are no longer available. Failures can also damage the mechanics or electronics of the technical system. Finally, such failures can often cause serious collateral damage, such as the leakage of coolant from a refrigerator or a reactor vessel.

When developing a software-intensive technical system, the reliability of the system is therefore a key quality criterion. For analyzing the reliability of a technical system, a large number of reliability growth models are known from the prior art, with which the failure behavior of the technical system is predicted in the course of a test and / or correction process. examples For example, in Lawrence, Denis: "Software Reliability and Safety in Nuclear Reactor Protection Systems", US Nuclear Regulatory Commission, 1993, pages 101 to 105, forecast models are described with which the increase in the reliability of programming agents can be determined in a test period.

Nowadays, the selection of a suitable forecasting model is often based on the experience of experts. In addition, the goodness of adaptation was considered

Forecast model to the determined parameters of the technical system considered on the basis of error deviations. However, it is not possible to check the suitability of the forecast model with regard to long-term forecasts. There are also methods for evaluating the suitability of forecasting models (U-plot, Prequential Likelihood criterion or hold-out criterion, see e.g. Lyu, M.R .: "Handbook of Software Reliability Engineering", McGraw-Hill, New York, 1995). However, these also do not evaluate the suitability for long-term forecasts and are also computationally intensive.

The object of the invention is therefore to create a method for computer-aided evaluation of a forecast model, with which the suitability of the forecast model for a specific technical system can be checked in a simple manner, in particular also with regard to long-term forecasts.

This problem is solved by the independent claims. Further developments of the invention are defined in the dependent claims.

In the method according to the invention, a total time interval with a start point and an end point, in which a plurality of parameters of the technical system have been determined, is first divided into several subintervals, each subinterval lying between a subinterval start point and a subinterval end point in the overall interval. For Every partial interval is a partial adjustment of the forecast model to be evaluated to the parameters determined in the partial interval, whereby each partial interval is assigned to a partial adjustment. Finally, a forecast accuracy is determined for each partial adjustment, which is a measure of the accuracy of a forecast made with the partial adjustment of one or more parameters of the technical system. Finally, those that meet one or more predefined criteria are selected from the partial adjustments. A stability time measure is determined for these selected partial adaptations, which depends on a stability interval that lies essentially between the smallest and the largest partial interval end point of the partial intervals assigned to the selected partial adaptations. Finally, by means of the stability time measure and the

Forecast accuracy of the selected adjusted forecast models determines a forecast quality, which represents an evaluation measure for the quality of the forecast carried out with the forecast model.

Because a stability time measure is also included in the determination of the forecast quality, the suitability of the forecast model for long-term forecasts is also taken into account in the quality assessment, because it can be assumed that the longer-term the better the forecast, the more stable it is in the assessment time interval used.

Preferably, the subintervals are nested, i.e. each subinterval begins at the starting point of the total interval, so that subintervals with larger subinterval endpoints always include subintervals with smaller subinterval endpoints.

One of the specified criteria when selecting the partial adjustments is that all partial interval endpoints of the partial intervals assigned to the selected partial adjustments valle are temporally successive partial interval endpoints in the total interval and the largest partial interval endpoint of the partial intervals assigned to the selected partial adaptations is the end point of the total interval. A coherent stability time interval is therefore always considered, which always has the end point of the total interval as the end point.

In a preferred embodiment of the method according to the invention, the forecast accuracy of a respective partial adjustment is the deviation of one or more parameters forecast using the respective partial adjustment from one or more parameters determined in the total interval and / or one or more parameters forecast using a predetermined partial adjustment. The ascertained accuracy of the forecast of the respective partial adjustment is preferably the deviation of a parameter predicted by means of the respective partial adjustment at the end point of the total interval from a parameter determined at the end point of the total interval and / or from a parameter predicted by means of a predetermined partial adjustment at the end point of the total interval. The predefined partial adaptation is preferably an adaptation which is adapted to all parameters of the overall interval. In particular, the deviation of the predicted from the determined parameters is a relative deviation.

In one embodiment of the method according to the invention, the predetermined criterion when selecting the partial adjustments consists in the fact that the determined accuracy of the prognosis of a respective partial adjustment is better than a predetermined value. Thus, only if this criterion is met is the respective partial adjustment assigned to the group of selected partial adjustments.

In a further embodiment of the invention, there is a predetermined criterion for a respective partial adaptation. in that one or more parameters determined in the total interval lie within the forecasting interval of the respective partial adjustment. A forecast spreading interval is defined as a confidence interval for a chosen statistical significance when forecasting the target quantity.

In a further embodiment, a predefined criterion for a respective partial adjustment is selected such that the criterion is met if a parameter determined at the end point of the total interval lies within the forecasting interval of the respective partial adjustment.

In a preferred embodiment, the stability time measure is the length of the stability interval divided by the length of the total interval, i.e. the stability time measure is a relative value.

In a particularly preferred embodiment of the invention, the forecast quality is calculated using the following formula:

1-Δ Ml = - 3.003 «(l- +0.997

where Ml is the forecast quality, Δ the mean value of the forecast accuracy for the selected partial adjustments and L the length of the stability interval divided by the length of the total interval.

Alternatively, the forecast quality can be calculated using the following formula:

M2 = - ^{1 "Δ} 1 / (1- (1-) ² )

where M2 is the forecast quality, Δ the mean of the forecast accuracy for the selected partial adjustments and L the Length of the stability interval divided by the length of the total interval.

With the two above-mentioned forecast qualities M1 and M2, the greater the relative length L of the stability interval, the greater the value of the forecast quality. This means that a high value of the forecast quality stands for better suitability for long-term forecasts.

In a further preferred embodiment of the method according to the invention, the partial adaptation of the forecast model to the parameters determined in the respective partial interval takes place according to the maximum likelihood method and / or the method of the smallest squares of deviations. These methods are well known to the person skilled in the art from the prior art.

In a particularly preferred embodiment of the method according to the invention, the forecast to be evaluated is a reliability forecast, in particular a forecast carried out using a reliability growth model, and the parameters are values which represent the reliability of the technical system. The parameters preferably include the number of total failures of the technical system at the time the respective parameter was determined and / or the average time until a failure of the technical system occurred at the time the respective parameter was determined.

The method according to the invention is particularly suitable for a technical system which has processor means on which program means are executed, the evaluated prognosis being a reliability prognosis of the program means.

The method according to the invention can be used with different types of forecasting accuracy and / or with different predetermined criteria for the selection of the partial adjustment are carried out, an overall forecast quality being determined from the forecast qualities determined using the different types of forecast accuracy and / or predetermined criteria. In particular, this overall forecast quality is a weighted average of the forecast qualities determined using the different types of forecast accuracy and / or specified criteria. In this way, the forecast quality can be adjusted depending on the intended use of the forecast model to be evaluated. In particular, the forecast quality can be selected on the basis of the forecast targets set by a user of the method.

The total time interval used in the method according to the invention is preferably a test and correction phase of the technical system, in which phase the technical system was continuously adapted to improve its reliability. The forecast model evaluated with the method according to the invention thus serves in particular to estimate whether the length of a given test and correction phase of the technical system is sufficient to ensure a certain reliability of the system when it is used later.

The method according to the invention can in each case be repeated for a number of different forecast models, as a result of which a plurality of forecast qualities are obtained. The individual forecast qualities can then be compared and the forecast model with the best forecast quality can be used for the forecast of the parameters of the technical system.

In addition to the method described above, the invention further relates to an arrangement for computer-aided evaluation of the prognosis of parameters of a technical system carried out by means of a forecast model, the arrangement being designed such that the method according to the invention can be carried out with this arrangement. In addition, the invention relates to a computer program product, which in the Memory of a computer can be loaded and includes software code sections with which the inventive method is carried out when the program product is running on the computer.

Embodiments of the invention are described in detail with reference to the accompanying figures.

Show it:

Figure 1 is a diagram showing the sequence of an embodiment of the method according to the invention;

FIG. 2 shows a diagram which illustrates the determination of partial adaptations in the method according to the invention; and

Figure 3 shows a technical arrangement for performing the inventive method.

In the embodiment of the method according to the invention described below, a technical system is considered which has processor means on which program means can be executed. The aim of the method is to evaluate a prognosis that predicts the reliability of the program resources running on the technical system. Reliability parameters of the system were determined beforehand in a total time interval I, which represents a test phase of the technical system, the reliability parameters in the embodiment described here being the cumulative total failure numbers of the technical system determined at a predetermined point in time. In the test phase, the program resources of the technical system were subjected to extensive tests, whereby the errors found in the program resources were corrected. In a first step S101, the total interval I is divided into a plurality of subintervals Ii, I ₂ ,..., I, the subintervals always starting at the starting point of the total interval I and gradually increasing, so that the last subinterval corresponds to the total interval I. In one

Step S102 partially adjusts a forecast model to be evaluated with the method to the parameters A (t) (t is a point in time in the total interval I and A (t) is the total number of failures at point in time). Any of those known from the prior art can be used

Reliability growth model can be used as a forecast model, for example the model by Musa and Okumoto or a logarithmic model. These models include parameters that are adapted in step S102 to the failure numbers A (t) of the technical system in each subinterval Ii, I ₂ , ..., I. In this way, a large number of partial adjustments P- ^ are obtained.

Finally, in the next step S103, a forecast accuracy Δj_ is determined for each partial adjustment. In the embodiment described here, the forecast accuracy Δ ^ is the amount of the difference between the number of failures predicted at the end point of the total interval and the number of failures actually determined at the end point divided by the number of failures actually determined at the end point. Mathematically, Δ ^ can therefore be written as follows:

where A ± (t _{eri (} -ι) is the number of failures determined with the partial adjustment Pi at the end point t _en d of the total interval I and A (t _en d) is the actually determined number of failures at the time t _en d.

Alternatively, the relative deviation between the number of default predicted at the end point t _en and the number of default can also be used are considered, which was predicted with a partial adjustment adapted to all parameters of the total interval. In the latter case, the sensitivity of the method to random fluctuations in the event of failures at the end of the total interval is significantly reduced.

In the next step S104, those partial adaptations are selected that have a forecast accuracy Δi that is smaller than a predetermined maximum value α. Since the prognosis becomes better the larger the subinterval considered when adapting the forecast model, the subinterval endpoints that are assigned to the selected subadaptations are normally in the vicinity of the end point t _{θnd of} the total _interval I.

In order to take into account the sub-interval ranges in which a forecast with an accuracy smaller than the predetermined value α is achieved, a stability time measure L is determined in step S105, which depends on a stability interval. The stability interval lies between that

»The smallest and the largest partial interval end point of the partial intervals assigned to the selected partial adjustments. The stability time measure is the relative length of the stability interval and can be written mathematically as follows:

T - end * k _

where tk is the smallest partial interval end point of the partial intervals assigned to the selected partial adaptations.

Finally, a forecast quality M1 is determined in step S106 using the following formula:

Ml = - 3.003 • (1-LY +0.997 where Ml is the forecast quality, Δ is the mean value of the forecast accuracy for the selected adapted forecast models and L is the stability measure of time.

The constants 3.003 and 0.997 are chosen so that the term 1 — Δ, which becomes greater the higher the accuracy of the forecast, for a stability measure of time, which corresponds to 90% of the length of the total interval, by the value of the denominator, which for large Stability times are less than 1, is still reinforced. In contrast, the forecasting quality Ml is maximally quartered by the denominator for a short stability interval. By choosing Ml in this way, the period in which the forecasts are stable is taken into account when determining the quality of the forecast. The longer the stability interval, the greater the forecast quality, so that the value of the forecast quality expresses in particular whether the forecast model used is suitable for long-term forecasts.

Alternatively, the forecast quality can be represented by the following value M2:

M2 = - 1 / (1- (1-)

Here M2 stands for the forecast quality, Δ is the mean value of the forecast accuracy for the selected partial adjustments and L is the stability time measure defined above. The correction term in the denominator is an expression that is unlimited over the interval [0,1], whereby this expression can have a very large value in the case of a short stability phase and a value close to 1 in the case of a long stability interval. This means that the longer the stability interval, the greater the value M2, and M2 is also suitable for evaluating forecast models for long-term forecasts. By performing the method for a large number of prognostic models, the forecast model can thus be determined that is most suitable for a long-term forecast based on the forecast quality.

FIG. 2 shows a diagram which illustrates the adaptation of a forecast model to the parameters of a technical system. The abscissa of the diagram is the time axis t and the ordinate represents the number of total failures A. In the diagram, the total failures A (t) of the technical system measured in a total interval I between t ₀ and t _e at predetermined times are in the form of Shown measuring points. Different subintervals are taken into account for adaptation, the subintervals Iχ, I2 and I3 being shown in FIG. All subintervals start at time tg = 0 and increase in sequence. Interval Ii extends from 0 to tj_, interval I2 from 0 to 2 and interval I3 from 0 to t.3. The failure numbers A (t) in the individual partial intervals are used to adapt the forecast model under consideration. For every interval In _. , I2 and I3 thus result in three curves P _f Ε ^> 2 ^unc * 3r which represent a corresponding forecast of the number of failures on the basis of the measured number of failures in the corresponding intervals. As expected, the deviation of the number of failures determined with the forecast P3 from the actual number of failures A (t _en d) at the end point t _en d is smallest. According to the method according to the invention, a maximum percentage deviation α between the predicted and the actually ascertained number of failures at the end point t _e n _{d is} determined and then the partial adaptations of the forecast model are selected in which the deviation of the predicted from the actually ascertained number of failures is less than α , The forecast quality can then be determined with the aid of these partial adjustments and the stability time measure described in the preceding.

The method according to the invention was used for the forecasting model by Musa and Okumoto and for the forecasting model Log Power carried out. The values 10% and 5% were considered as the maximum deviation α of the forecast accuracy. It has been shown here that for α = 10% both forecast models are not very suitable for long-term forecasts, although the Log Power model received a higher forecast quality and thus a better rating due to better forecast accuracy. At α = 5%, the Model Log Power has a much higher forecast quality than the model from Musa and Okumoto, which is due to the fact that the stability time model of the Log Power model is much longer than the stability time model of the Musa and Okumoto model.

In the embodiment of the method according to the invention described above, the number of total failures was considered as a parameter. However, it is also possible to use other parameters to evaluate the forecast model, e.g. the average time until a technical system failure occurs. Furthermore, the criterion for selecting a partial adjustment does not have to be fixed by a fixed predetermined value α. It is also conceivable that a partial adjustment is always selected when the determined parameters of the technical system lie within the forecasting interval of the respective partial adjustment. Furthermore, forecast qualities can be determined for different types of parameters or selection criteria, which can then be combined to form an overall forecast quality according to the forecast goals desired by a user.

FIG. 3 shows a technical arrangement with processor means PRZE, on which program means can be executed. The processor means PRZE comprise a processor CPU, a memory MEM and an input / output interface IOS, which is used in different ways via an interface IFC. Output is visible on a monitor MON and / or output on a printer PRT via a graphic interface. An entry is made with the mouse MAS or KEYBOARD. The processor means PRZE also have a data bus BUS, which ensures the connection to the memory MEM, the processor CPU and the input / output interface IOS. Furthermore, additional components can be connected to the data bus BUS, for example additional memories, data memories in the form of a hard disk or a scanner. The technical arrangement can be used as a device for evaluating the prognosis of parameters of a technical system carried out by means of a prognosis model. In addition, the computer program product for carrying out the method according to the invention can be loaded into the memory MEM. It is also conceivable that the technical arrangement of FIG. 3 represents the technical system, the parameters of which are predicted using a forecast model, the forecast model being evaluated using the method according to the invention.

Claims

claims

1. Method for computer-aided evaluation of the prognosis of parameters of a technical system carried out by means of a prognosis model:

- In which a total time interval (I) with a starting point (t ₀ ) and an end point (t _end ) ri ⁿ a plurality of parameters (A <t)) of the technical system were determined, in several sub-intervals (Iχ, I _2nd , ...., I) is divided, with each subinterval lying between a subinterval start point (to) and a subinterval end point (t, t ₂ , ..., ten) i ™ - total interval (I); in which for each partial interval (Iχ, I ₂ , ...., I) a partial adjustment (P- ^) of the forecast model to the parameters determined in the partial interval (Ii, I ₂ , ...., I) (A (t )) is carried out, whereby each partial interval (Ii, I ₂ , ...., I) is assigned to a partial adjustment (Pj;

- in which a forecast accuracy (Δi) is determined for each partial adjustment (Pj), which is a measure of the accuracy of a forecast made with the partial adjustment (Pj) of one or more of the parameters (A (t)) of the technical system;

- Which are selected from the partial adjustments (P ^) that meet one or more predetermined criteria; in which a stability time measure (L) is determined for the selected partial adjustments (Pj), which depends on a stability interval iw between the smallest and the largest partial interval end point (ti, t ₂ , ..., t _end ) of the selected partial adjustments (Pj_) assigned subintervals (Ii, I ₂ , ...., I);

- In which a forecast quality (Ml; M2) is determined by means of the stability time measure (L) and the forecasting inaccuracies (Δi) of the selected partial adjustments (Pj) Represents a measure of the quality of the forecast carried out with the forecast model.

2. The method according to claim 1, wherein each sub-interval (Ii, I, ...., I) begins at the starting point (t ₀ ) of the total interval (I).

3. The method according to claim 1 or 2, in which a predetermined criterion consists in that all partial interval end points (ti, t ₂ , ..., t _end ) of the partial intervals (Ii, I ₂ , ..) assigned to the selected partial adaptations (Pi). .., I) are successive partial interval end points (Pj in the total interval (Iα, I ₂ , ...., I) and the largest partial interval end point of the partial intervals (Iχ, I ₂ , ...., assigned to the selected partial adaptations (P), I) is the end point (t _end ) of the total interval (I).

4. The method according to any one of the preceding claims, in which the determined accuracy of the forecast (A) of a respective partial adjustment (Pi) is the deviation of one or more parameters (Äi (t)) predicted by means of the respective partial adjustment (P) from one or more in the total interval ( I) determines the determined parameters (A (t)) and / or one or more parameters (Äi (t)) predicted by means of a predetermined partial adjustment.

5. The method according to any one of the preceding claims, in which the determined accuracy of the forecast (Δi) of a respective partial adaptation (Pi) is the deviation of a parameter (Äi (t)) predicted by means of the respective partial adaptation (Pi) at the end point (t _enc j) of the total interval (I) from a parameter (A (t)) determined at the end point (t _e n) of the total interval (I) and / or from a parameter (Äi (t)) predicted by means of a predetermined partial adjustment (Pi) at the end point ( t _end ) of the total interval (I).

6. The method according to claim 4 or 5, wherein the predetermined partial adjustment is an adjustment of the forecast model to all determined parameters (A (t)) of the total interval (I).

7. The method according to any one of claims 4 to 6, wherein the deviation of the predicted parameters (Äi (t)) from the determined parameters (A (t)) is a relative deviation.

8. The method according to any one of claims 4 to 7, wherein a predetermined criterion for a respective partial adjustment (Pi) consists in that the determined accuracy of the forecast (Δi) of a respective partial adjustment (Pi) is better than a predetermined value (α).

9. The method as claimed in one of claims 1 to 3, in which a predetermined criterion for a respective partial adjustment (Pi) is that one or more parameters (A (t)) determined in the total interval (I) lie within the forecasting interval of the respective partial adjustment ,

10. The method as claimed in claim 9, in which a predetermined criterion for a respective partial _adjustment consists in that a parameter (A (t)) determined at the end point (t _enC i) of the total _interval (I) within the forecasting interval of the respective partial adjustment (Pi) lies.

11. The method according to any one of the preceding claims, wherein the stability time measure (L) is the length of the stability interval divided by the length of the total interval (I).

12. The method according to any one of the preceding claims, wherein the forecast quality (Ml; M2) is as follows: Ml = - ^1_Δ 3.003 • (1-L) ³ +0.997 where Ml is the forecast quality, Δ is the mean of the forecast accuracy (Δi) for the selected partial adjustments (Pi) and L is the length of the stability interval divided by the length of the total interval (I).

13. The method according to any one of the preceding claims, wherein the forecast quality (Ml; M2) is as follows:

where M2 is the forecast quality, Δ the mean of the forecast accuracy (Δ) for the selected partial adjustments (Pi) and L the length of the stability interval divided by the length of the total interval (I).

14. The method as claimed in one of the preceding claims, in which the partial adaptation of the forecast model to the parameters (A (t),...) Determined in the respective partial interval (Ii, I ₂ , ...., I) according to the maximum likelihood method and / or the method of least squares.

15. The method as claimed in claim 1, in which the evaluated forecast is a reliability forecast, in particular a forecast carried out using a reliability growth model, and the parameters (A (t)) are values which represent the reliability of the technical system.

16. The method according to claim 15, wherein the parameters (A (t)) the number of total failures of the technical system for

Include the time of determination of the respective parameter (A (t)) and / or the average time until a failure of the technical system occurs at the time of determination of the respective parameter (A (t)).

17. The method according to claim 15 or 16, wherein the technical system has processor means on which program means are executed, the evaluated prognosis being a reliability prognosis of the program means.

18. The method as claimed in one of the preceding claims, in which the method is carried out with different types of forecasting accuracy (Δi) and / or with different predefined criteria for the selection of the partial adjustments (Pi) and from the with the different types of forecasting accuracy ( Δi) and / or predefined criteria determined forecast quality (Ml; M2) an overall forecast quality is determined.

19. The method according to claim 18, wherein the overall forecast quality is a weighted average of the forecast quality (Ml; M2) determined with the different types of forecast accuracy (Δi) and / or predetermined criteria.

20. The method according to any one of the preceding claims, wherein the total time interval (I) represents a test and correction phase of the technical system, in which phase the technical system was continuously adapted to improve its reliability.

21. The method according to any one of the preceding claims, in which the method is repeated for several different forecast models, whereby a plurality of forecast qualities (Ml; M2) is obtained.

22. Arrangement for the computer-aided evaluation of the prognosis of parameters (A (t)) of a technical system carried out by means of a prognosis model, the arrangement being designed in such a way that a method according to one of the preceding claims can be carried out.

23. A computer program product that can be loaded into the memory of a computer and comprises software code sections with which a method according to one of claims 1 to 21 is carried out when the program product is running on the computer.