WO2022018912A1

WO2022018912A1 - Prediction system, prediction method, and display device

Info

Publication number: WO2022018912A1
Application number: PCT/JP2021/015566
Authority: WO
Inventors: 晃治陰山; 茂寿崎村; 智裕山本; 秀之田所; 亜由美渡部
Original assignee: 株式会社日立製作所
Priority date: 2020-07-21
Filing date: 2021-04-15
Publication date: 2022-01-27
Also published as: JP2022021042A; JP7436313B2

Abstract

This prediction system for presenting a prediction value by a prediction model acquired by adjusting parameters through usage of training data, presents an operator with a degree of reliability of the prediction value in consideration of both of an influence of input values of respective input items on an output value and a width of extrapolation of input values serving as an extrapolation condition. This prediction system includes a prediction model which is trained by using training data at a training stage. The prediction system is characterized by being provided with: an input unit for providing an input value for each input item to be provided to the prediction model at a prediction stage; a sensitivity coefficient calculation unit that calculates a sensitivity coefficient indicating an influence of each input item of the prediction model on the prediction value obtained from the prediction model; a distance coefficient calculation unit that calculates a distance coefficient indicating a degree of deviation from a statistic value of training data regarding the input value for each input item; an out-of-training range index calculation unit that calculates an out-of-training range index from the sensitivity coefficient and a value of the distance coefficient; and a display unit that displays one or both of the sensitivity coefficient and the distance coefficient, or a value of the out-of-training range index as a numerical value or a drawing.

Description

Prediction system, prediction method, and display device

The present invention relates to a prediction system, a prediction method, and a display device that support an operator who judges driving based on a prediction value of machine learning.

Predictive systems that obtain predicted values using statistical prediction models such as neural networks, deep learning, and multiple regression analysis, which are machine learning, are being put to practical use in various fields. These prediction models give training data in which the values of input items and output items are aligned in advance at the learning stage, and adjust the values of the parameters in the prediction model to appropriate values so that the calculated values of the prediction model match as much as possible. .. After that, in the prediction stage, an input value is given to this prediction model and the prediction value is calculated.

Since the prediction model is adjusted to match the training data given to the training stage as much as possible, the prediction value obtained when the input value within the range of the training data is given is likely to be close to the correct value, but the training If the data is out of range, that is, if the input value of the extrapolation condition is given, an appropriate predicted value may not be obtained, and the reliability is low.

Especially when using predictive models such as neural networks and deep learning, it is desirable to use a sufficiently large amount of training data at the learning stage. If it is necessary to move to the prediction stage before a sufficiently large amount of training data can be obtained due to time constraints, etc., the input value of the extrapolation condition may be given in the prediction stage. Since neural networks and deep learning are non-linear, the predicted values when the input values of extrapolation conditions are given may deviate significantly.

As a result, it is difficult for operators who operate plants and equipment with reference to the predicted values to judge how much they can trust the predicted values. Even if it is found that an input value that is too large or too small compared to the time of learning is included, if the sensitivity characteristic of the input item is not known, it is not possible to know how much the predicted value is affected.

In relation to such a problem, the techniques described in Patent Document 1 and Patent Document 2 are known.
In Patent Document 1, when operating in an operating region outside the data range on which the parameters and torque statistical model of the internal combustion engine are based (that is, in the case of extrapolation conditions), the physical quantity affected by the torque deviates from the reference value. If so, it is corrected by the correction coefficient. As a result, it is possible to provide a vehicle control device capable of controlling the engine torque of the internal combustion engine that drives the vehicle with high accuracy.

Patent Document 2 describes that a redundant input is detected by analyzing the sensitivity characteristic of the output when the input of the neural network is changed. By deleting the detected redundant input, the optimum input can be selected.

Japanese Unexamined Patent Publication No. 2009-287520 Patent No. 3329806

However, Patent Document 1 does not describe the function of presenting the degree of extrapolation conditions to the operator. In addition, there is no description about the sensitivity characteristics of how much the input value of each input item affects the output value. Patent Document 2 has no description related to extrapolation conditions.

Therefore, if A: the input item that has a large influence on the predicted value is an extreme extrapolation condition and the reliability of the predicted value is significantly low, B: the input item that has a large influence on the predicted value is extrapolated slightly. If the condition is a condition and the reliability of the predicted value is slightly low, C: If the input item that has an extremely small effect on the predicted value is an extreme extrapolation condition and the reliability of the predicted value is slightly low, It was not possible to distinguish and judge. Also, there is no description about the function of presenting the degree of extrapolation condition to the user.

The present invention has been made in view of these problems. The problem to be solved by the present invention is the influence of the input value of each input item on the output value and the extrapolation condition in the prediction system that presents the predicted value by the prediction model obtained by adjusting the parameters using the training data. The degree of reliability of the predicted value is presented to the operator in consideration of both the extrapolation width of the input value.

From the above, in the present invention, "a prediction system including a prediction model that learns using learning data in the learning stage, and an input unit that gives an input value of each input item given to the prediction model in the prediction stage. The sensitivity coefficient calculation unit that calculates the sensitivity coefficient that shows the effect of each input item of the prediction model on the predicted value obtained by the prediction model, and the distance coefficient that shows the degree of deviation from the statistical value of the learning data regarding the input value of each input item. A distance coefficient calculation unit that calculates It is a prediction system characterized by having a display unit that displays a numerical value or a figure. "

Further, in the present invention, "a prediction method for making a prediction by a prediction model learned using training data in the learning stage, and in the prediction stage, the input value of each input item given to the prediction model is obtained and the prediction model is obtained. Calculate the sensitivity coefficient that indicates the effect of each input item on the predicted value obtained by the prediction model, calculate the distance coefficient that indicates the degree of deviation from the statistical value of the training data for the input value of each input item, and use the sensitivity coefficient. A prediction method characterized by calculating an out-of-learning index from the value of the distance coefficient and displaying the value of either or both of the sensitivity coefficient and the distance coefficient, or the value of the out-of-learning index as a numerical value or a figure. " It is a thing.

Further, in the present invention, "a display device used in a prediction system including a prediction model that learns using training data in the learning stage, and the display device predicts the input value of each input item given to the prediction model. A sensitivity coefficient that indicates the effect on the predicted value obtained by the model, a distance coefficient that indicates the degree of deviation from the statistical value of the training data regarding the input value of each input item, and a learning range calculated from the values of the sensitivity coefficient and the distance coefficient. The external index is a display device characterized by displaying either or both of the sensitivity coefficient and the distance coefficient, or the value of the index outside the learning range as a numerical value or a figure. "

According to the present invention, the operator can know the index value in consideration of both the influence of the input value of the input item as the extrapolation condition on the predicted value and the degree of extrapolation.

As a result, according to the embodiment of the present invention, the operator can say, for example, when A: an input item having a large influence on the predicted value is an extreme extrapolation condition and the reliability of the predicted value is significantly low, B: prediction. If the input item that has a large effect on the value has a slight extrapolation condition and the reliability of the predicted value is slightly low, C: The input value that has an extremely small effect on the predicted value is an extrapolation condition. If the reliability of the predicted value is not so low, it can be judged quantitatively by distinguishing.

That is, the operator grasps the reliability of the predicted value calculated by the prediction system, believes the predicted value when the reliability is high, and considers the safety factor in the predicted value when the reliability is low. The values can be used to operate plants and equipment. As a result, the operator can reduce the risk of driving failure.

The functional block diagram which shows the whole structure example of the prediction system which concerns on embodiment of this invention. The figure which shows the function example which obtains the distance coefficient from the input value of an input item when two or more large and small statistical values are used. The figure which shows the function example which obtains the distance coefficient from the input value of an input item when one statistical value is used. The figure which shows the display example which displays the predicted value, the measured value, and the value of the index outside the learning range as a line graph. The figure which shows the display example which displays the predicted value and the measured value as a bar graph. The figure which shows the display example which displays the value of the index outside the learning range as a bar graph. The figure which shows the display example which displays the breakdown of the value of the index out of learning range as a stacked area graph. The figure which shows the display example which displays the breakdown of the value of the index out of learning range as a stacked bar graph. The figure which shows the example of the simulated data generated by the formula (3) for learning the formula (4). The figure which shows the correlation example of the value of the simulated data and the predicted value by a prediction model. The figure which shows the bar graph example of the prediction error with respect to the value of input X ₁ and X _2. The figure which shows the bar graph example of the index which is out of learning range for the value of input X ₁ and X _2. The figure which shows the correlation example of the index which is out of learning range and prediction error.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

The target of the prediction system of the present invention is specifically the operation of a rainwater pump, the operation of a water treatment plant, the operation of an industrial plant, etc., but the prediction model is identified by training data at the learning stage such as a neural network. It is not limited to these as long as a person obtains some predicted value based on the predicted model and sets an operating condition with reference to the predicted value.

FIG. 1 is a functional block diagram showing an overall configuration example of a prediction system according to an embodiment of the present invention. The present invention is a prediction system that supports an operator who makes a judgment based on a prediction value of machine learning, and the entire system includes a learning stage and a prediction stage, but FIG. 1 mainly shows a prediction stage. The prediction system that performs the processing of is described.

Therefore, the learning data D1 used in the learning stage is stored in the statistical value storage unit DB of the learning data. In the present invention, the learning method for obtaining the learning data D1 does not matter, but the learning data D1 is composed of normal data and does not include abnormal data, and is data having a value within a predetermined range. It is configured.

On the other hand, in the input unit 10, input values D2 of each of a plurality of input items, which are appropriately detected from a plant or the like, are generally obtained. In this case, the input value D2 may be given an input value outside the range of the training data, that is, an extrapolation condition. In FIG. 1, each input value D2 of an input item is given from the input unit 10 to the input value normalization calculation unit 12. The normalization calculation unit 12 converts the value of each given input item into a numerical value having a certain width, and outputs the normalized input value D2U. The normalized input value D2U is given to the distance coefficient calculation unit 18 and the sensitivity coefficient calculation unit 22.

The distance coefficient calculation unit 18 is also given the statistical value D1U of the training data after normalization. The statistic value D1U of the training data after normalization is a value obtained by normalizing the statistic value D1 of the training data of each input item given from the statistic value storage unit DB of the training data in the statistic value normalization calculation unit 15. be. The distance coefficient calculation unit 18 calculates the distance coefficient L based on the input value D2U after normalization and the statistical value D1U of the learning data after normalization, and gives it to the index calculation unit 26 outside the learning range.

On the other hand, the sensitivity coefficient calculation unit 22 gives the input value D2U1 in which one variable of the input value D2U after normalization is perturbed to the prediction model M1 for sensitivity analysis. The prediction model M1 for sensitivity analysis calculates the predicted value DS1 when a perturbation is given and gives it to the sensitivity coefficient calculation unit 22. The sensitivity coefficient calculation unit 22 obtains the sensitivity coefficient K based on the predicted value DS1 when a perturbation is given, and gives it to the out-of-learning range index calculation unit 26.

The out-of-learning index calculation unit 26 calculates the out-of-learning index D3 based on the distance coefficient L and the sensitivity coefficient K and gives it to the display unit 30. The display unit 30 displays the given learning range index D3 on the screen. Further, the input value D2U after normalization is also given to the prediction model M2 for predicting value calculation, and the predicted value DS2 is obtained. If the predicted value DS2 is obtained, the predicted value D2 may be given to the predicted value calculation prediction model M2 as well. The predicted value DS2 is also given to the display unit 30, and is displayed on the screen together with the learning range index D3.

The display unit 30 is given a distance coefficient L and a sensitivity coefficient K, and these may be displayed. Although not shown, not only the out-of-learning index D3 and the predicted value DS2 but also various data D1 and D2 input to the prediction system and various data D1 and D2 input to the prediction system are used for the display unit 30. All data including various intermediate data generated by the processing can be displayed, and the display format can be appropriate.

An example of a specific method for calculating the input value D2U after normalization in the input value normalization calculation unit 12 will be described below. As the input value D2 of each input item, it is assumed that numerical values of various scales are given. For example, the digits may differ depending on the input item, such as 5 ° C to 40 ° C at room temperature and 980 hPa to 1020 hPa at atmospheric pressure. If these numerical values are used as they are, the influence of input items with large numerical values may become excessive, so dimensionlessization is made so that they can be used on an equal footing. A linear transformation may be used for normalization, or an affine transformation may be used. By this treatment, the above-mentioned room temperature and atmospheric pressure are all converted into values in the range of 0.0 to 1.0, for example. The input value D2U obtained in this way after normalization is given to the distance coefficient calculation unit 18 and the sensitivity coefficient calculation unit 22.

An example of a specific method for calculating the statistical value D1U of the learning data after normalization in the statistical value normalization calculation unit 15 will be described below. First, the statistical value D1 of the learning data of each input item is given from the statistical value storage unit DB of the learning data. As the statistical value D1 of the learning data of each input item, it is conceivable that numerical values of various scales are given.

For example, the minimum temperature is 5.6 ° C, the maximum temperature is 28.4 ° C, the average temperature is 16.45 ° C, the standard deviation is 7.3 ° C, and the minimum pressure is 1004.8 hPa, the maximum pressure is 1016.7 hPa, and the average. In some cases, the digits are significantly different, such as air temperature 1011.2 hPa and standard deviation 3.69 hPa. If these numerical values are used as they are, the influence of input items with large numerical values may become excessive, so dimensionlessization is made so that they can be used on an equal footing.

A linear transformation may be used for normalization, or an affine transformation may be used. By this treatment, the above-mentioned room temperature and atmospheric pressure are all converted into values in the range of 0.0 to 1.0, for example. The normalization method needs to be exactly the same as the input value normalization calculation unit 12. As a result, the input value D2U after normalization and the statistical value D1U of the learning data after normalization can be compared and evaluated on the same scale by the distance coefficient calculation unit 18.

An example of a specific method for calculating the distance coefficient L in the distance coefficient calculation unit 18 will be described below. This distance coefficient L is an index showing how much extrapolated points are extrapolated. FIG. 2 is an example showing a function for obtaining a distance coefficient from a value of an input item when two or more large and small statistical values are used among the learning data D1 stored in the statistical value storage unit DB of the learning data. The horizontal axis shows the input value X, and the vertical axis shows the distance coefficient L. For example, when there are two large and small statistical values such as the maximum value and the minimum value of the learning data D1 stored in the statistical value storage unit DB of the learning data, this is used as the input value X on the horizontal axis in FIG. 2 to be the maximum. The values Xmax and the minimum value Xmin are described. If the value of the input value X is between Xmin and Xmax, it is considered to be close to the interpolation condition or the interpolation condition of the training data D1, and the value of the distance coefficient L in FIG. 2 is set to 0. When the input value X is smaller than Xmin, the value of the distance coefficient L becomes smaller as the value increases from Xmin. When the input value X is larger than Xmax, the value of the distance coefficient L becomes larger as the value increases from Xmax. The line graph in FIG. 2 is shown as a straight line for the sake of simplicity, but it does not have to be a straight line.

If the "exclusion condition" is defined accurately, the pair of Xmin and Xmax will be "minimum value, maximum value", but if the value at the end of the distribution can be regarded as close to the extrapolation condition, other than this. Also, "mean value-standard deviation x α, mean value + standard deviation x α (where α is a positive number)" or the like may be used.
The larger the value of α, the closer the pair of Xmin and Xmax is to the “minimum value, maximum value”, and the closer to the more accurate “extrapolation condition”. In the present invention, the method of determining Xmin and Xmax is not limited to these if two large and small values are used.

FIG. 3 shows an example showing a function for obtaining the distance coefficient from the value of the input item when one statistical value is used in the distance coefficient calculation unit 18. FIG. 3 also shows the input value X on the horizontal axis and the distance coefficient L on the vertical axis. For example, regarding the learning data D1 stored in the statistical value storage unit DB of the learning data, when one statistical value exists like the average value of the learning data D1, this is described as Xcenter in FIG. The closer the value of the input value X is to the Xcenter, the closer the value of the distance coefficient L in FIG. 3 is to 0. When the input value X is smaller than the Xcenter, the value of the distance coefficient L becomes smaller as the value becomes farther from the Xcenter, taking a negative value. When the input value X is larger than the Xcenter, the value of the distance coefficient L becomes larger as the value is farther from the Xcenter. For the sake of simplicity, the function of FIG. 3 shows a quadratic function folded at the point of Xcenter, but the function is not particularly limited as long as it shows the same tendency.
As this Xcenter, a mode value, a median value, or the like may be used in addition to the average value.

An example of a specific method for calculating the sensitivity coefficient K in the sensitivity coefficient calculation unit 22 will be described below. The sensitivity coefficient K indicates how much the predicted value of the sensitivity analysis prediction model M1 changes when a minute perturbation is applied to the input value of the sensitivity analysis prediction model M1. The sensitivity coefficient calculation unit 22 first gives a perturbation according to a preset perturbation width to the given input value D2U after normalization, and first generates an input value D2U1 to which the perturbation is given. If the perturbation width is, for example, 0.02 and the value of one variable with the normalized input value D2U is 0.5, then 0.5-0.02 / 2 = 0.49 and 0.5 + 0.02 /. Find two points of 2 = 0.51. Then, the values of the other input items are given to the sensitivity analysis prediction model M1 as the input value D2U1 to which the perturbation is given without changing.

In the sensitivity analysis prediction model M1, the predicted value when the value of one variable is 0.49 and 0.51, that is, the predicted value DS1 when a perturbation is given is calculated and given to the sensitivity coefficient calculation unit 22. .. The sensitivity coefficient calculation unit 22 calculates the influence of the input item to which the perturbation is given on the predicted value, that is, the sensitivity coefficient K, by dividing the difference of the predicted value DS1 when the perturbation is given by the perturbation width. This value can be a positive number or a negative number.

It is desirable to use the normalized input value D2U as the target to be perturbed by the sensitivity coefficient calculation unit 22, but for example, when the given input value is a jump value, the prediction when the perturbation is given. The value DS1 and the sensitivity coefficient K can be extreme values. Since this has an undesired effect on the calculation of the out-of-learning index D3, the latest value is not used as the normalized input value D2U as the target to be perturbed by the sensitivity coefficient calculation unit 22. It is also possible to obtain in advance a value considering the past values such as the average value, the mode value, and the median value of the past n points including the point of, and give a perturbation to it.

The out-of-learning index calculation unit 26 calculates the out-of-learning index D3 from the distance coefficient L and the sensitivity coefficient K. An example of a specific method of this calculation method will be described below. First, it is assumed that there are three input values of the prediction model M1 for sensitivity analysis, and they are called X ₁ , X ₂ , and X ₃ , respectively. Further, the predicted value obtained by the sensitivity analysis prediction model M1 is referred to as Y. Since the sensitivity coefficient K is a change in the predicted value when a minute perturbation is applied to the input value as described above, it can be expressed as the following equation (1).

Further, the distance coefficient L can be calculated by giving _{X 1} , X ₂ , and X ₃ to a function as shown in FIG. 2, for example. Since there are three types of input values, three distance coefficients L are also obtained, and these are called L ₁ , L ₂ , and L ₃ . Using these variables, the out-of-learning index D3 can be obtained, for example, by the following equation (2).

When the value of the out-of-learning index D3 is 0, there is no input value as an extrapolation condition, or there is an input value as an extrapolation condition, but the sum of the three terms has no effect on the predicted value Y. Means that. It can be said that the predicted value DS2 obtained by the prediction model M2 for calculating the predicted value in this case has high reliability. On the contrary, when the value of the index D3 outside the learning range deviates from 0, it means that the input value of the extrapolation condition is included and the influence on the predicted value Y cannot be ignored. It can be said that the predicted value DS2 obtained by giving the input value in this case to the predicted value calculation prediction model M2 has low reliability. The value of the out-of-learning index D3 obtained by Eq. (2) is a measure of how much the predicted value DS2 obtained by the predicted value calculation prediction model M2 deviates to either the positive or negative direction. Become.

Equation (2) is _{the sum of three terms for the three input values X 1} , X ₂ , and X ₃ , and each term can be either a positive value or a negative value. If there are positive and negative terms, the effects of each term will be offset. When offset, the value of the out-of-learning index D3 approaches 0, but the individual input values themselves may be extreme extrapolation conditions. If it is not necessary to estimate how much the predicted value DS2 obtained by the predicted value calculation prediction model M2 deviates in the positive or negative direction, and only whether or not it is an extrapolation condition is evaluated, the learning range. The extrapolation index D3 may be calculated as the sum of the values obtained by taking the absolute values for each term, for example, as shown by the equation (3).

The mathematical formulas shown in the formulas (2) and (3) are the simplest examples, and are particularly suitable for formulas that calculate the out-of-learning index D3 based on the values of the sensitivity coefficient K and the distance coefficient L. There is no limitation.

The out-of-learning index D3 calculated by the out-of-learning index calculation unit 26 is displayed on the display unit 30. The display unit may be a computer, a tablet, a screen of a mobile terminal, or the like. Further, the predicted value DS2 calculated by the predicted value calculation prediction model M2 is also displayed on the display unit 30. These displays may be numbers, but may also be figures such as graphs. In addition, by displaying not only the latest predicted value DS2 but also the past predicted value and the measured value, it is possible to confirm how correct the past predicted value was, and at the same time, the prediction error at that time and the index D3 outside the learning range. It is even more useful because you can understand the relationship between them. An example of the display screen is shown in FIGS. 4 to 8.

FIG. 4 is a diagram showing a display example in which the predicted value DS2, the measured value D2U, and the value of the out-of-learning range index D3 are all displayed as a line graph. An example of the display screen 90 of FIG. 4 is composed of two graphs, an upper graph and a lower graph. The upper graph is a trend graph with a broken line showing the measured value D2U and the predicted value DS2 on the vertical axis with respect to the elapsed time on the horizontal axis. The elapsed time 0 indicates the present, and the measured value D2U is plotted because there are values up to this elapsed time 0, and for the future elapsed time 1, there is no measured value D2U yet and only the predicted value DS2 is shown. There is. In this way, it is possible to visually confirm how much the predicted value DS2 deviates from the measured value D2U by the elapsed time 0, and at the same time, the predicted value DS2 at the future elapsed time 1 can be known at a glance.

The lower graph of FIG. 4 is a trend graph with a polygonal line of the out-of-learning range index D3 having the same horizontal axis of elapsed time as the upper graph. In this way, the transition of the out-of-learning index D3 up to the future elapsed time 1 can be visually confirmed. In the example of FIG. 4, it can be seen that the value of the out-of-learning index D3 of the future elapsed time 1 is close to 0 and small. Since this small value means that the degree of extrapolation condition is small, it can be trusted that the predicted value DS2 of the future elapsed time 1 in the upper graph will be correct. Since the value of the out-of-learning index D3 of the future elapsed time 1 is close to the value of the out-of-learning index D3 of the elapsed time -5, the predicted value DS2 is the measured value at the time of the elapsed time -5 in the upper graph. It can be inferred that it will match.

The horizontal axis of FIG. 4 shows the elapsed time from -6 to 1, but the width of this elapsed time may be changed. Further, although the future value is also shown here up to the elapsed time 1, not only one point but also a plurality of future predicted values DS2 and the value of the out-of-learning range index D3 may be shown. When showing the future predicted value DS2 at multiple times and the value of the out-of-learning range index D3, the predicted value one hour ahead as the predicted value DS2 displayed from the elapsed time -6 to 0 in this FIG. There are multiple forecast values two hours ahead and future forecast values. A plurality of graphs may be displayed, or may be narrowed down and displayed when it becomes complicated.

The graph on the upper side of FIG. 4 shows an example of a trend graph with a line, but it may be a bar graph as shown in FIG. FIG. 5 is a diagram showing a display example in which the predicted value DS2 and the measured value D2U are displayed as a bar graph. In this figure, two bar graphs of the measured value D2U and the predicted value DS2 are displayed according to the passage of time. Further, although the graph on the lower side of FIG. 4 is also a trend graph with a line, it may be a bar graph as shown in FIG. FIG. 6 is a diagram showing a display example in which the value of the index D3 outside the learning range is displayed as a bar graph. For the purpose of comparison with past values, the bar graphs shown in FIGS. 5 and 6 may be easier to see.

When calculating the out-of-learning index D3 by summing the absolute values of the terms of a plurality of input items as in the above equation (2), the out-of-learning index is displayed by displaying the value of each term. It is possible to indicate which input item affects the value of D3 to what extent. A display example in that case is shown in FIG.

FIG. 7 is an example of a display unit that displays the breakdown of the value of the index D3 outside the learning range as a stacked area graph. For example, in the breakdown of the learning range indicator D3 in terms of the elapsed time 0, we can be seen that the influence of the input X ₃ accounted for the majority. Therefore, it is possible to check the obtained data, measuring instruments, communication environment, etc., such as whether the value of the input X _{3 is an outlier such as noise.} The stacked area graph may be colored, or may be colored according to whether the value in the term before the absolute value is given is a positive value or a negative value. This makes it possible to grasp which term and which term cancel each other out.

To show the degree of each item, a stacked bar graph may be used as shown in FIG. 8 instead of the stacked area graph. Even in this figure, it is possible to determine which input item causes the out-of-learning range index D3 to increase. The stacked bar graph may also be colored, or may be colored according to whether the value in the term before the absolute value is given is a positive value or a negative value.

As shown in FIG. 4, it is also possible to display the predicted value DS2 and the measured value D2U on the upper side of the display screen 90 of the display unit 30, and display the out-of-learning range index D3 on the lower side, but reduce the information to be viewed. For the purpose of making it easier to understand, only the upper figure may be displayed. In that case, the result of the out-of-learning index D3 will be included in the upper figure. This can be achieved by changing it according to the value. The value of the out-of-learning index D3 is small to 0 and the reliable predicted value DS2 is a dark black line or legend. Conversely, the value of the out-of-learning index D3 is extremely large or extremely small, so it is considered to be unreliable. In this case, the predicted value DS2 may be a light black line or a legend, or a line or a legend close to red.

The above display example is based on the distance coefficient L indicating how much the input value D2 taken into the prediction system deviates from the learning data D1 used in the learning stage, and the sensitivity coefficient K when a perturbation is given. , These two indexes are aggregated into one index, the out-of-learning range index D3, and expressed as various graphs and trends as time-series information in the prediction system. Further, it is expressed as various graphs and trends in comparison with the predicted value DS2 and the measured value D2U.

These notations are for the purpose of providing the operator with judgment material that allows the operator to judge whether or not the predicted value should be adopted for the operation of the plant, for example, through the display, and the method of providing the data is other than the graph and the trend display. There is also a display (numerical display) of raw data. In the raw data display, various data of each input time are displayed in a table format, which includes a distance coefficient L, a sensitivity coefficient K, and an out-of-learning range index D3. In the numerical display of the present invention, it is preferable to display either or both of the distance coefficient L and the sensitivity coefficient K, or the out-of-learning index D3 which is one index obtained by aggregating these. Further, it is preferable that the numerical display includes the predicted value DS2 and the measured value D2U.

Hereinafter, in order to specifically show the effect of the present invention, an example of evaluation calculation will be described. _{It is assumed that there are input values X 1} and X ₂ consisting of two input items, and the value Y affected by the input values is actually in the relationship of equation (4). Operator is not at all know this (4) of the relationship, only the data of X _{1, X} 2, _Y is that obtained as the real measured value D2U.

It is assumed that the predicted value is calculated by the simple prediction model of Eq. (5) for this actual measured value D2U. The prediction model in this case is the prediction model M2 of FIG.

This prediction model contains three coefficients a, b, and c, which need to be identified by the training data D1 at the learning stage. FIG. 9 is a diagram showing an example of simulated data generated by the equation (3) for learning the equation (4). _{Therefore, simulated data X 1} and X ₂ are generated as shown in FIG. 9 according to the relationship of Eq. (5), and the value of Y at this time is obtained. From these results, these are used as learning data D1 and three coefficients a, The values of b and c were identified. Specifically, X ₁ is changed in 0.2 increments as a range from 0 to 1.0, and _{X 2} is as a range from 0 to 1.0 in 0.2 increments for each value of _{X 1.} By changing, the value of the output Y at the time of each combination was obtained.

As a result, a = 1, b = 0.01, c = −0.13467 were obtained. The value of the simulated data and the predicted value by the prediction model of the formula (5) are shown in FIG. 10 as a correlation diagram. In FIG. 10, the horizontal axis represents the value in the range of 0 to 1.0 as the value of the simulated data, and the vertical axis represents the value in the range of 0 to 1.0 as the predicted value by the prediction model. ^{In this case, R 2} which is an index showing the degree of correlation is 0.9214, which is a _{relatively good model under the interpolation conditions of X 1} and X _{2 in} FIG. 9 (both are in the range of 0 to 1). You can see that.

On the other hand, we evaluated how much the predicted value by the prediction model differs from the reality when the input value that is the extrapolation condition is given. As shown in FIG. 9, the training data is in the range of 0 to 1.0 for both _{X 1} and X ₂ _{, so when the value of X 1} is set to 1.1 to 1.5 as an example of extrapolation conditions. And X ₂ are assumed to be 1.1 to 1.5. FIG. 11 shows the result of obtaining the prediction error of the predicted value calculated by the prediction model. FIG. 11 is a bar graph of prediction errors for the values of _{inputs X 1} and X _2. In this figure, _{a combination of two} _{inputs X 1} and X 2 is adopted on the horizontal axis, and a prediction error is shown on the vertical axis. The left side of FIG. 11 shows _{the case where X 1} _{is changed from 1.0 to 1.5 while X 2} is fixed to 1.0, and the right side of FIG. 11 shows the case where X ₁ is fixed to 1.0. This _{is the case where X 2} is changed from 1.0 to 1.5.

First, looking at the graph on the left side of FIG. 11, here, the absolute value of the prediction error under the _{condition of the left end (X 1} = 1.0, X _{2 = 1.0), which is the insertion condition, is about 0.12.} However, when X ₁ becomes 1.1 to 1.5, the absolute value of the prediction error increases about four times to about 0.4. On the other hand, looking at the graph on the right side of FIG. 11, _{it can be seen that even if X 2} has a maximum of 1.5, the prediction error hardly increases from about 0.12 at the left end.

If the prediction model is a very simple multiple regression equation as shown in equation (5), and the operator knows the value of the identified coefficient, _{then X 2} is the _{extrapolation condition when X 1} is the extrapolation condition. It can be estimated in advance that the prediction error will be larger than when the extrapolation condition is used.

However, it may not be easy to estimate to that extent. Of course, if the prediction model is a black box model such as a neural network or deep learning and cannot be easily grasped by a simple mathematical formula, it is almost impossible to estimate not only the degree of prediction error but also the magnitude. Is.

FIG. 12 shows an example of the result of calculating the out-of-learning range index D3. FIG. 12 is a bar graph of out-of-learning index for the values of _{inputs X 1} and X _2. In this figure, _{a combination of two} _{inputs X 1} and X 2 is adopted on the horizontal axis, and the out-of-learning range index D3 is shown on the vertical axis. The left side of FIG. 12 shows _{the case where X 1} _{is changed from 1.0 to 1.5 with X 2} fixed to 1.0, and the right side of FIG. 11 shows the case where X ₁ is fixed to 1.0. This _{is the case where X 2} is changed from 1.0 to 1.5.
Looking at the graph on the left side of FIG. 12, since the condition at the left end (X ₁ = 1.0, X ₂ = 1.0) is an interpolation condition, the value of the out-of-learning index D3 is 0. There is. It can be seen that the value of the out-of-learning range index D3 increases as the input X _{1 deviates from 1.0.} On the other hand, looking at the graph on the right side of FIG. 12, even if the input X ₂ deviates from 1.0 and becomes 1.5, the index outside the learning range remains almost 1.0.

Not only the degree of extrapolation, the sensitivity coefficient K is also considered, such difference occurs between X ₁ and X _2. It can be seen that the value of the out-of-learning range index D3 has a shape similar to the value of the prediction error shown in FIG. In this way, if the value of the index D3 outside the learning range is known, it can be said that the magnitude of the predicted value DS2 may be different from the measured value D2U in advance.

FIG. 13 shows the prediction error shown in FIG. 11 and the value of the out-of-learning range index D3 shown in FIG. 12 as a correlation diagram. FIG. 13 is a diagram showing an example of correlation between an index outside the learning range and a prediction error. In this figure, the out-of-learning index D3 is adopted on the horizontal axis, and the prediction error is shown on the vertical axis. If such a correlation diagram can also be displayed on the display unit 30, it is possible to grasp in advance how much the measured value D2U may deviate from the value of the index D3 outside the learning range of the predicted value in the future, which is more useful. be.

If the value of the out-of-learning index D3 is greater than or equal to a certain value or less than or equal to a certain value, the reliability of the predicted value DS2 is considered to be low. You may add a function that does not display the value DS2 itself on the screen. By adding such a function, it becomes possible to drive on the safe side with further reduced risk.

The configuration of the prediction system has been explained with reference to the configuration of FIG. 1, but this idea can be realized as it is as a prediction method using a prediction model. The invention of the prediction method is "a prediction method for making a prediction by a prediction model learned by using training data in a learning stage, and in the prediction stage, an input value of each input item given to the prediction model is obtained and predicted. Calculate the sensitivity coefficient that shows the effect of each input item of the model on the predicted value obtained by the prediction model, calculate the distance coefficient that shows the degree of deviation from the statistical value of the training data regarding the input value of each input item, and calculate the sensitivity. It can be realized by calculating the out-of-learning index from the values of the coefficient and the distance coefficient, and displaying either or both of the sensitivity coefficient and the distance coefficient, or the value of the out-of-learning index as a numerical value or a figure.

Further, the present invention has a feature as a display content displayed on the display device. This is, for example, "a display device used in a prediction system including a prediction model for learning using learning stage training data, and the display device has an input value of each input item given to the prediction model in the prediction model. Sensitivity coefficient indicating the effect on the required predicted value, distance coefficient indicating the degree of deviation from the statistical value of the training data regarding the input value of each input item, and the out-of-learning index calculated from the values of the sensitivity coefficient and the distance coefficient. A display device characterized by displaying the value of either or both of the sensitivity coefficient and the distance coefficient, or an index outside the learning range as a numerical value or a figure.

10: Input unit, 12: Input value normalization calculation unit, 15: Statistical value normalization calculation unit, 18: Distance coefficient calculation unit, 22: Sensitivity coefficient calculation unit, 26: Out-of-learning index calculation unit, 30: Display unit , D1: Statistical value of training data of each input item, D1U: Statistical value of training data after normalization, D2: Each input value of input item, D2U: Input value after normalization, D2U1: Perturbation Input value, D3: Out-of-learning index, DB: Statistical value storage of training data, DS1: Predicted value when perturbation is given, DS2: Predicted value, K: Sensitivity coefficient, L: Distance coefficient, M1: Sensitivity Prediction model for analysis, M2: Prediction model for calculation of predicted value

Claims

It is a prediction system that includes a prediction model that learns using learning data at the learning stage.
An input unit that gives an input value of each input item given to the prediction model at the prediction stage, and a sensitivity coefficient calculation unit that calculates a sensitivity coefficient that indicates the influence of each input item of the prediction model on the prediction value obtained by the prediction model. And the distance coefficient calculation unit that calculates the distance coefficient indicating the degree of deviation from the statistical value of the learning data regarding the input value of each input item, and the learning that calculates the out-of-learning index from the sensitivity coefficient and the value of the distance coefficient. A prediction system including a out-of-range index calculation unit and a display unit that displays the value of the out-of-range index, either or both of the sensitivity coefficient and the distance coefficient, or the value of the out-of-range index as a numerical value or a figure.
The prediction system according to claim 1.
It is equipped with an input value normalization calculation unit that normalizes the input value of each input item, and a statistical value normalization calculation unit that normalizes the statistical value of the training data for each input item.
It is characterized in that at least either the sensitivity coefficient calculation unit or the distance coefficient calculation unit gives the input value after normalization and the statistical value of the training data after normalization in order to calculate the sensitivity coefficient or the distance coefficient. Prediction system to do.
The prediction system according to claim 1 or 2.
The out-of-learning index calculation unit is a prediction system characterized in that the product of the sensitivity coefficient and the distance coefficient of each input item is added for all input items to calculate the out-of-learning index.
The prediction system according to claim 1 or 2.
The out-of-learning index calculation unit is a prediction system characterized in that the out-of-learning index is calculated by adding the absolute value of the product of the sensitivity coefficient and the distance coefficient of each input item for all input items.
The prediction system according to any one of claims 1 to 4.
Two or more large and small statistical values are used as the statistical values given to the distance coefficient calculation unit, and when the value of the input item is larger than the maximum statistical value, the larger the value, the larger the distance coefficient value, and the minimum. If the value of the input item is smaller than the statistic value, the value of the distance coefficient becomes smaller as it is smaller, and if the value of the input item is between the minimum statistic value and the maximum statistic value, the input item It is characterized by giving the value of the distance coefficient when the value of is equal to the minimum statistical value and the value of the distance coefficient when the value of the input item is equal to the maximum statistical value. Prediction system.
The prediction system according to claim 5.
The maximum value of the statistical values given to the distance coefficient calculation unit, the maximum value of the training data, the maximum value of the training data after normalization, the average value of the training data plus a constant multiple of the standard deviation, Using either the normalized average value plus a constant multiple of the standard deviation, the minimum statistical value given to the distance coefficient calculation unit is the minimum value of the training data and the normalized value. It is characterized by using either the minimum value of the training data, the value obtained by subtracting the constant multiple of the standard deviation from the average value of the training data, or the value obtained by subtracting the constant multiple of the standard deviation from the average value of the normalized training data. Prediction system.
The prediction system according to any one of claims 1 to 4.
One value is used as the statistical value given to the distance coefficient calculation unit, and when the value of the input item is larger than the statistical value, the value of the distance coefficient becomes larger as the distance increases, and the value is input compared to the statistical value. A prediction system characterized in that when the value of an item is small, the value of the distance coefficient becomes smaller as the value of the item becomes smaller.
The prediction system according to claim 7.
A prediction system characterized in that any one of a mean value, a mode value, and a median value is used as a statistical value given to the distance coefficient calculation unit.
The prediction system according to any one of claims 1 to 8.
The display unit is a prediction system characterized in that the value of the index outside the learning range calculated in the past is also displayed as a numerical value or a figure.
The prediction system according to any one of claims 1 to 9.
The display unit is a prediction system characterized in that one axis is time and the other axis is the value of an index outside the learning range, and the magnitude is visually recognizable with respect to the past value.
The prediction system according to any one of claims 1 to 10.
A prediction system characterized in that a graph in which one axis is time and the other axis is past measured values and predicted values can also be displayed on the display unit.
The prediction system according to any one of claims 1 to 10.
The prediction system is characterized in that the display unit can also display a graph in which one axis is time and the other axis is a prediction error which is a difference between a measured value and a predicted value.
The prediction system according to any one of claims 1 to 10.
The display unit is characterized in that one axis can display a prediction error of a predicted value with respect to a past measured value, and the other axis can display a correlation diagram which is an index outside the learning range.
It is a prediction method for making predictions by a prediction model that learns using learning data at the learning stage.
In the prediction stage, the input value of each input item given to the prediction model is obtained, the sensitivity coefficient indicating the influence of each input item of the prediction model on the prediction value obtained by the prediction model is calculated, and the input of each input item is calculated. A distance coefficient indicating the degree of deviation from the statistical value of the training data regarding the value is calculated, an index outside the learning range is calculated from the sensitivity coefficient and the value of the distance coefficient, and either or both of the sensitivity coefficient and the distance coefficient are calculated. , Or a prediction method comprising a table displaying the value of the index outside the learning range as a numerical value or a figure.
A display device used in a prediction system that includes a prediction model that learns using training data in the learning stage.
The display device has a sensitivity coefficient indicating the influence of the input value of each input item given to the prediction model on the prediction value obtained by the prediction model, and a deviation from the statistical value of the training data regarding the input value of each input item. With respect to the distance coefficient indicating the degree and the out-of-learning index calculated from the sensitivity coefficient and the value of the distance coefficient, either or both of the sensitivity coefficient and the distance coefficient, or the value of the out-of-learning index is numerically or A display device characterized by displaying as a figure.