KR20210003004A - Method and apparatus for abnormality diagnose and prediction based on ensemble model - Google Patents

Abstract

The present invention relates to a technique for processing an ensemble by optimizing primary output results in various models in applying a diagnosis and prediction technique for abnormal situations occurring in facilities and devices. A sequence of methods of the present invention configures and processes a learning network for constructing an optimal ensemble model according to the estimation of each probability distribution by using a response variable having an adaptive threshold value and a specific probability distribution outputted from the various models as an input of the ensemble model. Through this, in diagnosing an abnormal situation related to arbitrary facilities or devices, the abnormality diagnosis technique based on the optimal ensemble model can be implemented.

Description

TECHNICAL FIELD [Method and apparatus for abnormality diagnose and prediction based on ensemble model}

The present invention relates to a technique for processing an ensemble by optimizing primary output results from various models in applying a diagnosis and prediction technique for abnormal situations occurring in facilities and devices. More specifically, the present invention relates to the configuration of a learning network for constructing an optimal ensemble model according to estimation of each probability distribution by using an adaptive threshold output from the various models and a response variable having a specific probability distribution as inputs of an ensemble model. will be.

The conventional ensemble-based classification prediction technique uses a bagging technique such as random forest to classify or predict values from multiple decision trees to derive results by means of averaging, multiplication, or majority voting, or several weak learners. Active attempts such as the Boosting technique, a technique that connects with dogs and trains with weights on training samples that were not suitable in the previous weak model, finally creates a strong learner have been actively made. Recently, not only such a bagging or boosting technique, but also an ensemble classification technique based on deep learning has been widely used. The ensemble classification method based on deep learning can see the effect of ensemble processing similar to the existing method by applying the Dropout method on the layer within the model.

The bagging technique divides training datasets into several and trains each dataset independently. Therefore, since the weights in each bagging model are ensembled and averaged, each bagging model can be viewed as an independent model, and through such a bagging model, the effect of reducing the variance of the response variable can be seen.

The boosting method divides the training data set into multiple pieces in the same way as the Bagging method, but for data with large errors when training in the cascading previous classifier, it is appropriate to learn by adjusting the extraction probability in the next classifier to be higher. As a way to achieve this, a weight is also given according to the classifier performance. Through this boosting model, the effect of reducing the bias of the response variable can be achieved.

There is also a case in which an ensemble technique that combines a deep learning-based model that is popular in recent years, such as bagging or boosting techniques, which are techniques used in the conventional computer vision field, is used. The ensemble technique is a method of constructing bagging, boosting, and deep learning models, respectively, and harmonizing and processing the results of the response variables. In this case, the ensemble method uses the average of the output variables, the product, and the majority vote.

However, the existing ensemble method does not properly reflect the skewness of the probability distribution of the model response variable used in the first step by using only the response variable value as the input variable of the second-stage ensemble model without considering the threshold in the first step output. There was a problem that couldn't. Depending on the results of the first-stage model, the probability distribution of the response variable can have various forms such as uniform distribution, exponential distribution, beta distribution, Weibull distribution, and normal distribution, but the existing methods properly reflect these distribution characteristics. There are limitations that cannot be done.

In the present invention, in implementing an ensemble model based on various types of learning models, by applying an optimal probability distribution model to adopt the response variable of each individual model, the adaptive threshold of each individual model is newly reflected in the ensemble model learning. To learn. In other words, in the ensemble model, we implement a model that can learn by including adaptive thresholds, rather than learning only by the response variables of the previous individual models.

Through this, in diagnosing an abnormal situation related to an arbitrary facility or device, it is possible to implement an abnormality diagnosis technique based on an optimal ensemble model.

The present invention is one of the methods of implementing an ensemble model, and when constructing a hierarchical learning model, the threshold and probability distribution of the response variables of each individual model are generated through one-step output, and the threshold and probability distribution of the generated response variables A method of constructing a two-stage learning model based on optimized learning based on estimation of the probability distribution of a first-stage response variable is provided by using as an input variable of a two-stage learning model. In this case, the optimization learning for the first-stage response variable is constructed in conjunction with the weight product according to the importance of each model in the first-stage in the second-stage learning model.

In order to implement this, the present invention is a method for diagnosing and predicting anomalies based on an ensemble model, in which training data is input to perform optimization learning in an individual model, and an adaptive threshold and a response variable value (Probability) are output. An ensemble model-based abnormality diagnosis and prediction method is provided, including: performing optimization learning of the ensemble model based on the threshold values and response variable values of each model, and diagnosing and predicting the abnormality based on the ensemble model.

In addition, the present invention provides a means for performing optimization learning in individual models by inputting training data, a means for outputting an adaptive threshold and a response variable value, and an ensemble based on the threshold and response variable values of each model. An ensemble model-based anomaly diagnosis and prediction apparatus including a means for performing optimization learning of a model and a means for diagnosing and predicting an anomaly based on an ensemble model is provided.

The configuration and operation of the present invention introduced above will become more apparent through specific embodiments described with reference to the drawings.

Unlike the conventional invention, the present invention uses the threshold value and probability distribution of the response variable of each individual model in the output of the previous stage as input variables of the next stage learning model when constructing an ensemble-based hierarchical learning model. It is possible to optimize learning of the ensemble model based on the estimation of the probability distribution of the response variable of.

If the present invention is used in the field of diagnosing abnormal signals for the occurrence of partial discharge in a power line, after determining whether an abnormal signal has occurred for the power line through several learners, the probability distribution for each output variable and threshold bundle of the preceding primary learners An optimal partial discharge abnormal signal diagnosis is possible through an estimation-based secondary ensemble learner.

1 is a conceptual diagram of an ensemble model-based abnormality diagnosis and prediction processing method, FIG. 2 is a flow chart of an ensemble model-based abnormality diagnosis and prediction method, FIG. 3 is a block diagram of an ensemble model processing unit, FIG. 4 is a flow chart of an ensemble model processing unit, and FIG. 5 is several learning models. An example of a probability model of a response variable estimated from, Figure 6 is an example of a threshold result estimated from several learning models

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms different from each other, only this embodiment makes the disclosure of the present invention complete, and those skilled in the art to which the present invention pertains. It is provided to fully inform the scope of the invention to the person, and the invention is defined by the description of the claims.

On the other hand, terms used in the present specification are for explaining examples and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, "comprises" or "comprising" refers to the presence of one or more other components, steps, actions and/or elements other than the recited elements, steps, actions and/or elements, or Does not exclude addition.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to elements of each drawing, even though they are indicated on different drawings, the same elements are assigned the same reference numerals as much as possible, and in describing the present invention, a detailed description of related known configurations or functions If the gist of the present invention may be obscure, detailed description thereof will be omitted.

1 is a conceptual diagram illustrating an ensemble model-based abnormality diagnosis and prediction method. First, the entire training dataset 10 or a dataset processed with random samples is input to each individual model processing unit 20A, 20B, ??, and 20N. Individual model processing units (20A, 20B, ??, 20N) perform weight training using the input training data set 10, and then complete the prediction model to provide a response variable (or output variable) (probability) and model-based A threshold for determining whether there is an abnormality in the data of is outputted.

At this time, the probability distribution model of the response variable can be distributed in various forms, for example, uniform distribution (Fig. 5a), exponential distribution, beta distribution (Fig. 5b, c), wavel distribution, and normal distribution. It appears as a continuous probability distribution such as (d in Fig. 5).

In addition, as shown in Fig. 6, the threshold value for determining the presence or absence of an abnormality from the prediction results estimated in each learning model is not indicated by the median or average value of the output probability distribution of the response variable, but a specific value regardless of each learning model. Have.

Due to this result, when the response variable result value in the current step is directly input and processed by the ensemble model processing unit 30, which is the next step, the probability that the learning result does not show optimal performance increases. That is, as shown in FIG. 5, the response variable result value in the first-stage model generally exhibits a skewness phenomenon.

Accordingly, in the present invention, the process of correcting the result value by learning the response variable result value together with the threshold value is performed in the ensemble model processing unit 30, which is a next step. The ensemble model processing unit 30 performs optimization learning based on response variables and thresholds of each individual model, and performs a final abnormality diagnosis and visualization process 50 through a test dataset 40 to obtain a diagnosis result. (Diagnostic result) is output.

The above process will be described for each step with reference to the flow chart of a method for diagnosing and predicting an abnormality based on an ensemble model of FIG. 2.

S10: input training data

The training data is pre-processed to become sequential structured data according to a time sequence. That is, the training data appears in the form of images, images, voices, texts, numbers, etc. according to the characteristics of the data. Data such as video, image, and audio are subjected to preprocessing transformation such as FFT (Fast Fourier Transform), DCT (Discrete Cosine Transform), MFCC (Mel Frequency Cepstral Coefficient), etc. It is saved as an external file or directly input to the model for processing.

S20: Optimization training on each individual model

Since the form of the input data in step S10 has various structures according to the data characteristics, in step S20, each individual model processing unit 20A, 20B, ..., 20N processes input data according to the data characteristics, respectively. For example, the input data is based on the performance analysis of individual training models such as XGBoost, LightGBM, CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), etc., depending on the characteristics of video, image, voice, text, and numbers. Configurable. Alternatively, it is possible to divide input data having similar data characteristics into several clusters and perform optimization learning by applying different models for each cluster.

S30: Output of adaptive threshold and response variable value (Probability)

The model output in step S30 through optimizing learning of each input data in S20 is an adaptive threshold that distinguishes between normal and abnormal data and the probability of a response variable including the posterior distribution of the input data through the model. appear. Here, the threshold value can be adaptively selected by analyzing the prediction performance at the output through automatic input of several threshold values for the same sample.

S40: Ensemble model optimization training based on threshold and response variable values of each model

In step S40, the ensemble model is implemented based on the response variable value and the threshold value of each model. The expected value for the prediction performance of each model is proportional to the product of the posterior probability distribution for the model output of the input data, the prediction performance of the model, and the model weight. Ensemble model optimization learning is a learning process to find optimal weight parameters for ensemble the results of each of these models. For example, when the posterior probability distribution is in the form of a camel's bimodal, the weight of the critical value with the central value of the bimodal shape and the weight of the critical value with the peak normal value when the posterior probability distribution is in the form of a single peak have completely different weight reliability. . That is, in ensemble model training, optimization learning is performed by inputting the posterior probability distribution model, prediction performance, and threshold values of each model.

S50: Ensemble model-based abnormality diagnosis prediction

The ensemble model after optimization learning in step S40 outputs an adaptive threshold and prediction performance. In S50, the accuracy of prediction of abnormality diagnosis of the ensemble model can be checked through the test data set.

S60: Visualization of abnormality diagnosis results based on test data

In order to check the accuracy of the abnormality diagnosis result in step S50, in step S60, it is visualized based on the weight parameter of the model. For example, by applying a Grad-CAM technique or the like to a model performed to check the validity of an abnormality diagnosis model in an image or image, the abnormal data area can be visually checked.

3 shows a block diagram of the ensemble model processing unit 30 shown in FIG. 1. The ensemble model processing unit 30 is composed of an input processing unit 310, a data analysis unit 320, a feature processing unit 330, a model learning unit 340, an abnormality diagnosis unit 350, and a diagnosis output unit 360. have.

The input processing unit 310 is a module that stores a threshold value and a response variable value transmitted from the previous individual model processing unit (20 in FIG. 1). The data analysis unit 320 estimates a probability distribution of values of a response variable. At this time, the probability distribution of the received response variable values can be various continuous probability distributions such as uniform distribution, exponential distribution, beta distribution, wavel distribution, and normal distribution. Estimate for each individual model response variable value.

The feature processing unit 330 processes a variable value to be input to the model learning unit 340 by multiplying the modified threshold value through the processing of the threshold value in the probability distribution for each individual model, the weight for each model, and the response variable for each model. In this case, the value of the input variable of the model learning unit 340 is Equation 1 below.

라 할 때, 1차 개별모델처리부(20) 이후의 사후 확률 밀도함수는

가 되고(여기서,

: 개별모델처리부에서의 가중치 파라미터, x: 임계치 확률 변수), 특정 임계치 j상의 i번째 개별모델처리부(20)에서의 사후 확률밀도 값은

이 된다.In Equation 1, the probability density function of the data

When d, the posterior probability density function after the first individual model processing unit 20 is

Becomes (where,

: The weight parameter in the individual model processing unit, x: the threshold random variable), the posterior probability density value in the i-th individual model processing unit 20 above the specific threshold value j

Becomes.

는 i번째 개별모델처리부의 성능에 따른 가중치 파라미터이고,

는 i번째 개별모델처리부의 입력변수에 대한 반응변수의 확률 값이 된다. 일례로,

가 정규분포를 따른다 가정하면, x의 특정 임계치 j가 확률밀도함수의 평균값일 때,

값이 최대값을 갖게 되고, x의 특정 임계치 j의 치우침 정도에 따라

값이 달라지게 된다. 이 때의 정규분포 함수는 도 5의 (D)를 참고한다.Also, in Equation 1,

Is a weighting parameter according to the performance of the i-th individual model processor,

Is the response variable for the input variable of the i-th individual model processing unit. It becomes a probability value. For example,

Assuming that is a normal distribution, when a specific threshold j of x is the average value of the probability density function,

The value will have a maximum value, and depending on the degree of bias of the specific threshold j of x

The value will be different. For the normal distribution function at this time, refer to FIG. 5D.

The model learning unit 340 receives the corrected threshold value for each individual model processed by the feature processing unit 330, a weight for each model, and a response variable for each model to train a learning model. The abnormality diagnosis unit 350 checks whether or not the test data is abnormally diagnosed from the adaptive threshold value and the random variable value of the trained learning model, and visualizes and outputs it through the diagnosis output unit 360.

4 shows the processing process of the ensemble model processing unit 30, and each step will be described.

S110: Input of adaptive threshold and probability for each model. Here, the input of the ensemble model is the output of several individual learning models for each input data. In this case, the output value is a value of a response variable including a threshold for classifying normal/abnormal data and a posterior distribution of input data through an individual learning model.

S120: Ensemble model training based on weights, adaptive thresholds, and response variable values for each model. The ensemble model is trained based on each individual learning model response variable value and threshold. The expected value for the prediction performance of each model is proportional to the product of the posterior probability distribution for the model output of the input data, the prediction performance of the model, and the model weight. Ensemble model optimization learning is a learning process to find the optimal weight parameters to ensemble the results of each of these models. That is, in ensemble model learning, optimization learning is performed by inputting the posterior probability distribution model, prediction performance, and threshold values of each model.

S130: Ensemble model-based adaptive threshold and predictive performance value (Accuracy) output. The ensemble model after optimization learning outputs the adaptive threshold and prediction performance. Through the training data set and the separate test data set, the accuracy of prediction of abnormality diagnosis of the ensemble model is checked.

The present invention described above can be utilized in diagnosing an abnormal signal for the occurrence of partial discharge in a power line. For example, after determining whether an abnormal signal has occurred on the power line through various learners such as Xgboost, LightGBM, Lstm-Attention, etc. as the first order, the second order based on the estimation of the probability distribution for each output variable and threshold bundle of the preceding first learners. An optimal partial discharge abnormal signal can be diagnosed through the ensemble learner.

The present invention described above can be implemented in terms of a device or a method. In particular, a function or process of each component of the present invention is a digital signal processor (DSP), a processor, a controller, and an application-specific application-specific (ASIC). IC), programmable logic devices (FPGA, etc.), and at least one of other electronic devices, and a combination of these can be implemented as hardware elements. It can also be implemented as software in combination with hardware elements or independently, and this software can be stored on a recording medium.

Although the configuration of the present invention has been described in detail through a preferred embodiment of the present invention, those of ordinary skill in the art to which the present invention pertains to the present invention are different from the contents disclosed in the present specification without changing the technical idea or essential features. It will be appreciated that it can be implemented in other specific forms. It should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of protection of the present invention is determined by the claims described later rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalent concepts should be interpreted as being included in the technical scope of the present invention. .

10: training dataset, 40: test dataset, 50: abnormality diagnosis and visualization process

Claims (1)

As an ensemble model-based abnormality diagnosis and prediction method,
Inputting training data to perform optimization learning on individual models,
Outputting an adaptive threshold and a probability of a response variable,
Performing optimization learning of the ensemble model based on the threshold value and response variable value of each model,
An ensemble model-based abnormality diagnosis and prediction method comprising the step of diagnosing and predicting an abnormality based on the ensemble model.

Images