KR102366787B1 - Real-time sliding window based anomaly detection system for multivariate data generated by manufacturing equipment - Google Patents

Abstract

The present invention time-series the discrete data of multiple sensors collected in real-time in the facility, then extracts the multivariate analysis result in the form of an image using the sliding window technique, and returns the result to anomaly scores in real time through the image processing deep learning technique. It is about a system that outputs score).

Description

Real-time sliding window based anomaly detection system for multivariate data generated by manufacturing equipment}

The present invention time-series the discrete data of multiple sensors collected in real-time in the facility, then extracts the multivariate analysis result in the form of an image using the sliding window technique, and returns the result to anomaly scores in real time through the image processing deep learning technique. It is about a system that outputs score).

In general, research and technology development for early detection of equipment anomalies using sensor devices is being actively conducted, and conventional machines such as artificial neural networks, decision trees, and clustering, using the collected data Methods using learning models and algorithms are also being used.

And, machine learning generally refines the collected data, classifies it into training data and test data, learns/evaluates the training model, and uses the learned model after training It is a technology that accurately processes new incoming data to analyze and predict information necessary for a new environment.

Meanwhile, in recent years, as deep learning technology that imitates the human brain has developed significantly, machine learning based on deep learning technology is actively applied in various fields such as image recognition/processing, automatic voice recognition, video behavior recognition, and natural language processing. is a trend that

And, deep learning (or deep learning) is making successful applications in many areas recently.

New types of services combined with various domain knowledge are being derived in almost all fields from computer vision, natural language processing, medicine, robotics, and even art fields, and in many areas, automation based on the model created through deep learning is is being done

In addition, the core content of the data is identified through the combination of the nonlinear transformation method, and a regression model with a very complex structure can be implemented.

In particular, as a data learning method of machine learning, it is classified into supervised, unsupervised, and reinforcement learning according to the presence or absence of judgment criteria. Research on unsupervised learning has a lot of potential.

In other words, supervised learning trains a computer in a state where a label for data is given an explicit correct answer, and unsupervised learning is data without a label in which a target value is not given for an input value. because it is much larger.

In the manufacturing process, there are cases where there are defective standards for products, equipment, and accessories, but there are many cases where they do not.

And it is not easy to set new standards that do not exist.

Therefore, if unsupervised learning is used, it is expected that anomalies can be detected more efficiently through wasted data.

In relation to this technology, conventional patent No. 2018330 discloses a plant pipe abnormality detection device and method using multiple meta-learning, and the configuration is as shown in FIG. The feature extractor 151 collects data during a long-term trend period (Tl) consisting of several packet periods from a plurality of periodically measured time series data of M sensors, and extracts a plurality of N long-term trend image features for each sensor. extract

In addition, the configuration and operation principle of the remaining long-term trend multi-feature synthesizer 152 and meta-learning machine 153 and long-term trend optimizer 160 also include the short-term trend meta-learner 110 and short-term trend optimizer 120 configuration and It is the same as the operating principle, and in order to improve performance, the long-term trend meta-learning machine 150 can extend meta-learning by setting a plurality of long-term trend sections according to user definition.

In addition, the multi-trend meta-evaluator 170 is the short-term trend meta learner 110, the mid-term trend meta learner 130, and the long-term trend meta learner 150 for each trend. The results learned based on the combination are input, and then, the multi-trend meta-evaluator 170 aggregates the input results to finally evaluate the abnormality of the plant piping.

However, with the apparatus and method for detecting anomaly in the plant pipeline, images and volume features reflecting multiple trends are extracted from various sensor data installed on the plant pipeline, and image and volume features reflecting multiple trends are extracted and optimized fusion between these features and sensors. Since it is not possible to accurately detect abnormalities in the piping, such a technology is required.

It is an object of the present invention to improve the conventional problems as described above, to accurately detect anomaly of multivariate time series data in a manufacturing facility, to predict anomaly as well as to suggest the cause of the anomaly, , to provide a real-time multivariate anomaly detection system for manufacturing facilities using a sliding window technique that enables efficient manufacturing facility management and prediction of product defects.

The present invention, in order to achieve the above object, the step of receiving a certain amount of real-time discrete (discrete) data;

After converting the discrete data into time series data, multivariate analysis and matrix generation for each section for each window using a sliding window technique;

extracting feature values by inputting each matrix into the first neural networks;

reconstructing the feature value through a second neural network;

And it provides a real-time multivariate anomaly detection system of a manufacturing facility using a sliding window technique including the step of inferring new data based on the feature value and outputting an anomaly score.

In addition, in the multivariate analysis and matrix generation step of the present invention, any one method selected from correlation matrix, covariance matrix, and canonical correlation analysis is utilized,

As for dimensionality reduction, principal component analysis (PCA), t-SNE, and UMAP (reduction to two dimensions) can be selected.

The input value provides a real-time multivariate anomaly detection system of a manufacturing facility using a sliding window technique composed of image data corresponding to each window composed of matrix values or two-dimensional data.

And, in the combination of the first neural networks of the present invention, based on a 2D convolutional neural network (2D-CNN) to process image data, Convolutional Long Short-Term (ConvLSTM) that enables image input by performing recurrent neural network polymerization and convolution processing memory models),

Provided is a real-time multivariate anomaly detection system for manufacturing facilities using a sliding window technique that includes at least one of the attention mechanisms to which the weight information of the window to be more focused on in the correlation matrix according to the passage of time is applied.

In addition, in the second neural network of the present invention, a (2D-Deconvolutional network) image is restored through a 2D deconvolutional neural network, and a manufacturing facility using a sliding window technique to derive the final result by combining with the result of the attention mechanism for each layer of real-time multivariate anomaly detection system.

In addition, the first and second neural networks of the present invention extract a feature value by reducing the dimension through the entire process, and then restore it, and learn in a direction in which the difference between the input value and the restored value decreases. This applies,

The first and second neural networks provide a real-time multivariate anomaly detection system for manufacturing facilities using a sliding window technique that can transform an image processing process into 3D convolution instead of 2D convolution.

As described above, according to the present invention, in many manufacturing industries except for the semiconductor market, data derived from equipment data do not have labels, and these data are often discarded without being properly managed and standardized, but in the present invention, these data It is to realize the effect of reducing production and operating costs, as well as the costs derived from this, through predictive quality/non-prediction of quality, including preventive maintenance, using

1 is a block diagram illustrating a conventional anomaly detection apparatus and method.
2 is a block diagram illustrating an anomaly detection system combining the multivariate analysis result and the autoencoder method in a predetermined window section using a sliding window according to the present invention.
3 is a block diagram illustrating an autoencoder anomaly detection system using a convolutional neural network, a convolutional LSTM, and an attention mechanism through imaging of a multivariate analysis result using a sliding window according to the present invention.

Hereinafter, an embodiment of the present invention will be described in detail based on the accompanying drawings.

The drawings of the present invention and the content of the detailed description to explain them correspond to one example, and in the large picture of the combination of multivariate analysis including relational description and artificial neural network in time series data, detailed items or structures are variously changed talk about possible

That is, those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention.

First, there are four main types of machine learning systems.

Supervised learning, unsupervised learning, semi-supervised learning, and finally reinforcement learning.

Supervised learning corresponds to a case in which a variable to be predicted with respect to data has a label, whereas unsupervised learning is the opposite case, in which there is no label.

In addition, semi-supervised learning refers to a case in which only some of the data are labeled, and reinforcement learning allows the user to present an appropriate criterion to the machine to give a good score to learn more about it, and to learn more about it when it is not good. It is a method to induce less learning by subtracting the score.

In spite of collecting a large amount of data, it is not easy to establish strict standards for right and wrong for the physical phenomenon of equipment operation.

In addition, it is not easy to obtain an operational insight from the perspective of the entire facility because it is not possible to find an appropriate link between these data, so it is not possible to perform a meaningful analysis.

In this situation, it was derived that unsupervised learning using an appropriate combination of deep learning, which is a nonlinear machine learning method, and a form of multivariate analysis that can express correlation between facilities, causality, or the structure of multidimensional data, is appropriate.

When dealing with multivariate time series data of facility data, it is possible to detect anomalies according to the structural distribution of multivariate data. It can help the operator to respond in advance to a specific physical quantity related to the sensor.

The present invention combines data to transform it into multivariate time-series data, and converts non-stationary time-series data into stationary time-series data.

Thus, the mean, variance, and autocorrelation of the data are made constant regardless of the passage of time.

Second, by using a sliding-window technique, windows of different sizes are moved at regular intervals to form an index on the structural relationship or correlation of multivariate data captured within the window, and then analyze it to create a matrix do.

At this time, in the multivariate analysis and matrix generation step, any one method selected from correlation matrix, covariance matrix, and canonical correlation analysis is used, and with respect to dimension reduction, principal component analysis (PCA), t-SNE, and UMAP (reduced to 2D) may be selected, and the input value is composed of image data corresponding to each window composed of a matrix value or 2D data.

Then, the data is image-processed through a neural network.

<Multivariate analysis and dimensionality reduction>

There are several analysis methods in multivariate analysis, but let's look at an example that can be used as a representative.

First, the most basic method is to use a scatter plot.

That is, it is a method of representing the relationship between two variables using a Cartesian coordinate system.

Since this shows the amount of change of two variables as it is, not a numerical value indicating any linear relationship, it is also expressed as it is for the relationship between variables in a non-linear relationship.

The second method is to use a correlation diagram.

This is a method of summarizing statistics for each variable by utilizing correlation and covariance between variables.

However, since it is expressed numerically through the linear relationship, the non-linear relationship may not be clearly expressed.

And, if it is desired to determine the linear correlation between clusters of variables, canonical correlation analysis can be used.

For example, assuming that two types of sensors are installed in one facility, the variables of A and B sensors are measured, and the correlation between them is observed, but the correlation between linear combinations is maximal. It is a method in which the weights are adjusted and determined so that .

Next, we would like to suggest ways to reduce the dimensionality of high-dimensional data.

First of all, there is a principal component analysis method (Principle Component Analysis) that finds principal components expressed as linear combinations using variance-covariance relationships among other quantitative variables.

This is a method of projecting existing data to the new space by forming a new independent axis in the order of the largest data variance.

Because it is a linear combination, in the case of data with a non-linear relationship, there are cases where the original data cannot be explained well in a small dimension. (Interpretable)

In addition, it can be reduced using t-distributed stochastic neighbor embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP).

Both methods use the similarity index through distance to express the degree of data density in the high dimension in the low dimension.

However, t-SNE is mainly used for visualization and consumes a lot of memory during operation, while UMAP method is known to enable fast operation and express the global distribution of data in high and low dimensions relatively well. more preferred

Using this dimensionality reduction method, data that is difficult to visualize with higher dimensions or higher can be reduced and imaged in a way that expresses the relationship between the original data well.

By using these images as input values of the artificial neural network, they can be used in more various ways for detecting abnormalities.

The neural network is composed of a combination of first neural networks and a combination of second neural networks.

In this case, in the combination of the first neural networks, Convolutional Long Short-Term Memory models (ConvLSTM) that enable image input by performing recurrent neural network polymerization and convolution processing based on a 2D convolutional neural network (2D-CNN) to process image data ), and at least one of the Attention Mechanism to which the weight information of the window to be more focused on in the correlation matrix over time is applied.

And, in the second neural networks, an image is restored through a 2D deconvolutional neural network (2D-Deconvolutional network), and the final result is derived by combining the result of the attention mechanism for each layer.

In addition, the first and second neural networks extract the feature values by reducing the dimension through the entire process, then restore them, and the autoencoder method is applied to learn in a direction in which the difference between the input value and the restored value decreases. and, the first and second neural networks use those capable of transforming the image processing process into 3D convolution instead of 2D convolution.

<Auto Encoder>

Autoencoder is used as one of the representative deep learning methods of anomaly detection unsupervised learning.

This method is a machine learning model that extracts feature values that are well representative of the input values.

In this model, the input value and the output value are learned to be the same, and it can also be viewed as a self-replicating model. During the training process, a high-dimensional input value is reduced to a low-dimensional data, and the reduced information is restored to the original dimension. go through a series of processes.

And, this reduced information can be used to infer new data.

However, as shown in FIGS. 2 and 3, the autoencoder itself cannot include information on the flow of incoming data or an understanding of a later event based on an event that has occurred in the past, so one of the recurrent neural network models is By introducing an LSTM model that can remember information with a long time gap, a model that can recognize a long flow of data is constructed.

In addition, by using the attention mechanism, the entire batch data is checked at every time step, and more weight is given to the input value at the point in time to focus more, so that the loss of information about the flow can be further reduced. .

If the input value is an image (scatterplot, correlation matrix, and two-dimensionally reduced data image), more detailed anomaly detection is possible using a convolutional neural network.

Through the convolutional neural network layer, the necessary feature map is extracted through the convolution of the image and the filter (weight parameter) and then transferred to the next layer.

Through this, a simple autoencoder, a convolutional autoencoder (CNN autoencoder), and a simple LSTM can be used as CNN-LSTM.

An example picture for this can be found in FIG. 3 .

Claims (5)

Receiving a certain amount of real-time discrete data;
after converting the discrete data into time series data, multivariate analysis and matrix generation for each section for each window using a sliding window technique;
inputting each matrix into the first neural networks to extract feature values of image data generated by a dimensionality reduction method;
reconstructing the feature value through a second neural network;
and outputting an anomaly score after inferring new data based on the feature value,
The combination of the first neural networks includes Convolutional Long Short-Term Memory models (ConvLSTM) that enable image input by performing recurrent neural network polymerization and convolution processing based on a 2D convolutional neural network (2D-CNN) to process image data; It includes at least one of the Attention Mechanism to which the weight information of the window to be more focused on in the correlation matrix according to the passage of time is applied,
In the second neural network, a (2D-Deconvolutional network) image is restored through a 2D deconvolutional neural network, and real-time multivariate anomalies of manufacturing facilities using a sliding window technique to derive the final result by combining the results of the attention mechanism for each layer detection system. According to claim 1, wherein in the multivariate analysis and matrix generation step, any one method selected from a correlation matrix, a covariance matrix, and a canonical correlation analysis is utilized,
Regarding dimensionality reduction, principal component analysis (PCA), t-SNE, and UMAP (reduction to two dimensions) can be selected,
The input value is a real-time multivariate anomaly detection system of a manufacturing facility using a sliding window technique, characterized in that it consists of image data corresponding to each window composed of a matrix value or two-dimensional data. delete delete The autoencoder according to claim 1, wherein the first and second neural networks extract a feature value by reducing a dimension through the entire process and then restore it, and learn in a direction in which the difference between the input value and the restored value decreases. ) method is applied,
The first and second neural networks, a real-time multivariate anomaly detection system for manufacturing facilities using a sliding window technique, characterized in that it is possible to transform the image processing process into 3D convolution instead of 2D convolution.

Images