KR20200052421A - Apparatus and Method for Anomaly Detection of SMD Assembly Device Operation based on Deeplearnig - Google Patents

Abstract

The present invention relates to an apparatus for detecting motion anomalies of deep learning based surface mounted device (SMD) assembly equipment which comprises: an input processing unit for extracting the first sound signal from the operating sound waveform of the SMD assembly equipment; a sound restoration model trained based on deep learning to restore the first sound signal extracted by the input processor and output a second sound signal; an error calculation unit for calculating an error between the first sound signal and the second sound signal; and an equipment abnormality determination unit that determines whether the SMD assembly equipment is abnormal according to whether the error calculated in the error calculation unit is greater than or equal to a predetermined threshold.

Description

Apparatus and Method for Anomaly Detection of SMD Assembly Device Operation based on Deeplearnig}

The present invention relates to an operation abnormality detection technology of a surface mounted component (SMD) assembly equipment, and more particularly, to an apparatus and method for detecting an operation abnormality of an SMD assembly equipment based on deep learning.

Surface Mounted Device (SMD) is a device that attaches microelectronic components on a Printed Circuit Board (PCB), and SMD Assembly Equipment is a device that performs the task of producing such SMD. to be.

Since SMD assembly equipment operates on a production line that is very fast in an automated production process, if a failure occurs in one device in the production line, operation stops, and the entire production line is affected as well. Therefore, the anomaly detection of the SMD assembly equipment must be quickly performed.

However, the SMD assembly equipment produces a small amount of SMDs of various types, and as the SMD to be produced is changed, the assembly process is frequently replaced. Therefore, not only the standard for determining the anomaly of the operation of the SMD assembly equipment is different for each assembly process to be replaced, but there are also various types of malfunctions occurring in one assembly process. In addition, data on malfunctions having a history of occurrence can be acquired, but data on malfunctions without a history of occurrence is difficult to acquire, so it is difficult to detect unknown malfunctions. Accordingly, there is a need for a SMD assembly equipment abnormality detection technology that can quickly cope with frequent assembly processes and various malfunctions and can also detect malfunctions without a history of occurrence.

The present invention provides an apparatus and method for detecting motion anomalies of a deep learning-based SMD assembly equipment capable of flexibly and adaptively detecting motion anomalies as the assembly process of the SMD assembly equipment is changed.

The present invention provides an apparatus and method for detecting anomalies in a deep learning-based SMD assembly equipment capable of detecting malfunctions without occurrence history in the SMD assembly equipment.

The present invention is an apparatus for detecting anomalies of a deep learning based SMD assembly equipment, an input processing unit extracting a first sound signal from an operating sound waveform of a surface mounted device (SMD) assembly equipment, and extracted by the input processing unit A sound restoration model trained based on deep learning to restore the first sound signal and output the second sound signal, an error calculation unit calculating an error between the first sound signal and the second sound signal, and an error calculation unit And an equipment abnormality determination unit that determines whether the SMD assembly equipment is abnormal according to whether the error is greater than or equal to a predetermined threshold.

The present invention is a method for detecting motion anomalies of a deep learning-based SMD assembly equipment, and inputs a first sound signal extracted from an operating sound waveform when a surface mounted component (SMD) assembly equipment normally assembles an assembly target. Learning a sound restoration model using the labeling signal, restoring the first sound signal extracted from the working sound waveform of the SMD assembly equipment through the learned sound restoration model, and generating a second sound signal; And calculating an error between the signal and the second sound signal, and determining whether the SMD assembly equipment is abnormal according to whether the calculated error is greater than or equal to a predetermined threshold.

The apparatus and method for detecting motion anomalies of the deep learning-based SMD assembly equipment according to the present invention can significantly reduce the number of neurons in an artificial neural network to be learned compared to other artificial neural networks. The learning time is also reduced, and even if the SMD assembly equipment changes the assembly target, it can quickly respond to the new environment. Therefore, as the assembly process of the SMD assembly equipment is changed, it is possible to flexibly and adaptively detect an operation abnormality.

In addition, according to the present invention, since the deep learning model restores the input signal, it is possible to detect a malfunction without a history in the SMD assembly equipment.

1 is a block diagram of an apparatus for detecting an operation abnormality of a deep learning-based SMD assembly equipment according to an embodiment of the present invention.
2 is a diagram for explaining the structure of a fast adaptive RNN encoder / decoder model (FARED) according to an embodiment of the present invention.
3A to 3C are diagrams for explaining the operation of the input processing unit according to an embodiment of the present invention.
4 is a flowchart illustrating a method of detecting an operation abnormality of a deep learning-based SMD assembly equipment according to an embodiment of the present invention.

Hereinafter, an apparatus and a method for detecting an operation abnormality of a deep learning-based SMD assembly equipment according to a preferred embodiment will be described in detail with reference to the accompanying drawings. Here, the same reference numerals are used for the same components, and repeated descriptions and detailed descriptions of well-known functions and components that may unnecessarily obscure the subject matter of the invention are omitted. Embodiments of the invention are provided to more fully describe the present invention to those skilled in the art. Therefore, the shape and size of elements in the drawings may be exaggerated for a more clear description.

Combinations of each block in the accompanying block diagrams and steps of the flow charts may be performed by computer program instructions (execution engines), these computer program instructions being incorporated into a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment. Since it can be mounted, its instructions, which are executed through a processor of a computer or other programmable data processing equipment, create a means to perform the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions can also be stored in computer readable or computer readable memory that can be oriented to a computer or other programmable data processing equipment to implement a function in a particular way, so that computer readable or computer readable memory The instructions stored in it are also possible to produce an article of manufacture containing instructions means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And since computer program instructions may be mounted on a computer or other programmable data processing equipment, a series of operation steps are performed on a computer or other programmable data processing equipment to create a process that is executed by the computer to generate a computer or other programmable It is also possible for instructions to perform data processing equipment to provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or each step can represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that it is also possible for functions to occur out of sequence. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, and it is also possible that the blocks or steps are performed in the reverse order of the corresponding function as necessary.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention exemplified below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art.

In recent years, Deep Learning has made significant strides, especially in video, language and signal processing. Accordingly, the present invention provides an apparatus and method for detecting motion anomalies of an SMD assembly equipment based on deep learning.

The advantage of this deep learning-based motion anomaly detection technique is that the performance is excellent when the collected training data and training time for model training are sufficient. In particular, there is an advantage that a model can be trained using only training data when the SMD assembly equipment is in a normal state, and then it can be determined that it is anomaly when untrained data is input. For example, in the case of an electrocardiogram (Electrocardiography: ECG), a model capable of determining whether an abnormality of data is inputted later is used by learning using only normal training data. In general, a short-time ECG waveform pattern is characterized by one-to-one correspondence. That is, in the normal ECG pattern, the R waveform appears immediately after the Q waveform. Therefore, the ECG inspection model predicts a subsequent waveform sequence from the currently input waveform sequence.

However, the SMD assembly equipment has a characteristic that when the assembly object is changed, the operation sound generated by the equipment also changes. That is, unlike the ECG pattern, the sound pattern generated in the SMD assembly equipment has a one-to-many correspondence, so it is impossible to learn and predict the future waveform sequence from the currently input waveform sequence. Therefore, when the assembly object is changed, it is necessary to retrain the model using the steady state data corresponding to the changed assembly object. However, the number of objects of assembly of SMD assembly equipment is numerous, so it is very difficult to train a model whenever the assembly object is changed. Therefore, the present invention designs a sound restoration model that restores the currently input sequence data so that the abnormality of the SMD assembly equipment can be detected quickly and flexibly.

1 is a block diagram of an apparatus for detecting an operation of a deep learning-based SMD assembly equipment according to an embodiment of the present invention, and FIG. 2 is a fast adaptive RNN encoder / decoder model according to an embodiment of the present invention This is a diagram to explain the structure of Encoder-Decoder (FARED).

Referring to FIG. 1, the apparatus 100 for detecting anomalies of a deep learning-based SMD assembly equipment includes an input processing unit 10, a sound restoration model 20, an error calculation unit 30, and an abnormality determination unit 40 You can. Additionally, the model learning adjustment unit 50 may be further included.

,

,..,

를 제1 소리 신호로 소리 복원 모델(20)에 순차적으로 입력시킨다. 이때, 분할된 시퀀스 스펙트럼(Sequential Spectrum) 신호

,

,..,

는 연속되는 신호들 간에 일정 부분, 예컨대 50% 정도는 중첩될 수 있다. 이는 각각의 시퀀스들 간의 redundant 정보를 제공하기 때문에, 모델 학습을 좀 더 용이하게 한다. The input processing unit 10 inputs the first sound signal extracted from the operational sound waveform of the SMD assembly equipment into the sound restoration model 20. 2 shows an example of the operational sound waveform 210 of the SMD assembly equipment, the input processing unit 10 is a sequence spectrum (Sequential Spectrum) signal that divides (Slicing) the entire sound waveform 210 in units of a predetermined time length

,

, ..,

Is sequentially input to the sound restoration model 20 as the first sound signal. At this time, the divided sequence spectrum (Sequential Spectrum) signal

,

, ..,

May overlap some of the consecutive signals, for example, about 50%. This makes model training a little easier because it provides redundant information between each sequence.

The input processing unit 10 may input the sequence spectrum data extracted from the sound waveform 210 as it is, but may perform a pre-processing process according to an embodiment of the present invention. In other words, using the deep learning model, it is possible to derive the result by extracting the feature from the artificial neural network by itself without having to extract the feature from the data, but using raw data adds feature extraction. In addition to requiring additional parameters, additional training time is also required, so if the artificial neural network is trained on features extracted in advance from the data, neurons for feature extraction are no longer needed. Accordingly, the input processing unit 10 may use a short-time fourier transform (STFT) or a Mel-Frequency Cepstral Coefficients (MFCC) feature extraction technique as a pre-processing technique. A detailed description thereof will be described later with reference to FIG. 3.

,

, ...,

가 입력됨에 따라, 입력된 소리 신호들과 가까운 출력 데이터

,

, ...,

를 복원하도록 훈련된 딥러닝 기반 인공 지능 학습 모델이다. As the first sound signal is input from the input processing unit 10, the sound restoration model 20 restores the input first sound signal and outputs a second sound signal. The sound restoration model 20 uses a first sound signal identical to the input signal as a labeling signal, that is, a ground truth signal, to the input signal and the input signal when the SMD assembly equipment normally assembles the assembly target as training data. That is, referring to FIG. 2, the sound restoration model 20 is a sequence spectrum signal extracted from the sound waveform 210 as training data.

,

, ...,

Is input, the output data close to the input sound signals

,

, ...,

It is a deep learning-based artificial intelligence learning model trained to restore.

,

, ...,

와 라벨링된 신호

,

, ...,

간의 오차를 최소화되도록 소리 복원 모델(20)의 신경망을 역전파시켜 소리 복원 모델(20)을 최적화시킨다. Therefore, in order to train the sound restoration model 20, the adjustment unit 50 to be described later is output data calculated by the error calculation unit 30

,

, ...,

And labeled signals

,

, ...,

The sound restoration model 20 is optimized by back propagating the neural network of the sound restoration model 20 to minimize errors in the liver.

In addition, according to an embodiment of the present invention, the learning algorithm used for generating the sound restoration model is a Recurrent Neural Network (RNN) model or LSTM (Long-) using a GRU (Gated Recurrent Unit) suitable for processing sequence data. Short Term Memory).

Recurrent Neural Network (RNN) or Long-Short Term Memory (LSTM) is a learning algorithm that learns time series data that changes over time and predicts artificial intelligence. RNN is the deepest network structure among deep learnings. Examples of time-series data include a stock price, a person's movement, climate, the number of Internet users, search terms, and the like, including the transmission and reception signals as in the present invention. LSTM is a long-short term memory algorithm that solves the phenomenon of vanishing gradient by placing the gate unit for each node and forgetting data from a long time ago due to the deep neural network.

Also, the sound restoration model 20 may be a multi-layer neural network formed by connecting at least two recursive neural network units 21 in series. For example, FIG. 2 shows a multi-layer network in which three RNN cells are stacked. However, the present invention is not limited thereto, and the recursive neural network unit 21 may use various types of recursive neural networks such as RNN, LSTM, and GRU.

As described above, in the present invention, the benefit of designing an RNN-based deep learning model can significantly reduce the number of neurons in an artificial neural network to be learned compared to other artificial neural networks, thus reducing the time required for learning and the SMD assembly equipment to be assembled. Changes can quickly respond to new environments. Therefore, as the assembly process of the SMD assembly equipment is changed, it is possible to flexibly and adaptively detect an operation abnormality.

Meanwhile, the error calculator 30 calculates an error (Euclidean distance) between the first sound signal input to the sound recovery model 20 and the second sound signal output from the sound recovery model 20. That is, it is to calculate how close the sound is restored to the first sound signal input by the sound restoration model 20.

The equipment abnormality determination unit 40 determines whether the SMD assembly equipment is abnormal according to whether the value calculated by the error calculation unit 30 is greater than or equal to a predetermined threshold. That is, the sound restoration model 20 determines that the restoration fails when the untrained sound signal is input as the first sound signal rather than the sound that the SMD assembly equipment operates in a normal state, and the SMD assembly equipment is abnormal. Is to do.

3 is a view for explaining the operation of the input processing unit according to an embodiment of the present invention.

Referring to FIG. 3, the input processing unit 10 processes the first sound signal using the MFCC feature extraction technique. This MFCC feature extraction technique is very useful for speech recognition, which extracts frequency information from the spectrum to reduce the dimension of the data, thereby reducing computation and training time.

,

,..,

각각에 MFCC 특징 추출 기법을 적용한다. 즉, 입력 처리부(10)는 도 3의 (a)에 도시된 시퀀스 스펙트럼(Sequential Spectrum) 신호를 (b)에 도시된 바와 같이 소정 시간 길이로 분할하고, (c)에 도시된 바와 같이 분할된 신호를 시간 평균(time-averaged) 스펙트럼을 계산한다. 입력 처리부(10)는 전술한 바와 같이 산출된 시간 평균 스펙트럼을 제1 소리 신호 및 ground truth 신호로 사용할 수 있다. The input processing unit 10 is a sequence spectrum (Sequential Spectrum) signal by dividing (Slicing) the entire sound waveform 210 shown in Figure 2 in a predetermined time length unit

,

, ..,

MFCC feature extraction technique is applied to each. That is, the input processing unit 10 divides the sequence spectrum signal shown in (a) of FIG. 3 into a predetermined time length as shown in (b), and is divided as shown in (c). Compute the time-averaged spectrum of the signal. The input processing unit 10 may use the time average spectrum calculated as described above as a first sound signal and a ground truth signal.

Therefore, in addition to reducing the number of neurons in the artificial neural network primarily by using the RNN-based model, it is possible to reduce the neurons secondarily by extracting features from sound data.

4 is a flowchart illustrating a method of detecting an operation abnormality of a deep learning-based SMD assembly equipment according to an embodiment of the present invention.

,

, ...,

가 입력됨에 따라, 입력된 소리 신호들과 가까운 출력 데이터

,

, ...,

를 복원하도록 훈련된다. 훈련 과정은 출력 데이터

,

, ...,

와 라벨링된 신호

,

, ...,

간의 오차를 최소화되도록 소리 복원 모델(20)의 신경망을 역전파시켜 소리 복원 모델(20)을 최적화시키는 것이다. Referring to FIG. 4, a method for detecting an operation abnormality of a deep learning-based SMD assembly equipment is performed by the device 100, which firstly uses training data as an operation sound when the SMD assembly device normally assembles an assembly object. The sound restoration model 20 is trained (S410). That is, a labeling signal to an input signal and an input signal as training data, that is, a ground sound signal, uses the same first sound signal as the input signal. That is, referring to FIG. 2, the sound restoration model 20 is a sequence spectrum signal extracted from the sound waveform 210 as training data.

,

, ...,

Is input, the output data close to the input sound signals

,

, ...,

It is trained to restore. Training process output data

,

, ...,

And labeled signals

,

, ...,

The sound restoration model 20 is optimized by back propagating the neural network of the sound restoration model 20 so as to minimize errors in the liver.

Next, the device 100 restores the first sound signal extracted from the operational sound waveform of the SMD assembly equipment through the sound restoration model 20 to generate a second sound signal (S420). Here, according to one embodiment, the first sound signal is a spectrum (Spectrum) divided by a predetermined time length in the operating sound waveform of the SMD assembly equipment, may be overlapped by a certain portion between two consecutive spectra. According to another embodiment, the first sound signal may be a time average spectrum obtained by dividing each spectrum into a predetermined time length and averaging the divided signals.

Then, the apparatus 100 calculates an error (Euclidean distance) between the first sound signal input to the sound recovery model 20 and the second sound signal output from the sound recovery model 20 (S430). That is, it is to calculate how close the sound is restored to the first sound signal input by the sound restoration model 20.

The apparatus 100 determines whether the calculated error is greater than or equal to a predetermined threshold (S440). That is, when the error calculated in S440 is greater than or equal to a predetermined threshold, the device 100 determines that there is an abnormality in the SMD assembly equipment (S450). On the other hand, if the error calculated in S440 is not greater than a predetermined threshold, the device 100 determines that the SMD assembly equipment is normal (S460).

Claims (6)

An input processing unit for extracting the first sound signal from the operating sound waveform of the surface mounted device (SMD) assembly equipment,
A sound restoration model learned based on deep learning to restore the first sound signal extracted by the input processing unit and output the second sound signal;
An error calculation unit for calculating an error between the first sound signal and the second sound signal,
An apparatus for detecting anomalies of a deep learning-based SMD assembly equipment including an equipment abnormality determination unit that determines whether or not the SMD assembly equipment is abnormal according to whether the error calculated in the error calculation unit is greater than or equal to a predetermined threshold.
The method of claim 1, wherein the sound restoration model
Recurrent learning including Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), and Gated Recurrent Unit (GRU), which sequentially receives time samples and learns them sequentially. A device for detecting motion anomalies in deep-learning based SMD assembly equipment using artificial neural networks.
The method of claim 1, wherein the input processing unit
Operation of SMD assembly equipment More than the operation of deep learning-based SMD assembly equipment, which further includes an input processing unit that extracts features from the waveform of the sound using MFCC (Mel-Frequency Cepstral Coefficients) technique and generates a first sound signal and inputs it to a sound restoration model Detection device.
The method of claim 1, wherein the first sound signal
Spectrum divided into a predetermined length of time in the operational sound waveform of the SMD assembly equipment,
A device for detecting motion anomalies in deep learning-based SMD assembly equipment that overlaps a certain portion between two consecutive spectra.
The method of claim 5, wherein the first sound signal
A device for detecting anomalies in a deep learning-based SMD assembly equipment, which is a time-averaged spectrum obtained by dividing each spectrum into a predetermined time length and averaging the divided signals.
Learning a sound restoration model by using a first sound signal extracted from an operating sound waveform when a surface mounted component (SMD) assembly equipment normally assembles an assembly target;
Generating a second sound signal by restoring the first sound signal extracted from the operational sound waveform of the SMD assembly equipment through the learned sound restoration model;
Calculating an error between the first sound signal and the second sound signal,
And determining whether or not the SMD assembly equipment is abnormal according to whether the calculated error is greater than or equal to a predetermined threshold.

Images