KR20210129499A - Data preprocessing method for detecting work errors in the process for assembling surface mount device - Google Patents

Abstract

As a sound data preprocessing method for a work error detection device that detects a work error by analyzing the sound data generated by sequentially repeating unit works in a surface mount device (SMD) assembly process, the present invention comprises: a step of dynamically determining a window size based on a length of the output data and a length of the waveform data preset with respect to the waveform data; a step of generating frames by windowing the waveform data with a determined window size, converting the waveform data into a time-frequency domain, and averaging on a time axis; and a step of combining the generated frames in chronological order to generate the output data. Therefore, the present invention is capable of allowing a device that detects the work error to learn quickly and effectively.

Description

Data preprocessing method for detecting work errors in the assembly process of surface mount devices

The present invention relates to a sound-based error detection technology, and more particularly, to a method for pre-processing acoustic data by a surface mount device assembly process that is collected to detect an operation error.

Automated equipment for assembling a surface mounted device (SMD) is being produced by directly attaching a surface mounted component (SMC) to the surface of a printed circuit board (PCB). Automated equipment for assembling SMDs assembles multiple types of SMDs in a small amount, and the operation speed of the equipment in the assembly process is very fast.

The SMD assembly process has a certain work pattern in which unit operations including PCB board warehousing, component picking, component movement, component mounting, and PCB board release are sequentially repeated.

Since unit tasks have a work pattern that is repeated sequentially, if an operation is stopped due to a failure in a certain unit task, the entire SMD assembly process is affected and the automated assembly process stops.

Therefore, in the SMD assembly process, it is important to quickly detect errors in the unit operations and to quickly recover the unit operations in which the failure occurred.

In Korea Patent Publication No. 10-2019-0081594 (published on July 9, 2019), acoustic data generated in an automated production line is analyzed using an auto-encoder model, a deep learning model, to detect work errors in an automated production line. A method for predicting is disclosed.

When the SMD assembly equipment of the SMD assembly process assembles various products, an error detection model that can adapt very quickly to various assembly environments is required. It tends to be rather long. Therefore, if the target product to be assembled by the SMD assembly facility is frequently changed and the learning time is longer than the preparation time required to change to a new target product, it may be difficult to apply to the actual assembly process.

No. 10-2019-0081594

An object of the present invention is to provide a method of pre-processing acoustic data so that an apparatus for detecting an operation error by analyzing acoustic data generated in an SMD assembly process can learn quickly and effectively.

In addition, the present invention generates sequence data of a fixed length from waveform data of unit operations having periods of different lengths, so that a deep learning model for detecting operation errors has periodicity even if the periods of unit operations are different. Another purpose is to provide a pre-processing method that allows you to judge the situation.

According to an aspect of the present invention, an acoustic data preprocessing method for an operation error detection apparatus for detecting an operation error by analyzing acoustic data generated by sequentially repeating unit operations in a surface mount device (SMD) assembly process is a waveform (waveform) ) dynamically determining a window size based on the length of the waveform data and the length of the output data preset for the data, and windowing the waveform data with the determined window size to convert the waveform data into the time-frequency domain It includes the steps of generating frames by transforming and averaging on a time axis, and generating output data by combining the generated frames in chronological order.

In this case, the waveform data is data obtained by dividing the acoustic data collected in the surface mounting device assembly process into a length corresponding to a cycle of a repeated operation.

According to the present invention, it is possible to pre-process the acoustic data so that an apparatus for detecting an operation error by analyzing the acoustic data generated in the SMD assembly process can learn quickly and effectively.

In addition, the present invention can generate sequence data of a fixed length for the deep learning model to learn from waveform data of unit operations having periods of different lengths even when the periods of unit operations are different.

1 is a conceptual diagram illustrating a concept of a procedure for generating sequence data of a fixed length from waveform data of different lengths.
2 is an example of a deep learning model for learning preprocessed sequence data.
3 is a flowchart illustrating a procedure of a method for pre-processing sound data for an apparatus for detecting an operation error according to an aspect.

The foregoing and additional aspects are embodied through the embodiments described with reference to the accompanying drawings. It is understood that various combinations of elements in each embodiment are possible within the embodiments as long as there is no contradiction between them or other mentions. Each block in the block diagram may in some cases represent a physical component, but in other cases may be a part of the function of one physical component or a logical representation of a function across a plurality of physical components. Sometimes a block or part of an entity may be a set of program instructions. All or a part of these blocks may be implemented by hardware, software, or a combination thereof.

According to an aspect of the present invention, an acoustic data preprocessing method for an operation error detection apparatus for detecting operation errors by analyzing acoustic data generated by sequentially repeating unit operations in a surface mount device (SMD) assembly process is a window size dynamic It includes a determining step, an average frame generation step, and an output data generation step.

A surface mounted device (SMD) is a device in which small electronic components are attached on a printed circuit board (PCB). The SMD assembly process is a process of producing various types of SMD in small quantities through SMD assembly equipment operating in an automated production line. In the SMD assembly process, if one device fails and stops, not only the device but the entire automated production line will be affected and the SMD assembly will stop. Therefore, the SMD assembly process should be able to detect errors quickly.

The work error detection device is a device for detecting an error occurring in an automated SMD assembly process, and the present invention is a method applicable to a device for detecting an error using sound, that is, acoustic data generated in the SMD assembly process. The SMD assembly process is a process in which unit operations including PCB warehousing, component acquisition, component attachment, and PCB shipment are repeated according to a preset program. SMD assembly equipment used in the SMD assembly process operates according to a preset program sequence, so it always operates consistently except for exceptional circumstances such as a stop due to an error or lack of parts. Therefore, it is possible to detect an error from the operation sound of the SMD assembly equipment, and work error detection devices for detecting the error from the operation sound are being studied based on deep learning technology. Since the SMD assembly process is a process of producing multiple types of SMD in a small amount, the conversion of the target product to be produced occurs quickly. Therefore, it is necessary for error detection devices based on deep learning technology to quickly learn the production process of a new product, and for rapid learning, research is being conducted in a direction to reduce the learning time by preprocessing the acoustic data without using it as it is.

There is no limit to the deep learning techniques used by error detection devices. It may be based on a supervised learning model or it may be based on an unsupervised learning model.

Based on the supervised learning model, there is a model that detects errors based on acoustics using a classification model such as SVM (Support Vector Machine) or CNN (Convolution Neural Network). These classification models show high performance when classifying well-known classes. However, there is a problem in that it is necessary to collect all kinds of normal and abnormal data in order to train supervised learning models such as SVM and CNN, and there is a problem that data collection for some abnormal states may be practically impossible. In this case, the SVM and CNN models can classify the new steady-state data as an abnormal state, and vice versa, can classify the new abnormal state as a steady state, resulting in a classification error for new data. .

Based on the unsupervised learning model, there is a model that detects errors based on sound using an auto-encoder or a recurrent neural network (RNN) encoder-decoder model. Unlike supervised learning models, unsupervised learning models can be trained using only steady-state data without class distinction. Even in the case of an unsupervised learning model, since steady-state data can be collected through the assembly process, it is impossible to collect steady-state data for a specific SMD that has not undergone the assembly process. Therefore, there is a need for a pre-processing method that enables rapid learning within the waiting time for conversion of a target product to be assembled.

In general, the input data of a deep learning model uses a fixed length. Accordingly, devices for detecting an error in a work process using sound data also process sound data of a fixed length as input data. Mechanical devices in industrial sites, including SMD assembly devices, often repeat the same operation continuously, so the sound generated from the device often has a certain period. However, when acoustic data is divided into fixed lengths and used for training and error detection of deep learning models, information on the operating state may not be sufficiently transmitted. Therefore, in order to grasp the operating state from the sound data having periodicity, it is necessary to process the sound data as input data in units of a period. In this case, the operation of the cycle unit may be a unit operation or an operation of assembling one SMD device.

The acoustic data preprocessing method according to an aspect of the present invention may be implemented as a program instruction set executed in a computing device, and the device executing the method is a computing device including a PC, a server, and the like. According to an aspect of the invention, it may be implemented in an error detection device.

1 shows the concept of a procedure for generating sequence data of a fixed length from waveform data of different lengths. In order to apply the pre-processing method according to an aspect of the present invention, sound data must be divided in period units.

The window size dynamic determination step is a step of dynamically determining a window size based on a length of output data preset for waveform data and a length of the waveform data. In this case, the waveform data is data obtained by dividing the acoustic data collected in the surface mounting device assembly process into a length corresponding to the cycle of the unit operation for each unit operation. Additionally, the dynamic determination of the window size may be determined in consideration of an overlap ratio in addition to a preset length of output data and a length of waveform data.

In the example shown in FIG. 1 , a processing process when data is input at different cycles by using the assembling cycle of the target product as a cycle unit is illustrated. 1 shows an assembly process with a cycle of 8 seconds and an assembly process with a cycle of 16 seconds as examples. In the example of Fig. 1, the length T of the output data is set to 32, and the overlap ratio is set to 50%. In this case, the window size for the waveform data of the left 8-second period is 500 ms in length, and the window size of the waveform data of the right 16-second period is 1000 ms in length.

The average frame generation step generates frames by windowing the waveform data with a determined window size, converting the waveform data into a time-frequency domain, and averaging on the time axis. Windowing refers to multiplying the original signal by a window function in order to show only a part of the signal in signal processing, etc. There is no limitation on the window function applied at this time, but a Hanning window function may be applied. As an example of the time-frequency domain transformation, a Short-Time Furier Transform (STFT) may be used. That is, the converted frame may be a spectrogram converted by STFT. Even if the length of the output data is fixed, the size of the frame converted to the time-frequency domain by applying the window function varies according to the length of the assembly cycle. Therefore, it is difficult for the deep learning learning model to learn using this frame data as input data. The transformed frames are averaged on the time axis to make the frame size constant.

The output data generating step is a step of generating output data by combining the generated frames in chronological order. The length of the output data is equal to the size T of the set output data.

2 is an example of a deep learning model for learning preprocessed sequence data. The deep learning model shown in FIG. 2 is an RNN encoder-decoder model based on Bidirectional Long Short Term Memory (BLSTM). The input vector xi is a frequency component of a certain section of the acoustic data generated in the SMD assembly process, and the intermediate vector zi expresses the characteristics of the acoustic data in a compressed space as an output of the encoder. The decoder generates an output vector yi using the intermediate vector sequence as input data, and is trained in a direction that minimizes the difference between yi and xi. BLSTM consists of forward LSTM and backward LSTM. The model for learning the preprocessed sequence data may be another deep learning model, and it may also be an RNN encoder-decoder model and constructed using LSTMs instead of BLSTMs.

3 is a flowchart illustrating a procedure of a method for pre-processing sound data for an apparatus for detecting an operation error according to an aspect. Referring to FIG. 3 , the sound data preprocessing apparatus first receives and sets the length and overlap ratio of output data through a user interface or the like ( S1000 ). In this case, when the overlap ratio is fixed, the overlap ratio may not be input. The acoustic data pre-processing apparatus divides the acoustic data generated in the SMD assembly process based on the assembly cycle of the target products (S1020). The sound data preprocessing apparatus determines a window size based on the set output data length, overlap ratio, and the length of the divided waveform data (S1040), and windows the waveform data into a window having the determined window size (S1060). In this case, the applied window function may be a Hanning window function. The sound data preprocessor converts the windowed waveform data into STFT transform, that is, time-frequency domain (S1080). The sound data preprocessor generates one frame by averaging the windowed STFT-converted data on the time axis (S1100). Although the length of the windowed and STFT-converted data varies depending on the period, frame data of a constant size can be generated regardless of the period by averaging on the time axis. The sound data pre-processing apparatus generates output data by configuring the generated frames in a time order ( S1120 ). The output data is pre-processed data and may be used as input data of the error detection device.

Although the present invention has been described above with reference to the accompanying drawings, the present invention is not limited thereto, and it should be construed to encompass various modifications that can be apparent from those skilled in the art. The claims are intended to cover such variations.

Claims (3)

In the sound data preprocessing method for a work error detection device that detects work errors by analyzing sound data generated by sequentially repeating unit tasks in a surface mounted device (SMD) assembly process, the method comprising:
dynamically determining a window size based on a length of output data preset with respect to waveform data and a length of the waveform data;
generating frames by windowing the waveform data with a determined window size, converting the waveform data into a time-frequency domain and averaging on the time axis; and
generating output data by combining the generated frames in chronological order;
including,
Waveform data is an acoustic data preprocessing method for a work error detection device, which is data obtained by dividing the acoustic data collected in the surface mounting device assembly process into a length corresponding to the cycle of the repeated operation.
The method of claim 1,
The dynamic determination of the window size is an acoustic data preprocessing method for a work error detection apparatus that is determined based on a preset length and overlap ratio of output data and a length of waveform data.
The method of claim 1,
Windowing is an acoustic data preprocessing method for a work error detection device performed by applying a Hanning window function.

Images