TWI744177B - Method and system for vision-based defect detection - Google Patents

Abstract

A method and a system for vision-based defect detection are proposed. The method includes the following steps. A test audio signal is outputted to a device-under-test (DUT), and a response signal of the DUT with respect to the test audio signal is received to generate a received audio signal. Signal processing is performed on the received audio signal to generate a spectrogram, and whether the DUT has an unacceptable defect with respect to a predefined auditory standard is determined through computer vision according to the spectrogram.

Description

Defect detection visualization method and system

The invention relates to a detection technology, and in particular to a defect detection visualization method and system.

A speaker is a transducer that converts an electrical signal into an acoustic signal. It is widely used in audio equipment, earphones and other equipment, and its performance affects the use of these equipment. In the past, speaker assembly defects were detected by experienced listeners at the end of the production line. This type of detection requires the application of log-swept sine chirps to the speaker, and the use of human auditory detection to analyze whether the response signal is normal. However, the results detected by the human ear evaluation will vary with subjective factors such as the listener's age, mood changes, hearing fatigue, etc., and may easily cause occupational injury to the listener.

The present invention provides a defect detection visualization method and system, which can use computer vision technology to detect whether an object to be tested has unacceptable defects in a pre-established auditory standard from a time-spectrogram.

In an embodiment of the present invention, the above-mentioned method includes the following steps. The test audio signal is output to the object under test, and the response signal of the object under test to the test audio signal is received to generate a radio audio signal. Signal processing is performed on the radio audio signal to generate a time-spectrogram, and based on the time-spectrogram, it is visually determined whether the object under test has unacceptable defects in the pre-established auditory standard.

In an embodiment of the present invention, the aforementioned system includes a signal output device, a microphone, an analog-to-digital converter, and a processing device. The signal output device is used to output test audio signals to the object under test. The microphone is used to receive the response signal of the object under test to the test audio signal. The analog-to-digital converter is used to convert the response signal into a radio audio signal. The processing device is used for signal processing for the radio audio signal to generate a time-spectrogram, and based on the time-spectrogram, to visually determine whether the object under test has an unacceptable defect in the predetermined auditory standard.

Part of the embodiments of the present invention will be described in detail in conjunction with the accompanying drawings. The reference symbols in the following description will be regarded as the same or similar elements when the same symbol appears in different drawings. These embodiments are only a part of the present invention, and do not disclose all the possible implementation modes of the present invention. More precisely, these embodiments are just examples of methods and systems within the scope of the patent application of the present invention.

FIG. 1 is a block diagram of a defect detection system according to an embodiment of the present invention, but this is only for convenience of description, and is not intended to limit the present invention. First, Figure 1 first introduces all the components and configuration relationships in the defect detection system, and detailed functions will be disclosed in conjunction with Figure 2.

1, the defect detection system 100 includes a signal output device 110, a microphone 120, an analog-to-digital converter 130, and a processing device 140, which are used to detect whether the object T to be tested is defective.

The signal output device 110 is used to output the test audio signal to the object T under test. It can be, for example, an electronic device with a digital audio output interface, and outputs the test audio signal to the object T under test in a wireless or wired manner. The microphone 120 is used to receive the response of the object T under test to the test audio signal, and it can be arranged in the vicinity of the object T under test or at the best receiving position relative to the object T under test. The analog-to-digital converter 130 is connected to the microphone 120 for converting the analog audio signal received by the microphone 120 into a digital audio signal.

The processing device 140 is connected to the analog-to-digital converter 130 for processing the digital audio signal received from the analog-to-digital converter 130 to detect whether the object U under test is defective. The processing device 140 includes a memory and a processor. The memory can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (flash memory), hard disk Discs or other similar devices, integrated circuits, and combinations thereof. The processor can be, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessors or digital signal processors. (Digital signal processor, DSP) or other similar devices, integrated circuits, and combinations thereof.

It should be noted that, in one embodiment, the signal output device 110, the microphone 120, the analog-to-digital converter 130, and the processing device 140 can be divided into four independent devices. In an embodiment, the signal output device 110 and the processing device 140 can be integrated into the same device, and the processing device 140 can control the output of the signal output device 110. In one embodiment, the signal output device 110, the microphone 120, the analog-to-digital converter 130, and the processing device 140 may be an all-in-one computer system. The present invention does not impose any limitation on the integration of the signal output device 110, the microphone 120, the analog-to-digital converter 130, and the processing device 140, as long as the system including these devices belongs to the category of the defect detection system 100.

The following is a list of embodiments to illustrate the detailed steps of the defect detection system 100 performing the defect detection method for the object T to be tested. In the following embodiments, an electronic device with a speaker will be used as the object T to be tested, and the defect detected by the defect detection system 100 is a rub and buzz of the object T to be tested.

FIG. 2 is a flowchart of a defect detection visualization method according to an embodiment of the present invention. The process of FIG. 2 will be executed by the defect detection system 100 of FIG. 1.

1 and 2 at the same time, the signal output device 110 will output the test audio signal to the object T under test (step S202), the microphone 120 will receive the response signal of the object T under test to the test audio signal (step S204), analog and digital The converter 130 converts the response signal into a radio audio signal (step S206). Here, the audio range of the test audio signal can be 1K～20Hz, where the amplitude of 1K～500Hz is -25dB, the amplitude of 500Hz～300Hz is -15dB, and the amplitude of 300Hz～20Hz is -8dB. However, because the assembled abnormal sound will resonate with the specific frequency point of the test audio signal, and in order to prevent the resonance of the non-assembled abnormal sound from affecting the detection of the abnormal sound (such as button resonance), the audio range and amplitude of the test audio signal will be in accordance with the expected frequency. It may be adjusted depending on the test object T. The object T under test will generate a response signal to the test audio signal, and the microphone 120 will receive the response signal from the object T under test. Then, the analog-to-digital converter 130 performs analog-to-digital conversion of the analog response signal to generate a digital response signal (hereinafter referred to as “audio signal”).

The processing device 140 performs signal processing on the received audio signal to generate a time-spectrogram (step S208), and visually determines whether the object T under test has a defect based on the time-spectrogram (step S210). The processing device 140 may perform Fast Fourier Transform (FFT) for the radio audio signal to generate a time-frequency spectrum. The reason why the radio audio signal is converted into a time-spectrogram here is that the abnormal sound does not have significant characteristics in the radio audio signal, but the abnormal sound has time continuity when it resonates with the test audio signal. Therefore, if the time-domain signal is converted into a time-frequency spectrum After the picture, the abnormal sound feature will show the phenomenon of continuous time and energy clustering in the time-spectrogram, so as to use computer vision technology to achieve defect detection of the object under test.

Taking the schematic diagram of the time spectrogram shown in FIG. 3 according to an embodiment of the present invention as an example, the time spectrogram 310 corresponds to a sound signal without abnormal sound, and the time spectrogram 320 corresponds to a sound signal with abnormal sound. It must be noted that those with ordinary knowledge in the field should understand that the time spectrogram represents the distribution of signal strength over time and frequency, while the time spectrogram 310 and time spectrogram 320 simply use a curve to simply indicate the obvious signal strength. To explain. Here, the abnormal sound signal has the characteristics of time continuity and energy clustering in the time-spectrogram 320, such as the abnormal sound characteristic RB. Therefore, if the processing device 140 is used to analyze the time-frequency spectrogram with computer vision technology, it can be judged whether the device T under test has abnormal sound due to assembly defects.

In the following embodiments, a classifier will be used for image recognition. Therefore, before the processing device 140 determines whether the object T to be tested has a defect, it will obtain a trained classifier. The classifier here can be trained by the processing device 140 itself, or a trained classifier obtained from another processing device, and the present invention is not limited here.

4 is a functional block flow chart of constructing a classifier according to an embodiment of the present invention. From the following description, a similar processing device 140 (hereinafter referred to as a “training system”) will be used to construct the classifier.

Please refer to FIG. 4. First, the training system will collect multiple training data 402. The training data here can be N1 non-defective training sound samples and N2 defective training sound samples respectively generated by N1 non-defective training objects and N2 defective training objects in a manner similar to steps S202 to S204, where this N1+ The N2 training objects are the same as the object T to be tested, but have been tested for defects in advance.

Then, the training system converts the training data into a time-spectrogram 404. In order to reduce the computational complexity and to avoid low-frequency noise and high-frequency noise images, the training system will select a preset frequency range of, for example, 3K～15K Hz as the detection area. Taking FIG. 3 as an example, the area 315 is the detection area of the time spectrogram 310, and the area 325 is the detection area of the time spectrogram 320. For the convenience of explanation, the detection area in the time-spectrogram corresponding to the defect-free training sound sample is referred to as the "defect-free detection area image", and the detection area in the time-spectrogram corresponding to the defect-free training sound sample is referred to as the "defect Detection area image".

After that, the training system will obtain each defect detection area image and the feature values corresponding to different areas in each defect detection area image, and obtain the texture between each defect detection area image and the defect detection area image and the reference model 408. The texture correlation 406 is used as the spatial feature 410 to train the classifier 412 to generate a classifier 414 for detecting whether the object T to be tested is defective.

200的像素尺寸）。在本實施例中，若是子區塊的尺寸太大，將會降低異音特徵的比重，而若是子區塊的尺寸太小，將會無法涵蓋異音特徵而影響後續的辨識結果。因此，訓練系統可以例如是以圖5根據本發明一實施例所繪示的功能方塊流程圖來取得各個缺陷檢測區域影像以及無缺陷檢測區域影像的空間特徵。 Here, the training system will first perform image segmentation of all defect detection area images and non-defect detection area images to generate multiple sub-blocks of the same size (for example,

200 pixel size). In this embodiment, if the size of the sub-block is too large, the proportion of the abnormal sound feature will be reduced, and if the size of the sub-block is too small, the abnormal sound feature will not be covered and the subsequent identification results will be affected. Therefore, the training system may, for example, obtain the spatial characteristics of each defect detection area image and the non-defect detection area image according to the functional block flowchart shown in FIG. 5 according to an embodiment of the present invention.

800。T11為影像分割後的其中一個子區塊（以下稱為「訓練子區塊」），其像素尺寸為

200。另一方面，T0為T1經過影像金字塔處理（縮小處理）後所產生的影像，其像素尺寸為

400。T01為影像分割後的其中一個訓練子區塊，其像素尺寸將與訓練子區塊T11相同，即

200的像素尺寸。 Please refer to Figure 5, the training system will perform image pyramid processing H (image pyramid) on the images of each defect detection area and the images of the non-defect detection area to generate images of different scales. This embodiment will have two scales, namely the original image size and 1/4 of the original image size (reducing the length and width of the original image to 1/2 of the original image size). Here, one of the defect detection area images will be used to illustrate the flow of FIG. 5, and those with ordinary knowledge in the art can analogize the processing methods of other defect detection area images and non-defect detection area images. Assume that T1 is one of the defect detection area images, and its pixel size is

800. T11 is one of the sub-blocks after image segmentation (hereinafter referred to as the "training sub-block"), and its pixel size is

200. On the other hand, T0 is the image generated after T1 undergoes image pyramid processing (reduction processing), and its pixel size is

400. T01 is one of the training sub-blocks after image segmentation, and its pixel size will be the same as the training sub-block T11, namely

200 pixel size.

（standard deviation）以及直方圖偏態

（Kurtosis）至少之一者來做為每個訓練子區塊的特徵值，然而本發明不以此為限。此外，為了提高無缺陷與缺陷的差異性，訓練系統更可以根據N1個無缺陷檢測區域影像來產生關聯於無缺陷的參考模型。舉例來說，訓練系統可以是將相同尺度的N1個無缺陷檢測區域影像的像素值進行平均，以獲得參考模型。因此，各個尺度分別有其對應的參考模型。在本實施例中，訓練系統將會產生對應於影像T1的參考模型R1以及對應於影像T0的參考模型R0。在此的影像T1與參考模型R1具有相同尺度，因此影像T1中的訓練子區塊將會在參考模型R1中找到對應的子區塊（以下稱為「參考子區塊」）。類似地，T0與參考模型R0具有相同尺度，因此影像T0中的訓練子區塊將會在參考模型R0中找到對應的參考子區塊。 Then, the training system will perform feature extraction FE for each of the non-defect detection area images of different scales and each training sub-block segmented from the defect detection area images. In this embodiment, the training system may, for example, calculate the standard deviation of the pixel value of each training sub-block of various scales.

(Standard deviation) and histogram skewness

(Kurtosis) At least one of them is used as the feature value of each training sub-block, but the present invention is not limited to this. In addition, in order to improve the difference between defect-free and defect-free, the training system can also generate a reference model related to the defect-free based on the images of N1 defect-free detection areas. For example, the training system may average the pixel values of N1 images of the defect-free detection area of the same scale to obtain a reference model. Therefore, each scale has its corresponding reference model. In this embodiment, the training system will generate a reference model R1 corresponding to the image T1 and a reference model R0 corresponding to the image T0. The image T1 here has the same scale as the reference model R1, so the training sub-blocks in the image T1 will find the corresponding sub-blocks in the reference model R1 (hereinafter referred to as “reference sub-blocks”). Similarly, T0 and the reference model R0 have the same scale, so the training sub-block in the image T0 will find the corresponding reference sub-block in the reference model R0.

（coefficient）。 Then, the training system will calculate the texture correlation between each sub-block of each scale and the reference sub-block in the corresponding reference model. Specifically, the training system will calculate the texture correlation between the training sub-block T11 and the reference sub-block R11 and the texture correlation between the sub-block T01 and the reference sub-block R01. The texture correlation here can be the correlation coefficient of the local binary pattern (LBP) between the sub-block and the reference sub-block

(Coefficient).

，每張影像將會有各自的影像特徵向量

，其中 n為子區塊的數量。以圖5為例，缺陷檢測區域影像T1將會有影像特徵向量

，其中

為缺陷檢測區域影像T1中訓練子區塊的數量。類似地，影像T0將會有影像特徵向量

，其中

為影像T0中訓練子區塊的數量。之後，訓練系統可將兩個尺度的影像特徵向量連接（concatenate）為特徵向量

來輸入至分類器M。 Here, each sub-block will have its own feature vector

, Each image will have its own image feature vector

, Where n is the number of sub-blocks. Take Figure 5 as an example, the defect detection area image T1 will have an image feature vector

,in

Is the number of training sub-blocks in the defect detection area image T1. Similarly, image T0 will have image feature vector

,in

Is the number of training sub-blocks in the image T0. After that, the training system can concatenate the image feature vectors of the two scales into feature vectors

To input to the classifier M.

After the training system has input all the feature vectors corresponding to all N1+N2 training data to the classifier, it will start to train the classifier M. The classifier here can be a support vector machine (support vector machines, SVM) classifier, and the training system will calculate the optimal separating hyperplane of the SVM classifier to determine whether the test object T Defective basis.

FIG. 6 is a functional block flow chart of a defect detection method according to an embodiment of the present invention, and the flow of FIG. 6 is applicable to the defect detection system 100. Before performing the process of FIG. 6, the processing device 140 will pre-store the reference model and the classifier mentioned in FIG. 5.

1 and 6 at the same time, first, similar to step S206 and step S208, the processing device 140 will obtain the test data 602 (that is, the radio audio signal corresponding to the object T under test), and convert the test data into time Spectrogram 604. "The test data here is the radio audio signal in step S206.

Next, the processing device 140 will obtain a plurality of sub-blocks associated with the time-spectrogram to obtain the spatial feature 610 therefrom to input to the classifier 612. In this embodiment, the processing device 140 also selects a preset frequency range of, for example, 3K-15K Hz as the detection area to generate the detection area image. In an embodiment, the processing device 140 may directly divide the detection area image, and directly generate multiple sub-blocks of the same size. In another embodiment, the processing device 140 may perform image pyramid processing on the detection area image to generate multiple detection area images of different scales. Then, the processing device 140 divides the detection area images of different scales to generate a plurality of sub-blocks of the same size.

After that, the processing device 140 will obtain the feature value of each sub-block, and obtain the texture correlation 606 between each sub-block and the reference model 608. The feature value here is, for example, at least one of the standard deviation of the pixel value of the sub-block and the histogram skewness, but it needs to meet the input requirements of the pre-stored classifier. The texture correlation here may be the correlation coefficient of the local binary pattern between the sub-block and the reference sub-block corresponding to the reference model. Then, the processing device 140 inputs the feature value and texture correlation corresponding to each sub-block to the classifier 612 to generate an output result, and the output result will indicate whether the object T to be tested has a defect.

In this embodiment, in order to achieve a more rigorous inspection, so as to prevent the object T to be tested that is actually defective from being misjudged as non-defective, the processing device 140 may furthermore when the output result indicates that the object T to be tested does not have a defect. Further confirm according to the reliability of the output result. In detail, taking the SVM classifier as an example, the processing device 140 may obtain a confidence level of the output result, and determine whether the confidence value is greater than a preset confidence threshold 614, where the preset confidence threshold may be 0.75. If so, the processing device 140 will determine that the object T to be tested does not have a defect. Otherwise, the processing device 140 will determine that the object T to be tested has a defect.

In this embodiment, the defect detected by the defect detection system 100 is an assembly abnormal sound of the object T to be tested. Since different types of assembly abnormal sounds will generate resonance harmonics when playing a specific audio signal, the processing device 140 can further use the frequency and harmonic frequency range of the abnormal sounds in the time-spectrogram to determine the assembly abnormalities in the object T under test. Tone parts. From another point of view, the processing device 140 will determine the component that caused the assembly abnormal sound based on the specific region of the time-frequency spectrum.

For example, FIG. 7 is a schematic diagram of a time-spectrogram according to an embodiment of the present invention, and the following only illustrates a partial area of the time-spectrogram. Both the time spectrogram 710 and the time spectrogram 720 have assembled abnormal sounds. Since the resonant frequency point when the screw is not tightened is a single-point resonance of 460 Hz, the processing device 140 can obtain from the time spectrum diagram 710 that the screw of the device T under test is not tightened. Since the resonant sound caused by the iron filings in the speaker unit will resonate at 460-350 Hz, the processing device 140 can obtain from the time spectrum diagram 720 that there is iron filings in the device T under test.

In practice, in order to avoid overkill, when the object to be tested is judged to be defective due to assembly of abnormal sound, the tester can judge whether the abnormal sound is acceptable or unacceptable according to the volume of the abnormal sound. When the abnormal sound is acceptable (difficult or undetectable by human ears), the object under test will be regarded as an "OK" object under test. When the abnormal sound is unacceptable, the object under test will be regarded as "NG" object under test. Visually, FIG. 8 is a schematic diagram of a time-spectrogram 810 with acceptable abnormal sounds and a time-spectrogram 820 with unacceptable abnormal sounds according to an embodiment of the present invention, wherein the time-spectrogram 820 includes Clear clusters of high brightness. In subsequent embodiments, a machine learning-based quantization mechanism will be proposed based on this observation, which can classify abnormal sounds into acceptable and unacceptable, so as to reduce the overkill rate.

FIG. 9 is a block diagram of a defect detection system according to an embodiment of the present invention, but this is only for convenience of description and is not intended to limit the present invention. First, Figure 9 first introduces all the components and configuration relationships in the defect detection system, and detailed functions will be disclosed in conjunction with Figure 10.

Please refer to FIG. 9, the defect detection system 900 includes a signal output device 910, a microphone 920, an analog-to-digital converter 930, and a processing device 940, wherein similar numbers prefixed with "9" in the first code are used to indicate components similar to those in FIG. The defect detection system 900 is used to determine whether the object T to be tested has an unacceptable defect in the predetermined auditory standard. Here, the pre-established auditory standard may be a range set according to human ear perception, a customized range in response to customer needs, a range defined by a third party, and so on. In the following embodiments, an electronic device with a speaker will also be used as the object T to be tested for description, and the defect detected by the defect detection system 900 is an assembly abnormal sound of the object T to be tested.

FIG. 10 is a flowchart of a defect detection visualization method according to an embodiment of the present invention. The process of FIG. 10 will be executed by the defect detection system 900 of FIG. 9.

9 and 10 at the same time, the signal output device 910 will output the test audio signal to the object T under test (step S1002), and the microphone 920 will receive the response signal of the object T under test to the test audio signal (step S1004). Next, the analog-to-digital converter 930 converts the response signal into a radio audio signal (step S1006), and the processing device 940 performs signal processing on the radio audio signal to generate a time-spectrogram (step S1008). Here, for the details of steps S1002 to S1008, please refer to the related descriptions of steps S202 to S208, which will not be repeated here. then. The processing device 940 visually determines whether the object T to be tested has an unacceptable defect in the pre-established auditory standard based on the time-spectrogram (step S1010). In this embodiment, the processing device 940 may first determine whether the object T to be tested has a defect based on the time-frequency spectrum in a manner similar to step S208. If so, the processing device 940 may further determine whether the defect is unacceptable in the pre-established auditory standard according to the time-spectrogram, so as to avoid excessive elimination.

Based on this, before the processing device 940 determines whether the test object T has an unacceptable defect, another classifier will be trained and constructed. The classifier here can be trained by the processing device 140 itself, or a trained classifier obtained from another processing device, and the present invention is not limited here. In the following embodiments, a similar processing device 940 (hereinafter referred to as a “training system”) will be used to construct a classifier. First, the training system will collect multiple training data, and these training data are training objects marked as "acceptable defects" in the pre-established auditory standards. The training system will perform projection conversion and feature quantization processing on the time-spectrogram corresponding to each training sound sample according to the time and space characteristics presented in the time-spectrogram of the object under test with abnormal sound.

FIG. 11 is a functional diagram of converting a time-spectrogram to a projection curve according to an embodiment of the present invention.

Please refer to FIG. 11, the training system extracts the region of interest 1115 from the time-spectrogram 1110, where the region of interest 1115 can be, for example, the region identified in FIG. area. Next, the training system divides the region of interest 1115 of the time-spectrogram 1110 into multiple sub-time-spectrograms (for example, three regions R1 to R3 in this embodiment) relative to different frequency levels (ie, horizontal division). The training system converts the two-dimensional sub-time spectrograms R1 to R3 into one-dimensional projection curves R1 to R3, respectively. For example, this conversion may be to average the energy values (ie, the vertical direction) of each time in each sub-time spectrogram R1 to R3.

For the projection curves CR1 to CR3 in which the horizontal axis represents time and the vertical axis represents energy, the projection value of the sub-time spectrogram with abnormal sound characteristics will be higher. When the projection value is continuously high in time, it is likely to have serious abnormal noise. In addition, the abnormal sound characteristics will be further classified into unacceptable (severe) and acceptable abnormal sound characteristics in the pre-established auditory standards. It is assumed that the pre-established auditory standards are set according to the range of human hearing perception. The human ear will be more sensitive to certain frequencies. For example, when the abnormal sound feature only appears in the sub-time spectrogram R1 (the frequency is about greater than 10K), the abnormal sound may be acceptable. However, when the abnormal sound feature appears in all sub-time spectrograms R1 to R3, the abnormal sound may be unacceptable. In other words, unacceptable (serious) abnormal sounds have the following characteristics: (1) higher projection energy, (2) longer duration, and (3) wider frequency range. Next, the training system will perform feature quantification.

、

、

、

以及

。子時頻譜圖

中的第

區域中的第

區段所對應的各個區段

的特徵值將會涉及到在對應的區段的資料點的統計參數以及對應的子時頻譜圖所分配到的權重，例如根據方程式(1)：

(1)

在此，

為區段

中數值大於區段平均值

的平均值，並且

為大於該區段平均值的k倍標準差。

以及

分別為

以及

的平均值，

以及

分別為

以及

的數量。在此，

以及

組成的

集合代表區段

中大於區段平均值

的數值，

為區段

中資料點的數量，

為不同子時頻譜圖的權重。L為各個子時頻譜圖的係數，其中越低頻的子時頻譜圖的係數越低，並且該區間的異音越為重要。 In detail, in order to highlight local features, each projection curve will be further divided into multiple segments (ie, vertical division) according to different time intervals. For example, the projection curve CR1 in Figure 11 can be further divided into five sections as shown in Figure 12.

,

,

,

as well as

. Sub-time spectrogram

In the

In the area

Each section corresponding to the section

The characteristic value of will be related to the statistical parameters of the data points in the corresponding section and the weight assigned to the corresponding sub-time spectrogram, for example, according to equation (1):

(1)

here,

For segment

The median value is greater than the segment average

The average of and

It is k times the standard deviation greater than the average value of the segment.

as well as

Respectively

as well as

average of,

as well as

Respectively

as well as

quantity. here,

as well as

consist of

Collection representative section

Medium is greater than the segment average

The value of

For segment

The number of data points in the

Is the weight of the spectrogram at different sub-times. L is the coefficient of each sub-time spectrogram, where the lower the frequency of the sub-time spectrogram, the lower the coefficient, and the more important the abnormal sound in this interval.

，則

以及

。假設

，則

。假設

以及

，則

，

，

，以及

。假設低頻子時頻譜圖的權重為

，則

。區段

的量化結果可以表示成

。 For convenience and clarity, suppose

,but

as well as

. Hypothesis

,but

. Hypothesis

as well as

,but

,

,

,as well as

. Assuming that the weight of the low frequency sub-time spectrogram is

,but

. Section

The quantitative result of can be expressed as

.

，將會根據習知的機器學習或深度學習模型來建立並且訓練用來判斷可接受的異音的單一類別支援向量機（one-class SVM，OCSVM）分類器。之後，此分類器將可判別出可接受與不可接受的異音。 When the training system calculates each sub-time of the training object with acceptable defects, the feature quantization result of the spectrogram V=

, Will build and train a single-class support vector machine (one-class SVM, OCSVM) classifier for judging acceptable abnormal sounds based on conventional machine learning or deep learning models. After that, this classifier will be able to distinguish acceptable and unacceptable abnormal sounds.

Please go back to Figure 10, it should be clear that S1010 has details about visually judging whether the object T under test has unacceptable defects in the pre-established auditory standard based on the time-spectrogram and will correspond to the training OCSVM classification. Steps. In detail, when the processing device 940 receives the time spectrogram, it will extract the region of interest from the time spectrogram, and divide the region of interest according to different frequency levels to generate multiple sub-time spectrograms. Then, the processing device 940 converts all the sub-time spectrograms into one-dimensional projection curves, and further divides all the projection curves into multiple sections according to different time intervals. The processing device 940 calculates and inputs the feature quantization result of each sub-time spectrogram to the OCSVM classifier. The processing device 940 will obtain the confidence level of the output result (hereinafter referred to as the “defect confidence level”), and determine whether the defect confidence level is greater than the preset defect confidence level, where the preset defect confidence level may be 0. The preset defect confidence level here can be adjusted according to actual applications. When the determination result is affirmative, the processing device 940 will determine that the object T to be tested has an acceptable abnormal sound. When the determination result is negative, the processing device 940 will determine that the object T under test has an unacceptable abnormal sound.

For example, FIG. 13 is a data diagram of abnormal sound classification detection according to an embodiment of the present invention, wherein each data point represents an object to be tested. The defect confidence level of the object to be tested corresponding to point 1301 is 0.1, so it has an acceptable abnormal sound. In fact, the objects to be tested corresponding to all points in the cluster 1300 have acceptable abnormal sounds. The defect confidence level of the object under test corresponding to point 1303 is -0.38, and the defect confidence level of the object under test corresponding to point 1304 is -0.04, so both have unacceptable abnormal sounds. The object to be tested corresponding to point 1302 is an outlier, so it will be tested again.

Table 1 summarizes the experimental results of using defect detection visualization methods based on machine learning quantization mechanism without import (such as Figure 2) and with import (such as Figure 10) to distinguish acceptable and unacceptable abnormal sounds. Without importing the quantification mechanism, out of 1361 objects to be tested, 941 objects to be tested will be classified as "OK" objects to be tested (OK rate is 0.691), and 421 objects to be tested will be classified The category is "NG" the object to be tested (NG rate is 0.309). With the introduction of a quantization mechanism, additional abnormal sound classification will be performed based on the severity of the pre-established auditory standard. Among the 421 "NG" objects to be tested, 177 objects to be tested have acceptable abnormal sounds (the overall OK rate is 0.821), and 244 objects to be tested have unacceptable abnormal sounds (the overall NG rate is 0.179). Obviously, Table 1 reflects that the introduction of a quantitative mechanism can reduce the overall NG rate by 13%. In terms of product manufacturing and management, the cost of testing and retesting will be significantly reduced, and lower over-emissions will increase product yield. In certain applications, the quality of the NG object to be tested can be graded according to the severity of the abnormal sound, which can be used as a future product market plan. OK rate NG rate Defect detection with imported quantitative mechanism 0.691 0.309 Defect detection without introducing quantitative mechanism 0.821 0.179 Table 1

In summary, the defect detection method and system proposed by the present invention can use computer vision technology to detect whether the object under test has unacceptable defects in the pre-established auditory standard from the time-spectrogram. In this way, the present invention can not only provide more accurate defect detection than the subjective judgment of human ears, but also reduce related occupational injuries.

100, 900: Defect detection system 110, 910: signal output device 120, 920: Microphone 130, 930: Analog-to-digital converter 140, 940: processing device T: Object to be tested S202～S210, S1002～S1010: steps 310, 320, 710, 720, 810, 820, 1100: Time-spectrogram 315, 325: detection area RB: Abnormal sound characteristics 402～414, 602～614: Process H: image pyramid T1, T0: image T11, T01: sub-block R1, R0: Reference model R11, R01: Reference sub-block FE: Feature extraction M: classifier 1115: Region of Interest R1～R3: sub-time spectrum diagram CR1～CR3: Projection curve 1300: cluster 1301～13014: point

FIG. 1 is a block diagram of a defect detection system according to an embodiment of the invention. FIG. 2 is a flowchart of a method for visualizing defect detection according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a time-frequency spectrum diagram according to an embodiment of the present invention. FIG. 4 is a functional block flowchart of constructing a classifier according to an embodiment of the invention. FIG. 5 is a functional block flowchart of a method for obtaining spatial features according to an embodiment of the present invention. FIG. 6 is a functional block flowchart of a defect detection visualization method according to an embodiment of the present invention. FIG. 7 is a schematic diagram of a time-frequency spectrum diagram according to an embodiment of the present invention. FIG. 8 is a schematic diagram illustrating a time-spectrogram with acceptable abnormal sounds and a time-spectrogram with unacceptable abnormal sounds according to an embodiment of the present invention. FIG. 9 is a block diagram of a defect detection system according to an embodiment of the invention. FIG. 10 is a flowchart of a defect detection visualization method according to an embodiment of the present invention. FIG. 11 is a functional diagram of converting a time-spectrogram into a projection curve according to an embodiment of the present invention. FIG. 12 is a schematic diagram of dividing projection curves according to an embodiment of the present invention. FIG. 13 is a data diagram of the hierarchical detection of abnormal sounds according to an embodiment of the present invention.

S1002~S1010: steps

Claims (19)

A visual method for defect detection, including: Output test audio signal to the object under test; Receiving the response signal of the object under test to the test audio signal to generate a radio audio signal; Perform signal processing on the radio audio signal to generate a time-spectrogram; and According to the time-frequency spectrum, it is visually judged whether the object under test has unacceptable defects in the pre-established auditory standards. According to the method described in item 1 of the scope of patent application, the step of receiving the response signal of the object under test to the test audio signal to generate the radio audio signal includes: Use a microphone to receive the response signal; and The analog-digital conversion is performed on the response signal to generate the radio audio signal. For the method described in item 2 of the scope of patent application, the steps of performing signal processing on the radio audio signal to generate the time spectrogram include: Fast Fourier transform is performed on the radio audio signal to generate the time spectrogram. For the method described in item 1 of the scope of patent application, the step of visually judging whether the object under test has the unacceptable defect in the pre-established auditory standard according to the time-spectrogram includes: According to the time spectrum chart, visually judge whether the object under test has defects; and When the object to be tested has the defect, it is determined whether the defect is unacceptable in the pre-established auditory standard according to the time-spectrogram. For the method described in item 1 of the scope of patent application, the step of visually judging whether the object under test has the unacceptable defect in the pre-established auditory standard according to the time-spectrogram includes: Obtain multiple sub-time spectrograms associated with the time spectrogram; Respectively converting each of the sub-time spectrograms into projection curves; Obtain a plurality of sections associated with each of the projection curves; Generating a feature quantization result of each of the segments corresponding to each of the projection curves; and According to the feature quantification result and the classifier, it is determined whether the object to be tested has the unacceptable defect. For the method described in item 5 of the scope of patent application, the step of obtaining the sub-time spectrogram associated with the time spectrogram includes: Extract a region of interest from the spectrogram at that time, where the region of interest corresponds to a preset frequency range; and Divide the region of interest according to different frequency levels to generate the sub-time spectrogram. For the method described in item 5 of the scope of patent application, the step of converting each of the sub-time spectrograms into the projection curve respectively includes: Calculate the energy average value of each time in each sub-time spectrogram to generate the projection curve. The method according to item 5 of the scope of patent application, wherein the step of obtaining the segments associated with each of the projection curves includes: According to different time intervals, each of the projection curves is divided into the segments. The method according to item 5 of the scope of patent application, wherein the feature quantization result of each of the segments corresponding to each of the projection curves is related to the statistical parameters of the multiple data points of the corresponding segment and the corresponding The weight assigned to this sub-time spectrogram. For the method described in item 5 of the scope of patent application, the step of judging whether the object under test has the unacceptable defect according to the feature quantification result and the classifier includes: Input the feature quantification results corresponding to all the segments of all the projection curves to the classifier; Receive the output of the classifier; and According to the output result of the classifier, it is determined whether the object to be tested has the unacceptable defect. The method described in item 10 of the scope of patent application, wherein the classifier is a support vector machine classifier, and is constructed based on a plurality of defect training objects with acceptable defects in the predetermined auditory standard. For example, the method described in item 10 of the scope of patent application, wherein the step of judging whether the object under test has the unacceptable defect according to the output result of the classifier includes: Defect confidence level to obtain the output result; Determine whether the confidence level of the defect is greater than the preset confidence level of the defect; When the defect confidence level is greater than the preset defect confidence level, it is determined that the object under test has an acceptable defect; and When the defect confidence level is not greater than the preset defect confidence level, it is determined that the object to be tested has the unacceptable defect. The method described in item 1 of the scope of patent application, wherein the object to be tested is an electronic device with a speaker. The method described in item 1 of the scope of patent application, wherein the defect is an abnormal sound in the assembly of the object to be tested. A defect detection system includes: Signal output device for outputting test audio signals to the object under test; The microphone is used to receive the response signal of the object under test to the test audio signal; An analog-to-digital converter for converting the response signal into a radio audio signal; and The processing device is used for signal processing for the radio audio signal to generate a time-spectrogram, and based on the time-spectrogram, to visually determine whether the object under test has unacceptable defects in the pre-established auditory standard. For example, the system described in item 15 of the scope of patent application, wherein the processing device visually judges whether the object to be tested has a defect based on the time-spectrogram, and when the object to be tested has the defect, according to the time-spectrogram, Determine whether the defect is unacceptable in the pre-established auditory standard. For the system described in item 15 of the scope of patent application, the processing device further stores the classifier in advance, and the processing device obtains multiple sub-time spectrograms associated with the time spectrogram, and converts each of the sub-time spectrograms into Projection curve, obtain multiple segments associated with each of the projection curves, generate feature quantization results corresponding to each of the segments of each projection curve, and determine the pending feature based on the feature quantization results and the classifier Test whether the object has the unacceptable defect. The system described in item 16 of the scope of patent application, wherein the object to be tested is an electronic device with a speaker. The system described in item 16 of the scope of patent application, wherein the defect is an abnormal sound of the assembly of the object to be tested.

Images