TWI770817B - Defect detecting method, electronic device, and storage medium - Google Patents
Defect detecting method, electronic device, and storage medium Download PDFInfo
- Publication number
- TWI770817B TWI770817B TW110105186A TW110105186A TWI770817B TW I770817 B TWI770817 B TW I770817B TW 110105186 A TW110105186 A TW 110105186A TW 110105186 A TW110105186 A TW 110105186A TW I770817 B TWI770817 B TW I770817B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- data
- dimensionality reduction
- product image
- linear
- Prior art date
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
本發明涉及外觀檢測技術領域,尤其涉及一種瑕疵檢測方法、電子裝置及存儲介質。 The invention relates to the technical field of appearance detection, and in particular, to a defect detection method, an electronic device and a storage medium.
隨著科學技術的發展,基於深度學習的神經網路模型,例如卷積神經網路,廣泛應用於人工智慧領域,可以實現各種功能的自動化,例如調製資料、圖像資料等資料的自動化檢測及分類。在對圖像資料中的瑕疵信號進行檢測時,通常需要根據資料集訓練卷積神經網路,並對訓練過程中的最後一層卷積層進行特徵提取,然後進行後續的瑕疵信號檢測。然而,當原始圖像資料的相似性較高時,提取的特徵的相似性也會較高而不易分類,從而降低瑕疵信號的檢測準確度。 With the development of science and technology, neural network models based on deep learning, such as convolutional neural networks, are widely used in the field of artificial intelligence, which can realize the automation of various functions, such as automatic detection of modulation data, image data and other data. Classification. When detecting defect signals in image data, it is usually necessary to train a convolutional neural network according to the data set, and perform feature extraction on the last convolutional layer in the training process, and then perform subsequent defect signal detection. However, when the similarity of the original image data is high, the similarity of the extracted features is also high and it is difficult to classify, thereby reducing the detection accuracy of the defect signal.
有鑒於此,有必要提供一種瑕疵檢測方法、電子裝置及存儲介質,對於待檢測的圖像資料直接進行降維處理,並結合高斯混合模型進行瑕疵檢測,以提高檢測準確度。 In view of this, it is necessary to provide a defect detection method, an electronic device and a storage medium, which directly perform dimension reduction processing on the image data to be detected, and perform defect detection in combination with a Gaussian mixture model, so as to improve the detection accuracy.
本發明的第一方面提供一種瑕疵檢測方法,所述方法包括:將待檢測的複數產品圖像劃分為線性圖像或非線性圖像; 根據複數降維演算法對經過圖像劃分的所述產品圖像進行降維處理而得到複數降維資料;確定複數所述降維資料中的最佳降維資料;將所述最佳降維資料輸入高斯混合模型而得到所述產品圖像的評分數據;將所述評分數據與閾值進行比對,判斷所述評分數據是否小於所述閾值;及當所述評分數據小於所述閾值時,確定所述產品圖像中存在瑕疵。 A first aspect of the present invention provides a flaw detection method, the method comprising: dividing a complex product image to be detected into a linear image or a nonlinear image; Perform dimensionality reduction processing on the product image divided by the image according to the complex dimensionality reduction algorithm to obtain complex dimensionality reduction data; determine the best dimensionality reduction data in the complex dimensionality reduction data; The data is input into a Gaussian mixture model to obtain the scoring data of the product image; the scoring data is compared with a threshold to determine whether the scoring data is less than the threshold; and when the scoring data is less than the threshold, Defects are determined to be present in the product image.
優選地,所述方法還包括:當所述評分數據大於或等於所述閾值時,確定所述產品圖像中不存在瑕疵。 Preferably, the method further comprises: when the rating data is greater than or equal to the threshold, determining that there is no defect in the product image.
優選地,所述將待檢測的複數產品圖像劃分為線性圖像或非線性圖像包括:對待檢測的所述產品圖像進行歸一化處理;將所述產品圖像的圖元值輸入線性評分函數而得到線性圖像及非線性圖像的分類分值;判斷所述線性圖像的分類分值是否大於所述非線性圖像的分類分值;當判定所述線性圖像的分類分值大於所述非線性圖像的分類分值時,將所述產品圖像劃分為線性圖像;及當所述線性圖像的分類分值不大於所述非線性圖像的分類分值時,將所述產品圖像劃分為非線性圖像。 Preferably, the dividing the complex product image to be detected into a linear image or a nonlinear image includes: normalizing the product image to be detected; inputting the primitive value of the product image Linear scoring function to obtain the classification scores of the linear image and the nonlinear image; determine whether the classification score of the linear image is greater than the classification score of the nonlinear image; when determining the classification score of the linear image When the score is greater than the classification score of the nonlinear image, classify the product image as a linear image; and when the classification score of the linear image is not greater than the classification score of the nonlinear image , the product image is divided into non-linear images.
優選地,所述根據複數降維演算法對經過圖像劃分的所述產品圖像進行降維處理而得到複數降維資料包括:當所述產品圖像為線性圖像時,分別採用主成分分析演算法及隨機投影演算法對所述產品圖像進行降維;及當所述產品圖像為非線性圖像時,分別採用等度量映射演算法及t分佈隨機鄰域嵌入演算法對所述產品圖像進行降維。 Preferably, performing dimensionality reduction processing on the product image divided by the image according to the complex dimensionality reduction algorithm to obtain complex dimensionality reduction data includes: when the product image is a linear image, using principal components respectively The analysis algorithm and the random projection algorithm are used to reduce the dimension of the product image; and when the product image is a non-linear image, the equal-metric mapping algorithm and the t-distributed random neighborhood embedding algorithm are respectively used to perform dimensionality reduction on the product image. Dimensionality reduction of the product image.
優選地,所述確定複數所述降維資料中的最佳降維資料:計算每種降維演算法得到的複數降維資料之間的距離平均值;及確定距離平均值最大的複數降維資料為所述最佳降維資料。 Preferably, the determining the best dimensionality reduction data in the complex dimensionality reduction data: calculating the average distance between the complex dimensionality reduction data obtained by each dimensionality reduction algorithm; and determining the complex dimensionality reduction with the largest distance average The data are the best dimensionality reduction data described.
優選地,所述將所述最佳降維資料輸入高斯混合模型而得到所述產品圖像的評分數據包括:根據待檢測瑕疵的類型的數量確定所述高斯混合模型中高斯模型的數量;及根據所述最佳降維資料及EM演算法計算所述高斯混合模型的期望值及每次反覆運算的模型參數值;及根據每次反覆運算計算出的期望值及模型參數值推算出所述高斯混合模型的最佳參數作為所述評分數據。 Preferably, inputting the best dimensionality reduction data into a Gaussian mixture model to obtain the scoring data of the product image comprises: determining the number of Gaussian models in the Gaussian mixture model according to the number of types of defects to be detected; and Calculate the expected value of the Gaussian mixture model and the model parameter values of each iteration according to the optimal dimensionality reduction data and the EM algorithm; and calculate the Gaussian mixture according to the expected value and model parameter values calculated by each iteration The best parameters of the model are used as the scoring data.
優選地,所述閾值為每次反覆運算計算出的模型參數值的均值與三倍所述評分數據的標準差之間的差值。 Preferably, the threshold is the difference between the mean value of the model parameter values calculated by each repeated operation and three times the standard deviation of the scoring data.
優選地,所述方法還包括:在所述產品圖像上標示存在瑕疵的區域,並在顯示螢幕上顯示標示後的所述產品圖像。 Preferably, the method further comprises: marking the defective area on the product image, and displaying the marked product image on a display screen.
本發明的第二方面提供一種電子裝置,包括:處理器;以及記憶體,所述記憶體中存儲有複數程式模組,所述複數程式模組由所述處理器載入並執行上述的瑕疵檢測方法。 A second aspect of the present invention provides an electronic device, comprising: a processor; and a memory, wherein the memory stores a plurality of program modules, the plurality of program modules are loaded by the processor and execute the above-mentioned defects Detection method.
本發明的第三方面提供一種存儲介質,其上存儲有至少一條電腦指令,所述指令由處理器並載入執行上述瑕疵檢測方法。 A third aspect of the present invention provides a storage medium on which at least one computer instruction is stored, and the instruction is loaded by a processor to execute the above-mentioned defect detection method.
上述瑕疵檢測方法、電子裝置及存儲介質對於待檢測的圖像資料直接進行降維處理,並結合高斯混合模型進行瑕疵檢測,在資料處理過程中無需進行卷積特徵的提取,簡化了運算過程,避免丟失資訊,從而有效提高了瑕疵檢測的準確度。 The above flaw detection method, electronic device and storage medium directly perform dimensionality reduction processing on the image data to be detected, and combine the Gaussian mixture model to perform flaw detection, which does not need to extract convolution features during the data processing process, which simplifies the operation process. Avoid loss of information, thereby effectively improving the accuracy of flaw detection.
1:電子裝置 1: Electronic device
10:處理器 10: Processor
100:瑕疵檢測系統 100: Defect Detection System
101:獲取模組 101: Get Mods
102:劃分模組 102: Divide modules
103:降維模組 103: Dimensionality Reduction Module
104:確定模組 104: Determine the module
105:計算模組 105: Computing Modules
106:判斷模組 106: Judgment Module
107:顯示模組 107: Display Module
20:記憶體 20: Memory
30:電腦程式 30: Computer Programs
40:顯示螢幕 40: Display screen
S301~S309:步驟 S301~S309: Steps
圖1是本發明較佳實施方式提供的電子裝置的結構示意圖。 FIG. 1 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention.
圖2是本發明較佳實施方式提供的瑕疵檢測系統的結構示意圖。 FIG. 2 is a schematic structural diagram of a defect detection system provided by a preferred embodiment of the present invention.
圖3是本發明較佳實施方式提供的瑕疵檢測方法的流程圖。 FIG. 3 is a flowchart of a defect detection method provided by a preferred embodiment of the present invention.
為了能夠更清楚地理解本發明的上述目的、特徵和優點,下面結合附圖和具體實施例對本發明進行詳細描述。需要說明的是,在不衝突的情況下,本申請的實施例及實施例中的特徵可以相互組合。 In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.
在下面的描述中闡述了很多具體細節以便於充分理解本發明,所描述的實施例僅僅是本發明一部分實施例,而不是全部的實施例。基於本發明中的實施例,本領域普通技術人員在沒有做出創造性勞動前提下所獲得的所有其他實施例,都屬於本發明保護的範圍。 In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
除非另有定義,本文所使用的所有的技術和科學術語與屬於本發明的技術領域的技術人員通常理解的含義相同。本文中在本發明的說明書中所使用的術語只是為了描述具體的實施例的目的,不是旨在於限制本發明。 Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.
請參閱圖1所示,為本發明較佳實施方式提供的電子裝置的結構示意圖。 Please refer to FIG. 1 , which is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention.
本發明中的瑕疵檢測方法應用在電子裝置1中。所述電子裝置1可以為安裝有瑕疵檢測程式的電子設備,例如個人電腦、伺服器等,其中,所述伺服器可以是單一的伺服器、伺服器集群或雲端伺服器等。
The defect detection method in the present invention is applied in the
所述電子裝置1所處的網路包括但不限於互聯網、廣域網路、都會區網路、局域網、虛擬私人網路絡(Virtual Private Network,VPN)等。
The network where the
所述電子裝置1包括,但不僅限於,處理器10、記憶體20、存儲在所述記憶體20中並可在所述處理器10上運行的電腦程式30及顯示螢幕40。例如,所述電腦程式30為瑕疵檢測程式。所述處理器10執行所述電腦程式30時實現瑕疵檢測方法中的步驟,例如圖3所示的步驟S301~S309。或者,所述處理器10執行所述電腦程式30時實現瑕疵檢測系統中各模組/單元的功能,例如圖2中的模組101-107。
The
示例性的,所述電腦程式30可以被分割成一個或複數模組/單元,所述一個或者複數模組/單元被存儲在所述記憶體20中,並由所述處理器10執行,以完成本發明。所述一個或複數模組/單元可以是能夠完成特定功能的一系列電腦程式指令段,所述指令段用於描述所述電腦程式30在所述電子裝置1中的執行過程。例如,所述電腦程式30可以被分割成圖3中的獲取模組101、劃分模組102、降維模組103、確定模組104、計算模組105、判斷模組106及顯示模組107。各模組具體功能參見瑕疵檢測系統實施例中各模組的功能。
Exemplarily, the
本領域技術人員可以理解,所述示意圖僅僅是電子裝置1的示例,並不構成對電子裝置1的限定,可以包括比圖示更多或更少的部件,或者組合某些部件,或者不同的部件,例如所述電子裝置1還可以包括輸入輸出設備、網路接入設備、匯流排等。
Those skilled in the art can understand that the schematic diagram is only an example of the
所稱處理器10可以是中央處理單元(Central Processing Unit,CPU),還可以是其他通用處理器、數位訊號處理器(Digital Signal Processor,DSP)、專用積體電路(Application Specific Integrated Circuit,ASIC)、現成可程式設計閘陣列(Field-Programmable Gate Array,FPGA)或者其他可程式設計邏輯器件、分立門或者電晶體邏輯器件、分立硬體元件等。通用處理器可以是微處理器或者所述處理器10也可以是任何常規的處理器等,所述處理器10是所述電子裝置1的控制中心,利用各種介面和線路連接整個電子裝置1的各個部分。
The so-called
所述記憶體20可用於存儲所述電腦程式30和/或模組/單元,所述處理器10藉由運行或執行存儲在所述記憶體20內的電腦程式和/或模組/單元,以及調用存儲在記憶體20內的資料,實現所述電子裝置1的各種功能。所述記憶體20可主要包括存儲程式區和存儲資料區,其中,存儲程式區可存儲作業系統、至少一個功能所需的應用程式(比如聲音播放功能、圖像播放功能等)等;存儲資料區可存儲根據電子裝置1的使用所創建的資料(比如音訊資料、電話本等)等。此外,記憶體20可以包括易失性記憶體,還可以包括非易失性記憶體,例如硬碟、記憶體、插接式硬碟,智慧存儲卡(Smart Media Card,SMC),安全數位(Secure Digital,SD)卡,快閃記憶體卡(Flash Card)、至少一個磁碟記憶體件、快閃記憶體器件、或其他記憶體件。
The
請參閱圖2所示,為本發明較佳實施方式提供的瑕疵檢測系統的功能模組圖。 Please refer to FIG. 2 , which is a functional module diagram of a defect detection system provided by a preferred embodiment of the present invention.
在一些實施方式中,瑕疵檢測系統100運行於所述電子裝置1中。所述瑕疵檢測系統100可以包括複數由程式碼段所組成的功能模組。所述瑕疵檢測系統100中的各個程式段的程式碼可以存儲於電子裝置1的記憶體20中,並由所述至少一個處理器10所執行,以實現瑕疵檢測功能。
In some embodiments, the
本實施方式中,瑕疵檢測系統100根據其所執行的功能,可以被劃分為複數功能模組。參閱圖2所示,所述功能模組可以包括獲取模組101、劃分模組102、降維模組103、確定模組104、計算模組105、判斷模組106及顯示模組107。本發明所稱的模組是指一種能夠被至少一個處理器所執行並且能夠完成固定功能的一系列電腦程式段,其存儲在記憶體20中。可以理解的是,在其他實施例中,上述模組也可為固化於所述處理器10中的程式指令或固件(firmware)。
In this embodiment, the
所述獲取模組101用於當接收到瑕疵檢測請求時,根據所述瑕疵檢測請求獲取待檢測的產品圖像。
The obtaining
在本實施方式中,所述瑕疵檢測請求可以由用戶觸發(例如:藉由預設功能按鍵進行觸發),也可以在預設時間內自動觸發。 In this embodiment, the defect detection request can be triggered by a user (eg, triggered by a preset function button), or can be triggered automatically within a preset time.
在本實施方式中,所述瑕疵檢測請求至少包括檢測物件資訊。具體的,解析所述瑕疵檢測請求,以獲得所述瑕疵檢測請求中的檢測物件資訊,根據所述檢測物件資訊從待檢測圖像庫中獲取待檢測的所述產品圖像。其中,所述檢測物件資訊可以是產品名稱、產品型號等。 In this embodiment, the defect detection request includes at least detection object information. Specifically, the defect detection request is parsed to obtain the detection object information in the defect detection request, and the to-be-detected product image is acquired from the to-be-detected image library according to the detection object information. Wherein, the detection object information may be a product name, a product model, and the like.
所述劃分模組102用於將待檢測的複數產品圖像劃分為線性圖像或非線性圖像。
The
在本實施方式中,所述劃分模組102對待檢測的所述產品圖像進行歸一化處理,將所述產品圖像的圖元值輸入線性評分函數而得到線性圖像及非線性圖像的分類分值,判斷所述線性圖像的分類分值是否大於所述非線性圖像的分類分值,當判定所述線性圖像的分類分值大於所述非線性圖像的分類分值時,將所述產品圖像劃分為線性圖像,當所述線性圖像的分類分值不大於所述非線性圖像的分類分值時,將所述產品圖像劃分為非線性圖像。
In this embodiment, the
例如,所述線性評分函數為f(x i ,W,b)=Wx i +b。假設每個圖像資料都被拉長為一個長度為D的列向量,大小為[D x 1]。其中,大小為[KxD]的矩陣W和大小為[Kx1]列向量b為所述線性評分函數的參數,xi包含了第i個圖像的所有圖元資訊,這些圖元資訊被拉成為一個[px1]的列向量,W的大小為[qxp],b的大小為[qx1]。其中,p為產品圖像的圖元點個數,q為類別數量。在本實施方式中,q為2。因此,P個數位(原始圖元數值)輸入所述線性評分函數,所述線性評分函數輸出2個數位(不同分類得到的分值),然後將所述不同分類的分值進行比對。其中,參數W為權重,b被稱為偏差向量。 For example, the linear scoring function is f ( x i , W , b ) = Wx i + b . Suppose each image data is stretched into a column vector of length D of size [D x 1]. Among them, the matrix W of size [KxD] and the column vector b of size [Kx1] are the parameters of the linear scoring function, xi contains all the primitive information of the i-th image, and these primitive information are pulled into a A column vector of [px1] with W of size [qxp] and b of size [qx1]. Among them, p is the number of primitive points of the product image, and q is the number of categories. In this embodiment, q is 2. Therefore, P digits (original primitive values) are input into the linear scoring function, and the linear scoring function outputs 2 digits (scores obtained from different classifications), and then the scores of the different classifications are compared. Among them, the parameter W is the weight, and b is called the bias vector.
所述降維模組103用於根據複數降維演算法對經過圖像劃分的所述產品圖像進行降維處理而得到複數降維資料。
The
在本實施方式中,當所述產品圖像為線性圖像時,所述降維模組103分別採用主成分分析演算法(PCA)及隨機投影演算法(Random Project)對所述產品圖像進行降維。當所述產品圖像為非線性圖像時,所述降維模組103分別採用等度量映射演算法(Isomap)及t分佈隨機鄰域嵌入演算法(t-SNE)對所述產品圖像進行降維。
In this embodiment, when the product image is a linear image, the
具體的,採用主成分分析演算法進行降維時,所述降維模組103首先將樣本資料減去樣本平均值,然後藉由計算資料協方差矩陣計算資料的主成分,最後將資料藉由轉換矩陣映射到主成分上,從而實現產品圖像的降維。
Specifically, when using the principal component analysis algorithm for dimensionality reduction, the
採用隨機投影演算法進行降維時,所述降維模組103首先選擇映射矩陣RRK×N,用亂數填充所述映射矩陣,歸一化所述映射矩陣中的每一行,然後藉由y=RX對資料降維。其中,X為輸入資料,y降維後的資料。
When the random projection algorithm is used for dimension reduction, the
採用等度量映射演算法進行降維時,所述降維模組103確定樣本資料中,即產品圖像圖元點的k近鄰,將圖元點與k近鄰之間的距離設置為歐氏距離,與其他點的距離設置為無窮大,採用最短路徑演算法計算任一兩個圖元點之間的距離,將所述距離作為多維標度分析演算法的輸入資料,藉由所述多維標度分析演算法輸出產品圖像圖元點在低維空間的投影,從而獲得降維的圖像。
When using the isometric mapping algorithm for dimensionality reduction, the
採用t分佈隨機鄰域嵌入演算法(t-SNE)進行降維時,所述降維模組103藉由隨機鄰接嵌入(SNE)將樣本資料,即產品圖像的圖元點之間的高維歐幾裡得距離轉換為表示相似性的條件概率,對於高維資料點Xi和Xj的低維對應點,計算類似的條件概率,使用梯度下降法最小化KL距離,並定義困惑度,從而獲得對應的低維資料。
When the t-distributed stochastic neighbor embedding algorithm (t-SNE) is used for dimension reduction, the
所述確定模組104用於確定複數所述降維資料中的最佳降維資料。
The determining
在本實施方式中,所述確定模組104計算每種降維演算法得到的複數降維資料之間的距離平均值,確定距離平均值最大的複數降維資料為所述最佳降維資料。需要說明的是,降維資料之間的距離越大,說明降維效果越好。在本實施方式中,所述複數降維資料之間的距離為資料點在鄰接圖中的距離。
In this embodiment, the determining
所述計算模組105用於將所述最佳降維資料輸入高斯混合模型而得到所述產品圖像的評分數據。
The
在本實施方式中,所述計算模組105對所述最佳降維資料進行均值池化處理,得到目標向量,然後將所述目標向量輸入至預先訓練好的高斯混合模型中,得到所述評分數據。其中,所述高斯混合模型能夠利用高斯概率密度函數(正態分佈圖像曲線)精確地量化目標向量對應的評分。
In this embodiment, the
具體的,所述計算模組105根據待檢測瑕疵的類型的數量確定所述高斯混合模型中高斯模型的數量,根據所述最佳降維資料及EM(Expectation-Maximum)演算法計算所述高斯混合模型的期望值及每次反覆運算的模型參數值,及根據每次反覆運算計算出的期望值及模型參數值推算出所述高斯混合模型的最佳參數作為所述評分數據。在本實施方式中,所述高斯混合模型中高斯模型的數量為2。
Specifically, the
在本實施方式中,所述計算模組105可以將所述訓練樣本資料劃分為訓練集、測試集及驗證集,然後基於最大期望演算法,反覆運算訓練所述訓練集中的複數低維向量,得到學習器,然後利用所述測試集中的複數低維向量測試所述學習器,得到測試結果,當所述測試結果小於配置值時,所述計算模組105利用所述驗證集中的複數低維向量調整所述學習器中的參數,得到所述預先訓練好的高斯混合模型。在本實施方式中,所述計算模組105計算所述複數
低維向量的數量,當所述數量小於預設數量時,利用資料增強演算法增加所述複數低維向量的數量。
In this embodiment, the
所述判斷模組106用於將所述評分數據與閾值進行比對,判斷所述評分數據是否小於所述閾值。
The judging
在本實施方式中,所述閾值為每次反覆運算計算出的模型參數值的均值與三倍所述評分數據的標準差之間的差值。 In this embodiment, the threshold value is the difference between the mean value of the model parameter values calculated by each repeated operation and three times the standard deviation of the scoring data.
所述確定模組104還當所述評分數據小於所述閾值時,確定所述產品圖像中存在瑕疵。
The determining
所述確定模組104還當所述評分數據大於或等於所述閾值時,確定所述產品圖像中不存在瑕疵。
The determining
所述顯示模組107用於在所述產品圖像上標示存在瑕疵的區域,並在顯示螢幕上顯示標示後的所述產品圖像。
The
請參閱圖3所示,為本發明較佳實施方式提供的瑕疵檢測方法的流程圖。根據不同的需求,所述流程圖中步驟的順序可以改變,某些步驟可以省略。 Please refer to FIG. 3 , which is a flowchart of a defect detection method provided by a preferred embodiment of the present invention. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.
S301,當接收到瑕疵檢測請求時,根據所述瑕疵檢測請求獲取待檢測的產品圖像。 S301, when a defect detection request is received, acquire a product image to be detected according to the defect detection request.
在本實施方式中,所述瑕疵檢測請求可以由用戶觸發(例如:藉由預設功能按鍵進行觸發),也可以在預設時間內自動觸發。 In this embodiment, the defect detection request can be triggered by a user (eg, triggered by a preset function button), or can be triggered automatically within a preset time.
在本實施方式中,所述瑕疵檢測請求至少包括檢測物件資訊。具體的,解析所述瑕疵檢測請求,以獲得所述瑕疵檢測請求中的檢測物件資訊,根據所述檢測物件資訊從待檢測圖像庫中獲取待檢測的所述產品圖像。其中,所述檢測物件資訊可以是產品名稱、產品型號等。 In this embodiment, the defect detection request includes at least detection object information. Specifically, the defect detection request is parsed to obtain the detection object information in the defect detection request, and the to-be-detected product image is acquired from the to-be-detected image library according to the detection object information. Wherein, the detection object information may be a product name, a product model, and the like.
S302,將待檢測的複數產品圖像劃分為線性圖像或非線性圖像。 S302: Divide the complex product image to be detected into a linear image or a nonlinear image.
在本實施方式中,對待檢測的所述產品圖像進行歸一化處理,將所述產品圖像的圖元值輸入線性評分函數而得到線性圖像及非線性圖像的分類分值,判斷所述線性圖像的分類分值是否大於所述非線性圖像的分類分值,當判定所述線性圖像的分類分值大於所述非線性圖像的分類分值時,將所述產品圖像劃分為線性圖像,當所述線性圖像的分類分值不大於所述非線性圖像的分類分值時,將所述產品圖像劃分為非線性圖像。 In this embodiment, the product image to be detected is subjected to normalization processing, and the primitive values of the product image are input into a linear scoring function to obtain the classification scores of the linear image and the nonlinear image. Whether the classification score of the linear image is greater than the classification score of the nonlinear image, when it is determined that the classification score of the linear image is greater than the classification score of the nonlinear image, the product The image is divided into linear images, and when the classification score of the linear image is not greater than the classification score of the non-linear image, the product image is divided into non-linear images.
例如,所述線性評分函數為f(x i ,W,b)=Wx i +b。假設每個圖像資料都被拉長為一個長度為D的列向量,大小為[D x 1]。其中,大小為[KxD]的矩陣W和大小為[Kx1]列向量b為所述線性評分函數的參數,xi包含了第i個圖像的所有圖元資訊,這些圖元資訊被拉成為一個[px1]的列向量,W的大小為[qxp],b的大小為[qx1]。其中,p為產品圖像的圖元點個數,q為類別數量。在本實施方式中,q為2。因此,P個數位(原始圖元數值)輸入所述線性評分函數,所述線性評分函數輸出2個數位(不同分類得到的分值),然後將所述不同分類的分值進行比對。其中,參數W為權重,b被稱為偏差向量。 For example, the linear scoring function is f ( x i , W , b ) = Wx i + b . Suppose each image data is stretched into a column vector of length D of size [D x 1]. Among them, the matrix W of size [KxD] and the column vector b of size [Kx1] are the parameters of the linear scoring function, xi contains all the primitive information of the i-th image, and these primitive information are pulled into a A column vector of [px1] with W of size [qxp] and b of size [qx1]. Among them, p is the number of primitive points of the product image, and q is the number of categories. In this embodiment, q is 2. Therefore, P digits (original primitive values) are input into the linear scoring function, and the linear scoring function outputs 2 digits (scores obtained from different classifications), and then the scores of the different classifications are compared. Among them, the parameter W is the weight, and b is called the bias vector.
S303,根據複數降維演算法對經過圖像劃分的所述產品圖像進行降維處理而得到複數降維資料。 S303 , performing a dimensionality reduction process on the product image divided by the image according to a complex dimensionality reduction algorithm to obtain complex dimensionality reduction data.
具體的,採用主成分分析演算法進行降維時,首先將樣本資料減去樣本平均值,然後藉由計算資料協方差矩陣計算資料的主成分,最後將資料藉由轉換矩陣映射到主成分上,從而實現產品圖像的降維。 Specifically, when using the principal component analysis algorithm for dimensionality reduction, first subtract the sample mean from the sample data, then calculate the principal components of the data by calculating the data covariance matrix, and finally map the data to the principal components by using the transformation matrix. , so as to achieve dimensionality reduction of product images.
採用隨機投影演算法進行降維時,首先選擇映射矩陣RRK×N,用亂數填充所述映射矩陣,歸一化所述映射矩陣中的每一行,然後藉由y=RX對資料降維。其中,X為輸入資料,y降維後的資料。 When using the random projection algorithm for dimensionality reduction, first select the mapping matrix R RK×N, fill the mapping matrix with random numbers, normalize each row in the mapping matrix, and then reduce the dimension of the data by y=RX. Among them, X is the input data, and y is the data after dimension reduction.
採用等度量映射演算法進行降維時,確定樣本資料中,即產品圖像圖元點的k近鄰,將圖元點與k近鄰之間的距離設置為歐氏距離,與其他點的 距離設置為無窮大,採用最短路徑演算法計算任一兩個圖元點之間的距離,將所述距離作為多維標度分析演算法的輸入資料,藉由所述多維標度分析演算法輸出產品圖像圖元點在低維空間的投影,從而獲得降維的圖像。 When using the isometric mapping algorithm for dimensionality reduction, determine the k-nearest neighbors of the product image primitive points in the sample data, and set the distance between the primitive points and the k-nearest neighbors as the Euclidean distance. The distance is set to infinity, the shortest path algorithm is used to calculate the distance between any two primitive points, the distance is used as the input data of the multi-dimensional scaling analysis algorithm, and the product is output by the multi-dimensional scaling analysis algorithm Projection of image primitive points in a low-dimensional space to obtain a reduced-dimensional image.
採用t分佈隨機鄰域嵌入演算法(t-SNE)進行降維時,藉由隨機鄰接嵌入(SNE)將樣本資料,即產品圖像的圖元點之間的高維歐幾裡得距離轉換為表示相似性的條件概率,對於高維資料點Xi和Xj的低維對應點,計算類似的條件概率,使用梯度下降法最小化KL距離,並定義困惑度,從而獲得對應的低維資料。 When using the t-distributed stochastic neighbor embedding algorithm (t-SNE) for dimensionality reduction, the sample data, that is, the high-dimensional Euclidean distance between the primitive points of the product image, is converted by the stochastic neighbor embedding (SNE). In order to express the conditional probability of similarity, for the low-dimensional counterparts of high-dimensional data points Xi and Xj, the similar conditional probability is calculated, the gradient descent method is used to minimize the KL distance, and the perplexity is defined to obtain the corresponding low-dimensional data.
S304,確定複數所述降維資料中的最佳降維資料。 S304, determine the best dimensionality reduction data in the complex dimensionality reduction data.
在本實施方式中,計算每種降維演算法得到的複數降維資料之間的距離平均值,確定距離平均值最大的複數降維資料為所述最佳降維資料。 In this embodiment, the average distance between the complex dimension reduction data obtained by each dimension reduction algorithm is calculated, and the complex dimension reduction data with the largest distance average value is determined as the optimal dimension reduction data.
S305,將所述最佳降維資料輸入高斯混合模型而得到所述產品圖像的評分數據。 S305, inputting the best dimensionality reduction data into a Gaussian mixture model to obtain rating data of the product image.
在本實施方式中,對所述最佳降維資料進行均值池化處理,得到目標向量,然後將所述目標向量輸入至預先訓練好的高斯混合模型中,得到所述評分數據。其中,所述高斯混合模型能夠利用高斯概率密度函數(正態分佈圖像曲線)精確地量化目標向量對應的評分。 In this embodiment, mean pooling is performed on the optimal dimensionality reduction data to obtain a target vector, and then the target vector is input into a pre-trained Gaussian mixture model to obtain the scoring data. Wherein, the Gaussian mixture model can accurately quantify the score corresponding to the target vector by using a Gaussian probability density function (normal distribution image curve).
具體的,根據待檢測瑕疵的類型的數量確定所述高斯混合模型中高斯模型的數量,根據所述最佳降維資料及EM(Expectation-Maximum)演算法計算所述高斯混合模型的期望值及每次反覆運算的模型參數值,及根據每次反覆運算計算出的期望值及模型參數值推算出所述高斯混合模型的最佳參數作為所述評分數據。 Specifically, the number of Gaussian models in the Gaussian mixture model is determined according to the number of types of defects to be detected, and the expected value and each Gaussian mixture model are calculated according to the optimal dimensionality reduction data and the EM (Expectation-Maximum) algorithm. The model parameter value of the repeated operation, and the optimal parameter of the Gaussian mixture model is calculated according to the expected value and the model parameter value calculated in each repeated operation as the scoring data.
在本實施方式中,可以將所述訓練樣本資料劃分為訓練集、測試集及驗證集,然後基於最大期望演算法,反覆運算訓練所述訓練集中的複數低 維向量,得到學習器,然後利用所述測試集中的複數低維向量測試所述學習器,得到測試結果,當所述測試結果小於配置值時,利用所述驗證集中的複數低維向量調整所述學習器中的參數,得到所述預先訓練好的高斯混合模型。在本實施方式中,計算所述複數低維向量的數量,當所述數量小於預設數量時,利用資料增強演算法增加所述複數低維向量的數量。 In this embodiment, the training sample data can be divided into a training set, a test set and a validation set, and then based on the maximum expectation algorithm, repeated operations are performed to train the complex numbers in the training set to be low dimensional vector to obtain the learner, then use the complex low-dimensional vector in the test set to test the learner to obtain the test result, when the test result is less than the configuration value, use the complex low-dimensional vector in the verification set to adjust the parameters in the learner to obtain the pre-trained Gaussian mixture model. In this embodiment, the number of the complex low-dimensional vectors is calculated, and when the number is less than a preset number, the data augmentation algorithm is used to increase the number of the complex low-dimensional vectors.
S306,將所述評分數據與閾值進行比對,判斷所述評分數據是否小於所述閾值。 S306, compare the scoring data with a threshold, and determine whether the scoring data is less than the threshold.
在本實施方式中,所述閾值為每次反覆運算計算出的模型參數值的均值與三倍所述評分數據的標準差之間的差值。當所述評分數據小於所述閾值時,所述流程進入S307。當所述評分數據大於或等於所述閾值時,所述流程進入S309。 In this embodiment, the threshold value is the difference between the mean value of the model parameter values calculated by each repeated operation and three times the standard deviation of the scoring data. When the score data is less than the threshold, the process goes to S307. When the score data is greater than or equal to the threshold, the process proceeds to S309.
S307,確定所述產品圖像中存在瑕疵。 S307, it is determined that there is a defect in the product image.
S308,在所述產品圖像上標示存在瑕疵的區域,並在顯示螢幕上顯示標示後的所述產品圖像。 S308 , marking the defective area on the product image, and displaying the marked product image on the display screen.
S309,確定所述產品圖像中不存在瑕疵。 S309, it is determined that there is no defect in the product image.
所述電子裝置1集成的模組/單元如果以軟體功能單元的形式實現並作為獨立的產品銷售或使用時,可以存儲在一個電腦可讀取存儲介質中。基於這樣的理解,本發明實現上述實施例方法中的全部或部分流程,也可以藉由電腦程式來指令相關的硬體來完成,所述的電腦程式可存儲於一電腦可讀存儲介質中,所述電腦程式在被處理器執行時,可實現上述各個方法實施例的步驟。其中,所述電腦程式包括電腦程式代碼,所述電腦程式代碼可以為原始程式碼形式、物件代碼形式、可執行檔或某些中間形式等。所述電腦可讀介質可以包括:能夠攜帶所述電腦程式代碼的任何實體或裝置、記錄介質、隨身碟、
移動硬碟、磁碟、光碟、電腦記憶體、唯讀記憶體(ROM,Read-Only Memory)、隨機存取記憶體(RAM,Random Access Memory)等。
If the modules/units integrated in the
本發明所提供的瑕疵檢測方法、電子裝置及存儲介質對於待檢測的圖像資料直接進行降維處理,並結合高斯混合模型進行瑕疵檢測,在資料處理過程中無需進行卷積特徵的提取,簡化了運算過程,避免丟失資訊,從而有效提高了瑕疵檢測的準確度。 The defect detection method, electronic device and storage medium provided by the present invention directly perform dimensionality reduction processing on the image data to be detected, and combine the Gaussian mixture model for defect detection, which does not need to extract convolution features in the data processing process, simplifying the process The operation process is improved to avoid loss of information, thereby effectively improving the accuracy of defect detection.
對於本領域技術人員而言,顯然本發明不限於上述示範性實施例的細節,而且在不背離本發明的精神或基本特徵的情況下,能夠以其他的具體形式實現本發明。因此,無論從哪一點來看,均應將實施例看作是示範性的,而且是非限制性的,本發明的範圍由所附申請專利範圍而不是上述說明限定,因此旨在將落在申請專利範圍的等同要件的含義和範圍內的所有變化涵括在本發明內。不應將申請專利範圍中的任何附圖標記視為限制所涉及的申請專利範圍。此外,顯然“包括”一詞不排除其他單元或步驟,單數不排除複數。裝置申請專利範圍中陳述的複數單元或裝置也可以由同一個單元或裝置藉由軟體或者硬體來實現。第一,第二等詞語用來表示名稱,而並不表示任何特定的順序。 It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and not restrictive, and the scope of the present invention is defined by the appended claims rather than the foregoing description, and is therefore intended to fall within the scope of the application. All changes within the meaning and scope of equivalents to the scope of the patent are included in the present invention. Any reference signs in the patentable scope should not be construed as limiting the claimed scope. Furthermore, it is clear that the word "comprising" does not exclude other units or steps and the singular does not exclude the plural. Plural units or devices stated in the scope of the device application can also be realized by software or hardware by the same unit or device. The terms first, second, etc. are used to denote names and do not denote any particular order.
綜上所述,本發明符合發明專利要件,爰依法提出專利申請。惟,以上所述者僅為本發明之較佳實施方式,舉凡熟悉本案技藝之人士,於爰依本發明精神所作之等效修飾或變化,皆應涵蓋於以下之申請專利範圍內。 To sum up, the present invention complies with the requirements of an invention patent, and a patent application can be filed in accordance with the law. However, the above descriptions are only the preferred embodiments of the present invention, and for those who are familiar with the art of the present invention, equivalent modifications or changes made in accordance with the spirit of the present invention should all be covered within the scope of the following patent application.
S301~S309:步驟 S301~S309: Steps
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW110105186A TWI770817B (en) | 2021-02-09 | 2021-02-09 | Defect detecting method, electronic device, and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW110105186A TWI770817B (en) | 2021-02-09 | 2021-02-09 | Defect detecting method, electronic device, and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI770817B true TWI770817B (en) | 2022-07-11 |
TW202232383A TW202232383A (en) | 2022-08-16 |
Family
ID=83439295
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW110105186A TWI770817B (en) | 2021-02-09 | 2021-02-09 | Defect detecting method, electronic device, and storage medium |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI770817B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104867144A (en) * | 2015-05-15 | 2015-08-26 | 广东工业大学 | IC element solder joint defect detection method based on Gaussian mixture model |
CN109978824A (en) * | 2019-02-19 | 2019-07-05 | 深圳大学 | A kind of transparent membrane defect method for measuring shape of palaemon and system |
US20190244337A1 (en) * | 2018-02-05 | 2019-08-08 | Nec Laboratories America, Inc. | Unsupervised image-based anomaly detection using multi-scale context-dependent deep autoencoding gaussian mixture model |
TW202038110A (en) * | 2018-12-20 | 2020-10-16 | 以色列商應用材料以色列公司 | Classifying defects in a semiconductor specimen |
-
2021
- 2021-02-09 TW TW110105186A patent/TWI770817B/en active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104867144A (en) * | 2015-05-15 | 2015-08-26 | 广东工业大学 | IC element solder joint defect detection method based on Gaussian mixture model |
US20190244337A1 (en) * | 2018-02-05 | 2019-08-08 | Nec Laboratories America, Inc. | Unsupervised image-based anomaly detection using multi-scale context-dependent deep autoencoding gaussian mixture model |
TW202038110A (en) * | 2018-12-20 | 2020-10-16 | 以色列商應用材料以色列公司 | Classifying defects in a semiconductor specimen |
CN109978824A (en) * | 2019-02-19 | 2019-07-05 | 深圳大学 | A kind of transparent membrane defect method for measuring shape of palaemon and system |
Also Published As
Publication number | Publication date |
---|---|
TW202232383A (en) | 2022-08-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10977523B2 (en) | Methods and apparatuses for identifying object category, and electronic devices | |
CN108038474B (en) | Face detection method, convolutional neural network parameter training method, device and medium | |
CN107944020B (en) | Face image searching method and device, computer device and storage medium | |
CN110020592B (en) | Object detection model training method, device, computer equipment and storage medium | |
US11055571B2 (en) | Information processing device, recording medium recording information processing program, and information processing method | |
US20190266434A1 (en) | Method and device for extracting information from pie chart | |
WO2020258502A1 (en) | Text analysis method and apparatus, computer apparatus and computer storage medium | |
WO2019051941A1 (en) | Method, apparatus and device for identifying vehicle type, and computer-readable storage medium | |
CN105144239A (en) | Image processing device, program, and image processing method | |
CN112418212B (en) | YOLOv3 algorithm based on EIoU improvement | |
US20200125836A1 (en) | Training Method for Descreening System, Descreening Method, Device, Apparatus and Medium | |
CN114943674A (en) | Defect detection method, electronic device and storage medium | |
CN108959474B (en) | Entity relation extraction method | |
CN113269257A (en) | Image classification method and device, terminal equipment and storage medium | |
TWI803243B (en) | Method for expanding images, computer device and storage medium | |
CN111210402A (en) | Face image quality scoring method and device, computer equipment and storage medium | |
CN111444807A (en) | Target detection method, device, electronic equipment and computer readable medium | |
JP6989450B2 (en) | Image analysis device, image analysis method and program | |
CN113269752A (en) | Image detection method, device terminal equipment and storage medium | |
TWI762193B (en) | Image defect detection method, image defect detection device, electronic device and storage media | |
TWI770817B (en) | Defect detecting method, electronic device, and storage medium | |
CN111967383A (en) | Age estimation method, and training method and device of age estimation model | |
TWI795729B (en) | Image defect detection method, image defect detection device, electronic device and storage media | |
TWI755176B (en) | Method and device for calculating cell distribution density, electronic device, and storage unit | |
WO2023025063A1 (en) | Image signal processor optimization method and device |