TWI842292B - Appearance defect inspection method and system for automated production line products - Google Patents

Appearance defect inspection method and system for automated production line products Download PDF

Info

Publication number
TWI842292B
TWI842292B TW111149651A TW111149651A TWI842292B TW I842292 B TWI842292 B TW I842292B TW 111149651 A TW111149651 A TW 111149651A TW 111149651 A TW111149651 A TW 111149651A TW I842292 B TWI842292 B TW I842292B
Authority
TW
Taiwan
Prior art keywords
image file
image
learning
files
image files
Prior art date
Application number
TW111149651A
Other languages
Chinese (zh)
Other versions
TW202426898A (en
Inventor
黃靖瑋
許仁覺
林奕安
Original Assignee
偲倢科技股份有限公司
Filing date
Publication date
Application filed by 偲倢科技股份有限公司 filed Critical 偲倢科技股份有限公司
Application granted granted Critical
Publication of TWI842292B publication Critical patent/TWI842292B/en
Publication of TW202426898A publication Critical patent/TW202426898A/en

Links

Abstract

An appearance defect inspection method and an appearance defect inspection system for automated production line products are disclosed. The system includes a test fixture, an image capture device, an automatic learning module and a light source. With learning image files obtained by the image capture device learned by the automatic learning module and these learning image files having been normalized, the system can more accurately predict the defects of the products, and then eliminate the defective products. Model training provided by the present invention is accurate, and also reduces the misjudgment rate when selecting defective products online.

Description

自動化產線產品外觀瑕疵檢驗方法及系統 Automated production line product appearance defect inspection method and system

本發明關於一種外觀瑕疵檢驗方法及系統,特別是一種自動化產線產品的外觀瑕疵檢驗方法及系統。 The present invention relates to a method and system for inspecting appearance defects, in particular, a method and system for inspecting appearance defects of automated production line products.

隨著工業自動化設備的普及應用,許多產品的製作程序得以一致化,製造產能也能大幅提升。可想而知,對這些大數量的產品進行品檢是一個不輕鬆的任務,尤其是當這些產品的體積輕巧,瑕疵不容易分辨時。因此,僅依靠人(眼)力來執行品檢工作的速率,已經跟不上產能,需要額外的輔助設備來解決這問題。 With the popularization of industrial automation equipment, the production process of many products has been standardized, and the manufacturing capacity can be greatly improved. It is conceivable that quality inspection of these large quantities of products is not an easy task, especially when these products are small and the defects are not easy to distinguish. Therefore, the speed of quality inspection work relying solely on human (eye) power can no longer keep up with the production capacity, and additional auxiliary equipment is needed to solve this problem.

依照不同產品的特性,輔助品檢設備有著不同的特性。比如要檢測大型產品,如預鑄水管的內部裂損情形,可以使用超聲波技術;要檢測飲料類食品的成分,可以藉由化學成分分析;如果要檢測外觀可見的缺失,則可以透過產品外觀圖像的分析或比對來進行。對於自動化產線上體積小且瑕疵不明顯的產品外觀檢測,除了前述的圖像分析或比對外,還需要佐以自動化影像處理。 Auxiliary inspection equipment has different characteristics according to the characteristics of different products. For example, if you want to inspect large products, such as the internal cracks of precast water pipes, you can use ultrasonic technology; if you want to inspect the ingredients of beverages, you can use chemical composition analysis; if you want to detect visible defects, you can do it through product appearance image analysis or comparison. For the appearance inspection of small products with inconspicuous defects on the automated production line, in addition to the aforementioned image analysis or comparison, automated image processing is also required.

在一些外觀瑕疵檢驗的先前技術中,有一些可以提供解決方案的參考。例如中華民國發明專利第I479145號提出一種外觀瑕疵檢測系統及方法,該系統用於獲取檢測裝置的攝像單元攝取的待測樣本的當前影像,該 當前影像包括該待測樣本的側邊影像及投射影像;對待測樣本的側邊影像進行瑕疵檢測作業,並根據待測樣本的投射影像確定待測樣本的轉動角度;當待測樣本的側邊影像中檢測出瑕疵時,根據該待測樣本的轉動角度獲取對應的側邊影像範本;將待測樣本的側邊影像與該獲取的側邊影像範本進行比對,確定待測樣本中出現瑕疵的側邊區塊。此專利屬於利用圖片對比找出產品的外觀瑕疵,然而對外觀瑕疵的定義僅限於既有資料,在資料儲備不完整的情況下,比對結果會有漏網之魚。另外,中華民國發明專利第I667575號提出一種瑕疵檢測系統,連接於自動外觀檢測裝置,包括下列裝置:複檢伺服站接收瑕疵影像與瑕疵位置;訓練終端儲存已訓練模組;分類終端接收瑕疵影像與瑕疵位置,讀取已訓練模組中相應瑕疵影像的目標訓練模組,依據目標訓練模組分類瑕疵影像以產生並傳送標記瑕疵影像至複檢伺服站;複檢終端接收標記瑕疵影像,接收並傳送相應標記瑕疵影像的驗證操作至複檢伺服站;及標記複檢終端接收驗證操作與標記瑕疵影像,並接收相應標記瑕疵影像的標記結果。複檢伺服站傳送標記結果與標記瑕疵影像至訓練終端,以使訓練終端據此訓練對應的訓練模組。前述專利利用人工智能學習瑕疵品的外觀瑕疵,進而找出缺陷處並挑出瑕疵品。然而,該專利的影像使用範圍過大,且影像沒有經過正規化,除了造成人工智能在學習階段的不精確,在上線挑選瑕疵品時也會導致錯判率的增加。 In some prior art of appearance defect inspection, there are some references that can provide solutions. For example, the Republic of China Patent No. I479145 proposes an appearance defect detection system and method, which is used to obtain a current image of a sample to be tested taken by a camera unit of a detection device, the current image including a side image and a projection image of the sample to be tested; perform defect detection on the side image of the sample to be tested, and determine the rotation angle of the sample to be tested according to the projection image of the sample to be tested; when a defect is detected in the side image of the sample to be tested, obtain a corresponding side image template according to the rotation angle of the sample to be tested; compare the side image of the sample to be tested with the obtained side image template to determine the side block where the defect appears in the sample to be tested. This patent is about using image comparison to find out the appearance defects of products. However, the definition of appearance defects is limited to existing data. If the data storage is incomplete, the comparison results may miss some defects. In addition, the Republic of China's invention patent No. I667575 proposes a defect detection system connected to an automatic appearance inspection device, including the following devices: a re-inspection server station receives defect images and defect positions; a training terminal stores trained modules; a classification terminal receives defect images and defect positions, reads a target training module corresponding to the defect image in the trained module, and classifies the defect image according to the target training module to generate and transmit a marked defect image to the re-inspection server station; the re-inspection terminal receives the marked defect image, receives and transmits the verification operation of the corresponding marked defect image to the re-inspection server station; and the marked re-inspection terminal receives the verification operation and the marked defect image, and receives the marking result of the corresponding marked defect image. The re-inspection server station transmits the marking results and the marked defect image to the training terminal so that the training terminal can train the corresponding training module accordingly. The aforementioned patent uses artificial intelligence to learn the appearance defects of defective products, and then finds the defects and picks out the defective products. However, the image usage range of the patent is too large, and the image has not been normalized. In addition to causing inaccuracy in the learning stage of artificial intelligence, it will also lead to an increase in the error rate when selecting defective products online.

為了解決習知技術中存在的問題,因此有本發明之濫觴。 In order to solve the problems existing in the prior art, the present invention is proposed.

本段文字提取和編譯本發明的某些特點。其它特點將被揭露於後續段落中。其目的在涵蓋附加的申請專利範圍之精神和範圍中,各式的修改和類似的排列。 This paragraph extracts and compiles certain features of the invention. Other features will be revealed in subsequent paragraphs. Its purpose is to cover various modifications and similar arrangements within the spirit and scope of the attached patent application.

為了解決習知技術中存在的問題,本發明揭露一種自動化產線產品外觀瑕疵檢驗方法,包含步驟:於一資料準備階段中,執行:a)以定距離拍攝一產品的複數個瑕疵品的一定方向外觀,以獲得複數個學習影像檔;b)製作關於每一學習影像檔的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯(Sobel)影像圖檔及一邊緣效果(Edge)影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同;及c)將步驟b)中的各影像圖檔串接為一訓練圖檔;於一模型訓練階段中,執行:d)以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類;以及於一品管階段中,執行:e)取得同步驟a)之定距離拍攝的該產品的一受檢品的該定方向外觀之一確認影像檔;f)製作關於該確認影像檔的該紅原色影像圖檔、該藍原色影像圖檔、該綠原色影像圖檔、該索伯影像圖檔及該邊緣效果影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同;g)將步驟f)中的各影像圖檔串接為一確認圖檔;及h)將該確認圖檔輸入該估測模型中以獲得對應的瑕疵種類。 In order to solve the problems existing in the prior art, the present invention discloses an automated production line product appearance defect inspection method, comprising the steps of: in a data preparation stage, executing: a) photographing the appearance of a plurality of defective products of a product in a certain direction at a fixed distance to obtain a plurality of learning image files; b) generating a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file and an edge effect (Edge) image file for each learning image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same; and c) concatenating the image files in step b) into a training image file; in a model training stage, executing: d) using all the image files to obtain a training image file; The training image and the corresponding defect type are used to generate an estimation model through a convolution neural network to estimate the defect types of other image files; and in the first quality control stage, the following steps are performed: e) obtaining a confirmation image file of the appearance of a product under inspection of the product in the fixed direction photographed at a fixed distance in step a); f) generating the red primary color image file, the blue primary color image file, the green primary color image file, the Sobel image file and the edge effect image file of the confirmation image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same; g) concatenating the image files in step f) into a confirmation image file; and h) inputting the confirmation image file into the estimation model to obtain the corresponding defect type.

所述之自動化產線產品外觀瑕疵檢驗方法中,步驟b)中可進一步為每一學習影像檔製作一二值化(Threshold)影像圖檔,步驟f)中進一步製作關於該確認影像檔的該二值化影像圖檔。 In the automated production line product appearance defect inspection method, in step b), a binarized (Threshold) image file can be further produced for each learning image file, and in step f), the binarized image file related to the confirmation image file can be further produced.

所述之自動化產線產品外觀瑕疵檢驗方法中,步驟b)中還可進一步為每一學習影像檔製作一高通濾波器(High Pass Filter)處理影像圖檔,步驟f)中進一步製作關於該確認影像檔的該高通濾波器處理影像圖檔。 In the automated production line product appearance defect inspection method, in step b), a high-pass filter (High Pass Filter) processed image file can be further produced for each learning image file, and in step f), the high-pass filter processed image file for the confirmation image file can be further produced.

最好,取得該些學習影像檔的背景光亮度及色溫的差異不超過1%。 Preferably, the differences in background brightness and color temperature of the learning image files should not exceed 1%.

最好,任二學習影像檔在平面任一方向上的像素數量的差異比例小於0.5%。 Preferably, the difference in the number of pixels between any two learning image files in any direction on the plane is less than 0.5%.

本發明還揭露一種自動化產線產品外觀瑕疵檢驗系統,包含:一檢驗治具,系統地排放一產品的複數個瑕疵品或受檢品;一影像擷取裝置,與該檢驗治具上的每一瑕疵品或受檢品保持實質相近的距離,以依序拍攝該些瑕疵品的一定方向外觀以獲得複數個學習影像檔,及依序拍攝該些受檢品的該定方向外觀以獲得複數個確認影像檔;及一自動學習模組,安裝於一工作主機中,包含以下子模組:一資料前處理子模組,接收來自該影像擷取裝置的該些學習影像檔與該些確認影像檔,並去除無關該產品的背景影像;一影像產生子模組,執行以下作業:為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯(Sobel)影像圖檔及一邊緣效果影像圖檔,各影像圖檔的像素總數及像素排列方式相同;將來源自同一學習影像檔之製作的影像圖檔的串接為一訓練圖檔;及將來源自同一確認影像檔之製作的影像圖檔的串接為一確認圖檔;一模型學習子模組,以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類;及一品質判斷子模組,將該些確認圖檔輸入該估測模型中以獲得對應的瑕疵種類。 The present invention also discloses an automated production line product appearance defect inspection system, comprising: an inspection jig, which systematically arranges a plurality of defective products or inspected products of a product; an image capture device, which maintains a substantially similar distance from each defective product or inspected product on the inspection jig, to sequentially photograph the appearance of the defective products in a certain direction to obtain a plurality of learning image files, and to sequentially photograph the appearance of the inspected products in the certain direction to obtain a plurality of confirmation image files; and an automatic learning module, which is installed in a working host and comprises the following submodules: a data pre-processing submodule, which receives the learning image files and the confirmation image files from the image capture device and removes background images irrelevant to the product; and an image generation submodule, which performs the following operations: generating a plurality of learning image files for the learning images; The file and each of the confirmation image files are used to generate a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file and an edge effect image file, and the total number of pixels and pixel arrangement of each image file are the same; the image files generated from the same learning image file are concatenated into a training image file; and the image files generated from the same confirmation image file are concatenated into a confirmation image file; a model learning submodule, using all the training images and the corresponding defect types, generates an estimation model through a convolutional neural network to estimate the defect types of other image files; and a quality judgment submodule, inputs the confirmation images into the estimation model to obtain the corresponding defect types.

依照本發明,該影像產生子模組可進一步為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的二值化影像圖檔。 According to the present invention, the image generation submodule can further generate a binary image file associated with each of the learning image files and the confirmation image files.

依照本發明,該影像產生子模組還可進一步為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的高通濾波器處理影像圖檔。 According to the present invention, the image generation submodule can further generate a high-pass filter processed image file associated with each of the learning image files and the confirmation image files.

所述之自動化產線產品外觀瑕疵檢驗系統可進一步包含一光源,發光時維持該影像擷取裝置所在封閉空間中的背景光亮度及色溫的差異不超過1%。 The automated production line product appearance defect inspection system may further include a light source that maintains the difference in background light brightness and color temperature in the closed space where the image capture device is located within 1% when emitting light.

最好,任二學習影像檔在平面任一方向上的像素數量的差異比例小於0.5%。 Preferably, the difference in the number of pixels between any two learning image files in any direction on the plane is less than 0.5%.

透過自動學習模組學習影像擷取裝置取得的學習影像檔,且該些學習影像檔已正規化,本系統可以較精準地預測產品的瑕疵,進而剔除之。本發明的模型訓練精確,在上線挑選瑕疵品時也會減少錯判率。 By using the automatic learning module to learn the learning image files obtained by the image capture device, and these learning image files have been normalized, the system can more accurately predict product defects and then eliminate them. The model training of the present invention is accurate, and the error rate will also be reduced when selecting defective products online.

10:檢驗治具 10: Inspection fixture

10’:檢驗治具 10’: Inspection fixture

20:影像擷取裝置 20: Image capture device

20’:第一影像擷取裝置 20’: First image capture device

20”:第二影像擷取裝置 20”: Second image capture device

30:自動學習模組 30: Automatic learning module

31:資料前處理子模組 31: Data pre-processing submodule

32:影像產生子模組 32: Image generation submodule

33:模型學習子模組 33: Model learning submodule

34:品質判斷子模組 34: Quality judgment submodule

40:光源 40: Light source

A:封閉空間 A: Closed space

B:藍原色影像圖檔 B: Blue primary color image file

E:邊緣效果影像圖檔 E: Edge effect image file

G:綠原色影像圖檔 G: Green primary color image file

H:高通濾波器處理影像圖檔 H: High-pass filter processing image file

L:學習影像檔 L: Learning video file

P:產品 P:Products

R:紅原色影像圖檔 R: Red primary color image file

S:索伯影像圖檔 S: Sauber image file

S01~S08:步驟 S01~S08: Steps

圖1為本發明實施例的一種自動化產線產品外觀瑕疵檢驗方法的流程圖。 Figure 1 is a flow chart of an automated production line product appearance defect inspection method according to an embodiment of the present invention.

圖2顯示由一學習影像檔製作的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯影像圖檔及一邊緣效果影像圖檔。 FIG2 shows a red primary color image file, a blue primary color image file, a green primary color image file, a Sauber image file, and an edge effect image file made from a learning image file.

圖3顯示由該學習影像檔製作的該紅原色影像圖檔、該藍原色影像圖檔、該綠原色影像圖檔、該索伯影像圖檔、該邊緣效果影像圖檔、一二值化影像圖檔與一高通濾波器處理影像圖檔。 FIG3 shows the red primary color image file, the blue primary color image file, the green primary color image file, the Sobel image file, the edge effect image file, a binarized image file, and a high-pass filter processed image file made from the learning image file.

圖4為本發明實施例的一種自動化產線產品外觀瑕疵檢驗系統的元件示意圖。 Figure 4 is a schematic diagram of the components of an automated production line product appearance defect inspection system according to an embodiment of the present invention.

圖5為該自動化產線產品外觀瑕疵檢驗系統的影像擷取裝置與檢驗治具之另一種態樣。 Figure 5 shows another view of the image capture device and inspection fixture of the automated production line product appearance defect inspection system.

圖6繪示一自動學習模組的架構。 Figure 6 shows the architecture of an automatic learning module.

本發明將藉由參照下列的實施方式而更具體地描述。 The present invention will be described in more detail with reference to the following embodiments.

本發明揭露一種自動化產線產品外觀瑕疵檢驗方法(以下簡稱本發明)。本發明分為三個實施階段:一資料準備階段、一模型訓練階段與一品管階段。以下分別針對不同階段的各個具體步驟,依序說明。 The present invention discloses an automated production line product appearance defect inspection method (hereinafter referred to as the present invention). The present invention is divided into three implementation stages: a data preparation stage, a model training stage, and a quality control stage. The following describes the specific steps of different stages in order.

在資料準備階段中,本方法執行的第一個步驟為以定距離拍攝一產品的複數個瑕疵品的一定方向外觀,以獲得複數個學習影像檔(S01)。瑕疵品無論以怎樣的方式排列,比如直線、環狀排列,都要讓每個瑕疵品的同一面(定方向)面向拍攝影像的影像擷取裝置,且每一個瑕疵品的該面與影像擷取裝置的距離盡量固定一致,以確保影像擷取裝置擷取的每個學習影像檔中,瑕疵品的大小及所佔據像素範圍差異性小。依照本發的要求明,任二學習影像檔在平面任一方向上(垂直與水平方向)的像素數量的差異比例小於0.5%。舉例來說瑕疵品A在其學習影像檔中,水平方向最長處佔了4,000個像素,瑕疵品B在其學習影像檔中,對應處所佔的像素只能介於3,980~4,020個。實作上,可將所有瑕疵品在該處所佔的像素數量進行平均,除去超過平均值0.5%以上的瑕疵品學習影像檔。在垂直方向上也做同樣的處理。理論上來說,自動化產線產品有一致性的外觀,在前述的影像擷取條件下,會有一致的影像位置與像素數量。但這理論值會因為操作人員的失誤、治具上安裝的差異、溫度濕度影響光線,甚至是影像擷取裝置本身的問題,每個學習影像檔都不一樣。為了讓後續瑕疵學習的模型訓練過程更有效,減少學習上的失誤,因此剔除掉不合前述限制的學習影像檔來進行模型訓練。由於瑕疵品(這裡指同一種瑕疵,比如龜裂)的數量可以由大貨中挑出很多,因此學習影像檔的總數量不會是個問題。此外,背景光源也會影響學習影像檔的品質。因此,依照本發明,取得該些學習影像檔的背景光亮度及色溫的差異不超過1%,超過標準的學習影像檔也應該被剔除。影像 擷取裝置可以是,但不限於電荷耦合器件(Charge-Coupled Device,CCD)影像擷取裝置或互補式金屬氧化物半導體(Complementary Metal-Oxide-Semiconductor,CMOS)影像擷取裝置。型態上可以是個攝影機(不限制為有線或無線款式)。學習影像檔的檔案格式不在本發明的限制中,常用的影像檔格式皆可,但要能依照學習影像檔製作出下一步驟所指定的圖檔。 In the data preparation stage, the first step of the method is to shoot the appearance of a plurality of defective products of a product in a certain direction at a fixed distance to obtain a plurality of learning image files (S01). Regardless of how the defective products are arranged, such as in a straight line or in a ring, the same side (in a fixed direction) of each defective product must face the image capture device for shooting images, and the distance between the side of each defective product and the image capture device must be as fixed as possible to ensure that the size of the defective product and the pixel range occupied in each learning image file captured by the image capture device have small differences. According to the requirements of the present invention, the difference ratio of the number of pixels in any direction (vertical and horizontal directions) of any two learning image files on the plane is less than 0.5%. For example, in the learning image file of defective product A, the longest horizontal part occupies 4,000 pixels, and in the learning image file of defective product B, the corresponding location can only be between 3,980 and 4,020 pixels. In practice, the number of pixels occupied by all defective products in that location can be averaged, and the defective learning image files that exceed the average value by more than 0.5% can be removed. The same process is done in the vertical direction. In theory, the products of the automated production line have a consistent appearance, and under the aforementioned image capture conditions, there will be consistent image positions and pixel numbers. However, this theoretical value will be different for each learning image file due to operator errors, differences in installation on the fixture, temperature and humidity affecting light, and even problems with the image capture device itself. In order to make the subsequent defect learning model training process more effective and reduce learning errors, the learning image files that do not meet the above restrictions are eliminated for model training. Since the number of defective products (here refers to the same defect, such as cracks) can be picked out from the bulk, the total number of learning image files will not be a problem. In addition, the background light source will also affect the quality of the learning image files. Therefore, according to the present invention, the difference in the brightness and color temperature of the background light obtained for the learning image files does not exceed 1%, and the learning image files that exceed the standard should also be eliminated. Image The capture device may be, but is not limited to, a charge-coupled device (CCD) image capture device or a complementary metal-oxide-semiconductor (CMOS) image capture device. The type may be a camera (not limited to a wired or wireless type). The file format of the learning image file is not limited by the present invention, and any commonly used image file format may be used, but the image file specified in the next step must be generated according to the learning image file.

資料準備階段中執行的第二個步驟為製作關於每一學習影像檔的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯(Sobel)影像圖檔及一邊緣效果(Edge)影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同(S02)。關於相關的影像圖檔說明,請見圖2,該圖顯示由一學習影像檔L製作的一紅原色影像圖檔R、一藍原色影像圖檔B、一綠原色影像圖檔G、一索伯影像圖檔S及一邊緣效果影像圖檔E。紅原色影像圖檔R、藍原色影像圖檔B與綠原色影像圖檔G是利用三原色光模式,將學習影像檔L的色彩分成紅、藍與綠色為基底的影像。紅原色影像圖檔R、藍原色影像圖檔B與綠原色影像圖檔G可以合成原始的學習影像檔L。在圖2中,由於瑕疵品的原始外觀顏色以灰階為主,其製作的紅原色影像圖檔R、藍原色影像圖檔B與綠原色影像圖檔G都偏向灰階顏色的組合。為了方便說明,圖2學習影像檔L中的瑕疵以右上方三條彩線為例來表示。紅原色影像圖檔R、藍原色影像圖檔B與綠原色影像圖檔G在相同位置也有三條對應的瑕疵線。索伯影像圖檔S是利用索伯算子,套用圖像的梯度作為判斷依據,對於不同方向性的邊界是用分開的面罩(Mask)來偵測。當梯度變化超過一個閥值,即判斷為邊界。邊緣效果影像圖檔E是應用邊緣檢測(Edge Detection)演算法,由學習影像檔L中找出特徵邊緣。要注意的是,每一個製作的影像圖檔的像素總數及像素排列方式相同,比如每一個影像圖檔都 是4,000 x 3,000,可以不必與學習影像檔L的像素總數及像素排列方式相同,但最好一致。這個作法是為了下一個步驟。 The second step performed in the data preparation stage is to generate a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file, and an edge effect (Edge) image file for each learning image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same (S02). For the description of the related image files, please see FIG. 2, which shows a red primary color image file R, a blue primary color image file B, a green primary color image file G, a Sobel image file S, and an edge effect image file E generated from a learning image file L. The red primary color image file R, the blue primary color image file B, and the green primary color image file G are images based on red, blue, and green by using the three primary color light mode to separate the colors of the learning image file L. The red primary image file R, the blue primary image file B and the green primary image file G can be synthesized into the original learning image file L. In Figure 2, since the original appearance color of the defective product is mainly gray, the red primary image file R, the blue primary image file B and the green primary image file G are all biased towards a combination of gray colors. For the convenience of explanation, the defects in the learning image file L in Figure 2 are represented by the three colored lines in the upper right corner. The red primary image file R, the blue primary image file B and the green primary image file G also have three corresponding defect lines at the same position. The Sobel image file S uses the Sobel operator and applies the gradient of the image as the judgment basis. Separate masks are used to detect boundaries of different directions. When the gradient change exceeds a threshold value, it is judged as a boundary. The edge effect image file E uses the edge detection algorithm to find the characteristic edges from the learning image file L. It should be noted that the total number of pixels and the arrangement of pixels in each produced image file are the same. For example, if each image file is 4,000 x 3,000, it does not have to be the same as the total number of pixels and the arrangement of pixels in the learning image file L, but it is best to be consistent. This is for the next step.

資料準備階段中執行的第三個步驟為將步驟S02中的各影像圖檔串接為一訓練圖檔(S03)。如圖2所示,「+」表示串接的順序:紅原色影像圖檔R、藍原色影像圖檔B、綠原色影像圖檔G、索伯影像圖檔S及邊緣效果影像圖檔E。因此,在訓練圖檔中可以看見5個不同的圖案,不限制是橫向排列或縱向排列,惟其排列順序要一致。有多少個學習影像檔L,就會生成對應數量的訓練圖檔。訓練圖檔的格式不限,可以是光柵圖像檔案,如JPEG檔、向量圖像檔案,如WMF檔,或無失真壓縮的點陣圖圖形檔案,如PNG檔。 The third step performed in the data preparation phase is to concatenate the image files in step S02 into a training image file (S03). As shown in Figure 2, "+" indicates the order of concatenation: red primary image file R, blue primary image file B, green primary image file G, Sobel image file S and edge effect image file E. Therefore, 5 different patterns can be seen in the training image file, regardless of whether they are arranged horizontally or vertically, but the arrangement order must be consistent. The number of training images will be generated for the number of learning image files L. The format of the training image file is not limited, and can be a raster image file such as a JPEG file, a vector image file such as a WMF file, or a lossless compressed bitmap image file such as a PNG file.

依照本發明,於模型訓練階段中執行以下步驟:以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類(S04)。所有訓練圖檔指的是用來輸入學習而不分適合種瑕疵的訓練圖檔的全部,每一個訓練圖檔對應的瑕疵品可能有一個以上的瑕疵,比如裂隙、雷雕位置偏移、塗漆色偏等,這些都是前述的瑕疵種類。卷積神經網路由一個或多個卷積層和頂端的全連通層組成,同時也包括關聯權重和池化層,卷積神經網路能夠在現有輸入資料的二維結構與特性上,預測其他同質二維結構的特性。卷積神經網路在圖像辨識方面有很好結果。本發明並未限制卷積神經網路的設計,只是以特殊技術優化輸入訓練圖檔,讓相同的卷積神經網路結構,在學習訓練圖檔的過程中,對於預測特定瑕疵種類能有較準確的預估結果。 According to the present invention, the following steps are performed in the model training stage: all training images and corresponding defect types are used to generate an estimation model through a convolutional neural network to estimate the defect types of other image files (S04). All training images refer to all training images used for input learning regardless of the type of defect. Each defective product corresponding to each training image may have more than one defect, such as cracks, laser engraving position deviation, paint color deviation, etc., which are all the aforementioned defect types. The convolutional neural network consists of one or more convolutional layers and a fully connected layer at the top, as well as associated weights and pooling layers. The convolutional neural network can predict the characteristics of other homogeneous two-dimensional structures based on the two-dimensional structure and characteristics of the existing input data. The convolutional neural network has good results in image recognition. The present invention does not limit the design of the convolutional neural network, but only optimizes the input training image file with special technology, so that the same convolutional neural network structure can have more accurate prediction results for predicting specific defect types in the process of learning training images.

在品管階段中,本發明依序執行以下步驟。第一、取得同步驟S01之定距離拍攝的該產品的一受檢品的該定方向外觀之一確認影像(S05)檔。這裡,受檢品與瑕疵品都是同種產品,只是受檢品是要用估測模型來 進行瑕疵種類預估的標的,可以是產線上一系列生產的產品。受檢品用與瑕疵品相同的拍攝方法(定方向、定距離、同色溫、同亮度)進行拍攝,以取得確認影像檔。確認影像檔與學習影像檔的內容型態與檔案格式相同。第二、製作關於該確認影像檔的該紅原色影像圖檔、該藍原色影像圖檔、該綠原色影像圖檔、該索伯影像圖檔及該邊緣效果影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同(S06)。相同地,確認影像檔和學習影像檔都要製作前述的5種影像圖檔,才有一致比較的基礎,產品所佔的像素數差異比例也在0.5%以內。第三、將步驟S06中的各影像圖檔串接為一確認圖檔(S07)。確認圖檔和訓練圖檔一樣,是由紅原色影像圖檔、藍原色影像圖檔、綠原色影像圖檔、索伯影像圖檔及邊緣效果影像圖檔串接製成。確認圖檔和訓練圖檔的格式與大小相同,影像圖檔串接順序也相同。第四、將該確認圖檔輸入該估測模型中以獲得對應的瑕疵種類(S08),也就是深度學習的結果。 In the quality control stage, the present invention performs the following steps in sequence. First, obtain a confirmation image (S05) file of the fixed direction appearance of an inspected product of the product photographed at a fixed distance by the synchronization step S01. Here, the inspected product and the defective product are the same product, but the inspected product is the target for defect type estimation using the estimation model, and can be a series of products produced on the production line. The inspected product is photographed using the same shooting method as the defective product (fixed direction, fixed distance, same color temperature, same brightness) to obtain a confirmation image file. The content type and file format of the confirmation image file are the same as those of the learning image file. Second, prepare the red primary color image file, the blue primary color image file, the green primary color image file, the Sauber image file and the edge effect image file for the confirmation image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same (S06). Similarly, both the confirmation image file and the learning image file must prepare the aforementioned five types of image files to have a consistent basis for comparison, and the difference ratio of the number of pixels occupied by the product is also within 0.5%. Third, concatenate the image files in step S06 into a confirmation image file (S07). The confirmation image file is the same as the training image file, which is concatenated by the red primary color image file, the blue primary color image file, the green primary color image file, the Sauber image file and the edge effect image file. The format and size of the confirmation image file and the training image file are the same, and the order of concatenating the image files is also the same. Fourth, the confirmed image file is input into the estimation model to obtain the corresponding defect type (S08), which is the result of deep learning.

在前述的方法中,如果要再增加卷積神經網路的學習成效,產出更精準的估測模型,可以在步驟S02中進一步為每一學習影像檔製作一二值化(Threshold)影像圖檔。同步地,步驟S06中也進一步製作關於該確認影像檔的該二值化影像圖檔。二值化影像是根據特定的限幅值(clip-value)將灰度圖轉換為黑白圖,從而找出產品的影像邊緣,並標示出其瑕疵。請見圖3,學習影像檔L的二值化影像圖檔T如圖所示。估測模型學習的對象多了學習影像檔L的二值化影像圖檔T,預估時也會依據確認圖檔中二值化影像圖檔的部分作判斷。 In the aforementioned method, if you want to further increase the learning effect of the convolutional neural network and produce a more accurate estimation model, you can further generate a binarized (Threshold) image file for each learning image file in step S02. Simultaneously, step S06 also generates the binarized image file for the confirmation image file. The binarized image is to convert the grayscale image into a black and white image according to a specific clip-value, so as to find the image edge of the product and mark its defects. See Figure 3, the binarized image file T of the learning image file L is shown in the figure. The estimation model has an additional object of learning the binarized image file T of the learning image file L, and the estimation will also be based on the part of the binarized image file in the confirmation image file for judgment.

依照本發明,前述的方法還可再增加卷積神經網路的學習成效,產出最精準的估測模型。在此情況下,步驟S02中再進一步為每一學習影像檔製作一高通濾波器(High Pass Filter)處理影像圖檔,步驟S06中也再進 一步製作關於該確認影像檔的該高通濾波器處理影像圖檔。「高通」指的是銳化或邊緣強化圖形的動作,透過執行的濾波器(軟體)獲得的影像圖檔就是高通濾波器處理影像圖檔。請見復見圖3,學習影像檔L的高通濾波器處理影像圖檔H如圖所示,紅原色影像圖檔R、藍原色影像圖檔B、綠原色影像圖檔G、索伯影像圖檔S、邊緣效果影像圖檔E、二值化影像圖檔T與高通濾波器處理影像圖檔H依次串接,形成最完整的訓練圖檔或確認圖檔,以進行學習或預估。 According to the present invention, the aforementioned method can further increase the learning effect of the convolutional neural network and produce the most accurate estimation model. In this case, in step S02, a high-pass filter (High Pass Filter) processing image file is further produced for each learning image file, and in step S06, the high-pass filter processing image file for the confirmation image file is further produced. "High pass" refers to the action of sharpening or edge enhancement graphics. The image file obtained by executing the filter (software) is the high-pass filter processing image file. Please refer to Figure 3 again. The high-pass filter processed image file H of the learning image file L is shown in the figure. The red primary color image file R, the blue primary color image file B, the green primary color image file G, the Sauber image file S, the edge effect image file E, the binarized image file T and the high-pass filter processed image file H are sequentially connected in series to form the most complete training image file or confirmation image file for learning or estimation.

請見圖4,該圖為本發明實施例的一種自動化產線產品外觀瑕疵檢驗系統(以下簡稱本系統)的元件示意圖。本系統是要安裝在一個不受外部光源的干擾的封閉空間A中,以便能精準地取封閉空間A中瑕疵品或受檢品的影像,作為智能學習及判斷的依據。本系統包含了一檢驗治具10、一影像擷取裝置20、一自動學習模組30與一光源40。這些技術元件的型態、功能及整體的運作,將配合相關圖式詳盡說明。 Please see Figure 4, which is a schematic diagram of the components of an automated production line product appearance defect inspection system (hereinafter referred to as the system) of an embodiment of the present invention. The system is to be installed in a closed space A that is not disturbed by external light sources, so that images of defective products or inspected products in the closed space A can be accurately taken as a basis for intelligent learning and judgment. The system includes an inspection fixture 10, an image capture device 20, an automatic learning module 30 and a light source 40. The types, functions and overall operations of these technical components will be explained in detail with the relevant diagrams.

檢驗治具10系統地排放一產品P的數個瑕疵品或受檢品,「系統」意味以規律的方式排列以成系統地進行操作。當產品P是瑕疵品時,檢驗治具10用來執行前述方法的資料準備階段的各步驟;當產品P是受檢品時,檢驗治具10用來執行前述方法的品管階段的各步驟。 The inspection jig 10 systematically arranges several defective products or inspected products of a product P. "System" means arranging them in a regular manner to operate systematically. When the product P is a defective product, the inspection jig 10 is used to perform the steps of the data preparation stage of the above method; when the product P is an inspected product, the inspection jig 10 is used to perform the steps of the quality control stage of the above method.

影像擷取裝置20與檢驗治具10上的每一瑕疵品或受檢品保持實質相近的距離,以依序拍攝該些瑕疵品的一定方向外觀以獲得複數個學習影像檔(用於資料準備階段的各步驟),及依序拍攝該些受檢品的該定方向外觀以獲得複數個確認影像檔(用於品管階段的各步驟)。影像擷取裝置20的形態以於前文揭露,此處不再贅述。影像擷取裝置20與檢驗治具10互動以完成拍攝的作業。在本實施例中,檢驗治具10是個固定不動的裝置,上方依序、等間隔固定安裝了產品P(全為瑕疵品或全為受檢品)。影像擷 取裝置20依序由左(以實線圖像表示)到右(以虛線圖像表示),依序拍攝產品P。這是一種影像擷取裝置20與檢驗治具10的組合。在其它實施例中,如圖5所示,本系統的影像擷取裝置與檢驗治具之組合有另一種態樣。在這實施例中,檢驗治具10’是個環狀裝置,其上依序、等間隔放置了產品P,檢驗治具10’可以轉動(順時鐘方向)以帶動產品P。產品P有兩個表面需要進行外觀品檢(產品P以一個梯形繪示,兩個表面分別為上底與下底處對應的表面),第一影像擷取裝置20’用來拍攝產品P的前表面,第一影像擷取裝置20”用來拍攝產品P的後表面。這種組合是檢驗治具10’移動但影像擷取裝置不移動,便於多影像擷取裝置的操作。 The image capture device 20 maintains a substantially similar distance from each defective product or inspected product on the inspection jig 10, so as to sequentially photograph the appearance of the defective products in a certain direction to obtain a plurality of learning image files (used in each step of the data preparation stage), and sequentially photograph the appearance of the inspected products in the certain direction to obtain a plurality of confirmation image files (used in each step of the quality control stage). The shape of the image capture device 20 has been disclosed in the previous text and will not be repeated here. The image capture device 20 interacts with the inspection jig 10 to complete the shooting operation. In this embodiment, the inspection jig 10 is a fixed device, on which products P (all defective products or all inspected products) are fixedly installed in sequence and at equal intervals. The image capture device 20 sequentially photographs the product P from the left (indicated by the solid line image) to the right (indicated by the dotted line image). This is a combination of the image capture device 20 and the inspection jig 10. In other embodiments, as shown in FIG5, the combination of the image capture device and the inspection jig of the present system has another form. In this embodiment, the inspection jig 10' is a ring-shaped device, on which the products P are placed sequentially and at equal intervals, and the inspection jig 10' can be rotated (clockwise) to drive the product P. Product P has two surfaces that need to be inspected for appearance (product P is drawn as a trapezoid, and the two surfaces are the surfaces corresponding to the upper and lower bases respectively). The first image capture device 20' is used to photograph the front surface of product P, and the first image capture device 20" is used to photograph the back surface of product P. This combination is that the inspection fixture 10' moves but the image capture device does not move, which facilitates the operation of multiple image capture devices.

自動學習模組30安裝於一工作主機1中,藉由操作工作主機1而完成特定的作業。本系統的工作主機1的硬體架構和一般伺服器架構無大差異,可包含中央處理器、記憶體、儲存裝置(比如硬碟)、輸出入單元等。基於不同的需求,各種組成硬體的特性及功能會有差異。這些硬體為伺服器領域的技術人員所應了解的架構。此外,以下所介紹關於本系統的自動學習模組30的各個模組及其子模組,為利用或配合上述現有硬體設備而運行的技術要件。因此,它們可以是軟體,包含了特定的程式碼與資料,而在作業系統下運行於至少一部份的硬體架構中(比如程式碼與相關資料檔案儲存於儲存裝置中,在作業系統的運作下暫存於記憶體,而為中央處理器動態的調用執行)。另一方面,該些模組也可以是特製硬體,比如特殊應用積體電路(Application-Specific Integrated Circuit,ASIC)或外接卡,用以執行該些模組或子模組所賦予的作用。更有甚者,這些技術要件可以是部分是軟體、部分是硬體,依照產品設計人員的需求而有效整合,都在本專利所主張的技術範圍內。 The automatic learning module 30 is installed in a working host 1, and a specific operation is completed by operating the working host 1. The hardware architecture of the working host 1 of this system is not much different from the general server architecture, and may include a central processing unit, memory, storage device (such as a hard disk), input and output units, etc. Based on different needs, the characteristics and functions of various components of the hardware will be different. These hardware are architectures that technical personnel in the server field should understand. In addition, the various modules and sub-modules of the automatic learning module 30 of this system introduced below are technical requirements for operating by utilizing or cooperating with the above-mentioned existing hardware equipment. Therefore, they can be software, including specific program codes and data, and run in at least part of the hardware architecture under the operating system (for example, the program codes and related data files are stored in the storage device, temporarily stored in the memory under the operation of the operating system, and dynamically called and executed by the central processor). On the other hand, these modules can also be special hardware, such as application-specific integrated circuits (ASIC) or external cards, to perform the functions assigned by these modules or sub-modules. What's more, these technical requirements can be partly software and partly hardware, and can be effectively integrated according to the needs of product designers, all within the technical scope advocated by this patent.

請見圖6,該圖繪示自動學習模組30的架構。自動學習模組30包含了以下子模組:一資料前處理子模組31、一影像產生子模組32、一模型學習子模組33與一品質判斷子模組34。 Please see Figure 6, which shows the architecture of the automatic learning module 30. The automatic learning module 30 includes the following sub-modules: a data pre-processing sub-module 31, an image generation sub-module 32, a model learning sub-module 33 and a quality judgment sub-module 34.

資料前處理子模組31接收來自影像擷取裝置10的該些學習影像檔與該些確認影像檔,並去除無關該產品的背景影像。如此,學習影像檔與確認影像檔會更「乾淨」,但檔案大小(像素數量)彼此差異在0.5%以內。影像產生子模組32可執行以下作業:第一、為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯影像圖檔及一邊緣效果影像圖檔,各影像圖檔的像素總數及像素排列方式相同。任二學習影像檔在平面任一方向上的像素數量的差異比例小於0.5%,影像產生子模組32可以把不符合要求的學習影像檔排除。第二、將來源自同一學習影像檔之製作的影像圖檔的串接為一訓練圖檔,用於資料準備階段。第三、將來源自同一確認影像檔之製作的影像圖檔的串接為一確認圖檔,用於品管階段。模型學習子模組33以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類。運作方式同前述方法步驟S04所述,本處不予以贅述。品質判斷子模組34則將該些確認圖檔輸入該估測模型中以獲得對應的瑕疵種類。 The data pre-processing submodule 31 receives the learning image files and the confirmation image files from the image capture device 10, and removes the background image that is not related to the product. In this way, the learning image files and the confirmation image files will be "cleaner", but the file size (number of pixels) will differ from each other within 0.5%. The image generation submodule 32 can perform the following operations: First, for each of the learning image files and the confirmation image files, a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file and an edge effect image file are produced. The total number of pixels and the pixel arrangement of each image file are the same. If the difference ratio of the number of pixels in any direction of the plane between any two learning image files is less than 0.5%, the image generation submodule 32 can exclude the learning image files that do not meet the requirements. Second, the image files produced from the same learning image file are concatenated into a training image file for the data preparation stage. Third, the image files produced from the same confirmation image file are concatenated into a confirmation image file for the quality control stage. The model learning submodule 33 uses all the training images and the corresponding defect types to generate an estimation model through a convolutional neural network to estimate the defect types of other image files. The operation method is the same as that described in the aforementioned method step S04, and will not be repeated here. The quality judgment submodule 34 inputs the confirmation images into the estimation model to obtain the corresponding defect types.

影像產生子模組32還進一步為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的二值化影像圖檔,加入紅原色影像圖檔、藍原色影像圖檔、綠原色影像圖檔、索伯影像圖檔及邊緣效果影像圖檔中,一起進行學習或用於預估瑕疵種類。加入二值化影像圖檔內容進入訓練圖檔中,可增加卷積神經網路的學習成效,產出更精準的估測模型。相似地,影像產生子模組32也可進一步為該些學習影像檔與該些確認影像檔中的每 一者製作與其相關的高通濾波器處理影像圖檔,依序串接紅原色影像圖檔、藍原色影像圖檔、綠原色影像圖檔、索伯影像圖檔、邊緣效果影像圖檔、二值化影像圖檔與高通濾波器處理影像圖檔形成訓練圖檔或確認圖檔,以用於學習或預估瑕疵種類。 The image generation submodule 32 further generates a binary image file related to each of the learning image files and the confirmation image files, and adds the binary image file to the red primary color image file, the blue primary color image file, the green primary color image file, the Sobel image file, and the edge effect image file for learning or for estimating the defect type. Adding the binary image file content to the training image file can increase the learning effect of the convolutional neural network and produce a more accurate estimation model. Similarly, the image generation submodule 32 can also further generate high-pass filter processed image files related to each of the learning image files and the confirmation image files, and sequentially connect the red primary color image file, the blue primary color image file, the green primary color image file, the Sauber image file, the edge effect image file, the binarization image file and the high-pass filter processed image file to form a training image file or a confirmation image file for learning or estimating the defect type.

光源40具有控制影像擷取裝置20獲取影像品質的功能。發光時光源40維持影像擷取裝置40所在封閉空間A中的背景光亮度及色溫的差異不超過1%,以輔助學習影像檔與確認影像檔的正規化。 The light source 40 has the function of controlling the image quality of the image capture device 20. When emitting light, the light source 40 maintains the difference in the background light brightness and color temperature in the closed space A where the image capture device 40 is located within 1%, so as to assist in learning the image file and confirm the normalization of the image file.

雖然本發明已以實施方式揭露如上,然其並非用以限定本發明,任何所屬技術領域中具有通常知識者,在不脫離本發明之精神和範圍內,當可作些許之更動與潤飾,因此本發明之保護範圍當視後附之申請專利範圍所界定者為準。 Although the present invention has been disclosed in the form of implementation as above, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of the patent application attached hereto.

S01~S08:步驟 S01~S08: Steps

Claims (10)

一種自動化產線產品外觀瑕疵檢驗方法,包含步驟:於一資料準備階段中,執行:a)以定距離拍攝一產品的複數個瑕疵品的一定方向外觀,以獲得複數個學習影像檔;b)製作關於每一學習影像檔的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯(Sobel)影像圖檔及一邊緣效果(Edge)影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同;及c)將步驟b)中的各影像圖檔串接為一訓練圖檔;於一模型訓練階段中,執行:d)以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類;以及於一品管階段中,執行:e)取得同步驟a)之定距離拍攝的該產品的一受檢品的該定方向外觀之一確認影像檔;f)製作關於該確認影像檔的該紅原色影像圖檔、該藍原色影像圖檔、該綠原色影像圖檔、該索伯影像圖檔及該邊緣效果影像圖檔,其中各影像圖檔的像素總數及像素排列方式相同;g)將步驟f)中的各影像圖檔串接為一確認圖檔;及h)將該確認圖檔輸入該估測模型中以獲得對應的瑕疵種類。 An automated production line product appearance defect inspection method comprises the steps of: in a data preparation phase, executing: a) photographing the appearance of a plurality of defective products of a product in a certain direction at a fixed distance to obtain a plurality of learning image files; b) generating a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file and an edge effect (Edge) image file for each learning image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same; and c) concatenating the image files in step b) into a training image file; in a model training phase, executing: d) using all training images and corresponding defect images to obtain a training image file; Type, through a convolution neural network to generate an estimation model for estimating the defect type of other image files; and in the first quality control stage, perform: e) obtain a confirmation image file of the fixed direction appearance of a product under inspection of the product photographed at a fixed distance in synchronization with step a); f) generate the red primary color image file, the blue primary color image file, the green primary color image file, the Sobel image file and the edge effect image file related to the confirmation image file, wherein the total number of pixels and the pixel arrangement method of each image file are the same; g) concatenate the image files in step f) into a confirmation image file; and h) input the confirmation image file into the estimation model to obtain the corresponding defect type. 如申請專利範圍第1項所述之自動化產線產品外觀瑕疵檢驗方法,其中步驟b)中進一步為每一學習影像檔製作一二值化(Threshold)影像圖檔,步驟f)中進一步製作關於該確認影像檔的該二值化影像圖檔。 As described in the first item of the patent application scope, the automated production line product appearance defect inspection method, wherein in step b), a binarization (Threshold) image file is further produced for each learning image file, and in step f), the binarization image file related to the confirmation image file is further produced. 如申請專利範圍第2項所述之自動化產線產品外觀瑕疵檢驗方法,其中步驟b)中進一步為每一學習影像檔製作一高通濾波器(High Pass Filter)處理影像圖檔,步驟f)中進一步製作關於該確認影像檔的該高通濾波器處理影像圖檔。 As described in the second item of the patent application scope, the automated production line product appearance defect inspection method, wherein in step b), a high pass filter (High Pass Filter) processing image file is further produced for each learning image file, and in step f), the high pass filter processing image file of the confirmation image file is further produced. 如申請專利範圍第1項所述之自動化產線產品外觀瑕疵檢驗方法,其中取得該些學習影像檔的背景光亮度及色溫的差異不超過1%。 As described in Item 1 of the patent application scope, the method for inspecting appearance defects of products on an automated production line, wherein the difference in the background light brightness and color temperature of the learning image files does not exceed 1%. 如申請專利範圍第1項所述之自動化產線產品外觀瑕疵檢驗方法,其中任二學習影像檔中的瑕疵品在平面任一方向上的像素數量的差異比例小於0.5%。 As described in Item 1 of the patent application scope, the method for inspecting appearance defects of products on an automated production line, wherein the difference in the number of pixels of defective products in any two learning image files in any direction on the plane is less than 0.5%. 一種自動化產線產品外觀瑕疵檢驗系統,包含:一檢驗治具,系統地排放一產品的複數個瑕疵品或受檢品;一影像擷取裝置,與該檢驗治具上的每一瑕疵品或受檢品保持實質相近的距離,以依序拍攝該些瑕疵品的一定方向外觀以獲得複數個學習影像檔,及依序拍攝該些受檢品的該定方向外觀以獲得複數個確認影像檔;及一自動學習模組,安裝於一工作主機中,包含以下子模組:一資料前處理子模組,接收來自該影像擷取裝置的該些學習影像檔與該些確認影像檔,並去除無關該產品的背景影像;一影像產生子模組,執行以下作業:為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的一紅原色影像圖檔、一藍原色影像圖檔、一綠原色影像圖檔、一索伯(Sobel)影像圖檔及一邊緣效果影像圖檔,各影像圖檔的像素總數及像素排列方式相同;將來源自同一學習影像檔之製作的影像圖檔的串接為一訓練圖檔;及將來源自同一確認影像檔之製作的影像圖檔的串接為一確認圖檔; 一模型學習子模組,以所有訓練圖檔及對應的瑕疵種類,透過一卷積神經網路以產生出一估測模型,用以預估其它影像檔的瑕疵種類;及一品質判斷子模組,將該些確認圖檔輸入該估測模型中以獲得對應的瑕疵種類。 An automated production line product appearance defect inspection system comprises: an inspection jig, which systematically arranges a plurality of defective products or inspected products of a product; an image capture device, which maintains a substantially similar distance from each defective product or inspected product on the inspection jig, to sequentially photograph the appearance of the defective products in a certain direction to obtain a plurality of learning image files, and to sequentially photograph the appearance of the inspected products in the certain direction to obtain a plurality of confirmation image files; and an automatic learning module, which is installed in a working host and comprises the following submodules: a data pre-processing submodule, which receives the learning image files and the confirmation image files from the image capture device and removes background images irrelevant to the product; an image generation submodule, which performs the following operations: generating a plurality of learning image files and the confirmation image files; Each of the confirmed image files generates a red primary color image file, a blue primary color image file, a green primary color image file, a Sobel image file and an edge effect image file, and the total number of pixels and pixel arrangement of each image file are the same; the image files generated from the same learning image file are concatenated into a training image file; and the image files generated from the same confirmed image file are concatenated into a confirmed image file; a model learning submodule, using all the training images and the corresponding defect types, generates an estimation model through a convolutional neural network to estimate the defect types of other image files; and a quality judgment submodule, inputs the confirmed images into the estimation model to obtain the corresponding defect types. 如申請專利範圍第6項所述之自動化產線產品外觀瑕疵檢驗系統,其中該影像產生子模組進一步為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的二值化影像圖檔。 As described in Item 6 of the patent application scope, the automated production line product appearance defect inspection system, wherein the image generation submodule further produces a binary image file associated with each of the learning image files and the confirmation image files. 如申請專利範圍第7項所述之自動化產線產品外觀瑕疵檢驗系統,其中該影像產生子模組進一步為該些學習影像檔與該些確認影像檔中的每一者製作與其相關的高通濾波器處理影像圖檔。 As described in Item 7 of the patent application scope, the automated production line product appearance defect inspection system, wherein the image generation submodule further produces a high-pass filter processed image file associated with each of the learning image files and the confirmation image files. 如申請專利範圍第6項所述之自動化產線產品外觀瑕疵檢驗系統,進一步包含一光源,發光時維持該影像擷取裝置所在封閉空間中的背景光亮度及色溫的差異不超過1%。 The automated production line product appearance defect inspection system as described in Item 6 of the patent application further includes a light source that maintains the difference in background light brightness and color temperature in the closed space where the image capture device is located within 1% when emitting light. 如申請專利範圍第6項所述之自動化產線產品外觀瑕疵檢驗系統,其中任二學習影像檔中的瑕疵品在平面任一方向上的像素數量的差異比例小於0.5%。 As described in Item 6 of the patent application scope, the automated production line product appearance defect inspection system, wherein the difference ratio of the number of pixels of defective products in any two learning image files in any direction of the plane is less than 0.5%.
TW111149651A 2022-12-23 Appearance defect inspection method and system for automated production line products TWI842292B (en)

Publications (2)

Publication Number Publication Date
TWI842292B true TWI842292B (en) 2024-05-11
TW202426898A TW202426898A (en) 2024-07-01

Family

ID=

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018165753A1 (en) 2017-03-14 2018-09-20 University Of Manitoba Structure defect detection using machine learning algorithms

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018165753A1 (en) 2017-03-14 2018-09-20 University Of Manitoba Structure defect detection using machine learning algorithms

Similar Documents

Publication Publication Date Title
US11017259B2 (en) Defect inspection method, defect inspection device and defect inspection system
JP4935109B2 (en) Substrate inspection device, inspection logic setting method and inspection logic setting device
CN110779928B (en) Defect detection device and method
US20200265575A1 (en) Flaw inspection apparatus and method
TWI743837B (en) Training data increment method, electronic apparatus and computer-readable medium
JP2007147442A (en) Method, device, and program for inspecting wood
JP3741672B2 (en) Image feature learning type defect detection method, defect detection apparatus, and defect detection program
CN115830004A (en) Surface defect detection method, device, computer equipment and storage medium
CN116543247A (en) Data set manufacturing method and verification system based on photometric stereo surface reconstruction
Huang et al. Deep learning object detection applied to defect recognition of memory modules
JP2020112483A (en) Exterior appearance inspection system, calculation model construction method and calculation model construction program
TWI842292B (en) Appearance defect inspection method and system for automated production line products
CN110274911B (en) Image processing system, image processing apparatus, and storage medium
CN113592859B (en) Deep learning-based classification method for defects of display panel
JP2013117490A (en) Inspection system and method for setting recipe
CN112257514B (en) Infrared vision intelligent detection shooting method for equipment fault inspection
KR20230082346A (en) Method of detecting defect of manufactured product based on high-resolution image using autoencoder and convolutional neural network
CN114581536A (en) Image color difference detection method based on feature perception and multi-channel learning
JP3652589B2 (en) Defect inspection equipment
Xie et al. Evolving PCB visual inspection programs using genetic programming
CN113034424A (en) Model training method and electronic device
TWI786894B (en) Detection method
JP2006284543A (en) Method and device for inspecting mounted circuit board
TWI841834B (en) Apparatus and method for checking line defects on panel
TWI780465B (en) Defect-inspecting method of goggles and a system thereof