WO2024181209A1 - 処理方法およびそれを利用した処理装置 - Google Patents

処理方法およびそれを利用した処理装置 Download PDF

Info

Publication number
WO2024181209A1
WO2024181209A1 PCT/JP2024/005818 JP2024005818W WO2024181209A1 WO 2024181209 A1 WO2024181209 A1 WO 2024181209A1 JP 2024005818 W JP2024005818 W JP 2024005818W WO 2024181209 A1 WO2024181209 A1 WO 2024181209A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing
size
image
unit
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2024/005818
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
翔吾 待井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2025503792A priority Critical patent/JPWO2024181209A1/ja
Publication of WO2024181209A1 publication Critical patent/WO2024181209A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • This disclosure relates to processing technology, in particular to a processing method for performing processing on an image and a processing device that utilizes the same.
  • FCN head computed tomography
  • the neural network is required to measure the size of an object in an image.
  • This disclosure has been made in light of these circumstances, and its purpose is to provide a technology for measuring the size of an object in an image using a neural network.
  • a processing device includes an input unit that inputs an inspection image to be inspected, a processing unit that executes neural network processing on the inspection image input to the input unit, a first output unit that outputs information on the class of defective items contained in the inspection image as a processing result of the processing unit, and a second output unit that outputs information on the size of the defective items contained in the inspection image as a processing result of the processing unit.
  • the input unit inputs a learning image in which the class and size of the defective items are known
  • the processing unit trains the neural network so that, when the input unit inputs the learning image, the first output unit outputs information on the class of the known defective items and the second output unit outputs information on the size of the known defective items.
  • Another aspect of the present disclosure is a processing method.
  • This method includes the steps of inputting an inspection image to be inspected, executing neural network processing on the input inspection image, outputting information on the class of defective items contained in the inspection image as a processing result, and outputting information on the size of the defective items contained in the inspection image as a processing result.
  • the neural network is trained to output information on the class of the known defective items and to output information on the size of the known defective items when a learning image in which the class and size of the defective items are known is input.
  • FIG. 13 is a diagram showing a configuration of a neural network in a processing device that is a comparison target of this embodiment.
  • 2(a)-(b) are diagrams showing the input and output in the neural network of FIG.
  • FIG. 1 is a diagram illustrating a configuration of a processing apparatus according to an embodiment of the present invention.
  • FIG. 4 is a diagram showing a configuration of a neural network in the processing unit of FIG. 3 .
  • 5(a)-(c) are diagrams showing the inputs and outputs in the neural network of FIG.
  • the example relates to a processing device that judges an inspection image showing a product to be inspected. The judgment is whether the product is good or defective.
  • a product is good if it does not contain any defective parts and if the size of the defective parts contained in the product is smaller than the reference value.
  • a product is defective if the size of the defective parts contained in the product is equal to or larger than the reference value.
  • the processing device is required to identify the presence or absence of a defective item, which defective item it is, and the size of the defective item.
  • FIG. 1 shows the configuration of a neural network in a processing device 10 that is compared to this embodiment.
  • the processing device 10 includes a first convolutional layer 40a to a fourth convolutional layer 40d, collectively referred to as convolutional layers 40, and a first pooling layer 42a to a fourth pooling layer 42d, collectively referred to as pooling layers 42.
  • the number of convolutional layers 40 and pooling layers 42 included in the processing device 10 is not limited to "4", and the number of convolutional layers 40 and the number of pooling layers 42 may be different.
  • this neural network is an FCN structure that does not include a fully connected layer.
  • an image is input to the first convolutional layer 40a, and processing is performed in the order of the first convolutional layer 40a, the first pooling layer 42a, the second convolutional layer 40b, the second pooling layer 42b, the third convolutional layer 40c, the third pooling layer 42c, the fourth convolutional layer 40d, and the fourth pooling layer 42d.
  • the convolutional processing and the pooling processing are repeatedly performed.
  • the size of the image is successively reduced.
  • the number of channels in the neural network is determined based on the number of defective items to be detected, and is set to, for example, "3.”
  • Each convolutional layer 40 performs spatial filtering on an image by shifting a spatial filter whose size is smaller than the size of the image. Since spatial filtering is a known technique, a description thereof will be omitted here, but this spatial filtering corresponds to a convolution process, and image features are extracted by the convolution process. Padding and the like may also be performed in the convolutional layer 40. Furthermore, the convolutional layer 40 may perform multiple spatial filtering in parallel on an image by using multiple spatial filters in parallel. The number of images increases by using multiple spatial filters in parallel in this way. The number of spatial filters used in parallel in the convolutional layer 40 is called the number of channels mentioned above.
  • Each pooling layer 42 reduces the size of the image by combining multiple pixels contained in any region in the image into one pixel.
  • average pooling is performed to combine multiple pixels into one pixel, but max pooling may also be performed.
  • average pooling the average value of multiple pixel values in the region is used for one pixel
  • max pooling the maximum value of multiple pixel values in the region is used for one pixel.
  • the pooling process is performed to enhance robustness against translational movement of the representative value or average value in the region of interest.
  • a scoring layer (not shown) may be included after the fourth pooling layer 42d, and an accuracy metric such as SoftMax with Loss may be used in the scoring layer.
  • the image inspection process is performed after the learning process.
  • an image with known defective items hereinafter referred to as the "learning image”
  • the spatial filter coefficients of each convolutional layer 40 are learned so that the output from the fourth pooling layer 42d, which is the result of processing the learning image, becomes the defective items contained in the learning image (hereinafter also referred to as "teaching data").
  • an image to be inspected hereinafter referred to as the “inspection image”
  • the processing result for the inspection image is output from the fourth pooling layer 42d.
  • the output from the fourth pooling layer 42d indicates the defective items.
  • the learning image and the inspection image are collectively referred to as the "image” mentioned above.
  • Figures 2(a)-(b) show the input and output in a neural network.
  • Figure 2(a) shows an inspection image.
  • the inspection image includes a foreign object 200 as a defective item.
  • Figure 2(b) shows the processing result.
  • a three-channel configuration including three types of defective items, "OK”, “foreign object”, and “flash”, is shown, and the processing result shows the probability for each defective item. Since the probability of a foreign object is the highest at "0.9", the processing result shows a foreign object.
  • Such a processing device 10 can identify defective items, as mentioned above, it is not possible to measure the size of the defective items, for example the size of the foreign object 200. Below, a configuration that can measure the size of the foreign object 200 will be described.
  • FIG. 3 shows the configuration of a processing device 100 according to this embodiment.
  • the processing device 100 includes an input unit 110, a processing unit 120, and a first output unit 130a and a second output unit 130b, which are collectively referred to as an output unit 130.
  • the input unit 110 inputs learning images in which the class and size of defective items are known.
  • the input unit 110 inputs an inspection image to be inspected.
  • the processing unit 120 includes a neural network that processes the learning images or inspection images.
  • FIG. 4 shows the configuration of a neural network in the processing unit 120.
  • the processing unit 120 includes a first convolutional layer 140a to a fifth convolutional layer 140e collectively referred to as convolutional layers 140, and a first pooling layer 142a to a fifth pooling layer 142e collectively referred to as pooling layers 142.
  • the number of convolutional layers 140 and pooling layers 142 included in the processing unit 120 is not limited to "5", and the number of convolutional layers 140 and the number of pooling layers 142 may be different.
  • the first convolution layer 140a is connected to the input unit 110 in FIG. 3, and processing is performed in the order of the first convolution layer 140a, the first pooling layer 142a, the second convolution layer 140b, the second pooling layer 142b, the third convolution layer 140c, the third pooling layer 142c, the fourth convolution layer 140d, and the fourth pooling layer 142d.
  • the fourth pooling layer 142d is connected to the first output unit 130a in FIG. 3 via a scoring layer (not shown) or the like.
  • the first convolution layer 140a to the fourth pooling layer 142d from the input unit 110 to the first output unit 130a may be called the "first path". Comparing the first path with FIG.
  • the convolution layer 140 corresponds to the convolution layer 40
  • the pooling layer 142 corresponds to the pooling layer 42
  • the configuration of the first path corresponds to the configuration of FIG. 1.
  • the number of channels in the neural network is determined based on the number of defects to be detected, and is set to, for example, "3.”
  • processing is performed in the order of the first convolution layer 140a, the first pooling layer 142a, the second convolution layer 140b, the second pooling layer 142b, the third convolution layer 140c, the third pooling layer 142c, the fifth convolution layer 140e, and the fifth pooling layer 142e.
  • the fifth pooling layer 142e is connected to the second output unit 130b in FIG. 3.
  • the first convolution layer 140a to the fifth pooling layer 142e from the input unit 110 to the second output unit 130b may be called the "second path".
  • the first convolution layer 140a, the first pooling layer 142a, the second convolution layer 140b, the second pooling layer 142b, the third convolution layer 140c, and the third pooling layer 142c are commonly used.
  • the image inspection process is performed after the learning process.
  • a learning process for the first path hereinafter referred to as the "first learning process”
  • a learning process for the second path hereinafter referred to as the "second learning process”
  • the first learning process a learning image is input to the first convolutional layer 140a, and the coefficients of the spatial filters of each convolutional layer 140 are learned so that the output from the fourth pooling layer 142d, which is the result of the processing of the learning image, becomes the defective items contained in the learning image (hereinafter also referred to as the "first teacher data").
  • the first learning process is similar to the learning process for the neural network in FIG. 1.
  • the processing unit 120 trains the neural network so that when the input unit 110 inputs a learning image, the first output unit 130a outputs information on the class of known defective items, and the second output unit 130b outputs information on the size of known defective items.
  • the processing unit 120 executes neural network processing on the inspection image input to the input unit 110.
  • the first output unit 130a outputs information on the class of defective items contained in the inspection image as the processing result of the processing unit 120. This corresponds to outputting a judgment value map for each class in the inspection image.
  • the second output unit 130b outputs information on the size of defective items contained in the inspection image as the processing result of the processing unit 120. This corresponds to outputting a size map for each class in the inspection image. In the size judgment on the size map, blocks with a certain brightness value and area value or more in the judgment value map are identified as defective parts, and the maximum brightness value in the size map on the obtained defective parts is judged to be the defective size.
  • the processing device 100 may determine whether the product shown in the inspection image is good or bad based on the class information output from the first output unit 130a and the size information output from the second output unit 130b. The processing device 100 determines that the product is good if the class information is "OK". The processing device 100 also determines that the product is good if the class information is not "OK” but the size is smaller than a reference value. On the other hand, the processing device 100 determines that the product is bad if the class information is not "OK" and the size is equal to or larger than the reference value.
  • Figures 5(a)-(c) show the input and output in the neural network.
  • Figure 5(a) shows an inspection image.
  • the inspection image includes a foreign object 200 as a defective item, as in Figure 2(a).
  • the size of the foreign object 200 is, for example, "13.0 pix”.
  • Figure 5(b) shows the result of classification among the processing results.
  • the defective items are classified into three classes, "OK”, “foreign object”, and “flash”, and the result of classification shows the probability for each defective item. Since the probability of a foreign object is the highest at "0.9", the result of classification shows a foreign object.
  • Figure 5(c) shows the result of size measurement among the processing results.
  • a three-channel configuration including three types of defective items, "OK”, “foreign object", and “flash”, is shown, and the result of size measurement shows the size for each defective item.
  • the size of the foreign object is shown as "12.9 pix”.
  • the subject of the device, system, or method disclosed herein includes a computer.
  • the computer executes a program to realize the functions of the subject of the device, system, or method disclosed herein.
  • the computer includes a processor that operates according to a program as its main hardware configuration. The type of processor is not important as long as it can realize the functions by executing the program.
  • the processor is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or an LSI (Large Scale Integration).
  • the multiple electronic circuits may be integrated into one chip or may be provided on multiple chips.
  • the multiple chips may be integrated into one device or may be provided on multiple devices.
  • the program is recorded on a non-transitory recording medium such as a computer-readable ROM, optical disk, or hard disk drive.
  • the program may be stored in the recording medium in advance, or may be supplied to the recording medium via a wide area communication network including the Internet.
  • a learning image in which the class and size of the defective items are known is input, and the neural network is trained to output information on the known class of the defective items and information on the known size of the defective items, so that the size of an object in an image can be measured in the neural network.
  • a common convolutional layer 140 is used in part of the first path from the input unit 110 to the first output unit 130a and in part of the second path from the input unit 110 to the second output unit 130b, so that the efficiency of the learning process can be improved.
  • a processing device (100) of one aspect of the present disclosure includes an input unit (110) that inputs an inspection image to be inspected, a processing unit (120) that executes neural network processing on the inspection image input to the input unit (110), a first output unit (130a) that outputs information on the class of defective items contained in the inspection image as a processing result of the processing unit (120), and a second output unit (130b) that outputs information on the size of the defective items contained in the inspection image as a processing result of the processing unit (120).
  • the input unit (110) inputs a learning image in which the class and size of the defective items are known, and the processing unit (120) learns the neural network so that when the input unit (110) inputs the learning image, the first output unit (130a) outputs information on the class of the known defective items and the second output unit (130b) outputs information on the size of the known defective items.
  • the neural network in the processing unit (120) may use a common convolutional layer in a portion of the first path from the input unit (110) to the first output unit (130a) and in a portion of the second path from the input unit (110) to the second output unit (130b).
  • Another aspect of the present disclosure is a processing method.
  • This method includes the steps of inputting an inspection image to be inspected, executing neural network processing on the input inspection image, outputting information on the class of defective items contained in the inspection image as a processing result, and outputting information on the size of the defective items contained in the inspection image as a processing result.
  • the neural network is trained to output information on the class of the known defective items and to output information on the size of the known defective items when a learning image in which the class and size of the defective items are known is input.
  • the processing unit 120 is configured with multiple convolution layers 140 and multiple pooling layers 142 arranged alternately.
  • the processing unit 120 may have a GoogleNet-based or DenseNet-based configuration, for example. This modified example allows for greater flexibility in the configuration.
  • the processing unit 120 is configured such that multiple convolutional layers 140 and multiple pooling layers 142 are arranged alternately.
  • the processing unit 120 may include only the convolutional layer 140 and perform only the convolution process, or may include only the pooling layer 142 and perform only the pooling process. This modified example allows for greater flexibility in the configuration.
  • 100 Processing device 110 Input unit, 120 Processing unit, 130 Output unit, 140 Convolution layer, 142 Pooling layer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
PCT/JP2024/005818 2023-02-27 2024-02-19 処理方法およびそれを利用した処理装置 Ceased WO2024181209A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2025503792A JPWO2024181209A1 (https=) 2023-02-27 2024-02-19

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023-028806 2023-02-27
JP2023028806 2023-02-27

Publications (1)

Publication Number Publication Date
WO2024181209A1 true WO2024181209A1 (ja) 2024-09-06

Family

ID=92590470

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/005818 Ceased WO2024181209A1 (ja) 2023-02-27 2024-02-19 処理方法およびそれを利用した処理装置

Country Status (2)

Country Link
JP (1) JPWO2024181209A1 (https=)
WO (1) WO2024181209A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021086350A (ja) * 2019-11-27 2021-06-03 富士フイルム株式会社 画像学習装置、画像学習方法、ニューラルネットワーク、及び画像分類装置
JP2021092887A (ja) * 2019-12-09 2021-06-17 コニカミノルタ株式会社 外観検査装置、外観検査システム、特徴量推定装置および外観検査プログラム
JP2022506485A (ja) * 2018-11-02 2022-01-17 ケーエルエー コーポレイション ブランクレチクル上の欠陥のタイプおよびサイズを判定するためのシステムおよび方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022506485A (ja) * 2018-11-02 2022-01-17 ケーエルエー コーポレイション ブランクレチクル上の欠陥のタイプおよびサイズを判定するためのシステムおよび方法
JP2021086350A (ja) * 2019-11-27 2021-06-03 富士フイルム株式会社 画像学習装置、画像学習方法、ニューラルネットワーク、及び画像分類装置
JP2021092887A (ja) * 2019-12-09 2021-06-17 コニカミノルタ株式会社 外観検査装置、外観検査システム、特徴量推定装置および外観検査プログラム

Also Published As

Publication number Publication date
JPWO2024181209A1 (https=) 2024-09-06

Similar Documents

Publication Publication Date Title
US11017259B2 (en) Defect inspection method, defect inspection device and defect inspection system
JP7087397B2 (ja) 基板の欠陥検査装置、基板の欠陥検査方法及び記憶媒体
JP6952268B2 (ja) 処理方法およびそれを利用した処理装置
CN112614086B (zh) 学习装置、检查装置、学习方法以及检查方法
JP7241292B2 (ja) 処理方法およびそれを利用した処理装置
WO2021181749A1 (ja) 学習装置、画像検査装置、学習済みパラメータ、学習方法、および画像検査方法
US12499532B2 (en) Computer and visual inspection method for identifying defective products
CN112561852A (zh) 图像判断装置和图像判断方法
CN112461839A (zh) 外观检查方法及外观检查装置
Schwebig et al. Compilation of training datasets for use of convolutional neural networks supporting automatic inspection processes in industry 4.0 based electronic manufacturing
CN115937059A (zh) 具有生成式训练模型的零件检查系统
CN114894801B (zh) 检查系统、检查方法及检查程序
JP2020112483A (ja) 外観検査システム、計算モデル構築方法及び計算モデル構築プログラム
JP7258509B2 (ja) 画像処理装置、画像処理方法、及び画像処理プログラム
WO2024181209A1 (ja) 処理方法およびそれを利用した処理装置
CN117083630A (zh) 信息处理装置、控制程序以及控制方法
JP4629086B2 (ja) 画像欠陥検査方法および画像欠陥検査装置
JP7557796B2 (ja) 処理方法およびそれを利用した処理装置
JP7774231B2 (ja) 処理方法およびそれを利用した処理装置
WO2021033372A1 (ja) 処理方法およびそれを利用した処理装置
JP7711637B2 (ja) ウェーハの判定方法、判定プログラム、判定装置、ウェーハの製造方法及びウェーハ
TWI902049B (zh) 檢查裝置、檢查方法以及電腦可讀取的程式
US20250209597A1 (en) Substrate inspecting apparatus and method of inspecting substrate
CN121329933A (zh) 一种pcb电路板生产质量检测方法
CN118446956A (zh) 检查装置以及检查方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24763690

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025503792

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025503792

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24763690

Country of ref document: EP

Kind code of ref document: A1