WO2022130814A1 - 指標選択装置、情報処理装置、情報処理システム、検査装置、検査システム、指標選択方法、および指標選択プログラム - Google Patents
指標選択装置、情報処理装置、情報処理システム、検査装置、検査システム、指標選択方法、および指標選択プログラム Download PDFInfo
- Publication number
- WO2022130814A1 WO2022130814A1 PCT/JP2021/040525 JP2021040525W WO2022130814A1 WO 2022130814 A1 WO2022130814 A1 WO 2022130814A1 JP 2021040525 W JP2021040525 W JP 2021040525W WO 2022130814 A1 WO2022130814 A1 WO 2022130814A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- index
- defective product
- index selection
- score
- input data
- Prior art date
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 66
- 230000010365 information processing Effects 0.000 title claims abstract description 56
- 238000010187 selection method Methods 0.000 title claims abstract description 24
- 230000002950 deficient Effects 0.000 claims abstract description 160
- 238000004364 calculation method Methods 0.000 claims abstract description 43
- 230000002159 abnormal effect Effects 0.000 claims description 80
- 230000005856 abnormality Effects 0.000 claims description 52
- 238000009826 distribution Methods 0.000 claims description 24
- 238000000034 method Methods 0.000 claims description 18
- 230000002547 anomalous effect Effects 0.000 claims description 4
- 230000007547 defect Effects 0.000 abstract description 27
- 238000001514 detection method Methods 0.000 abstract description 14
- 230000006870 function Effects 0.000 description 28
- 238000010586 diagram Methods 0.000 description 20
- 238000012545 processing Methods 0.000 description 20
- 238000003860 storage Methods 0.000 description 10
- 238000013135 deep learning Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Definitions
- the present invention relates to an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection program.
- An inspection device equipped with an image processing function for judging the quality / defect of a work by using an input image obtained by photographing a production object (work) on a production line is known.
- an image processing algorithm is used to extract the feature amount of the input image, and the quality / defect of the work is determined based on the threshold value for separating the non-defective product / the defective product.
- Patent Document 1 in order to improve the inspection accuracy, rules and threshold values (inspection logic) that specify a method for determining whether the work is good or bad are dynamically set according to changes in the production environment. It is disclosed to do.
- the present invention has been made to solve such a problem, and has improved the accuracy of defect detection, such as an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection device.
- the purpose is to provide an indicator selection program.
- a score calculation unit that calculates an abnormal score of a non-defective product and a defective product based on a plurality of input data of a non-defective product and a defective product and a reference data corresponding to the input data, and the non-defective product and the non-defective product.
- An index selection device comprising an index selection unit for selecting any one of the plurality of indexes according to an abnormality score of a defective product.
- the abnormality score is the difference between the result of calculating the index using the input data and the result of calculating the index using the reference data for each of the non-defective product and the defective product.
- the index selection device according to 1).
- the index selection unit selects an index having the largest difference between the distribution of the abnormal score of the good product and the distribution of the abnormal score of the defective product with respect to the plurality of input data and the reference data.
- the index selection device according to 1) or (2).
- the index selection unit uses the feature amount of a plurality of input data of the non-defective product and the defective product as an explanatory variable, and the difference between the distribution of the abnormal score of the non-defective product and the distribution of the abnormal score of the defective product as the objective variable.
- the index according to any one of (1) to (4) above, wherein one of the plurality of indexes is selected by using the trained model trained so as to have the largest difference. Selection device.
- the input data is color image data
- the index selection unit calculates an abnormal score based on hue and / or saturation as the index, whichever is 1 of (1) to (5) above.
- the index selection device described in 1.
- the input data acquisition unit for acquiring input data, the input data acquired by the input data acquisition unit, the reference data corresponding to the input data, and any one of claims 1 to 6.
- An information processing device having a score calculation unit that calculates an abnormal score based on an index selected by the index selection device.
- An inspection device having a determination unit for determining a non-defective product or a defective product based on the abnormality score output by the information processing apparatus according to (7) above.
- An information processing system including the information processing device according to (7) above and a display device for displaying an abnormal score by the score calculation unit.
- An inspection system including the inspection device according to (8) above and a display device for displaying a determination result by the determination unit.
- An index selection method comprising a step (b) of selecting any one of the plurality of indexes according to an abnormality score of a non-defective product and a defective product.
- the abnormality score is the difference between the result of calculating the index using the input data and the result of calculating the index using the reference data for each of the non-defective product and the defective product.
- step (b) the index having the largest difference between the distribution of the abnormal score of the good product and the distribution of the abnormal score of the defective product with respect to the plurality of input data and the reference data is selected.
- the above (11) to (14) further include a step (c) of generating reconstruction data as reference data based on the input data by a generation model learned using input data of a plurality of non-defective products.
- the index selection method according to any one of 13).
- the feature amount of the plurality of input data of the non-defective product and the defective product is used as an explanatory variable, and the difference between the distribution of the abnormal score of the non-defective product and the distribution of the abnormal score of the defective product is used as the objective variable.
- the input data is color image data, and in step (b), any one of the above (11) to (15) for calculating an abnormality score due to hue and / or saturation as the index.
- An index selection program for causing a computer to execute a process included in the index selection method according to any one of (11) to (16) above.
- the index selection device of the present invention one of a plurality of indexes is selected according to the abnormal score of the non-defective product and the defective product, so that an index effective for separating the non-defective product / the defective product can be obtained.
- the information processing device calculates the abnormality score using the index selected by the index selection device, so that the user can confirm the position and area of the abnormality to be inspected from the abnormality score map displayed on the display. Further, the inspection device can improve the determination accuracy of the non-defective product / defective product to be inspected based on the abnormality score calculated by the information processing device.
- the index selection device the information processing device, the information processing system, the inspection device, the inspection system, the index selection method, and the index selection program according to the embodiment of the present invention will be described with reference to the drawings.
- the same elements are designated by the same reference numerals, and duplicate description will be omitted.
- FIG. 1 is a schematic block diagram illustrating a hardware configuration of an information processing apparatus 100 according to an embodiment
- FIG. 2 illustrates a main function of a control unit 110 when the information processing apparatus 100 functions as an index selection apparatus. It is a functional block diagram.
- FIG. 3 is a schematic block diagram illustrating the function of the control unit 110 shown in FIG. 2
- FIG. 4 is a schematic diagram for explaining the calculation of the abnormal score by the score calculation unit 230 shown in FIG.
- FIG. 5 is a schematic diagram illustrating the calculation results of the abnormal scores 1 to N by the index parameters 1 to N.
- the information processing apparatus 100 calculates an abnormality score based on a plurality of index parameters (indexes) based on a plurality of input images and a plurality of reference images corresponding to these input images, and a plurality of indexes according to the abnormality score. It functions as an index selection device that selects any one of the parameters.
- the input image is labeled as a good product (normal inspection target) or a defective product (inspection target including abnormalities such as defects), and by learning the index parameters (deep learning or machine learning), it is not good or bad.
- Index parameters that are effective in separating from non-defective products are selected.
- the index parameters can be set, for example, individually or in combination of hue, saturation, density, object shape, size, and the like, which are color attributes of the input image and the reconstructed image. .. The details of the calculation method of the abnormal score will be described later.
- the inspection target is not particularly limited, but examples thereof include parts used in industrial products. Inspection includes detection of abnormalities such as breaks, bends, chips, scratches, and stains. Further, the information processing apparatus 100 acquires a plurality of non-defective input images (hereinafter, also referred to as "normal input images"), learns these images as correct image, and generates a reconstructed image from the input image. Deep learning. Generate a model (generative model). The reference image can be a reconstructed image of the input image.
- the information processing apparatus 100 has a control unit 110, a communication unit 120, and an operation display unit 130. These components are connected to each other via the bus 101.
- the information processing device 100 may be, for example, a computer such as a personal computer or a server.
- the control unit 110 includes a CPU (Central Processing Unit) 111, a RAM (Random Access Memory) 112, a ROM (Read Only Memory) 113, and an auxiliary storage unit 114.
- CPU Central Processing Unit
- RAM Random Access Memory
- ROM Read Only Memory
- the CPU 111 executes programs such as an OS (Operating System), an index selection program, and an inspection program developed in the RAM 112, and controls the operation of the information processing apparatus 100.
- the index selection program and the inspection program are stored in the ROM 113 or the auxiliary storage unit 114 in advance.
- the RAM 112 stores data or the like temporarily generated by the processing of the CPU 111.
- the ROM 113 stores a program executed by the CPU 111, data used for executing the program, parameters, and the like.
- the auxiliary storage unit 114 has, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.
- control unit 110 functions as an image acquisition unit 210, an image reconstruction unit 220, a score calculation unit 230, and an index selection unit 240 by executing an index selection program by the CPU 111.
- the image acquisition unit 210 functions as an input data acquisition unit and acquires an input image (input data) by cooperating with the communication unit 120.
- the image reconstruction unit 220 functions as a data reconstruction unit, and is used as reference data based on the input image acquired by the image acquisition unit 210 by the generation model learned by using the input images of a plurality of non-defective products related to the same inspection target. Generates a reconstructed image (reconstructed data) of. More specifically, the image reconstruction unit 220 extracts a feature amount from the input image by the trained first neural network, and reconstructs (restores) the input image based on the extracted feature amount. Generate a reconstructed image.
- the first neural network has a multi-layered convolutional neural network, and is pre-learned by supervised learning so that the difference between the reconstructed image and the input image disappears or becomes as small as possible at the time of learning.
- the first neural network functions as a generative model having an encoder / decoder structure, for example.
- an autoencoder AE: AutoEncoder
- VAE Variational AutoEncoder
- AE and VAE are known techniques, detailed description thereof will be omitted.
- the image reconstruction unit 220 has, for example, a VAE including an encoder 221 and a decoder 222 as a generative model.
- VAE extracts only the essential elements of the input image by extracting the features of the input image, and reconstructs them using the extracted features to exclude the non-essential elements of the input image. Generates and outputs a reconstructed image. That is, since VAE learns only with a normal input image, it is configured so that a corresponding feature amount can be generated with respect to the normal input image, and is an inspection target including a defective product, that is, an abnormality such as a defect or a scratch. For the input image (hereinafter, also referred to as "abnormal input image”), the feature amount corresponding to the abnormality cannot be generated, and the reproducibility is not obtained.
- the input image includes an image of the member M1 as an inspection target.
- the member M1 originally has a linear texture T1. Further, when the member M1 is a defective product, for example, the member M1 includes a scratch S1 generated in the middle of the manufacturing process.
- the reconstructing process of the input image is executed, the reconstructed image obtained by reconstructing the image of the member M1 is output.
- the reconstructed image is an image in which only essential elements are left from the image of the member M1 of the input image and unnecessary elements are removed. Since the texture T1 is originally provided by the member M1 (essential element), it is reconstructed. On the other hand, the wound S1 is abnormal (not an essential element) and is not reconstructed.
- the image reconstruction unit 220 generates and outputs a reconstruction image in which non-essential elements in the input image are excluded. Therefore, in the case of a defective product rather than a good product, the input image and the reconstruction image are combined. The difference becomes large.
- a predetermined image related to the same inspection target can be used as the reference image instead of the reconstructed image.
- the predetermined image may be, for example, a typical non-defective image for the same inspection subject that is not used for the input image.
- it is necessary to perform correction processing for the position shift or size it is necessary to perform correction processing for the position shift or size.
- the above correction process is not necessary.
- the score calculation unit 230 calculates an abnormality score based on a plurality of index parameters based on a plurality of input images of non-defective products and defective products and a plurality of reconstructed images corresponding to the input images. As shown in FIG. 4, the score calculation unit 230 is an index based on, for example, a plurality of input images labeled as non-defective products or defective products, and a plurality of reconstructed images corresponding to these input images. Anomalous scores 1 to N for non-defective products and defective products are calculated according to parameters 1 to N.
- the anomaly score may be the difference between the result of calculating the index parameter value using the input image and the result of calculating the index parameter value using the reconstructed image for each of the non-defective product and the defective product.
- hue and saturation can be used as index parameters.
- FIG. 5 shows the normalized anomaly score and the anomaly score for good and defective products in Cartesian coordinates with the number of samples (input images) used to select the indicator parameters on the horizontal and vertical axes, respectively.
- the distribution of the values of is illustrated.
- the abnormal scores due to the index parameters 1 to N correspond to the abnormal scores 1 to N, respectively.
- the good sample is distributed in a region where the normalized abnormal score is relatively small, and the defective sample is distributed in a region where the normalized abnormal score is relatively large.
- the data of the abnormality scores 1 to N are stored in the auxiliary storage unit 114.
- the index selection unit 240 selects any one of the index parameters 1 to N according to the abnormality scores 1 to N of the non-defective product and the defective product.
- the abnormal score of the non-defective product the shaded area in the figure
- the abnormal score of the defective product the gray part in the figure
- the index selection unit 240 selects the index parameter N having the largest difference between the distribution of the abnormal score of the non-defective product and the distribution of the abnormal score of the defective product.
- the selection of index parameters is selected by performing deep learning.
- the index selection unit 240 has, for example, a multi-layered convolutional neural network (second neural network), and deep learning (deep learning) of index parameters for a plurality of input images of good and defective products related to the same inspection target. Or machine learning). By inputting a plurality of input images of the same inspection target, an abnormality score is accumulated for each pair of the input image and the reference image, and the learning of the index parameter is deepened.
- second neural network second neural network
- deep learning deep learning
- the index selection unit 240 uses the feature quantities of a plurality of input images of the non-defective product and the defective product as explanatory variables, and the difference between the distribution of the abnormal score of the non-defective product and the distribution of the abnormal score of the defective product (that is, the abnormality of the non-defective product). With the distance between the statistic of the score and the statistic of the abnormal score of the defective product as the objective variable, the second neural network is trained so that the above difference (distance) is the largest, and a trained model is generated. ..
- the statistic can be, for example, a histogram.
- it may be configured to learn index parameters so that the false positive rate of non-defective products is reduced.
- index parameter may be selected so that (distance) is the largest.
- the index selection unit 240 selects one of a plurality of index parameters (index parameter N in the example of FIG. 5) for one inspection target using the trained model.
- the communication unit 120 is an interface circuit (for example, a LAN card or the like) for communicating with an external device via a network.
- an interface circuit for example, a LAN card or the like
- the operation display unit 130 has an input unit and an output unit.
- the input unit is provided with, for example, a keyboard, a mouse, etc., and is used for the user to perform various instructions (inputs) such as character input by the keyboard, mouse, etc., and various settings.
- the output unit includes a display (display device) and displays an input image or the like.
- the inspection target is photographed by an image pickup device such as a camera, for example.
- the image pickup apparatus transmits the captured image data of the inspection target to the information processing apparatus 100.
- the information processing apparatus 100 acquires image data as an input image. Further, the images to be inspected taken in advance by the imaging device are stored in a storage device outside the information processing device 100, and the information processing device 100 stores a predetermined number of images to be inspected stored in the storage device. It can also be configured to be sequentially acquired as an input image.
- the image pickup device when the inspection target is a part of an industrial product, the image pickup device is installed in the inspection process, takes a picture of a shooting range including the inspection target, and outputs image data including the inspection target.
- the image pickup apparatus outputs, for example, data of a black-and-white image or a color image to be inspected of a predetermined pixel (for example, 128 pixels ⁇ 128 pixels).
- FIG. 6 is a functional block diagram illustrating the main functions of the control unit 110 when the information processing device 100 functions as an information processing device. Further, FIG. 7 is a schematic block diagram illustrating the function of the control unit 110 shown in FIG. 6, and FIG. 8 is a functional block diagram illustrating the function of the score calculation unit 330 shown in FIG. 7.
- control unit 110 functions as an image acquisition unit 310, an image reconstruction unit 320, and a score calculation unit 330 by executing an inspection program by the CPU 111.
- the image acquisition unit 310 functions as an input data acquisition unit and acquires an input image (input data) by cooperating with the communication unit 120.
- the image reconstruction unit 320 functions as a data reconstruction unit, and is based on the input image acquired by the image acquisition unit 310 by the generation model learned using the input images of a plurality of non-defective products. Generate a reconstructed image (reconstructed data) as reference data. More specifically, the image reconstruction unit 320 extracts a feature amount from the input image by the trained first neural network, and reconstructs (restores) the input image based on the extracted feature amount. Generate a reconstructed image.
- the determination unit 340 determines whether the product is non-defective or defective, but if the input image contains noise, the abnormality score may not be calculated appropriately due to the noise.
- the filter can be a denoising filter (eg, a low pass filter, a high pass filter, or a band pass filter). With such a configuration, the influence of noise on the abnormal score is alleviated, and it is possible to prevent or suppress the deterioration of the determination performance (separation performance) of the non-defective product / defective product due to the noise.
- the score calculation unit 330 calculates an abnormal score between the input image acquired by the image acquisition unit 310 and the reconstructed image reconstructed by the image reconstruction unit 320. As shown in FIG. 8, the score calculation unit 330 has a calculation processing unit 331. The arithmetic processing unit 331 uses at least one of the index parameters 1 to N to calculate the abnormal score as the output of the score calculation unit 330.
- the score calculation unit 330 calculates the value of the index parameter used by the arithmetic processing unit 331 among the index parameters 1 to N, and calculates the abnormal score based on the value of the index parameter. For example, when the score calculation unit 330 is set so that only the abnormal score of the index parameter 1 (luminance in the figure) is used, the value of the index parameter 1 and the abnormal score thereof are used for the input image and the reconstructed image. Is calculated. In this case, the arithmetic processing unit 331 calculates the abnormal score as the output of the score calculation unit 330 based only on the abnormal score of the index parameter 1.
- the abnormality score of the index parameter 1 may be, for example, the difference between the luminance value of the input image and the luminance value of the reconstructed image.
- the abnormality score of the index parameter 2 may be, for example, a difference between the result of calculating the index parameter 2 using the input image and the result of calculating the index parameter 2 using the reconstructed image. The same applies to the method of calculating the abnormal score of other index parameters.
- the arithmetic processing unit 331 can set use / non-use for each index parameter 1 to N abnormality score based on the index parameter selected by the index selection device and stored in the RAM 212.
- the arithmetic processing unit 331 has the abnormality score of the index parameter 1. Based on the abnormal scores of both the index and the abnormal score of the index parameter 2, the abnormal score as the output of the score calculation unit 330 is calculated.
- the abnormal score of the score calculation unit 330 can be calculated by weighting the abnormal score of the index parameter 1 and the abnormal score of the index parameter 2 with predetermined coefficients and adding them together.
- the operation display unit 130 displays an abnormal score (abnormal score map) calculated by the score calculation unit 330 on the display. As a result, the user can confirm the position and area of the abnormality to be inspected from the abnormality score map displayed on the display.
- the control unit 110 and the operation display unit 130 constitute an information processing system.
- FIG. 9 is a functional block diagram illustrating the main functions of the control unit when the information processing apparatus functions as an inspection apparatus.
- the control unit 110 has a determination unit 340 in addition to the image acquisition unit 310, the image reconstruction unit 320, and the score calculation unit 330.
- the determination unit 340 determines whether the inspection target is a non-defective product or a defective product based on the abnormal score calculated by the score calculation unit 330. For example, when the determination unit 340 is set so that only the abnormal score of the index parameter 1 is used in the score calculation unit 330, the maximum value of the abnormal score (abnormal score map) of the luminance is set to a predetermined value set in advance. If it is equal to or less than the first threshold value, it can be determined that the inspection target is a non-defective product, and if it exceeds the first threshold value, it can be determined that the inspection target is a defective product.
- the region exceeding the first threshold value is presumed to be a defect region such as a scratch in the inspection target.
- the first threshold can be determined experimentally by the user, for example, based on anomalous score maps calculated for a plurality of images to be inspected, including non-defective and defective products.
- the determination unit 340 can determine a non-defective product / a defective product by using only the abnormal brightness score, but for example, when it is difficult to detect the defective region only by the abnormal brightness score, the brightness of the index parameter 1 A good product / defective product can be determined using the abnormal score and the abnormal score calculated based on the abnormal score of the hue of the index parameter 2. Alternatively, the determination unit 340 can determine a non-defective product / a defective product by using only the abnormal score of the index parameter 2.
- the determination unit 340 presets a second threshold value for separating the non-defective product and the defective product for each index parameter based on the distribution of the abnormal scores of the non-defective product and the defective product calculated by the score calculation unit 230 of the index selection device. do.
- the determination unit 340 determines that the inspection target is a non-defective product and sets the second threshold value. If it exceeds, it can be determined that the inspection target is a defective product. Therefore, in the abnormality score map, the region exceeding the second threshold value is presumed to be a defect region such as a scratch in the inspection target.
- the second threshold value is calculated for, for example, a plurality of images to be inspected including non-defective products and defective products. It can be determined experimentally by the user based on the anomaly score.
- the determination result is stored in the RAM 112. Further, the determination result may be displayed on the display of the operation display unit 130.
- the information processing device 100 constitutes an inspection system.
- FIG. 10 is a flowchart illustrating the processing procedure of the index selection method of the present embodiment.
- the processing of the flowchart of FIG. 10 is realized by the CPU 111 executing the index selection program.
- the input image is acquired (step S101).
- the image acquisition unit 210 acquires a plurality of input images to be inspected from, for example, an external image pickup device or storage device of the information processing device 100. These input images are pre-labeled as non-defective or defective.
- the image acquisition unit 210 transmits a plurality of input images to the image reconstruction unit 220 and the score calculation unit 230.
- the image reconstruction unit 220 generates a plurality of reconstructed images corresponding to the plurality of input images by the generation model based on the plurality of input images acquired by the image acquisition unit 210.
- the score calculation unit 230 calculates anomalous scores 1 to N according to index parameters 1 to N based on a plurality of input images of non-defective products and defective products and a plurality of reconstructed images corresponding to the input images.
- the index selection unit 240 selects any one of the index parameters 1 to N according to the abnormality scores 1 to N of the non-defective product and the defective product. More specifically, the index selection unit 240 learns index parameters for a plurality of input images of non-defective products and defective products, and from index parameters 1 to N, distribution of abnormal scores of non-defective products and abnormal scores of defective products. Select the index parameter that has the largest difference from the distribution.
- FIG. 11 is a flowchart illustrating the processing procedure of the inspection method of the present embodiment.
- the processing of the flowchart of FIG. 11 is realized by the CPU 111 executing the inspection program.
- FIG. 12 is a schematic diagram for explaining prevention of over-detection.
- an input image is acquired (step S201).
- the image acquisition unit 310 acquires an input image to be inspected from, for example, an external image pickup device or storage device of the information processing device 100.
- the input image is an image to be inspected (unknown) for which a non-defective product / defective product has not been determined.
- the image acquisition unit 310 transmits the input image to the image reconstruction unit 320 and the score calculation unit 330.
- the image reconstruction unit 320 generates a reconstruction image corresponding to the input image by the generation model based on the input image acquired by the image acquisition unit 310.
- the score calculation unit 330 calculates an abnormal luminance score based on the input image and the reconstructed image corresponding to the input image. Further, the score calculation unit 330 calculates an abnormality score based on the index parameter based on the input image, the reconstructed image corresponding to the input image, and the index parameter selected in step S104 of the index selection method. The index parameters are selected for each inspection target.
- the determination unit 340 determines whether the inspection target is a non-defective product or a defective product based on the abnormality score of the luminance calculated by the score calculation unit 330 and the abnormality score by the index parameter. In the example shown in FIG. 7, since the region corresponding to the scratch S1 exceeds the second threshold value in the abnormality score map based on the index parameter, it is determined that the inspection target is a defective product.
- defect detection can be performed for the difference between the luminance value of the input image and the luminance value of the reconstructed image.
- the input image IM1 contains a linear defect F added in the manufacturing process in addition to the linear feature C originally provided in the non-defective product.
- the reconstructed image IM2 corresponding to the input image IM1 the feature C of the input image IM1 is reconstructed, but the defect F is not reconstructed. Therefore, the difference image IM3 is generated by subtracting the brightness value of the reconstructed image IM2 from the brightness value of the input image IM1, and the defect F is left in the difference image IM3.
- the linear defect F can be detected by, for example, a linear defect detection algorithm or the like.
- the input image IM1 includes the feature C of a good product and the linear defect F even if the linear defect detection algorithm is used. Therefore, not only the defect F but also the feature C may be detected as a defect.
- a detection algorithm when performing defect detection, a detection algorithm can be used for the difference between the luminance value of the input image and the luminance value of the reconstructed image. Therefore, even when the characteristics of the non-defective product and the characteristics of the defect match or are similar to each other, over-detection of the non-defective product can be more effectively prevented or suppressed as compared with the prior art.
- the index selection device According to the index selection device, the information processing device, and the inspection device of the present embodiment described above, the following effects can be obtained.
- the index selection device selects one of a plurality of indexes according to the abnormal score of the non-defective product and the defective product, an index effective for separating the non-defective product / the defective product can be obtained.
- the information processing device calculates the abnormality score using the index selected by the index selection device, so that the user can confirm the position and area of the abnormality to be inspected from the abnormality score map displayed on the display.
- the inspection device can improve the determination accuracy of the non-defective product / defective product to be inspected based on the abnormality score calculated by the information processing device. Further, it is possible to improve the robustness and the accuracy of defect detection for various inspection targets and defect types.
- the index selection device, the information processing device, the information processing system, the inspection device, the inspection system, the index selection method, and the index selection program described above have described the main configurations in explaining the features of the above-described embodiment.
- the configuration is not limited to the above, and various modifications can be made within the scope of the claims. Further, it does not exclude the configuration provided in a general inspection device or the like.
- the case where the information processing device 100 also serves as the index selection device, the information processing device, and the inspection device has been described, but the case is not limited to such a case, and the index selection device, the information processing device, and the information processing device, and The inspection device may be implemented on separate hardware.
- the means and methods for performing various processes in the above-mentioned device can be realized by either a dedicated hardware circuit or a programmed computer.
- the program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versaille Disc) -ROM, or may be provided online via a network such as the Internet.
- the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk.
- the above program may be provided as a single application software, or may be incorporated into software of a device such as an index selection device or an information processing device as one function.
- 100 information processing equipment 110 Control unit, 111 CPU, 112 RAM, 113 ROM, 114 Auxiliary storage, 120 communication unit, 130 operation display unit, 210 Image acquisition department, 220 Image Reconstruction Unit, 221 encoder, 222 decoder, 230 Score Calculator, 240 Index selection unit, 310 image acquisition unit, 320 Image Reconstruction Department, 321 encoder, 322 Decoder, 330 Score Calculator, 331 arithmetic processing unit, 340 Judgment unit.
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Analytical Chemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Biochemistry (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
Description
情報処理装置100は、複数の入力画像と、これらの入力画像に対応する複数の参照画像とに基づいて、複数の指標パラメーター(指標)による異常スコアを算出し、異常スコアに応じて複数の指標パラメーターのうちのいずれか1つを選択する指標選択装置として機能する。入力画像は、良品(正常な検査対象)または不良品(欠陥等の異常を含む検査対象)のラベルが付けられており、指標パラメーターを学習(深層学習または機械学習)することにより、良品と不良品とを分離するのに有効な指標パラメーターが選択される。指標パラメーターは、例えば、入力画像および再構成画像の色の属性である色相(Hue)、彩度(Saturation)、濃度、オブジェクトの形状、大きさ等を各々単独、あるいはそれらを組み合わせとして設定されうる。異常スコアの算出方法の詳細については後述する。
上述のように、本実施形態では、色相、彩度を指標パラメーターとして使用しうる。これにより、例えば、検出対象が、輝度値のみでは検出することが困難な微細な色の変化を含む欠陥を有する場合であっても、欠陥領域を検出できる。
図6は情報処理装置100が情報処理装置として機能する場合における制御部110の主要な機能を例示する機能ブロック図である。また、図7は図6に示す制御部110の機能を例示する概略ブロック図であり、図8は図7に示すスコア算出部330の機能を例示する機能ブロック図である。
図9は、情報処理装置が検査装置として機能する場合における制御部の主要な機能を例示する機能ブロック図である。制御部110は、画像取得部310、画像再構成部320、およびスコア算出部330に加えて、判定部340を有する。
図10は、本実施形態の指標選択方法の処理手順を例示するフローチャートである。図10のフローチャートの処理は、CPU111が指標選択プログラムを実行することにより実現される。
図11は、本実施形態の検査方法の処理手順を例示するフローチャートである。図11のフローチャートの処理は、CPU111が検査プログラムを実行することにより実現される。また、図12は、過検出の防止について説明するための模式図である。
110 制御部、
111 CPU、
112 RAM、
113 ROM、
114 補助記憶部、
120 通信部、
130 操作表示部、
210 画像取得部、
220 画像再構成部、
221 エンコーダー、
222 デコーダー、
230 スコア算出部、
240 指標選択部、
310 画像取得部、
320 画像再構成部、
321 エンコーダー、
322 デコーダー、
330 スコア算出部、
331 演算処理部、
340 判定部。
Claims (17)
- 良品および不良品の複数の入力データと、前記入力データに対応する複数の参照データと、に基づいて、複数の指標による良品および不良品の異常スコアを算出するスコア算出部と、
前記良品および不良品の異常スコアに応じて、前記複数の指標のうちのいずれか1つを選択する指標選択部と、を有する、指標選択装置。 - 前記異常スコアは、良品および不良品の各々について、前記入力データを用いて前記指標を算出した結果と、前記参照データを用いて前記指標を算出した結果との差分である、請求項1に記載の指標選択装置。
- 前記指標選択部は、前記複数の入力データおよび前記複数の参照データに対する、前記良品の異常スコアの分布と前記不良品の異常スコアの分布との差異が最も大きくなる指標を選択する、請求項1または2に記載の指標選択装置。
- 複数の良品の入力データを用いて学習した生成モデルにより、前記入力データに基づいて、前記参照データとしての再構成データを生成するデータ再構成部をさらに有する、請求項1~3のいずれか1項に記載の指標選択装置。
- 前記指標選択部は、
良品および不良品の複数の入力データの特徴量を説明変数とし、前記良品の異常スコアの分布と前記不良品の異常スコアの分布との差異を目的変数として、前記差異が最も大きくなるように学習された学習済みモデルを使用して、前記複数の指標のうちのいずれか1つを選択する、請求項1~4のいずれか1項に記載の指標選択装置。 - 前記入力データは、カラーの画像データであり、
前記指標選択部は、前記指標として色相、および/または彩度による異常スコアを算出する、請求項1~5のいずれか1項に記載の指標選択装置。 - 入力データを取得する入力データ取得部と、
前記入力データ取得部によって取得された入力データと、前記入力データに対応する参照データと、請求項1~6のいずれか1項に記載の指標選択装置によって選択された指標と、に基づいて異常スコアを算出するスコア算出部を有する、情報処理装置。 - 請求項7に記載の情報処理装置の出力した異常スコアに基づいて、良品または不良品の判定を行う判定部を有する検査装置。
- 請求項7に記載の情報処理装置と、
前記スコア算出部による異常スコアを表示する表示装置と、を有する情報処理システム。 - 請求項8に記載の検査装置と、
前記判定部による判定結果を表示する表示装置と、を有する検査システム。 - 良品および不良品の複数の入力データと、前記入力データに対応する複数の参照データと、に基づいて複数の指標による良品および不良品の異常スコアを算出するステップ(a)と、
前記良品および不良品の異常スコアに応じて、前記複数の指標のうちのいずれか1つを選択するステップ(b)と、を有する、指標選択方法。 - 前記異常スコアは、良品および不良品の各々について、前記入力データを用いて前記指標を算出した結果と、前記参照データを用いて前記指標を算出した結果との差分である、請求項11に記載の指標選択方法。
- 前記ステップ(b)において、前記複数の入力データおよび前記複数の参照データに対する、前記良品の異常スコアの分布と前記不良品の異常スコアの分布との差異が最も大きくなる指標を選択する、請求項11または12に記載の指標選択方法。
- 複数の良品の入力データを用いて学習した生成モデルにより、前記入力データに基づいて、前記参照データとしての再構成データを生成するステップ(c)をさらに有する、請求項11~13のいずれか1項に記載の指標選択方法。
- 前記ステップ(b)において、良品および不良品の複数の入力データの特徴量を説明変数とし、前記良品の異常スコアの分布と前記不良品の異常スコアの分布との差異を目的変数として、前記差異が最も大きくなるように学習された学習済みモデルを使用して、前記複数の指標のうちのいずれか1つを選択する、請求項11~14のいずれか1項に記載の指標選択方法。
- 前記入力データは、カラーの画像データであり、
前記ステップ(b)において、前記指標として色相、および/または彩度による異常スコアを算出する、請求項11~15のいずれか1項に記載の指標選択方法。 - 請求項11~16のいずれか1項に記載の指標選択方法が含む処理をコンピューターに実行させるための指標選択プログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/037,820 US20240005477A1 (en) | 2020-12-16 | 2021-11-04 | Index selection device, information processing device, information processing system, inspection device, inspection system, index selection method, and index selection program |
JP2022569759A JPWO2022130814A1 (ja) | 2020-12-16 | 2021-11-04 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020-208476 | 2020-12-16 | ||
JP2020208476 | 2020-12-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022130814A1 true WO2022130814A1 (ja) | 2022-06-23 |
Family
ID=82057553
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/040525 WO2022130814A1 (ja) | 2020-12-16 | 2021-11-04 | 指標選択装置、情報処理装置、情報処理システム、検査装置、検査システム、指標選択方法、および指標選択プログラム |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240005477A1 (ja) |
JP (1) | JPWO2022130814A1 (ja) |
WO (1) | WO2022130814A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230368372A1 (en) * | 2021-12-03 | 2023-11-16 | Contemporary Amperex Technology Co., Limited | Fast anomaly detection method and system based on contrastive representation distillation |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017049974A (ja) * | 2015-09-04 | 2017-03-09 | キヤノン株式会社 | 識別器生成装置、良否判定方法、およびプログラム |
JP2020173496A (ja) * | 2019-04-08 | 2020-10-22 | 国立大学法人岐阜大学 | 異常品判定方法 |
CN111833300A (zh) * | 2020-06-04 | 2020-10-27 | 西安电子科技大学 | 一种基于生成对抗学习的复合材料构件缺陷检测方法和装置 |
JP2020187655A (ja) * | 2019-05-16 | 2020-11-19 | 株式会社キーエンス | 画像検査装置 |
JP2020187657A (ja) * | 2019-05-16 | 2020-11-19 | 株式会社キーエンス | 画像検査装置 |
JP2020197797A (ja) * | 2019-05-31 | 2020-12-10 | 株式会社東芝 | 画像処理装置及び画像処理方法 |
-
2021
- 2021-11-04 US US18/037,820 patent/US20240005477A1/en active Pending
- 2021-11-04 JP JP2022569759A patent/JPWO2022130814A1/ja active Pending
- 2021-11-04 WO PCT/JP2021/040525 patent/WO2022130814A1/ja active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017049974A (ja) * | 2015-09-04 | 2017-03-09 | キヤノン株式会社 | 識別器生成装置、良否判定方法、およびプログラム |
JP2020173496A (ja) * | 2019-04-08 | 2020-10-22 | 国立大学法人岐阜大学 | 異常品判定方法 |
JP2020187655A (ja) * | 2019-05-16 | 2020-11-19 | 株式会社キーエンス | 画像検査装置 |
JP2020187657A (ja) * | 2019-05-16 | 2020-11-19 | 株式会社キーエンス | 画像検査装置 |
JP2020197797A (ja) * | 2019-05-31 | 2020-12-10 | 株式会社東芝 | 画像処理装置及び画像処理方法 |
CN111833300A (zh) * | 2020-06-04 | 2020-10-27 | 西安电子科技大学 | 一种基于生成对抗学习的复合材料构件缺陷检测方法和装置 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230368372A1 (en) * | 2021-12-03 | 2023-11-16 | Contemporary Amperex Technology Co., Limited | Fast anomaly detection method and system based on contrastive representation distillation |
US12020425B2 (en) * | 2021-12-03 | 2024-06-25 | Contemporary Amperex Technology Co., Limited | Fast anomaly detection method and system based on contrastive representation distillation |
Also Published As
Publication number | Publication date |
---|---|
JPWO2022130814A1 (ja) | 2022-06-23 |
US20240005477A1 (en) | 2024-01-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020031984A1 (ja) | 部品の検査方法及び検査システム | |
KR20230124713A (ko) | 결함 검출 방법, 장치 및 시스템 | |
JP7393313B2 (ja) | 欠陥分類装置、欠陥分類方法及びプログラム | |
JP2013224833A (ja) | 外観検査装置、外観検査方法及びコンピュータプログラム | |
KR20210141060A (ko) | 머신러닝 기반의 이미지 이상 탐지 시스템 | |
WO2022130814A1 (ja) | 指標選択装置、情報処理装置、情報処理システム、検査装置、検査システム、指標選択方法、および指標選択プログラム | |
CN112767331B (zh) | 基于零样本学习的图像异常检测方法 | |
JP7459697B2 (ja) | 異常検知システム、学習装置、異常検知プログラム、学習プログラム、異常検知方法、および学習方法 | |
US7646892B2 (en) | Image inspecting apparatus, image inspecting method, control program and computer-readable storage medium | |
JP7240199B2 (ja) | パラメータ決定方法、パラメータ決定装置、及びプログラム | |
JP2022029262A (ja) | 画像処理装置、画像処理方法、画像処理プログラム、および学習装置 | |
JP7460391B2 (ja) | 学習装置、画像検査装置、プログラム、および学習方法 | |
US11120541B2 (en) | Determination device and determining method thereof | |
KR20230036650A (ko) | 영상 패치 기반의 불량 검출 시스템 및 방법 | |
JP2021174194A (ja) | 学習用データ処理装置、学習装置、学習用データ処理方法、およびプログラム | |
WO2022201968A1 (ja) | 情報処理装置、制御プログラム、および制御方法 | |
JP2023008416A (ja) | 異常検知システムおよび異常検知方法 | |
JP7459696B2 (ja) | 異常検知システム、学習装置、異常検知プログラム、学習プログラム、異常検知方法、および学習方法演算装置の学習方法 | |
JP7436752B2 (ja) | 検知装置および検知方法 | |
US20230005120A1 (en) | Computer and Visual Inspection Method | |
JP7372072B2 (ja) | 画像処理装置、画像処理方法、画像処理プログラム、および画像検査システム | |
WO2023243202A1 (ja) | 画像生成方法及び外観検査装置 | |
JP7386287B1 (ja) | 検査方法及び検査装置 | |
WO2022244406A1 (ja) | 信号処理装置、信号処理方法、及びプログラム | |
JP7437610B2 (ja) | 画像検査装置、画像検査方法及び学習済みモデル生成装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21906178 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022569759 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18037820 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21906178 Country of ref document: EP Kind code of ref document: A1 |