WO2023089846A1 - 検査装置および検査方法並びにこれに用いるプログラム - Google Patents

検査装置および検査方法並びにこれに用いるプログラム Download PDF

Info

Publication number
WO2023089846A1
WO2023089846A1 PCT/JP2022/018250 JP2022018250W WO2023089846A1 WO 2023089846 A1 WO2023089846 A1 WO 2023089846A1 JP 2022018250 W JP2022018250 W JP 2022018250W WO 2023089846 A1 WO2023089846 A1 WO 2023089846A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
inspection
images
channel
captured
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/018250
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴之 石黒
仁 星野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Roxy Corp
Original Assignee
Roxy Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Roxy Corp filed Critical Roxy Corp
Priority to JP2023562110A priority Critical patent/JPWO2023089846A1/ja
Publication of WO2023089846A1 publication Critical patent/WO2023089846A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to an inspection apparatus and inspection method for inspecting the presence or absence of defects in an inspection target using an image of the inspection target, and a program used therefor.
  • Patent Document 1 discloses that a composite image is generated using a plurality of inspection images capable of detecting defects of different types, such as a specular reflection image, a diffuse reflection image, and a shape image. An inspection apparatus is described that inspects the presence or absence of defects in a workpiece using a composite image.
  • the non-defective part and the defective part in the composite image are set as follows. That is, among the composite image, a region included in the pixel value range recognized as a non-defective portion in a plurality of inspection images is set as a non-defective region, and a region not included in the pixel value range is set as a defective region. . This allows the inspector to detect different types of defects simply by looking at the composite image.
  • the presence or absence of defects is automatically determined from an image of the inspection object using a convolutional neural network. Also in the inspection apparatus described in the above publication, it is possible to automatically determine the presence or absence of a defect by inputting a synthesized image into a convolutional neural network.
  • a composite image is generated from a plurality of inspection images as in the inspection apparatus described in the above-mentioned publication, some of the plurality of information possessed by each of the plurality of inspection images may be missing.
  • the present invention has been made in view of the above, and one of its purposes is to provide a technology that contributes to improving learning accuracy and inspection accuracy. Another object of the present invention is to provide a technique that contributes to shortening the learning time and the inspection time.
  • an inspection apparatus is configured that inspects the presence or absence of defects in an inspection target using an image of the inspection target.
  • the inspection apparatus includes an image acquiring section, an inspection image generating section, a reversible image synthesizing section, a feature extractor, and a determiner.
  • the image acquisition unit is capable of acquiring at least one captured image of the inspection object.
  • the inspection image generator can generate at least first and second inspection images suitable for detecting defects to be inspected from the captured image.
  • the reversible image synthesizing unit combines the first image information of the first inspection image and the second image information of the second inspection image in a state in which the first and second inspection images are aligned.
  • the reversible image synthesizing unit can separate a first inspection image having all of the first image information, a second inspection image having all of the second image information, and a synthesized image.
  • the feature quantity extractor can extract feature quantities from the synthesized image using machine learning.
  • the determiner can determine the presence/absence of a defect to be inspected based on the feature quantity.
  • the "first inspection image” and the "second inspection image” in the present invention refer to a filtered image generated by subjecting a captured image to irreversible arithmetic filtering, or a captured image itself and a filtered image.
  • An image that has been subjected to any image processing including "Different imaging mode” refers to a mode of imaging with a combination of various cameras capable of imaging with light of different wavelengths and lighting capable of irradiating light of different wavelengths, or imaging with different camera positions or different lighting positions.
  • first image information and “second image information” in the present invention typically include the number of channels indicating the number of types of image information, position information indicating the physical position of an object to be inspected, A pixel value indicating information measured as each position corresponds to this.
  • the first and second inspection images are used to learn the feature amount extractor and the determiner.
  • learning time can be shortened.
  • inspection time can be shortened because the inspection can be performed only by the feature amount extractor and the determiner that have been trained using only the synthesized image. Since the synthesized image has all the image information (first image information and second image information) of each of the first and second inspection images, it is possible to realize stable learning and inspection of the feature amount extractor and the determiner. can be done. As a result, learning accuracy and inspection accuracy can be improved.
  • the feature extractor and the determiner can be trained using the first and second inspection images, respectively. It is possible to improve the learning accuracy and the inspection accuracy as compared with the configuration in which the Here, the "relationship between the first and second inspection images" typically corresponds to the relationship between the feature amount of the first inspection image and the feature amount of the second inspection image. For example, when the feature amount of the first inspection image is in the first state and the feature amount of the second inspection image is in the second state, it is determined that there is a defect in the inspection object. Things are mentioned.
  • the feature amount of the first inspection image and the second inspection image may be used. is in a predetermined relationship (the feature amount of the first inspection image is in the first state and the feature amount of the second inspection image is in the second state), it can be determined that there is a defect. become. Furthermore, since the composite image is generated by superimposing the first and second inspection images, positional information is shared between the superimposed images, so an increase in image size can be minimized. As a result, it is possible to prevent an increase in the learning time and inspection time of the feature quantity extractor and the determiner.
  • a synthesized image having the first image information and the second image information can be synthesized from the first inspection image having the first image information and the second inspection image having the second image information, and Since the synthesized image having the information and the second image information can be separated into the first inspection image having the first image information and the second inspection image having the second image information, the feature amount extractor and the judgment When learning the instrument, it is possible to annotate both the composite image and the first and second inspection images. The annotation is then shared between the composite image and the first and second inspection images. In other words, by annotating one of the composite image and the first and second inspection images, the other image can be annotated, that is, a common annotation can be affixed. can be done.
  • the composite image comprises at least first and second image channels.
  • the reversible image synthesizing unit assigns the first and second inspection images to the first and second image channels, respectively, thereby generating a synthetic image in which the first and second inspection images are synthesized.
  • the "image channel" in the present invention typically corresponds to a color channel or an alpha channel.
  • the first and second inspection images are only assigned to the first and second image channels, respectively, the first inspection image having all the first image information and the second image information having all the The first and second inspection images are superimposed in a state separable from the second inspection image in a state in which all the first and second image information are included and in a state in which the first and second inspection images are aligned. It is possible to easily realize a configuration capable of generating a combined image.
  • the inspection image generation unit can generate a third inspection image capable of detecting defects to be inspected from the captured image.
  • the composite image also has a third image channel.
  • the reversible image synthesizing unit assigns the first, second and third inspection images to the first, second and third image channels, respectively, thereby synthesizing the first, second and third inspection images. to generate a composite image.
  • the first, second, and third inspection images are only assigned to the first, second, and third image channels, respectively, the first inspection image having all the first image information, a state having all the first, second and third image information in a manner separable into a second inspection image having all the second image information and a third inspection image having all the third image information; It is possible to easily realize a configuration capable of generating a composite image in which the first, second and third inspection images are superimposed while the first, second and third inspection images are aligned.
  • an inspection method for inspecting the presence or absence of defects in an inspection target using an image of the inspection target includes (a) acquiring at least one captured image of an inspection target, (b) generating at least first and second inspection images capable of detecting defects of the inspection target from the captured image, and (c ) having all the first and second image information and the first and second image information in a manner separable into a first inspection image having all the first image information and a second inspection image having all the second image information generating a composite image in which the first and second test images are superimposed while the two inspection images are aligned; (d) extracting feature values from the composite image by a feature value extractor using machine learning; (e) Determining whether or not there is a defect to be inspected based on the feature amount using a determiner.
  • first inspection image and the “second inspection image” in the present invention refer to a filtered image generated by subjecting a captured image to irreversible arithmetic filtering, or a captured image itself and a filtered image. or when the captured images are a plurality of images captured in different imaging modes, the plurality of captured images themselves or the plurality of captured images are filtered.
  • An image that has been subjected to any image processing including "Different imaging mode” means a mode of capturing images with a combination of various cameras capable of capturing images with light of different wavelengths and lighting capable of irradiating light of different wavelengths, a mode of capturing images at different camera positions or different lighting positions, Imaging in different imaging modes with different camera exposure times and light irradiation times, imaging by irradiating different patterns of light onto the object to be inspected, and measuring the reflection time of light, sound waves, etc. to measure the object to be inspected
  • a mode of imaging a shape and the like correspond to this.
  • first image information and “second image information” in the present invention typically include the number of channels indicating the number of types of image information, position information indicating the physical position of an object to be inspected, A pixel value indicating information measured as each position corresponds to this.
  • first image information and “second image information” in the present invention typically include the number of channels indicating the number of types of image information, position information indicating the physical position of an object to be inspected, A pixel value indicating information measured as each position corresponds to this.
  • the first and second inspection images are used to learn the feature amount extractor and the determiner.
  • learning time can be shortened.
  • inspection time can be shortened because the inspection can be performed only by the feature amount extractor and the determiner that have been trained using only the synthesized image. Since the synthesized image has all the image information (first image information and second image information) of each of the first and second inspection images, it is possible to realize stable learning and inspection of the feature amount extractor and the determiner. can be done. As a result, learning accuracy and inspection accuracy can be improved.
  • the feature extractor and the determiner can be trained using the first and second inspection images, respectively. It is possible to improve the learning accuracy and the inspection accuracy as compared with the configuration in which the Here, the "relationship between the first and second inspection images" typically corresponds to the relationship between the feature amount of the first inspection image and the feature amount of the second inspection image. For example, when the feature amount of the first inspection image is in the first state and the feature amount of the second inspection image is in the second state, it is determined that there is a defect in the inspection object. Things are mentioned.
  • the feature amount of the first inspection image and the second inspection image may be used. is in a predetermined relationship (the feature amount of the first inspection image is in the first state and the feature amount of the second inspection image is in the second state), it can be determined that there is a defect. become. Furthermore, since the composite image is generated by superimposing the first and second inspection images, positional information is shared between the superimposed images, so an increase in image size can be minimized. As a result, it is possible to prevent an increase in the learning time and inspection time of the feature quantity extractor and the determiner.
  • a synthesized image having the first image information and the second image information can be synthesized from the first inspection image having the first image information and the second inspection image having the second image information, and Since the synthesized image having the information and the second image information can be separated into the first inspection image having the first image information and the second inspection image having the second image information, the feature amount extractor and the judgment When learning the instrument, it is possible to annotate both the composite image and the first and second inspection images. The annotation is then shared between the composite image and the first and second inspection images. In other words, by annotating one of the composite image and the first and second inspection images, the other image can be annotated, that is, a common annotation can be affixed. can be done.
  • step (c) assigns the first and second inspection images to at least first and second image channels of the combined image, respectively. generating a composite image with the second inspection image composited therewith;
  • the "image channel” in the present invention typically corresponds to a color channel and an alpha channel.
  • the composite image can be automatically generated simply by assigning the first and second inspection images to the first and second image channels, respectively.
  • a composite image can be generated by superimposing the first and second inspection images in a state in which the first and second inspection images are all aligned and the first and second inspection images are respectively It is possible to easily realize a configuration in which the first and second inspection images can be separated from the combined image while having all the first and second image information.
  • step (b) includes the step of generating a third inspection image capable of detecting defects to be inspected from the captured image. Then, step (c) assigns the first, second and third inspection images to the first, second and third image channels of the synthesized image, respectively. generating a composite image in which the second and third inspection images are combined;
  • the first, second, and third inspection images are only assigned to the first, second, and third image channels, respectively, the first inspection image having all the first image information, a state having all the first, second and third image information in a manner separable into a second inspection image having all the second image information and a third inspection image having all the third image information; It is possible to easily realize a configuration capable of generating a composite image in which the first, second and third inspection images are superimposed while the first, second and third inspection images are aligned.
  • a program for inspecting the presence or absence of defects in the inspection object using the image of the inspection object is configured.
  • the program is for causing one or a plurality of computers to execute each step of the inspection method according to the present invention in any one of the aspects described above.
  • the program may be recorded on a computer-readable recording medium such as a hard disk, ROM, SSD, flash memory (USB memory, SD card, etc.), floppy disk, CD, DVD, etc., or may be recorded on a transmission medium, For example, it may be distributed from one computer to another computer via a communication network such as the Internet or LAN, or may be exchanged in any other manner.
  • each step of the inspection method according to any one of the aspects of the present invention described above is executed by causing one computer to execute the program or by causing a plurality of computers to share and execute each step. Therefore, the same effect as the inspection method according to the present invention described above, for example, the effect that learning time and inspection time can be shortened, the effect that learning accuracy and inspection accuracy can be improved, It is possible to obtain the effect of being able to suppress the attachment of inappropriate annotations.
  • the present invention it is possible to improve learning accuracy and inspection accuracy. Moreover, according to the present invention, it is possible to shorten the learning time and the inspection time.
  • FIG. 1 is a configuration diagram showing a schematic configuration of a computer 1 functioning as an inspection device according to an embodiment of the present invention
  • FIG. 1 is a functional block diagram showing the functional configuration of a computer 1 functioning as an inspection device according to an embodiment of the present invention
  • FIG. 6 is an explanatory diagram showing an example of an image setting window 62
  • FIG. 11 is an explanatory diagram showing a selected image display window 63 when a synthesized image SYim is selected
  • FIG. 11 is an explanatory diagram showing a selected image display window 63 when a gloss suppression image GRim is selected
  • FIG. 9 is an explanatory diagram showing a selected image display window 63 when a shape-enhanced image Sim is selected
  • FIG. 11 is an explanatory diagram showing a selected image display window 63 when a stereoscopic image Zim is selected
  • 4 is a flowchart showing an example of defect detection processing
  • the inspection apparatus captures an image of an inspection object 90 conveyed by, for example, a belt conveyor BC with a camera 70 while being irradiated with pattern light from an illumination 72. It is an apparatus for inspecting the presence or absence of defects in the inspection object 90 using the captured image of the inspection object 90, and is configured as a general-purpose computer 1 centered around a CPU 2, a ROM 4, and a RAM 6.
  • FIG. The computer 1 functioning as an inspection apparatus according to the present embodiment includes a GPU 8 that performs calculation processing and matrix operation processing necessary when performing image processing, various application programs (simply referred to as applications), and various data including image data.
  • the computer 1 is an example of an implementation configuration corresponding to the "inspection device" in the present invention.
  • the computer 1 includes an input device 14 such as a keyboard and a mouse for the user to input various commands, a display 60 for displaying various information, and the like.
  • the CPU 2, the ROM 4, the RAM 6, the GPU 8, the HDD 10, the I/F 12, the input device 14, the display 60, etc. are electrically connected by a bus 80, and are configured to exchange various control signals and data with each other.
  • the computer 1 has a function of executing an operation according to the input operation when the user inputs a cursor or the like displayed on the display 60 through the input device 14 . Further, the computer 1 executes various processes by an application stored in the HDD 10, specifically, a defect detection application for detecting the presence or absence of a defect in the inspection object 90. function as an inspection apparatus according to the form of In the present embodiment, the configuration is such that the inspection device can be implemented by the computer 1, but it may be implemented as a dedicated device.
  • a defect detection application is an example of an implementation structure corresponding to the "program" in the present invention.
  • the computer 1 includes the above-described hardware resources such as the CPU 2, the ROM 4, the RAM 6, the GPU 8, the HDD 10, the I/F 12, the input device 14, and the display 60 (see FIG. 1).
  • the extractor 30, the determiner 32, the image display control section 34, the storage section 36, etc. are configured as functional blocks.
  • these units (the image acquisition unit 20, the inspection image generation unit 22, the reversible image synthesis unit 24, the annotation addition unit 26, the division unit 28, the feature extractor 30, the determination unit 32, the image display control unit 34, storage unit 36, etc.), each component (CPU 2, ROM 4, RAM 6, GPU 8, HDD 10, I/F 12, input device 14, display 60, etc.) can be said to be a function realized by operating alone or in cooperation.
  • the image acquiring unit 20, the inspection image generating unit 22, the reversible image synthesizing unit 24, the annotation adding unit 26, the dividing unit 28, the feature amount extractor 30, the determining unit 32, the image display control unit 34, the storage unit 36, etc. are electrically connected by bus lines 82 such as an address bus and a data bus.
  • the image acquisition unit 20 acquires a plurality of captured images of the inspection target 90 captured by the camera 70 .
  • the plurality of captured images are captured by the camera 70 each time the inspection target 90 is irradiated with a plurality of patterns of light (for example, irradiation from a plurality of angles, or a plurality of grid-like or striped lights) in sequence.
  • a plurality of patterns of light for example, irradiation from a plurality of angles, or a plurality of grid-like or striped lights
  • the camera 70 is a monochrome camera
  • the captured image is a grayscale image.
  • the inspection image generation unit 22 acquires a plurality of captured images from the image acquisition unit 20, and performs image processing on the acquired plurality of captured images (for example, using the principle of a phase shift measurement method) to obtain an inspection object.
  • a plurality of inspection images capable of detecting 90 defects for example, normal image Nim, specular reflection light image SRim, diffuse reflection light image DRim, shape emphasized image Sim, gloss suppression image GRim, stereoscopic image Zim, etc.
  • the normal image Nim is an average image
  • the specular reflected light image SRim is an image obtained by extracting only the specular reflected light component
  • the diffuse reflected light image DRim is an image obtained by extracting the diffuse reflected light component.
  • a gloss suppression image GRim is an image obtained by extracting a portion with a change in gloss
  • a stereoscopic image Zim is an image obtained by extracting height information of an object.
  • the inspection image generation unit 22 uses, for example, binarization processing, the Canny algorithm, or the like to use each inspection image (for example, a normal image Nim, a specular reflected light image SRim, or a diffuse image). position alignment of the reflected light image DRim, the shape-enhanced image Sim, the gloss-suppressed image GRim, the stereoscopic image Zim, etc.).
  • the stereoscopic image Zim, the specular reflected light image SRim, and the shape-enhanced image Sim correspond to the "first inspection image,” the "second inspection image,” and the "third inspection image” in the present invention. It is an example of a configuration.
  • the reversible image synthesizing unit 24 generates an inspection image generated by the inspection image generation unit 22 (for example, a normal image Nim, a specular reflection light image SRim, a diffuse reflection light image DRim, a shape emphasized image Sim, a gloss suppression image GRim, a three-dimensional image Zim, etc.) to generate a synthesized image SYim.
  • the reversible image synthesizing unit 24 allocates the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim to the R channel CHr, G channel CHg, and B channel CHb of the synthesized image SYim, respectively.
  • the inspection images normal image Nim, specular reflected light image SRim, diffuse reflected light image DRim, shape-enhanced image Sim, gloss-suppressed image GRim, stereoscopic image Zim, etc.
  • the inspection images normal image Nim, specular reflection light image SRim, diffuse reflection light image DRim, shape-enhanced image Sim, gloss-suppressed image GRim, stereoscopic image Zim, etc.
  • the inspection images normal image Nim, specular reflection light image SRim, diffuse reflection light image DRim, shape-enhanced image Sim, gloss-suppressed image GRim, stereoscopic image Zim, etc.
  • the synthetic image SYim is generated by allocating the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim to the R channel CHr, G channel CHg, and B channel CHb of the synthetic image SYim, respectively. Therefore, it is possible to generate a composite image SYim having all the image information (for example, the number of channels, position information, pixel values, etc.) of the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim.
  • each of the gloss suppression image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim has A state in which all image information (for example, the number of channels, position information, pixel values, etc.), in other words, the image information (for example, the number of channels , position information, pixel values, etc.) can be separated (extracted) from the composite image SYim into the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim.
  • the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim are assigned to the R channel CHr, the G channel CHg, and the B channel CHb while being aligned with each other.
  • the R channel CHr, G channel CHg, and B channel CHb are examples of implementation configurations corresponding to the "first image channel”, "second image channel”, and "third image channel” in the present invention.
  • image information (for example, number of channels, position information, pixel value, etc.) is an example of implementation configuration corresponding to "first image information" and "second image information" in the present invention.
  • the annotation adding unit 26 adds annotations to the composite image SYim, the gloss-suppressed image GRim assigned to the R channel CHr, the shape-enhanced image Sim assigned to the G channel CHg, and the stereoscopic image Zim assigned to the B channel CHb. .
  • Annotations given to any one of the synthesized image SYim, gloss suppression image GRim, shape-enhanced image Sim and stereoscopic image Zim are automatically reflected (given) to the other image.
  • common annotations can be attached to the composite image SYim, the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim.
  • the annotation adding unit 26 stores the annotated synthetic image SYim in the storage unit 36 together with the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim. Giving annotations to the synthesized image SYim, the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim is performed by the user's input operation via the input device 14 when learning the feature quantity extractor 30 and the determiner 32.
  • the user selects the defective area of the image of the inspection object 90 having the defect displayed on the display 60, that is, the defective area of the synthesized image SYim, the gloss suppression image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim. This is done by specifying by clicking, dragging, etc., and inputting the fact that it is a defect (for example, "mark").
  • the dividing unit 28 divides the synthesized image SYim generated by the reversible image synthesizing unit 24 into a plurality of divided images of a predetermined size.
  • the feature amount extractor 30 extracts feature amounts for the divided images.
  • machine learning more specifically, deep metric learning is performed so as to extract a feature amount that can accurately determine whether or not there is a defect in the inspection target 90.
  • the configuration is such that learning is performed by correcting the calculation parameters using That is, the feature amount extractor 30 applies deep metric learning to a series of processes for calculating the distance between the extracted feature amounts and correcting the parameters by the error backpropagation method and the gradient descent method based on the calculated distance. bottom.
  • the feature quantity extractor 30 performs learning in advance using a plurality of learning synthesized images SYim to which annotations have been added (with teacher signals), and stores them in the storage unit 36 as one trained model 31. Keep
  • the determiner 32 determines whether or not the inspection object 90 has a defect based on the feature quantity extracted by the feature quantity extractor 30 . Note that the determination result is displayed on the display 60 .
  • the determiner 32 includes, for example, gradient boosting, support vector machines, random forests, neural networks, Gaussian normalization, ensemble testing, and other machine learning other than machine learning using neural networks including deep learning, so-called statistics. target machine learning can be used.
  • the image display control unit 34 displays a predetermined window on the display 60 when the inspection application is started. In addition, the image display control unit 34 operates the image setting window 62 shown in FIG. The image display window 63 is displayed.
  • the selected image display window 63 displays images for inspection selected by the user's input operation via the input device 14 (composite image SYim (FIG. 4), gloss suppression image GRim (FIG. 5) forming the composite image SYim, A shape-enhanced image Sim (FIG. 6) and a stereoscopic image Zim (FIG. 7) are displayed. Furthermore, the image display control unit 34 displays the determination result by the determiner 32 in a determination result window (not shown).
  • the storage unit 36 is secured in at least one of the RAM 6 and the HDD 10, and stores inspection images generated by the inspection image generation unit 22 (for example, normal image Nim, specular reflection light image SRim, diffuse reflection light image DRim, shape enhancement image Sim, gloss-suppressed image GRim, stereoscopic image Zim, etc.), synthesized image SYim generated by reversible image synthesizing unit 24, and learned model 31 are stored.
  • the synthesized image SYim is combined with inspection images (normal image Nim, specular reflected light image SRim, shape-enhanced image Sim, and stereoscopic image Zim) assigned to the R channel CHr, G channel CHg, and B channel CHb. remembered.
  • FIG. 4 is a main flowchart showing an example of defect detection processing.
  • the defect detection process is mainly performed by the image acquisition unit 20 of the computer 1, the inspection image generation unit 22, the reversible image synthesis unit 24, the division unit 28, the feature amount extractor 30, the determination unit 32, the image display control unit 34, and It is executed by the storage unit 36 .
  • the image acquisition unit 20 first executes a process of acquiring a plurality of captured images captured by the camera 70 (step S10). Subsequently, the inspection image generation unit 22 generates a plurality of inspection images capable of detecting defects in the inspection target 90 from the acquired plurality of captured images. , a process of generating a stereoscopic image Zim is executed, and a process of storing the generated inspection images (gloss-suppressed image GRim, shape-enhanced image Sim, and stereoscopic image Zim) in the storage unit 36 is executed (step S12). .
  • the reversible image synthesizing unit 24 follows the user's instruction in the image setting window 62 to transfer the gloss-suppressed image GRim to the R channel.
  • a process of allocating the shape-enhanced image Sim to the G channel CHg and the stereoscopic image Zim to the B channel CHb is executed for CHr (step S14).
  • a composite image SYim having all the image information (for example, the number of channels, position information, pixel values, etc.) of the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim is generated.
  • the reversible image combining unit 24 stores the generated combined image SYim in the storage unit 36 .
  • the dividing unit 28 executes processing for dividing the composite image SYim into a plurality of divided images of a predetermined size (step S16). Then, the feature amount extractor 30 executes processing for extracting feature amounts for the plurality of divided images using the trained model 31 (step S18). Subsequently, the determiner 32 determines whether or not there is a defect in the inspection object 90 based on the extracted feature amount (step S20), and the image display control unit 34 outputs the determination result. After executing the process (step S22), the process ends.
  • inspection images normal image Nim, specular reflected light image SRim, and , shape-enhanced image Sim
  • inspection images gloss-suppressed image GRim, shape-enhanced image Sim, and stereoscopic image Zim
  • inspection can be performed only by the feature amount extractor 30 and the determiner 32 that are trained using only the synthesized image SYim, so the inspection time can be shortened.
  • the synthesized image SYim has all of the image information (for example, the number of channels, position information, pixel values, etc.) of each of the inspection images (the gloss suppression image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim)
  • the feature amount Stable learning and inspection of the extractor 30 and the determiner 32 can be realized, and the relationship between inspection images (gloss suppression image GRim, shape emphasized image Sim, and stereoscopic image Zim), that is, each inspection It is also possible to learn the relationship between the feature amounts of the original image (the feature amount of the gloss-suppressed image GRim, the feature amount of the shape-enhanced image Sim, and the feature amount of the stereoscopic image Zim).
  • the defect of the inspection object 90 can be detected only with the feature amount of each inspection image (the feature amount of the gloss suppression image GRim, the feature amount of the shape-enhanced image Sim, and the feature amount of the stereoscopic image Zim). is difficult, the relationship between the feature amounts of each inspection image (the feature amount of the gloss-suppressed image GRim, the feature amount of the shape-enhanced image Sim, and the feature amount of the stereoscopic image Zim) , it is possible to determine whether or not the inspection object 90 has a defect. As a result, learning accuracy and inspection accuracy can be improved.
  • each inspection image (the gloss suppression image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim) is superimposed, that is, the position information is stored between the superimposed images. is shared, the increase in image size can be minimized. As a result, it is possible to prevent the learning time of the feature extractor 30 and the determiner 32 and the learning time and inspection time of the feature extractor 30 and the determiner 32 from increasing.
  • image information for example, the number of channels, position information, pixel values, etc.
  • image information for example, the number of channels, position information, pixel values, etc.
  • the inspection images gloss-suppressed image GRim, shape-enhanced image Sim, and stereoscopic image Zim
  • R channel CHr the number of channels, position information, pixel values, etc.
  • image information for example, channel number, position information, pixel value, etc.
  • Image information for example, the number of channels, position information, pixel values, etc.
  • each inspection image gloss suppression image GRim, shape-enhanced image Sim, and stereoscopic image Zim
  • each inspection image gloss suppression image GRim, shape-enhanced image Sim, and stereoscopic image Zim
  • a plurality of captured images are acquired by varying the irradiation pattern of the illumination 72 that irradiates the inspection target 90, but the present invention is not limited to this.
  • a configuration that acquires a plurality of captured images by varying the position of the illumination 72 that irradiates the inspection target 90 or a configuration that acquires a plurality of captured images by varying the type of illumination 72 that irradiates the inspection target 90.
  • a configuration for acquiring a plurality of captured images by varying the imaging mode of the camera 70 a configuration for acquiring a plurality of captured images by varying the position of the camera 70, and a configuration for acquiring a plurality of captured images by varying the type of the camera 70.
  • a configuration for acquiring an image may also be used.
  • a mode for changing the positions of the camera 70 and the lighting 72 there is a mode in which the positions of one camera 70 and one lighting 72 are changed each time an image of the inspection target 90 is taken.
  • a mode is conceivable in which the inspection object 90 is imaged while sequentially switching the plurality of cameras 70 and the plurality of illuminations 72 .
  • a plurality of inspection images (for example, a normal image Nim, a regular reflected light image SRim, a diffuse reflected light image DRim, a shape-enhanced image Sim, and a gloss suppression image are generated by performing image processing on a plurality of captured images.
  • GRim stereoscopic image Zim, etc.
  • a plurality of inspection images may be generated by subjecting one captured image to irreversible arithmetic filtering, or one captured image and the captured image may be subjected to irreversible arithmetic filtering.
  • a plurality of inspection images may be generated by irreversibly synthesizing the processed images.
  • the plurality of captured images themselves may be used as the plurality of inspection images.
  • the inspection image generation unit 22 executes processing for selecting an image from which the presence or absence of defects in the inspection object 90 can be detected from the plurality of captured images acquired by the image acquisition unit 20 .
  • the inspection image generation unit 22 aligns each inspection image when generating the inspection image from the captured image.
  • the present invention is not limited to this.
  • the reversible image synthesizing unit 24 may align the respective inspection images.
  • three inspection images are divided into three image channels (R channel CHr, G channel CHg, and B channel CHb).
  • the configuration is not limited to this.
  • two inspection images may be assigned to two image channels, or four or more inspection images may be assigned to four or more image channels.
  • the number of image channels does not necessarily have to be equal to the number of inspection images, and may be equal to or greater than the number of inspection images. If the number of image channels is equal to or greater than the number of inspection images, inspection images can be assigned to any image channel, and image channels to which inspection images have not been assigned can be left blank.
  • the gloss-suppressed image GRim, the shape-enhanced image Sim, and the stereoscopic image Zim are assigned to the R channel CHr, G channel CHg, and B channel CHb of the composite image SYim, respectively.
  • the normal image Nim is assigned to the G channel CHg
  • the specular image SRim is assigned to the B channel CHb
  • the shape-enhanced image Sim is assigned to the R channel CHr.
  • the images assigned to the R channel CHr, the G channel CHg, and the B channel CHb of the composite image SYim are assigned to the plurality of generated examinations. can be arbitrarily selected from the images for
  • RGB red, green, blue, blue, and red, blue, a suppressed image GRim, a stereoscopic image Zim, etc.
  • RGB red, green, blue, blue, a suppressed image GRim, a stereoscopic image Zim, etc.
  • R channels CHr, G An alpha channel may be used in addition to or in place of the channel CHg and the B channel CHb.
  • CMYK which is a color channel
  • an inspection image for example, a normal image Nim, a specular reflected light image SRim, a diffuse reflected light image DRim, a shape-enhanced image Sim, a gloss-suppressed image GRim, a stereoscopic image Zim, etc.
  • system C, M, Y, and K channels may be used, or HSV system H, S, and V channels consisting of three components of hue, saturation, and brightness may be used, Alternatively, the HSL H, S, and L channels, which consist of three components of hue, saturation, and luminance, may be used.
  • various applications including the defect detection application are stored in the HDD 10, but the configuration is not limited to this.
  • various applications including the defect detection application may be configured to be distributed from other computers to the computer 1 as the inspection apparatus according to the present embodiment via a transmission medium, for example, a communication network such as the Internet or LAN.
  • This embodiment shows an example of a form for carrying out the present invention. Therefore, the present invention is not limited to the configuration of this embodiment.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
PCT/JP2022/018250 2021-11-22 2022-04-20 検査装置および検査方法並びにこれに用いるプログラム Ceased WO2023089846A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2023562110A JPWO2023089846A1 (https=) 2021-11-22 2022-04-20

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021189118 2021-11-22
JP2021-189118 2021-11-22

Publications (1)

Publication Number Publication Date
WO2023089846A1 true WO2023089846A1 (ja) 2023-05-25

Family

ID=86396563

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/018250 Ceased WO2023089846A1 (ja) 2021-11-22 2022-04-20 検査装置および検査方法並びにこれに用いるプログラム

Country Status (2)

Country Link
JP (1) JPWO2023089846A1 (https=)
WO (1) WO2023089846A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000275185A (ja) * 1999-03-26 2000-10-06 Toshiba Eng Co Ltd 欠陥検査装置及び欠陥検査方法
JP2017049974A (ja) * 2015-09-04 2017-03-09 キヤノン株式会社 識別器生成装置、良否判定方法、およびプログラム
WO2020085327A1 (ja) * 2018-10-23 2020-04-30 旭化成株式会社 診断装置及び診断方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000275185A (ja) * 1999-03-26 2000-10-06 Toshiba Eng Co Ltd 欠陥検査装置及び欠陥検査方法
JP2017049974A (ja) * 2015-09-04 2017-03-09 キヤノン株式会社 識別器生成装置、良否判定方法、およびプログラム
WO2020085327A1 (ja) * 2018-10-23 2020-04-30 旭化成株式会社 診断装置及び診断方法

Also Published As

Publication number Publication date
JPWO2023089846A1 (https=) 2023-05-25

Similar Documents

Publication Publication Date Title
JPWO2019151393A1 (ja) 食品検査システム、食品検査プログラム、食品検査方法および食品生産方法
US20170091925A1 (en) Detecting Defects on a Wafer Using Defect-Specific and Multi-Channel Information
CN110658198A (zh) 光学检测方法、光学检测装置及光学检测系统
JP6256249B2 (ja) 計測装置、基板検査装置、及びその制御方法
TWI327292B (en) Board inspection apparatus and method and apparatus for setting inspection logic thereof
JP2016502750A5 (https=)
JP2017049974A (ja) 識別器生成装置、良否判定方法、およびプログラム
CN107315011A (zh) 图像处理装置、图像处理方法及存储介质
KR20120014886A (ko) 검사 레시피 생성 및 검사 레시피를 기반으로 한 검사
CN117274258B (zh) 主板图像的缺陷检测方法、系统、设备及存储介质
JP2020112483A (ja) 外観検査システム、計算モデル構築方法及び計算モデル構築プログラム
JP2023008416A (ja) 異常検知システムおよび異常検知方法
JP2022114331A (ja) 検査システム、検査方法及び検査プログラム
JP7718429B2 (ja) 指標選択装置、情報処理装置、情報処理システム、検査装置、検査システム、指標選択方法、および指標選択プログラム
WO2023089846A1 (ja) 検査装置および検査方法並びにこれに用いるプログラム
WO2022201968A1 (ja) 情報処理装置、制御プログラム、および制御方法
CN114689604B (zh) 具有光滑表面的待测物于光学检测上的影像处理方法及其检测系统
JPWO2018198828A1 (ja) 欠陥検査用画像の生成装置および欠陥検査用画像の生成方法
JP2023152482A (ja) 検査装置、検査方法及び検査用コンピュータプログラム
JP2024075062A (ja) 外観検査装置および外観検査用識別器の生成方法
JP2022123217A (ja) 画像検査装置、画像検査方法、及び学習済みモデル生成装置
JP2023152476A (ja) 検査装置、検査方法及び検査用コンピュータプログラム
JP7771768B2 (ja) 機械学習装置、機械学習方法、機械学習プログラム、検査装置、検査方法、および検査プログラム
JP2023140822A (ja) 検査装置、検査方法及びプログラム
KR102171773B1 (ko) 검사 영역 결정 방법 및 이를 이용하는 외관 검사 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22895135

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023562110

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05/09/2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22895135

Country of ref document: EP

Kind code of ref document: A1