CN109916906B - Defect detection device and method - Google Patents
Defect detection device and method Download PDFInfo
- Publication number
- CN109916906B CN109916906B CN201811636492.8A CN201811636492A CN109916906B CN 109916906 B CN109916906 B CN 109916906B CN 201811636492 A CN201811636492 A CN 201811636492A CN 109916906 B CN109916906 B CN 109916906B
- Authority
- CN
- China
- Prior art keywords
- defect
- image
- defect detection
- images
- learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/956—Inspecting patterns on the surface of objects
- G01N21/95607—Inspecting patterns on the surface of objects using a comparative method
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Abstract
The invention provides a defect detection device and a method thereof, according to an embodiment, the defect detection device comprises: a storage part storing a predetermined one of the standard images and at least one photographed image obtained by photographing the inspection object; and a control unit that calculates a difference between the standard image and each of the captured images, generates a difference object corresponding to each of the captured images, synthesizes the standard image, each of the captured images, and each of the difference images, generates at least one color image corresponding to each of the captured images, and performs at least one of learning of a defect detection model and defect detection using the generated color image.
Description
Technical Field
The present invention relates to a defect detection apparatus and method, and more particularly, to a defect detection apparatus and method for performing defect training and detection by using an image processing method capable of improving defect detection performance when detecting a defect included in an object using a captured image of the object under inspection.
Background
There are various ways of detecting whether a mass-produced product having uniform quality is defective. Such as intermediate or final products in the production process of printed products such as packaging paper or label paper, printed circuit boards or assembled circuits and the like, image analysis techniques are actively applied to the actual products or the products with the same shape, so as to reduce the time and labor required for detecting defects and improve the detection quality.
At present, the image analysis technology of artificial intelligence is used for detecting the defects of products, thereby obtaining the technical achievements of reducing false detection or over-detection rate and the like.
As described above, a general method for detecting a product defect using an artificial intelligence image analysis technique is to learn a model necessary for determining a defect using a sufficient number of images of normal and/or defective products, and then determine whether an image of an inspection target is normal or defective using the learned model. However, the existing defect training and detecting method cannot identify all the defects, so that the overall defect detecting performance is poor.
For example, according to the conventional defect detection method of the printed circuit board, as shown in fig. 1, a model is learned using a photographed image 10 photographed and obtained by a printed circuit board to be produced, a position of a defect 11 is marked, a defective region and a background region are trained, and then the defect detection model learns a non-defective background region and a defective region, and then when a photographed image of a test object is input, the photographed image including whether the defect is included or not may be output.
However, in the conventional defect detection method for the printed circuit board, a defect detection rate for a structure such as OPEN (OPEN; defect occurring when the regions to which the circuit patterns should be connected are disconnected) or SHORT (SHORT; defect occurring when the circuit patterns should not be connected to each other are connected to each other) is not high, and a defect at a position where a HOLE (HOLE) should be formed, a SPACE (SPACE) generated by maintaining an interval between the circuit patterns insufficiently and a defect cannot be detected.
Another method for detecting product defects using an artificial intelligence image analysis technique is to generate a Difference image using a sufficient number of captured images of normal and/or defective products and a preset standard image, learn a model using the Difference image, and then determine whether the image of the inspection target is normal or defective using the learned model. However, even in such a method, the detection rate of the structural defect is not sufficiently improved, and the defect detection rate is reduced because the color or brightness of a part of the region of the differential image is similar to that of the background or the edge portion.
A related prior art document, korean registered patent No. 10-1128322, relates to an optical inspection apparatus and method for a printed circuit board, and describes a technique for comparing a standard PCB pattern image with a photographed PCB pattern image to determine whether it is defective or not. However, in the prior art, the brightness mode of the standard image and the shot image is compared with a numerical value to judge whether the standard image and the shot image are normal or not, so that the tiny defects are difficult to detect, and the types or the correct positions of the defects cannot be distinguished. There is therefore a need for a technique that can solve the above-mentioned problems.
The above background is technical information which is owned by the inventor or grasped in obtaining the present invention, and is not a known technology which has been disclosed to the general public between applications of the present invention.
Disclosure of Invention
Technical problem
An object of the embodiments disclosed in the present specification is to provide a defect detecting apparatus and method.
An object of the embodiments disclosed in the present specification is to improve defect detection performance of an inspection object using a 3-channel color image.
An object of the embodiments disclosed in the present specification is to determine the type of a defect by learning the type of a detected defect.
An object of the embodiments disclosed in the present invention is to improve accuracy of defect detection and classification using defect detection models that are learned separately from defect positions.
Technical scheme
In order to solve the above problem, the present invention adopts a technical solution that, as an apparatus for detecting a defect of an inspection object according to an embodiment, includes: a storage part storing a predetermined one of the standard images and at least one photographed image obtained by photographing the test object; and a control unit that calculates a difference between the standard image and each of the captured images, generates a difference object corresponding to each of the captured images, synthesizes the standard image, each of the captured images, and each of the difference images, generates at least one color image corresponding to each of the captured images, and performs at least one of learning a defect detection model and detecting a defect using the generated color image.
In order to solve the technical problem, a technical solution adopted by the present invention is a method for detecting a defect of an inspection object, implemented by a defect detecting apparatus according to an embodiment, including: a step of obtaining at least one photographed image generated by photographing the test object; a step of generating a difference image corresponding to each photographed image using a difference between each photographed image and a predetermined one of the standard images; synthesizing the standard image, the captured images, and the difference images to generate at least one color image corresponding to each captured image; and a step of performing at least one of learning of a defect detection model or defect detection using the generated color image.
Further, to solve the technical problem, an aspect of the present invention is a defect detection method in a computer-readable recording medium recording a program for implementing the defect detection method, according to another embodiment, including: a step of obtaining at least one photographed image generated by photographing the test object; a step of generating a difference image corresponding to each photographed image using a difference between each photographed image and a predetermined one of the standard images; synthesizing the standard image, the captured images, and the difference images to generate at least one color image corresponding to each captured image; and a step of performing at least one of learning of a defect detection model or defect detection using the generated color image.
In order to solve the technical problem, the present invention adopts a technical solution that, according to another embodiment, in a computer program implemented by a defect detecting apparatus and stored in a medium for implementing a defect detecting method, the defect detecting method includes: a step of obtaining at least one photographed image generated by photographing the test object; a step of generating a difference image corresponding to each photographed image using a difference between each photographed image and a predetermined one of the standard images; synthesizing the standard image, the captured images, and the difference images to generate at least one color image corresponding to each captured image; and a step of performing at least one of learning of a defect detection model or defect detection using the generated color image.
Advantageous effects
According to one of the above technical solutions, a defect detection apparatus and method can be provided.
According to one of the technical schemes, the defect detection performance of the inspection object is improved by using the 3-channel color image.
According to the first technical solution, a defect detecting apparatus and a method for determining a defect type by learning a detected defect type can be provided.
According to one of the above technical solutions, the accuracy of defect detection and classification is improved by using defect detection models respectively learned according to defect positions.
Effects obtainable by the disclosed embodiments are not limited to the above effects, and other effects not referred to will be apparent to those of ordinary skill in the art from the following description.
Drawings
FIG. 1 is an explanatory view illustrating an example of a photographed image of a printed circuit board marking a defective position;
FIG. 2 is a block diagram illustrating the structure of a defect detection apparatus of an embodiment;
fig. 3 is an exemplary view illustrating an example of a reference image, a captured image, and a difference image for defect detection in the defect detection apparatus according to the embodiment;
fig. 4 is an explanatory view illustrating an example of a color image generated using the reference image, the captured image, and the difference image in the detection apparatus of the embodiment;
FIG. 5 is an illustration of a process for learning the types of defects detected in the defect detection apparatus according to one embodiment;
fig. 6 and 7 are a flow chart illustrating the steps of a defect detection method of an embodiment.
Description of the symbols
100: a defect detecting device;
110: an input/output unit;
120: a storage unit;
130: a communication unit;
140: a control unit;
200: a photographing device.
Detailed Description
Various embodiments will be described in detail below with reference to the accompanying drawings. The embodiments described below can be modified into other embodiments. For more specific explanation, details which are widely known to those skilled in the art to which the following embodiments belong will not be described in detail. In the drawings, portions not related to the description of the embodiments are omitted, and like reference numerals are used for like portions in the specification.
The description that a certain element is "connected" to another element means that the element is not directly connected but may be connected to another element. When a certain element is described as being "included" in the specification, it is not intended to exclude other elements unless the description is specifically contrary to the description, but it is also intended to include other elements.
Various embodiments are described in detail below with reference to the following figures.
The following terms are defined before the description.
The "captured image" is an image obtained by directly capturing the test object. Hereinafter, a printed circuit board will be described as an "inspection target", but any product may be used as long as it has a predetermined pattern in production, such as an intermediate or final product of an electronic product, or a printed product such as a wrapping paper or a label paper. The captured image is not limited to a simple image captured by visible light, and may be an image obtained by various optical imaging methods such as X-ray and infrared rays depending on the type and properties of the detection target.
The "standard image" is an image showing an ideal pattern that the test object needs to have, and is an actual standard of the captured images of the respective test objects, and the captured images of any test object are used together with the standard image to generate a differential image described later.
The "difference image" is an image generated using the captured image and the standard image, and is calculated using the difference between the captured image and the standard image. For example, for a binary image, the difference object may be generated only by the difference between the respective pixel values of the two images, and for a non-binary image, it may be obtained directly using the difference between the respective pixel values or using the result of comparison with a threshold value. The pixel value is a luminance value of each pixel included in the image, and may have one of 8-bit values, i.e., 0 to 255. The differential image may be obtained not only by the above-described simple subtraction, but also by giving a weighted value to the value of the captured image or the standard image as needed, or by converting the difference between the pixel values into a plurality of gradation values by using two or more threshold values. As described above, in one embodiment, the difference image may be calculated in any manner as long as it can appropriately compensate for the type or property of the inspection object in the calculation method. Further, in order to obtain the difference image, a preprocessing process may be performed on at least one of the captured image and the standard image.
When there are a plurality of captured images, the difference images are generated in the number corresponding to each captured image.
The "color image" below is an image in which the captured image, the standard image, and the differential image are each a color channel. To form a color image, the photographed image, the standard image, and the differential image forming each image channel may be formed as 8-bit grayscale images, respectively. Therefore, if necessary, when the defect detection device described later is a captured image, the standard image, and the difference image are not 8-bit grayscale images, the captured image and the standard image can be converted into 8-bit grayscale images. The 8-bit gray image at this time means a 1-channel image in which each pixel value has a value of 8 bits, i.e., one luminance value among 0 to 255. That is, there is no color channel, and an image is constituted with pixels having one luminance value among 0 to 255.
In addition, a color image may be formed as a 3-channel 24-bit image in which a captured image, a standard image, and a differential image formed from the 8-bit grayscale image described above are formed as different color channels, respectively. For example, the standard image may form a RED (RED) channel, the photographed image may form a GREEN (GREEN) channel, the differential image may form a BLUE (BLUE) channel, and each pixel value of each channel may display a luminance value of a corresponding color of each pixel of the color image. In this case, since the captured image is plural, a color image can be generated corresponding to each captured image and the difference image.
The color image is then used for learning a defect detection model in the defect detection apparatus and method of an embodiment. Furthermore, the color image finally generated by using the shot image of the detection object can be used for detecting the defect of the actual detection object by using the learned defect detection model.
Other words to be described than the above-defined words are described below.
Fig. 2 is a block diagram illustrating the structure of a defect detection apparatus of an embodiment.
The defect inspection apparatus 100 is an apparatus for inspecting a test object for defects by learning and inspecting the defects by using the learned model.
The defect detection apparatus 100 may be implemented by an electronic terminal or a server-client system (or cloud system), which may include an electronic terminal that installs a web service application required for interaction with a user.
The electronic terminal 10 may be implemented as a computer, a portable terminal, a television, a Wearable Device (Wearable Device), or the like, which is connected to a remote server through a network (N), or may be connected to other terminals and servers. Examples of the computer include a notebook computer, a desktop computer (desktop), and a laptop computer (laptop) equipped with a WEB Browser (WEB Browser), and a portable terminal (laptop) such as a Wireless Communication device capable of ensuring portability and mobility may include PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal portable System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile telecommunications) -2000, CDMA (Code Division Multiple Access) -2000, W-CDMA (W-Code Division Multiple Access), wibro (Wireless Broadband Access), internet Smart Phone (Smart Phone), and WiMAX (Mobile world Interoperability) based Wireless Communication devices. The TV may include IPTV (Internet Protocol Television), internet TV (Internet Television), wireless TV, cable TV, and the like. The wearable device is an information processing device which can be worn by a human body directly, such as a watch, glasses, ornaments, clothes, shoes and the like, and is connected to a remote server or other terminals directly or through other information processing devices through a network.
The defect detecting apparatus 100 of an embodiment includes at least a part of the input/output section 110, the storage section 120, the communication section 130, and the control section 140.
The input/output unit 110 may include an input unit for obtaining user input required in a learning process or a defect inspection process for inspecting a defect of an inspection object or required for selecting a desired document, and an output unit for displaying information such as an operation execution result or a state of the defect inspection apparatus 100. For example, the input/output unit 110 may include an operation panel (operation panel) for receiving user input, a display panel (display panel) for displaying a screen, and the like.
Specifically, the input unit may include various types of devices for receiving user input, such as a keyboard, a physical button, a touch screen, a camera, or a microphone. The output section may include a display panel or a speaker, etc. The input/output unit 110 may include various structures for supporting input/output.
The defect detection apparatus 100 may include a storage section 120. The storage unit 120 may temporarily store a plurality of captured images and standard images required for learning the defect detection model, and may temporarily store at least a difference image generated by using the captured images and the standard images. Further, at least the captured image and the standard image and the color image obtained using the differential image may be temporarily stored. In the process of learning the defect detection model using the color image, the memory unit 120 updates and stores the learned model each time learning is performed. The storage unit 120 may store each defect inspection model in a separate inspection file.
The memory 120 stores the color image with the defect locations marked for the learning process.
The defect detection apparatus 100 may further include a communication section 130.
The communication section 130 may implement wireless communication with other devices or networks. To this end, the communication part 130 may include a communication module supporting at least one of various wired and wireless communication methods. For example, the communication module may be implemented in the form of a chipset (chipset).
Examples of the Wireless Communication supported by the Communication unit 130 include Wi-Fi (Wireless Fidelity), wi-Fi Direct (Wi-Fi Direct), bluetooth (Bluetooth), UWB (Ultra Wide Band) and NFC (Near Field Communication). The wired communication supported by the communication unit 130 is, for example, USB (universal serial bus), HDMI (High Definition Multimedia Interface), or the like.
In particular, the communication unit 130 may communicate with the camera 200 to receive the photographed image from the camera 200. In this case, the imaging device 200 is a device provided with an optical device such as a camera. In this case, the imaging device 200 may be a device that can image only light having a wavelength in the visible spectrum, or may be a device that can image light having another wavelength such as infrared rays or X-rays.
The control unit 140 controls the overall operation of the defect detection apparatus 100 and may include a processor such as a CPU. The control unit 140 may control other components included in the defect detection apparatus 100, for example, to perform an operation corresponding to the user input received through the input/output unit 110.
For example, the control unit 140 may execute a program stored in the storage unit 120, read a file stored in the storage unit 120, or store a new file in the storage unit 120.
According to an embodiment described in the present specification, the control section 140 may obtain a captured image. The captured image is an image obtained by capturing an inspection object as described above, and is generated by the above-described image capturing device 200 and transmitted to the defect detecting device 100. The captured image at this time may include a plurality of images captured by a plurality of test objects, respectively.
As shown in fig. 3, the control unit 140 may generate a difference image (c) corresponding to each captured image using the standard image (a) and each captured image (b). As described above, the difference image (c) is calculated by the control unit 140 by calculating the difference between the captured image (b) and the standard image (a).
For example, the standard image (a) may be stored in the storage unit 120 as an 8-bit grayscale image, and the control unit 140 may convert each captured image (b) into an 8-bit grayscale image. The control unit 140 subtracts the pixel values of the captured images (b) from the pixel values of the standard image (a) to generate a difference image (c) from an 8-bit grayscale image having the calculated values as pixel values.
The control unit 140 performs an additional operation such as addition of a predetermined weight to the pixel value of the standard image (a) or the captured image (b) or comparison of the value calculated by the subtraction with one or more threshold values to associate each pixel value with a plurality of predetermined level values, thereby generating a difference image (c).
Further, as shown in fig. 4, the control unit 140 may obtain a color image (d) from the standard image (c) generated through the above-described process, each captured image (b), and the difference image (c) corresponding to each captured image (b). In this case, the color images (d) may be generated in the number corresponding to each captured image (b).
Specifically, the control section 140 may generate a 3-channel color image (d) having the standard image (a), each captured image (b), and the difference object (c) corresponding to each captured image (b) as a color channel. In this case, each of the captured image (b), the standard image (a), and the difference image (c) corresponding to each captured image can be formed as an 8-bit grayscale image as described above, and when some of them do not match the 8-bit grayscale image, the control section 140 converts the image into an 8-bit grayscale image to prepare for generating a color image (d).
Then, the control section 140 generates a 3-channel 24-bit color image (d) in which each 8-bit grayscale image is regarded as a red channel, a green channel, and a blue channel. The generated color image (d) may be an image in which the pixel values of the captured images (b), the standard image (a), and the difference image (c) corresponding to the captured images (b) are set as the red, green, and blue luminance values of the pixels of the color image (d). According to an embodiment, three images including the standard image (a), each of the photographed images (b), and the differential image (c) corresponding to each of the photographed images (b) may respectively form mutually different color channels in a different order from the above-described red, green, and blue.
The control unit 140 performs learning of the defect detection model using the one or more color images (d) generated as described above. In this case, the control unit 140 first performs learning of a defect detection model so that the suspected defect positions can be detected from the color image (d) at a time.
In this case, the learning color image (d) used by the control unit 140 to perform the learning of the defect detection model may be marked with the defective position. For this purpose, the defect detection apparatus 100 may receive the defect position information included in each color image (d) as a learning object by inputting it from the user through the input/output section 110, or receive the color image (d) in which the defect position has been marked through the communication section 130 by another apparatus.
The control unit 140 may perform learning to learn the learning target color image (d) for marking the defect position and recognize the suspected defect position at a time. Specifically, the control unit 140 may perform learning of a defect detection model for identifying a suspected defect position, and for example, the control unit 140 may perform learning of a defect detection model composed of an artificial neural network using machine technology, for example, using deep learning technology. The control unit 140 learns the defect detection model by a process of obtaining a defect detection model constant value using the information of the defect suspected position as a result value, using each pixel value of the 3-channel 24-bit color image (d) as an input variable.
Further, the defect detection model is learned so that the suspected position of a defect can be found in the color image (d).
The defect detection model learned by the control unit 140 at this time may have characteristics such as the appearance, area, relative relationship with peripheral pixel values, and the like of adjacent pixels limited within a specific range based on RGB values to detect the suspected position of a defect. For example, in the example shown in fig. 4, after yellow dots included in the color image (d) are marked as defect positions, the defect detection model is learned using these images. The yellow dots can display the leak defects in the printed circuit board, and the common color, size, appearance, position characteristics and the like of the leak defects can be learned through the defect detection model. Further, when a plurality of color images (d) including other types of defects are learned in addition to the leak defect, the defect detection model learns various types of defects to detect the defect suspicious site.
The control unit 140 additionally performs learning of a defect detection model using a trimming image created by trimming a predetermined region with the defect suspected position as the center.
For example, as shown in fig. 5, the control unit 140 may cut a certain region of the color image (d) to generate a cut picture (e). In this case, the trimming image (e) is, for example, a region image having a predetermined size with the defect suspicious location as the center.
The control unit 140 obtains a trimming image (e) from the learning target color image (d) including the defect with the defect position as the center, and then performs learning of the defect detection model using the trimming image (e). At this time, whether the image (e) is defective or not and/or the type of defect may be marked on the trimming image (e). Specifically, as shown in fig. 5, when the defect in the trimming image (e) is a non-defect that has been detected, the user selects "0. Pass" and if it is a defect, one of a plurality of defect type selection buttons (CLS) such as "1.rc", "2. Bump", "3. Short" is selected according to the type of the defect, and the defect or the defect type of the trimming image (e) is marked. The learning of the defect detection model can be further performed using the trimming image (e) marked as described above for the presence or absence of defects or the type of defects.
Furthermore, the defect detection model not only learns to judge whether the defect exists more accurately, but also can distinguish the type of the defect.
In the example shown in fig. 5, since the defect included in the trimming image (e) is a leak defect, the user can select the "9. Leak" button, and the control unit 140 performs the learning of the defect detection model by marking the trimming image (e) as a leak.
In this case, the defect detection model for learning the color image for detecting the suspected defect position and the defect detection model for learning the trimming image for detecting the presence or absence and/or type of the defect may be formed by different detection models.
The defect inspection apparatus 100 according to an embodiment can perform learning of a defect inspection model with different criteria according to the defect suspicious location. For example, when detecting a defect of a printed circuit board, a certain degree of defect is allowed for the outer contour of the printed circuit board, but the defect is discriminated on a strict standard for a circuit region.
The control section 140 divides the color image (d) into a plurality of regions according to the actual configuration of the printed circuit board corresponding to the color image (d). For example, after the color image (d) region is divided into a plurality of regions such as an outline, a Mesh (Mesh), a circuit, a bonding pad, and a pad in advance, it is possible to determine which region the suspected defect position detected corresponds to. In this case, the control unit 140 may learn different detection models for each region in order to set the defect detection criteria differently for each region, and the different detection models for each region may be stored in the storage unit 120 in the form of different detection documents.
The control unit 140 performs learning of a defect detection model corresponding to a region including the defect-suspected location using the trimmed image (e) of the defect-suspected location.
Therefore, different defect detection models are learned according to which region the region including the defect suspicious position corresponds to, so that different detection standards can be set according to the defect suspicious position.
Further, the control unit 140 performs defect detection using the defect detection model learned as described above. For this reason, the defect detection apparatus 100 can obtain a photographed image (b) of the photographic subject for distinguishing whether an actual defect is present or absent. As shown in fig. 3, the defect detection apparatus 100 generates a difference image (c) by using the difference between the acquired captured image (b) and the standard image (a).
As shown in fig. 4, the defect detection apparatus 100 can generate a 3-channel 24-bit color image (d) in which the captured image (b), the standard images (a), and the difference image (c) are RGB channels.
The control section 140 then discriminates whether or not there is a defect included in the color image (d) and the type of the defect using the learned defect detection model. At this time, the control section 140 may first extract the defect suspicious location from the color image (d). In order to detect defect suspicious locations, a defect detection model that learns color images may be utilized.
When the control unit 140 detects the defect detection position, the difference image (e) is obtained with the defect detection position as the center. The control unit 140 also determines whether or not the difference image (e) is defective and/or the type of defect using the defect detection model. In this case, a defect detection model for learning the difference image may be used to determine whether or not a defect is present and/or the type of defect.
Further, when a plurality of detection models learned from regions including the suspected defect locations are stored as respective detection documents, the control unit 140 reads the difference image (e) using the detection model corresponding to the suspected defect location detected.
Then, whether the detected object has the defect or not is correctly identified, and the detected object has the defect or not is finely classified according to relative standards on positions.
Fig. 6 to 7 are sequence diagrams illustrating a defect detection method according to an embodiment.
The defect detection method of the embodiment illustrated in fig. 6 to 7 includes steps processed in time series in the defect detection apparatus 100 illustrated in fig. 1 or 2. Although omitted below, the above description about the defect detection apparatus 100 illustrated in fig. 1 or 2 can be applied to the defect detection method of the embodiment illustrated in fig. 6 to 7.
As shown in fig. 6, the defect detecting apparatus 100 in the defect detecting method of an embodiment may obtain a photographed image obtained by directly photographing an inspection object. The defect detection apparatus 100 obtains the photographed image directly including the photographing means or receives the photographed image from other photographing apparatuses 200 including the photographing means (S610).
The defect detection apparatus 100 calculates a difference between the standard image and the captured image to obtain a difference image (S620).
The defect detection apparatus 100 then converts the standard image and each of the photographed images and the differential image corresponding to each of the photographed images into 8-bit grayscale images, respectively (S630). When at least a part of the standard image, the captured image, and the difference image has been generated or converted into an 8-bit grayscale image, the conversion process may not be performed for the image. For example, the standard image may be stored as an 8-bit grayscale image, and the captured image is also transferred after being converted into an 8-bit grayscale image in the camera 200.
Further, the standard image and the captured image are already calculated in the state of 8-bit grayscale images, and the difference image may be generated as an 8-bit grayscale image.
The defect detection apparatus 100 may then generate 3-channel 24-bit color images having the respective grayscale images as red, green, and blue channels (S640). For example, the captured image may form a color image red channel, the standard image may form a green channel, the differential image may form a blue channel, and the color image may form an image having red, green, and blue luminance values for each pixel.
The defect detection apparatus 100 performs learning of the defect detection model using the color image thus generated, or inputs the color image to the learned defect detection model to detect whether or not there is a defect (S650).
As described above, in order to determine whether a defect is present or absent by using color images of red, green, and blue 3 channels and to determine whether a defect is present or absent by using a gray image, the difference in luminance between the defect and the background is not large, and thus the problem of being unable to correctly read is solved.
On the other hand, in the defect detection method described above, as a specific learning or defect detection method that can further improve the defect detection performance, the defect detection apparatus 100 in the defect detection method according to the other embodiment may first confirm the suspected positions of defects in the color image (S710). For this reason, the suspected defect positions are marked in the color images of the respective learning objects, and the color images can be used to learn the suspected defect positions detected by the defect detection model.
The defect detection apparatus 100 may obtain a cropped image corresponding to a suspected defect location (S720). When learning of the defect detection model is performed, the clipped image may be set as a region selected by the user in the color image. Or after the defect suspicious position is detected by using a defect detection model which is learned in a position capable of detecting the defect suspicious position in advance, the cutting image is obtained by taking the defect suspicious position as the center.
The defect detection apparatus 100 then uses the obtained trimming image to learn whether or not a defect is present or not or type thereof, or to detect a defect.
To this end, according to an embodiment, the defect detecting apparatus 100 may select a detection file corresponding to a defective position (S730). When different detection models are used for each region, the defect detection apparatus 100 calls out a detection document in which the detection model corresponding to the region to which the defect position belongs is stored, and determines the defect by learning or analyzing the cut-out image using the detection model of the called-out detection document (S740).
When learning the defect detection model, the defect detection apparatus 100 learns the trimming image corresponding to the defect suspected position using the detection model corresponding to the defect suspected position. At this time, whether the defect exists or not and/or the defect type can be marked in each cutting object, and on the basis, whether the defect exists or not or the defect type can be learned by the detection model corresponding to the suspicious defect position.
When the defect detection model is used to determine the defect of the inspection object, the defect detection apparatus 100 calls out a detection document corresponding to the suspected defect position to determine whether the defect is present, and inputs a trimming image obtained with the suspected defect position as the center to the defect detection model to determine whether the defect is present or not or the type of the defect.
As described above, the defect inspection apparatus 100 can perform the learning of detecting the suspected defect position using the color image, and perform the two-step learning of learning whether or not a specific defect is present or not or the type of the defect using the trimming image obtained centering on the suspected defect position. When detecting a defect, the defect detecting apparatus 100 may also perform a two-step detection process of extracting a defect suspicious location by using a color image, obtaining a cut image with the extracted defect suspicious location as a center, and then determining whether or not a specific defect is present or not.
Further, the defect inspection apparatus 100 performs learning of inspection models according to the difference in the defect position, and further establishes different determination criteria for each suspected defect position according to the characteristics of the inspection object such as a printed circuit board, thereby flexibly and reasonably performing defect inspection.
The term "part" used in the above embodiments refers to a hardware component such as software, an FPGA (field programmable gate array), or an ASIC, and "part" executes a certain role. But the meaning of "section" is not limited to software or hardware. The "part" may be disposed on a processable storage medium, or may be disposed so as to reproduce one or more processors. As an example, the "part" includes a plurality of components such as a software component, an object-oriented software component, a class component, and a task component, a plurality of processors, a plurality of functions, a plurality of attributes, a plurality of programs, a plurality of subroutines, a plurality of segments of program patent code, a plurality of drivers, firmware, microcode, circuitry, data, databases, a plurality of data structures, a plurality of tables, a plurality of arrays, and a plurality of variables.
The functions provided in the plurality of components and the plurality of "-" parts "may be combined with a smaller number of components and the plurality of" - "parts" or may be separated from a plurality of additional components and "-" parts ".
Furthermore, the plurality of components and the plurality of units can be realized by reproducing one or more CPUs in the device or the secure multimedia card.
The defect detection method according to the embodiment illustrated in fig. 6 and 7 may be implemented in the form of a medium that stores data of commands executable on a computer and that can be read by the computer. The commands and data may be stored as program code means that is executed by a processor to generate predetermined modules of program code to perform predetermined operations. Computer readable media can be any available media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. The computer readable medium may include a computer recording medium. Computer recording media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as data, computer readable instructions, data structures, program modules or other data. For example, the computer recording medium may be a magnetic storage medium such as an HDD and an SSD, an optical recording medium such as a CD, a DVD, and a blu-ray disc, or a memory included in a server accessible through a network.
The defect detection method of the embodiment illustrated by fig. 6 and 7 may be implemented by a computer program (or computer program product) including a command executable by a computer. The computer program includes a programmable command processed by the processor, and may be implemented in a High-level Programming Language (High-level Programming Language), an Object-oriented Programming Language (Object-oriented Programming Language), an assembly Language, a mechanical Language, or the like. The computer program may be recorded on a computer-readable recording medium such as a memory, a hard disk, a magnetic/optical medium, or an SSD (Solid-State Drive).
Further, the defect detection method of the embodiment illustrated by fig. 6 and 7 may be implemented by running a computer program as described above on a computing device. The computing device includes at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. The components are interconnected using various busses, respectively, and may be mounted on a common motherboard or assembled in other suitable manners.
The processor may process commands within the computing device, such as commands stored on memory or storage for displaying graphical information needed to provide a GUI (graphical User Interface) on an external input output device, such as a display connected to a high speed Interface. In another embodiment, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and storage modalities. The processor may be implemented in a chipset of chips that include separate analog and/or digital processors.
Memory is the storage of information within a computing device. For example, the memory may be comprised of volatile memory cells or a collection thereof. As another example, the memory may be comprised of nonvolatile memory cells or a collection thereof. The memory may also be other forms of computer readable media such as magnetic or optical disks.
The storage device may provide mass storage space for the computing device. The storage device may be or include a computer-readable medium. For example, a plurality of devices or other components within a SAN (Storage Area Network) may be included, which may be a hard disk drive, a hard disk device, an optical disk device, or a tape device, a flash memory, other semiconductor Storage devices or device arrays similar thereto.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: it is still possible to modify the technical solutions described in the foregoing embodiments without causing the essence of the corresponding technical solutions to depart from the scope of the technical solutions described in the embodiments of the present invention. For example, the components described as singular may be implemented in a distributed manner, and the components described as distributed may be implemented in a combined manner.
The scope of the present invention should be construed in accordance with the scope of the following claims, and any variations, modifications, etc. within the scope and range of equivalents thereof should be construed as being included in the scope of the claims.
Claims (11)
1. A defect detecting apparatus is characterized in that,
an apparatus for detecting a defect of an inspection object, comprising:
a storage part storing a predetermined one of the standard images and at least one photographed image obtained by photographing the test object; and
a control unit that calculates a difference between the standard image and each of the captured images to generate a difference image corresponding to each of the captured images, synthesizes the standard image, each of the captured images, and each of the difference images to generate at least one color image corresponding to each of the captured images, and performs at least one of learning a defect detection model and detecting a defect using the generated color image; wherein the control section forms the standard image, the respective captured images, and a difference image generated corresponding to the respective captured images into an 8-bit grayscale image; the control section generates a 3-channel 24-bit color image in which the standard image and the respective captured images formed as an 8-bit grayscale image and a difference image generated corresponding to the respective captured images are 3 color channels different from each other.
2. The defect detection apparatus of claim 1,
the control part is used for implementing the learning of the defect detection model for the first time when the color image is used for implementing the learning of the defect detection model, the defect detection model can be implemented for the first time when the defect suspicious position can be detected in the color image, and the defect detection model can be implemented for the second time when the cut image obtained by cutting the image of the certain area of the defect suspicious position can identify whether the defect exists or not or the type of the defect.
3. The defect detection apparatus of claim 2,
the storage part is used for storing each detection document required for learning the defect detection model according to different positions;
the control unit performs the learning of the cut image using a detection document corresponding to a defect suspected position of the color image.
4. The defect detection apparatus of claim 1,
the control unit detects a suspected defect position from the color image when the defect detection is performed using the color image, and performs the determination of whether or not the defect or the type of the defect using a cut image obtained by cutting a fixed-area image of the suspected defect position detected.
5. The defect detection apparatus of claim 4,
the storage part is used for storing each detection document for implementing defect detection by using defect detection models different according to positions;
the control unit determines whether or not the cropped image is defective or a type of defect using a detection document corresponding to the detected defective suspicious location.
6. A defect detection method is characterized in that,
as a method of detecting a defect of an inspection object by a defect detecting apparatus, comprising:
a step of obtaining at least one photographed image generated by photographing the test object;
a step of generating a difference image corresponding to each of the photographed images using a difference between each of the photographed images and a predetermined one of the standard images;
synthesizing the standard image, the captured images, and the difference images to generate at least one color image corresponding to each captured image; wherein the generating of the color image comprises: a step of forming the standard image and the respective captured images, and differential images generated corresponding to the respective captured images into respective 8-bit grayscale images; and a step of generating a 3-channel 24-bit color image in which the standard image and the respective captured images formed in 8-bit grayscale images, and differential images generated corresponding to the respective captured images are regarded as 3 color channels different from each other; and
and a step of performing at least one of learning of a defect detection model or defect detection using the generated color image.
7. The defect detection method of claim 6,
the step of performing at least one of learning a defect detection model or defect detection using the generated color image includes:
a step of detecting a defect suspicious location from the color image;
a step of obtaining a cut image by cutting an image of a certain area of the detected defect suspicious location;
and judging whether the defect exists or not or learning the defect type by using the cutting image.
8. The defect detection method of claim 7,
the step of performing at least one of learning a defect detection model or defect detection using the generated color image includes:
and implementing the step of learning required for detecting the defect suspicious position by using the color image.
9. The defect detection method of claim 7,
the defect detection method further comprises: a step of holding a defect detection model that differs according to position;
the step of judging whether the defect exists or not or learning the defect type by using the cutting image comprises the following steps: and a step of judging whether or not the defect is present in the cut image or learning the type of the defect by using a defect detection model corresponding to the detected defect suspicious location.
10. A computer-readable recording medium recording a program implementing the method of claim 6.
11. A computer program implemented by a defect detection apparatus and stored on a medium for implementing the method of claim 6.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR10-2018-0120853 | 2018-10-11 | ||
| KR1020180120853A KR102175286B1 (en) | 2018-10-11 | 2018-10-11 | Apparatus and method for inspecting for defects |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109916906A CN109916906A (en) | 2019-06-21 |
| CN109916906B true CN109916906B (en) | 2022-10-14 |
Family
ID=66959925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811636492.8A Active CN109916906B (en) | 2018-10-11 | 2018-12-29 | Defect detection device and method |
Country Status (3)
| Country | Link |
|---|---|
| KR (1) | KR102175286B1 (en) |
| CN (1) | CN109916906B (en) |
| TW (1) | TWI718573B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10977509B2 (en) | 2017-03-27 | 2021-04-13 | Samsung Electronics Co., Ltd. | Image processing method and apparatus for object detection |
| KR102252592B1 (en) * | 2019-08-16 | 2021-05-17 | 라온피플 주식회사 | Apparatus and method for inspecting substrate defect |
| CN110487803A (en) * | 2019-08-20 | 2019-11-22 | Oppo（重庆）智能科技有限公司 | The defect inspection method and device of infrared light-emitting component |
| CN110763699B (en) * | 2019-10-12 | 2022-12-20 | 广州兴森快捷电路科技有限公司 | Analysis method for inner layer interconnection defects of circuit board and circuit board |
| CN110726724A (en) * | 2019-10-22 | 2020-01-24 | 北京百度网讯科技有限公司 | Defect detection method, system and device |
| CN111008961B (en) * | 2019-11-25 | 2021-10-19 | 深圳供电局有限公司 | Transmission line equipment defect detection method and system, equipment and medium thereof |
| JP2021089219A (en) * | 2019-12-05 | 2021-06-10 | 東洋製罐グループホールディングス株式会社 | Image inspection system and image inspection method |
| CN113516612B (en) * | 2020-04-13 | 2022-06-14 | 阿里巴巴集团控股有限公司 | Data processing method, device, equipment and storage medium |
| CN111507975B (en) * | 2020-04-23 | 2022-02-08 | 西南交通大学 | Method for detecting abnormity of outdoor insulator of traction substation |
| CN111754456B (en) * | 2020-05-15 | 2022-10-14 | 清华大学 | Two-dimensional PCB appearance defect real-time automatic detection technology based on deep learning |
| US11423577B2 (en) | 2020-07-08 | 2022-08-23 | International Business Machines Corporation | Printed circuit board assembly defect detection |
| TWI770561B (en) * | 2020-07-22 | 2022-07-11 | 鴻海精密工業股份有限公司 | Product defect detection method, computer device and storage medium |
| CN112085722B (en) * | 2020-09-07 | 2024-04-09 | 凌云光技术股份有限公司 | Training sample image acquisition method and device |
| CN112184689A (en) * | 2020-10-12 | 2021-01-05 | 罗建华 | Semiconductor device detection method and device, intelligent terminal and storage medium |
| KR102505484B1 (en) * | 2020-12-18 | 2023-03-03 | 한국공학대학교산학협력단 | Method and apparatus for inspecting tire appearance based on deep learning |
| CN113111828B (en) * | 2021-04-23 | 2022-03-18 | 中国科学院宁波材料技术与工程研究所 | Three-dimensional defect detection method and system for bearing |
| KR102394093B1 (en) * | 2021-09-13 | 2022-05-04 | 오병후 | Method of reading defects using X-ray images of helicopter blades |
| KR102471441B1 (en) | 2021-12-20 | 2022-11-28 | 주식회사 아이코어 | Vision inspection system for detecting failure based on deep learning |
| CN114862856B (en) * | 2022-07-07 | 2022-10-11 | 成都数之联科技股份有限公司 | Panel defect area identification method and device, electronic equipment and medium |
| CN115115625B (en) * | 2022-08-26 | 2022-11-04 | 聊城市正晟电缆有限公司 | Cable production abnormity detection method based on image processing |
| WO2024049201A1 (en) * | 2022-08-31 | 2024-03-07 | 주식회사 엘지에너지솔루션 | Device and method for inspecting battery electrode |
| KR102610783B1 (en) * | 2022-11-03 | 2023-12-07 | 시냅스이미징(주) | Deep learning inspection system implementing image preprocessing of CAD image |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010085679A3 (en) * | 2009-01-26 | 2010-10-14 | Kla-Tencor Corporation | Systems and methods for detecting defects on a wafer |
| JP2014092402A (en) * | 2012-11-01 | 2014-05-19 | Mitsubishi Electric Corp | Defect inspection method for display device |
| CN105894514A (en) * | 2016-04-06 | 2016-08-24 | 广东工业大学 | Printed matter defect detection method and system based on GPU parallel operation |
| CN108596892A (en) * | 2018-04-23 | 2018-09-28 | 西安交通大学 | A kind of identification of Weld Defects based on improvement LeNet-5 models |
| CN108896550A (en) * | 2018-03-30 | 2018-11-27 | 湖北工程学院 | Facial mask print quality inspection method and system |
| CN109084957A (en) * | 2018-08-31 | 2018-12-25 | 华南理工大学 | The defects detection and color sorting process and its system of photovoltaic solar crystal-silicon battery slice |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3344338B2 (en) * | 1997-12-25 | 2002-11-11 | 日本電気株式会社 | Image defect detection apparatus and defect detection method |
| US7015418B2 (en) * | 2002-05-17 | 2006-03-21 | Gsi Group Corporation | Method and system for calibrating a laser processing system and laser marking system utilizing same |
| US7917554B2 (en) * | 2005-08-23 | 2011-03-29 | Ricoh Co. Ltd. | Visibly-perceptible hot spots in documents |
| US9241731B2 (en) * | 2012-04-09 | 2016-01-26 | Ethicon Endo-Surgery, Inc. | Rotatable electrical connection for ultrasonic surgical instruments |
| CN103234476B (en) * | 2013-04-01 | 2014-04-02 | 廖怀宝 | Method for identifying object two-dimensional outlines |
| JP6395481B2 (en) * | 2014-07-11 | 2018-09-26 | キヤノン株式会社 | Image recognition apparatus, method, and program |
| JP2016212539A (en) | 2015-05-01 | 2016-12-15 | 富士通株式会社 | Information processor, defective part determination program, and defective part determination method |
| CN106530284A (en) * | 2016-10-21 | 2017-03-22 | 广州视源电子科技股份有限公司 | Solder joint type detection method and apparatus based on image identification |
-
2018
- 2018-10-11 KR KR1020180120853A patent/KR102175286B1/en active IP Right Grant
- 2018-12-29 CN CN201811636492.8A patent/CN109916906B/en active Active
-
2019
- 2019-06-25 TW TW108122230A patent/TWI718573B/en active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010085679A3 (en) * | 2009-01-26 | 2010-10-14 | Kla-Tencor Corporation | Systems and methods for detecting defects on a wafer |
| JP2014092402A (en) * | 2012-11-01 | 2014-05-19 | Mitsubishi Electric Corp | Defect inspection method for display device |
| CN105894514A (en) * | 2016-04-06 | 2016-08-24 | 广东工业大学 | Printed matter defect detection method and system based on GPU parallel operation |
| CN108896550A (en) * | 2018-03-30 | 2018-11-27 | 湖北工程学院 | Facial mask print quality inspection method and system |
| CN108596892A (en) * | 2018-04-23 | 2018-09-28 | 西安交通大学 | A kind of identification of Weld Defects based on improvement LeNet-5 models |
| CN109084957A (en) * | 2018-08-31 | 2018-12-25 | 华南理工大学 | The defects detection and color sorting process and its system of photovoltaic solar crystal-silicon battery slice |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109916906A (en) | 2019-06-21 |
| TWI718573B (en) | 2021-02-11 |
| KR102175286B1 (en) | 2020-11-06 |
| TW202014985A (en) | 2020-04-16 |
| KR20200046137A (en) | 2020-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109916906B (en) | Defect detection device and method | |
| TWI620441B (en) | Machine learning of real-time image capture parameters | |
| US10255663B2 (en) | Image processing device, image processing method, computer program product | |
| US11367282B2 (en) | Subtitle extraction method and device, storage medium | |
| US10740912B2 (en) | Detection of humans in images using depth information | |
| US9706111B2 (en) | No-reference image and video quality evaluation | |
| WO2021027135A1 (en) | Cell detection model training method and apparatus, computer device and storage medium | |
| US20150371387A1 (en) | Local adaptive histogram equalization | |
| US9070044B2 (en) | Image adjustment | |
| TW201537956A (en) | Object tracking in encoded video streams | |
| CN105046254A (en) | Character recognition method and apparatus | |
| US20170039444A1 (en) | Object detection using directional filtering | |
| US20180197577A1 (en) | Thumbnail generation for video | |
| US11348254B2 (en) | Visual search method, computer device, and storage medium | |
| CN112711971A (en) | Terminal message processing method, image recognition method, device, medium, and system thereof | |
| CN116596928B (en) | Quick peanut oil impurity detection method based on image characteristics | |
| CN112052730A (en) | 3D dynamic portrait recognition monitoring device and method | |
| CN116233534A (en) | Video processing method and device, electronic equipment and storage medium | |
| JP6834126B2 (en) | Information processing equipment, defect detection methods and programs | |
| CN110310341B (en) | Method, device, equipment and storage medium for generating default parameters in color algorithm | |
| US20160086377A1 (en) | Determining an image target's suitability for color transfer in an augmented reality environment | |
| EP4150557A1 (en) | Optimizing high dynamic range (hdr) image processing based on selected regions | |
| CN113409199A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| KR102252590B1 (en) | Method and apparatus for setting test environment based on machine learning environment | |
| US20150279047A1 (en) | Exemplar-based color classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40009201 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |