TWI718573B - Apparatus and method for inspecting for defects - Google Patents

Abstract

The present invention provides a defect detection device and method. According to an embodiment, the defect detection device includes: a storage unit storing a predetermined standard image and at least one shot image obtained by shooting an inspection object; and a control unit, calculating the standard image and each shot image Differential images corresponding to each shot image are generated, the standard image, each shot image, and each difference image are synthesized to generate at least one color image corresponding to each shot image, and the generated color images are used to implement defect detection model learning or defect detection At least one of them.

Description

Defect detection device and method

A number of embodiments disclosed in the specification of the present invention relate to defect detection devices and methods. Specifically, when a defect contained in an object is detected by using a photographed image of an inspection object, an image processing method that can improve defect detection performance is used for defect training and detection. Defect detection device and method.

There are many ways to detect the defects of mass-produced products with uniform quality. Printed products such as packaging paper or label paper, or intermediate or final products in the production process of electronic products such as printed circuit boards or assembled circuits. Actively apply image analysis technology to production products with the same actual appearance or shape to reduce detection It takes time and labor to improve the quality of inspection.

At present, artificial intelligence image analysis technology is used to detect product defects, so as to obtain technical results such as reducing false detection or over-detection rate.

As mentioned above, the usual method of using artificial intelligence image analysis technology to detect product defects is to use a sufficient number of images of normal and/or defective products to learn the models required to determine the defects, and then use the learning model to determine whether the images of the inspection object are normal or Bad or not. However, it is impossible to identify all kinds of defects by using the existing defect training and detection methods, which leads to the problem of poor overall defect detection performance.

For example, according to the existing defect detection method for printed circuit boards, as shown in Figure 1, the model is learned from the photographed image 10 obtained by photographing the produced printed circuit board, and the position of the defect 11 is marked, and the defect area and the background area After training, the defect detection model learns the defect-free background area and the defective area, and then when the photographed image of the inspection object is input, it can output whether the photographed image contains defects or not.

However, this existing defect detection method for printed circuit boards is to detect disconnection (OPEN; the defect caused by the disconnection of the area where the circuit pattern should be connected) or short circuit (SHORT; the circuit pattern that should not be connected to each other. The defect detection rate of the structure such as the occurrence of defects) is not high, and there should be holes (HOLE). The defect of the hole or the circuit pattern and the interval between the patterns are not maintained sufficiently and the space that occurs (SPACE) does not reach the defect, etc. Undetectable.

Another method of using artificial intelligence image analysis technology to detect product defects is to use a sufficient number of normal and/or defective product images and pre-set standard images to generate a difference image. After the model is learned using the difference image, Use the learned model to determine whether the image of the test subject is normal or bad. However, even in this way, the detection rate of structural defects has not been fully improved, and the defect detection rate is reduced because the color or brightness of a part of the differential image is similar to the background or edge portion.

The related prior art document, Korean Registered Patent No. 10-1128322, relates to an optical inspection device and method for printed circuit boards, and describes a technique for comparing standard PCB pattern images and shooting PCB pattern images to determine whether they are defective. However, the conventional technology only compares the standard image and the brightness mode of the shot image with the value to determine whether it is normal or defective. It is difficult to detect small defects, nor can it distinguish the type or correct position of the defect. Therefore, a technology that can solve the above-mentioned problems is needed.

The above-mentioned background art is technical information possessed by the inventor or mastered in the process of obtaining the present invention in order to obtain the present invention, and is not a conventional technique that has been disclosed to the general public before the present invention is applied for.

The purpose of the embodiments disclosed in this specification is to provide a defect detection device and method.

The purpose of the embodiments disclosed in this specification is to use 3-channel color images to improve the defect detection performance of the inspection object.

The purpose of the embodiments disclosed in this specification is to determine the type of the defect by learning the type of the detected defect.

The purpose of the embodiments disclosed in the present invention is to improve the accuracy of defect detection and classification by using defect detection models that are separately learned according to defect positions.

To solve the problem, the technical solution adopted by the present invention is, according to an embodiment, as a device for detecting defects of an inspection object, including: a storage unit storing a predetermined standard image and at least one photographed image obtained by photographing the inspection object; and The control unit calculates the difference between the standard image and each shot image, generates a difference image corresponding to each shot image, combines the standard image, each shot image, and each difference image to generate at least one color image corresponding to each shot image, At least one of learning of a defect detection model or defect detection is implemented using the generated color image.

In order to solve the technical problem, the technical solution adopted by the present invention is, according to an embodiment, as a method for detecting defects of an inspection object implemented by a defect detection device, including: obtaining a photographic inspection The step of examining at least one photographed image generated by the subject; the step of using the difference between each photographed image and a predetermined standard image to generate a differential image corresponding to each photographed image; combining the standard image, each photographed image, and each differential image The step of generating at least one color image corresponding to each captured image; and using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

To further solve the technical problem, the technical solution adopted by the present invention is that, according to another embodiment, the defect detection method in the computer-readable recording medium that records the program for implementing the defect detection method includes: At least one step of photographing images; a step of generating a differential image corresponding to each photographed image by using the difference between each photographed image and a predetermined standard image; combining the standard image, each photographed image, and each differential image to generate a corresponding The step of capturing at least one color image of the image; and the step of using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

In order to solve the technical problem, the technical solution adopted by the present invention is, according to another embodiment, implemented by a defect detection device and stored in a computer program in a medium for implementing the defect detection method, the defect detection method includes : The step of obtaining at least one shot image generated by the shooting test object; the step of using the difference between each shot image and a predetermined standard image to generate a difference image corresponding to each shot image; combining the standard image, each shot image, and each shot image The step of synthesizing the difference images to generate at least one color image corresponding to each captured image; and using the generated color image to implement at least one of the learning of the defect detection model or the defect detection.

According to one of the above technical solutions, a defect detection device and method can be proposed.

According to one of the above technical solutions, 3-channel color images are used to improve the defect detection performance of the inspection object.

According to the first technical solution mentioned above, it is possible to provide a defect detection device and method that learns the defect type to be detected and determines the defect type.

According to one of the above technical solutions, the defect detection model learned separately according to the defect position is used to improve the accuracy of defect detection and classification.

The effects that can be obtained by the disclosed embodiments are not limited to the above effects. According to the following description, those with ordinary knowledge in the art can clearly understand other effects that are not involved.

a: Standard image

b, 10: Shooting images

c: Differential image

CLS: Defect type selection button

d: color image

e: crop image

S610, S620, S630, S640, S650, S710, S720, S730, S740: Defect detection method steps

11: Defects

100: Defect detection device

110: Input and output department

120: Storage Department

130: Ministry of Communications

140: Control Department

200: Camera

Fig. 1 is an exemplary diagram showing an example of a photographed image of a printed circuit board marking the position of a defect; Fig. 2 is a block diagram illustrating the structure of a defect detection device according to an embodiment; Fig. 3 is a diagram showing a defect detection according to an embodiment An illustration of an example of a reference image, a captured image, and a differential image used for defect detection in the device; Figure 4 shows an example of a color image generated using the reference image, the captured image, and the differential image in the detection device of an embodiment Illustrative diagram; FIG. 5 is an illustrative diagram illustrating the process of learning the type of defects detected in the defect detection device according to an embodiment; FIG. 6 and FIG. 7 are flowcharts illustrating the steps of a defect detection method according to an embodiment .

Various embodiments will be described in detail below with reference to the accompanying drawings. The embodiments described below can also be modified into other different forms for implementation. In order to explain more clearly, the technical field of the following embodiments The content of the domain that has been widely known by ordinary knowledgeable persons will not be explained in detail. In the figure, parts irrelevant to the description of the embodiments are omitted, and similar numbers are used for similar parts in the specification.

The description in the specification that a certain element is "connected" to other elements not only means that it is directly connected, but there may be other elements in between. When a component is described in the specification as "comprising" a certain element, provided that there is no description to the contrary, it does not mean that other elements are excluded, but it is stated that other elements may also be included.

A number of embodiments will be described in detail below with reference to the drawings.

Before explaining, define the terms used below.

"Shooting image" is an image obtained by directly shooting the test object. The following uses printed circuit boards as the "test object" for description, but as long as it is an intermediate or final product of an electronic product, or printed matter such as packaging paper or label paper, it can be a product that has a predetermined pattern at the time of production. The captured image is not limited to a simple image captured with visible light, but may also be an image obtained using various optical imaging methods such as X-rays or infrared rays according to the type or nature of the object to be detected.

The "standard image" is an image that shows the ideal mode that the test object needs to have, and is the actual standard for shooting images of each test object. Any image of the test object and the standard image are used together to generate the differential image described later.

"Differential image" is an image generated using the above-mentioned shooting image and standard image, and calculating the difference between the shooting image and the standard image. For example, for a binary image, the differential image can be generated only by the difference between the pixel values of the two images. For a non-binary image, the difference between the pixel values can be used directly or the result of comparison with a threshold value can be used. The pixel value here is the brightness value of each pixel included in the image, and can have an 8-bit value, that is, a value from 0 to 255. The difference image can not only be obtained by the above-mentioned simple subtraction operation, but also can be obtained by assigning a weighted value to the value of the captured image or standard image as needed, or converting the difference between each pixel value into multiple level values using two or more thresholds. As mentioned above, in one embodiment, as long as the differential image can appropriately compensate for the type or nature of the test object in the calculation method, Any calculation method can be used. Further, in order to obtain a differential image, a preprocessing process of at least one of the captured image and the standard image may be carried out.

When there are multiple shot images, the number of difference images corresponding to each shot image will be generated.

The following "color image" is an image that uses the captured image, standard image, and differential image as the color channels. In order to form color images, the captured images, standard images, and differential images of each image channel can be formed as 8-bit grayscale images, respectively. For this reason, according to requirements, the defect detection device described later can convert into 8-bit gray-scale images when shooting images, standard images, and differential images are not 8-bit gray-scale images. At this time, an 8-bit grayscale image refers to a 1-channel image in which each pixel value has an 8-bit value, that is, a brightness value from 0 to 255. In other words, there is no color channel, and an image composed of pixels with a brightness value from 0 to 255.

In addition, the color image can be formed by using the captured image, the standard image, and the differential image formed by the aforementioned 8-bit grayscale image as 3-channel 24-bit images with different color channels, respectively. For example, a standard image can form a red (RED) channel, a captured image can form a green (GREEN) channel, and a differential image can form a blue (BLUE) channel. Each pixel value of each channel can display the brightness of the corresponding color of each pixel of the color image. value. At this time, there are multiple shot images, so color images can be generated corresponding to each shot image and difference image.

At this time, the color image is used for learning the defect detection model in the defect detection device and method of an embodiment. In addition, the color image finally generated by the shooting image of the inspection object can also be used to detect the defects of the actual inspection object by using the learned defect detection model.

In addition to the terms defined above, other terms that need to be explained are separately explained below.

Fig. 2 is a block diagram illustrating the structure of a defect detection device according to an embodiment.

The defect detection device 100 is a device for learning required to detect defects of an inspection object and detecting defects using a learned model.

The defect detection apparatus 100 may be implemented by an electronic terminal or a server-client system (or cloud system), and the system may include an electronic terminal for installing online service applications required for interaction with users.

The electronic terminal 10 can be realized by connecting a remote server via a network (N), or a computer that can be connected to other terminals and servers, or a portable terminal, a television, a wearable device, and the like. Computers here include, for example, laptops, desktops, and laptops with a web browser (WEB Browser). Portable terminals are, for example, wireless communication devices that can ensure portability and mobility. It can include PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication)-2000, CDMA ( Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), smart phone (Smart Phone), mobile WiMAX (Mobile Worldwide Interoperability for Microwave Access) and other types of handheld-based Handheld wireless communication device. The television may include IPTV (Internet Protocol Television), Internet TV (Internet Television), wireless television, cable television, and so on. Further, wearable devices are information processing devices that can be directly worn by the human body such as watches, glasses, accessories, clothing, shoes, etc., directly or through other information processing devices connected to a remote server or other terminals via a network.

The defect detection apparatus 100 of an embodiment includes at least a part of an input/output unit 110, a storage unit 120, a communication unit 130, and a control unit 140.

The input and output unit 110 may include a user input required to obtain the learning process required for detecting the defect of the inspection object or the user input required in the defect detection process or an input unit required to select a required file, and an input unit used to display the operation execution result or a defect detection device 100 status and other information output unit. For example, lose The input/output unit 110 may include an operation panel (operation panel) that receives user input, a display panel (display panel) that displays a screen, and the like.

Specifically, the input unit may include various types of devices that receive user input, such as a keyboard, physical buttons, touch screen, camera, or microphone. The output unit may include a display panel or a speaker. However, it is not limited to this, and the input and output unit 110 may include various structures that support input and output.

The defect detection apparatus 100 may include a storage unit 120. The storage unit 120 can temporarily store a plurality of photographed images and standard images required for learning the defect detection model, and at least can temporarily store the difference images generated using these. Furthermore, at least the shot image and the standard image and the color image obtained by using the differential image can be temporarily stored. In the process of learning the defect detection model by using the color image, the storage unit 120 updates and stores the learned model each time learning is performed. The storage unit 120 can separate each defect inspection model into independent inspection files for storage.

The defect location is marked in the color image stored in the storage unit 120 for the learning process.

The defect detection device 100 may further include a communication unit 130.

The communication unit 130 may implement wireless communication with other devices or networks. To this end, the communication unit 130 may include a communication module that supports at least one of various wireless communication methods. For example, the communication module can be implemented in the form of a chipset.

The wireless communication supported by the communication unit 130 is, for example, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth (Bluetooth), UWB (Ultra Wide Band), or NFC (Near Field Communication). The wired communication supported by the communication unit 130 is, for example, USB or HDMI (High Definition Multimedia Interface).

In particular, the communication unit 130 can communicate with the camera 200 to receive the shot image from the camera 200. In this case, the imaging device 200 is, for example, a device provided with optical equipment such as a camera. At this time, the imaging device 200 may be a device that can only capture light of wavelengths in the visible light spectrum, or a device that can capture light of other wavelengths such as infrared rays or X-rays.

The control unit 140 controls the overall operation of the defect detection apparatus 100, and may include a processor such as a CPU. The control unit 140 may control other structures included in the defect detection apparatus 100 in order to perform an action corresponding to the user input received by the input/output unit 110, for example.

For example, the control unit 140 can run a program stored in the storage unit 120, or read a file stored in the storage unit 120, or store a new file in the storage unit 120.

According to an embodiment described in this specification, the control unit 140 can obtain a captured image. As described above, the captured image is the image obtained by shooting the inspection object, which is generated by the above-mentioned imaging device 200 and transmitted to the defect detection device 100. At this time, the captured image may include multiple images respectively captured by multiple inspection objects.

As shown in FIG. 3, the control unit 140 can use the standard image a and each captured image b to generate a differential image c corresponding to each captured image. Here, the difference image c is calculated by the control unit 140 using the difference between the captured image b and the standard image a as described above.

For example, the standard image a can be pre-stored as an 8-bit grayscale image in the storage unit 120, and the control unit 140 can convert each captured image b into an 8-bit grayscale image. The control unit 140 subtracts each pixel value of each captured image b from each pixel value of the standard image a, and generates an 8-bit grayscale image with each calculated value as the pixel value as a differential image c.

The control unit 140 performs additional operations such as subtraction or more, for example, to assign a set weight to the pixel values of the standard image a or the captured image b, or to compare the value calculated by the subtraction with one or more thresholds to obtain each pixel value A difference image c is generated by performing operations corresponding to a plurality of set level values.

Furthermore, as shown in FIG. 4, the control unit 140 can obtain a color image d using the standard image a, each shot image b, and the difference image c corresponding to each shot image b generated through the above process. At this time, the color image d can also be generated in a quantity corresponding to each shot image b.

Specifically, the control unit 140 may generate a 3-channel color image d with a standard image a, each captured image b, and a differential image c corresponding to each captured image b as color channels. At this time, each shot image b, standard image a, and the difference image c corresponding to each shot image can be formed as 8-bit grayscale images as described above, and when some of them do not conform to the 8-bit grayscale image, the control unit 140 Converted into 8-bit grayscale image to prepare for the generation of color image d.

Then the control unit 140 generates a 3-channel 24-bit color image d with each 8-bit grayscale image as the red, green, and blue channels. Furthermore, the generated color image d can be an image in which each pixel value of each captured image b, the standard image a, and the difference image c corresponding to each captured image b is used as the red, green, and blue brightness value of each pixel of the color image d. According to the embodiment, the three images including the standard image a, each shot image b, and the difference image c corresponding to each shot image b may respectively form mutually different color channels in a different order from the above red, green, and blue.

The control unit 140 uses the one or more color images d generated as described above to implement the learning of the defect detection model. At this time, the control unit 140 first implements the learning of the defect detection model in order to detect the suspicious position of the defect from the color image d at one time.

At this time, the learning target color image d used by the control unit 140 to implement the learning of the defect detection model may be respectively marked with defective defect positions. To this end, the defect detection device 100 can receive the defect location information contained in each color image d, which is the learning object, from the user via the input and output unit 110, or receive the marked defect location via the communication unit 130 via other devices. Color image d.

The control unit 140 can perform learning to identify the suspicious defect position by learning the learning target color image d of the marked defect position. Specifically, the control unit 140 may implement learning of a defect detection model for identifying suspicious locations of defects. For example, the control unit 140 may use machine technology to further implement learning of a defect detection model composed of artificial neural networks, such as deep learning technology. The control unit 140 takes each pixel value of the 3-channel 24-bit color image d as an input variable, and implements the learning of the defect detection model through the process of obtaining the defect detection model constant value by using the information of the suspected defect location as the result value.

Furthermore, the defect detection model can be learned to find the suspicious position of the defect in the color image d.

At this time, the defect detection model learned by the control unit 140 may have characteristics such as the appearance, area, and relative relationship with surrounding pixel values of adjacent pixels defined in a specific range based on the RGB values to detect the suspicious location of the defect. For example, in the example shown in Figure 4, after the yellow dots included in the color image d are marked as defect locations, these images are used to implement the learning of the defect detection model. At this time, the yellow dots can display leak defects in the printed circuit board, for example, and the defect detection model can learn the common color, size, appearance, and location characteristics of these leak defects through learning. Further, in addition to leak defects, when learning multiple color images d containing other types of defects, the defect detection model learns various types of defects and detects suspicious locations of the defects.

The control unit 140 additionally implements learning of a defect detection model using a cropped image produced by cropping a certain area with the suspicious defect position as the center.

For example, as shown in FIG. 5, the control unit 140 may crop a certain area of the color image d to generate a cropped picture e. At this time, the cropped image e is, for example, an area image with a set size centered on the suspected defect position.

After obtaining the cropped image e from the learning target color image d containing the defect with the defect position as the center, the control unit 140 implements the learning of the defect detection model using the cropped image e. At this time, the cropped image e can indicate the presence or absence of defects and/or the type of defects. Specifically, as shown in Figure 5, when the defect of the cropped image e is a non-defect that has passed inspection, the user is allowed to select "0. Pass", and if it is a defect, select "1.RC" and "2" according to the defect type. One of multiple defect type selection buttons (CLS) such as "bump", "3. Short circuit", etc., enables the presence or absence of defects in the cropped image e or the defect type to be marked. Using the cropped image e of the presence or absence of the marked defect or the defect type as described above, the learning of the defect detection model can be further implemented.

Furthermore, the defect detection model can not only learn to determine the existence of defects more accurately, but also distinguish the types of defects.

In the example shown in Figure 5, the defect contained in the cropped image e is a leak defect, so the user can select the "9. Leak" button, and the control unit 140 marks the cropped image e as a leak for implementation Learning of defect detection models.

At this time, the defect detection model for learning color images to detect the suspicious location of the defect and the defect detection model for learning cropped images to detect the presence or absence of defects and/or defect types can be composed of different detection models.

The defect detection apparatus 100 according to an embodiment can implement the learning of the defect detection model according to different standards of the suspicious defect position. For example, when detecting the defects of printed circuit boards, a certain degree of defects are allowed for the outline of the printed circuit board, but the defects are identified according to strict standards for the circuit area.

The control unit 140 divides the color image d into a plurality of regions according to the actual structure of the printed circuit board corresponding to the color image d. For example, after pre-dividing the color image d area into multiple areas such as outline, mesh, circuit, bonding, pad, etc., it is possible to identify which area the suspicious location of the detected defect corresponds to. At this time, in order to set the defect detection standards differently for each area, the control unit 140 can implement the learning of different detection models for each area, and the different detection models in each area are stored in the storage unit in the form of mutually different detection files. 120.

The control unit 140 uses the cropped image e of the suspicious defect position to perform learning of the defect detection model corresponding to the area included in the suspicious defect position.

In this way, different defect detection models can be learned according to which area the area containing the suspicious location of the defect corresponds to, so different detection standards can be set according to the suspicious location of the defect.

In addition, the control unit 140 performs defect detection using the defect detection model learned as described above. For this reason, the defect detection apparatus 100 can obtain a photographed image b of the photographed object in order to distinguish whether the actual defect exists or not. As shown in FIG. 3 again, the defect detection device 100 uses the obtained difference between the captured image b and the standard image a to generate a differential image c.

As shown in FIG. 4, the defect detection device 100 can generate a 3-channel 24-bit color image d using the captured image b, each standard image a, and the differential image c as RGB channels.

Then, the control unit 140 uses the learned defect detection model to discriminate whether there are defects in the color image d and the defect types. At this time, the control unit 140 can first extract the suspected defect location from the color image d. In order to detect the suspicious location of the defect, a defect detection model that learns color images can be used.

Then, when the control unit 140 detects the defect detection position, it obtains the differential image e with the defect detection position as the center. In addition, the control unit 140 uses the defect detection model to identify the presence or absence of a defect in the differential image e and/or the defect type. At this time, in order to distinguish the existence of defects and/or defect types, the defect detection model of learning differential images can be used.

Furthermore, when a plurality of detection models learned separately from the area including the suspicious defect location are stored as respective detection files, the control unit 140 reads the difference image e using the detection model corresponding to the suspicious defect location detected.

Then correctly distinguish whether there is a defect or defect type, and finely classify whether the inspection object has a defect according to the relative standard of the position.

Fig. 6 and Fig. 7 are sequence diagrams for explaining a defect detection method according to an embodiment.

The defect detection method of the embodiment shown in FIG. 6 and FIG. 7 includes the steps of processing in a time sequence in the defect detection apparatus 100 shown in FIG. 1 or FIG. 2. Even if the content is omitted below, the content described above with respect to the defect detection device 100 shown in Fig. 1 or Fig. 2 can also be applied to the defect detection method of the embodiment shown in Fig. 6 and Fig. 7.

As shown in FIG. 6, in one embodiment of the defect detection method, the defect detection device 100 can obtain the photographed image obtained by directly photographing the inspection object. The defect detection device 100 directly includes a photographing means to obtain a photographed image, or receives a photographed image from another photographing device 200 including a photographing means (step S610).

The defect detection device 100 calculates the difference between the standard image and the captured image to obtain a difference image (step S620).

Then, the defect detection device 100 converts the standard image, each shot image, and the difference image corresponding to each shot image into 8-bit grayscale images (step S630). When at least a part of the standard image, the captured image, and the differential image has been generated or converted into an 8-bit grayscale image, the conversion program may not be implemented for the image. For example, the standard image may have been stored as an 8-bit gray-scale image, and the captured image is also converted into an 8-bit gray-scale image in the shooting device 200 to be transmitted.

Further, the standard image and the captured image are already 8-bit grayscale images, and the differential image can also be generated as an 8-bit grayscale image.

Then, the defect detection apparatus 100 can generate a 3-channel 24-bit color image with each grayscale image as the red, green, and blue channels (step S640). For example, a captured image can form a color image red channel, a standard image can form a green channel, a differential image can form a blue channel, and a color image can form an image with red, green, and blue brightness values for each pixel.

The defect detection apparatus 100 implements learning of a defect detection model using the color image generated in this way, or inputs the color image to the learned defect detection model to detect the presence or absence of a defect (step S650).

As mentioned above, the color image of red, green, and blue is used to determine whether the defect exists or not, and the gray image is used to determine whether the defect exists or not. The brightness difference between the defect and the background is not large, so it is solved that it cannot be read correctly. The problem.

On the other hand, in the above-mentioned defect detection method, as a specific learning or defect detection method that can further improve the defect detection performance, the defect detection device 100 in the defect detection method according to other embodiments can first confirm the suspicious location of the defect in the color image ( Step S710). For this reason, the suspicious location of the marked defect in the color image of each learning object can be learned by using the color image to make the defect detection model detect the suspicious location of the defect.

The defect detection device 100 can obtain a cropped image corresponding to the suspicious location of the defect (step S720). When implementing the learning of the defect detection model, the cropped image can be set as the area selected by the user in the color image. Or a defect detection model that is learned in advance to detect the suspicious location of the defect is used to first detect the suspicious location of the defect, and then obtain a cropped image with the suspicious location as the center.

Subsequently, the defect detection device 100 uses the obtained cropped image to learn the existence or type of the defect, or to detect the defect.

To this end, according to an embodiment, the defect inspection apparatus 100 may select an inspection file corresponding to the defect position (step S730). When using different inspection models by area, the defect inspection device 100 calls up the inspection file in which the inspection model corresponding to the area of the defect location belongs, and uses the inspection model of the called inspection file to learn or analyze the cut image to determine the defect ( Step S740).

When implementing the learning of the defect detection model, the defect detection device 100 uses the detection model corresponding to the suspicious location of the defect to learn the cropped image corresponding to the suspicious location of the defect. At this time, the presence or absence of defects and/or defect types can be marked in each cropped object. Based on this, the detection model corresponding to the suspicious location of the defect can learn the presence or absence or type of the defect.

When the defect inspection model is used to determine the defect of the inspection object, the defect inspection device 100 calls the inspection file corresponding to the suspicious location of the defect to determine whether the defect exists or not, and inputs the cropped image obtained from the suspicious location of the defect as the center to the defect inspection model to determine the defect Existence or defect type.

As described above, the defect detection apparatus 100 can use color images to implement learning to detect the suspicious location of the defect, and use cropped images obtained centering on the suspicious defect location to implement a two-step learning of the existence or type of a specific defect. When detecting a defect, the defect detection device 100 may also implement a two-step detection process of first extracting the suspicious location of the defect by using the color image, and then obtaining a cropped image with the extracted suspicious location as the center, and then determining the existence or type of the specific defect.

Furthermore, the defect detection device 100 implements learning of different detection models according to different defect positions, and then establishes different judgment standards for each suspicious position of defects according to the characteristics of inspection objects such as printed circuit boards, and uses this to implement defect detection flexibly and reasonably.

The term "~ part" used in the above embodiments refers to hardware components such as software or FPGA (field Programmable gate array) or ASIC, and the "~ part" performs a certain role. But the meaning of "~部" is not limited to software or hardware. The "~ part" can be placed on a storage medium that can be processed, or one or more processors can be reproduced. As an example, "~" includes software components, object-oriented software components, class components, and task components, multiple components, multiple processors, multiple functions, multiple attributes, multiple programs, multiple sub-elements, etc. Programs, multiple segments of program patent codes, multiple drivers, firmware, micro-commands, circuits, data, databases, multiple data structures, multiple tables, multiple arrays, and multiple variables.

The functions provided in the multiple constituent elements and multiple "~parts" can be combined with a smaller number of constituent elements and multiple "~parts", or separated from the added multiple constituent elements and "~parts".

Not only that, but multiple constituent elements and multiple "~parts" can be realized by reproducing one or more CPUs in the device or the secure multimedia card.

The defect detection method according to the embodiment illustrated in Figs. 6 and 7 can be implemented in the form of a medium that can store data of commands that can be run on a computer and can be read by a computer. Commands and data can be stored in the form of program code, and a predetermined degree module is generated when the processor is processed to execute a predetermined operation. Computer-readable media is any available media that the computer can access, including volatile and non-volatile media, separate and non-separable media. The computer-readable medium may include a computer recording medium. Computer recording media include computer-readable commands, data structures, program modules, or other volatile, non-volatile, discrete, and non-separable media realized by any method or technology for the purpose of storing information such as data. For example, computer recording media may be magnetic storage media such as HDD and SSD, CD, DVD And optical recording media such as Blu-ray Discs, or storage included in servers accessible via the Internet.

The defect detection method of the embodiment described in FIGS. 6 and 7 can be implemented by a computer program (or a computer program product) that includes commands that can be run by a computer. Computer programs include programmable commands processed by the processor, and can be implemented in high-level programming language, object-oriented programming language, assembly language, or mechanical language. The computer program can be recorded on a computer-readable recording medium (such as a memory, a hard disk, a magnetic/optical medium, or an SSD (Solid-State Drive)).

Furthermore, the defect detection method of the embodiment illustrated in FIGS. 6 and 7 can be implemented by running the computer program as described above on a computing device. The computing device includes at least a part of a processor, a storage, a storage device, a high-speed port connected to the storage and a high-speed expansion port, and a low-speed port connected to a low-speed bus and the storage device. The components are connected to each other by various bus bars, and can be installed on a general-purpose motherboard or assembled in other suitable ways.

The processor can process commands in the computing device. The commands are, for example, connected to a display on a high-speed port, and are stored in the memory in order to display the graphic information required by the GUI (Graphic User Interface) on the external input and output device. Or a command on the storage device. In another embodiment, multiple processors and/or multiple buses can be used with multiple memories and storage forms as appropriate. The processor can be implemented by a chipset consisting of multiple chips including independent multiple analog and/or digital processors.

The storage is to store information in the computing device. For one example, the storage may consist of volatile storage units or a collection thereof. Another example is that the storage can be composed of non-volatile storage units or a collection thereof. The storage may also be other forms of computer readable media such as floppy disks or optical discs.

The storage device can provide a large-capacity storage space for the computing device. The storage device may be a computer-readable medium or a composition containing these media. For example, it can contain SAN (Storage Area Multiple devices or other components in the Network) can be hard disk drives, hard disk devices, optical disk devices or tape devices, flash, and other similar semiconductor storage devices or device arrays.

The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still compare the foregoing embodiments. The described technical solutions are modified, and these modifications do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions described in the embodiments of the present invention. For example, various constituent elements described as a single type can be implemented in a dispersed manner, and constituent elements described as dispersed can also be implemented in a combined form.

The patent scope of the present invention should be interpreted according to the following patent application scope of the invention, and any change or modification within its equivalent scope shall belong to the patent scope of the present invention.

100: Defect detection device

110: Input and output department

120: Storage Department

130: Ministry of Communications

140: Control Department

200: Camera

Claims (14)

A defect detection device, as a device for detecting defects of an inspection object, comprises: a storage part storing a predetermined standard image and at least one shot image obtained by shooting the inspection object; and a control part calculating the standard image and each The difference of the shot images generates a difference image corresponding to each shot image, the standard image, each shot image, and each difference image are synthesized to generate at least one color image corresponding to each shot image, and the color image Mark the position of the defect in, thereby implementing the learning of the defect detection model, or input the color image to the learned defect detection model and extract the suspicious location of the defect from the color image, thereby implementing the defect detection. According to the defect detection device described in item 1 of the scope of patent application, the control unit forms the standard image, each of the photographed images, and the differential image generated corresponding to each of the photographed images into 8-bit grayscale images. For the defect detection device described in item 2 of the scope of patent application, wherein the control unit generates the standard image formed with 8-bit grayscale images, each of the shot images, and the difference image generated corresponding to each of the shot images as mutual 3-channel 24-bit color image with 3 different color channels. For the defect detection device described in item 1 of the scope of patent application, when the control unit uses the color image to implement the learning of the defect detection model, the control unit has two learning steps for the defect detection model, and the first step is in the The suspicious location of the defect can be detected in the color image. The second step is to mark the existence of the defect or the type of the defect in a cropped image of a certain area of the suspicious location of the defect. For example, the defect detection device described in item 4 of the scope of patent application, wherein the storage unit stores various detection files required for learning different defect detection models according to the location; the control The manufacturing department uses the detection file corresponding to the suspicious location of the color image to implement the learning of the cropped image. For example, the defect detection device described in item 1 of the scope of patent application, wherein when the control unit uses the color image to implement defect detection, it detects the suspicious position of the defect from the color image, and uses the image of a certain area of the detected suspicious position of the defect Crop to obtain a cropped image to determine whether the defect exists or not or the type of the defect. For example, the defect detection device described in item 6 of the scope of patent application, wherein the storage unit stores various inspection files for defect detection using different defect detection models according to the location; the control unit uses the information corresponding to the suspicious location of the detected defect The inspection file determines whether the defect of the cropped image exists or not or the defect type. A defect detection method, as a method for detecting defects of an inspection object by a defect detection device, includes: obtaining at least one photographed image generated by photographing the inspection object; using the difference between each of the photographed images and a predetermined standard image The step of generating a difference image corresponding to each shot image by difference; the step of synthesizing the standard image, each shot image, and each difference image to generate at least one color image corresponding to each shot image; and in the color image Mark the defect location, thereby implementing the step of defect detection model learning, or input the color image into the learned defect detection model and extract the suspicious location of the defect from the color image, thereby implementing the step of defect detection. According to the defect detection method described in item 8 of the scope of patent application, the step of generating the color image includes: forming the standard image, each of the shot images, and the difference image generated corresponding to each of the shot images into 8-bit grays The steps to obtain an image; and The step of generating the standard image formed by the 8-bit grayscale image, each of the shot images, and the difference image generated corresponding to each of the shot images as a 3-channel 24-bit color image of three different color channels. For example, the defect detection method described in item 9 of the scope of patent application, wherein the use of the generated color image to implement the learning of the defect detection model or the defect detection includes: the step of detecting the suspicious position of the defect from the color image; and the step of detecting the suspicious position of the defect The step of cropping a certain area of the image to obtain a cropped image; and marking the presence or absence of the defect or the type of the defect in the cropped image, thereby implementing the step of learning the defect detection model, or inputting the cropped image to the learned The defect detection model is used to determine the presence or absence of the defect or the type of the defect in the cropped image, thereby implementing the step of defect determination. For the defect detection method described in item 10 of the scope of patent application, the use of the generated color image to implement the learning of the defect detection model or the defect detection includes: implementing the step of learning required to use the color image to detect the suspicious location of the defect. For example, the defect detection method described in item 10 of the scope of patent application, wherein the defect detection method further includes the step of maintaining different defect detection models according to the position; marking the presence or absence of defects or defects in the cropped image Type, by which the step of learning the defect detection model is implemented, or the cropped image is input to the learned defect detection model and the presence or absence of the defect or the type of the defect is determined in the cropped image, whereby the step of implementing the defect determination further includes : Use the defect detection model corresponding to the suspicious position of the detected defect to implement the step of learning or judging whether the defect exists or not or the defect type of the cropped image. A computer-readable recording medium that records the implementation program of the method described in item 8 of the scope of patent application. A computer program implemented by a defect detection device and stored in a medium for implementing the method described in item 8 of the scope of the patent application.

Images