CN116091503A - Method, device, equipment and medium for discriminating panel foreign matter defects - Google Patents
Method, device, equipment and medium for discriminating panel foreign matter defects Download PDFInfo
- Publication number
- CN116091503A CN116091503A CN202310374308.1A CN202310374308A CN116091503A CN 116091503 A CN116091503 A CN 116091503A CN 202310374308 A CN202310374308 A CN 202310374308A CN 116091503 A CN116091503 A CN 116091503A
- Authority
- CN
- China
- Prior art keywords
- target
- image
- defect
- line
- panel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The application discloses a method, a device, equipment and a medium for discriminating a panel foreign matter defect, relates to the technical field of panel defect recognition, and aims to solve the technical problem that the prior art cannot discriminate the severity of the foreign matter defect in a panel. The method comprises the following steps: acquiring an image to be identified of a target panel; inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; inputting the first image into a classification model for classification to obtain a classification result; carrying out gray processing on the first image with the OK classification result to obtain a target line area; and obtaining a defect judging result of the target panel based on the line width value of the target line area.
Description
Technical Field
The present disclosure relates to the field of panel defect identification technologies, and in particular, to a method, an apparatus, a device, and a medium for identifying a panel foreign object defect.
Background
The application field of the PCB is very wide, so long as the PCB is an electronic device, various defects exist in the PCB-mounted production process, different defects have different acceptance degrees, certain potential safety hazards exist in the defects of the foreign matters of the PCB, if the PCB has serious surface foreign matters, certain safety influence can be brought to the PCB, the PCB is burnt down, and fire hazard can be brought greatly.
Therefore, a method for accurately determining the defect of the panel foreign matter is needed.
Disclosure of Invention
The main purpose of the application is to provide a method, a device, equipment and a medium for discriminating the foreign matter defects of a panel, which aim to solve the technical problem that the prior art cannot discriminate the severity degree of the foreign matter defects in the panel.
In order to solve the above technical problems, the embodiments of the present application provide: a method for judging a foreign matter defect of a panel comprises the following steps:
acquiring an image to be identified of a target panel;
inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; the first image comprises target defects and corresponding labeling information;
inputting the first image into a classification model for classification to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG;
carrying out gray processing on the first image with the OK classification result to obtain a target line area;
and obtaining a defect judging result of the target panel based on the line width value of the target line area.
As some optional embodiments of the present application, the inputting the image to be identified into the target detection model to detect the target defect, and obtaining the first image includes:
inputting the image to be identified into a target detection model to detect target defects, and obtaining an image to be identified containing defect labels;
and cutting the image to be identified containing the defect labels based on the target defects to obtain a first image.
As some optional embodiments of the present application, after performing gray-scale processing on the first image with the classification result of OK, obtaining a target line area includes:
gray processing is carried out on the first image with the OK classification result, and then an OK second image is obtained;
and extracting the maximum connected domain in the OK second image to obtain a target line area.
As some optional embodiments of the present application, the extracting the largest connected domain in the OK second image to obtain the target line area includes:
extracting the largest connected domain in the OK second image, and then cutting to obtain a first circuit area image;
extracting a circuit area which is not covered by the foreign object defect in the first circuit area image, and obtaining a second circuit area image;
and obtaining the minimum circumscribed rectangle of the circuit area in the second circuit area image, and performing clipping treatment to obtain a target circuit area.
As some optional embodiments of the present application, the obtaining the defect discrimination result of the target panel based on the line width value of the target line area includes:
judging the line direction of the target line area based on the line width value of the target line area; wherein the line direction of the target line area includes a horizontal direction or a vertical direction;
and obtaining a defect judging result of the target panel based on the line direction.
As some optional embodiments of the present application, the determining, based on the line width value of the target line area, the line direction of the target line area includes:
extracting the outline of the target line area to obtain a closed area outline and boundary pixel points of the closed area outline; traversing boundary pixel points of the closed region outline to obtain a line width value of the closed region outline;
and judging the line direction of the target line area based on the line width value of the target line area.
As some optional embodiments of the present application, the determining, based on the line width value of the target line area, the line direction of the target line area includes:
calculating pixel differences between the target line area and the nearest horizontal line to obtain a first pixel difference value;
calculating pixel differences between the target line area and the nearest vertical line to obtain a second pixel difference;
and judging the line direction of the target line area based on the first pixel difference value and the second pixel difference value.
As some optional embodiments of the present application, the determining, based on the first pixel difference value and the second pixel difference value, a line direction of the target line area includes:
if the first pixel difference value is larger than the second pixel difference value, the line direction of the target line area is a vertical direction;
if the first pixel difference value is smaller than the second pixel difference value, the line direction of the target line area is a horizontal direction.
As some optional embodiments of the present application, the determining, based on the line width value of the target line area, the line direction of the target line area includes:
comparing the line width value of the target line area with a preset standard width value, and judging the line direction of the target line area; wherein the preset standard width value comprises a preset horizontal standard width value or a preset vertical standard width value.
As some optional embodiments of the present application, the preset horizontal standard width value is 2 times the vertical standard width value.
As some optional embodiments of the present application, the obtaining, based on the line direction, a defect discrimination result of the target panel includes:
if the line direction is the horizontal direction, the defect judgment result of the target panel is NG;
if the line direction is the vertical direction, the defect judgment result of the target panel is OK.
As some optional embodiments of the present application, the target detection model is obtained through training of the following steps:
acquiring a foreign matter defect sample image; wherein the foreign matter defect sample image includes an OK sample image and a NG sample image; the OK sample image refers to that the foreign matter defect coincides with a line in the vertical direction; the NG sample image refers to the superposition of the foreign matter defect and the horizontal line;
acquiring position coordinate information of a foreign object defect in the foreign object defect sample image, and cutting based on the position coordinate information of the foreign object defect to obtain a first defect sample image;
performing data enhancement processing on the first defect sample image to obtain a first defect sample image set;
training an initial target detection model based on the first defect sample image set so that the initial target detection model detects and marks the foreign object defects in the foreign object defect sample image and then outputs a second defect sample image.
As some optional embodiments of the present application, the data enhancement process includes at least one processing method of a rotation process, a flipping process, a scaling process, and an affine change process.
As some optional embodiments of the application, the initial target detection model is a preliminary graph judging model constructed by a yolo algorithm, and the two classification models are Resnet50 network models.
In order to solve the above technical problems, the embodiment of the present application further provides: a discriminating device for panel foreign matter defect includes:
the first acquisition module is used for acquiring an image to be identified of the target panel;
the target defect detection module is used for inputting the image to be identified into a target detection model to detect target defects and obtain a first image; the first image comprises target defects and corresponding labeling information;
the classification module is used for inputting the first image into a classification model to classify, so as to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG;
the target line area extraction module is used for obtaining a target line area after gray processing is carried out on the first image with the OK classification result;
and the judging module is used for obtaining a defect judging result of the target panel based on the line width value of the target line area.
In order to solve the above technical problems, the embodiment of the present application further provides: an electronic device includes a memory in which a computer program is stored, and a processor that executes the computer program to implement the method of discriminating a panel foreign matter defect as described above.
In order to solve the above technical problems, the embodiment of the present application further provides: a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method of discriminating a panel foreign matter defect as described above.
Compared with the prior art, the method for identifying the panel foreign matter defects comprises the following steps: after an image to be identified of a target panel is obtained, the image to be identified is input into a target detection model to detect and label target defects, and a first image containing the target defects and corresponding labeling information is obtained; through the steps, the target defects in the target panel can be primarily identified, so that the position areas of the foreign object defects on the target panel can be primarily judged. Inputting a first image containing target defects and corresponding labeling information into a classification model for classification, wherein the classification model judges whether a classification result of the first image is OK or NG by judging the line direction of the first image, which coincides with the foreign object defects, and the classification result is OK when the target defects coincide with lines in the vertical direction; and when the target defect coincides with the horizontal line, the classification result is NG. In order to avoid that the classification model misjudges the image of the unqualified product as an OK image, after the classification result is obtained, carrying out gray processing on the OK image in the classification result, obtaining a target line area and line width values of the target line area, judging the direction of the target line based on the line width values of the target line area, judging the judging result as unqualified if the direction of the target line is the horizontal direction, and judging the judging result as qualified if the direction of the target line is the vertical direction. The method can effectively avoid the phenomenon of missing detection or false detection in the defect detection process, thereby improving the defect detection efficiency.
Drawings
FIG. 1 is a schematic diagram of an electronic device architecture of a hardware operating environment according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a method for identifying a foreign object defect of a panel according to an embodiment of the present disclosure;
FIG. 3 is a schematic view of an image to be identified of a target panel according to an embodiment of the present application;
FIG. 4 is a schematic view of a first image according to an embodiment of the present application;
FIG. 5 is a schematic view of a first image after cropping according to an embodiment of the present application;
FIG. 6 is a schematic view of an OK second image in accordance with an embodiment of the present application;
FIG. 7 is a schematic diagram of a second line area image in accordance with an embodiment of the present application;
FIG. 8 is a schematic diagram of a target line area according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a device for discriminating a foreign matter defect of a panel according to an embodiment of the present application;
wherein 1001-processor, 1002-communication bus, 1003-user interface, 1004-network interface, 1005-memory.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The continuous improvement of the industrial production and manufacturing level accelerates the updating of the electronic products, so that higher requirements are put forward on the underlying infrastructure hardware. The quality of the printed circuit board (Printed Circuit board, PCB) as an important base component of the integrated circuit determines the overall performance of the electronic product.
But it is difficult to avoid defects in the PCB produced by the fabrication due to various factors during the PCB production process. The traditional PCB defect detection is manually marked with the defect information, the efficiency is low, the accuracy is low, a defect detection algorithm based on deep learning is gradually researched in recent years, but the traditional convolutional neural network is difficult to consider the feature information of the global and detail positions for the defect detection of the PCB image, and the accuracy of detection and the accuracy of marking are low. Therefore, for the problem of detecting foreign matter defects of PCB images, the main solutions of the embodiments of the present application are: provided are a method, apparatus, device and medium for identifying a foreign matter defect of a panel, the method comprising the steps of: acquiring an image to be identified of a target panel; inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; the first image comprises target defects and corresponding labeling information; inputting the first image into a classification model for classification to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG; carrying out gray processing on the first image with the OK classification result to obtain a target line area; and obtaining a defect judging result of the target panel based on the line width value of the target line area.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device in a hardware running environment according to an embodiment of the present application.
As shown in fig. 1, the electronic device may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) Memory or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 1 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and an electronic program may be included in the memory 1005 as one type of storage medium.
In the electronic device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention may be disposed in the electronic device, where the electronic device invokes the device for identifying a panel foreign object defect stored in the memory 1005 through the processor 1001, and executes the method for identifying a panel foreign object defect provided in the embodiment of the present application.
Referring to fig. 2, embodiments of the present application provide: a method for identifying panel foreign matter defects comprises the following steps:
s10, acquiring an image to be identified of a target panel; the image to be identified is shown in fig. 3.
The target panel is a circuit panel that needs to detect whether a foreign object defect exists, and the target panel may or may not have a foreign object defect. In addition, the image to be identified of the target panel is obtained by manual photographing or AOI (Automated Optical Inspection, automatic optical inspection) based on the target panel, and the image to be identified of the target panel may also be obtained after being processed by a computer vision algorithm. Wherein the AOI is a device for detecting common defects encountered in welding production based on optical principles. AOI is a new test technology, but is rapidly developed, and various manufacturers have proposed AOI test equipment. When automatically detecting, the machine automatically scans the PCB through the camera, collects images and the like. The computer vision algorithm uses a camera and a computer to replace human eyes to recognize, track and measure targets and perform graphic processing, so that the computer is processed into images which are more suitable for human eyes to observe or transmit to an instrument to recognize. The panel refers to an industrial panel produced in an industrial manufacturing process, such as a process panel, a liquid crystal panel, an electronic panel, and the like.
Step S20, inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; the first image comprises target defects and corresponding labeling information.
In practical application, whether the target defect exists in the image to be identified can be detected by performing feature identification through an R-CNN algorithm, a YOLO algorithm, an SSD algorithm and the like, but the R-CNN algorithm is poor in detection efficiency and has a great amount of waste in storage resources. Therefore, in the embodiment of the application, when the first target defect area is identified in the image to be identified, a preliminary graph judgment model constructed by a yolo algorithm is adopted.
It should be noted that, in order to improve the detection accuracy of the preliminary graph judgment model, the target detection model in the embodiment of the application is obtained through training in the following steps: acquiring a foreign matter defect sample image; wherein the foreign matter defect sample image includes an OK sample image and a NG sample image; the OK sample image refers to that the foreign matter defect coincides with a line in the vertical direction; the NG sample image refers to the superposition of the foreign matter defect and the horizontal line; acquiring position coordinate information of a foreign object defect in the foreign object defect sample image, and cutting based on the position coordinate information of the foreign object defect to obtain a first defect sample image; performing data enhancement processing on the first defect sample image to obtain a first defect sample image set; training an initial target detection model based on the first defect sample image set so that the initial target detection model detects and marks the foreign object defects in the foreign object defect sample image and then outputs a second defect sample image. Wherein, in order to enlarge the sample size, the data enhancement processing includes at least one processing method of rotation processing, flipping processing, scaling processing, and affine change processing.
Specifically, in step S20, the inputting the image to be identified into the target detection model to detect the target defect, and acquiring the first image includes: and inputting the image to be identified into a target detection model to detect target defects, and obtaining the image to be identified containing defect labels, as shown in fig. 4. The defect label in the step can be a type label, a size label or a position label for the defect. In this embodiment of the present application, since the setting of this step is to distinguish whether the image to be identified includes the target foreign object defect, the defect labeling described herein is a preferred position labeling, so that the image to be identified is cut based on the position labeling information of the defect, so as to obtain a first image only including the defect area, that is, the cut first image is shown in fig. 5.
S30, inputting the first image into a classification model for classification, and obtaining a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; and when the target defect coincides with the horizontal line, the classification result is NG.
When training the classification model, training the classification model by using a sample image set obtained by taking a cut defect area image as a sample image and performing data enhancement processing on the sample image to obtain a classification result. The two classification models are Resnet50 network models, the network is mainly composed of four groups of convolution blocks, each group of convolution blocks comprises different numbers of convolution layers, and features with different scales can be effectively obtained and fused. In addition, a residual network is added, and a jump connection mode is adopted, so that the network parameter complexity is increased, and the gradient disappearance phenomenon in the back propagation is reduced, thereby helping the classification of foreign object defects. The training loss function adopts a cross entropy loss function to help guide the model to realize the classification effect.
It should be noted that, the original data image, usually a color image, is obtained by manually photographing or AOI photographing the target panel, so that the feature information in the obtained original data image is not obvious and contains a lot of noise information irrelevant to the feature; therefore, after the input image is input to the target detection model and the classification model, the output image is still a color image, so in order to remove noise interference in the image to highlight the feature information, the embodiment of the application performs image denoising enhancement processing, such as image graying, gray histogram equalization processing or denoising filtering processing, on the first image with the classification result of OK after the first image is acquired. Namely:
and step S40, carrying out gray processing on the first image with the OK classification result, and acquiring a target line area.
It should be noted that, in daily life, an RGB color space is often used for describing a color space, and a three-dimensional color RGB space is formed by three primary colors of red, green and blue (RGB), and an image may be understood as being formed by overlapping three primary colors according to different proportions. When the image captured by the camera is digitally discretized, the pixel value of each pixel in the image is in the interval range of [0, 225], the image can be converted into a three-dimensional array [ H, W, C ], wherein H represents the transverse quantity of the image, W represents the column vector of the image, C represents the channel quantity of the image, and the three channels [35] of RGB are formed under the RGB color space, when the computer operates the image data, the pixel values of the three channels are required to be calculated, the operation quantity is too large, so that the color image of the three channels can be converted into a single-channel gray-scale image [36], and the parameter calculation quantity is greatly reduced, which is the gray-scale processing.
In practical application, step S40 of performing gray processing on the first image with the OK classification result to obtain a target line area includes:
step S41, gray processing is carried out on the first image with the OK classification result, and then an OK second image is obtained. At the moment, after the OK second image is subjected to gray processing, the original image of the PCB defect is converted into a single-channel gray image, the color information is lost, and the contrast of the foreground and the background and the overall and local brightness information is more highlighted.
Step S42, extracting the maximum connected domain in the OK second image, and obtaining the target line area in the OK second image. In the obtained OK second image, only the target line and a part of the defect area are contained and previously not connected, and in practical application, the largest connected domain in the image is the line area with a large probability, so that the largest connected domain in the OK second image is extracted to obtain the target line area in the image. As a preferred solution, in step S42, the extracting the largest connected domain in the OK second image to obtain the target line area in the OK second image includes:
step S421, extracting the largest connected domain in the OK second image, and then clipping to obtain a first line area image. Through the above steps, the first line area image may include several portions of the line area, and the foreign object defect area.
Step S422, extracting a circuit area not covered by the foreign object defect in the first circuit area image, and obtaining a second circuit area image. After removing the foreign matter defect area, the second circuit area image obtained through the steps extracts the circuit area which is not covered by the foreign matter defect in the first circuit area image.
Step S423, obtaining the minimum circumscribed rectangle of the line area in the second line area image, and performing clipping processing to obtain a target line area. Since the second line area image only includes the line area and the line width of the line is uniform, the line width calculation is only required for the line in a certain step, and therefore, the minimum circumscribed rectangle clipping is performed on the line area in the second line area image to obtain the target line area.
For the technical content of the step S40, for example, the following is:
operating the cut OK image containing the defects, and firstly graying the cut OK image to obtain a corresponding gray level image, namely an OK second image is shown in figure 6; the maximum connected domain in the gray scale map is extracted, a line region is obtained, a threshold interval is set to (165, 255), clipping is performed, and a value between pixel values (79.3, 200) is further adopted as a reserved area region in consideration of that the residual of part of defects is still on the line, and foreign matter defect regions in the area region are removed, as shown in fig. 7. Since the line widths are uniform, only a part of the line area is reserved for calculating the line width, that is, the upper part of the reserved line, and then the cutting is performed after the minimum external moment of the part of the line is acquired, as shown in fig. 8. The width of the external moment can be obtained directly by a function, namely the width can be expressed as the line width.
And step S50, obtaining a defect judging result of the target panel based on the line width value of the target line area. Specifically, if the line direction is a horizontal direction, the defect determination result of the target panel is NG; if the line direction is the vertical direction, the defect judgment result of the target panel is OK.
In practical application, step S50 of obtaining the defect discrimination result of the target panel based on the line width value of the target line area includes: judging the line direction of the target line area based on the line width value of the target line area; wherein the line direction of the target line area includes a horizontal direction or a vertical direction; and obtaining a defect judging result of the target panel based on the line direction.
It should be noted that, in the actual circuit board printing process, a layer of lines in the horizontal direction needs to be printed first, and a layer of lines in the vertical direction needs to be printed last, and the timing of occurrence of the foreign object defect is generally in the printing process. Therefore, if the foreign matter defect falls on the line in the horizontal direction, the performance of the subsequent product is larger, and if the foreign matter defect falls on the line in the vertical direction, the performance of the subsequent product is smaller.
In an actual application, the determining the line direction of the target line area based on the line width value of the target line area includes: extracting the outline of the target line area to obtain a closed area outline and boundary pixel points of the closed area outline; traversing boundary pixel points of the closed region outline to obtain a line width value of the closed region outline; and judging the line direction of the target line area based on the line width value of the target line area.
In an actual application, the determining the line direction of the target line area based on the line width value of the target line area includes: calculating pixel differences between the target line area and the nearest horizontal line to obtain a first pixel difference value; calculating pixel differences between the target line area and the nearest vertical line to obtain a second pixel difference; and judging the line direction of the target line area based on the first pixel difference value and the second pixel difference value. Wherein the determining the line direction of the target line area based on the first pixel difference value and the second pixel difference value includes: if the first pixel difference value is larger than the second pixel difference value, the line direction of the target line area is a vertical direction; if the first pixel difference value is smaller than the second pixel difference value, the line direction of the target line area is a horizontal direction.
In practical application, other methods may be used to determine the direction of the target line, for example: comparing the line width value of the target line area with a preset standard width value, and judging the line direction of the target line area; wherein the preset standard width value comprises a preset horizontal standard width value or a preset vertical standard width value. Wherein the preset horizontal standard width value is 2 times of the vertical standard width value.
In the actual business scenario, the foreign object defect is OK on the vertical line and NG on the horizontal line. In a given provided line width value, the line in the horizontal direction is 2 times the line width in the vertical direction. If the calculated line width is 2.687, the pixel difference from the given vertical line is 2-3 pixels, and the length is about 1/2 of the width of the horizontal line, so that it can be determined that the foreign object is defective on the vertical line, and thus the final defect determination result is OK.
Compared with the prior art, the method for identifying the panel foreign matter defects comprises the following steps: after an image to be identified of a target panel is obtained, the image to be identified is input into a target detection model to detect and label target defects, and a first image containing the target defects and corresponding labeling information is obtained; through the steps, the target defects in the target panel can be primarily identified, so that the position areas of the foreign object defects on the target panel can be primarily judged. Inputting a first image containing target defects and corresponding labeling information into a classification model for classification, wherein the classification model judges whether a classification result of the first image is OK or NG by judging the line direction of the first image, which coincides with the foreign object defects, and the classification result is OK when the target defects coincide with lines in the vertical direction; and when the target defect coincides with the horizontal line, the classification result is NG. In order to avoid that the classification model misjudges the image of the unqualified product as an OK image, after the classification result is obtained, carrying out gray processing on the OK image in the classification result, obtaining a target line area and line width values of the target line area, judging the direction of the target line based on the line width values of the target line area, judging the judging result as unqualified if the direction of the target line is the horizontal direction, and judging the judging result as qualified if the direction of the target line is the vertical direction. The method can effectively avoid the phenomenon of missing detection or false detection in the defect detection process, thereby improving the defect detection efficiency.
Based on the same inventive concept, as shown in fig. 9, the present application further provides: a discriminating device for panel foreign matter defect includes:
the first acquisition module is used for acquiring an image to be identified of the target panel;
the target defect detection module is used for inputting the image to be identified into a target detection model to detect target defects and obtain a first image; the first image comprises target defects and corresponding labeling information;
the classification module is used for inputting the first image into a classification model to classify, so as to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG;
the target line area extraction module is used for obtaining a target line area after gray processing is carried out on the first image with the OK classification result;
and the judging module is used for obtaining a defect judging result of the target panel based on the line width value of the target line area.
It should be noted that, each module in the device for determining a panel foreign object defect in this embodiment corresponds to each step in the method for determining a panel foreign object defect in the foregoing embodiment, so the specific implementation of this embodiment may refer to the implementation of the foregoing method for determining a panel foreign object defect, and will not be described herein.
Furthermore, in an embodiment, embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, the processor executing the computer program to implement the method as described above.
In some embodiments, the computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; but may be a variety of devices including one or any combination of the above memories. The computer may be a variety of computing devices including smart terminals and servers.
In some embodiments, the executable instructions may be in the form of programs, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and they may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.
As an example, the executable instructions may, but need not, correspond to files in a file system, may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a hypertext markup language (HTML, hyper Text Markup Language) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).
As an example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices located at one site or, alternatively, distributed across multiple sites and interconnected by a communication network.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.
From the above description of embodiments, it will be clear to a person skilled in the art that the above embodiment method may be implemented by means of software plus a necessary general hardware platform, but may of course also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising several instructions for causing a multimedia terminal device (which may be a mobile phone, a computer, a television receiver, or a network device, etc.) to perform the method described in the embodiments of the present application.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the claims, and all equivalent structures or equivalent processes using the descriptions and drawings of the present application, or direct or indirect application in other related technical fields are included in the scope of the claims of the present application.
Claims (17)
1. The method for judging the foreign matter defect of the panel is characterized by comprising the following steps of:
acquiring an image to be identified of a target panel;
inputting the image to be identified into a target detection model to detect target defects, and obtaining a first image; the first image comprises target defects and corresponding labeling information;
inputting the first image into a classification model for classification to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG;
carrying out gray processing on the first image with the OK classification result to obtain a target line area;
and obtaining a defect judging result of the target panel based on the line width value of the target line area.
2. The method for determining a foreign object defect of a panel according to claim 1, wherein inputting the image to be identified into a target detection model to detect the target defect, and obtaining the first image comprises:
inputting the image to be identified into a target detection model to detect target defects, and obtaining an image to be identified containing defect labels;
and cutting the image to be identified containing the defect labels based on the target defects to obtain a first image.
3. The method for determining a foreign object defect of a panel according to claim 1, wherein the step of obtaining the target line area after performing the gray-scale processing on the first image having the classification result of OK comprises:
gray processing is carried out on the first image with the OK classification result, and then an OK second image is obtained;
and extracting the maximum connected domain in the OK second image to obtain a target line area in the OK second image.
4. The method for determining a foreign object defect on a panel according to claim 3, wherein extracting the largest connected domain in the OK second image to obtain the target line area in the OK second image comprises:
extracting the largest connected domain in the OK second image, and then cutting to obtain a first circuit area image;
extracting a circuit area which is not covered by the foreign object defect in the first circuit area image, and obtaining a second circuit area image;
and obtaining the minimum circumscribed rectangle of the circuit area in the second circuit area image, and performing clipping treatment to obtain a target circuit area.
5. The method according to claim 1, wherein the obtaining the defect determination result of the target panel based on the line width value of the target line area comprises:
judging the line direction of the target line area based on the line width value of the target line area; wherein the line direction of the target line area includes a horizontal direction or a vertical direction;
and obtaining a defect judging result of the target panel based on the line direction.
6. The method according to claim 5, wherein determining the line direction of the target line area based on the line width value of the target line area comprises:
extracting the outline of the target line area to obtain a closed area outline and boundary pixel points of the closed area outline; traversing boundary pixel points of the closed region outline to obtain a line width value of the closed region outline;
and judging the line direction of the target line area based on the line width value of the target line area.
7. The method according to claim 6, wherein the determining the line direction of the target line area based on the line width value of the target line area includes:
calculating pixel differences between the target line area and the nearest horizontal line to obtain a first pixel difference value;
calculating pixel differences between the target line area and the nearest vertical line to obtain a second pixel difference;
and judging the line direction of the target line area based on the first pixel difference value and the second pixel difference value.
8. The method according to claim 7, wherein determining the line direction of the target line area based on the first pixel difference value and the second pixel difference value, comprises:
if the first pixel difference value is larger than the second pixel difference value, the line direction of the target line area is a vertical direction;
if the first pixel difference value is smaller than the second pixel difference value, the line direction of the target line area is a horizontal direction.
9. The method according to claim 6, wherein the determining the line direction of the target line area based on the line width value of the target line area includes:
comparing the line width value of the target line area with a preset standard width value, and judging the line direction of the target line area; wherein the preset standard width value comprises a preset horizontal standard width value or a preset vertical standard width value.
10. The method according to claim 9, wherein the predetermined horizontal standard width value is 2 times the vertical standard width value.
11. The method according to claim 5, wherein obtaining the defect discrimination result of the target panel based on the line direction comprises:
if the line direction is the horizontal direction, the defect judgment result of the target panel is NG;
if the line direction is the vertical direction, the defect judgment result of the target panel is OK.
12. The method for discriminating a foreign matter defect of a panel according to claim 1 wherein said target detection model is obtained by training the steps of:
acquiring a foreign matter defect sample image; wherein the foreign matter defect sample image includes an OK sample image and a NG sample image; the OK sample image refers to that the foreign matter defect coincides with a line in the vertical direction; the NG sample image refers to the superposition of the foreign matter defect and the horizontal line;
acquiring position coordinate information of a foreign object defect in the foreign object defect sample image, and cutting based on the position coordinate information of the foreign object defect to obtain a first defect sample image;
performing data enhancement processing on the first defect sample image to obtain a first defect sample image set;
training an initial target detection model based on the first defect sample image set so that the initial target detection model detects and marks the foreign object defects in the foreign object defect sample image and then outputs a second defect sample image.
13. The method according to claim 12, wherein the data enhancement processing includes at least one processing method of rotation processing, flipping processing, scaling processing, and affine change processing.
14. The method for determining a foreign object defect on a panel according to claim 12, wherein the initial target detection model is a preliminary graph determination model constructed by a yolo algorithm, and the classification model is a Resnet50 network model.
15. A device for discriminating a foreign matter defect of a panel, comprising:
the first acquisition module is used for acquiring an image to be identified of the target panel;
the target defect detection module is used for inputting the image to be identified into a target detection model to detect target defects and obtain a first image; the first image comprises target defects and corresponding labeling information;
the classification module is used for inputting the first image into a classification model to classify, so as to obtain a classification result; wherein the classification result comprises OK and NG; wherein, when the target defect coincides with the vertical line, the classification result is OK; when the target defect coincides with the horizontal line, the classification result is NG;
the target line area extraction module is used for obtaining a target line area after gray processing is carried out on the first image with the OK classification result;
and the judging module is used for obtaining a defect judging result of the target panel based on the line width value of the target line area.
16. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method for discriminating a panel foreign object defect according to any one of claims 1 to 14.
17. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and a processor executes the computer program to implement the method for discriminating a panel foreign matter defect according to any one of claims 1 to 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310374308.1A CN116091503B (en) | 2023-04-10 | 2023-04-10 | Method, device, equipment and medium for discriminating panel foreign matter defects |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310374308.1A CN116091503B (en) | 2023-04-10 | 2023-04-10 | Method, device, equipment and medium for discriminating panel foreign matter defects |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116091503A true CN116091503A (en) | 2023-05-09 |
| CN116091503B CN116091503B (en) | 2023-06-13 |
Family
ID=86210588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310374308.1A Active CN116091503B (en) | 2023-04-10 | 2023-04-10 | Method, device, equipment and medium for discriminating panel foreign matter defects |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116091503B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116542975A (en) * | 2023-07-05 | 2023-08-04 | 成都数之联科技股份有限公司 | Defect classification method, device, equipment and medium for glass panel |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6831998B1 (en) * | 2000-06-22 | 2004-12-14 | Hitachi, Ltd. | Inspection system for circuit patterns and a method thereof |
| WO2010024067A1 (en) * | 2008-08-29 | 2010-03-04 | 株式会社 日立ハイテクノロジーズ | Defect inspection method, and defect inspection device |
| US20130011043A1 (en) * | 2010-01-22 | 2013-01-10 | Hitachi High-Technologies Corporation | Defect inspecting device and defect inspecting method |
| CN105092604A (en) * | 2014-05-15 | 2015-11-25 | 由田新技股份有限公司 | Line information and non-line area information acquisition method and line defect detection method |
| CN107367682A (en) * | 2017-06-26 | 2017-11-21 | 王俊 | Printed circuit board repair method and system |
| CN108510476A (en) * | 2018-03-14 | 2018-09-07 | 五邑大学 | A kind of Mobile phone screen wireline inspection method based on machine vision |
| CN113688807A (en) * | 2021-10-26 | 2021-11-23 | 中科慧远视觉技术(北京)有限公司 | Self-adaptive defect detection method, device, recognition system and storage medium |
| CN113870256A (en) * | 2021-12-01 | 2021-12-31 | 成都数之联科技有限公司 | PCB defect evaluation method, device, equipment and medium |
| CN113935960A (en) * | 2021-09-28 | 2022-01-14 | 景旺电子科技(珠海)有限公司 | Defect overhauling method, device and equipment |
| CN114186481A (en) * | 2021-11-26 | 2022-03-15 | 江门市浩远科技有限公司 | Efficient circuit board washing method based on machine learning |
| CN115082466A (en) * | 2022-08-22 | 2022-09-20 | 江苏庆慈机械制造有限公司 | PCB surface welding spot defect detection method and system |
| CN115359043A (en) * | 2022-10-18 | 2022-11-18 | 启东市航新实用技术研究所 | Intelligent detection method for foreign matters on PCB surface |
| CN115731220A (en) * | 2022-11-30 | 2023-03-03 | 广州致景信息科技有限公司 | Grey cloth defect positioning and classifying method, system, equipment and storage medium |
| CN115843156A (en) * | 2022-11-23 | 2023-03-24 | 惠州中京电子科技有限公司 | PCB circuit repairing method and device |
-
2023
- 2023-04-10 CN CN202310374308.1A patent/CN116091503B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6831998B1 (en) * | 2000-06-22 | 2004-12-14 | Hitachi, Ltd. | Inspection system for circuit patterns and a method thereof |
| WO2010024067A1 (en) * | 2008-08-29 | 2010-03-04 | 株式会社 日立ハイテクノロジーズ | Defect inspection method, and defect inspection device |
| US20130011043A1 (en) * | 2010-01-22 | 2013-01-10 | Hitachi High-Technologies Corporation | Defect inspecting device and defect inspecting method |
| CN105092604A (en) * | 2014-05-15 | 2015-11-25 | 由田新技股份有限公司 | Line information and non-line area information acquisition method and line defect detection method |
| CN107367682A (en) * | 2017-06-26 | 2017-11-21 | 王俊 | Printed circuit board repair method and system |
| CN108510476A (en) * | 2018-03-14 | 2018-09-07 | 五邑大学 | A kind of Mobile phone screen wireline inspection method based on machine vision |
| CN113935960A (en) * | 2021-09-28 | 2022-01-14 | 景旺电子科技(珠海)有限公司 | Defect overhauling method, device and equipment |
| CN113688807A (en) * | 2021-10-26 | 2021-11-23 | 中科慧远视觉技术(北京)有限公司 | Self-adaptive defect detection method, device, recognition system and storage medium |
| CN114186481A (en) * | 2021-11-26 | 2022-03-15 | 江门市浩远科技有限公司 | Efficient circuit board washing method based on machine learning |
| CN113870256A (en) * | 2021-12-01 | 2021-12-31 | 成都数之联科技有限公司 | PCB defect evaluation method, device, equipment and medium |
| CN115082466A (en) * | 2022-08-22 | 2022-09-20 | 江苏庆慈机械制造有限公司 | PCB surface welding spot defect detection method and system |
| CN115359043A (en) * | 2022-10-18 | 2022-11-18 | 启东市航新实用技术研究所 | Intelligent detection method for foreign matters on PCB surface |
| CN115843156A (en) * | 2022-11-23 | 2023-03-24 | 惠州中京电子科技有限公司 | PCB circuit repairing method and device |
| CN115731220A (en) * | 2022-11-30 | 2023-03-03 | 广州致景信息科技有限公司 | Grey cloth defect positioning and classifying method, system, equipment and storage medium |
Non-Patent Citations (7)
| Title |
|---|
| THI TRAM ANH PHAM 等: "Defect Detection in Printed Circuit Boards Using Semi-Supervised Learning", 《SENSORS》, vol. 23, no. 06, pages 1 - 17 * |
| 何止戈: "基于深度学习方法的PCB图像缺陷检测", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 2020, pages 135 - 340 * |
| 张波 等: "基于小波变换的PCB缺陷视觉检测技术研究", 《系统仿真学报》, no. 08, pages 1864 - 1866 * |
| 李云峰 等: "基于梯度方向信息熵的印刷电路板缺陷检测", 《中国机械工程》, vol. 28, no. 06, pages 695 - 701 * |
| 王伯雄 等: "基于Legendre正交矩的亚像素线宽测量方法", 《清华大学学报(自然科学版)》, vol. 56, no. 02, pages 218 - 222 * |
| 苑玮琦 等: "基于局部马尔科夫模式的FPC薄浆缺陷检测", 《仪器仪表学报》, vol. 39, no. 06, pages 207 - 214 * |
| 邹梓龙: "基于无人机航拍图像的输电线提取与异物检测方法研究", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》, no. 2023, pages 031 - 13 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116542975A (en) * | 2023-07-05 | 2023-08-04 | 成都数之联科技股份有限公司 | Defect classification method, device, equipment and medium for glass panel |
| CN116542975B (en) * | 2023-07-05 | 2023-09-12 | 成都数之联科技股份有限公司 | Defect classification method, device, equipment and medium for glass panel |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116091503B (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114266773B (en) | Display panel defect positioning method, device, equipment and storage medium | |
| CN108182383B (en) | Vehicle window detection method and device | |
| CN111080622A (en) | Neural network training method, workpiece surface defect classification and detection method and device | |
| CN114202543B (en) | Method, device, equipment and medium for detecting dirt defects of PCB (printed circuit board) | |
| CN116542975B (en) | Defect classification method, device, equipment and medium for glass panel | |
| CN115205223B (en) | Visual inspection method and device for transparent object, computer equipment and medium | |
| CN116091503B (en) | Method, device, equipment and medium for discriminating panel foreign matter defects | |
| CN115578377B (en) | Panel defect detection method, training method, device, equipment and medium | |
| CN109389110B (en) | Region determination method and device | |
| CN115239734A (en) | Model training method, device, storage medium, equipment and computer program product | |
| CN113850749A (en) | Method for training defect detector | |
| CN115861327A (en) | PCB color change defect detection method, device, equipment and medium | |
| CN111626249A (en) | Method and device for identifying geometric figure in topic image and computer storage medium | |
| CN117392042A (en) | Defect detection method, defect detection apparatus, and storage medium | |
| CN115546219B (en) | Detection plate type generation method, plate card defect detection method, device and product | |
| CN114937037B (en) | Product defect detection method, device and equipment and readable storage medium | |
| CN113822836A (en) | Method of marking an image | |
| CN115578362A (en) | Defect detection method and device for electrode coating, electronic device and medium | |
| CN111402185A (en) | Image detection method and device | |
| CN111935480B (en) | Detection method for image acquisition device and related device | |
| CN114926429A (en) | Method, device and equipment for detecting trace length and readable storage medium | |
| CN112967224A (en) | Electronic circuit board detection system, method and medium based on artificial intelligence | |
| CN113449617A (en) | Track safety detection method, system, device and storage medium | |
| CN114627114B (en) | Method, system, device and medium for measuring severity of product defect | |
| CN117495846B (en) | Image detection method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |