CN111080615A - PCB defect detection system and detection method based on convolutional neural network - Google Patents

PCB defect detection system and detection method based on convolutional neural network Download PDF

Info

Publication number
CN111080615A
CN111080615A CN201911273062.9A CN201911273062A CN111080615A CN 111080615 A CN111080615 A CN 111080615A CN 201911273062 A CN201911273062 A CN 201911273062A CN 111080615 A CN111080615 A CN 111080615A
Authority
CN
China
Prior art keywords
pcb
image
defect
detecting
target image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911273062.9A
Other languages
Chinese (zh)
Other versions
CN111080615B (en
Inventor
黄家水
张发恩
徐华泽
唐永亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ainnovation Chongqing Technology Co ltd
Original Assignee
Ainnovation Chongqing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ainnovation Chongqing Technology Co ltd filed Critical Ainnovation Chongqing Technology Co ltd
Priority to CN201911273062.9A priority Critical patent/CN111080615B/en
Publication of CN111080615A publication Critical patent/CN111080615A/en
Application granted granted Critical
Publication of CN111080615B publication Critical patent/CN111080615B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a PCB defect detection system based on a convolutional neural network, which comprises: the first image input module is used for inputting a PCB image to be detected; the key point detection module is used for detecting at least two defect detection key points in the PCB image and extracting a PCB defect target image from the PCB image through each detected defect detection key point; the standard image acquisition module is used for acquiring and outputting a standard image corresponding to the PCB defect target image; the second image input module is used for simultaneously inputting the PCB defect target image and the standard image into a PCB image segmentation module; the PCB image segmentation module is used for segmenting the PCB defect target image based on the standard image and finally obtaining the specific area with the defect in the PCB defect target image.

Description

PCB defect detection system and detection method based on convolutional neural network
Technical Field
The invention relates to a PCB defect detection system, in particular to a PCB defect detection system and a detection method based on a convolutional neural network for detecting defects of a PCB printed circuit board.
Background
Pcb (printed Circuit board), which is called printed Circuit board (pcb) and is also called printed Circuit board (pcb), is an important electronic component, a support for electronic components, and a carrier for electrical connection of electronic components. Whether the PCB has defects or not directly affects the performance of equipment using the PCB, so that the defect detection of the PCB is particularly necessary.
At present, the defect detection of the PCB circuit board is mainly the traditional manual visual inspection, the manual visual inspection has higher omission factor and false detection rate, the detection efficiency of the visual inspection is very low, a large amount of manpower is consumed, or the production cost of an enterprise is directly improved, and the market competitiveness of the product is reduced.
Disclosure of Invention
The invention aims to provide a PCB defect detection system based on a convolutional neural network and a method for detecting defects of a PCB printed circuit board by the PCB defect detection system based on the convolutional neural network, so as to solve the technical problems.
In order to achieve the purpose, the invention adopts the following technical scheme:
a PCB defect detection system based on a convolutional neural network is provided, which is used for detecting whether a PCB printed circuit board has manufacturing defects or not, and comprises:
a first image input module for inputting PCB image I to be detectedo；
A key point detection module connected with the first image input module and used for detecting the PCB image IoDetecting at least two defect detection key points, and detecting each defect detection key point from the PCB image IoExtracting a PCB defect target image I't；
A standard image acquisition module for acquiring the PCB defect target image I'tCorresponding standard image IsAnd outputting;
second image input modules respectively connected to the switchesThe key point detection module and the standard image acquisition module are used for acquiring the PCB defect target image I'tAnd the standard image IsSimultaneously inputting the images into a PCB image segmentation module;
the PCB image segmentation module is connected with the second image input module and used for being based on the standard image IsFor the PCB defect target image I'tCarrying out defect image segmentation and finally obtaining the PCB defect target image I'tIn which the specific area of the defect is present.
The invention also provides a PCB defect detection method based on the convolutional neural network, which is realized by applying the PCB defect detection system and comprises the following steps:
step S1, the PCB defect detection system acquires the PCB image I to be detectedo；
Step S2, the PCB defect detection system carries out detection on the PCB image IoDetecting defect key points to obtain at least two defect detection key points;
step S3, the PCB defect detecting system detects the key points based on each defect and from the PCB image IoExtracting the PCB defect target image I't；
Step S4, the PCB defect detection system enables the PCB defect target image I'tAnd a preset standard image IsMatching detection is carried out, and the target image I 'of the PCB defects is obtained'tCarrying out defect image segmentation and finally obtaining the PCB defect target image I'tThe specific area in which the defect exists.
As a preferable aspect of the present invention, the step S1 includes an image preprocessing process, and the image preprocessing process specifically includes the following steps:
step S11, the PCB defect detection system scales the PCB image I according to a preset scaleoReducing to obtain a scaled PCB image Ir；
Step S12, the PCB defect detection system converts the PCB image IrPerforming image normalizationProcessing to obtain a normalized PCB image Ind。
As a preferred embodiment of the present invention, in the step S2, the PCB defect detecting system detects the PCB image IndThe specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system converts the PCB image IndInputting the data into a preset key point detection model, and then outputting a score map S representing the confidence coefficient of the defect key pointd；
Step S22, the PCB defect detecting system calculates the score map SdSelecting the pixel point with the maximum confidence as the corresponding defect detection key point corresponding to the channel, and calculating to obtain the defect detection key point in the PCB image IrThe specific coordinates in (1);
step S23, the PCB defect detecting system calculates the scaling of the step S11 by reverse-deduction to obtain the PCB image I of each defect detecting key pointoThe specific coordinates in (1).
As a preferred embodiment of the present invention, in step S3, the PCB defect detecting system detects the PCB defect from the PCB image IoExtracting the PCB defect target image I'tThe specific method comprises the following steps:
step S31, the PCB defect detecting system detects the critical point based on each defect in the PCB image IOFrom the PCB image IOIn-process cutting out target image It；
Step S32, the PCB defect detection system enables the target image ItIs adjusted to a preset size to obtain the PCB defect target image I 'after size adjustment't。
As a preferred embodiment of the present invention, in the step S4, the PCB defect detecting system detects the PCB defect target image I'tThe specific method for segmenting the defect image comprises the following steps:
step S41, the PCB defect detection system acquires the PCB defect target image I'tAnd the PCB defect target image I'tCorresponding standard image Is；
Step S42, the PCB defect detection system enables the PCB defect target image I'tAnd the standard image IsConnected in the channel direction to form a channel image Ic；
Step S43, the PCB defect detection system converts the channel image IcCarrying out image normalization processing to obtain a normalized PCB image Ins；
Step S44, the PCB defect detection system converts the PCB image InsInputting the PCB image I into an image segmentation modelnsPredicting whether there is defect and outputting a defect probability map SS；
Step S45, the PCB defect detecting system judges the PCB image InsWhether the pixel value corresponding to each pixel point in the image is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains the PCB defect target image I 'according to the judgment result'tThe specific area in which the defect exists.
As a preferable mode of the present invention, the threshold value in the step S45 is 0.5.
The PCB defect detection system provided by the invention can automatically detect whether the product defects exist in the PCB based on the deep learning convolutional neural network, solves the problems of easy missing detection and false detection in the traditional manual visual detection mode, greatly improves the detection efficiency and the detection accuracy, is beneficial to improving the product quality of production enterprises and improves the market competitiveness of products.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a schematic structural diagram of a PCB defect detection system provided by the present invention;
FIG. 2 is a diagram of the steps of a method for detecting PCB defects of a PCB printed circuit board using the PCB defect detecting system provided by the present invention;
FIG. 3 is a diagram of an input PCB image I in the PCB defect detection method provided by the inventionoA method step diagram for image preprocessing;
FIG. 4 is a diagram of a PCB image I in the PCB defect detection method provided by the inventionndCarrying out a specific step diagram of defect key point detection;
FIG. 5 is a diagram of a slave PCB image I in the PCB defect detection method provided by the inventionoExtracting a PCB defect target image I'tThe specific steps of (1);
FIG. 6 is a target image I 'of PCB defects in the PCB defect detection method provided by the invention'tAnd (3) a specific method step diagram for carrying out defect image segmentation.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
Wherein the showings are for the purpose of illustration only and are shown by way of illustration only and not in actual form, and are not to be construed as limiting the present patent; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not indicated or implied that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and are not to be construed as limitations of the present patent, and the specific meanings of the terms may be understood by those skilled in the art according to specific situations.
In the description of the present invention, unless otherwise explicitly specified or limited, the term "connected" or the like, if appearing to indicate a connection relationship between the components, is to be understood broadly, for example, as being fixed or detachable or integral; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or may be connected through one or more other components or may be in an interactive relationship with one another. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Referring to fig. 1, the present embodiment provides a PCB defect detecting system based on a convolutional neural network, for detecting whether a manufacturing defect exists in a PCB, the PCB defect detecting system includes:
a first image input module 1 for inputting a PCB image I to be detectedo；
The key point detection module 2 is connected with the first image input module 1 and used for detecting the PCB image IoDetecting at least two defect detection key points, and detecting each defect detection key point from the PCB image IoExtracting a PCB defect target image I't；
A standard image obtaining module 3 for obtaining a PCB defect target image I'tCorresponding standard image IsAnd outputting;
the second image input module 4 is respectively connected with the key point detection module 2 and the standard image acquisition module 3, and is used for acquiring a PCB defect target image I'tAnd a standard image IsSimultaneously inputting the images into a PCB image segmentation module;
a PCB image segmentation module 5 connected with the second image input module 4 and used for based on the standard image IsTo PCB Defect target image I'tCarrying out defect image segmentation and finally obtaining a PCB defect target image I'tIn which the specific area of the defect is present.
The invention also provides a method for detecting the PCB defect based on the convolutional neural network, which is realized by applying the PCB defect detection system, and please refer to FIG. 2, and the method specifically comprises the following steps:
step S1, the PCB defect detection system acquires a PCB image I to be detectedo；
Step S2, PCB Defect detection System for PCB image IoDetecting defect key points to obtain at least two defect detection key points;
step S3, the PCB defect detecting system detects the key points based on each defect from the PCB image IoExtracting a PCB defect target image I't；
Step S4, the PCB defect detection system obtains a PCB defect target image I'tAnd a preset standard image IsMatching detection is carried out, and a PCB defect target image I 'is subjected to'tCarrying out defect image segmentation and finally obtaining a PCB defect target image I'tIn which the specific area of the defect is present.
Referring to fig. 3, the step S1 includes an image preprocessing process, which specifically includes the following steps:
step S11, the PCB defect detecting system scales the PCB image I according to the preset scaleoReducing to obtain a scaled PCB image Ir；
Step S12, the PCB defect detection system will print the PCB image IrCarrying out image normalization processing to obtain a normalized PCB image Ind。
In the above technical solution, specifically, the PCB defect detecting system acquires the PCB image I through the camerao∈R1920 ×1080×3(numerals 1920,1080. 3 respectively represent PCB images IoLength, width, number of channels);
in this embodiment, the PCB defect detecting system scales the input PCB image I by 3 timesoPCB image I reduced to 640 x 360 sizer∈R640×360×3(numerals 640, 360 and 3 denote PCB image I, respectivelyrLength, width, and number of channels);
subsequently, the PCB defect detection system will take the PCB image IrThe pixel values of all the pixel points in the image are divided by 255 (the pixel values are normalized from the range of 0-255 to the range of 0-1) to obtain a normalized PCB image Ind∈R640×360×3(numerals 640, 360 and 3 denote PCB image I, respectivelyndLength, width, and number of channels).
Referring to fig. 4, in step S2, the PCB defect detecting system detects a PCB image IndThe specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system will print the PCB image IndInputting the data into a preset key point detection model, and then outputting a score map S representing the confidence coefficient of the defect key pointd；
Step S22, the PCB defect detecting system calculates score chart SdSelecting the pixel point with the maximum confidence as a defect detection key point corresponding to the corresponding channel, and calculating to obtain the defect detection key point in the PCB image IrThe specific coordinates in (1);
step S23, the PCB defect detecting system calculates the PCB image I of each defect detecting key point according to the scaling in the step S11 by reverse-deductionoThe specific coordinates in (1).
In this embodiment, the number of defect detection key points detected by the system may be two, and taking the system detecting two defect detection key points as an example, specifically, in the above technical solution, the system will preprocess the PCB image IndInputting into the above-mentioned key point detection model, PCB image IndExtracting feature graph with output size of 160 x 90 x 270 from image features passing through the key point detection model, and then carrying out system operation on the feature graphThe graph is up-sampled to obtain and input a PCB image IndThe feature size of the same size, i.e., the feature size, was converted from 160 x 90 x 270 to 640 x 360 x 2. Then the system carries out further image feature extraction on the converted feature graph through two convolution layers and a Sigmoid function activation layer to output a fraction graph S with the size of 640 x 360 x 2d. The score map SdThere are two channels, corresponding to PCB defect target image I't(i.e. the target detection region) and the pixel value of each pixel point in each channel represents the confidence that the pixel point is predicted as the vertex (upper left corner or lower right corner) corresponding to the channel.
The system then calculates a score map SdAnd selecting the pixel point as a defect detection key point corresponding to the channel so as to obtain two vertexes of the upper left corner and the lower right corner of the target detection area in the PCB image IrOf (a) { (x) } {, a specific coordinate of each1,y1),(x2,y2)}；
x1Used for representing the first defect detection key point in the PCB image IrThe transverse coordinate value of (1);
y1used for representing the first defect detection key point in the PCB image IrLongitudinal coordinate values of (1);
x2for indicating the second defect detection key point in PCB image IrThe lateral coordinate value of (a);
y2for indicating the second defect detection key point in PCB image IrLongitudinal coordinate values of (a).
Finally, the system calculates and obtains the original PCB image I of the first defect detection key point based on the 3 parts of scalingoCoordinate of (3 x)1,3y1)；
And calculating to obtain a PCB image I of the second defect detection key point at the beginningoCoordinate of (3 x)2,3y2)。
In the above technical scheme, the key point detection model is formed by training a first convolutional neural network, and the first convolutional neural network realizes detection and identification of the defect detection key point through a first trunk network and a key point detection network. In this embodiment, the specific network structure and network parameters of the first convolutional neural network used in the system are as follows:
layer(s) Filter Step size Output size
Input 640x360x3
HRNet18-Det 4 160x90x270
UpSampling 640x360x270
Conv1 1x1x256 1 640x360x256
Conv2 1x1x4 1 640x360x2
Sigmoid 640x360x2
TABLE a
In table a, english input is the input layer of the first convolutional neural network, and is used for inputting PCB image I with size of 640 × 360 × 3nd。
HRNet18-Det represents the keypoint detection model, which is pre-trained, and the training method is the model training method existing in the prior art, and the training method is not the scope of the claimed invention, so the specific method steps for training the keypoint detection model are not described herein. PCB image I with size 640 x 360 x 3ndAnd extracting a feature map with the output size of 160 × 90 × 270 from the image features passing through the key point detection model.
UpSampling is an UpSampling layer, and the feature map with the size of 160 × 90 × 270 is processed by the UpSampling layer to obtain and input a PCB image IndA feature map of the same dimensions;
conv1 is the first convolution layer, the convolution kernel size is 1 × 256, the step size is 1, and the feature map output by the upsampling layer processing is convolved by the first convolution layer to obtain a feature map with the size of 640 × 360 × 256.
Conv2 was the second convolution layer with a convolution kernel size of 1 x 4 and a step size of 1, which transformed the feature map with size 640 x 360 x 256 into a feature map with size 640 x 360 x 2.
Sigmoid is a function activation layer, and the feature graph with the size of 640 x 360 x 2 outputs a fraction graph S with the size of 640 x 360 x 2 after being processed by the function activation layerd。
Referring to fig. 5, in step S3, the PCB defect detecting system detects a defect from the PCB image IoExtracting a PCB defect target image I'tThe specific method comprises the following steps:
step S31, the PCB defect detecting system detects the specific coordinates of the key points in the PCB image based on each defect, and the specific coordinates are obtained from the PCB image IOIn-process cutting out target image It；
Step S32, the PCB defect detection system sends the target image ItIs adjusted to a preset size to obtain a PCB defect target image I 'after size adjustment't。
In the embodiment, specifically, the system extracts the original PCB image I based on the coordinates of two vertices of the upper left corner and the lower right corner of the target detection areaoIn-process cutting out target image It＝Io[3y1:3y2,3x1:3x2,:]；
The system then maps the target image ItPCB defect target image I 'adjusted to size 640 x 360 x 3't∈R640 ×360×3。
Referring to FIG. 6, in step S4, the PCB defect detecting system targets the PCB defect target image I'tThe specific method for segmenting the defect image comprises the following steps:
step S41, acquiring a PCB defect target image I 'by the PCB defect detection system'tAnd PCB Defect target image I'tCorresponding standard image Is；
Step S42, the PCB defect detection system obtains a PCB defect target image I'tAnd a standard image IsConnected in the channel direction to form a channel image Ic；
Step S43, the PCB defect detection system sends the channel image IcCarrying out image normalization processing to obtain a normalized PCB image Ins；
Step S44, the PCB defect detection system will print the PCB image InsInputting the data into an image segmentation model, and performing image I on PCBnsPredicting whether there is defect and outputting a defect probability map SS；
Step S45, the PCB defect detecting system judges the PCB image InsWhether the pixel value corresponding to each pixel point in the image is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains a PCB defect target image I 'according to the judgment result'tIn which the specific area of the defect is present.
In the technical scheme, the image segmentation model is formed by training through a second convolutional neural network, and the second convolutional neural network realizes image segmentation of the PCB image through a second trunk network and an image segmentation network. In this embodiment, the specific network structure and network parameters of the second convolutional neural network used in the system are as follows:
Figure BDA0002314755600000081
Figure BDA0002314755600000091
table b
In the above table b, english input represents an input layer of the second convolutional neural network, and an input image of the input layer is a PCB defect target image I'tWhich PCB defect target image I'tCorresponding standard image IsPCB Defect target image I'tAnd a standard image IsAll 640 x 360 x 3, so the image size input into the input layer is 640 x 360 x 6 (length x width channels);
HRNet18-Seg is used for representing an image segmentation model, and 6-channel input images are subjected to image feature extraction of the image segmentation model and output feature maps with the size of 160 × 90 × 270;
UpSampling is an UpSampling layer. The system then upsamples the feature map with size 160 x 90 x 270, resulting in a feature map with size 640 x 360 x 270.
Conv1 is the first convolution layer in the second convolutional neural network, and the system further performs image feature extraction on the feature map with the size of 640 × 360 × 270 through the first convolution layer to obtain the feature map with the size of 640 × 360 × 256. The convolution kernel size of the first convolution layer is 1 x 256, with a step size of 1.
Conv2 is a second convolution layer in the second convolutional neural network, and the system further performs image feature extraction on the feature map with the size of 640 × 360 × 256 through the second convolution layer to obtain the feature map with the size of 640 × 360 × 1. The convolution kernel size of the second convolution layer is 1 x 1, step size is 1.
Finally, the system outputs a defect probability map S with the size of 640 x 360 x 1 based on the feature map with the size of 640 x 360 x 1 through a Sigmoid function activation layerS。
In the above technical solution, the preset threshold is preferably 0.5. That is, when the defect probability map SSIf the pixel value of the pixel point in the image is more than 0.5, the defect of the pixel point is shown, and if the defect probability map S is adopted, the defect probability map S is adoptedSIf the pixel value of the pixel point in (1) is less than or equal to 0.5, it indicates that the pixel point has no defect.
In addition, in the above technical solution, the training methods of the keypoint detection model and the image segmentation model are both the existing training methods of the recognition models, and the specific training process of the convolutional neural network on the two recognition models is not described here.
In conclusion, the PCB detection device can automatically detect whether the PCB has product defects or not, solves the problems that the prior manual visual detection mode is easy to miss detection and false detection, and greatly improves the detection efficiency and the detection accuracy.
It should be understood that the above-described embodiments are merely preferred embodiments of the invention and the technical principles applied thereto. It will be understood by those skilled in the art that various modifications, equivalents, changes, and the like can be made to the present invention. However, such variations are within the scope of the invention as long as they do not depart from the spirit of the invention. In addition, certain terms used in the specification and claims of the present application are not limiting, but are used merely for convenience of description.

Claims (7)

1. A PCB defect detection system based on a convolutional neural network is used for detecting whether manufacturing defects exist in a PCB printed circuit board or not, and is characterized by comprising the following components:
a first image input module for inputting PCB image I to be detectedo；
A key point detection module connected with the first image input module and used for detecting the PCB image IoDetecting at least two defect detection key points, and detecting each defect detection key point from the PCB image IoExtracting a PCB defect target image I't；
A standard image acquisition module for acquiring the PCB defect target image I'tCorresponding standard image IsAnd outputting;
a second image input module respectively connected with the key point detection module and the standard image acquisition module and used for enabling the PCB defect target image I'tAnd the standard image IsSimultaneously inputting the images into a PCB image segmentation module;
the PCB image segmentation module is connected with the second image input module and used for being based on the standard image IsFor the PCB defect target image I'tCarrying out defect image segmentation and finally obtaining the PCB defect target image I'tIn which the specific area of the defect is present.
2. A PCB defect detection method based on a convolutional neural network is realized by applying the PCB defect detection system as the claim 1, and is characterized by comprising the following steps:
step S1, the PCB defect detection system acquires the PCB image I to be detectedo；
Step S2, the PThe CB defect detection system is used for detecting the PCB image IoDetecting defect key points to obtain at least two defect detection key points;
step S3, the PCB defect detecting system detects the key points based on each defect and from the PCB image IoExtracting the PCB defect target image I't；
Step S4, the PCB defect detection system enables the PCB defect target image I'tAnd a preset standard image IsMatching detection is carried out, and the target image I 'of the PCB defects is obtained'tCarrying out defect image segmentation and finally obtaining the PCB defect target image I'tThe specific area in which the defect exists.
3. The PCB defect detecting method of claim 2, wherein the step S1 includes an image preprocessing process, and the image preprocessing process specifically includes the following steps:
step S11, the PCB defect detection system scales the PCB image I according to a preset scaleoReducing to obtain a scaled PCB image Ir；
Step S12, the PCB defect detection system converts the PCB image IrCarrying out image normalization processing to obtain a normalized PCB image Ind。
4. The PCB defect detecting method of claim 3, wherein in the step S2, the PCB defect detecting system applies the PCB image IndThe specific method for detecting the defect key points comprises the following steps:
step S21, the PCB defect detection system converts the PCB image IndInputting the data into a preset key point detection model, and then outputting a score map S representing the confidence coefficient of the defect key pointd；
Step S22, the PCB defect detecting system calculates the score map SdAnd setting the coordinates of the pixel points with the maximum confidence in each channel, and selecting the pixel points with the maximum confidence as the coordinatesCalculating the defect detection key points corresponding to the corresponding channels in the PCB image IrThe specific coordinates in (1);
step S23, the PCB defect detecting system calculates the scaling of the step S11 by reverse-deduction to obtain the PCB image I of each defect detecting key pointoThe specific coordinates in (1).
5. The PCB defect detecting method of claim 4, wherein in the step S3, the PCB defect detecting system detects the PCB image I from the PCB imageoExtracting the PCB defect target image I'tThe specific method comprises the following steps:
step S31, the PCB defect detecting system detects the critical point based on each defect in the PCB image IOFrom the PCB image IOIn-process cutting out target image It；
Step S32, the PCB defect detection system enables the target image ItIs adjusted to a preset size to obtain the PCB defect target image I 'after size adjustment't。
6. The PCB defect detection method of claim 5, wherein in the step S4, the PCB defect detection system is used for detecting the PCB defect target image I'tThe specific method for segmenting the defect image comprises the following steps:
step S41, the PCB defect detection system acquires the PCB defect target image I'tAnd the PCB defect target image I'tCorresponding standard image Is；
Step S42, the PCB defect detection system enables the PCB defect target image I'tAnd the standard image IsConnected in the channel direction to form a channel image Ic；
Step S43, the PCB defect detection system converts the channel image IcCarrying out image normalization processing to obtain a normalized PCB image Ins；
Step S44, the PCB defect detection system converts the PCB image InsInputting the PCB image I into an image segmentation modelnsPredicting whether there is defect and outputting a defect probability map SS；
Step S45, the PCB defect detecting system judges the PCB image InsWhether the pixel value corresponding to each pixel point in the image is larger than a preset threshold value,
if yes, judging that the pixel point has defects;
if not, judging that the pixel point has no defect;
and obtaining a judgment result;
step S46, the PCB defect detection system obtains the PCB defect target image I 'according to the judgment result'tThe specific area in which the defect exists.
7. The PCB defect detecting method of claim 6, wherein the threshold value in the step S45 is 0.5.
CN201911273062.9A 2019-12-12 2019-12-12 PCB defect detection system and detection method based on convolutional neural network Active CN111080615B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911273062.9A CN111080615B (en) 2019-12-12 2019-12-12 PCB defect detection system and detection method based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911273062.9A CN111080615B (en) 2019-12-12 2019-12-12 PCB defect detection system and detection method based on convolutional neural network

Publications (2)

Publication Number Publication Date
CN111080615A true CN111080615A (en) 2020-04-28
CN111080615B CN111080615B (en) 2023-06-16

Family

ID=70314112

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911273062.9A Active CN111080615B (en) 2019-12-12 2019-12-12 PCB defect detection system and detection method based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN111080615B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111899241A (en) * 2020-07-28 2020-11-06 华中科技大学 Quantitative on-line detection method and system for defects of PCB (printed Circuit Board) patches in front of furnace
CN113609897A (en) * 2021-06-23 2021-11-05 阿里巴巴新加坡控股有限公司 Defect detection method and defect detection system
CN114489914A (en) * 2022-02-17 2022-05-13 广州嘉普信息科技有限公司 Method for generating template key area of PCB intelligent detection system and electronic equipment
EP4220552A4 (en) * 2021-12-07 2023-12-27 Contemporary Amperex Technology Co., Limited Image processing method and system

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130142769A (en) * 2012-06-20 2013-12-30 한국기술교육대학교 산학협력단 Apparatus and method for vision inspecting using image division
CN106897994A (en) * 2017-01-20 2017-06-27 北京京仪仪器仪表研究总院有限公司 A kind of pcb board defect detecting system and method based on layered image
CN107389701A (en) * 2017-08-22 2017-11-24 西北工业大学 A kind of PCB visual defects automatic checkout system and method based on image
CN107610085A (en) * 2016-07-11 2018-01-19 富强 A kind of welding point defect detecting system based on computer vision
CN108122247A (en) * 2017-12-25 2018-06-05 北京航空航天大学 A kind of video object detection method based on saliency and feature prior model
US20180374245A1 (en) * 2017-06-26 2018-12-27 Elekta, Inc. Image quality in cone beam computed tomography images using deep convolutional neural networks
CN109101983A (en) * 2018-08-02 2018-12-28 大连恒锐科技股份有限公司 A kind of shoe pattern and footprint critical point detection method based on deep learning
KR101950681B1 (en) * 2018-05-04 2019-02-21 주식회사 스켈터랩스 Defect inspection device, system, method and computer readable storage medium
CN109829900A (en) * 2019-01-18 2019-05-31 创新奇智(北京)科技有限公司 A kind of steel coil end-face defect inspection method based on deep learning
CN109859207A (en) * 2019-03-06 2019-06-07 华南理工大学 A kind of defect inspection method of high density flexible substrate
CN109919908A (en) * 2019-01-23 2019-06-21 华灿光电（浙江）有限公司 The method and apparatus of light-emitting diode chip for backlight unit defects detection
CN110320285A (en) * 2019-07-08 2019-10-11 创新奇智(青岛)科技有限公司 A kind of steel rail defect recognition methods, system and electronic equipment based on ultrasonic signal
CN110443777A (en) * 2018-10-09 2019-11-12 广东仙童智能机器人科技有限公司 A kind of PCB image alignment method, device and computer storage medium

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130142769A (en) * 2012-06-20 2013-12-30 한국기술교육대학교 산학협력단 Apparatus and method for vision inspecting using image division
CN107610085A (en) * 2016-07-11 2018-01-19 富强 A kind of welding point defect detecting system based on computer vision
CN106897994A (en) * 2017-01-20 2017-06-27 北京京仪仪器仪表研究总院有限公司 A kind of pcb board defect detecting system and method based on layered image
US20180374245A1 (en) * 2017-06-26 2018-12-27 Elekta, Inc. Image quality in cone beam computed tomography images using deep convolutional neural networks
CN107389701A (en) * 2017-08-22 2017-11-24 西北工业大学 A kind of PCB visual defects automatic checkout system and method based on image
CN108122247A (en) * 2017-12-25 2018-06-05 北京航空航天大学 A kind of video object detection method based on saliency and feature prior model
KR101950681B1 (en) * 2018-05-04 2019-02-21 주식회사 스켈터랩스 Defect inspection device, system, method and computer readable storage medium
CN109101983A (en) * 2018-08-02 2018-12-28 大连恒锐科技股份有限公司 A kind of shoe pattern and footprint critical point detection method based on deep learning
CN110443777A (en) * 2018-10-09 2019-11-12 广东仙童智能机器人科技有限公司 A kind of PCB image alignment method, device and computer storage medium
CN109829900A (en) * 2019-01-18 2019-05-31 创新奇智(北京)科技有限公司 A kind of steel coil end-face defect inspection method based on deep learning
CN109919908A (en) * 2019-01-23 2019-06-21 华灿光电（浙江）有限公司 The method and apparatus of light-emitting diode chip for backlight unit defects detection
CN109859207A (en) * 2019-03-06 2019-06-07 华南理工大学 A kind of defect inspection method of high density flexible substrate
CN110320285A (en) * 2019-07-08 2019-10-11 创新奇智(青岛)科技有限公司 A kind of steel rail defect recognition methods, system and electronic equipment based on ultrasonic signal

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JIAN CHENG;PEI-SONG WANG;GANG LI;QING-HAO HU;HAN-QING LU;: "Recent advances in efficient computation of deep convolutional neural networks" *
邓桂平;申莉;田忠春;杨建荣;: "智能电能表质量一致性检测系统设计" *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111899241A (en) * 2020-07-28 2020-11-06 华中科技大学 Quantitative on-line detection method and system for defects of PCB (printed Circuit Board) patches in front of furnace
CN111899241B (en) * 2020-07-28 2022-03-18 华中科技大学 Quantitative on-line detection method and system for defects of PCB (printed Circuit Board) patches in front of furnace
CN113609897A (en) * 2021-06-23 2021-11-05 阿里巴巴新加坡控股有限公司 Defect detection method and defect detection system
EP4220552A4 (en) * 2021-12-07 2023-12-27 Contemporary Amperex Technology Co., Limited Image processing method and system
US11967125B2 (en) * 2021-12-07 2024-04-23 Contemporary Amperex Technology Co., Limited Image processing method and system
CN114489914A (en) * 2022-02-17 2022-05-13 广州嘉普信息科技有限公司 Method for generating template key area of PCB intelligent detection system and electronic equipment

Also Published As

Publication number Publication date
CN111080615B (en) 2023-06-16

Similar Documents

Publication Publication Date Title
CN111080615A (en) PCB defect detection system and detection method based on convolutional neural network
CN108108746B (en) License plate character recognition method based on Caffe deep learning framework
CN108305243B (en) Magnetic shoe surface defect detection method based on deep learning
CN111709909B (en) General printing defect detection method based on deep learning and model thereof
CN104751142B (en) A kind of natural scene Method for text detection based on stroke feature
CN106548182B (en) Pavement crack detection method and device based on deep learning and main cause analysis
CN111899241B (en) Quantitative on-line detection method and system for defects of PCB (printed Circuit Board) patches in front of furnace
CN110033431B (en) Non-contact detection device and detection method for detecting corrosion area on surface of steel bridge
CN110991448A (en) Text detection method and device for nameplate image of power equipment
CN110751154B (en) Complex environment multi-shape text detection method based on pixel-level segmentation
CN111680690B (en) Character recognition method and device
CN108830133A (en) Recognition methods, electronic device and the readable storage medium storing program for executing of contract image picture
CN112308872A (en) Image edge detection method based on multi-scale Gabor first-order derivative
CN111210417B (en) Cloth defect detection method based on convolutional neural network
CN111754502A (en) Method for detecting surface defects of magnetic core based on fast-RCNN algorithm of multi-scale feature fusion
CN111241891B (en) Face image cutting method and device and computer readable storage medium
CN116934762A (en) System and method for detecting surface defects of lithium battery pole piece
CN111950556A (en) License plate printing quality detection method based on deep learning
CN111091550A (en) Multi-size self-adaptive PCB solder paste area detection system and detection method
CN114299020A (en) Scooter display screen defect detection method and system based on machine vision
CN108171750A (en) The chest handling positioning identification system of view-based access control model
CN112734742A (en) Method and system for improving industrial quality inspection accuracy
CN112132054A (en) Document positioning and segmenting method based on deep learning
CN111798429B (en) Visual inspection method for defects of printed matter
CN115424121B (en) Electric power pressing plate switch inspection method based on computer vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant