CN110070536A - A kind of pcb board component detection method based on deep learning - Google Patents

A kind of pcb board component detection method based on deep learning Download PDF

Info

Publication number
CN110070536A
CN110070536A CN201910333652.XA CN201910333652A CN110070536A CN 110070536 A CN110070536 A CN 110070536A CN 201910333652 A CN201910333652 A CN 201910333652A CN 110070536 A CN110070536 A CN 110070536A
Authority
CN
China
Prior art keywords
network
component
text
pcb board
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910333652.XA
Other languages
Chinese (zh)
Other versions
CN110070536B (en
Inventor
高�浩
杨泽宇
胡海东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Post and Telecommunication University
Original Assignee
Nanjing Post and Telecommunication University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Post and Telecommunication University filed Critical Nanjing Post and Telecommunication University
Priority to CN201910333652.XA priority Critical patent/CN110070536B/en
Publication of CN110070536A publication Critical patent/CN110070536A/en
Application granted granted Critical
Publication of CN110070536B publication Critical patent/CN110070536B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/63—Scene text, e.g. street names
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30141—Printed circuit board [PCB]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The pcb board component detection method based on deep learning that the invention discloses a kind of, comprising: obtain a large amount of pcb board images and it is marked for being trained to network；Faster-rcnn is trained to detect component locations and cut down；A simple convolutional network is trained to judge component polarity；Training EAST network detects the position of component image context this frame and is cut into；The content of text in text box image that training CRNN Network Recognition is cut into；Polarity is compared with content of text with PCB design file and is obtained a result.The present invention realizes the full-automatic identification to object identifier, solves the problems, such as that each detection-phase docking is difficult at present.

Description

A kind of pcb board component detection method based on deep learning
Technical field
The present invention relates to pcb board component test technique automatic field, specially a kind of pcb boards based on deep learning Component detection method.
Background technique
PCB, that is, printed circuit board is the important component of various electronic equipments, is the supporter of electronic component, several in life Each our common electronic equipments, such as electronic watch, calculator, computer, telecommunications etc. require to use PCB Version.It is indivisible why pcb board can obtain the characteristics of more and more extensive development is with its high reliability and high density, these Characteristic also determines that it is very high to the accuracy requirement of each component, so carrying out scale detection to pcb board becomes Pcb board produces one of important process.
In recent years, many kinds of as the size of component on PCB becomes smaller, the pure method manually estimated in accuracy rate and The needs that production is no longer satisfied in speed are continued to develop by the method for be powered detection and vision-based detection, it would be desirable to PCB The indexs such as the appearance, type of component, position, polarity, model realize automatic monitoring on plate.Traditional visible detection method Most of to compare and analyze by workbench, mechanical arm, CCD camera lens etc. with standard picture, this method speed is slow and automatic Change degree is not high.With flourishing for deep learning, more object detection methods neural network based are detected as pcb board Popular research direction, this method speed is fast, and precision is high, while detection scheme end to end, the degree of automation may be implemented Also relatively high, but function is relatively single, it is most of that only the position of component and classification are detected and lack a height Integrated comprehensive automation detection system detects together including information such as polarity, models to component.
Summary of the invention
In response to the problems existing in the prior art, the purpose of the present invention is to provide a kind of pcb board member device based on deep learning The high-efficient automatic detection of wrong component may be implemented on pcb board in part detection method, this method, can will be at traditional image Reason, target detection, text recognition algorithms are packaged unification, improve the degree of automation and accuracy of detection.
To achieve the above object, the technical solution adopted by the present invention is that:
A kind of pcb board component detection method based on deep learning, comprising:
S1 is obtained and is needed the PCB data set detected and pre-processed；
S2 identifies the position of component on pcb board and type using target detection network, and is cut into single member The image of device；
S3 constructs neural network model, detects the polarity of component in each component image；
S4 detects the text box in each component image using EAST network, using in CRNN Network Recognition text box Text；
S5, the polarity for the component that will test out and the text identified are compared with PCB design file, are detected As a result.
Preferably, S1 further comprises: a large amount of sample images for needing detected pcb board of shooting, by marker's Mode carries out Attitude estimation to camera；Radiation transformation is carried out according to angular coordinate and perspective transform obtains image flame detection, then to figure Component information as in is labeled the PCB data set needed.
Specifically, a large amount of sample images for needing detected pcb board of shooting, it is contemplated that each component tag mark Orientation angle is different, and the camera for needing to set up four different directions collects the image information of different directions.Due to the image taken Angle is unfavorable for image detection and text identification, thus must carry out image flame detection, in order to improve correction accuracy, may be selected by The mode of marker carries out Attitude estimation to camera.
Specifically, four marker in fixed position are set, by being detected using edge detection to image binaryzation The profile and angular coordinate of marker carries out the standard drawing after radiation transformation is corrected with perspective transform according to angular coordinate Picture is labeled the component information in image.
Preferably, in S2, the target detection network uses faster-rcnn network structure.
It specifically, is residual error neural network resnet-101 for the basic network of target detection.Compared to traditional convolution The complexity of neural network such as VGG, residual error neural network resnet-101 reduce, the parameter decline needed；It is profound in building The problem of being not in gradient disperse when network structure；It also solves the degenerate problem of deep learning network simultaneously, so right High-precision target identification effect is more preferable.
Specifically, using faster-rcnn as target detection frame, compared to other such as yolo, ssd detection blocks Frame, faster-rcnn detection accuracy is higher but time-consuming longer, detects since the present invention is applied to product defect, so for reality When property requires not high, therefore faster-rcnn is more suitable this scene.
Specifically, the apex coordinate and device item name of the available each component of target detection, later and standard Design drawing is compared, when it classification and design drawing it is not consistent when by error message feed back to system maintenance personnel into Row manual confirmation.For there is the component of detailed coordinate information in design drawing, when detecting, classification is correct, can be according to drawing Standard coordinate information is adjusted, and is conducive to subsequent further detection.
Preferably, in S3, the neural network model use by five layers of convolutional network, two layers of fully-connected network and The sorter network of softmax activation primitive composition.
Specifically, in detected component, since some devices on pcb board are there are the differentiation of positive-negative polarity, because This need to consider polar detection.It is not obvious enough due to distinguishing polar feature, so in target detection and identifier Text region The step of (i.e. TextField.text identification) individually increases by one between two stages and judges polarity, and error message is fed back into dimension Shield personnel.The step of polarity judges is simple two classification task.
Preferably, in S4, there are two the workflows in stage for the EAST network tool, wherein the first stage uses complete Convolutional network model, directly generates word or line of text level prediction, and second stage is pre- by the word of generation or line of text rank Survey is sent to non-maximum suppression to generate final result, and the realization of the two stages is trained end to end.
Specifically, text is divided into for the identification of identifier and is accurately positioned (i.e. text box detection) and text identification.Due to Text is made of discontinuous character, and simple algorithm of target detection cannot accurately position text, while in order to The degree of automation of raising system considers the differentiation and correction of text orientation, using based on text detection algorithm EAST.This algorithm There are two the workflows in stage, wherein the first stage uses complete convolutional network (FCN) model, which directly generates list Word or line of text level prediction, eliminate redundancy and at a slow speed intermediate steps；Second stage sends the text prediction of generation to non- To generate final result, this network configuration may be implemented to train and optimize end to end maximum suppression.
Preferably, in S4, the CRNN network is arbitrarily long to handle using image as sequence inputting shot and long term memory network The continuous text of degree.
Specifically, after the coordinate for extracting each component identifier, that is, the text box of each component image is determined Later, it needs to carry out text identification to identifier (that is, text in text box).
Specifically, a kind of deep neural network frame CRNN by CNN, RNN, CTC multiple network structure fusion is selected, it The sequence for having the advantages that end-to-end training, can handle random length is not related to character segmentation or horizontal vertical standard Change, in no dictionary and scene text Cognitive task based on dictionary, can be smaller without predefined word, model, it is more suitable for Practical application scene possesses stronger generalization.
Specifically, final detection result is obtained according to each detecting step, by the type, polarity, the model that detect component Etc. information compared with standard design information, it is multiple that error message and the lower component information of confidence level are supplied to artificial detection personnel It looks into.
Compared with prior art, the beneficial effects of the present invention are: the present invention is by by target detection and text detection and knowledge Other frame carries out interface encapsulation, realizes the full-automatic identification to object identifier, and it is tired to solve each detection-phase docking at present Difficult problem；CRNN network frame of the invention has unified the loss function of three components, utilizes prediction label and true tag Logarithm loss come weighing result gap, can also realize and train end to end.
Detailed description of the invention
Fig. 1 is the flow diagram according to the method for the present invention of embodiment；
Fig. 2 is the EAST schematic network structure according to embodiment；
Specific embodiment
Below in conjunction with the attached drawing in the present invention, technical solution of the present invention is clearly and completely described, it is clear that Described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based on the implementation in the present invention Example, those of ordinary skill in the art's all other embodiment obtained under the conditions of not making creative work belong to The scope of protection of the invention.
The present invention provides a kind of pcb board component detection method based on deep learning, comprising:
S1 is obtained and is needed the PCB data set detected and pre-processed；
S2 identifies the position of component on pcb board and type using target detection network, and is cut into single member The image of device；
S3 constructs neural network model, detects the polarity of component in each component image；
S4 detects the text box in each component image using EAST network, using in CRNN Network Recognition text box Text；
S5, the polarity for the component that will test out and the text identified are compared with PCB design file, are detected As a result.
Embodiment
The PCB component detection method based on deep learning that the present embodiment provides a kind of, as shown in Figure 1, its specific steps Are as follows:
(1) prepare data set:
Shoot the sample image of a large amount of pcb boards, it is contemplated that the orientation angle of each component tag mark is different, needs frame If the camera of four different directions collects the image information of different directions.Due to the image angle taken be unfavorable for image detection and Text identification, in order to improve correction accuracy, may be selected to carry out camera by the mode of marker so image flame detection must be carried out Attitude estimation.Four marker in fixed position are set, by utilizing edge detection detection marker's to image binaryzation Profile and angular coordinate carry out the standard picture after radiation transformation is corrected with perspective transform according to angular coordinate, use Labelme is labeled the component information in image.
Mark is divided into two steps: the first step first marks the type of each component and position on image, then by all first devices Part is cut into form a large amount of independent entry device images from original image；Second step to the text on independent entry device image into Rower note is used to train text detection network.Two above data set is required through affine transformation (rotation, translation, scaling etc.) Acquisition data are expanded with image transformation (plus noise, color offset, Gaussian Blur, sharpening etc.), and separately design training Collection, verifying collection and test set.Generally sample is divided into independent three parts training set (train set), verifying collection (validation set) and test set (test set).The division that three set are carried out to initial data, is to be able to select Effect is best out, the optimal model of generalization ability.
The division that three data sets are carried out to initial data, also for preventing model over-fitting.It is all when having used Initial data goes training pattern, and obtained result is likely to the model and has farthest been fitted initial data, that is, the mould Type is existed to be fitted all initial data.But changed other data sets test this modelling effect may be just less good ?.It when new sample appearance, reuses the model and is predicted, effect may might as well only use the mould of a part of data training Type.
One typical divides is the 50% of the total sample of training set Zhan, and it is other respectively account for 25%, three parts are all from sample In randomly select.Situation lesser for sample size is usually used to stay small part to do test set.Then remaining N number of sample is adopted Cross-validation method is rolled over K.That is, sample is upset, then uniformly it is divided into K parts, selects wherein K-1 parts of training in turn, remaining one Part is verified, and Prediction sum squares are calculated, and K Prediction sum squares are finally done average alternatively optimal models again The foundation of structure.Special K takes N, is exactly leaving-one method (leave one out).
(2) utilize algorithm of target detection, detect PCB version on all components position coordinates and device name, and with mark Quasi- design drawing carries out check and correction and provides error message:
Basic network for target detection is residual error neural network resnet-101, compared to traditional convolutional neural networks Such as VGG, its complexity is reduced, the parameter decline needed；It is not in gradient disperse when constructing profound network structure Problem；It solves the degenerate problem of deep learning network simultaneously, so more preferable to high-precision target identification effect.This implementation In example, target detection frame uses faster-rcnn, compared to other such as yolo, ssd detection frameworks, faster-rcnn Detection accuracy is higher but time-consuming longer, since the present embodiment is detected applied to product defect, so not for requirement of real-time Height, faster-rcnn are more suitable this scene.
Frame introduction: a picture comprising multiple RoI (regions of interest) inputs connecting entirely for a multilayer It connects in network, obtains characteristic pattern, then each RoI is melted into the characteristic pattern of a fixed size by pond, is fully connected layer later It is drawn into a feature vector.For each RoI, the feature vector obtained after full articulamentum is finally shared: one It carries out being used to do softmax recurrence after full connection, for doing object identification to the region RoI, another is after full connection Amendment positioning is done for being b-box regression, so that posting is more accurate.According to posting by component from original Image segmentation is sent into subsequent network out and is judged.
(3) neural network model is constructed, the polarity of each component is detected:
Polarity judgement can be classified as more classification problems.Full articulamentum can be eventually adding by several layers of convolution ponds to solve.
First layer input data be original 227*227*3 image, this image by the convolution kernel of 96 11*11*3 into Row convolution algorithm.The characteristic pattern pixel for generating 55*55*96 is formed to the pixel layer after original image convolution.These pixels Layer passes through the processing of relu unit, generates activation pixel layer, size is still the pixel layer data of 55*55*96.These pixel layers warp Cross the processing of pool operation (pond operation), the scale of pond operation is 3*3, step-length 2, then the size of Chi Huahou image is (55-3)/2+1=27.I.e. the scale of Chi Huahou pixel is 27*27*96；Then pass through normalized, normalize the ruler of operation Degree is 5*5；The scale of the pixel layer formed after first convolutional layer operation is 27*27*96.Respectively correspond 96 convolution kernel institutes Operation is formed.
The characteristic pattern that second layer input is 27*27*96 is protected using the convolution kernel operation of 256 5*5*96 and using 0 filling The characteristic pattern that characteristic pattern size generates 27*27*256 is held, these pixel layers pass through the processing of relu unit, generate activation pixel Layer, size is still the characteristic pattern of 27*27*256.Passing through 3*3, the pool operation that step-length is 2 obtains the feature of 13*13*256 Figure.
Third layer exports upper layer using the convolution kernel of 384 3*3*256 and carries out convolution algorithm, keeps figure using 0 filling As the characteristic pattern that size generates 13*13*384 generates activation pixel layer by relu.4th layer identical as third layer.
The characteristic pattern of pool layers of generation 6*6*512 is added in layer 5 on the basis of third layer, is prepared for input to full connection Layer.
Layer 6 and layer 7 are full articulamentum, and size is respectively 4096*4096 and 4096* polar orientation number.It trained Dropot operation, which is added, in full articulamentum in journey prevents over-fitting.
It is most followed by softmax activation primitive and carrys out output category probability, be maximized the direction that input picture can be obtained.
(4) it is directed to each component, determines the exact position of identifier text on each component:
The present invention, which uses, is based on text detection algorithm EAST, and there are two the workflows in stage for this algorithm, as shown in Figure 2. The process first stage uses complete convolutional network (FCN) model, which directly generates word or line of text level prediction, arranges In addition to redundancy and at a slow speed intermediate steps.Second stage sends non-maximum suppression for the text prediction of generation and is most terminated with generating Fruit, this network structure may be implemented to train and optimize end to end.
Since the size variation in block domain is larger, it is thus determined that needing the spy from neural network rearward when the position of big word Sign, and predict the characteristic information that early stage is needed when including the region of small character.So used here as the structure for being similar to U-net Each grade of another characteristic is merged in realization step by step, is born with realizing the utilization of Analysis On Multi-scale Features while not increasing too many calculating Load.Convolutional network complete in this way can be roughly divided into three each sections: feature extraction core network, Fusion Features branch and output layer.
Feature extraction core network can be used on ImageNet with the classical bone such as PVANet, VGG16 for training Dry network shows feature extraction by removing full articulamentum reservation convolution fructufy.
Fusion Features branch is actually a upper sampling process, begins through unpool from the last layer of core network Layer reduces characteristic pattern port number simultaneously for twice of characteristic pattern dimension enlargement, then connects with upper one layer of characteristic pattern of backbone network, so Afterwards by two layers of convolution be sent into next layer unpool repeat aforesaid operations, eventually by the convolution of several 1*1 obtain with it is original The character area geometry prognostic chart of the sizes such as image and corresponding pixel confidence score chart.Pass through a preset threshold Value, the biggish geometry prognostic chart of confidence level is remained, and is predicted, is obtained using the geometry of non-maxima suppression algorithm removal redundancy It the bounding box of text and splits on to final original part and is sent into following process.
It is more as far as possible to allow network to extract to do pre-training on natural scene detection data collection COCO-Text first in training Feature prevents over-fitting, finely tunes after the completion on the PCB component image data collection marked to adapt to business datum.
(5) text identification is carried out to text box in first device, and is compared with standard type information:
The present embodiment selects a kind of deep neural network frame CRNN by CNN, RNN, CTC multiple network structure fusion to come Identify that word content, the network frame of CRNN are broadly divided into three layers, first layer: depth convolutional layer (DCNN) will be originally inputted figure Shape be compressed to identical size using convolution, Chi Hua, the mode that connects carries out feature extraction entirely, finally obtain under characteristic pattern is used as One layer of input, a column of characteristic pattern are equivalent to a matrix area of original image at this time, and specific mapping relations are according to volume Product is related with the selection in pond.
The second layer is two-way sequence signature extract layer (Bidirectional Recurrent Neural Network), this When characteristic pattern matrix row vector be equivalent to feature, column vector just becomes a sequence data, exports one after this layer Length is equal to column, the sequence signature that width is 1.The reason of selecting two-way LSTM to extract as sequence signature has at 5 points: one, it is right The longer character that a column cannot indicate in characteristic pattern of text and width with contextual relation has good Feature capturing Ability；Two, it can propagate back to DCNN layers, and such whole network frame can share a loss function；Three, it can be handled The text of random length；Four, when text is longer, it can overcome long-term Dependence Problem；Five, two-way propagation be able to solve front and back according to Rely problem.
Third layer is sequence labelling (Transcription Layer) layer, is that sequence signature is turned in the task of this layer Label required for being melted into, this conversion is divided into based on dictionary and without dictionary two ways, when being based on dictionary, obtained label It is the word of maximum probability in dictionary, label is made of the character of each maximum probability when no dictionary, does not need entirely to mark Label are all in dictionary.Final output form is in the second layer
Z={ z1……zT}
T is characterized figure matrix column number, then
Indicate that some character is the probability of π, and π is the label of each column of matrix, B (π) indicates the label predicted, it is Remove what duplicate and blank character obtained by π, correct label is l in training, and the correct probability of the label of prediction is
Correct label is not known in test, obtains then having as a result, the first is to be not based on dictionary there are two types of method
L=B (argmaxπp(π|Z))
Be based on dictionary second, all labels in traversal dictionary, select the label of a maximum probability as output as a result, But this method, which has a problem that, is exactly, and when the number of labels in dictionary is too big, the spent time is too many, so in base We first determine the length of text with the mode of non-dictionary in the mode of dictionary, then select label based on dictionary again, can be with Greatly reduce the number of tags of traversal.Final prediction result and standard model are compared into the selection it can be learnt that the component It is whether correct.
So far, the original part orientation problem in the detection of PCB original part, polarity decision problem, type identifier problem are all addressed.
The present invention improves unified neural network framework for the detection to pcb board component, solution by constructing one The PCB that determined manufactures the requirement in link to high reliability and high precision, greatly reduces manpower consumption and the biography of artificial detection The hardware device requirement of system mechanical means, can be used as detection platform or detection system to assembly line by trained model Pcb board is measured in real time, and solves the problems, such as to can not achieve real-time detection in traditional detection method.
The beneficial effects of the present invention are: the present invention is by carrying out interface envelope for target detection and text detection and identification framework Dress realizes the full-automatic identification to object identifier, solves the problems, such as that each detection-phase docking is difficult at present；Of the invention CRNN network frame has unified the loss function of three components, is lost using the logarithm of prediction label and true tag to measure knot The gap of fruit can also be realized and be trained end to end.
It although an embodiment of the present invention has been shown and described, for the ordinary skill in the art, can be with A variety of variations, modification, replacement can be carried out to these embodiments without departing from the principles and spirit of the present invention by understanding And modification, the scope of the present invention is defined by the appended.

Claims (6)

1. a kind of pcb board component detection method based on deep learning characterized by comprising
S1 is obtained and is needed the PCB data set detected and pre-processed；
S2 identifies the position of component on pcb board and type using target detection network, and is cut into single component Image；
S3 constructs neural network model, detects the polarity of component in each component image；
S4 detects the text box in each component image using EAST network, utilizes the text in CRNN Network Recognition text box Word；
S5, the polarity for the component that will test out and the text identified are compared with PCB design file, obtain detection knot Fruit.
2. a kind of pcb board component detection method based on deep learning according to claim 1, which is characterized in that S1 Further comprise: a large amount of sample images for needing detected pcb board of shooting carry out posture to camera by the mode of marker Estimation；Carry out radiation transformation according to angular coordinate and perspective transform obtain image flame detection, then to the component information in image into Rower infuses the PCB data set needed.
3. a kind of pcb board component detection method based on deep learning according to claim 1, which is characterized in that S2 In, the target detection network uses faster-rcnn network structure.
4. a kind of pcb board component detection method based on deep learning according to claim 1, which is characterized in that S3 In, the neural network model uses point being made of five layers of convolutional network, two layers of fully-connected network and softmax activation primitive Class network.
5. a kind of pcb board component detection method based on deep learning according to claim 1, which is characterized in that S4 In, there are two the workflows in stage for the EAST network tool, wherein the first stage uses complete convolutional network model, directly It generates word or the word of generation or line of text level prediction are sent non-maximum suppression by line of text level prediction, second stage To generate final result, the realization of the two stages is trained end to end.
6. a kind of pcb board component detection method based on deep learning according to claim 1, which is characterized in that S4 In, the CRNN network is using image as sequence inputting shot and long term memory network, to handle the continuous text of random length.
CN201910333652.XA 2019-04-24 2019-04-24 Deep learning-based PCB component detection method Active CN110070536B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910333652.XA CN110070536B (en) 2019-04-24 2019-04-24 Deep learning-based PCB component detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910333652.XA CN110070536B (en) 2019-04-24 2019-04-24 Deep learning-based PCB component detection method

Publications (2)

Publication Number Publication Date
CN110070536A true CN110070536A (en) 2019-07-30
CN110070536B CN110070536B (en) 2022-08-30

Family

ID=67368585

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910333652.XA Active CN110070536B (en) 2019-04-24 2019-04-24 Deep learning-based PCB component detection method

Country Status (1)

Country Link
CN (1) CN110070536B (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111079815A (en) * 2019-12-11 2020-04-28 常州大学 Automatic high-value electronic device identification method for waste circuit board disassembly
CN111079749A (en) * 2019-12-12 2020-04-28 创新奇智(重庆)科技有限公司 End-to-end commodity price tag character recognition method and system with attitude correction function
CN111274961A (en) * 2020-01-20 2020-06-12 华南理工大学 Character recognition and information analysis method for flexible IC substrate
CN111462094A (en) * 2020-04-03 2020-07-28 联觉(深圳)科技有限公司 PCBA component detection method and device and computer readable storage medium
CN111461122A (en) * 2020-05-18 2020-07-28 南京大学 Certificate information detection and extraction method
CN111626177A (en) * 2020-05-22 2020-09-04 深圳技术大学 PCB element identification method and device
CN111783590A (en) * 2020-06-24 2020-10-16 西北工业大学 Multi-class small target detection method based on metric learning
CN111814919A (en) * 2020-08-31 2020-10-23 江西小马机器人有限公司 Instrument positioning and identifying system based on deep learning
CN111914508A (en) * 2020-08-19 2020-11-10 创新奇智(南京)科技有限公司 Component detection method and device, electronic equipment and readable storage medium
CN112102242A (en) * 2020-08-13 2020-12-18 南京航空航天大学 PCB component detection method based on lightweight network
CN112132215A (en) * 2020-09-22 2020-12-25 平安国际智慧城市科技股份有限公司 Method and device for identifying object type and computer readable storage medium
CN112381175A (en) * 2020-12-05 2021-02-19 中国人民解放军32181部队 Circuit board identification and analysis method based on image processing
US20210082101A1 (en) * 2019-09-17 2021-03-18 Battelle Memorial Institute System and method for rapid inspection of printed circuit board using multiple modalities
CN112733924A (en) * 2021-01-04 2021-04-30 哈尔滨工业大学 Multi-patch component detection method
WO2021135065A1 (en) * 2020-01-03 2021-07-08 平安科技（深圳）有限公司 Vehicle component identification method and apparatus, and computer device and storage medium
CN113205511A (en) * 2021-05-25 2021-08-03 中科芯集成电路有限公司 Electronic component batch information detection method and system based on deep neural network
CN113450321A (en) * 2021-06-18 2021-09-28 电子科技大学 Single-stage target detection method based on edge detection
CN113674207A (en) * 2021-07-21 2021-11-19 电子科技大学 Automatic PCB component positioning method based on graph convolution neural network
CN113688821A (en) * 2021-09-07 2021-11-23 四川中电启明星信息技术有限公司 OCR character recognition method based on deep learning
CN113804704A (en) * 2020-06-11 2021-12-17 广东美的白色家电技术创新中心有限公司 Circuit board detection method, visual detection equipment and device with storage function
CN114004815A (en) * 2021-11-03 2022-02-01 哈工大机器人（合肥）国际创新研究院 PCBA appearance detection method and device
CN114428082A (en) * 2020-10-29 2022-05-03 技嘉科技股份有限公司 Electronic component image capturing method and capacitor polarity determination method using same
TWI775140B (en) * 2019-09-19 2022-08-21 日商斯庫林集團股份有限公司 Learning apparatus, inspection apparatus, learning method and inspection method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150066180A1 (en) * 2012-05-09 2015-03-05 Vayo (Shanghai) Technology Co., Ltd. Quick processing system and method for smt equipment
CN104463209A (en) * 2014-12-08 2015-03-25 厦门理工学院 Method for recognizing digital code on PCB based on BP neural network
CN105469400A (en) * 2015-11-23 2016-04-06 广州视源电子科技股份有限公司 Method and system for quickly identifying and marking polarity direction of electronic element

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150066180A1 (en) * 2012-05-09 2015-03-05 Vayo (Shanghai) Technology Co., Ltd. Quick processing system and method for smt equipment
CN104463209A (en) * 2014-12-08 2015-03-25 厦门理工学院 Method for recognizing digital code on PCB based on BP neural network
CN105469400A (en) * 2015-11-23 2016-04-06 广州视源电子科技股份有限公司 Method and system for quickly identifying and marking polarity direction of electronic element

Cited By (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210082101A1 (en) * 2019-09-17 2021-03-18 Battelle Memorial Institute System and method for rapid inspection of printed circuit board using multiple modalities
US11906578B2 (en) * 2019-09-17 2024-02-20 Battelle Memorial Institute System and method for rapid inspection of printed circuit board using multiple modalities
TWI775140B (en) * 2019-09-19 2022-08-21 日商斯庫林集團股份有限公司 Learning apparatus, inspection apparatus, learning method and inspection method
CN111079815A (en) * 2019-12-11 2020-04-28 常州大学 Automatic high-value electronic device identification method for waste circuit board disassembly
CN111079749A (en) * 2019-12-12 2020-04-28 创新奇智(重庆)科技有限公司 End-to-end commodity price tag character recognition method and system with attitude correction function
CN111079749B (en) * 2019-12-12 2023-12-22 创新奇智(重庆)科技有限公司 End-to-end commodity price tag character recognition method and system with gesture correction
WO2021135065A1 (en) * 2020-01-03 2021-07-08 平安科技（深圳）有限公司 Vehicle component identification method and apparatus, and computer device and storage medium
CN111274961A (en) * 2020-01-20 2020-06-12 华南理工大学 Character recognition and information analysis method for flexible IC substrate
CN111274961B (en) * 2020-01-20 2021-12-07 华南理工大学 Character recognition and information analysis method for flexible IC substrate
CN111462094A (en) * 2020-04-03 2020-07-28 联觉(深圳)科技有限公司 PCBA component detection method and device and computer readable storage medium
CN111461122A (en) * 2020-05-18 2020-07-28 南京大学 Certificate information detection and extraction method
CN111461122B (en) * 2020-05-18 2024-03-22 南京大学 Certificate information detection and extraction method
CN111626177A (en) * 2020-05-22 2020-09-04 深圳技术大学 PCB element identification method and device
CN111626177B (en) * 2020-05-22 2023-11-21 深圳技术大学 PCB element identification method and device
WO2021232670A1 (en) * 2020-05-22 2021-11-25 深圳技术大学 Pcb component identification method and device
CN113804704A (en) * 2020-06-11 2021-12-17 广东美的白色家电技术创新中心有限公司 Circuit board detection method, visual detection equipment and device with storage function
CN111783590A (en) * 2020-06-24 2020-10-16 西北工业大学 Multi-class small target detection method based on metric learning
CN112102242B (en) * 2020-08-13 2024-09-20 南京航空航天大学 PCB component detection method based on lightweight network
CN112102242A (en) * 2020-08-13 2020-12-18 南京航空航天大学 PCB component detection method based on lightweight network
CN111914508A (en) * 2020-08-19 2020-11-10 创新奇智(南京)科技有限公司 Component detection method and device, electronic equipment and readable storage medium
CN111814919A (en) * 2020-08-31 2020-10-23 江西小马机器人有限公司 Instrument positioning and identifying system based on deep learning
CN112132215B (en) * 2020-09-22 2024-04-16 平安国际智慧城市科技股份有限公司 Method, device and computer readable storage medium for identifying object type
CN112132215A (en) * 2020-09-22 2020-12-25 平安国际智慧城市科技股份有限公司 Method and device for identifying object type and computer readable storage medium
CN114428082A (en) * 2020-10-29 2022-05-03 技嘉科技股份有限公司 Electronic component image capturing method and capacitor polarity determination method using same
CN112381175A (en) * 2020-12-05 2021-02-19 中国人民解放军32181部队 Circuit board identification and analysis method based on image processing
CN112733924A (en) * 2021-01-04 2021-04-30 哈尔滨工业大学 Multi-patch component detection method
CN113205511B (en) * 2021-05-25 2023-09-29 中科芯集成电路有限公司 Electronic component batch information detection method and system based on deep neural network
CN113205511A (en) * 2021-05-25 2021-08-03 中科芯集成电路有限公司 Electronic component batch information detection method and system based on deep neural network
CN113450321B (en) * 2021-06-18 2022-05-03 电子科技大学 Single-stage target detection method based on edge detection
CN113450321A (en) * 2021-06-18 2021-09-28 电子科技大学 Single-stage target detection method based on edge detection
CN113674207B (en) * 2021-07-21 2023-04-07 电子科技大学 Automatic PCB component positioning method based on graph convolution neural network
CN113674207A (en) * 2021-07-21 2021-11-19 电子科技大学 Automatic PCB component positioning method based on graph convolution neural network
CN113688821B (en) * 2021-09-07 2023-05-23 四川中电启明星信息技术有限公司 OCR text recognition method based on deep learning
CN113688821A (en) * 2021-09-07 2021-11-23 四川中电启明星信息技术有限公司 OCR character recognition method based on deep learning
CN114004815A (en) * 2021-11-03 2022-02-01 哈工大机器人（合肥）国际创新研究院 PCBA appearance detection method and device

Also Published As

Publication number Publication date
CN110070536B (en) 2022-08-30

Similar Documents

Publication Publication Date Title
CN110070536A (en) A kind of pcb board component detection method based on deep learning
Yuliang et al. Detecting curve text in the wild: New dataset and new solution
CN113160192B (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN106650721B (en) A kind of industrial character identifying method based on convolutional neural networks
CN111833306B (en) Defect detection method and model training method for defect detection
CN110636715A (en) Self-learning-based automatic welding and defect detection method
Khan et al. Comparative study of hand gesture recognition system
CN110866529B (en) Character recognition method, device, electronic equipment and storage medium
CN108520273A (en) A kind of quick detection recognition method of dense small item based on target detection
CN110991435A (en) Express waybill key information positioning method and device based on deep learning
CN109919934A (en) A kind of liquid crystal display panel defect inspection method based on the study of multi-source domain depth migration
Zhang et al. Keypoint-graph-driven learning framework for object pose estimation
CN107862234A (en) A kind of recognition methods of color bar code, electronic equipment and storage medium
CN114155527A (en) Scene text recognition method and device
CN111079815B (en) Automatic high-value electronic device identification method for disassembly of waste circuit board
CN106874913A (en) A kind of vegetable detection method
CN110929795A (en) Method for quickly identifying and positioning welding spot of high-speed wire welding machine
CN110689000A (en) Vehicle license plate identification method based on vehicle license plate sample in complex environment
CN114627435A (en) Intelligent light adjusting method, device, equipment and medium based on image recognition
CN114429445A (en) PCB defect detection and identification method based on MAIRNet
CN113780484A (en) Industrial product defect detection method and device
CN116258990A (en) Cross-modal affinity-based small sample reference video target segmentation method
CN116091470A (en) Intelligent industrial control method and device based on 5G technology, electronic equipment and storage medium
Mirani et al. Object recognition in different lighting conditions at various angles by deep learning method
Wang et al. Conditional TransGAN‐Based Data Augmentation for PCB Electronic Component Inspection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant