CN113221867A - Deep learning-based PCB image character detection method - Google Patents
Deep learning-based PCB image character detection method Download PDFInfo
- Publication number
- CN113221867A CN113221867A CN202110513119.9A CN202110513119A CN113221867A CN 113221867 A CN113221867 A CN 113221867A CN 202110513119 A CN202110513119 A CN 202110513119A CN 113221867 A CN113221867 A CN 113221867A
- Authority
- CN
- China
- Prior art keywords
- character
- image
- pcb
- network
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 90
- 238000013135 deep learning Methods 0.000 title claims abstract description 16
- 238000010586 diagram Methods 0.000 claims abstract description 68
- 238000009826 distribution Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 11
- 241000132092 Aster Species 0.000 claims abstract description 6
- 230000004927 fusion Effects 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 12
- 238000005520 cutting process Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 238000003466 welding Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 2
- 150000001875 compounds Chemical class 0.000 claims description 2
- 238000003860 storage Methods 0.000 abstract description 3
- 239000012634 fragment Substances 0.000 description 15
- 238000004880 explosion Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 239000002360 explosive Substances 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 150000003071 polychlorinated biphenyls Chemical class 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
The invention provides a PCB image character detection method based on deep learning. The method comprises the following steps: constructing a PCB character detection network, and inputting a PCB image to be detected into the trained network; the network generates feature graphs with different sizes through downsampling, and upsamples the fusion features layer by layer; respectively obtaining a character distribution thermodynamic diagram and a frame distribution thermodynamic diagram through a character detection head and a frame detection head; subtracting the two thermodynamic diagrams pixel by pixel to obtain a final character distribution thermodynamic diagram; carrying out binarization processing on the thermodynamic diagram by using a preset threshold value, and judging whether characters and edge positions thereof exist at the position to obtain a character detection result; and recognizing characters by using the ASTER of the character recognition network, and storing character information into a database. The PCB character detection network and the ASTER character recognition network are combined, so that the tasks of PCB image character detection, recognition and character information storage are completed, and further the tracing of the PCB fragment image is realized.
Description
Technical Field
The invention belongs to the technical field of character detection of computer vision, and particularly relates to a PCB character detection technology based on deep learning.
Background
Explosion cases are extremely destructive, are the key points of concern of public security departments, and have important significance in timely detecting such cases. Remote controls, timers, etc. are key components of explosive devices, and Printed Circuit Board (PCB) scrap of such components is found in the field evidence after explosion. Tracing the incomplete film, namely finding out the related information of the whole board with the same type as the incomplete film, and providing clues for case detection; and tracing the PCB remnant images by using a computer vision technology is an efficient case handling mode. Different from general image retrieval and identification and PCB fragment source tracing, the PCB fragment image identification becomes a visual difficulty because the fragment has great difference with the whole PCB of the same model in area size, extremely unequal characteristics and random fragment position and shape and is difficult to be registered with the whole PCB of the same model. But for PCB fragments with printed characters, the character detection and recognition are a feasible tracing way: character detection and recognition are not limited by constraints such as image registration. Therefore, the method detects and identifies the characters on the PCB fragment image, retrieves the whole PCB image by using the character identification result, and obtains the fragment related information by using the pre-stored information matched with the whole PCB, thereby being a feasible and effective way for solving the problem of tracing the source of the circuit board fragment.
The character detection of the PCB image belongs to the character detection task in vision, but has the particularity that: (1) the size of characters in a natural scene image is generally large, and the characters in a PCB image are relatively small; the deep learning model is closely related to a training data set, and the existing character detection model based on deep learning does not have PCB image data in the training set, so that the character detection model is used for detecting characters in a PCB image, and omission is inevitable. (2) Many mark lines, lead wires and welding points are marked in the PCB image, and the neural network is easy to confuse the factors and characters to generate false detection. Therefore, it is desirable to provide a character detection task that is well suited for PCB images.
Disclosure of Invention
The circuit board has various varieties, the sizes and the distribution positions of characters are not unified, and various possible detection interferences exist on the circuit board, for example, the characteristics of element pins, marked lines and welding points are similar to the characteristics of the characters, and simple characteristics are difficult to distinguish. In order to obtain an accurate character detection result, the invention fully considers the characteristics of a circuit board image when designing a circuit board character detection scheme, provides a character detection scheme based on deep learning, and designs a network model for realizing the character detection of the PCB image so as to extract the depth characteristic with more representation power in a character region and improve the character detection accuracy.
In the detection process, firstly, extracting image features by using a network model obtained by training to obtain image features of different scales; secondly, carrying out feature fusion of different scales layer by layer to obtain a feature map simultaneously having high-level abstract information and low-level detail information, and predicting the character position; thirdly, accurately identifying the detected characters, capturing a single-line character area, designing a network structure by adopting two detection heads to respectively predict thermodynamic diagrams of the character area and the character boundary box distribution area in order to prevent multiple lines of characters from being detected into a connected area; fourthly, the two predictive thermodynamic diagrams are subtracted pixel by pixel, the difference diagram is used as a new character distribution thermodynamic diagram, the pixel value of the area between the characters in different rows on the character distribution thermodynamic diagram is reduced, the character area is divided on the diagram, and the character areas in different rows can be well distinguished. And finally, positioning a PCB image character area according to a set threshold value and a character distribution thermodynamic diagram, and completing a PCB image character detection task.
In the network model optimization process, in order to enable a network to better learn the characteristics of characters in a PCB image, the method adopts a mean square error loss function to constrain a prediction thermodynamic diagram pixel by pixel, and adopts single-scale structural similarity to constrain the prediction thermodynamic diagram from local structural pattern similarity so as to obtain a high-performance PCB character detection network model.
Because a large number of data samples are needed for deep network model training, PCB image data are collected as much as possible, and character positions are labeled; secondly, in order to increase the number of samples, data enhancement is carried out by adopting an image rotation strategy; thirdly, random cutting, multi-scale training strategy, overlapping cutting, splicing and multi-scale testing strategy are adopted to make up for the data deficiency.
The designed PCB character detection network is combined with the ASTER character recognition network, so that the tasks of PCB image character detection, recognition and character information storage are realized, and the PCB image fragment detection network is integrated into a related explosive fragment analysis system to automatically trace the source of the PCB fragment image.
The invention fully considers the characteristics of the PCB image, can accurately detect character areas on different types of PCB images, and carries out character detection in 50 PCB images containing 1777 target areas, wherein the accuracy rate is 95.6 percent, and the recall rate is 92.4 percent.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of PCB image character detection based on deep learning according to an embodiment of the present invention;
FIG. 2 is a diagram of a PCB graphic character detection network based on ResNet in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
S101: and constructing a character detection network, and inputting the image to be detected into the character detection network.
The structure of the character detection network designed by the invention is shown in figure 2 and is divided into three parts: down sampling feature extraction, up sampling feature integration and a detection head at the end of a network.
S102: and generating feature maps with different sizes through downsampling, and splicing, upsampling and feature fusion the obtained feature maps.
In the down-sampling feature extraction part, a ResNet structure is adopted as a backbone network to solve the problem of gradient dispersion. Firstly, in a path from bottom to top, an input image generates feature maps with different sizes in each CNN network layer through a 6-layer convolutional neural network; and secondly, splicing, upsampling and feature fusion are carried out on the feature graph obtained from the bottom-up path layer by adopting 3 layers of upsampling convolutional layers in the top-down path. By the structural design, each network layer can obtain deep semantic information and shallow detail information.
S103: and obtaining a character distribution thermodynamic diagram through the character distribution detection head, and obtaining a character frame thermodynamic diagram through the character frame distribution detection head.
A character distribution detection head and a character frame distribution detection head are added in a detection head part at the tail end of the network, so that the network simultaneously outputs a character distribution thermodynamic diagram and a character frame thermodynamic diagram, and character boundaries are adaptively positioned. In the process of sending the output feature map into the two detection heads, in order to reduce the number of parameters, 1 convolution layer can be adopted to reduce the dimension of the feature map, and then 2 convolution layers are adopted to respectively generate thermodynamic diagrams for obtaining character distribution and character frame distribution
S104: and subtracting the character distribution thermodynamic diagram and the character frame distribution thermodynamic diagram pixel by pixel to obtain a final character distribution thermodynamic diagram.
Because the characters on the circuit board are densely distributed, it may happen that the network detects a plurality of lines of characters as a single line of characters. To be provided withTo better capture single line character regions and distinguish dense character regions, the present invention uses a character distribution thermodynamic diagram StextThermodynamic diagram S distributed with character frameboxCarrying out pixel-by-pixel subtraction to obtain the final character distribution thermodynamic diagram
In the formula, (x, y) represents a pixel point coordinate. The strategy effectively reduces the pixel value of the character frame area in the character distribution thermodynamic diagram, so that the pixel value of the area between the characters of multiple lines in the thermodynamic diagram is reduced, the problem of adhesion among the character areas of the multiple lines in the thermodynamic diagram is avoided, and the network can obtain the character distribution thermodynamic diagram in a self-adaptive mode. This not only effectively solves the problem of character boundary threshold setting, but also effectively prevents an error in detecting lines of characters as one line of characters.
S105: and carrying out binarization processing on the character distribution thermodynamic diagram by using a preset character threshold value and a character edge threshold value, judging whether characters exist at the position or not by using the character threshold value, and judging the edge position of the characters by using the character edge threshold value to obtain a character detection result.
The final character detection box is determined on the character distribution thermodynamic diagram predicted by the model, and two thresholds are required to be preset: a character threshold and a character edge threshold. The character threshold is used to determine whether a character exists at the position, and the character edge threshold is used to determine where the edge of the character is. Generally, the larger the character threshold, the higher the accuracy of the network and the lower the recall rate; the smaller the character threshold, the lower the network accuracy and the higher the recall rate. The threshold value of the character edge is moderate, and if the threshold value is too large, the detection frame cannot completely frame out the character; if the threshold is too small, the detection frame is too large, and much interference is framed. Through multiple experiments, when the character threshold value is set to be 0.7 and the character edge threshold value is set to be 0.3, the detection effect of the network is best, namely F1-Measure is highest, and the calculation formula of F1 is as follows:
in the formula, P represents the accuracy of the method, and R represents the recall rate of the method.
S106: and recognizing characters by using the ASTER of the character recognition network, and storing character information into a database.
And for the character area in the PCB image detected by the character detection network, recognizing characters by using a character recognition network ASTER, and storing character information into a database for retrieving and tracing the PCB fragment image.
The deep learning model needs more training samples to obtain a robust model; too little training data may cause the model to overfit. In order to solve the problem of network overfitting, the data set is expanded by adopting data enhancement and a multi-scale training strategy, and the specific method is as follows.
S201: adding a PCB image without characters and containing interference of pins, welding spots and the like into the data set.
Some negative sample images, namely PCB images without characters and containing interference of pins, welding spots and the like are added into the data set, so that the types and the number of the negative samples are increased, and the balance of positive and negative samples is achieved as much as possible; and expanding the types of the PCBs in the training set, and eliminating similar PCB images to balance sample distribution.
S202: and rotating the PCB image in four directions, and expanding the data set to form a training set.
The character directions in the PCB image mainly comprise four directions (up, down, left and right), and according to the characteristics, the PCB image is rotated in the four directions, and the data set is expanded to 1192 images to form a training set.
S203: and performing multi-scale data enhancement on the image, and performing scaling on the original image in different scales.
Although the PCB image data set is enhanced to 1192 images, the number of PCB images in the training set is still not enough, the size and the type of image characters are still limited, the network model still has the problem of overfitting, and the character with the overlarge or undersize in the test set is difficult to accurately detect. In order to make the network robust to the detection of characters with different sizes, the invention performs multi-scale data enhancement on the image, namely, the original image is scaled by different scales (0.8x, 1.0x and 1.2 x). Training is performed using multi-scale images so that the network can detect more sized characters.
S204: and performing data enhancement in a random cropping mode to keep the size of the image consistent and perform data enhancement at the same time.
In the training process, images with the same size are required to be input into a network, but the sizes of the circuit board images are different, and if the images are scaled to be the same size, the shape of characters is distorted; therefore, data enhancement is performed in a random cropping mode on the basis of the above operation so as to keep the image size consistent and perform data enhancement at the same time.
In the character detection process, in order to improve the robustness of the network model to the character size, a multi-scale and image cutting and splicing method is adopted for prediction, and an image to be detected is scaled to be different scales (0.5x, 1.0x and 1.5x) and input into a character detection network for character detection; and restoring the character distribution thermodynamic diagrams obtained from the images of all scales into the size of the original image, and performing weighted summation to generate the final character distribution thermodynamic diagram. The weighted sum calculation is:
in the formula (I), the compound is shown in the specification,
respectively representing the thermodynamic diagrams obtained under the sizes of 0.5x, 1x and 1.5x during the test,
is the final resulting thermodynamic diagram.
Preferably, in the character detection process, in order to maintain the resolution of the original image and prevent character distortion, an overlap cropping strategy is adopted to process the image to be detected. The method comprises the steps of cutting an image into image blocks with uniform sizes to meet the size of an input image required by a network, splicing detection results of each image cutout according to original positions, and averaging values of heat maps of repeated overlapping areas to obtain a final detection result.
In order to make the character detection more accurate, the invention also needs to optimize the constructed character detection network.
The character detection network adopts Adam as an optimizer (optimizer), and adopts Mean Square Error (MSELoss) and local mode consistency Loss based on single-scale Structural Similarity (SSIM) as a Loss function (Loss function).
Mselos is a common loss function in thermodynamic diagrams:
in the formula, S is a network predicted character distribution/character border distribution thermodynamic diagram, T is a label thermodynamic diagram, S (x, y) represents a pixel value of a predicted thermal image pixel (x, y), T (x, y) represents a pixel value of a label image pixel (x, y), and N is an image pixel point number.
The loss function optimizes the network through the constraint of pixel-by-pixel in the thermodynamic diagram, but does not consider the structural relationship constraint between the pixels. Given a set of pixel values, different pixel structural relationships can form different image appearances, so the introduction of structural relationship constraints among pixels in a thermodynamic diagram is undoubtedly of great benefit for optimizing a network model. Therefore, the invention introduces SSIM index to carry out network optimization in the network training stage, and SSIM is calculated for each position (x, y) in the thermodynamic diagram:
in the formula, C1，C2Is a very small constant, usually C is chosen1＝(K1L)2，C2＝(K2L)2L isDynamic range of pixel values, constant K1,K1＜＜1；μs、
Respectively, mean and variance, mu, of a pixel point (x, y) in the prediction thermodynamic diagram S in its neighborhoodT、
Respectively, the mean and variance of a pixel point (x, y) in the label thermodynamic diagram T in a neighborhood, wherein the neighborhood is n multiplied by n, n is 11, and sigma isSTCalculating the correlation coefficient of the prediction graph and the label graph according to the formula:
in the formula, I represents S or T.
For the character distribution thermodynamic diagram, the mean value and the variance in the local area can well describe the character distribution condition in the local area, and the correlation coefficient reflects the consistency of the correlation between the two distribution diagrams. Local mode consistency constraint in the distribution diagram necessarily constrains the consistency of the global mode, so that the network model is better optimized in the training process. The local mode consistency loss function is defined as:
wherein N represents the number of pixels of the thermodynamic diagram, p represents the pixel point, and LLPCThe difference in local patterns between the prediction and the tag can be measured.
The network total loss function is expressed as:
loss＝MsELoss(s,t)+γ·LLPC(s,t) (8)
in the formula, γ is used to balance two loss functions, and in the experiment, γ is 0.01.
In order to realize the PCB image character detection method based on deep learning, a PCB image character labeling data set is needed to carry out model training and testing. Due to the lack of such public data sets, the invention also needs to collect and carry out PCB character detection and labeling to complete the preparation of the data sets.
The traditional label generation algorithm has poor positioning effect of the network on the character edge due to the fact that the pixel score at the character edge is low, and particularly the positioning effect of the single character boundary is poor. In order to obtain a heat map which can describe character areas better, the invention provides a new label generation algorithm.
First, four vertexes P of the character frame are utilized1～P4And (3) calculating the width W and the height H of a character frame by coordinates:
in the formula, d (·, ·) represents the distance between two points. Secondly, in order to match the thermodynamic diagram pixel value of the label with the character semantic intensity distribution of the image, the smooth Gaussian kernel radius of the character area is set to be KtGaussian kernel radius K with smooth character framebAdaptively calculating K according to the width W and height H of the character frametAnd Kb：
Secondly, calculating the midpoint position P of the character framecpComprises the following steps:
in the formula (x)cp,ycp) Represents the center coordinates of the character frame, (x)i,yi) Representing character bounding box vertex PiThe coordinates of (a). Then, the distance K from each vertex is taken from the connecting line of each vertex and the midpointtMake up a new rectangle, i.e. take PiAnd PcpDistance P on the connecting lineiIs KtPoint P'iWill be made of P'iThe inner pixel forming the rectangle D for the vertex is set to 1, and then the radius is KtThe gaussian kernel of (a) is smoothed. And finally, carrying out normalization processing to generate a character distribution thermodynamic diagram. In addition, will be made of P'iThe vertices are grouped into pixels on the D side of the rectangle with 1, and then the radius is KbAnd performing smoothing processing and finally performing normalization processing to generate a character frame distribution thermodynamic diagram.
The character distribution label obtained by the algorithm describes the character distribution area more properly, the corner positions of the characters are more fit with the external rectangle of the actual character area, and the phenomenon that the target areas are overlapped cannot occur in the label.
The invention realizes the tasks of character detection, character recognition and character information storage of the PCB image, and is integrated into a related explosive fragment analysis system to automatically trace the source of the PCB fragment image.
The PCB image character detection method based on deep learning can accurately detect the character position area on the circuit board, is applied to tracing of the circuit board fragments of the explosion device in the explosion case, can save a large amount of time and energy required by manual comparison, and provides important technical support for the rapid detection of the explosion case by the public security department.
Claims (7)
1. A PCB image character detection method based on deep learning is characterized by comprising the following steps:
step 1, constructing a PCB character detection network, and inputting a PCB image to be detected into the trained character detection network;
step 2, in a PCB character detection network, generating feature maps with different sizes through down-sampling, and splicing, up-sampling and feature fusion the obtained feature maps;
step 3, obtaining a character distribution thermodynamic diagram through a character distribution detection head in the PCB character detection network, and obtaining a character frame thermodynamic diagram through a character frame distribution detection head in the PCB character detection network;
step 4, subtracting the character distribution thermodynamic diagram and the character frame distribution thermodynamic diagram pixel by pixel to obtain a final character distribution thermodynamic diagram;
step 5, carrying out binarization processing on the character distribution thermodynamic diagram by using a preset character threshold value and a character edge threshold value, judging whether a character exists at the position or not through the character threshold value, and judging the edge position of the character through the character edge threshold value to obtain a character detection result;
and 6, recognizing characters by using the ASTER, and storing character information into a database.
2. The PCB image character detection method based on deep learning of claim 1, wherein in a bottom-up path, an input image generates feature maps with different sizes in each CNN network layer through a 6-layer convolutional neural network; in a top-down path, splicing, upsampling and feature fusion are carried out on a feature map obtained in the bottom-up path layer by adopting 3 layers of upsampling convolutional layers; and sending the output feature graph into two detection heads, firstly adopting 1 convolution layer to reduce the dimension of the feature graph, and then adopting 2 convolution layers to respectively generate thermodynamic diagrams for obtaining character distribution and character frame distribution.
3. The deep learning-based PCB image character detection method of claim 1, wherein in step 5, a character threshold of 0.7 and a character edge threshold of 0.3 are set.
4. The deep learning-based PCB image character detection method of claim 1, wherein in the PCB character detection network training process, a data set is extended by adopting a data enhancement and multi-scale training strategy, specifically:
adding a PCB image which has no characters and contains interference of pins, welding spots and the like into the data set to balance the positive and negative samples; rotating the PCB image in four directions to expand a data set; carrying out multi-scale data enhancement on the image, and carrying out scaling (0.8x, 1.0x and 1.2x) on the original image in different scales; and performing data enhancement by adopting a random cutting mode, keeping the sizes of the images consistent, and simultaneously performing data enhancement.
5. The deep learning-based PCB image character detection method of claim 1, wherein in the character detection process, a multi-scale and image cropping and re-splicing method is adopted for prediction, and specifically the method comprises the following steps:
the method comprises the steps of (1) scaling a PCB image to be detected into different scales (0.5x, 1.0x and 1.5x), inputting the images into a character detection network, and performing character detection; and restoring the character distribution thermodynamic diagrams obtained from the images of all scales into the size of the original image, and performing weighted summation to generate the final character distribution thermodynamic diagram. The weighted sum calculation is:
in the formula (I), the compound is shown in the specification,
respectively representing the thermodynamic diagrams obtained under the sizes of 0.5x, 1x and 1.5x during the test,
is the final generated thermodynamic diagram;
and processing the image to be detected by adopting an overlapping cutting strategy, cutting one image into image blocks with uniform sizes so as to meet the size of the input image required by the network, splicing the detection result of each image cutout block according to the original position, and repeatedly performing average processing on the value of the heat map of the overlapping area so as to obtain the final detection result.
6. The method for detecting the characters of the PCB image based on the deep learning of claim 1, wherein the constructed PCB character detection network is further optimized, in the process of network optimization, Adam is used as an optimizer, and the loss of consistency of a local mode of mean square error MSELoss and SSIM based on the similarity of a single-scale structure is used as a loss function, and specifically comprises the following steps:
in the network training stage, SSIM indexes are introduced for network optimization, and SSIM is calculated for each position (x, y) in the thermodynamic diagram:
in the formula, C1，C2Is a very small constant, usually C is chosen1＝(K1L)2，C2＝(K2L)2L is the dynamic range of the pixel value, constant K1,K1＜＜1；μS、
Respectively, mean and variance, mu, of a pixel point (x, y) in the prediction thermodynamic diagram S in its neighborhoodT、
Respectively, the mean and variance of a pixel point (x, y) in the label thermodynamic diagram T in a neighborhood, wherein the neighborhood is n multiplied by n, n is 11, and sigma isSTCalculating the correlation coefficient of the prediction graph and the label graph according to the formula:
in the formula, I represents S or T;
the local mode consistency loss function is defined as:
wherein N represents the number of pixels of the thermodynamic diagram, p represents the pixel point, and LLPCThe difference of local patterns between the prediction result and the label can be measured;
the network total loss function is expressed as:
loss＝MSELoss(s,t)+γ·LLPC(s,t)
in the formula, γ is used to balance two loss functions, and in the experiment, γ is 0.01.
7. The deep learning-based PCB image character detection method of claim 4, wherein the character detection labeling is performed on the data set to generate a character distribution label, and specifically the method comprises the following steps: using four vertices P of the character frame1～P4And (3) calculating the width W and the height H of a character frame by coordinates:
wherein d (·,) represents the distance between two points; setting smooth Gaussian kernel radius of character region as KtGaussian kernel radius K with smooth character framebAdaptively calculating K according to the width W and height H of the character frametAnd Kb：
Calculating the midpoint position P of the character framecp：
In the formula (x)cp,ycp) Represents the center coordinates of the character frame, (x)i,yi) Representing character bounding box vertex PiThe coordinates of (a); taking the distance K from each vertex on the connecting line of each vertex and the midpointtMake up a new rectangle, i.e. take PiAnd PcpDistance P on the connecting lineiIs KtPoint P'iWill be made of P'iThe inner pixel forming the rectangle D for the vertex is set to 1, using a radius KtSmoothing the Gaussian kernel; carrying out normalization processing to generate a character distribution thermodynamic diagram; in addition, will be made of P'iThe pixels on the sides of the rectangle D are formed for the vertices by 1, using a radius KbAnd performing smoothing processing and normalization processing to generate a character frame distribution thermodynamic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110513119.9A CN113221867A (en) | 2021-05-11 | 2021-05-11 | Deep learning-based PCB image character detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110513119.9A CN113221867A (en) | 2021-05-11 | 2021-05-11 | Deep learning-based PCB image character detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113221867A true CN113221867A (en) | 2021-08-06 |
Family
ID=77095175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110513119.9A Pending CN113221867A (en) | 2021-05-11 | 2021-05-11 | Deep learning-based PCB image character detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113221867A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114495106A (en) * | 2022-04-18 | 2022-05-13 | 电子科技大学 | MOCR (metal-oxide-semiconductor resistor) deep learning method applied to DFB (distributed feedback) laser chip |
| CN114549413A (en) * | 2022-01-19 | 2022-05-27 | 华东师范大学 | Multi-scale fusion full convolution network lymph node metastasis detection method based on CT image |
| CN115116047A (en) * | 2022-08-29 | 2022-09-27 | 松立控股集团股份有限公司 | License plate character region thermodynamic diagram-based license plate detection method |
| CN116071625A (en) * | 2023-03-07 | 2023-05-05 | 北京百度网讯科技有限公司 | Training method of deep learning model, target detection method and device |
| CN117542023A (en) * | 2024-01-04 | 2024-02-09 | 广汽埃安新能源汽车股份有限公司 | Traffic sign detection method, device, electronic equipment and storage medium |
| WO2024130858A1 (en) * | 2022-12-22 | 2024-06-27 | 上海媒智科技有限公司 | Rubber hose defect detection method and system based on deep learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463209A (en) * | 2014-12-08 | 2015-03-25 | 厦门理工学院 | Method for recognizing digital code on PCB based on BP neural network |
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN110929727A (en) * | 2020-02-12 | 2020-03-27 | 成都数联铭品科技有限公司 | Image labeling method and device, character detection method and system and electronic equipment |
| CN111583187A (en) * | 2020-04-14 | 2020-08-25 | 佛山市南海区广工大数控装备协同创新研究院 | PCB defect detection method based on CNN visualization |
| CN112287941A (en) * | 2020-11-26 | 2021-01-29 | 国际关系学院 | License plate recognition method based on automatic character region perception |
| CN112580629A (en) * | 2020-12-23 | 2021-03-30 | 深圳市捷顺科技实业股份有限公司 | License plate character recognition method based on deep learning and related device |
| CN112580507A (en) * | 2020-12-18 | 2021-03-30 | 合肥高维数据技术有限公司 | Deep learning text character detection method based on image moment correction |
-
2021
- 2021-05-11 CN CN202110513119.9A patent/CN113221867A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463209A (en) * | 2014-12-08 | 2015-03-25 | 厦门理工学院 | Method for recognizing digital code on PCB based on BP neural network |
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN110929727A (en) * | 2020-02-12 | 2020-03-27 | 成都数联铭品科技有限公司 | Image labeling method and device, character detection method and system and electronic equipment |
| CN111583187A (en) * | 2020-04-14 | 2020-08-25 | 佛山市南海区广工大数控装备协同创新研究院 | PCB defect detection method based on CNN visualization |
| CN112287941A (en) * | 2020-11-26 | 2021-01-29 | 国际关系学院 | License plate recognition method based on automatic character region perception |
| CN112580507A (en) * | 2020-12-18 | 2021-03-30 | 合肥高维数据技术有限公司 | Deep learning text character detection method based on image moment correction |
| CN112580629A (en) * | 2020-12-23 | 2021-03-30 | 深圳市捷顺科技实业股份有限公司 | License plate character recognition method based on deep learning and related device |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114549413A (en) * | 2022-01-19 | 2022-05-27 | 华东师范大学 | Multi-scale fusion full convolution network lymph node metastasis detection method based on CT image |
| CN114549413B (en) * | 2022-01-19 | 2023-02-03 | 华东师范大学 | Multi-scale fusion full convolution network lymph node metastasis detection method based on CT image |
| CN114495106A (en) * | 2022-04-18 | 2022-05-13 | 电子科技大学 | MOCR (metal-oxide-semiconductor resistor) deep learning method applied to DFB (distributed feedback) laser chip |
| CN115116047A (en) * | 2022-08-29 | 2022-09-27 | 松立控股集团股份有限公司 | License plate character region thermodynamic diagram-based license plate detection method |
| WO2024130858A1 (en) * | 2022-12-22 | 2024-06-27 | 上海媒智科技有限公司 | Rubber hose defect detection method and system based on deep learning |
| CN116071625A (en) * | 2023-03-07 | 2023-05-05 | 北京百度网讯科技有限公司 | Training method of deep learning model, target detection method and device |
| CN117542023A (en) * | 2024-01-04 | 2024-02-09 | 广汽埃安新能源汽车股份有限公司 | Traffic sign detection method, device, electronic equipment and storage medium |
| CN117542023B (en) * | 2024-01-04 | 2024-04-19 | 广汽埃安新能源汽车股份有限公司 | Traffic sign detection method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113221867A (en) | Deep learning-based PCB image character detection method | |
| CN109902622B (en) | Character detection and identification method for boarding check information verification | |
| TWI774659B (en) | Image text recognition method and device | |
| CN109255344B (en) | Machine vision-based digital display type instrument positioning and reading identification method | |
| CN111160352B (en) | Workpiece metal surface character recognition method and system based on image segmentation | |
| CN109800698B (en) | Icon detection method based on deep learning, icon detection system and storage medium | |
| CN111460927B (en) | Method for extracting structured information of house property evidence image | |
| CN112418216B (en) | Text detection method in complex natural scene image | |
| CN110766008A (en) | Text detection method facing any direction and shape | |
| CN110180186B (en) | Topographic map conversion method and system | |
| Lu et al. | Defect detection of PCB based on Bayes feature fusion | |
| CN110751154B (en) | Complex environment multi-shape text detection method based on pixel-level segmentation | |
| CN110503103B (en) | Character segmentation method in text line based on full convolution neural network | |
| CN111680690A (en) | Character recognition method and device | |
| CN110751619A (en) | Insulator defect detection method | |
| CN111832659A (en) | Laser marking system and method based on feature point extraction algorithm detection | |
| CN107516085A (en) | A kind of method that black surround is automatically removed based on file and picture | |
| CN113435452A (en) | Electrical equipment nameplate text detection method based on improved CTPN algorithm | |
| JP2016103759A (en) | Image processing apparatus, image processing method, and program | |
| CN111626145A (en) | Simple and effective incomplete form identification and page-crossing splicing method | |
| CN117437647B (en) | Oracle character detection method based on deep learning and computer vision | |
| CN113435219B (en) | Anti-counterfeiting detection method and device, electronic equipment and storage medium | |
| CN111274863B (en) | Text prediction method based on text mountain probability density | |
| CN114445814B (en) | Character region extraction method and computer-readable storage medium | |
| CN110766001A (en) | Bank card number positioning and end-to-end identification method based on CNN and RNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210806 |