CN116245882A - Circuit board electronic element detection method and device and computer equipment - Google Patents
Circuit board electronic element detection method and device and computer equipment Download PDFInfo
- Publication number
- CN116245882A CN116245882A CN202310527853.XA CN202310527853A CN116245882A CN 116245882 A CN116245882 A CN 116245882A CN 202310527853 A CN202310527853 A CN 202310527853A CN 116245882 A CN116245882 A CN 116245882A
- Authority
- CN
- China
- Prior art keywords
- image
- detection
- images
- circuit board
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 228
- 238000000034 method Methods 0.000 claims abstract description 64
- 238000012549 training Methods 0.000 claims description 69
- 230000011218 segmentation Effects 0.000 claims description 36
- 238000012216 screening Methods 0.000 claims description 26
- 238000002372 labelling Methods 0.000 claims description 24
- 238000001914 filtration Methods 0.000 claims description 18
- 238000007689 inspection Methods 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012795 verification Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 abstract description 18
- 230000008569 process Effects 0.000 abstract description 14
- 238000003709 image segmentation Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 239000003990 capacitor Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- RVCKCEDKBVEEHL-UHFFFAOYSA-N 2,3,4,5,6-pentachlorobenzyl alcohol Chemical compound OCC1=C(Cl)C(Cl)=C(Cl)C(Cl)=C1Cl RVCKCEDKBVEEHL-UHFFFAOYSA-N 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 238000005452 bending Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30148—Semiconductor; IC; Wafer
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and computer equipment for detecting electronic elements of a circuit board, which adopt a target detection algorithm based on a Swin-converter network structure to detect the elements on the electronic circuit board, replace the existing process flow which is manually arranged and can only carry out off-line detection, realize automatic real-time on-line detection, greatly improve the detection efficiency and have higher target recognition accuracy compared with other target detection algorithms. In addition, a method of scanning and dividing images is adopted to divide the circuit board image to be detected into a plurality of sub-images which are suitable for the input size of the detection model, then the sub-images are input into the model for detection, then the detection results of all the sub-images are mapped back to the original circuit board image, and repeated detection results are filtered. The method can ensure that each electronic element on the circuit board image is completely distributed in each sub-image so as to prevent the condition of missing detection in the detection process, thereby improving the accuracy of the whole detection process flow.
Description
Technical Field
The present invention relates to the field of computer vision detection technology, and in particular, to a method and an apparatus for detecting electronic components of a circuit board, and a computer device.
Background
The circuit board is a central for controlling the operation of electronic equipment, and almost all electronic equipment such as computers and related products, communication equipment, consumer electronic equipment and the like can not be controlled by the electronic circuit board, so that the circuit board is very important for quality monitoring in the production and manufacturing process of the electronic circuit board. After the circuit board completes the plug-in of the electronic element, the plug-in of the electronic element needs to be detected to judge whether the electronic element such as a resistor, a capacitor, a diode and the like has defects of misplacement, polarity reverse insertion, breakage, element bending angle and the like. At present, a main detection technology generally needs to make a process first, identify electronic elements in an image shot by a CCD camera in a manual mode, then intercept the identified electronic elements from the image of the whole circuit board, and compare the electronic elements with electronic element pictures in a standard template library, thereby detecting whether the electronic elements generate defects during plug-in. However, the manual process greatly reduces the inspection efficiency, and the method is not suitable for on-line inspection, but is suitable for off-line inspection. By adopting the method based on the deep learning target detection, the electronic elements on the circuit board can be automatically classified and identified, and the positions and the sizes of the elements on the original image can be directly output. By utilizing the information, the electronic element can be automatically segmented, so that the detection efficiency is greatly improved. Compared with the traditional image processing algorithm, the target detection method based on deep learning does not need to carry out the technologies of filtering, binarization, edge detection, corrosion, expansion and the like and set various processing parameters, and can also identify the target in the image with higher accuracy for the image with complex background.
Currently, target detection algorithms can be divided into two main categories: one type is a region-nominated based deep learning detection algorithm. Firstly, a series of candidate target areas are generated by using algorithms such as Edge Boxes (Locating Object Proposals from Edges), selective search (selectesearch) and the like, then, the characteristics of the target candidate areas are extracted by using a deep neural network, finally, the extracted characteristics are utilized to carry out target classification and regression of the target real position and the frame length and width, and typical representatives of the algorithms are R-CNN, SPP-NET, fast RCNN and the like. The other type is a regression-based target detection algorithm, which directly carries out regression operation at a plurality of positions of an original image to infer the boundary of a target frame and the type of an object in the frame, and represents a SSD (Single Shot Detection) and YOLO series related algorithm.
Recently, some researchers have introduced a transducer network structure model into the field of computer vision for image target detection. The Transformer originally used for NLP (natural language processing) tasks uses Self-Attention structure instead of the commonly used recurrent neural network RNN (Recurrent Neural Network) structure.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems in the related art to some extent. Therefore, the invention aims to provide a method and a device for detecting electronic components of a circuit board and computer equipment.
To achieve the above object, in a first aspect, a method for inspecting electronic components of a circuit board according to an embodiment of the present invention includes:
a step of producing an image data set: collecting a plurality of original images of a circuit board containing electronic elements to form a training image data set;
an image dataset labeling step: respectively labeling the electronic elements of each image in the training image data set;
and (3) training a detection model: training and verifying a detection model based on a Swin-transducer neural network on an image data set to generate an image detection model;
an image detection step: and scanning and dividing the image to be detected, and then inputting the divided sub-images into the image detection model for detection.
And (3) screening detection results: screening the detection results of all the divided sub-images, filtering out repeated detection results, and outputting the finally detected type of the electronic element and the position of the electronic element on the original image.
Further, according to an embodiment of the present invention, the method for capturing an original image from different starting points to select a captured image containing an electronic component to form an image dataset includes: intercepting original images of each circuit board from different initial positions for a plurality of times, intercepting the original images into a plurality of images according to a first set size in each interception;
And screening the intercepted images, removing the images which do not contain the electronic elements, and reserving the intercepted images which contain the electronic elements to form the image data set.
Further, according to an embodiment of the present invention, the method for labeling electronic components on each image in the image dataset includes:
when the electronic element is displayed completely in the image, marking the electronic element;
when the electronic component is not displayed completely in the image, the electronic component is not marked.
Further, according to an embodiment of the present invention, the classifying the labeled image dataset into a training set and a verification set, training and verifying the detection model based on the Swin-transducer neural network, and generating the image detection model includes:
dividing the image data set into a training set and a verification set according to a certain proportion;
training and verifying the detection model based on the Swin-transducer neural network; when training the model, firstly loading a pre-training model, and then carrying out detection model training and verification on the marked image data.
Further, according to an embodiment of the present invention, the first dividing the original electronic circuit board image into detection images with a size suitable for the input size of the detection model by using a scan division method, and then inputting the detection images into the detection model for detection includes:
A first round of segmentation step: sequentially dividing the image to be detected from the [0,0] position according to a second set size, and filling with black when the size of the tail end of the image to be detected is not enough to the second set size;
a second round of segmentation step: sequentially dividing the image to be detected from the position of the pixel point [ width/2, 0 of the second set size ] according to the second set size, and when the right end size of the image to be detected is not enough to the second set size, reversely pushing the width of the second set size from the right end edge of the image to be detected to the left for the last time to divide;
a third wheel segmentation step: sequentially dividing the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size according to the second set size, and when the bottom end size of the image to be detected is not enough to the second set size, reversely pushing the height of the second set size upwards from the bottom end edge of the image to be detected for the last time to divide;
fourth wheel segmentation step: starting the image to be detected from the position of the width/2 of the second set size and the height/2 of the second set size, sequentially dividing the image according to the second set size, and removing the residual image when the width and the height of the tail end of the image to be detected are not smaller than the second set size;
And inputting the sub-images obtained by dividing each round in turn into the detection model for detection.
Further, according to an embodiment of the present invention, the method for filtering the detection results of all the divided images, removing the repeated detection results, and marking the type and the position of the finally detected electronic component on the original image includes:
mapping the detection results of all the sub-images from the sub-image coordinate space back to the image coordinate space to be detected, namely adding the coordinate value of the top left corner vertex of the target frame of the detection result in the sub-image to the coordinate value of the top left corner vertex of the sub-image in the original image to be detected;
then sorting according to the detected categories of the electronic elements;
in the same category of results, filtering is performed based on the positional information of the components to filter out duplicate test results, because only one electronic component is usually mounted on the same position on the circuit board.
Further, according to an embodiment of the present invention, the method for filtering out repeated detection results according to the detected position information of the component includes the steps of:
calculating the distance between each element in the same type of element and the top left corner vertex of the target frame of all other elements;
If the distance between one element and other elements is larger than the set threshold value, outputting the category name of the element and the position and size information of the target frame on the image to be detected;
if the distance between two or more elements is smaller than the set threshold value, selecting the element with the largest length size of the target frame as a detection result, and outputting the category name of the detection result and the position and size information of the target frame on the image to be detected; the target frame length dimension is its longer dimension in the X and Y directions.
In a second aspect, an embodiment of the present invention further provides an apparatus for detecting an electronic component of a circuit board, including:
and (3) an image dataset making module: the image data set making module is used for collecting original images of a plurality of circuit boards, intercepting the original images from different starting points, and selecting intercepted images containing electronic elements to form an image data set;
the image dataset labeling module: the image data set labeling module is used for respectively labeling the electronic elements of the images in the image data set;
and the detection model training module: the detection model training module is used for dividing the marked image data set into a training set and a verification set, training and verifying a detection model based on the Swin-transducer neural network, and generating an image detection model;
An image detection module: the image detection module is used for firstly dividing an original electronic circuit board image into detection images with the size suitable for the input size of a detection model by adopting a scanning division method, and then inputting the detection images into the detection model for detection.
And a detection result screening module: the detection result screening module is used for screening the detection results of all the segmented images, removing repeated detection results and marking the types and positions of finally detected electronic elements on the original image.
Further, according to an embodiment of the present invention, the image dataset making module includes:
the image intercepting module is used for intercepting each original image from different initial positions for a plurality of times, and intercepting the original image into a plurality of images according to the first set size in sequence;
and the image data set module is used for picking out images containing electronic elements from all the intercepted images to form the image data set.
Further, according to an embodiment of the present invention, the image detection model further comprises a complete image segmentation module, the complete image segmentation module comprising:
The first round of segmentation module is used for sequentially segmenting the image to be detected from the [0,0] position according to a second set size, and filling and supplementing black when the tail end size of the image to be detected is not enough to the second set size;
the second wheel segmentation module is used for sequentially segmenting the image to be detected from the position of the pixel point [ width/2, 0 of the second set size ] according to the second set size, and when the right end of the image to be detected is of the second set size which is insufficient in size, the second wheel segmentation module reversely pushes the width of the second set size from the right end edge of the image to be detected to the left for the last time for segmentation;
the third wheel segmentation module is used for sequentially segmenting the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size according to the second set size, and when the bottom end size of the image to be detected is not enough to the second set size, the image to be detected is reversely pushed upwards from the bottom end edge of the image to be detected for the last time to intercept the height of the second set size;
and the fourth-wheel segmentation module is used for sequentially segmenting the image to be detected from the position of [ width/2 of the second set size, height/2 of the second set size ] according to the second set size, and removing the residual image when the size of the right lower end of the image to be detected is not enough to the second set size.
Further, according to an embodiment of the present invention, the detection result screening module includes:
and the detection result mapping module is used for mapping all detection results from the respective sub-images back to the to-be-detected image space, namely, the coordinate value of the top left corner vertex of the target frame of the detection result in the sub-image is added with the coordinate value of the top left corner vertex of the sub-image in the to-be-detected image, and the position information of the detection result in the to-be-detected image can be obtained.
The detection result classification and sorting module is used for classifying and sorting all detection results of the image to be detected according to the category of the electronic element;
and the screening and filtering module is used for screening the elements according to the position information of the detected elements in the results of the same category so as to filter out repeated detection results.
Further, according to an embodiment of the present invention, the filtering module includes:
the distance calculation module is used for calculating the distance between the target frame of each element in the same element and the top left corner vertex of the target frame of other elements;
The screening module is used for judging whether the distances between the elements are close, and outputting the category name of the element and the position and size information of the target frame on the image to be detected if the distances between one element and other elements are larger than a set threshold value; if the distance between two or more elements is smaller than the set threshold value, selecting the longest length of the target frame from the elements as a detection result, and outputting the category name of the detection result and the position and size information of the target frame on the image to be detected.
In a third aspect, a computer device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the circuit board electronic component detection method as described above when executing the computer program.
According to the method, the device and the computer equipment for detecting the electronic element of the circuit board, which are provided by the embodiment of the invention, the method comprises the steps of manufacturing an image data set: collecting original images of a plurality of electronic circuit boards, intercepting the original images from different starting points, and selecting images containing electronic elements to form an image dataset; an image dataset labeling step: respectively labeling the electronic elements of each image in the training image data set; and (3) training a detection model: training and verifying a model based on a Swin-transducer neural network on the marked image data to generate an image detection model; an image detection step: firstly scanning and segmenting an image to be detected, then inputting segmented sub-images into the image detection model, and detecting electronic elements in the image to be detected; and (3) screening detection results: and mapping the detection results of all the sub-images back to the original image coordinate space to be detected from the divided sub-image coordinate space, filtering the detection results with similar positions, and outputting the screened element types and the position and length and width size information of the element types in the original image. The detection method of the electronic element of the circuit board is characterized in that a detection algorithm based on a Swin-transducer neural network is adopted for detection, the detection algorithm has higher target recognition accuracy than that of SSD, YOLO series, original Mask R-CNN and other deep learning detection algorithms, the requirement of the detection process flow of the whole circuit board can be met in detection instantaneity, automatic real-time online detection can be realized, the detection efficiency is greatly improved, in addition, a scanning and image segmentation method is adopted to segment the circuit board image to be detected into a plurality of sub-images which are suitable for the input size of a detection model, then the sub-images are input into the model for detection, the detection results of all the sub-images are mapped back to the original circuit board image, and the repeated detection results are filtered. The method can ensure that each electronic element on the circuit board image is completely distributed in each sub-image so as to prevent the condition of missing detection in the detection process, thereby improving the accuracy of the whole detection process flow.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for detecting electronic components of a circuit board according to an embodiment of the present invention;
FIG. 2 is a partially cut-out image of a circuit board that is to be inspected in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a network structure of a detection model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a first segmented image of a detection module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a second segmented image of a detection module according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a third segmented image of a detection module according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a fourth segmented image of a detection module according to an embodiment of the present invention;
FIG. 8 is a flowchart of a method for screening a detection result according to an embodiment of the present invention;
fig. 9 is a block diagram of a circuit board electronic component detecting device according to an embodiment of the present invention;
fig. 10 is a block diagram of a computer device according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
Referring to fig. 1, in one aspect, an embodiment of the present invention provides a method for detecting electronic components of a circuit board, including:
a step S101 of creating an image dataset: collecting original images of a plurality of circuit boards, intercepting the images from different starting points, and selecting images containing electronic elements from the intercepted images to form an image dataset;
Image dataset labeling step S102: respectively labeling the electronic elements of each image in the training image data set;
detection model training step S103: training and verifying a model based on the Swin-transducer neural network by using the marked image data set to generate an image detection model;
image detection step S104: dividing an image to be detected into sub-images with the size suitable for the input size of a detection model, inputting the sub-images into the image detection model, and detecting electronic elements in the image to be detected.
Detection result screening step S105: and mapping all detection results from the sub-image coordinate space back to the original image coordinate space to be detected, filtering repeated detection results, and outputting the screened element types and the position information thereof on the original image.
In particular, before the electronic component inspection of the image to be inspected, the inspection model based on the Swin-transducer neural network needs to be trained, and training the deep learning network model generally requires a large number of pictures, especially as the number of layers (depth) of the network increases, the number of pictures required is larger. Models trained through a large number of diversified images tend to be more accurate in recognition and are not easy to overfit. In the image dataset creation step S101: forming a training image data set by collecting a plurality of original images of a circuit board containing electronic elements; the method comprises the steps of firstly collecting original images of a plurality of circuit boards, then intercepting different areas of each original image according to the size of 420×420, and generating a plurality of training images. For example, for an original image of a collected circuit board, the size is 3456×2629, the image is 3425+.420+.8.2 in the X (width) direction, which can be divided into 9 segments, since there may also be electronic components to be identified at the edges of the image, the back of the 0.2 part is filled with black, the width of the image is complemented to a multiple of 420. Similarly, in the Y (height) direction, 2629+.420+.6.3, the image can be segmented into 7 segments, so the entire image, starting from the top left corner [0, 0] point, can be segmented into 9×7=63 images. But the number of pictures thus split is still too small. In order to divide more images in one original picture, each original picture can be divided for a plurality of times from different initial positions; for example, from the points [50,0], [0, 50], [100, 0], [0, 100], [150,0], [0, 150], etc., are sequentially cut.
In this way, the image is captured, and some of the same electronic components appear multiple times in different captured images, but this does not affect the diversity of the data set. Because they are different locations in different images, although they are the same elements. In addition, the electronic components on the PCBA boards often have respective material numbers, the shapes and the sizes of the components with the same material numbers are very similar, but the positions of the components on different PCBA boards are changed, so that by using the method, an original image is divided according to different areas, more training pictures can be manufactured, and meanwhile, the diversity of a data set can be ensured. Similarly, we also take this clipping approach to other circuit board images, then select the image containing the electronic components from the clipped images, and finally collect 2325 images to make up the dataset.
In step S102: and (3) carrying out electronic element labeling on each image in the image data set by using LabelImg software, wherein the format of a labeling file is xml. In the embodiment of the invention, five types of electronic elements are detected in total: as shown in fig. 2, the blue ring resistor BR, the electrolytic capacitor CD, the gray ring resistor GR, the faraday capacitor C, and the diode D. These classes of components are widely used in the electronics industry and are very versatile components.
In the image dataset labeling step S102, the method for labeling electronic components on each image in the training image dataset includes the steps of: when the electronic element is displayed completely in the image, marking the electronic element; when the electronic component is not displayed completely in the image, the electronic component is not marked. That is, when the training data set is created, the complete resistor and other electronic components are labeled, while incomplete electronic components at the edges of the image are not labeled. The annotation file contains the category of the electronic element and the coordinate information of the upper left corner and the lower right corner of the target frame in the image.
In the test model training step S103: dividing the image data set into a training set and a verification set, training and verifying the identification model based on the Swin-transducer network, and generating an image detection model. In the embodiment of the invention, the Swin-transducer is used for training a detection model instead of the original backbone network ResNet101, as shown in FIG. 3. Swin is specifically designed for application of a transducer network in the field of vision, and has two main characteristics: sliding windows and hierarchical representations. The function of the sliding window is to calculate self-attention in windows that do not overlap locally and allow for cross-window connections; the hierarchical structure allows the model to adapt to pictures of different scales and makes the computational complexity of the model linear with the image size. Compared with SSD, YOLO series, original Mask R-CNN and other algorithms, the electronic element detection algorithm has higher target recognition accuracy, and can meet the requirements of the detection process flow of the whole circuit board on detection instantaneity.
80% of the pictures of the data set are taken as training sets and 20% of the pictures are taken as verification sets. The training class number of the model is set to 5, the total training round number is set to 500, the Batch Size for training is set to 20, and the Batch Size is the number of samples selected for one training. In addition, although our dataset has more than 2000, there are too few parameters relative to the overall model 48,000,000, so we use the method of transfer learning in training. The transfer learning is to transfer knowledge in one domain (namely, source domain) to another domain (namely, target domain) so that the target domain can obtain better learning effect. Generally, the data volume of the source field is sufficient, and the data volume of the target field is small, and particularly in the industrial detection industry, it is very difficult to collect a great number of training images, and in this case, the training model is very suitable to use the idea of migration learning.
In one embodiment of the present invention, in the image detection step S103, when training a detection model, a pre-training model is loaded, which has been trained on a large dataset (source domain data) such as ImageNet-1k (public image database) having 1000 classifications, including 1,281,167 training images, 50,000 verification images and 100,000 test images. The pre-training model is loaded, so that the network of the detection model can obtain better initialization weight, and the pre-training model can help the new training model to identify the characteristics of the edge, texture, shape and the like of the object by training on a large data set, so that the difficulty of learning the new model is greatly reduced. After 500 rounds of training, the aMP@0.5 index of the detection model on the verification set is 0.979, and the requirement of the detection process of the electronic element of the circuit board is met. In addition, in the later production process, new circuit board images can be continuously supplemented into the data set, and along with the continuous increase of the scale and diversity of the data set, new detection models are retrained, and the detection accuracy of the models can be continuously improved.
After the model training is completed through the steps, an image detection model is generated and can be used for detecting the electronic elements on the new image. In the image detection step S104: and detecting the electronic element in the image to be detected through the image detection model. The method comprises the following steps: the image detection model is led into an industrial personal computer of the detection equipment, and each time a new circuit board is photographed by the CCD camera, then the picture is detected.
In an embodiment of the present invention, the image detection step S104 further includes, before detecting the image to be detected, dividing the image into detection images with a size suitable for the input size of the image detection model by using a scan division method:
a first round of segmentation step: and (3) starting the image to be detected from the [0,0] position, sequentially dividing the image according to the second set size, and filling with black when the terminal size of the image to be detected is not enough to the second set size.
A second round of segmentation step: and (3) starting the image to be detected from the position of the pixel point [ width/2, 0 of the second set size ], sequentially dividing the image according to the second set size, and when the tail end size of the image to be detected is not enough to the second set size, reversely pushing the width of the second set size from the right tail end edge of the image to be detected to the left for the last time to divide.
A third wheel segmentation step: and (3) starting the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size, sequentially dividing the image according to the second set size, and when the bottom end size of the image to be detected is not enough to the second set size, reversely pushing the height of the second set size upwards from the bottom end edge of the image to be detected for the last time to divide.
Fourth wheel segmentation step: and (3) starting the image to be detected from the position of [ width/2 of the second set size, height/2 of the second set size ], sequentially dividing the image according to the second set size, and removing the rest image when the size of the right lower end of the image to be detected is not enough to the second set size.
Specifically, since the size of the training set image is 420×420. In some applications, the newly acquired circuit board image is much larger than this, and the normal width and height are both over 1000. If the entire original image is reshaped (resize) to 420 x 420, then some of the electronic components on the image will be squeezed very small, with a large difference from the component size in the training set, ultimately affecting the detection accuracy.
Therefore, in the embodiment of the invention, before the image to be detected is input into the image detection model, the image to be detected is segmented, and then the segmented sub-image is input into the image detection model for detection, so that the detection precision of the image with larger size is improved.
Specifically in the first round of segmentation step: starting the image to be detected from the [0,0] position, and sequentially dividing the image according to a second set size; as an illustrative embodiment, as shown in fig. 4, we start the original image to be detected (marked by black solid line box) from the top left corner [0,0] point, and intercept the images sequentially according to 672×672 size, and fill in with black when the end size of the image is not enough 672, so that the end edge of the image can be covered. The size of 672×672 is larger than that of the training image, but the detection accuracy is not degraded by the test because the shape and size of the electronic element in the 672×672 cut image are not affected. In addition, if an original image is cut according to the size of 672×672, the number of times of cutting the original image is smaller than 420×420, and accordingly, the detection times are reduced, so that the detection efficiency is improved.
Since during the first round of image segmentation some electronic components may be located exactly at the edges of the segmented image, for example some resistors in the first half of the previous image and some resistors in the second half of the next image, in this case these resistors may not be detected, since when the training dataset is created (image dataset labeling step S102), we label all resistors and other electronic components, but incomplete electronic components at the edges of the image are not labeled. To completely inspect all the electronic components on the original circuit board, the electronic components are completely distributed in the segmented sub-image, that is, the edge portions of the sub-image segmented from the [0,0] point in the X direction and the Y direction and the crossing area portions of the X and Y edges are covered, so that three-wheel image segmentation is also required.
In the second round of dividing step: starting the image to be detected from the position of the pixel point [ width/2, 0 of the second set size ], and sequentially intercepting the image according to the second set size; specifically, the size of the divided sub-images is 672×672, and as shown in fig. 5, the division may be sequentially performed from the point [336,0 ]. If the width of the original image is 3456, 3525≡672≡5.2, then the original image is divided in the X width direction by dividing it 4 times in turn, and then the 5 th time is divided by pushing 672 the length from the right end edge of the image to the left. This round of segmentation covers the vertical edges in the X direction that result from the first round of segmentation starting from the [0,0] point.
In the third round of segmentation step: starting the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size, and sequentially dividing the image according to the second set size; specifically, in fig. 6, starting from the point [0,336], if the height of the original image is 2518, 2518≡672≡3.7, then it is cut out 2 times in the Y height direction in order, the 3 rd pass then cuts back 672 the height from the bottom end edge of the original image, covering the horizontal edge created by the first pass cut in the Y direction.
In the fourth wheel segmentation step: and (3) starting the image to be detected from the position of the width/2 of the second set size and the height/2 of the second set size, and sequentially intercepting the images according to the second set size. As shown in fig. 7, from the point [336,336], the cutting is performed in order 4 times in the X direction and 3 times in the Y direction, covering the intersection area portion of the first round of cutting X and Y edges.
Thus, through four-wheel segmentation, it is ensured that all electronic components to be detected of the circuit board can be completely present in the segmented sub-images as much as possible. In addition, in each image segmentation process, the position [ Xi, yi ] of the top left corner vertex of each sub-image in the original image to be detected is recorded in the program.
The segmentation of the image to be detected by the four-wheel scan described above allows the electronic components to be distributed throughout the sub-images, but this approach also causes the possibility that the same component may appear repeatedly in different sub-images. When mapping back to the original image, multiple approximate detection results appear, so repeated similar detection results need to be screened out.
Referring to fig. 8, in one embodiment of the present invention, the image detection result screening step S105 includes the steps of:
s1051, mapping the detection results of all the divided sub-images of the image to be detected from the sub-image coordinate space back to the original image coordinate space to be detected; specifically, adding the coordinate value of the detected top-left corner vertex of the target frame in the sub-image and the coordinate value [ Xi, yi ] of the top-left corner vertex of the sub-image in the original image;
in step S1052, all the electronic components mapped back to the original image space are sorted by category, for example, all the detected blue ring resistors BR are of the same category, and sorted;
In step S1053, the distance between each element in the same class element and the top left corner vertex of all other same class element target frames is calculated;
s1054, judging whether the positions of the electronic elements of the same category on the original image to be detected are close; specifically, two components are not usually mounted at the same position on the circuit board, and in the present invention, the threshold value for determining whether the two components are close is set to 20, because the shortest dimension in the width and length directions of the target frames of all electronic components in the data set is 20 through statistics.
S1055, if the distance between a certain element and other elements of the same class is larger than a threshold value, outputting the class name of the element and the position and length and width size information of the target frame in the original image as detection results.
S1056, if the distance between two or more elements is smaller than the threshold value, selecting the element with the largest size of the target frame in the length direction from the elements as a detection result. Specifically, the direction in which the target frame is larger in the X and Y directions is set as the length direction, and a target frame of a longer size is selected, more likely to cover the electronic component.
The detection method for the electronic element of the circuit board provided by the embodiment of the invention is used for detection by adopting a detection algorithm based on a Swin-transducer neural network, and the detection algorithm has higher target recognition accuracy than SSD, YOLO series, original Mask R-CNN and other deep learning detection algorithms, and can meet the requirements of the detection process flow of the whole circuit board on detection instantaneity. The method can realize automatic real-time online detection, greatly improves detection efficiency, and in addition, the method of scanning and dividing images is adopted to divide the circuit board image to be detected into a plurality of sub-images which are suitable for the input size of the detection model, then the sub-images are input into the model for detection, then the detection results of all the sub-images are mapped back to the original circuit board image, and repeated detection results are filtered. The method can ensure that each electronic element on the circuit board image is completely distributed in each sub-image so as to prevent the condition of missing detection in the detection process, thereby improving the accuracy of the whole detection process flow.
In a second aspect, referring to fig. 9, an embodiment of the present invention further provides an apparatus for detecting an electronic component of a circuit board, including: the method comprises the steps of manufacturing an image data set module, an image data set labeling module, a detection model training module, an image detection module and a detection result screening module: the image data set making module is used for collecting original images of a plurality of circuit boards, intercepting the original images from different starting points, and selecting intercepted images containing electronic elements to form an image data set.
The image data set labeling module is used for respectively labeling the electronic elements of the images in the image data set.
The detection model training module is used for dividing the marked image data set into a training set and a verification set, training and verifying the detection model based on the Swin-transducer neural network, and generating an image detection model.
The image detection module is used for firstly dividing an original electronic circuit board image into detection images with the size suitable for the input size of a detection model by adopting a scanning division method, and then inputting the detection images into the detection model for detection.
The detection result screening module is used for screening the detection results of the segmented sub-images, filtering out repeated detection results and outputting the finally detected electronic element types and the position and size information of the target frame on the original image.
Further, according to an embodiment of the present invention, the image dataset making module includes: the system comprises an image intercepting module and an image data set module, wherein the image intercepting module is used for intercepting each original image from different initial positions for a plurality of times, and each interception is sequentially divided according to a first set size to intercept the original image into a plurality of images.
The image data set module is used for picking out images containing electronic elements from all the intercepted images to form the image data set.
Further, according to an embodiment of the present invention, the image detection module further includes a complete image segmentation module, the complete image segmentation module including: the system comprises a first wheel segmentation module, a second wheel segmentation module, a third wheel segmentation module and a fourth wheel segmentation module, wherein the first wheel segmentation module is used for sequentially segmenting images to be detected from the [0,0] position according to a second set size, and black filling and filling are carried out when the tail end size of the images to be detected is not enough to the second set size.
The second round of segmentation module is used for sequentially segmenting the image to be detected according to the second set size from the position of the pixel point [ width/2, 0 of the second set size ], and when the right end of the image to be detected is of the second set size which is insufficient in size, the width of the second set size is reversely pushed to the left from the edge of the right end of the image to be detected for segmentation for the last time.
The third wheel segmentation module is used for sequentially segmenting the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size according to the second set size, and when the bottom end size of the image to be detected is not enough to be the second set size, the height of the second set size is reversely pushed upwards from the bottom end edge of the image to be detected for the last time to be intercepted.
The fourth segmentation module is used for sequentially segmenting the image to be detected from the position of [ width/2 of the second set size, height/2 of the second set size ] according to the second set size, and removing the residual image when the size of the right lower end of the image to be detected is not enough to the second set size.
Further, according to an embodiment of the present invention, the detection result filtering module includes a repeated result filtering module, and the repeated result filtering module includes: the detection result mapping module is used for mapping all detection results from respective sub-images back to the space of the image to be detected, namely, the coordinate value of the top left corner vertex of the target frame of the detection result in the sub-image is added with the coordinate value of the top left corner vertex of the sub-image in the image to be detected, and the position information of the detection result in the image to be detected can be obtained.
The detection result classification module is used for classifying and sorting all detection results of the image to be detected according to the category of the electronic element.
And the screening and filtering module is used for screening the elements according to the position information of the detected elements in the results of the same category so as to filter repeated detection results.
Further, according to an embodiment of the present invention, the filtering module includes: the distance calculation module is used for calculating the distance between the target frame of each element in the same element and the top left corner vertex of the target frame of other elements.
The screening module is used for judging whether the distances between the elements are close, and if the distances between one element and other elements are larger than a set threshold value, outputting the category name of the element and the position and size information of the target frame on the image to be detected; if the distance between two or more elements is smaller than the set threshold value, selecting the longest length of the target frame from the elements as a detection result, and outputting the category name of the detection result and the position and size information of the target frame on the image to be detected.
In a third aspect, referring to fig. 10, the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for detecting electronic components of a circuit board when executing the computer program.
The computer program may be divided into one or more modules/units, which are stored in the memory and executed by the processor to accomplish the present invention, for example. The one or more modules/units may be a series of computer program instruction segments capable of performing the specified functions, which instruction segments describe the execution of the computer program in the computer device.
The computer device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the figures are merely examples of computer devices and are not limiting of computer devices, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the computer devices may also include input and output devices, network access devices, buses, etc.
The processor may be a central processing unit (Central Processing Unit, CPU), or other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field ProgrammableGate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete preset hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The memory may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device. Further, the memory may also include both internal storage units and external storage devices of the computer device. The memory is used for storing the computer program and other programs and data required by the computer device. The memory may also be used to temporarily store data that has been output or is to be output.
In a fourth aspect, embodiments of the present invention also provide a computer storage medium having stored thereon a computer program which, when executed by a processor, implements a circuit board electronic component inspection method as described above.
The computer program may be stored in a computer readable storage medium, which computer program, when being executed by a processor, may carry out the steps of the various method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, randomAccess Memory), an electrical carrier signal, a telecommunication signal, a software distribution medium, and so forth.
It should be noted that the computer readable medium may include content that is subject to appropriate increases and decreases as required by jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is not included as electrical carrier signals and telecommunication signals.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs.
The modules or units in the system of the embodiment of the invention can be combined, divided and deleted according to actual needs.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided herein, it should be understood that the disclosed apparatus/computer device 600 and method may be implemented in other ways. For example, the above-described apparatus/computer device 600 embodiments are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (9)
1. A method for inspecting electronic components of a circuit board, comprising:
a step of producing an image data set: collecting a plurality of original images of the circuit board, intercepting the original images of the circuit board from different starting points, and selecting intercepted images containing electronic elements to form an image data set;
an image dataset labeling step: respectively labeling the electronic elements of each image in the image data set;
and (3) training a detection model: dividing the marked image data set into a training set and a verification set, training and verifying a detection model based on a Swin-transducer neural network, and generating an image detection model;
an image detection step: firstly, dividing an original electronic circuit board image into detection images suitable for the input size of the image detection model by adopting a scanning segmentation method, and then inputting the detection images into the image detection model for detection;
and (3) screening detection results: screening the detection results of all the detection images, removing repeated detection results, and marking the types and positions of the finally detected electronic elements on the corresponding original electronic circuit board images.
2. The method for inspecting electronic components of a circuit board according to claim 1, wherein said capturing of said original image of the circuit board from different starting points, selecting a captured image containing electronic components to form an image dataset, comprises:
intercepting each original image from different initial positions for multiple times, sequentially segmenting each time according to a first set size, and intercepting the original image into a plurality of images;
and selecting the image containing the electronic element from all the intercepted images to form the image data set.
3. The method for inspecting electronic components of a circuit board according to claim 1, wherein the method for labeling electronic components for each image in the image dataset comprises:
when the electronic element is displayed completely in the image, marking the electronic element; the annotation file comprises the category of the electronic element and the coordinate information of the upper left corner and the lower right corner of the target frame in the image;
some electronic components are positioned at the edge of the image, part of the electronic components are cut off, and when the display is incomplete, the electronic components are not marked.
4. The method for inspecting electronic components of circuit board according to claim 1, wherein the steps of dividing the labeled image data set into a training set and a verification set, training and verifying the inspection model based on the Swin-transducer neural network, and generating the image inspection model include:
Dividing the image data set into a training set and a verification set according to a certain proportion;
training and verifying the detection model based on the Swin-transducer neural network; when training the model, firstly loading a pre-training model, and then carrying out detection model training and verification on the marked image data.
5. The method for inspecting electronic components of a circuit board according to claim 1, wherein the method for inspecting an original electronic circuit board image by dividing the image into inspection images having a size suitable for an input size of the image inspection model and inputting the inspection images into the image inspection model comprises:
a first round of segmentation step: starting the original electronic circuit board image as an image to be detected from the [0,0] position, sequentially dividing the image according to a second set size, and filling with black when the tail end size of the image to be detected is not enough to the second set size;
a second round of segmentation step: sequentially dividing the image to be detected from the position of the pixel point [ width/2, 0 of the second set size ] according to the second set size, and when the width size of the tail end of the image to be detected is not enough to the second set size, reversely pushing the width of the second set size to the left from the edge of the right tail end of the image to be detected for the last time to divide;
A third wheel segmentation step: starting the image to be detected from the position of the pixel point [0, the height/2 ] of the second set size, sequentially dividing the image according to the second set size, and when the height size of the bottom end of the image to be detected is not enough to be the second set size, reversely pushing the height of the second set size upwards from the edge of the bottom end of the image to be detected to divide the image to be detected for the last time;
fourth wheel segmentation step: starting the image to be detected from the position of the width/2 of the second set size and the height/2 of the second set size, sequentially dividing the image according to the second set size, and removing the residual image when the width and the height of the tail end of the image to be detected are not enough to the second set size;
and inputting the sub-images obtained by dividing each round in turn into the detection model for detection.
6. The method for inspecting electronic components of circuit board according to claim 5, wherein said method for screening the inspection results of all the inspection images, removing the repeated inspection results, and marking the type and position of the finally inspected electronic components on the original electronic circuit board image comprises:
mapping all detection results of the sub-images from a sub-image space back to the image space to be detected;
Then sorting according to the detected categories of the electronic elements;
in the same category of results, filtering is performed based on the detected positional information of the element to filter out duplicate detection results.
7. The method for inspecting electronic components of circuit board according to claim 6, wherein said filtering based on the positional information of the inspected components to filter out repeated inspection results comprises:
calculating the distance between each element in the elements of the same class and the top left corner vertex of the target frame of all other elements;
if the distance between a certain element and other elements is larger than the set threshold value, outputting the category of the element and the position and size information of the target frame on the image to be detected;
if the distance between two or more elements is smaller than the set threshold value, selecting the element with the largest length dimension of the target frame from the elements as a final detection result, and outputting the element category of the detection result and the position and dimension information of the target frame on the image to be detected.
8. An apparatus for inspecting electronic components of a circuit board, comprising:
and (3) an image dataset making module: the image data set making module is used for collecting original images of a plurality of circuit boards, intercepting the original images from different starting points, and then selecting intercepted images containing electronic elements to form an image data set;
The image dataset labeling module: the image data set labeling module is used for labeling electronic elements on each image in the image data set;
and the detection model training module: the detection model training module is used for training and verifying a detection model based on a Swin-transducer network structure on an image data set to generate an image detection model;
an image detection module: the image detection module is used for firstly dividing an original electronic circuit board image into detection images suitable for the input size of the image detection model by adopting a scanning segmentation method, and then inputting the detection images into the image detection model for detection;
and a detection result screening module: the detection result screening module is used for screening the detection results of the segmented sub-images, filtering out repeated detection results and outputting the finally detected electronic element types and the position and size information of the target frame on the original image.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements a method of circuit board electronic component inspection according to any one of claims 1 to 7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310527853.XA CN116245882A (en) | 2023-05-11 | 2023-05-11 | Circuit board electronic element detection method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310527853.XA CN116245882A (en) | 2023-05-11 | 2023-05-11 | Circuit board electronic element detection method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116245882A true CN116245882A (en) | 2023-06-09 |
Family
ID=86629917
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310527853.XA Pending CN116245882A (en) | 2023-05-11 | 2023-05-11 | Circuit board electronic element detection method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116245882A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197248A (en) * | 2023-11-08 | 2023-12-08 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN118332984A (en) * | 2024-06-12 | 2024-07-12 | 成都信息工程大学 | Analysis method and device for digital circuit, electronic equipment and storage medium |
| CN118657177A (en) * | 2024-08-13 | 2024-09-17 | 昆明理工大学 | Circuit board defect identification transducer network distributed reasoning method based on IEC61499 standard |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017215241A1 (en) * | 2016-06-16 | 2017-12-21 | 广州视源电子科技股份有限公司 | Circuit board element missing detection method and system |
| CN113159064A (en) * | 2021-04-06 | 2021-07-23 | 高书俊 | Method and device for detecting electronic element target based on simplified YOLOv3 circuit board |
| WO2021232670A1 (en) * | 2020-05-22 | 2021-11-25 | 深圳技术大学 | Pcb component identification method and device |
| CN114155417A (en) * | 2021-12-13 | 2022-03-08 | 中国科学院空间应用工程与技术中心 | Image target identification method and device, electronic equipment and computer storage medium |
| CN115063803A (en) * | 2022-05-31 | 2022-09-16 | 北京开拓鸿业高科技有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN115731186A (en) * | 2022-11-22 | 2023-03-03 | 中国联合网络通信集团有限公司 | Fabric quality detection method, device, equipment and storage medium |
| CN115829995A (en) * | 2022-12-20 | 2023-03-21 | 浙江理工大学 | Cloth flaw detection method and system based on pixel-level multi-scale feature fusion |
-
2023
- 2023-05-11 CN CN202310527853.XA patent/CN116245882A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017215241A1 (en) * | 2016-06-16 | 2017-12-21 | 广州视源电子科技股份有限公司 | Circuit board element missing detection method and system |
| WO2021232670A1 (en) * | 2020-05-22 | 2021-11-25 | 深圳技术大学 | Pcb component identification method and device |
| CN113159064A (en) * | 2021-04-06 | 2021-07-23 | 高书俊 | Method and device for detecting electronic element target based on simplified YOLOv3 circuit board |
| CN114155417A (en) * | 2021-12-13 | 2022-03-08 | 中国科学院空间应用工程与技术中心 | Image target identification method and device, electronic equipment and computer storage medium |
| CN115063803A (en) * | 2022-05-31 | 2022-09-16 | 北京开拓鸿业高科技有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN115731186A (en) * | 2022-11-22 | 2023-03-03 | 中国联合网络通信集团有限公司 | Fabric quality detection method, device, equipment and storage medium |
| CN115829995A (en) * | 2022-12-20 | 2023-03-21 | 浙江理工大学 | Cloth flaw detection method and system based on pixel-level multi-scale feature fusion |
Non-Patent Citations (3)
| Title |
|---|
| RUI HUANG 等: "A Rapid Recognition Method for Electronic Components Based on the Improved YOLO-V3 Network", 《ELECTRONICS》, vol. 8, no. 8, pages 1 - 18 * |
| 刘小燕;李照明;段嘉旭;项天远;: "基于卷积神经网络的印刷电路板色环电阻检测与定位方法", 《电子与信息学报》, vol. 42, no. 09, pages 2302 - 2311 * |
| 刘岩;: "卷积神经网络在光学元件损伤检测中的应用", 《电脑知识与技术》, vol. 13, no. 04, pages 178 - 182 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197248A (en) * | 2023-11-08 | 2023-12-08 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
| CN117197248B (en) * | 2023-11-08 | 2024-01-26 | 成都数之联科技股份有限公司 | Electrolytic capacitor direction judging method, device, equipment and storage medium |
| CN118052861A (en) * | 2024-02-20 | 2024-05-17 | 上海赫立智能机器有限公司 | Zero plane acquisition method, system, medium and electronic equipment |
| CN118332984A (en) * | 2024-06-12 | 2024-07-12 | 成都信息工程大学 | Analysis method and device for digital circuit, electronic equipment and storage medium |
| CN118657177A (en) * | 2024-08-13 | 2024-09-17 | 昆明理工大学 | Circuit board defect identification transducer network distributed reasoning method based on IEC61499 standard |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108918536B (en) | Tire mold surface character defect detection method, device, equipment and storage medium | |
| CN109342456B (en) | Welding spot defect detection method, device and equipment and readable storage medium | |
| CN116245882A (en) | Circuit board electronic element detection method and device and computer equipment | |
| CN111444921A (en) | Scratch defect detection method and device, computing equipment and storage medium | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN111415329A (en) | Workpiece surface defect detection method based on deep learning | |
| JP2011214903A (en) | Appearance inspection apparatus, and apparatus, method and program for generating appearance inspection discriminator | |
| CN114549981A (en) | Intelligent inspection pointer type instrument recognition and reading method based on deep learning | |
| CN109859164A (en) | A method of by Quick-type convolutional neural networks to PCBA appearance test | |
| CN111310826B (en) | Method and device for detecting labeling abnormality of sample set and electronic equipment | |
| CN113205511B (en) | Electronic component batch information detection method and system based on deep neural network | |
| CN111126393A (en) | Vehicle appearance refitting judgment method and device, computer equipment and storage medium | |
| CN114723709A (en) | Tunnel disease detection method and device and electronic equipment | |
| CN113159064A (en) | Method and device for detecting electronic element target based on simplified YOLOv3 circuit board | |
| CN111461133A (en) | Express delivery surface single item name identification method, device, equipment and storage medium | |
| TW202127371A (en) | Image-based defect detection method and computer readable medium thereof | |
| CN111179263A (en) | Industrial image surface defect detection model, method, system and device | |
| CN112508935A (en) | Product packaging detection method and system based on deep learning and product packaging sorting system | |
| CN114926441A (en) | Defect detection method and system for machining and molding injection molding part | |
| CN111626249A (en) | Method and device for identifying geometric figure in topic image and computer storage medium | |
| CN114549493A (en) | Magnetic core defect detection system and method based on deep learning | |
| CN114445410A (en) | Circuit board detection method based on image recognition, computer and readable storage medium | |
| CN116777877A (en) | Circuit board defect detection method, device, computer equipment and storage medium | |
| CN114170168A (en) | Display module defect detection method, system and computer readable storage medium | |
| CN112561885B (en) | YOLOv 4-tiny-based gate valve opening detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230609 |