CN116012825A - Electronic component intelligent identification method based on multiple modes - Google Patents
Electronic component intelligent identification method based on multiple modes Download PDFInfo
- Publication number
- CN116012825A CN116012825A CN202310041442.XA CN202310041442A CN116012825A CN 116012825 A CN116012825 A CN 116012825A CN 202310041442 A CN202310041442 A CN 202310041442A CN 116012825 A CN116012825 A CN 116012825A
- Authority
- CN
- China
- Prior art keywords
- light source
- network
- circuit board
- yolov5
- source mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000004927 fusion Effects 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000012360 testing method Methods 0.000 claims abstract description 15
- 238000012795 verification Methods 0.000 claims abstract description 11
- 238000010586 diagram Methods 0.000 claims description 15
- 238000002372 labelling Methods 0.000 claims description 6
- 239000000463 material Substances 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 12
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Abstract
An intelligent recognition method of electronic components based on multiple modes relates to the technical field of circuit board detection, and comprises the steps of firstly shooting images of a target circuit board under two modes of a direct light source and a side light source, marking target components to be recognized in the images, dividing marking data into a training set, a verification set and a test set, then cutting out a plurality of sample images from the images of the two modes at random, constructing a YOLOv5 network by taking an Efficientlite lightweight model as a backstone and a self-adaptive feature fusion structure as a Head, taking the cut sample images as training data, training the YOLOv5 network, verifying the marking data of the verification set, and testing the marking data of the test set to finally obtain the electronic component recognition model based on the YOLOv5 network; and then the obtained model is used for identifying the electronic components on the subsequent circuit board.
Description
Technical Field
The invention relates to a circuit board detection technology, in particular to a technology based on a multi-mode electronic component intelligent identification method.
Background
In the manufacturing process of electronic equipment, electronic components on a PCB (printed circuit board) are often identified by using an intelligent vision technology to realize automatic production. At present, a traditional image recognition algorithm based on machine learning and an intelligent image algorithm based on deep learning are adopted for recognition of electronic components.
The traditional image recognition algorithm based on machine learning generally comprises the steps of firstly carrying out image segmentation on each component image in an original image of the surface of a circuit board to obtain each component image, then carrying out feature extraction and selection based on the segmented images, manually extracting features, and then recognizing different electronic components according to the extracted features. The algorithm relies on manual feature extraction, has complicated process, poor accuracy and poor generalization capability (accuracy of model prediction of new data).
The intelligent image algorithm based on deep learning usually uses the existing classical target detection network as a backbone, adopts a mode of transfer learning or de-novo training to train a self-defined data set, and then uses a model obtained by training to automatically identify electronic components in an image. The method has the defect that the classical target detection network model often ignores limited computing resources of an actual application platform and ignores inconsistent information of the feature map under different scales, so that the effect of feature fusion can be affected.
Disclosure of Invention
Aiming at the defects in the prior art, the technical problem to be solved by the invention is to provide the multi-mode-based intelligent electronic component identification method which can automatically extract the characteristics, has good characteristic fusion effect and high identification accuracy.
In order to solve the technical problems, the invention provides a multi-mode-based intelligent electronic component identification method which is characterized by comprising the following specific steps:
1) Shooting a direct light source mode image and a side light source mode image of a target circuit board;
the direct light source mode diagram refers to a top view of the target circuit board under the direct irradiation of the top light source, and the side light source mode diagram refers to a top view of the target circuit board under the irradiation of the side light source;
2) Labeling a direct light source mode image and a side light source mode image of a target circuit board, and labeling the position and the material type name of a target device to be identified in the image;
3) Dividing the marked data in the step 2) into a training set, a verification set and a test set according to a preset proportion, wherein the training set, the verification set and the test set are three data sets in total;
4) Randomly cutting out N sample images with the same size from each image according to a direct light source mode image and a side light source mode image of a target circuit board, and enabling each sample image to contain at least one marked target device;
5) Constructing a YOLOv5 network by taking an Efficientlite lightweight model as a backstone and a self-adaptive feature fusion structure as a Head;
the model formula of the self-adaptive feature fusion structure is as follows:
in the method, in the process of the invention,
is the eigenvector of the point with the coordinate position (i, j) in the layer 2 eigenvector of the YOLOv5 network,/o>
Scaling the nth layer characteristic diagram of the Yolov5 network to the specified size of the 2 nd layer characteristic diagram, and then obtaining characteristic vectors of points with the coordinate positions of (i, j) in the diagram;
in the method, in the process of the invention,
is the weight of the layer 1 feature map of the YOLOv5 network,/for example>
Is the weight of the layer 2 feature map of the YOLOv5 network,/for example>
Is the weight of the layer 3 feature map of the YOLOv5 network,/for example>
Calculated by Softmax regression algorithm, < - > and->
Is obtained by back propagation network learning;
6) Training the YOLOv5 network constructed in the step 5) by taking all the sample images obtained in the step 4) as training data, verifying by using the marking data of a verification set, and testing by using the marking data of a test set to finally obtain an electronic component identification model based on the YOLOv5 network;
7) And 4) identifying the electronic components on the subsequent circuit board by using the electronic component identification model based on the YOLOv5 network obtained in the step 6).
The electronic component intelligent recognition method based on the multiple modes provided by the invention adopts the self-adaptive feature fusion structure to automatically extract the features, and the mode images of two illumination angles can be used for better describing the features of the electronic component, capturing more feature information, improving the precision and robustness of a model, resisting noise better, and has the advantages of good feature fusion effect and high recognition accuracy. And moreover, the backlight of the YOLOv5 network adopts an Efficientlite lightweight model, so that parameters of the model are reduced, network computing overhead is reduced, and the self-adaptive feature fusion structure can balance precision loss caused by the lightweight of the model, so that feature fusion of the model under different scales is promoted, and the performance of the model is effectively improved.
Detailed Description
The present invention is described in further detail below with reference to specific examples, but the present invention is not limited to the specific examples, and all the similar structures and similar variations using the present invention should be included in the scope of the present invention, where the numbers represent the relationships between the letters and the letters, and the letters and letters in the present invention distinguish the letters and the letters.
The embodiment of the invention provides a multi-mode-based intelligent electronic component identification method, which is characterized by comprising the following specific steps:
1) Shooting a direct light source mode image and a side light source mode image of a target circuit board;
the direct light source mode diagram refers to a top view of the target circuit board under the direct irradiation of the top light source, and the side light source mode diagram refers to a top view of the target circuit board under the irradiation of the side light source;
2) Labeling a direct light source mode image and a side light source mode image of a target circuit board, and labeling the position and the material type name of a target device to be identified in the image;
if the target device in the image is a resistor with the model of R0603, the material class name of the target device is marked as R0603;
3) Dividing the marked data in the step 2) into a training set, a verification set and a test set according to a preset proportion (such as a proportion of 8:1:1), wherein the three data sets are all provided;
the training set is used for training the model and determining parameters, the verification set is used for verifying the performance of the model, and the test set is used for verifying the generalization capability of the model (the accuracy of predicting new data by the model);
4) Randomly cutting out N (N is a preset positive integer) sample images with the same size from each image according to a direct light source mode image and a side light source mode image of a target circuit board, and enabling each sample image to contain at least one marked target device;
for example, 100 sample images with the resolution of 640 multiplied by 640 are randomly cut out from a direct light source mode image and a side light source mode image respectively;
5) Constructing a YOLOv5 network by taking an Efficientlite lightweight model as a Backbone network and a self-adaptive feature fusion structure (ASFF) as a Head;
the model formula of the self-adaptive feature fusion structure is as follows:
in the method, in the process of the invention,
is the eigenvector of the point with the coordinate position (i, j) in the layer 2 eigenvector of the YOLOv5 network,/o>
Scaling the nth layer characteristic diagram of the Yolov5 network to the specified size of the 2 nd layer characteristic diagram, and then obtaining characteristic vectors of points with the coordinate positions of (i, j) in the diagram; n is 1, 2 or 3;
in the method, in the process of the invention,
is the weight of the layer 1 feature map of the YOLOv5 network,/for example>
Is the weight of the layer 2 feature map of the YOLOv5 network,/for example>
Is the weight of the layer 3 feature map of the YOLOv5 network,/for example>
Calculated by Softmax regression algorithm, their value is between 0 and 1 and the sum of the additions is 1 +.>
Is obtained by back propagation network learning;
the afflicientlite lightweight model is in the prior art, the convolutional neural network adopted by the traditional backlight of Yolov5 improves detection performance by deepening the network layer number, widening the channel number and increasing the image resolution, but a network structure too deep is easy to cause gradient disappearance and network precision gain reduction, and a network structure too wide can cause the detection network to be incapable of extracting rich semantic information of deeper layers of the image.
In the traditional Yolov5, feature graphs with different scales depend on a PANet network and an FPN network to perform feature fusion; the FPN network is a classical feature pyramid structure, when an image is input into the detection network, feature extraction is carried out by a trunk part, feature graphs with different spatial resolutions are generated at different depths of the network, the FPN network transmits the feature information from top to bottom at different depths, relevant spatial position information of the feature graphs under different scales is reserved, and the feature information is fused through 1X 1 convolution operation. However, the pyramid structure connected from top to bottom is easy to lose for shallow feature images, and the accuracy of small target detection is low. The PANet network compresses the characteristic channels through convolution operation, which inevitably brings about the problem of information redundancy. PANet networks have given equal attention to different levels of feature maps. However, feature maps with different levels, which sometimes contain conflicting information about the object instance size due to depth differences, can interfere with gradient computation during network training and reduce the effectiveness of feature pyramids.
In order to balance the precision loss brought by simplifying a network model, a large target is usually associated with a deep layer feature map, a small target is associated with a shallow layer feature map, a detection network deepens the layers of the network by downsampling layer by layer and further convolving, so that more semantic information is obtained, and feature maps with different depths are reserved. This is because the feature map of the deeper layer generally has more semantic information, while the shallow feature map has more positional information and less semantic information, both of which are important information for target detection.
To avoid losing information during feature fusion, the present embodiment uses the existing adaptive feature fusion structure (ASFF) as the Head of the YOLOv5 network, which consists of two parts of constant scaling and adaptive fusion, when one object is specified and considered as an object in one level of feature map, the corresponding region in the other level of feature map is considered as background, then adaptively learns the weight parameters through the network back propagation, gives less weight to the location where contradictory regions exist, gives greater weight to the region with consistent size information, and filters out redundant information that interferes with the normal detection of the network to train the network to find the optimal fusion point.
For constant scaling, the adaptive feature fusion Architecture (ASFF) uses different scaling strategies for different levels of spatial feature maps, since they have different channel numbers and resolutions. When the Level2 (layer 2 of the YOLOv5 network) feature map is specified as a positive value, the detection network first compresses the Level 1 (layer 1 of the YOLOv5 network) feature map by a 1×1 convolution operation, and then uses interpolation to increase resolution; for the Level3 (layer 3 of the YOLOv5 network) feature map, a 3 x 3 convolutional layer with a step size of 2 is used to reduce resolution.
6) Training the YOLOv5 network constructed in the step 5) by taking all the sample images obtained in the step 4) as training data, verifying by using the marking data of a verification set, and testing by using the marking data of a test set to finally obtain an electronic component identification model based on the YOLOv5 network;
7) And 4) identifying the electronic components on the subsequent circuit board by using the electronic component identification model based on the YOLOv5 network obtained in the step 6).
Claims (1)
1. An electronic component intelligent recognition method based on multiple modes is characterized by comprising the following specific steps:
1) Shooting a direct light source mode image and a side light source mode image of a target circuit board;
the direct light source mode diagram refers to a top view of the target circuit board under the direct irradiation of the top light source, and the side light source mode diagram refers to a top view of the target circuit board under the irradiation of the side light source;
2) Labeling a direct light source mode image and a side light source mode image of a target circuit board, and labeling the position and the material type name of a target device to be identified in the image;
3) Dividing the marked data in the step 2) into a training set, a verification set and a test set according to a preset proportion, wherein the training set, the verification set and the test set are three data sets in total;
4) Randomly cutting out N sample images with the same size from each image according to a direct light source mode image and a side light source mode image of a target circuit board, and enabling each sample image to contain at least one marked target device;
5) Constructing a YOLOv5 network by taking an Efficientlite lightweight model as a backstone and a self-adaptive feature fusion structure as a Head;
the model formula of the self-adaptive feature fusion structure is as follows:
in the method, in the process of the invention,
is the eigenvector of the point with the coordinate position (i, j) in the layer 2 eigenvector of the YOLOv5 network,/o>
Scaling the nth layer characteristic diagram of the Yolov5 network to the specified size of the 2 nd layer characteristic diagram, and then obtaining characteristic vectors of points with the coordinate positions of (i, j) in the diagram;
in the method, in the process of the invention,
is the weight of the layer 1 feature map of the YOLOv5 network,/for example>
Is the weight of the layer 2 feature map of the YOLOv5 network,
is the weight of the layer 3 feature map of the YOLOv5 network,/for example>
Obtained by calculation through a Softmax regression algorithm,
is obtained by back propagation network learning;
6) Training the YOLOv5 network constructed in the step 5) by taking all the sample images obtained in the step 4) as training data, verifying by using the marking data of a verification set, and testing by using the marking data of a test set to finally obtain an electronic component identification model based on the YOLOv5 network;
7) And 4) identifying the electronic components on the subsequent circuit board by using the electronic component identification model based on the YOLOv5 network obtained in the step 6).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310041442.XA CN116012825A (en) | 2023-01-13 | 2023-01-13 | Electronic component intelligent identification method based on multiple modes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310041442.XA CN116012825A (en) | 2023-01-13 | 2023-01-13 | Electronic component intelligent identification method based on multiple modes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116012825A true CN116012825A (en) | 2023-04-25 |
Family
ID=86031693
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310041442.XA Pending CN116012825A (en) | 2023-01-13 | 2023-01-13 | Electronic component intelligent identification method based on multiple modes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116012825A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932511A (en) * | 2020-08-04 | 2020-11-13 | 南京工业大学 | Electronic component quality detection method and system based on deep learning |
| CN113326735A (en) * | 2021-04-29 | 2021-08-31 | 南京大学 | Multi-mode small target detection method based on YOLOv5 |
| CN113793323A (en) * | 2021-09-16 | 2021-12-14 | 云从科技集团股份有限公司 | Component detection method, system, equipment and medium |
| CN114445706A (en) * | 2022-01-25 | 2022-05-06 | 南京工程学院 | Power transmission line target detection and identification method based on feature fusion |
| CN115116052A (en) * | 2022-06-14 | 2022-09-27 | 深圳职业技术学院 | Orchard litchi identification method, device, equipment and storage medium |
| CN115512206A (en) * | 2022-10-19 | 2022-12-23 | 桂林电子科技大学 | Improved YOLOv5 target detection method suitable for low-illumination environment |
-
2023
- 2023-01-13 CN CN202310041442.XA patent/CN116012825A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932511A (en) * | 2020-08-04 | 2020-11-13 | 南京工业大学 | Electronic component quality detection method and system based on deep learning |
| CN113326735A (en) * | 2021-04-29 | 2021-08-31 | 南京大学 | Multi-mode small target detection method based on YOLOv5 |
| CN113793323A (en) * | 2021-09-16 | 2021-12-14 | 云从科技集团股份有限公司 | Component detection method, system, equipment and medium |
| CN114445706A (en) * | 2022-01-25 | 2022-05-06 | 南京工程学院 | Power transmission line target detection and identification method based on feature fusion |
| CN115116052A (en) * | 2022-06-14 | 2022-09-27 | 深圳职业技术学院 | Orchard litchi identification method, device, equipment and storage medium |
| CN115512206A (en) * | 2022-10-19 | 2022-12-23 | 桂林电子科技大学 | Improved YOLOv5 target detection method suitable for low-illumination environment |
Non-Patent Citations (1)
| Title |
|---|
| 颜宏文 等: "基于改进Yolov5-Lite轻量级的配电组件缺陷识别", 《高电压技术》, 11 August 2022 (2022-08-11), pages 1 - 10 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111612763B (en) | Mobile phone screen defect detection method, device and system, computer equipment and medium | |
| CN108121991B (en) | Deep learning ship target detection method based on edge candidate region extraction | |
| Long et al. | Safety helmet wearing detection based on deep learning | |
| CN111402209B (en) | U-Net-based high-speed railway steel rail damage detection method | |
| CN109190446A (en) | Pedestrian's recognition methods again based on triple focused lost function | |
| CN109285179A (en) | A kind of motion target tracking method based on multi-feature fusion | |
| CN109034245A (en) | A kind of object detection method merged using characteristic pattern | |
| CN111612817A (en) | Target tracking method based on depth feature adaptive fusion and context information | |
| CN110210608A (en) | The enhancement method of low-illumination image merged based on attention mechanism and multi-level features | |
| CN108647695A (en) | Soft image conspicuousness detection method based on covariance convolutional neural networks | |
| CN109635814A (en) | Forest fire automatic testing method and device based on deep neural network | |
| CN114972312A (en) | Improved insulator defect detection method based on YOLOv4-Tiny | |
| CN111353396A (en) | Concrete crack segmentation method based on SCSEOCUnet | |
| CN115330757B (en) | Circuit board welding spot defect detection method and system | |
| CN116309427A (en) | PCB surface defect detection method based on improved YOLOv5 algorithm | |
| CN109086781A (en) | A kind of cabinet lamp state identification method based on deep learning | |
| CN110458019B (en) | Water surface target detection method for eliminating reflection interference under scarce cognitive sample condition | |
| CN108171229A (en) | A kind of recognition methods of hollow adhesion identifying code and system | |
| CN114861761A (en) | Loop detection method based on twin network characteristics and geometric verification | |
| CN111339932A (en) | Palm print image preprocessing method and system | |
| CN109815957A (en) | A kind of character recognition method based on color image under complex background | |
| CN116012825A (en) | Electronic component intelligent identification method based on multiple modes | |
| US20230386023A1 (en) | Method for detecting medical images, electronic device, and storage medium | |
| CN113420623B (en) | 5G base station detection method and system based on self-organizing mapping neural network | |
| CN110084777A (en) | A kind of micro parts positioning and tracing method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |