CN116309429A - Chip defect detection method based on deep learning - Google Patents
Chip defect detection method based on deep learning Download PDFInfo
- Publication number
- CN116309429A CN116309429A CN202310240534.0A CN202310240534A CN116309429A CN 116309429 A CN116309429 A CN 116309429A CN 202310240534 A CN202310240534 A CN 202310240534A CN 116309429 A CN116309429 A CN 116309429A
- Authority
- CN
- China
- Prior art keywords
- convolution
- chip
- feature
- network
- resnet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 title claims abstract description 47
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 238000000605 extraction Methods 0.000 claims abstract description 16
- 230000004927 fusion Effects 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 10
- 238000012360 testing method Methods 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 17
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 9
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 6
- 230000004048 modification Effects 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 230000008034 disappearance Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000001788 irregular Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims 1
- 230000005764 inhibitory process Effects 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a chip defect detection method based on deep learning, which comprises the following steps: acquiring different types of chip surface defect images by using an ultra-high-speed industrial camera; marking positions and types of image defects on the surface of the chip through an image marking tool, constructing a defect data set, and dividing the defect data set into a training data set D and a test data set T; based on the improved FasterR-CNN framework, constructing a deep learning network model for chip defect detection; the chip defect detection method based on deep learning improves the FasterR-CNN algorithm by adopting the measures of deformable convolution reconstruction feature extraction network, multi-scale feature fusion, region of interest alignment (ROI alignment), soft non-maximum value inhibition and the like, solves the problems of small size, complex shape and the like of the chip defect, and improves the defect detection precision.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a chip defect detection method based on deep learning.
Background
After the chip is electrified, a starting instruction is generated to transmit a signal 0 and transmit data, and the household appliances can be intelligently connected, so that the chip is a core foundation of high-end manufacturing industry. However, the manufacturing process of the chip is very complex, and some structures which are not in line with expectations are easily generated in the production process, so that target defects are caused. In recent years, deep learning technology has been rapidly developed, and many researchers have attempted to apply the deep learning technology to defect detection of products based on the strong learning ability and feature extraction ability of deep learning in a large amount of data, so that the efficiency of defect detection has been greatly improved. However, the problem of detecting small targets is still one of the difficulties in the field of detecting surface defects of industrial products, so that improvement of the defect detection rate of small targets such as chips is a problem to be solved.
Disclosure of Invention
The invention aims to provide a chip defect detection method based on deep learning aiming at the defects of the prior art so as to solve the problems in the background art.
In order to achieve the above purpose, the present invention provides the following technical solutions: the method comprises the following steps:
step one: collecting different types of chip surface defect images through a high-speed industrial camera;
step two: marking positions and types of image defects on the surface of the chip through an image marking tool, constructing a defect data set, and dividing the defect data set into a training data set D and a test data set T;
step three: based on the improved FasterR-CNN framework, a deep learning network model for chip defect detection is constructed:
1) The characteristic extraction network is selected, the classical characteristic extraction network comprises VggNet, resNet, googleNet and the like, wherein the ResNet network has the advantages of residual units of the structure, the gradient disappearance problem is solved, the accuracy of the network is improved, and the ResNet-50 is selected as the characteristic extraction network. The ResNet-50 network comprises 49 convolution layers and one full connection layer, and the ResNet-50 network structure can be divided into seven parts, wherein the first part is mainly responsible for calculating input data convolution, regularization, activation functions and maximization pools. Wherein the first part does not contain residual blocks, and the second, third, fourth and fifth part structures all contain residual blocks. In the ResNet-50 network structure, the residual blocks all have three convolutions, and the total of 49 convolution layers in the network and the total of the last full connection layer is 50 layers;
2) The deformable convolution reconstructing the feature extraction network means that a convolution kernel is additionally added with a parameter direction parameter on each element, so that the convolution kernel can be expanded to a large range in the training process. There is no fixed geometry due to the different shape of the chip surface defects. Therefore, the concept of deformable convolution is introduced aiming at ResNet-50 with poor unknown change capability and weak generalization capability, and the recognition capability of the neural network on irregular targets is improved. The realization process of the deformable convolution is as follows:
sub-step 1: and adjusting the input size of the acquired chip image and performing preprocessing operation.
Sub-step 2: from the input image, a feature map is extracted using a conventional convolution kernel.
Sub-step 3: the offset of the deformation resulting in the deformable convolution: and taking the obtained characteristic diagram as input, and adding a convolution layer to the characteristic diagram.
Sub-step 4: the offset layer is 2N (where 2 refers to both x and y directions and N refers to the convolution kernel size) and only the x and y values, representing the offsets in the x and y directions of the image data pixels, respectively, need to be changed since translation is required in the two-dimensional plane.
Wherein during training, the convolution kernels used to generate the output features and the convolution kernels used to generate the offsets are synchronously learned.
3) The multi-scale feature fusion is carried out by introducing a feature pyramid FPN because the chip defect detection is the characteristic of small target detection. Feature fusion is to fuse deep semantic information into a shallow feature map, enrich semantic information by using deep features and simultaneously use features of shallow features. Feature pyramid FPN is introduced into faster R-CNN, features of all layers are integrated into feature pyramid, and feature activation output of second, third, fourth and fifth residual blocks in ResNet-50 network is taken as input. The number of channels of C2-C5 is reduced to 256 by 1x1 convolution, resulting in M2-M5 (taking C2 and M2 as examples, C2 represents the feature matrix of a series of residual structures corresponding to Conv2, and M2 represents the feature matrix of the residual structure of C2 after 1x1 convolution). The shallow layer feature map and the depth feature map with the same size are added through upsampling to obtain the outputs P2-P5 of the FPN, and then 3x3 convolution is adopted. The feature map P6 is obtained by downsampling the two largest pools on the output P5 of the FPN, outputting a multi-scale fusion feature combination. Generating a target candidate frame by utilizing fusion characteristics in the RPN, and obtaining a detection result through classification;
4) Region of interest alignment (ROI alignment). ROIAlign differs from ROI pooling not only in that it is simply quantized and then pooled, but also in that it is converted into continuous operation using a region feature aggregation method. The specific implementation steps of the region of interest alignment are as follows:
sub-step 1: all candidate regions are accessed in a loop and the mapped candidate region floating point coordinates are kept unquantized.
Sub-step 2: the candidate region is divided into zxz cells and each cell is also not quantized.
Sub-step 3: floating point coordinates of the sampling points in each unit are determined, floating point coordinates of the sampling points are calculated using a bilinear interpolation method, and then ROI output of a fixed dimension can be obtained.
5) Soft non-maximum suppression (SoftNMS). Soft non-maximum suppression is a re-estimation of all candidate boxes, but the score is preserved for the worse boxes, and the final implementation result is that the SoftNMS makes a modification to the score of the boxes, preserving a higher recall. The SoftNMS may be represented by the following formula:
as a preferable technical scheme of the invention, si is the score of the detection frame i, M is the highest score frame, bi is the set of detection frames, nt is the set threshold, the NMS sets the score of the detection frame with IOU larger than the threshold to 0, and the soft NMS attenuates the score of the detection frame with IOU larger than the threshold, so that the problems of target missing detection and false detection can be relieved, the IOU threshold of the soft NMS is 0.5, and the lowest score is 0.05.
Compared with the prior art, the invention provides a chip defect detection method based on deep learning, which has the following beneficial effects:
aiming at the characteristics and the requirements of chip defect detection, the invention takes FasterR-CNN as a basic framework and consists of a feature extraction network, a region recommendation network and a detection network. Therefore, the chip defect detection algorithm based on deep learning is provided, and the method improves the FasterR-CNN algorithm by adopting measures such as deformable convolution reconstruction feature extraction network, multi-scale feature fusion, region of interest alignment (ROI alignment), soft non-maximum value inhibition and the like, so that the problems of small size, complex shape and the like of the chip defect are solved, and the defect detection precision is improved.
Drawings
FIG. 1 is a diagram of a ResNet50 network architecture of the present invention;
FIG. 2 is a schematic diagram of a deformable convolution of the present invention;
FIG. 3 is a block diagram of a multi-scale feature fusion network of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1 to 3, in the present embodiment: the method comprises the following steps:
step one: collecting different types of chip surface defect images through a high-speed industrial camera;
step two: marking positions and types of image defects on the surface of the chip through an image marking tool, constructing a defect data set, and dividing the defect data set into a training data set D and a test data set T;
step three: based on the improved FasterR-CNN framework, a deep learning network model for chip defect detection is constructed:
1) The characteristic extraction network is selected, the classical characteristic extraction network comprises VggNet, resNet, googleNet and the like, wherein the ResNet network has the advantages of residual units of the structure, the gradient disappearance problem is solved, the accuracy of the network is improved, and the ResNet-50 is selected as the characteristic extraction network. The ResNet-50 network comprises 49 convolution layers and one full connection layer, and the ResNet-50 network structure can be divided into seven parts, wherein the first part is mainly responsible for calculating input data convolution, regularization, activation functions and maximization pools. Wherein the first part does not contain residual blocks, and the second, third, fourth and fifth part structures all contain residual blocks. In the ResNet-50 network structure, the residual blocks all have three convolutions, and the total of 49 convolution layers in the network and the total of the last full connection layer is 50 layers;
2) The deformable convolution reconstructing the feature extraction network means that a convolution kernel is additionally added with a parameter direction parameter on each element, so that the convolution kernel can be expanded to a large range in the training process. There is no fixed geometry due to the different shape of the chip surface defects. Therefore, the concept of deformable convolution is introduced aiming at ResNet-50 with poor unknown change capability and weak generalization capability, and the recognition capability of the neural network on irregular targets is improved. The realization process of the deformable convolution is as follows:
sub-step 1: and adjusting the input size of the acquired chip image and performing preprocessing operation.
Sub-step 2: from the input image, a feature map is extracted using a conventional convolution kernel.
Sub-step 3: the offset of the deformation resulting in the deformable convolution: and taking the obtained characteristic diagram as input, and adding a convolution layer to the characteristic diagram.
Sub-step 4: the offset layer is 2N (where 2 refers to both x and y directions and N refers to the convolution kernel size) and only the x and y values, representing the offsets in the x and y directions of the image data pixels, respectively, need to be changed since translation is required in the two-dimensional plane.
Wherein during training, the convolution kernels used to generate the output features and the convolution kernels used to generate the offsets are synchronously learned.
3) The multi-scale feature fusion is carried out by introducing a feature pyramid FPN because the chip defect detection is the characteristic of small target detection. Feature fusion is to fuse deep semantic information into a shallow feature map, enrich semantic information by using deep features and simultaneously use features of shallow features. Feature pyramid FPN is introduced into faster R-CNN, features of all layers are integrated into feature pyramid, and feature activation output of second, third, fourth and fifth residual blocks in ResNet-50 network is taken as input. The number of channels of C2-C5 is reduced to 256 by 1x1 convolution, resulting in M2-M5 (taking C2 and M2 as examples, C2 represents the feature matrix of a series of residual structures corresponding to Conv2, and M2 represents the feature matrix of the residual structure of C2 after 1x1 convolution). The shallow layer feature map and the depth feature map with the same size are added through upsampling to obtain the outputs P2-P5 of the FPN, and then 3x3 convolution is adopted. The feature map P6 is obtained by downsampling the two largest pools on the output P5 of the FPN, outputting a multi-scale fusion feature combination. Generating a target candidate frame by utilizing fusion characteristics in the RPN, and obtaining a detection result through classification;
4) Region of interest alignment (ROI alignment). ROIAlign differs from ROI pooling not only in that it is simply quantized and then pooled, but also in that it is converted into continuous operation using a region feature aggregation method. The specific implementation steps of the region of interest alignment are as follows:
sub-step 1: all candidate regions are accessed in a loop and the mapped candidate region floating point coordinates are kept unquantized.
Sub-step 2: the candidate region is divided into zxz cells and each cell is also not quantized.
Sub-step 3: floating point coordinates of the sampling points in each unit are determined, floating point coordinates of the sampling points are calculated using a bilinear interpolation method, and then ROI output of a fixed dimension can be obtained.
5) Soft non-maximum suppression (SoftNMS). Soft non-maximum suppression is a re-estimation of all candidate boxes, but the score is preserved for the worse boxes, and the final implementation result is that the SoftNMS makes a modification to the score of the boxes, preserving a higher recall. The SoftNMS may be represented by the following formula:
it should be noted that: si is the score of the detection frame i, M is the highest score frame, bi is the set of the detection frames, nt is the set threshold, the score of the detection frame with IOU greater than the threshold is set to 0 by the NMS, the score of the detection frame with IOU greater than the threshold is attenuated by the soft NMS, the problems of target missing detection and false detection can be relieved, the IOU threshold of the soft NMS is 0.5, and the lowest score is 0.05.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (2)
1. A chip defect detection method based on deep learning is characterized in that: the method comprises the following steps:
step one: collecting different types of chip surface defect images through a high-speed industrial camera;
step two: marking positions and types of image defects on the surface of the chip through an image marking tool, constructing a defect data set, and dividing the defect data set into a training data set D and a test data set T;
step three: based on the improved Faster R-CNN framework, a deep learning network model for chip defect detection is constructed:
1) The characteristic extraction network is selected, the classical characteristic extraction network comprises VggNet, resNet, googleNet and the like, wherein the ResNet network has the advantages of residual units of the structure, the gradient disappearance problem is solved, the accuracy of the network is improved, and the ResNet-50 is selected as the characteristic extraction network. The ResNet-50 network comprises 49 convolution layers and one full connection layer, and the ResNet-50 network structure can be divided into seven parts, wherein the first part is mainly responsible for calculating input data convolution, regularization, activation functions and maximization pools. Wherein the first part does not contain residual blocks, and the second, third, fourth and fifth part structures all contain residual blocks. In the ResNet-50 network structure, the residual blocks all have three convolutions, and the total of 49 convolution layers in the network and the total of the last full connection layer is 50 layers;
2) The deformable convolution reconstructing the feature extraction network means that a convolution kernel is additionally added with a parameter direction parameter on each element, so that the convolution kernel can be expanded to a large range in the training process. There is no fixed geometry due to the different shape of the chip surface defects. Therefore, the concept of deformable convolution is introduced aiming at ResNet-50 with poor unknown change capability and weak generalization capability, and the recognition capability of the neural network on irregular targets is improved. The realization process of the deformable convolution is as follows:
sub-step 1: and adjusting the input size of the acquired chip image and performing preprocessing operation.
Sub-step 2: from the input image, a feature map is extracted using a conventional convolution kernel.
Sub-step 3: the offset of the deformation resulting in the deformable convolution: and taking the obtained characteristic diagram as input, and adding a convolution layer to the characteristic diagram.
Sub-step 4: the offset layer is 2N (where 2 refers to both x and y directions and N refers to the convolution kernel size) and only the x and y values, representing the offsets in the x and y directions of the image data pixels, respectively, need to be changed since translation is required in the two-dimensional plane.
Wherein during training, the convolution kernels used to generate the output features and the convolution kernels used to generate the offsets are synchronously learned.
3) The multi-scale feature fusion is carried out by introducing a feature pyramid FPN because the chip defect detection is the characteristic of small target detection. Feature fusion is to fuse deep semantic information into a shallow feature map, enrich semantic information by using deep features and simultaneously use features of shallow features. Feature pyramid FPN is introduced into faster R-CNN, features of all layers are integrated into feature pyramid, and feature activation output of second, third, fourth and fifth residual blocks in ResNet-50 network is taken as input. The number of channels of C2-C5 is reduced to 256 by 1x1 convolution, resulting in M2-M5 (taking C2 and M2 as examples, C2 represents the feature matrix of a series of residual structures corresponding to Conv2, and M2 represents the feature matrix of the residual structure of C2 after 1x1 convolution). The shallow layer feature map and the depth feature map with the same size are added through upsampling to obtain the outputs P2-P5 of the FPN, and then 3x3 convolution is adopted. The feature map P6 is obtained by downsampling the two largest pools on the output P5 of the FPN, outputting a multi-scale fusion feature combination. Generating a target candidate frame by utilizing fusion characteristics in the RPN, and obtaining a detection result through classification;
4) Region of interest alignment (ROI alignment). ROI alignment differs from ROI pooling not only in simple quantization and pooling, but also in converting it to continuous operation using a region feature aggregation method. The specific implementation steps of the region of interest alignment are as follows:
sub-step 1: all candidate regions are accessed in a loop and the mapped candidate region floating point coordinates are kept unquantized.
Sub-step 2: the candidate region is divided into zxz cells and each cell is also not quantized.
Sub-step 3: floating point coordinates of the sampling points in each unit are determined, floating point coordinates of the sampling points are calculated using a bilinear interpolation method, and then ROI output of a fixed dimension can be obtained.
5) Soft non-maximum suppression (Soft NMS). Soft non-maximum suppression is a re-estimation of all candidate boxes, but the score is preserved for the worse boxes, with the end result that Soft NMS makes a modification to the score of the boxes, preserving higher recall. The Soft NMS may be represented by the following formula:
2. the method for detecting the chip defects based on deep learning as claimed in claim 1, wherein: in the third step, si is the score of the detection frame i, M is the highest score frame, bi is the set of detection frames, nt is the set threshold, the NMS sets the score of the detection frame with the IOU greater than the threshold to 0, and the Soft NMS attenuates the score of the detection frame with the IOU greater than the threshold, so as to alleviate the problems of target missing detection and false detection, the IOU threshold of the Soft NMS is 0.5, and the lowest score is 0.05.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310240534.0A CN116309429A (en) | 2023-03-14 | 2023-03-14 | Chip defect detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310240534.0A CN116309429A (en) | 2023-03-14 | 2023-03-14 | Chip defect detection method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116309429A true CN116309429A (en) | 2023-06-23 |
Family
ID=86833799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310240534.0A Pending CN116309429A (en) | 2023-03-14 | 2023-03-14 | Chip defect detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116309429A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237683A (en) * | 2023-11-10 | 2023-12-15 | 闽都创新实验室 | Chip defect intelligent detection system based on improved neural network |
| CN118261877A (en) * | 2024-03-29 | 2024-06-28 | 泰山科技学院 | Method for evaluating and maintaining service life of printed circuit electronic component |
-
2023
- 2023-03-14 CN CN202310240534.0A patent/CN116309429A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237683A (en) * | 2023-11-10 | 2023-12-15 | 闽都创新实验室 | Chip defect intelligent detection system based on improved neural network |
| CN117237683B (en) * | 2023-11-10 | 2024-03-26 | 闽都创新实验室 | Chip defect intelligent detection system based on improved neural network |
| CN118261877A (en) * | 2024-03-29 | 2024-06-28 | 泰山科技学院 | Method for evaluating and maintaining service life of printed circuit electronic component |
| CN118261877B (en) * | 2024-03-29 | 2024-10-11 | 泰山科技学院 | Method for evaluating and maintaining service life of printed circuit electronic component |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112287940B (en) | Semantic segmentation method of attention mechanism based on deep learning | |
| CN112819910B (en) | Hyperspectral image reconstruction method based on double-ghost attention machine mechanism network | |
| CN116309429A (en) | Chip defect detection method based on deep learning | |
| CN113033570A (en) | Image semantic segmentation method for improving fusion of void volume and multilevel characteristic information | |
| CN113344110B (en) | Fuzzy image classification method based on super-resolution reconstruction | |
| CN113449691A (en) | Human shape recognition system and method based on non-local attention mechanism | |
| Zha et al. | Multifeature transformation and fusion-based ship detection with small targets and complex backgrounds | |
| CN112686830A (en) | Super-resolution method of single depth map based on image decomposition | |
| CN112257727A (en) | Feature image extraction method based on deep learning self-adaptive deformable convolution | |
| CN115936992A (en) | Garbage image super-resolution method and system of lightweight transform | |
| CN117095277A (en) | Edge-guided multi-attention RGBD underwater salient object detection method | |
| CN117392153B (en) | Pancreas segmentation method based on local compensation and multi-scale adaptive deformation | |
| CN110751271A (en) | Image traceability feature characterization method based on deep neural network | |
| Ren et al. | A lightweight object detection network in low-light conditions based on depthwise separable pyramid network and attention mechanism on embedded platforms | |
| Wang et al. | MeDERT: A metal surface defect detection model | |
| CN117830889A (en) | Video saliency detection method, system and storage medium based on optical flow graph feature fusion mechanism | |
| CN116823610A (en) | Deep learning-based underwater image super-resolution generation method and system | |
| CN115775214A (en) | Point cloud completion method and system based on multi-stage fractal combination | |
| CN116188363A (en) | Hydropower station insulator image detection system and method | |
| CN115115860A (en) | Image feature point detection matching network based on deep learning | |
| CN114332989A (en) | Face detection method and system of multitask cascade convolution neural network | |
| Wang et al. | Apple automatic classification method based on improved VGG11 | |
| Wu et al. | Tire defect detection based on low and high-level feature fusion | |
| CN115909045B (en) | Two-stage landslide map feature intelligent recognition method based on contrast learning | |
| Haque et al. | Multi scale object detection based on single shot multibox detector with feature fusion and inception network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |