CN113506286A - Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm - Google Patents

Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm Download PDF

Info

Publication number
CN113506286A
CN113506286A CN202110852074.8A CN202110852074A CN113506286A CN 113506286 A CN113506286 A CN 113506286A CN 202110852074 A CN202110852074 A CN 202110852074A CN 113506286 A CN113506286 A CN 113506286A
Authority
CN
China
Prior art keywords
chip
yolov5
image
algorithm
defect
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110852074.8A
Other languages
Chinese (zh)
Inventor
王树龙
陈栋梁
杜林�
王国生
刘钰
马兰
孙彪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202110852074.8A priority Critical patent/CN113506286A/en
Publication of CN113506286A publication Critical patent/CN113506286A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/0008Industrial image inspection checking presence/absence
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/95Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956Inspecting patterns on the surface of objects
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01RMEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/28Testing of electronic circuits, e.g. by signal tracer
    • G01R31/302Contactless testing
    • G01R31/308Contactless testing using non-ionising electromagnetic radiation, e.g. optical radiation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854Grading and classifying of flaws
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N21/88Investigating the presence of flaws or contamination
    • G01N21/8851Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details
    • G06T2207/20032Median filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30148Semiconductor; IC; Wafer

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Biochemistry (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Immunology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Signal Processing (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Electromagnetism (AREA)
  • Toxicology (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of artificial intelligence and microwave chips, and discloses a method for realizing microwave chip defect detection on a small sample data set based on a YOLOv5 algorithm, which is used for carrying out image preprocessing on a microwave chip serving as the data set, wherein the image preprocessing comprises gray level conversion, image rotation correction, background noise removal and other operations. And preprocessing the acquired image, classifying and labeling the defects, and dividing the data set subjected to defect classification and labeling into a training set and a test set. A YOLOv5 network structure is built, a training set labeled by defect classification is used for training a YOLOv5 network structure, network model weight parameters are adjusted through a back propagation algorithm, a test set is used for testing and analyzing a final network, and results show that the surface of a microwave chip subjected to amplification processing realizes automatic and rapid defect detection and positioning and achieves higher accuracy, recall rate and mAP value.

Description

Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm
Technical Field
The invention relates to the technical field of artificial intelligence and microwave chips, in particular to a method for realizing microwave chip defect detection based on a small sample data set of a YOLOv5 algorithm.
Background
As an important driving force of a new technological revolution and industrial change, artificial intelligence is accelerating to be deeply integrated with entity economy, so that not only are more application scenes found in the process of promoting industrial digitization, but also a unique rule for the development of the artificial intelligence industry is gradually formed. At present, five major trend characteristics of the development of the artificial intelligence industry in China comprise that the total data amount is in an explosive growth situation, a microwave computing chip becomes an important field, quantum machine learning becomes an important engine and the like.
With the rapid development of microwave chips in recent years, the preparation process of microwave chips is gradually improved, and the defects of microwave chips become the subject of prior research by researchers. The convolutional neural network algorithm has important application value and remarkable research significance in the fields of image recognition and classification, voice analysis and retrieval, target detection and monitoring and the like. The YOLO series algorithm is unique in its "simplicity" advantage and is widely popular in the industry. Convolutional neural networks were originally implemented for target detection by software programming methods, and researchers are gradually starting to use them for defect detection of microwave chips.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a method for realizing microwave chip defect detection based on a small sample data set of a YOLOv5 algorithm, which comprises the steps of preprocessing a microwave chip defect original data set, combining a YOLOv5 deep learning model and model performance testing, establishing research and application of key technologies for perfecting image processing, network training and defect identification, analyzing algorithm real-time performance and accuracy through tests, and showing that the result shows that the microwave chip surface subjected to amplification processing realizes automatic and rapid defect detection and positioning and achieves higher accuracy, recall rate and mAP value.
The technical idea of the invention is as follows: the method is characterized by integrating the mechanism of the production process of the chip and the reasons of defect generation, the basic concept and development of deep learning, the definition of a convolutional neural network, a common model and the hardware acceleration technology of the current deep learning. The method comprises the steps of carrying out image preprocessing on a microwave chip collected image serving as a data set, wherein the image preprocessing comprises gray level conversion, image rotation correction, background noise removal and the like. And preprocessing the acquired image, classifying and labeling the defects, and dividing the data set subjected to defect classification and labeling into a training set and a test set. The method comprises the steps of building a YOLOv5 network structure, training the YOLOv5 network structure by using a training set after defect classification and marking, adjusting network model weight parameters through a back propagation algorithm, testing and analyzing a final network by using a test set, replacing an NMS algorithm with a soft-NMS algorithm to improve prediction accuracy, optimizing the network structure finally, and verifying real-time performance and accuracy of a recognition algorithm.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme.
The microwave chip defect detection method based on the small sample data set of the YOLOv5 algorithm comprises the following steps:
step 1, carrying out amplification shooting on an area where a microwave chip defect is located by adopting a microscope to obtain a microwave chip defect original data set;
step 2, preprocessing the microwave chip defect original data set to obtain a preprocessed microwave chip defect data set;
step 3, classifying and labeling the preprocessed microwave chip defect data set into a training set and a testing set after defect classification and labeling;
step 4, building a YOLOv5 network structure, training the YOLOv5 network structure by adopting the training set, and adjusting weight parameters in the network by combining a back propagation algorithm and a random gradient descent algorithm to obtain a trained YOLOv5 network model;
and 5, testing the trained YOLOv5 network model by using the test set, and determining the type and the position of the microwave chip defect and the accuracy, the recall rate and the mAP value of the YOLOv5 network model.
The technical scheme of the invention has the characteristics and further improvements that:
(1) in step 2, the pretreatment comprises the following steps in sequence: gray conversion, image rotation correction and background noise removal operation.
(2) The step 2 specifically comprises the following substeps:
substep 2.1, grayscale conversion: the microwave chip defect original data set comprises an RGB three-channel color image, the RGB three channels are weighted and averaged to obtain a gray image with a gray value of Y, and the specific formula is as follows:
Y=0.3R+0.59G+0.11B
substep 2.2, rotation correction of the image: acquiring information of a straight line where a chip edge is located in a gray image by a Hough conversion method, determining a chip inclination angle according to an angle between the straight line where the chip edge is located and a reference horizontal line, and performing inclination correction on the gray image according to a chip rotation angle to obtain a chip image after rotation correction;
substep 2.3, removing background noise: and removing the background noise in the rotationally corrected chip image by adopting a median filtering algorithm.
(3) In substep 2.2, the specific solving process of the chip inclination angle is as follows:
l2=(Lmax-x)2+x2
wherein L is the original side length of the chip, LmaxThe side length of a square externally connected with the chip is shown, and x is the offset of the chip in the horizontal direction;
the offset of the chip in the horizontal direction takes a small non-negative value of the x solution, and the solution can be obtained as follows:
Figure BDA0003182729130000041
and the calculation formula of the chip inclination angle alpha is as follows:
Figure BDA0003182729130000042
(4) in sub-step 2.3, the median filtering algorithm specifically includes the following sub-steps:
substep 2.3.1, inputting the chip image matrix after rotation correction into a YOLOv5 network, and adding salt-pepper noise and Gaussian noise firstly and then;
substep 2.3.2, sorting the pixel gray values in the neighborhood of the denoising template, selecting the gray value intermediate value as the pixel value of the point, and writing the denoised image function as:
f′(i,j)=median(i,j)∈S{f(i,j)}
wherein S is a template region, f (i, j) represents the original gray value corresponding to the coordinate (i, j), and f' (i, j) represents the gray value of the whole block region by taking the middle value of all the gray values of the template region;
the randomly occurring salt and pepper noise is replaced by the pixel values of other points in the neighborhood, so that the salt and pepper noise is removed.
(5) In step 3, the defect classification label includes four categories of metallization layer defects, air bridge collapse, excess and scratch defects.
(6) In step 4, the YOLOv5 network structure comprises four parts, namely an input end, a Backbone, a tack and a Prediction.
Compared with the prior art, the invention has the beneficial effects that:
the invention adopts a brand-new YOLOv5 algorithm, applies the algorithm to the defect detection of the microwave chip, and integrates the production process of the chip, the reasons for the defect generation of the microwave chip and the most advanced new algorithm in deep learning. Aiming at the surface of the microwave chip after amplification treatment, automatic and rapid defect detection and positioning are realized, and higher accuracy, recall rate and mAP (mean Average precision) value are achieved.
Drawings
The invention is described in further detail below with reference to the figures and specific embodiments.
FIG. 1 is a gray scale image obtained by converting a microwave chip defect original image according to the present invention;
FIG. 2 is a schematic diagram of the calculation of the chip tilt angle used in the present invention;
FIG. 3 is a diagram of the noise processing process employed by the present invention; wherein, the image (a) is an original collected image; (b) the image after adding salt and pepper noise; (c) an image obtained by adding gaussian noise on the basis of (b); (d) the image is subjected to denoising processing by a median filtering algorithm;
FIG. 4 is a diagram illustrating a defect example of a microwave chip used in the present invention; wherein, the diagram (A) is a defect diagram of the metallization layer, and the diagram (B) is a defect-free diagram of the metallization layer; FIG. C is a diagram of a redundancy defect; graph (D) is a graph of the excess with no defects;
FIG. 5 is a diagram of the network architecture of YOLOv5 employed in the present invention;
FIG. 6 is a graph of evaluation indexes in the training of the YOLOv5 network model trained by the present invention; wherein, the graph (A) is a precision monitoring graph in the iterative process, and the graph (B) is a recall monitoring graph in the iterative process; and (C) is a mAP monitoring graph in an iterative process.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to examples, but it will be understood by those skilled in the art that the following examples are only illustrative of the present invention and should not be construed as limiting the scope of the present invention.
The microwave chip defect detection method based on the small sample data set of the YOLOv5 algorithm comprises the following steps:
step 1, carrying out amplification shooting on the area where the microwave chip defect is located by adopting a microscope to obtain a microwave chip defect original data set.
Specifically, the original data set of the microwave chip defects is obtained by manually using a microscope to perform local amplification shooting on the areas where the microwave chip defects are located by technicians, and finally acquired images have large differences according to different amplification factors and chip types during acquisition. The image storage mode is a TIFF format, the TIFF format is a flexible bitmap format, the color information of each pixel point is stored, the color change and the color fine transition can be expressed, the original information of a shot chip is reserved, and the size of each image is a 2560 x 1920 matrix.
Step 2, preprocessing the microwave chip defect original data set to obtain a preprocessed microwave chip defect data set; and dividing the preprocessed microwave chip defect data set into a training set and a testing set.
Specifically, step 2 comprises the following substeps:
substep 2.1, greyscale conversion
The original captured image comprises an RGB three-channel color image, and the image contains much color information, which is very unfavorable for the processing efficiency of the computer on the image. Because the quantity of the neural network parameters is positively correlated with the size of the input image, in order to accelerate the processing speed and accuracy of the neural network, the input image of the deep neural network needs to be subjected to gray scale conversion operation, the original color three-channel image is converted into a gray scale image, and the color information is converted into a gray scale image consisting of 0-255. Because the color information of the defects does not need to be considered when the defects on the surface of the chip are detected, and the classification of the defects is only related to the form and the position of the defects, the parameters can be effectively reduced by converting the images into the gray-scale images, and the training efficiency of the neural network model is accelerated. Referring to fig. 1, the present invention converts an image into a gray-scale image by weighting and averaging three channels of RGB to obtain a gray-scale value Y by Y being 0.3R +0.59G +0.11B, wherein the data sets are greatly different due to different amplification factors and different chip types.
Substep 2.2, image rotation correction
Referring to fig. 2, the present invention normalizes the gray-scale map using the Hough transform theory. Due to shooting errors or chip placement positions, images may be inclined to a certain extent in the bare chip image acquisition process, and therefore, necessary rotation correction and optimization work needs to be performed on the images before the images are input as training data for deep learning. The method is specifically used for firstly acquiring chip edge position information in a gray level image, and determining an image inclination angle according to an angle between a straight line where a chip edge is located and a reference horizontal line, so as to perform inclination correction. For the gray level image of the microwave chip, the longest straight line L where the outline of the chip is located is extracted by using Hough transformationmax. And solving the deviation x of the chip inclination angle alpha and the horizontal direction of the chip at the moment, converting the deviation into a mathematical formula to solve the problem, and correcting by using the calculated chip inclination angle alpha. The specific formula is as follows:
l2=(Lmax-x)2+x2
wherein L is the original side length of the chip, LmaxThe side length of a square is externally connected with the chip.
The offset of the chip in the horizontal direction takes a small non-negative value of the x solution, and the solution can be obtained as follows:
Figure BDA0003182729130000071
and the calculation formula of the chip inclination angle alpha is as follows:
Figure BDA0003182729130000072
and carrying out rotation correction operation on the gray level image according to the calculated chip inclination angle alpha to obtain a chip image after rotation correction.
Substep 2.3, removal of background noise
Due to the limitations of the image acquisition equipment, noise and artifacts will inevitably affect the quality of image acquisition. If the denoising step is absent, the noise will be retained in the gray scale image when the image is subjected to gray scale conversion. Noise in the image may have an effect on the results of the recognition algorithm, resulting in false detection. For microwave chip images, noise-containing images can cause errors in the detected defect data. Therefore, in the subsequent image preprocessing, a median filtering algorithm needs to be adopted to remove noise in the image. Referring to fig. 3, the present invention uses a median method to perform nonlinear filtering, and processes common salt and pepper noise in image acquisition, specifically as follows:
substep 2.3.1, inputting the chip image matrix after rotation correction into a YOLOv5 network, and adding salt-pepper noise and Gaussian noise firstly and then;
substep 2.3.2, sorting the pixel gray values in the neighborhood of the denoising template, selecting the gray value intermediate value as the pixel value of the point, and writing the denoised image function as:
f′(i,j)=median(i,j)∈S{f(i,j)}
wherein, S is the template region, f (i, j) represents the original gray value corresponding to the coordinate (i, j), and f' (i, j) represents taking the middle value of all gray values of the template region as the gray value of the whole region. The randomly occurring salt and pepper noise can be replaced by the pixel values of other points in the neighborhood, so that the final filtering effect has a good denoising effect on common salt and pepper noise in image acquisition.
Substep 2.3.3, Gaussian noise is a mean of 0 and variance of σ2Is an additive noise. Random-rand (640,640s) generates a normal distribution with a mean 0 and a variance of 1, then multiplies σ to adjust the variance to σ2And then the image is added on a picture.
And 3, classifying the preprocessed microwave chip defect data set into a training set and a testing set after defect classification and marking.
Specifically, in the classification and labeling of the microwave chip defects, the definition of the defects is the acceptance standard of the microwave bare chip under the real condition, and the defects can be specifically classified into four categories, namely, metallization layer defects, air bridge collapse, excess and scratch defects, which are as follows:
1. metallization layer defects
Defects of the metallization layer: the method comprises the following steps of discoloration of a metalized layer, particles on a metalized surface, damage of the metalized layer, swelling and foaming of the metalized layer and the like:
(1) discoloration of the metallized layer, particles of the metallized surface:
defects pertain to the discoloration of the metallization layer when the metallization layer of the chip develops a discoloration and when the defects are due to the corrosion of the metallization layer. When the metal surface particles are homogeneous metal particles, which are integrated with the metallization layer and have a size greater than 10um or form an overlap of the metal pattern, the defects belong to the particles of the metal surface.
(2) Damage of metallization layer
When the damage range exceeds the same electrical path (such as between a grid power supply path and a drain power supply path, between an upper electrode and a lower electrode of a capacitor), short circuit is possible between the paths, and the gap cannot be distinguished obviously under a microscope of 200 multiplied by the number of the paths, or when the damage causes the risk of falling of metal particles (the size is more than or equal to 10 mu m), or when the damage exceeds 50% of the width of an original metal strip and the damage depth exceeds 50% of the thickness of the metal layer, the defect belongs to the damage of a metallization layer under any condition.
(3) Bulging, blistering of metallised layers
The metallization layer of the chip may bulge, peel or blister, and when the adhesion of the metallization layer is poor, the defect may be the bulge or blister of the metallization layer.
2. Collapse of air bridge
The definition of air bridge collapse is: firstly, collapse of an air (coupling) bridge on a lower working metallization layer formed by a normal back process cannot be considered as mechanical damage, and when no visible gap exists between the air (coupling) bridge of a chip and the lower working metallization layer, defects are generated when the width of the undamaged metallization layer is smaller than 75% of the original metallization width due to holes, bubbles, particles, scratches and the like on the air (coupling) bridge metallization layer.
3. Surplus materials
The redundancy defects can be divided into redundancy of the medium area and redundancy of the active area according to the positions of the redundancy. When the surplus exists in the medium area, the size of the surplus is more than or equal to 10 mu m; or forming the lap joint of the working materials, wherein the gap can not be distinguished obviously under a microscope of 200 multiplied by the number; any one of which is considered defective.
When the excess is present in the active region, the excess is not movable and contacts the working material, and the gap is not clearly distinguished under a microscope of 500 × or more.
4. Scratch defect
The scribing defects can be classified into four types, namely cracks in the functional region, broken edges of the scribing groove, uneven sizes of the scribing groove and scribing groove defects according to the definition. When a crack occurs in the functional region of the chip, the defect type belongs to the crack in the functional region; when the size of the edge breakage caused by chip scribing exceeds 50% of the scribing groove, the defect belongs to the edge breakage of the scribing groove; when the defect causes that the size of the chip does not meet the detailed specification requirement of the product, the defect belongs to the uneven size of the scribing groove; when discoloration due to metal or dielectric layers occurs in the scribe line, the defect belongs to a defect in the scribe line.
Referring to fig. 4, the selected labeling software of the present invention is label, and according to the above defect definition rule, labels and defines the defect type in the collected microwave chip image set, wherein, in the diagram (a), the scratch damage of the metal layer exceeds 50% of the original metal strip width, and should be regarded as the damage of the metallization layer in the metallization layer defects; if the scratch is too small in the graph (B), the defect is not considered to be present. The presence of an excess in the media region and the formation of a material lap in graph (C) is considered an excess defect, and the absence of a defect in graph (D) where the defect location does not belong to the media region is not considered a defect.
The method for classifying and labeling the defects of the microwave chip comprises the following steps: opening the Anaconda Prompt first creates an environment named label for py35 individual environment for annotation data; adding a mirror image source to download and install Pyqt5 and labelimg; after the installation is finished, inputting a command labellimg, and opening a marking tool interface to mark data; opening a data set picture to be labeled, finding each type of defect according to a required document for labeling, and checking the labeling condition in real time in Anaconda Prompt during labeling to finally obtain the labeled data in the format of xml.
And 4, building a YOLOv5 network structure, training the YOLOv5 network structure by adopting the training set, and adjusting weight parameters in the network by combining a back propagation algorithm and a random gradient descent algorithm to obtain a trained YOLOv5 network model.
Specifically, as shown in fig. 5, the YOLOv5 network structure constructed by the present invention is divided into four parts, namely, an input end, a backhaul, a Neck, and a Prediction (Output), specifically as follows:
1. input end:
(a) and Mosaic data enhancement: 4 pictures are spliced in the modes of random scaling, random cutting and random arrangement, so that the detection data set is greatly enriched, and particularly, a plurality of small targets are added by random scaling, so that the robustness of the YOLOv5 network is better. The data of 4 pictures can be directly calculated, so that the Mini-batch size does not need to be large, and a good effect can be achieved by using one GPU.
(b) And (3) self-adaptive anchor frame calculation: in the YOLOv5 algorithm, for different data sets, an anchor frame with an initial set length and width is used in network training, a prediction frame is output by a network on the basis of the initial anchor frame, and then the prediction frame is compared with a real frame group, the difference between the two frames is calculated, and then reverse updating and network parameters are iterated. However, the YOLOv5 embeds this function in the code, and adaptively calculates the best anchor block value in different training sets each time training.
(c) Adaptive picture scaling: in a common target detection algorithm, different pictures are different in length and width, so that the common method is to uniformly scale the original picture to a standard size and then send the standard size to a detection network. For example, sizes such as 416 × 416, 640 × 640 are commonly used in the YOLOv5 algorithm.
2、Backbone:
(a) Focus structure: the original 640 × 3 image is input into a Focus structure, and is changed into a 320 × 12 feature map by a slicing operation, and is then subjected to a convolution operation of 32 convolution kernels, and finally changed into a 320 × 32 feature map.
(b) A Conv module: the Conv module in YOLOv5 contains 3 sections of the common convolution (Conv2d) and batch normalization BN (batch normalization) and activation function (SiLU), also called CBM modules.
Conv2d operation: the parameters calculated for the basic convolution contain (c _ in, c _ out, k, s), i.e. the number of input channels, the number of output channels, the convolution kernel size, the step size 4 parameters.
(c) CSP (Cross Stage partial) structure consisting of a plurality of CBS modules (Conv + BN + SiLU) plus residual unit ResUnit (consisting of two CBS modules plus a shorting operation) and a Concat operation. The learning ability of CNN can be enhanced, so that the accuracy is ensured while the weight is reduced; the calculation bottleneck is reduced; the memory cost is reduced.
(d) C3 Module CSP Bottleneck with 3 volumes, i.e. CSP structure containing 3 Conv modules
(e) And the SPP module is composed of a CBS module and 3 pooling layers with different sizes, and is input into a Conv layer for calculation and output after Concat short-circuit operation through the original output of the CBS module and the output of the 3 pooling layers.
3、Neck:
An FPN + PAN structure is used. In the nerck structure of Yolov4, a common convolution operation is adopted. In the Neck structure of YOLOv5, a CSP2 structure designed by referring to CSPnet is adopted to enhance the capability of network feature fusion.
(a) Concat operation: the function of the Concat layer is to splice two or more feature maps in the channel or num dimension.
(b) Upesample operation: the upsampling is done by an interpolation method.
4、Prediction:
A gIoU _ Loss penalty function is employed. IoU, the problem of non-overlapping boundary frames is solved.
The back propagation algorithm is used as the most common learning method in the convolutional neural network, and in order to achieve the purpose of mapping the data input and output relationship function by the network, the back propagation algorithm is required to calculate errors through a loss function and reversely transfer the errors to the weight matrix of each layer in the network to modify weight values. The purpose of back propagation is to train the convolutional neural network so that the feature representation inside the data can be learned, allowing it to learn arbitrary mapping of inputs to outputs.
In the specific training process, the forward propagation of data refers to the process of obtaining the output of the network finally by the steps of convolution, pooling, full connection and the like of input data through the network. The back propagation of the error refers to a process of calculating an error existing between a predicted value and a true value of the network through a loss function, then propagating the error back along the network into each layer, calculating a gradient through a chain derivation rule, and multiplying by a learning rate to change a weight value of the position. Before the network converges or reaches the expected effect, the forward propagation and backward propagation processes are continuously and alternately carried out, the weight matrix in the network is modified, and the network mapping capability is enhanced.
And 5, testing the trained YOLO network model by adopting the test set, determining the type and the position of the defect, and calculating each evaluation index (accuracy, recall rate and mAP) of the final YOLO network model, wherein AP is used for calculating the area under a certain type of P-R curve, and mAP is used for calculating the average value of the areas under all types of P-R curves.
Referring to fig. 6, the final test result of the neural network model trained by using a small sample based on YOLOv5 is shown. Based on a small sample dataset (1000 training and 200 testing), 3 different initialization weight parameter matrixes (yolov5s.pt, yolov5m.pt and yolov5l.pt) are adopted, iteration is carried out step by step, the hyper-parameters such as learning rate (learning rate), momentum value (momentum) and object Loss learning rate (object Loss gain) are optimized once, a neural network structure (the step length of a moving window and the pooling strategy of pooling layers in an SPP structure) is modified, and different Loss functions (IoU _ Loss, gIoU _ Loss and dIoU _ Loss) are tried to influence the result; wherein, mainly modifying the pooling strategy of the pooling layers in the SPP structure, and changing the largest pooling layer with the smallest size into the average pooling layer. And the pooling layer divides the input matrix into a plurality of 2-by-2 cells, outputs the maximum value of 4 values of each 2-by-2 cell as a result to be maximum pooling, and outputs the average value of 4 values as a result to be average pooling.
The evaluation index graphs in the training of the trained YOLOv5 network model of the invention are shown in FIG. 6; wherein, fig. 6(a) is the average value of various accuracy rates obtained by each iteration (i.e. inputting all training sets into the network for one training) in the training; FIG. 6(B) is a graph of the mean of the recall rates obtained for each iteration of the training (i.e., one training for all training sets input to the network); fig. 6(C) shows the resulting value of the ap per iteration of training (i.e., one training for all training sets input to the network).
According to the invention, soft-NMS is used for replacing original NMS, the NMS algorithm selects the prior frame with the highest confidence coefficient from the prior frames finally obtained in each batch, calculates the IoU values of the remaining prior frames and the prior frames, directly deletes the prior frame larger than a preset threshold value, stores the prior frame into a candidate frame sequence, removes the prior frame, selects the prior frame with the highest confidence coefficient, and repeatedly calculates until the prior frame sequence is empty. And the soft-NMS changes the step of deleting the prior boxes into the step of reducing the confidence coefficient of the prior boxes so as to achieve the aim of preventing the false deletion of the correct sample.
Although the present invention has been described in detail in this specification with reference to specific embodiments and illustrative embodiments, it will be apparent to those skilled in the art that modifications and improvements can be made thereto based on the present invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.

Claims (7)

1. The microwave chip defect detection method based on the small sample data set of the YOLOv5 algorithm is characterized by comprising the following steps of:
step 1, carrying out amplification shooting on an area where a microwave chip defect is located by adopting a microscope to obtain a microwave chip defect original data set;
step 2, preprocessing the microwave chip defect original data set to obtain a preprocessed microwave chip defect data set;
step 3, classifying and labeling the preprocessed microwave chip defect data set into a training set and a testing set after defect classification and labeling;
step 4, building a YOLOv5 network structure, training the YOLOv5 network structure by adopting the training set, and adjusting weight parameters in the network by combining a back propagation algorithm and a random gradient descent algorithm to obtain a trained YOLOv5 network model;
and 5, testing the trained YOLOv5 network model by using the test set, and determining the type and the position of the microwave chip defect and the accuracy, the recall rate and the mAP value of the YOLOv5 network model.
2. The method for realizing microwave chip defect detection based on the YOLOv5 algorithm small sample dataset of claim 1, wherein in step 2, the preprocessing comprises: gray conversion, image rotation correction and background noise removal operation.
3. The method for realizing microwave chip defect detection based on the YOLOv5 algorithm small sample dataset of claim 2, wherein the step 2 comprises the following substeps:
substep 2.1, grayscale conversion: the microwave chip defect original data set comprises an RGB three-channel color image, the RGB three channels are weighted and averaged to obtain a gray image with a gray value of Y, and the specific formula is as follows:
Y=0.3R+0.59G+0.11B
substep 2.2, rotation correction of the image: acquiring information of a straight line where a chip edge is located in a gray image by a Hough conversion method, determining a chip inclination angle according to an angle between the straight line where the chip edge is located and a reference horizontal line, and performing inclination correction on the gray image according to a chip rotation angle to obtain a chip image after rotation correction;
substep 2.3, removing background noise: and removing the background noise in the rotationally corrected chip image by adopting a median filtering algorithm.
4. The method for realizing microwave chip defect detection based on the YOLOv5 algorithm small sample dataset of claim 3, wherein in substep 2.2, the specific solving process of the chip tilt angle is as follows:
l2=(Lmax-x)2+x2
wherein L is the original side length of the chip, LmaxThe side length of a square externally connected with the chip is shown, and x is the offset of the chip in the horizontal direction;
the offset of the chip in the horizontal direction takes a small non-negative value of the x solution, and the solution can be obtained as follows:
Figure FDA0003182729120000021
and the calculation formula of the chip inclination angle alpha is as follows:
Figure FDA0003182729120000022
5. the method for detecting defects of a microwave chip based on a small sample dataset of YOLOv5 algorithm of claim 3, wherein in substep 2.3, the median filtering algorithm comprises the following substeps:
substep 2.3.1, inputting the chip image matrix after rotation correction into a YOLOv5 network, and adding salt-pepper noise and Gaussian noise firstly and then;
substep 2.3.2, sorting the pixel gray values in the neighborhood of the denoising template, selecting the gray value intermediate value as the pixel value of the point, and writing the denoised image function as:
f′(i,j)=median(i,j)∈S{f(i,j)}
wherein S is a template region, f (i, j) represents the original gray value corresponding to the coordinate (i, j), and f' (i, j) represents the gray value of the whole block region by taking the middle value of all the gray values of the template region;
the randomly occurring salt and pepper noise is replaced by the pixel values of other points in the neighborhood, so that the salt and pepper noise is removed.
6. The method for detecting defects of a microwave chip based on a small sample dataset of YOLOv5 algorithm in claim 1, wherein in step 3, the defect classification labels include four categories, namely metallization layer defects, air bridge collapse, excess and scratch defects.
7. The method for realizing microwave chip defect detection based on the YOLOv5 algorithm small sample dataset as claimed in claim 1, wherein in step 4, the YOLOv5 network structure comprises four parts of an input end, a backsbone, a tack and a Prediction.
CN202110852074.8A 2021-07-27 2021-07-27 Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm Pending CN113506286A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110852074.8A CN113506286A (en) 2021-07-27 2021-07-27 Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110852074.8A CN113506286A (en) 2021-07-27 2021-07-27 Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm

Publications (1)

Publication Number Publication Date
CN113506286A true CN113506286A (en) 2021-10-15

Family

ID=78014145

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110852074.8A Pending CN113506286A (en) 2021-07-27 2021-07-27 Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm

Country Status (1)

Country Link
CN (1) CN113506286A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067197A (en) * 2021-11-17 2022-02-18 河南大学 Pipeline defect identification and positioning method based on target detection and binocular vision
CN114120037A (en) * 2021-11-25 2022-03-01 中国农业科学院农业信息研究所 Germinated potato image recognition method based on improved yolov5 model
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术(天津)有限公司 Roof panel defect detection method and system
CN114678125A (en) * 2022-03-14 2022-06-28 浙江大学 Intelligent health management method for acute cardiovascular and cerebrovascular event risk group
CN114881987A (en) * 2022-05-23 2022-08-09 浙江理工大学 Improved YOLOv 5-based hot-pressing light guide plate defect visual detection method
CN116051473A (en) * 2022-12-21 2023-05-02 中国石油大学(北京) Weld defect identification model establishment method and device, and identification method and device
CN116109956A (en) * 2023-04-12 2023-05-12 安徽省空安信息技术有限公司 Unmanned aerial vehicle self-adaptive zooming high-precision target detection intelligent inspection method
CN117237330A (en) * 2023-10-19 2023-12-15 山东鑫润机电安装工程有限公司 Automatic bridge defect detection method based on machine vision
CN117809138A (en) * 2024-02-23 2024-04-02 中国电子科技集团公司第二十九研究所 Method and system for enhancing redundant detection image data set

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067197A (en) * 2021-11-17 2022-02-18 河南大学 Pipeline defect identification and positioning method based on target detection and binocular vision
CN114120037A (en) * 2021-11-25 2022-03-01 中国农业科学院农业信息研究所 Germinated potato image recognition method based on improved yolov5 model
CN114120037B (en) * 2021-11-25 2022-07-19 中国农业科学院农业信息研究所 Germinated potato image recognition method based on improved yolov5 model
CN114678125A (en) * 2022-03-14 2022-06-28 浙江大学 Intelligent health management method for acute cardiovascular and cerebrovascular event risk group
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术(天津)有限公司 Roof panel defect detection method and system
CN114881987B (en) * 2022-05-23 2024-06-07 浙江理工大学 Hot-pressing light guide plate defect visual detection method based on improvement YOLOv5
CN114881987A (en) * 2022-05-23 2022-08-09 浙江理工大学 Improved YOLOv 5-based hot-pressing light guide plate defect visual detection method
CN116051473A (en) * 2022-12-21 2023-05-02 中国石油大学(北京) Weld defect identification model establishment method and device, and identification method and device
CN116109956A (en) * 2023-04-12 2023-05-12 安徽省空安信息技术有限公司 Unmanned aerial vehicle self-adaptive zooming high-precision target detection intelligent inspection method
CN117237330A (en) * 2023-10-19 2023-12-15 山东鑫润机电安装工程有限公司 Automatic bridge defect detection method based on machine vision
CN117237330B (en) * 2023-10-19 2024-02-20 山东鑫润机电安装工程有限公司 Automatic bridge defect detection method based on machine vision
CN117809138A (en) * 2024-02-23 2024-04-02 中国电子科技集团公司第二十九研究所 Method and system for enhancing redundant detection image data set
CN117809138B (en) * 2024-02-23 2024-05-14 中国电子科技集团公司第二十九研究所 Method and system for enhancing redundant detection image data set

Similar Documents

Publication Publication Date Title
CN113506286A (en) Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN109376792B (en) Photovoltaic cell appearance defect classification method based on multi-channel residual error neural network
CN108918536B (en) Tire mold surface character defect detection method, device, equipment and storage medium
US7215829B2 (en) Method and system for object recognition using fractal map
US7257267B2 (en) Method and system for image segmentation
CN114627383B (en) Small sample defect detection method based on metric learning
CN113160192A (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN110544231B (en) Lithium battery electrode surface defect detection method based on background standardization and centralized compensation algorithm
CN113469951B (en) Hub defect detection method based on cascade region convolutional neural network
CN114782391A (en) Method, system and device for constructing defect detection model of few-sample industrial image
US20230222643A1 (en) Semantic deep learning and rule optimization for surface corrosion detection and evaluation
CN115880298A (en) Glass surface defect detection method and system based on unsupervised pre-training
CN113487563B (en) EL image-based self-adaptive detection method for hidden cracks of photovoltaic module
CN112883795B (en) Rapid and automatic table extraction method based on deep neural network
CN116630304B (en) Lithium battery mold processing detection method and system based on artificial intelligence
CN111833313A (en) Industrial product surface defect detection method and system based on deep active learning
CN112561875A (en) Photovoltaic cell panel coarse grid detection method based on artificial intelligence
CN115861190A (en) Comparison learning-based unsupervised defect detection method for photovoltaic module
CN115861307A (en) Fascia gun power supply drive plate welding fault detection method based on artificial intelligence
CN112381794B (en) Printing defect detection method based on deep convolution generation network
CN117522864B (en) European pine plate surface flaw detection method based on machine vision
CN113421223B (en) Industrial product surface defect detection method based on deep learning and Gaussian mixture
CN113538342A (en) Convolutional neural network-based quality detection method for coating of aluminum aerosol can
CN115830302B (en) Multi-scale feature extraction fusion power distribution network equipment positioning identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination