CN110763700A - Method and equipment for detecting defects of semiconductor component - Google Patents

Method and equipment for detecting defects of semiconductor component Download PDF

Info

Publication number
CN110763700A
CN110763700A CN201911007150.4A CN201911007150A CN110763700A CN 110763700 A CN110763700 A CN 110763700A CN 201911007150 A CN201911007150 A CN 201911007150A CN 110763700 A CN110763700 A CN 110763700A
Authority
CN
China
Prior art keywords
detection
image
detection element
defect
semiconductor component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911007150.4A
Other languages
Chinese (zh)
Inventor
朱岱
崔凯阳
刘永军
汪震
胡锦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenxuan Intelligent Technology Nanjing Co Ltd
Original Assignee
Shenxuan Intelligent Technology Nanjing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenxuan Intelligent Technology Nanjing Co Ltd filed Critical Shenxuan Intelligent Technology Nanjing Co Ltd
Priority to CN201911007150.4A priority Critical patent/CN110763700A/en
Publication of CN110763700A publication Critical patent/CN110763700A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B07C5/361—Processing or control devices therefor, e.g. escort memory
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G01N2021/888—Marking defects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Biochemistry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the field of semiconductor component defect detection, in particular to a semiconductor component defect detection method and device. A method for detecting defects of a semiconductor component comprises the following steps: s1, taking materials; s2, 2D detection; s3, 3D detection; s4, judging whether the detection element is overturned, if so, executing a step S6; if not, go to step S5; s5, overturning: turning over the detection element to enable the reverse side of the detection element to face the 2D data acquisition position and the 3D data acquisition position, and executing the step S2; s6, sorting: and judging whether the element to be detected is NG or OK. The method is used for acquiring 2D and 3D image data of the detection element, identifying and judging the structural defects and the learning defects respectively, and further sorting the defects, can identify the defects with irregular and various morphological changes, and is a great progress in the field.

Description

Method and equipment for detecting defects of semiconductor component
Technical Field
The invention relates to the field of semiconductor component defect detection, in particular to a semiconductor component defect detection method and device.
Background
In the semiconductor component industry at present, the defect detection is mainly divided into two parts: the human eye identification of quality testing staff and the traditional contact defect detection mode.
The traditional manual quality inspection has low detection efficiency and high labor intensity, the detection result is greatly influenced by subjective factors such as the working experience, emotion and degree of seriousness of quality inspectors, and the conditions of false inspection and omission inspection cannot be effectively controlled; the contact defect detection method has a high recognition effect on defects with large defect areas and relatively constant changes of height, distance and the like, but a defect detection method for semiconductor components with irregular formation factors and various defect forms is lacked.
Patent document No. 201811433958.4 discloses a visual inspection apparatus including a feeding system for feeding, a discharging system for discharging, a rotary table for performing inspection work, and a scanning system for recording inspection data.
The disadvantages of the prior art include: (1) multiple structural defects cannot be detected simultaneously; (2) inability to detect learning deficiencies; (3) defects cannot be marked and sorted automatically. Nowadays, an application technology combining machine vision and an artificial intelligence technology is widely applied to the fields of target identification, defect detection and the like. The deep learning algorithm can learn the defect characteristics by marking the workpiece images with defects, and further deduce the positions and characteristics of the defects from the unmarked images.
Therefore, it is highly desirable to invent a semiconductor defect identifying method capable of effectively identifying defects with irregular and varied shapes.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, the present invention provides a method and apparatus for detecting defects of semiconductor devices, which can simultaneously identify structural defects and learning defects of semiconductor devices and realize effective sorting.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for detecting defects of a semiconductor component comprises the following steps:
s1, taking materials: placing the detection element at a correction station, and moving the detection element to a shooting jig after correcting the position;
s2, 2D detection: moving the shooting jig to a 2D data acquisition position for 2D image acquisition, and identifying structural defects by using a morphological algorithm;
s3, 3D detection: moving the shooting jig to a 3D data acquisition position for 3D image acquisition, and identifying a learning type defect by using a deep learning algorithm;
s4, judging whether the detection element is overturned, if so, executing a step S6; if not, go to step S5;
s5, overturning: turning over the detection element to enable the reverse side of the detection element to face the 2D data acquisition position and the 3D data acquisition position, and executing the step S2;
s6, sorting: judging whether the element to be detected is NG or OK;
if the number of the detection elements is NG, the detection elements are placed into an NG discharging bin;
if the OK is the result, the detection element is placed into an OK discharging bin.
In the method for detecting defects in a semiconductor device, in step S1, the position correction criterion is that the workpiece is subjected to position correction with reference to a certain right angle of the correction station.
In the method for detecting defects of a semiconductor device, in step S2, the morphological method is preferably:
s21, edge judgment: selecting an N multiplied by N pixel area by taking a pixel point (x, y) as a center, setting a threshold value T, drawing a cross by the center to obtain a neighborhood pixel of the pixel point (x, y), adding 1 to the gray level similarity number G when the difference D between the pixel value of the neighborhood pixel and the center pixel is less than the threshold value T, and determining the point as an edge point when the counted G meets the interval [ a, b ]; when the point is judged as an edge point, directly taking the pixel value of the point as output, and when the point is judged as an inner point, carrying out nonlinear median filtering;
s22, mask optimization: constructing a template with a gray value of 0, and obtaining a blank area in the middle of an image by adopting a maximum external area; then, constructing a template with a gray value of 1, obtaining a black remaining area in the middle of the image by adopting the maximum external area, and summing the black remaining area and the result of the previous processing to obtain an optimized mask image;
s23, morphological defect judgment: and constructing a disc with the radius of R to perform closing operation and opening operation on the image subjected to the edge judgment and mask optimization, and performing difference on the image and the original image to obtain defect distribution.
Preferably, in the method for detecting defects of a semiconductor device, in step S3, the deep learning algorithm includes:
s31, inputting the 3D image data of the detection element;
s32, selectively searching for a plurality of candidate areas which may contain defects in the 3D image;
s33, scaling the extracted candidate regions into a uniform size, and inputting the uniform size into a convolutional neural network for learning type defect feature extraction;
and S34, inputting the learning type defect characteristics extracted from each candidate region into a support vector machine for learning type defect classification.
Preferably, in the method for detecting defects of a semiconductor device, in the steps S2 and S3, the detection elements are photographed in multiple regions according to the sizes of the detection elements, and the entire 2D images or 3D images of the detection elements are obtained by stitching.
Preferably, in the method for detecting defects of a semiconductor device, the step S6 further includes:
and spraying an identification code and a defect mark on the detection element.
A semiconductor component defect detection device comprises a feeding bin, a correction device, a shooting jig, a movable rotary table, a 2D image acquisition system, a 3D image acquisition system, a pose adjusting device, an NG discharging bin, an OK discharging bin and an automatic robot;
the automatic robot is used for placing the detection element from the feeding bin to the correction device, transferring the detection element with the corrected position from the correction device to the shooting jig, and transferring the detection element to the NG discharging bin or the OK discharging bin;
the shooting jig is arranged on the movable turntable;
the movable turntable is used for rotating the shooting jig so that the detection elements can be respectively parked at the positions of the pose adjusting device, the 2D image acquisition system and the 3D image acquisition system;
the 2D image acquisition system, the 3D image acquisition system, the pose adjusting device and the automatic robot are respectively connected with a server.
Preferably, the semiconductor component defect detecting device, the 3D image capturing system includes a 3D measuring sensor.
Preferably, the semiconductor component defect inspection apparatus further includes a dimension measuring device for measuring a dimension of the inspection element.
The semiconductor component defect detection equipment preferably further comprises a code spraying device for spraying an identification code to the detection element and marking the defect position.
Compared with the prior art, the method and the equipment for detecting the defects of the semiconductor component, provided by the invention, have the advantages that 2D and 3D image data acquisition is carried out on the detection element, the structural defects and the learning defects are respectively identified and judged at the same time, and then sorting is carried out, so that the defects which are irregular and have various form changes can be identified, and the method and the equipment are a great progress in the field.
Drawings
FIG. 1 is a flow chart of a method for detecting defects of a semiconductor device according to the present invention;
FIG. 2 is a schematic diagram of a defect inspection apparatus for semiconductor devices provided by the present invention;
FIG. 3 is a schematic view of a sensing element in an embodiment of the present invention prior to correction in a corrective device;
FIG. 4 is a schematic view of a sensing element after being straightened in a straightening device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Please refer to fig. 1-2, the present invention provides a semiconductor device defect detecting apparatus, which includes a feeding bin 1, a correcting device 2, a shooting fixture 3, a mobile turntable 4, a 2D image collecting system 5, a 3D image collecting system 6, a pose adjusting device 7, an NG discharging bin 8, an OK discharging bin 9 and an automatic robot 10; preferably, the automatic robot 10 is a six-axis robot;
the automatic robot 10 is used for placing detection elements from the feeding bin 1 to the correction device 2, transferring the detection elements with corrected positions from the correction device 2 to the shooting jig 3, and transferring the detection elements to the upper NG discharging bin 8 or the OK discharging bin 9;
the shooting jig 3 is arranged on the movable turntable 4;
the movable turntable 4 is used for rotating the shooting jig 3, so that the detection elements can be respectively parked at the positions of the pose adjusting device 7, the 2D image acquisition system 5 and the 3D image acquisition system 6;
the 2D image acquisition system 5, the 3D image acquisition system 6, the pose adjusting device 7 and the automatic robot 10 are respectively connected with a server.
Correspondingly, in order to better apply the semiconductor component defect detection equipment provided by the invention, the invention also provides a semiconductor component defect detection method, which comprises the following steps:
s1, taking materials: placing the detection element at a correction station, and moving the detection element to the shooting jig 3 after correcting the position; preferably, with reference to fig. 3 and 4, the position correction is performed with respect to a workpiece with a right angle of the correcting device 2 of the correcting station as a reference;
s2, 2D detection: moving the shooting jig 3 to a 2D data acquisition position for 2D image acquisition, and identifying structural defects by using a morphological algorithm;
s3, 3D detection: moving the shooting jig 3 to a 3D data acquisition position for 3D image acquisition, and identifying a learning type defect by using a deep learning algorithm;
s4, judging whether the detection element is overturned, if so, executing a step S6; if not, go to step S5;
s5, overturning: turning over the detection element to enable the reverse side of the detection element to face the 2D data acquisition position and the 3D data acquisition position, and executing the step S2;
s6, sorting: judging whether the element to be detected is NG or OK;
if the number of the detection elements is NG, the detection elements are placed into an NG discharging bin 8;
if the OK is the result, the detection element is placed in an OK discharging bin 9.
Specifically, in the implementation process, a detection element enters the feeding bin 1, the automatic robot 10 takes out the detection element from the feeding bin 1 and puts the detection element into the correction device 2 for position correction, and after the correction, the automatic robot 10 moves the detection element onto the shooting jig 3; the movable turntable 4 rotates, the shooting jig 3 is moved to the position of the 2D image acquisition system for 2D image acquisition, and structural defect identification is carried out by using a morphological algorithm; then the movable turntable 4 rotates, the shooting jig 3 is moved to the position of the 3D image acquisition system 6 for 3D image acquisition, and a deep learning algorithm is used for recognizing learning type defects; then the movable turntable 4 rotates to move the shooting jig 3 to the pose adjusting device 7 to adjust the pose of the detection element, so that the reverse side of the detection element can face the 2D data acquisition position and the 3D data acquisition position; rotating the movable turntable 4 again, respectively collecting the 2D images and the 3D images again, judging whether a detection element is NG or OK according to the defect analysis data of the 2D image collection system 5 and the 3D image collection system 6, sorting, and if the detection element is NG, placing the detection element in the NG discharging bin 8; if the OK is OK, the materials are put into the OK discharging bin 9.
Preferably, in this embodiment, the 2D image capturing system 5 includes a light source, an industrial camera, and a lens; the 3D image acquisition system 6 comprises a 3D measurement sensor.
In step S2, the morphological method includes:
s21, edge judgment: selecting an N multiplied by N pixel area by taking a pixel point (x, y) as a center, setting a threshold value T, drawing a cross by the center to obtain a neighborhood pixel of the pixel point (x, y), adding 1 to the gray level similarity number G when the difference D between the pixel value of the neighborhood pixel and the center pixel is less than the threshold value T, and determining the point as an edge point when the counted G meets the interval [ a, b ]; when the point is judged as an edge point, directly taking the pixel value of the point as output, and when the point is judged as an inner point, carrying out nonlinear median filtering;
s22, mask optimization: constructing a template with a gray value of 0, and obtaining a blank area in the middle of an image by adopting a maximum external area; then, constructing a template with a gray value of 1, obtaining a black remaining area in the middle of the image by adopting the maximum external area, and summing the black remaining area and the result of the previous processing to obtain an optimized mask image;
s23, morphological defect judgment: the image after edge judgment and mask optimization separates the element layer from the substrate layer area in the image, the image quality problem inevitably causes burrs and grooves in the edge area, at the moment, a disc with the radius of R is constructed to perform closing operation and opening operation on the image after edge judgment and mask optimization, and the difference between the disc and the original image is obtained to obtain defect distribution.
Specifically, the closed operation is to sequentially perform expansion treatment and then corrosion treatment on the pictures; the opening operation is to perform corrosion treatment and then expansion treatment on the picture.
The corrosion treatment comprises the following steps: erosion can "narrow" the target area, which essentially causes the image boundaries to shrink, and can be used to eliminate small and meaningless objects. The formula is as follows:
Figure BDA0002243110990000051
the expansion treatment comprises the following steps: the expansion makes the range of the target area "large", and the background points contacting the target area are combined into the target object, so that the boundary of the target is expanded outwards. The effect is to fill some holes in the target area and to eliminate small particle noise contained in the target area. The formula is as follows:
Figure BDA0002243110990000052
preferably, in this embodiment, in step S3, the deep learning algorithm includes:
s31, inputting the 3D image data of the detection element; specifically, the 3D image is measured by the 3D measurement sensor;
s32, selectively searching for a plurality of candidate areas which may contain defects in the 3D image;
s33, scaling the extracted candidate regions into a uniform size, and inputting the uniform size into a convolutional neural network for learning type defect feature extraction;
and S34, inputting the learning type defect characteristics extracted from each candidate region into a support vector machine for learning type defect classification.
In step S32, feature extraction is performed using a 3D point cloud technique. The 3D point cloud is an array with points reflecting x, y and z three-dimensional coordinates fundamentally, and when the sizes and the sequence of the x axis and the y axis are fixed, only the z axis data can be simplified into one [ x, y and z ] axis data]*[y]Is used for the two-dimensional matrix of (1). The data can be converted into numerical values in the interval of 0-255 through data normalization, and then Z-axis data can be simplified into a group of two-dimensional tensors approximate to gray data. Meanwhile, the 3D measurement sensor is acquiring 3D data because of illumination or the likeOther physical factors, the 3D data at some points is not a valid value, and the 3D measurement sensor returns a set of definitions [ x ]]*[y]Two-dimensional matrix [ x ] of whether each height point in the two-dimensional matrix is a valid point or not1y1]Referred to as a mask matrix.
Specifically, the 3D point cloud obtained by the system may be subjected to data normalization, and then three sets of tensors with the same size are obtained: gray scale, height, mask. Before the neural network input is carried out, defect labeling is carried out on three groups of data of the same workpiece once. The system uses the three sets of tensor data to replace the input of a traditional network training three-channel (RGB) image, and uses the resnet as a network for extracting features.
The convolutional neural network comprises an input layer, a convolutional layer, a pooling layer, a full-link layer and an output layer.
Before the convolutional neural network identifies the learning-type defects, each defect in the learning-type defects needs to be learned, and the learning process is as follows:
and (4) conveying the defect picture into the input layer, learning by using a gradient descent algorithm, and standardizing the input characteristics of the convolutional neural network. Specifically, before inputting the learning data into the convolutional neural network, the input data needs to be normalized in the channel or time/frequency dimension, and if the input data is a pixel, the original pixel values distributed in [0,255] can also be normalized to the interval [0,1 ].
The function of the convolutional layer is to extract the characteristics of input data, the convolutional layer internally comprises a plurality of convolutional kernels, and each element forming the convolutional kernels corresponds to a weight coefficient and a deviation amount, and is similar to a neuron of a feedforward neural network. Each neuron in the convolution layer is connected to a plurality of neurons in a closely located region in the previous layer, the size of which depends on the size of the convolution kernel, known in the literature as the "receptive field", which has a meaning analogous to that of the visual cortical cells. When the convolution kernel works, the convolution kernel regularly sweeps the input characteristics, matrix element multiplication summation is carried out on the input characteristics in the receptive field, and deviation quantity is superposed:
Figure BDA0002243110990000061
the summation part in the formula is equivalent to solving the first-time cross correlation, and b is a deviation value; zlAnd Zl+1Represents the convolved input and output of the l +1 th layer, also called a signature; l isl+1Is Zl+1The feature map length and width are assumed to be the same; z (x, y) corresponds to the pixel of the feature map, K is the channel number of the feature map, f, s0And p is a convolutional layer parameter, corresponding to the convolutional kernel size, convolutional step size, and number of filling layers.
After the feature extraction is performed on the convolutional layer, the output feature map is transmitted to the pooling layer for feature selection and information filtering. The pooling layer contains a pre-set pooling function whose function is to replace the result of a single point in the feature map with the feature map statistics of its neighboring regions. The step of selecting the pooling area by the pooling layer is the same as the step of scanning the characteristic diagram by the convolution kernel, and the pooling size, the step length and the filling are controlled. Using LpPooling, which has the calculation formula:
Figure BDA0002243110990000072
step length s in the formula0Pixel (i, j) has the same meaning as the convolution layer, and p is a pre-specified parameter. When p is 1, LpPooling is averaged in a pooling zone, referred to as mean pooling; when p → ∞ LpPooling maximizes within a region, referred to as maximal pooling. Mean pooling and max pooling are pooling methods that have long been used in the design of convolutional neural networks, both of which preserve the background and texture information of the image at the expense of partial information or size of the feature map. And p is L when 22Pooling is also used in some jobs.
The fully-connected layer in the convolutional neural network is equivalent to the hidden layer in the traditional feedforward neural network. The fully-connected layer is located at the last part of the hidden layer of the convolutional neural network and only signals are transmitted to other fully-connected layers. The feature map loses spatial topology in the fully connected layer, is expanded into vectors and passes through the excitation function.
The convolutional neural network is usually a fully-connected layer upstream of the output layer, and thus has the same structure and operation principle as the output layer in the conventional feedforward neural network. For the image classification problem, the output layer outputs the classification label using a logistic function or a normalized exponential function. In an object recognition problem, the output layer may be designed to output the center coordinates, size, and classification of the object. In the image semantic segmentation, the output layer directly outputs the classification result of each pixel.
The learning process is that the learning of the learning type defects is carried out on a plurality of defect pictures sequentially through an input layer, a convolution layer, a pooling layer and a full-connection layer. The identification process is to input the 3D image into the input layer, output whether the learning type defect exists from the output layer and identify.
Preferably, in this embodiment, the correcting device further includes a size measuring device for measuring the size of the detecting element.
Correspondingly, in the steps S2 and S3, the detecting elements are multi-area photographed according to the sizes of the detecting elements, and the complete 2D image or 3D image of the detecting elements is obtained by stitching.
Specifically, the correcting device measures the size of the detection workpiece while correcting the position of the detection workpiece, and sends size data to the 2D image acquisition system and the 3D image system, the 2D/3D image acquisition system determines whether the size of the detection workpiece exceeds a predetermined size, and if so, the 2D/3D image acquisition system uses a multi-region shooting mode; if not, the normal shooting mode is used. And the normal shooting mode is to carry out integral simultaneous image acquisition on the detection workpiece and is single-sheet shooting or single-angle shooting.
Correspondingly, the 2D image acquisition system has a 2D multi-region shooting mode, and the 3D image acquisition system has a 3D multi-region shooting mode. In a further embodiment, the correction device sends the size data of the detected workpiece to a server, and the server sends mode switching instructions to the 2D/3D image acquisition system respectively according to the size of the size.
The 2D/3D multi-region photographing mode is that in the steps S2/S3, it further includes:
dividing the detection element into a plurality of small areas for shooting, wherein a certain common overlapping area is formed between two adjacent small area images shot;
and performing CAD template matching on each small-area image by using a surf algorithm, performing affine transformation on the image picture matched with the CAD according to the size and the position of the sub-area in the CAD, filling the images of different areas into the CAD image to form an overlapped part, and performing image fusion display on the overlapped part to obtain the 2D/3D image of the complete workpiece.
As a preferred embodiment. In this embodiment, the apparatus further includes a code spraying device, configured to spray an identification code to the detection element and mark the defect position.
Correspondingly, the step S6 further includes:
and spraying an identification code and a defect mark on the detection element.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.

Claims (10)

1. A method for detecting defects of a semiconductor component is characterized by comprising the following steps:
s1, taking materials: placing the detection element at a correction station, and moving the detection element to a shooting jig after correcting the position;
s2, 2D detection: moving the shooting jig to a 2D data acquisition position for 2D image acquisition, and identifying structural defects by using a morphological algorithm;
s3, 3D detection: moving the shooting jig to a 3D data acquisition position for 3D image acquisition, and identifying a learning type defect by using a deep learning algorithm;
s4, judging whether the detection element is overturned, if so, executing a step S6; if not, go to step S5;
s5, overturning: turning over the detection element to enable the reverse side of the detection element to face the 2D data acquisition position and the 3D data acquisition position, and executing the step S2;
s6, sorting: judging whether the element to be detected is NG or OK;
if the number of the detection elements is NG, the detection elements are placed into an NG discharging bin;
if the OK is the result, the detection element is placed into an OK discharging bin.
2. A method for detecting defects in a semiconductor component as claimed in claim 1, wherein in the step S1, the position correction is performed on the workpiece based on a right angle of the correction station.
3. A method for detecting defects in a semiconductor component as claimed in claim 1, wherein in the step S2, the morphological method is:
s21, edge judgment: selecting an N multiplied by N pixel area by taking a pixel point (x, y) as a center, setting a threshold value T, drawing a cross by the center to obtain a neighborhood pixel of the pixel point (x, y), adding 1 to the gray level similarity number G when the difference D between the pixel value of the neighborhood pixel and the center pixel is less than the threshold value T, and determining the point as an edge point when the counted G meets the interval [ a, b ]; when the point is judged as an edge point, directly taking the pixel value of the point as output, and when the point is judged as an inner point, carrying out nonlinear median filtering;
s22, mask optimization: constructing a template with a gray value of 0, and obtaining a blank area in the middle of an image by adopting a maximum external area; then, constructing a template with a gray value of 1, obtaining a black remaining area in the middle of the image by adopting the maximum external area, and summing the black remaining area and the result of the previous processing to obtain an optimized mask image;
s23, morphological defect judgment: and constructing a disc with the radius of R to perform closing operation and opening operation on the image subjected to the edge judgment and mask optimization, and performing difference on the image and the original image to obtain defect distribution.
4. A method for detecting defects of a semiconductor component as claimed in claim 1, wherein in the step S3, the deep learning algorithm comprises the steps of:
s31, inputting the 3D image data of the detection element;
s32, selectively searching for a plurality of candidate areas which may contain defects in the 3D image;
s33, scaling the extracted candidate regions into a uniform size, and inputting the uniform size into a convolutional neural network for learning type defect feature extraction;
and S34, inputting the learning type defect characteristics extracted from each candidate region into a support vector machine for learning type defect classification.
5. A method as claimed in claim 1, wherein in steps S2 and S3, the inspection elements are multi-area photographed according to their sizes, and the entire 2D or 3D images of the inspection elements are obtained by stitching.
6. The method for detecting defects in a semiconductor component as claimed in claim 1, wherein the step S6 further includes:
and spraying an identification code and a defect mark on the detection element.
7. The semiconductor component defect detection equipment is characterized by comprising a feeding bin, a correction device, a shooting jig, a movable rotary table, a 2D image acquisition system, a 3D image acquisition system, a pose adjusting device, an NG discharging bin, an OK discharging bin and an automatic robot;
the automatic robot is used for placing the detection element from the feeding bin to the correction device, transferring the detection element with the corrected position from the correction device to the shooting jig, and transferring the detection element to the NG discharging bin or the OK discharging bin;
the shooting jig is arranged on the movable turntable;
the movable turntable is used for rotating the shooting jig so that the detection elements can be respectively parked at the positions of the pose adjusting device, the 2D image acquisition system and the 3D image acquisition system;
the 2D image acquisition system, the 3D image acquisition system, the pose adjusting device and the automatic robot are respectively connected with a server.
8. The semiconductor component defect detection apparatus of claim 7, wherein the 3D image acquisition system comprises a 3D measurement sensor.
9. The apparatus for detecting defects in a semiconductor component as claimed in claim 7, wherein the correcting device further comprises a dimension measuring device for measuring a dimension of the detecting element.
10. The semiconductor component defect detection apparatus of claim 7, further comprising a code spraying device for spraying an identification code to the detection element and marking a defect position.
CN201911007150.4A 2019-10-22 2019-10-22 Method and equipment for detecting defects of semiconductor component Pending CN110763700A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911007150.4A CN110763700A (en) 2019-10-22 2019-10-22 Method and equipment for detecting defects of semiconductor component

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911007150.4A CN110763700A (en) 2019-10-22 2019-10-22 Method and equipment for detecting defects of semiconductor component

Publications (1)

Publication Number Publication Date
CN110763700A true CN110763700A (en) 2020-02-07

Family

ID=69331405

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911007150.4A Pending CN110763700A (en) 2019-10-22 2019-10-22 Method and equipment for detecting defects of semiconductor component

Country Status (1)

Country Link
CN (1) CN110763700A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402251A (en) * 2020-04-01 2020-07-10 苏州苏映视图像软件科技有限公司 Visual inspection method and system for 3D defect detection
CN111507998A (en) * 2020-04-20 2020-08-07 南京航空航天大学 Depth cascade-based multi-scale excitation mechanism tunnel surface defect segmentation method
CN111539933A (en) * 2020-04-22 2020-08-14 大连日佳电子有限公司 Direct-insertion element detection method and system
CN111736331A (en) * 2020-06-18 2020-10-02 湖南索莱智能科技有限公司 Method for judging horizontal and vertical directions of glass slide and device using method
CN111986178A (en) * 2020-08-21 2020-11-24 北京百度网讯科技有限公司 Product defect detection method and device, electronic equipment and storage medium
CN113421065A (en) * 2021-06-30 2021-09-21 安徽富信半导体科技有限公司 Semiconductor production intelligence letter sorting system based on thing networking
CN113546861A (en) * 2020-04-23 2021-10-26 海太半导体（无锡）有限公司 Collecting and classifying method for automatically identifying defective products

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1180472A (en) * 1996-03-14 1998-04-29 松下电器产业株式会社 Device for aligning printed board
CN103543168A (en) * 2013-10-12 2014-01-29 华南理工大学 Method and system for X ray detection on multilayer package substrate defects
CN105466951A (en) * 2014-09-12 2016-04-06 江苏明富自动化科技股份有限公司 Automatic optical detection apparatus and detection method thereof
CN106272426A (en) * 2016-09-12 2017-01-04 佛山市南海区广工大数控装备协同创新研究院 Solar battery sheet series welding anterior optic location and angle sensing device and detection method
CN108280837A (en) * 2018-01-25 2018-07-13 电子科技大学 BGA soldered balls contour extraction method in radioscopic image based on transformation
CN109001230A (en) * 2018-05-28 2018-12-14 中兵国铁（广东）科技有限公司 Welding point defect detection method based on machine vision
CN109087274A (en) * 2018-08-10 2018-12-25 哈尔滨工业大学 Electronic device defect inspection method and device based on multidimensional fusion and semantic segmentation
CN109447989A (en) * 2019-01-08 2019-03-08 哈尔滨理工大学 Defect detecting device and method based on motor copper bar burr growth district
CN109584215A (en) * 2018-11-10 2019-04-05 东莞理工学院 A kind of online vision detection system of circuit board
CN110186938A (en) * 2019-06-28 2019-08-30 笪萨科技(上海)有限公司 Two-sided defect analysis equipment and defects detection and analysis system
CN110231340A (en) * 2018-03-02 2019-09-13 由田新技股份有限公司 Equipment, method, deep learning method and the media of strengthening workpiece optical signature

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1180472A (en) * 1996-03-14 1998-04-29 松下电器产业株式会社 Device for aligning printed board
CN103543168A (en) * 2013-10-12 2014-01-29 华南理工大学 Method and system for X ray detection on multilayer package substrate defects
CN105466951A (en) * 2014-09-12 2016-04-06 江苏明富自动化科技股份有限公司 Automatic optical detection apparatus and detection method thereof
CN106272426A (en) * 2016-09-12 2017-01-04 佛山市南海区广工大数控装备协同创新研究院 Solar battery sheet series welding anterior optic location and angle sensing device and detection method
CN108280837A (en) * 2018-01-25 2018-07-13 电子科技大学 BGA soldered balls contour extraction method in radioscopic image based on transformation
CN110231340A (en) * 2018-03-02 2019-09-13 由田新技股份有限公司 Equipment, method, deep learning method and the media of strengthening workpiece optical signature
CN109001230A (en) * 2018-05-28 2018-12-14 中兵国铁（广东）科技有限公司 Welding point defect detection method based on machine vision
CN109087274A (en) * 2018-08-10 2018-12-25 哈尔滨工业大学 Electronic device defect inspection method and device based on multidimensional fusion and semantic segmentation
CN109584215A (en) * 2018-11-10 2019-04-05 东莞理工学院 A kind of online vision detection system of circuit board
CN109447989A (en) * 2019-01-08 2019-03-08 哈尔滨理工大学 Defect detecting device and method based on motor copper bar burr growth district
CN110186938A (en) * 2019-06-28 2019-08-30 笪萨科技(上海)有限公司 Two-sided defect analysis equipment and defects detection and analysis system

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402251A (en) * 2020-04-01 2020-07-10 苏州苏映视图像软件科技有限公司 Visual inspection method and system for 3D defect detection
CN111507998A (en) * 2020-04-20 2020-08-07 南京航空航天大学 Depth cascade-based multi-scale excitation mechanism tunnel surface defect segmentation method
CN111507998B (en) * 2020-04-20 2022-02-18 南京航空航天大学 Depth cascade-based multi-scale excitation mechanism tunnel surface defect segmentation method
CN111539933A (en) * 2020-04-22 2020-08-14 大连日佳电子有限公司 Direct-insertion element detection method and system
CN111539933B (en) * 2020-04-22 2023-06-06 大连日佳电子有限公司 Direct-insert element detection method and system
CN113546861A (en) * 2020-04-23 2021-10-26 海太半导体（无锡）有限公司 Collecting and classifying method for automatically identifying defective products
CN111736331A (en) * 2020-06-18 2020-10-02 湖南索莱智能科技有限公司 Method for judging horizontal and vertical directions of glass slide and device using method
CN111986178A (en) * 2020-08-21 2020-11-24 北京百度网讯科技有限公司 Product defect detection method and device, electronic equipment and storage medium
CN113421065A (en) * 2021-06-30 2021-09-21 安徽富信半导体科技有限公司 Semiconductor production intelligence letter sorting system based on thing networking
CN113421065B (en) * 2021-06-30 2024-01-30 安徽富信半导体科技有限公司 Intelligent sorting system for semiconductor production based on Internet of things

Similar Documents

Publication Publication Date Title
CN110763700A (en) Method and equipment for detecting defects of semiconductor component
CN110314854B (en) Workpiece detecting and sorting device and method based on visual robot
CN107230203B (en) Casting defect identification method based on human eye visual attention mechanism
CN104458755B (en) Multi-type material surface defect detection method based on machine vision
CN111507976B (en) Defect detection method and system based on multi-angle imaging
CN113592845A (en) Defect detection method and device for battery coating and storage medium
CN107966454A (en) A kind of end plug defect detecting device and detection method based on FPGA
CN108169236A (en) A kind of cracks of metal surface detection method of view-based access control model
CN109087286A (en) A kind of detection method and application based on Computer Image Processing and pattern-recognition
CN111127417B (en) Printing defect detection method based on SIFT feature matching and SSD algorithm improvement
Ali et al. A cascading fuzzy logic with image processing algorithm–based defect detection for automatic visual inspection of industrial cylindrical object’s surface
Yusof et al. Automated asphalt pavement crack detection and classification using deep convolution neural network
CN113393426A (en) Method for detecting surface defects of rolled steel plate
CN115816460A (en) Manipulator grabbing method based on deep learning target detection and image segmentation
CN108109154A (en) A kind of new positioning of workpiece and data capture method
CN117085969B (en) Artificial intelligence industrial vision detection method, device, equipment and storage medium
CN114612418A (en) Method, device and system for detecting surface defects of mouse shell and electronic equipment
CN114331961A (en) Method for defect detection of an object
TWI822968B (en) Color filter inspection device, inspection device, color filter inspection method, and inspection method
CN109975307A (en) Bearing surface defect detection system and detection method based on statistics projection training
CN113319013A (en) Apple intelligent sorting method based on machine vision
Kuo et al. Automated inspection of micro-defect recognition system for color filter
CN117169247A (en) Metal surface defect multi-dimensional detection method and system based on machine vision
CN114187269B (en) Rapid detection method for surface defect edge of small component
Shi et al. A fast workpiece detection method based on multi-feature fused SSD

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200207