CN109785316B - Method for detecting apparent defects of chip - Google Patents
Method for detecting apparent defects of chip Download PDFInfo
- Publication number
- CN109785316B CN109785316B CN201910061382.1A CN201910061382A CN109785316B CN 109785316 B CN109785316 B CN 109785316B CN 201910061382 A CN201910061382 A CN 201910061382A CN 109785316 B CN109785316 B CN 109785316B
- Authority
- CN
- China
- Prior art keywords
- pin
- centroid
- vector
- image
- angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
A chip apparent defect detection method comprises shooting an SOP chip image by a color CCD camera, extracting the outline and the centroid of a chip circular mark and a pin through a series of pre-treatments, calculating the improved environment characteristic vector of each centroid of the circular mark and the pin, matching and positioning with a template image, calculating an affine transformation matrix, affine transforming the image into a template image coordinate system, and finally, judging whether the pins are lack of the improved environment vectors of the circular marks, extracting printing pixels and edges of the printing areas of the pins to judge whether printing information is defective, calculating minimum circumscribed rectangles of the outlines of the pins to judge whether the pins are upwarped, downwarped and inclined, and extracting oxidation and desoldering pixels of the pin areas to judge whether the pins are oxidized and desoldered. The method can automatically, quickly, conveniently and accurately judge the problems of pin loss, upwarp, downwarp, skew, desoldering and oxidation of the SOP chip, can also judge whether a printed information area is clear and complete, can effectively detect the appearance of the SOP chip product and reduce the labor intensity of workers.
Description
Technical Field
The invention discloses a chip apparent defect detection method, belongs to the field of machine vision detection, and relates to an integrated circuit package (SOP) chip apparent defect detection method based on an improved environment vector quick positioning technology.
Background
In the production process of chip devices, the apparent quality detection of chips is an indispensable link, the SOP chip apparent quality detection of the integrated circuit at present mainly adopts a manual visual detection method, the working strength is high, false detection is easy to cause, the detection speed and the detection precision are low, the detection efficiency is low, the requirement of enterprise scale production cannot be met, and the factors restrict the development of the integrated chip production industry in China to a greater extent.
At present, some inherent defects also limit the accuracy of SOP chip detection, such as detection speed and precision, to a certain extent.
Disclosure of Invention
In order to solve the problems, the invention provides a detection method which can automatically, quickly, conveniently and accurately judge the problems of pin loss, upwarping, downwarping, skewing, desoldering and oxidation of the SOP chip, can also judge whether a printed information area is clear and complete, and can effectively detect the apparent defects of the SOP chip product. The technical scheme is that the method for detecting the apparent defects of the chip comprises the following steps:
(1) A machine vision product defect detection hardware platform is built, a chip is detected, and a color image is obtained;
(2) Histogram equalizing the image obtained in step (1), and then median filtering the image;
(3) Graying the image obtained in the step (2), extracting a circular mark outline and a pin outline from the image, and calculating the centroids of the corresponding outlines of the pin and the circular mark;
(4) Calculating improved environment characteristic vectors of the round mark centroid and the pin outline centroid according to the centroid obtained in the step (3);
(5) The improved environment vector of the centroid obtained in the step (4) is matched with the improved environment vector of the centroid of the template image in a similar manner, wherein the template image circular mark and the pin environment vector are calculated in the steps (1) to (4) in advance;
(6) Calculating an affine transformation matrix through the matched centroid points obtained in the step (5), and then carrying out affine transformation on the color image obtained in the step (2) and the pin outline extracted in the step (3) to a reference image coordinate system;
(7) Comparing the improved environment characteristic vector of the circular mark centroid obtained in the step (4) with the improved environment characteristic vector of the circular mark centroid of the template image in a similar manner, wherein if the improved environment characteristic vector is similar to the improved environment characteristic vector of the circular mark centroid of the template image, the pin is absent, otherwise, the pin is absent;
(8) Graying the image obtained by the step (6), positioning and dividing the image into a printing information area, calculating the ratio of the number of font pixels and font edge pixels to the number of pixels in the whole printing information area, and judging whether the printing information is defective;
(9) HSV color model transformation the color image obtained after affine transformation in the step (6) is calculated, and the ratio of the number of the defective pixels of each pin area to the number of the pixels of the whole pin area is calculated, so that whether the pin is desoldered or oxidized is judged;
(10) And (4) calculating the length-width ratio and the horizontal inclination angle of the minimum external rectangle of the transformed pin outline obtained in the step (6), judging whether the pin is upwarped or downwarped according to the length-width ratio, and judging whether the pin is inclined according to the horizontal inclination angle.
Preferably: the step (4) comprises the following steps:
(4-1) calculating an improved environment feature vector of the centroid of the circular mark under the pin centroid set;
(4-2) calculating an improved environment characteristic vector of each pin centroid under the circle center mark and other pin centroid sets;
preferably: the step (5) comprises the following steps:
(5-1) calculating the similarity between the improved environment vector of the pin centroid of the input image and the improved environment vector of the pin centroid of the template image;
(5-2) matching the pin centroid of the input image of the most similar improved environment vector and the reference image with each other as a pair while directly matching the reference image circular flag centroid with the input image circular flag centroid;
preferably: the improved calculation of the environment feature vector comprises the following steps:
wherein the point c on the two-dimensional plane is in the point set P = { P = 1 ，p 2 ...p N The calculation steps of the improved environment feature vector Context are as follows:
step 1: calculating a unit vector of point c to point set P centroid
The calculation formula is as follows:
step 2: computing the vector of points c to all points on the set of points P
And vector
Angle of = { Angle 1 ，angle 2 ...angle N The included angle represents the range of 0-2 pi, and the calculation formula is as follows:
step 3: calculating the distances Dist = { Dist) from point c to all points on the set of points P 1 ，dist 2 ...dist N And the calculation formula is as follows:
step 4: initializing Angle histogram Angle [ L ] =0, L =1,. L, L and distance histogram DistHist [ L ] =0, L =1,. L, counting Angle and distance histograms through all Angle and distance sets Dist, wherein L represents the resolution of the statistics, and the statistics are as follows:
step 5: normalizing the angle histogram AngleHist and the distance histogram DistHist to obtain an angle environment feature vector AngleVector and a distance environment feature vector DistVector, wherein the environment feature vector Context consists of the angle environment feature vector AngleVector and the distance environment feature vector DistVector, and the normalization formula is as follows:
the detection method is described in more detail as follows:
(1) Constructing a machine vision product defect detection hardware platform, detecting a chip, and acquiring a 3-channel RGB image I 1 ；
(2) Histogram equalization of the image I obtained in step (1) 1 Then median filtering the image to obtain I 2 ；
(3) Graded image I 2 Obtaining an image I 3 In picture I 3 Extracting the MarkContour contour and the PinContour contour of the round mark k K = 1-K, calculating the centroids Mark Central and PinCentroid of the corresponding contours of the round mark and the pin k K = 1-K, K being the number of pins;
further, the step (3) comprises the following detailed steps:
(3-1) 3-channel RGB image I 2 Conversion into a grayscale image I 3 ；
(3-2) Global thresholding I 3 Obtaining an image I containing a chip pin area 4 The threshold thresh is generally greater than 200, and the global threshold segmentation formula is as follows:
wherein, I 3 (I, j) representing a grayscale image I 3 Pixel value at coordinate (I, j), I 4 (I, j) image I representing a global threshold segmentation 4 Thresh is an empirically chosen threshold at the pixel value of coordinate (i, j);
(3-3) extracting an image I by adopting a contour tracking algorithm 4 The pin profile of (a), setting a threshold to discard profiles having an area about 60% less than the normal pin area and 140% greater than the pin area to obtain PinContourer k K =1 to K, and the centroid PinCentroid thereof is calculated k K = 1-K, K is the number of pins, and the centroid calculation formula is as follows:
wherein (x) k ，y k ) Represents the kth contour PinCentroid k Centroid of, M k Indicates that the kth contour contains the total number of pixels, (x) km ，y km ) Coordinates representing the mth pixel on the contour;
(3-4) extracting a round mark outline MarkContour and the centroid of the round mark, namely the center MarkCentroid, by adopting Hough circle transformation;
(4) Centroids MarkCentroid and PinCentroid obtained in step (3) k K = 1-K, calculating MarkContext and PinContext improved environment feature vectors for the centroid of the circular mark and the centroid of the pin outline k ，k＝1～K；
Further, the step (4) comprises the following detailed steps:
(4-1) calculation of Mark Central centroid of circular Mark in set { PinCentroid 1 ...PinCentroid K The improved environment feature vector MarkContext under the condition of (1);
(4-2) calculating the centroid PinCentroid of each pin k K = 1-K in the set { MarkCentroid, pinCentroid 1 ...PinCentroid k-1 ，PinCentroid k+1 ...PinCentroid K Improved environment feature vector PinContext under } k ，k＝1～K；
(5) The improved environment vector MarkConte of the centroid obtained in the step (4)xt and PinContext k K = 1-K and improved environment vectors rMarkContext and rPinContext of the centroid of the template image k K = 1-K similarity match, and the pin centroid PinCentroid of the most similar improved environment vector k K = 1-K and the pin centroid rPinCentroid of the template image k K = 1-K are matched with each other to form a pair, and the circular mark centroids of the input image and the circular mark centroids of the template image are directly matched with each other, wherein the centroids of the circular marks and the pins of the template image and the environment vector are calculated in advance from the step (1) to the step (4);
the further step (5) comprises the following detailed steps:
(5-1) calculating an improved environment vector { PinContext) of the pin centroid obtained in the step (4) 1 ...PinContext K An improved environment vector of the centroid of the template image pin { rPinContext } 1 ...rPinContext K A similarity Sim between, the similarity formula is as follows:
where Sim (p, q) denotes PinContext p And rPinContext q The closer the Sim (p, q) is to 0, the higher the similarity, angleVector p And DistVector p Is a context feature vector PinContext p Angle and distance components of (1), angle vector q And DistVector q Is the context feature vector rPinContext q The angle and distance components of (a);
(5-2) Pin centroids of input images of the most similar improved Environment vectors k K = 1-K and reference image rPinCentroid k K = 1-K are matched with each other in pairs, and the reference image circular mark centroid MarkCentroid and the input image circular mark centroid rmankcentroid are directly matched at the same time;
(6) Calculating affine transformation matrix T by using random sampling consensus RANSANC algorithm through the matched centroid points obtained in the step (5), and then affine transforming the RGB image I obtained in the step (2) 2 And the pin contour PinContourer extracted by the step (3) k K = 1-K to the reference image coordinate system, obtaining an image I 5 And pin profile tPinContourer k ，k＝1～K；
(7) Comparing the improved environment feature vector MarkContext of the circular mark centroid obtained in the step (4) with the improved environment feature vector rMarkContext of the template image in a similar manner, if the improved environment feature vector MarkContext is approximate, indicating that the pin is absent, otherwise, the pin is absent, wherein a formula (4) is adopted for similarity calculation, and a judgment threshold value is selected by experience;
(8) Graying the image I transformed in step (6) 5 Positioning and segmenting a printing information region ROI, calculating the ratio of the number of font pixels and font edge pixels to the number of pixels of the whole printing information region ROI, and judging whether the printing information is defective or not;
further, the step (8) includes the following detailed steps:
(8-1) graying the image I converted in the step (6) 5 Positioning and segmenting a printing information region ROI;
(8-2) binarizing the printing information region ROI by a variance method between maximum classes, counting the number of printing font pixels, and judging that printing is incomplete if the ratio of the number to the total number of the printing information region ROI is lower than 80% of a normal value;
(8-3) extracting a printing information region ROI by an edge detection algorithm (Canny algorithm), counting the number of edge pixels of a printing font, and judging that printing is unclear if the ratio of the number to the total number of the ROI of the printing information region is lower than 80% of a normal value;
(9) HSV color model transformation the affine transformed RGB image I obtained in step (6) 5 Calculating the ratio of the number of the defective pixels in each pin area to the number of the pixels in the whole pin area, and judging whether the pin is desoldered and oxidized or not;
further, the step (9) includes the following detailed steps:
(9-1) HSV color model transformation the transformed RGB image I obtained in step (6) 5
(9-2) counting the number of pixels with hue H of 40-80 degrees and saturation S of more than 0.15 in each pin area, and judging pin desoldering if the number of the pixels accounts for more than 40% of the total number of the pins;
(9-3) counting the number of pixels with hue H of 70-130 degrees and lightness V of less than 0.97 in each pin area, and judging that the pins are oxidized if the number of the pixels accounts for more than 40 percent of the total number of the pins;
(10) Calculating the transformed pin contour tPinContourer obtained in the step (6) k K =1 to K aspect ratio of minimum bounding rectangle AspectRatio k K =1 to K and horizontal tilt angle horizon k K =1 to K, and is defined by the aspect ratio AspectRatio k K = 1-K, can judge whether the pin is upwarped or downwarped, and the horizontal inclination angle is HorizotalAngle k And K = 1-K can judge whether the pin is inclined or not.
Further, the step (10) includes the following detailed steps:
(10-1) calculating the transformed pin outline tPinContourer obtained in the step (6) k K = minimum circumscribed rectangle of 1-K;
(10-2) calculating the aspect ratio AspectRatio of the minimum bounding rectangle k K = 1-K, if the length-width ratio is more than or less than 20% of the normal value, the pin is upwarped or downwarped;
(10-3) calculating the horizontal inclination angle horizon angle of the minimum bounding rectangle k K = 1-K, indicating that the pin is skewed if the angle is greater or less than 20 ° from horizontal;
wherein the point c on the two-dimensional plane is in the point set P = { P = 1 ，p 2 ...p N The calculation steps of the improved environment feature vector Context are as follows:
step 1: calculating a unit vector of point c to point set P centroid
The calculation formula is as follows:
step 2: computing the vector of points c to all points on the set of points P
And vector
Angle = { Angle = [ ] 1 ，angle 2 ...angle N The included angle represents the range of 0-2 pi, and the calculation formula is as follows:
wherein
Representing a vector
The included angle between the coordinate system and the coordinate system,
representing a vector
Angle with polar coordinate n Representing a vector
And vector
The included angle of (A);
step 3: calculating Euclidean distances Dist = { Dist } from the point c to all the points on the point set P 1 ，dist 2 ...dist N And the calculation formula is as follows:
step 4: initializing Angle histogram anglehit [ L ] =0, L =1, ·, L and distance histogram disthit [ L ] =0, L =1,..., L, traversing all Angle and distance sets Angle and Dist statistical Angle and distance histograms, where L represents the resolution of the statistics as follows:
step 5: normalizing the angle histogram angleHist and the distance histogram DistHist to obtain an angle environment feature vector AngleVector and a distance environment feature vector DistVector, wherein the environment feature vector Context consists of the angle environment feature vector AngleVector and the distance environment feature vector DistVector, and the normalization formula is as follows:
drawings
FIG. 1 is a flow chart of an SOP chip apparent defect detection method based on an improved environment vector fast positioning technology;
FIG. 2 is a schematic diagram of a hardware structure of an SOP chip defect detection system according to the present invention;
FIG. 3 is a modified environment vector for an 8 pin SOP chip template image circle mark;
FIGS. 4-11 are improved environmental vectors for each pin of an 8-pin SOP chip template image;
FIG. 12 is an 8 pin SOP chip input image circle marker refinement environment vector;
FIGS. 13-20 are modified environmental vectors for each pin of an 8-pin SOP chip input image;
FIG. 21 is a result of centroid matching of an 8 pin SOP chip template image and an input image;
Detailed Description
The present invention is further illustrated by the following examples in conjunction with the figures and examples, and embodiments of the present invention include, but are not limited to, the following examples.
FIG. 2 is a schematic diagram of a hardware structure of the SOP chip defect detection system of the present invention. Before detection, an industrial color CCD camera and a light source are arranged above a target chip, the computer controls the camera to shoot and the conveyor belt to move, and the chip is shot when the chip moves to the position right below the camera. During detection, no object interference is required around the chip, the background of the conveyor belt is black, meanwhile, the illumination is sufficient, the circular mark is clearly illuminated, the brightness of the pins is relatively high, and the resolution of the camera is sufficiently clear.
Fig. 3 shows the improved environment vector of the circle mark of the 8-pin SOP chip template image, and fig. 4 to 11 show the improved environment feature vector of the 8-pin centroid of the template image. Similarly, fig. 12 shows the circle labeled modified environment vector of the input image, and fig. 13 to 20 show the 8 pin centroid modified environment feature vectors of the input image. The above is completed by step (4), and each graph represents the modified environment feature vector corresponding to the circle mark and the pin centroid, where the statistical resolution L of the modified environment feature vector is 9, the solid line distance represents the distance feature vector DistVector, and the dashed line angle represents the angle feature vector AngleHist. The method can intuitively obtain that the improved environment feature vectors of the template image and the corresponding pins of the input image are basically similar, and shows that the features extracted by the algorithm have invariance of scale, rotation and translation and have certain stability.
Fig. 21 is a result of centroid matching of an 8-pin SOP chip template image and an input image. The template image and the input image are both subjected to steps (1) to (5). In the step (3), the program needs to screen the pin outline extracted by the outline tracking algorithm, the screening method depends on the outline area, the area is similar to the pin area, and other parts are removed; meanwhile, the round mark extracted by the Hough circle is easily influenced by parameters, the round mark needs to be screened according to the size of the round radius, and a local area needs to be positioned to search for the round mark, so that the calculated amount is reduced, the noise is reduced, and the detection stability is improved. In step (4), the improved environment feature vectors corresponding to the pins have high similarity (the similarity is calculated by formula (4)), and it can be seen that the corresponding centroids of the template image and the input image are matched one by one.
As shown in fig. 1 to 21, a specific example of an SOP chip for detecting 8 pins based on the improved environment vector fast positioning technology includes the following specific steps:
(1) A hardware detection system as shown in figure 2 is set up, when the transmission belt is transported with the chip, the computer sends a shooting instruction to the camera to obtain a clear enough chip RGB three-channel input image I 1 。
(2) Histogram equalization of the image I obtained in step (1) 1 Then median filtering the image to obtain I 2 ；
(3) Graded image I 2 Obtaining an image I 3 Since the chip to be tested is an 8-pin chip, the number of chip pins K is 8, and in image I 3 Extracting the MarkContour contour and the PinContour contour of the round mark k K = 1-8, calculating the centroids MarkCentroid and PinCentroid of the corresponding outlines of the round mark and the pin k ，k＝1～8；
Further, the step (3) comprises the following detailed steps:
(3-1) 3-channel RGB image I 2 Conversion into a grayscale image I 3 ；
(3-2) Global thresholding I using equation (1) 3 Obtaining an image I containing a chip pin area 4 The threshold thresh is selected to be 220;
wherein, I 3 (I, j) representing a grayscale image I 3 Pixel value at coordinate (I, j), I 4 (I, j) image I representing a global threshold segmentation 4 Thresh is an empirically chosen threshold at the pixel value of coordinate (i, j);
(3-3) extracting an image I by adopting a contour tracking method 4 The area of the discarded pin profile is smaller than normalThe pin outline PinContourer is obtained by the pin area of 60% and the outline larger than 140% of the pin outline area k K =1 to 8, and the centroid PinCentroid thereof is calculated from the formula (2) and the formula (3) k ，k＝1～8；
Wherein (x) k ，y k ) Represents the kth contour PinCentroid k Centroid of, M k Indicating that the kth contour contains the total number of pixels, (x) km ，y km ) Coordinates representing the mth pixel on the contour;
(3-4) extracting a round mark outline MarkContour and the centroid of the round mark, namely the center MarkCentroid, by adopting Hough circle transformation;
(4) Centroids MarkCentroid and PinCentroid obtained in step (3) k K = 1-8, calculating improved environment feature vectors MarkContext and PinContext of the centroid of the circular mark and the centroid of the pin outline k The improved environment feature vector calculation results of k = 1-8,8 pin SOP chip detection example templates and input images are shown in fig. 3 to 11 and fig. 12 to 20;
further, the step (4) comprises the following detailed steps:
(4-1) calculation of Mark Central centroid of circular Mark in set { PinCentroid 1 ...PinCentroid 8 The improved environment feature vector MarkContext under the condition of (1);
(4-2) calculating the centroid PinCentroid of each pin k K =1 to 8 in the set { MarkCentroid, pinCentroid 1 ...PinCentroid k-1 ，PinCentroid k+1 ...PinCentroid 8 The modified context feature vector PinContext of k ，k＝1～8；
(5) Improved environment vectors MarkContext and PinContext of the centroid obtained in the step (4) k ，k＝1～8 improved environment vectors rMarkContext and rPinContext with template image centroid k K = 1-8 similarity match, improving the pin centroid PinCentroid of the most similar environment vector k K = 1-8 and the pin centroid rPinCentroid of the template image k K =1 to 8, which are matched with each other in pairs, and match the input image circular mark centroid MarkCentroid and the template image circular mark centroid rmankcentroid at the same time, the matching result of the 8-pin SOP chip detection example is shown in fig. 21;
the further step (5) comprises the following detailed steps:
(5-1) calculating the improved environment vector { PinContext) of the pin centroid obtained in the step (4) by using the formula (4) 1 ...PinContext 8 An improved environment vector of the centroid of the template image pin { rPinContext } 1 ...rPinContext 8 A similarity Sim between;
where Sim (p, q) denotes PinContext p And rPinContext q The closer to 0 Sim (p, q) is, the higher the similarity is, and the AngleVector p And DistVector p Is a context feature vector PinContext p Angle and distance components of (1), angle vector q And DistVector q Is a context feature vector rPinContext q The angle and distance components of;
(5-2) Pin centroid of input image of most similar improved environment vector k K = 1-8 and reference image rPinCentroid k K = 1-8 are matched with each other in pairs, and the reference image circular mark centroid rMarkCentroid and the input image circular mark centroid is directly matched at the same time;
(6) Calculating affine transformation matrix T by using random sampling consensus RANSANC method through the matching centroid points obtained in the step (5), and then affine transforming the RGB image I obtained in the step (2) 2 And the pin profile PinContourer extracted by the step (3) k K = 1-8 to the reference image coordinate system, I is obtained 5 And tPinContourer k ，k＝1～8；
(7) Comparing the improved environment feature vector MarkContext of the circular mark centroid obtained in the step (4) with the improved environment feature vector rMarkContext of the template image in a similar manner, if the improved environment feature vector MarkContext is approximate, indicating that the pin is absent, otherwise, indicating that the pin is absent, wherein a formula (4) similarity calculation formula is adopted, and a judgment threshold value is selected by experience;
(8) Graying the image I transformed by (6) 5 Positioning and segmenting a printing information region ROI, calculating the ratio of the number of font pixels and font edge pixels to the number of pixels of the whole printing information region ROI, and judging whether the printing information is defective or not;
further, the step (8) includes the following detailed steps:
(8-1) graying the image I converted in (6) 5 Positioning and segmenting a printing information region ROI;
(8-2) binarizing the printing information region ROI by a variance method between maximum classes, counting the number of printing font pixels, and judging that printing is incomplete if the ratio of the number to the total number of the printing information region ROI is lower than 80% of a normal value;
(8-3) extracting a printing information region ROI by using an edge detection method (Canny method), counting the number of edge pixels of a printing font, and judging that printing is unclear if the ratio of the number to the total number of the ROI of the printing information region is lower than 80% of a normal value;
(9) HSV color model transformation the affine transformed RGB image I obtained from (6) 5 Calculating the ratio of the number of the defective pixels of each pin area to the number of the pixels of the whole pin area, and judging whether the pin is desoldered and oxidized or not;
further, the step (9) includes the following detailed steps:
(9-1) HSV color model transformation the RGB image I transformed in (6) is obtained 5
(9-2) counting the number of pixels with hue H of 40-80 degrees and saturation S of more than 0.15 in each pin area, and determining pin desoldering if the number of the pixels accounts for more than 40% of the total number of the pins;
(9-3) counting the number of pixels with hue H of 70-130 degrees and lightness V of less than 0.97 in each pin area, and judging that the pins are oxidized if the number of the pixels accounts for more than 40 percent of the total number of the pins;
(10) Calculating the transformed pin contour tPinContourer obtained in the step (6) k K = aspect ratio of minimum bounding rectangle of 1 to 8 k K =1 to 8 and horizontal tilt angle horizon angle k K =1 to 8, and is defined by the aspect ratio AspectRatio k K = 1-8 can judge whether the pin is upwarping or downwarping, and the pin is horizontally inclined at a certain angle k And k = 1-8 can judge whether the pin is skewed or not.
Further, the step (10) includes the following detailed steps:
(10-1) calculating the transformed pin contour tPinContourer obtained in (6) k K = minimum circumscribed rectangle of 1 to 8;
(10-2) calculating the aspect ratio AspectRatio of the minimum bounding rectangle k K = 1-8, if the length-width ratio is more than or less than 20% of the normal value, the pin is warped upwards or downwards;
(10-3) calculating the horizontal inclination angle horizon angle of the minimum bounding rectangle k K = 1-8, indicating that the pin is skewed if the angle is greater or less than 20 ° from horizontal;
wherein the point c on the two-dimensional plane is in the point set P = { P = 1 ，p 2 ...p N The computation steps of the modified Context feature vector Context are as follows:
step 1: calculating a unit vector from point c to the centroid of the set of points P
The calculation formula is as follows:
step 2: computing vectors of point c to all points on the set of points P
And vector
Angle = { Angle = [ ] 1 ，angle 2 ...angle N The included angle represents the range of 0-2 pi, and the calculation formula is as follows:
wherein
Representing a vector
The included angle between the coordinate system and the coordinate system,
representing a vector
Angle of polar coordinates n Representing a vector
And vector
The included angle of (A);
step 3: calculating Euclidean distances Dist = { Dist) from point c to all points on point set P 1 ，dist 2 ...dist N The calculation formula is as follows:
step 4: initializing Angle histogram anglehit [ L ] =0, L =1, ·, L and distance histogram disthit [ L ] =0, L =1,..., L, traversing all Angle and distance sets Angle and Dist statistical Angle and distance histograms, where L represents the resolution of the statistics as follows:
step 5: normalizing the angle histogram angleHist and the distance histogram DistHist to obtain an angle environment feature vector AngleVector and a distance environment feature vector DistVector, wherein the environment feature vector Context consists of the angle environment feature vector AngleVector and the distance environment feature vector DistVector, and the normalization formula is as follows:
the present invention can be preferably realized according to the above-described embodiments. It should be noted that, based on the above structural design, in order to solve the same technical problems, even if some insubstantial changes or modifications are made in the present invention, the spirit of the adopted technical solution is the same as the present invention, and therefore, the technical solution should be within the scope of the present invention.
Claims (4)
1. A method for detecting apparent defects of a chip is characterized by comprising the following steps: the method comprises the following steps:
(1) A machine vision product defect detection hardware platform is built, a chip is detected, and a color image is obtained;
(2) Histogram equalizing the image obtained in step (1), and then median filtering the image;
(3) Graying the image obtained in the step (2), extracting a circular mark outline and a pin outline from the image, and calculating the centroids of the corresponding outlines of the pin and the circular mark;
(4) Calculating improved environment characteristic vectors of the round mark centroid and the pin outline centroid according to the centroid obtained in the step (3);
(5) The improved environment vector of the centroid obtained in the step (4) is matched with the improved environment vector of the centroid of the template image in a similar manner, wherein the template image circular mark and the pin environment vector are calculated in the steps (1) to (4) in advance;
(6) Calculating an affine transformation matrix through the matched centroid points obtained in the step (5), and then carrying out affine transformation on the color image obtained in the step (2) and the pin outline extracted in the step (3) to a reference image coordinate system;
(7) Comparing the improved environment characteristic vector of the circular mark centroid obtained in the step (4) with the improved environment characteristic vector of the circular mark centroid of the template image in a similar manner, wherein if the improved environment characteristic vector is similar to the improved environment characteristic vector of the circular mark centroid of the template image, the pin is absent, otherwise, the pin is absent;
(8) Graying the image obtained by the step (6), positioning and dividing the image into a printing information area, calculating the ratio of the number of font pixels and font edge pixels to the number of pixels in the whole printing information area, and judging whether the printing information is defective or not;
(9) HSV color model transformation the color image obtained after affine transformation in the step (6) is calculated, and the ratio of the number of the defective pixels of each pin area to the number of the pixels of the whole pin area is calculated, so that whether the pin is desoldered or oxidized is judged;
(10) And (4) calculating the length-width ratio and the horizontal inclination angle of the minimum external rectangle of the transformed pin outline obtained in the step (6), judging whether the pin is upwarped or downwarped according to the length-width ratio, and judging whether the pin is inclined according to the horizontal inclination angle.
2. The method of claim 1, wherein the method comprises the steps of: the step (4) comprises the following steps:
(4-1) calculating an improved environment feature vector of the centroid of the circular mark under the pin centroid set;
and (4-2) calculating the improved environment characteristic vector of each pin centroid under the circle center mark and other pin centroid sets.
3. The method of claim 1, wherein the method comprises: the step (5) comprises the following steps:
(5-1) calculating the similarity between the improved environment vector of the pin centroid of the input image and the improved environment vector of the pin centroid of the template image;
(5-2) matching the pin centroid of the input image and the reference image of the most similar improved environment vector to each other as a pair while directly matching the reference image circular flag centroid with the input image circular flag centroid.
4. The method of claim 1, wherein the method comprises: the improved environment feature vector calculation of the step (4) and the step (7) comprises the following steps:
wherein the point c on the two-dimensional plane is in the point set P = { P = { P = 1 ，p 2 ...p N The calculation steps of the improved environment feature vector Context are as follows:
step S001: calculating a unit vector of point c to point set P centroid
The calculation formula is as follows:
step S002: computing the vector of points c to all points on the set of points P
And vector
Angle = { Angle = [ ] 1 ，angle 2 ...angle N The included angle represents the range of 0-2 pi, and the calculation formula is as follows:
wherein
Representing a vector
The included angle between the coordinate system and the coordinate system,
representing a vector
Angle with polar coordinate n Representing a vector
And vector
The included angle of (c);
step S003: calculating Euclidean distances Dist = { Dist) from point c to all points on point set P 1 ，dist 2 ...dist N And the calculation formula is as follows:
step S004: initializing Angle histogram Angle [ L ] =0, L =1,. L, L and distance histogram DistHist [ L ] =0, L =1,. L, counting Angle and distance histograms through all Angle and distance sets Dist, wherein L represents the resolution of the statistics, and the statistics are as follows:
step S005: normalizing the angle histogram angleHist and the distance histogram DistHist to obtain an angle environment feature vector AngleVector and a distance environment feature vector DistVector, wherein the environment feature vector Context consists of the angle environment feature vector AngleVector and the distance environment feature vector DistVector, and the normalization formula is as follows:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910061382.1A CN109785316B (en) | 2019-01-22 | 2019-01-22 | Method for detecting apparent defects of chip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910061382.1A CN109785316B (en) | 2019-01-22 | 2019-01-22 | Method for detecting apparent defects of chip |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109785316A CN109785316A (en) | 2019-05-21 |
| CN109785316B true CN109785316B (en) | 2022-12-02 |
Family
ID=66501894
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910061382.1A Active CN109785316B (en) | 2019-01-22 | 2019-01-22 | Method for detecting apparent defects of chip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109785316B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334760B (en) * | 2019-07-01 | 2021-06-08 | 成都数之联科技有限公司 | Optical component damage detection method and system based on RESUnet |
| CN110930390B (en) * | 2019-11-22 | 2020-09-22 | 深圳市海芯微迅半导体有限公司 | Chip pin missing detection method based on semi-supervised deep learning |
| CN110992326B (en) * | 2019-11-27 | 2022-08-09 | 江苏理工学院 | QFN chip pin image rapid inclination correction method |
| CN111260621B (en) * | 2020-01-14 | 2023-05-09 | 湖南大学 | Positioning and identifying method for surface defects of printed circuit board |
| CN113447455B (en) * | 2020-02-25 | 2022-08-12 | 苏州鱼得水电气科技有限公司 | Fiber material detection system based on terahertz technology |
| CN111693546A (en) * | 2020-06-16 | 2020-09-22 | 湖南大学 | Defect detection system, method and image acquisition system |
| CN112419224B (en) * | 2020-07-17 | 2021-08-27 | 宁波智能装备研究院有限公司 | Spherical pin chip positioning method and system |
| CN114061480B (en) * | 2020-08-03 | 2024-04-05 | 上海飞机制造有限公司 | Method for detecting appearance of workpiece |
| CN112001917B (en) * | 2020-09-04 | 2024-08-09 | 南京大学金陵学院 | Circular perforated part form and position tolerance detection method based on machine vision |
| CN112348840A (en) * | 2020-10-19 | 2021-02-09 | 江苏师范大学 | QFP chip pin defect discrimination method based on pixel region growth |
| CN112345556B (en) * | 2020-11-23 | 2023-07-21 | 兰州大学 | Fault diagnosis system and method for integrated circuit |
| CN113289911B (en) * | 2021-06-04 | 2023-04-14 | 宁波舜宇仪器有限公司 | Method and system for detecting defects of polyhedral material |
| CN115266743B (en) * | 2022-05-17 | 2024-02-02 | 江苏汤谷智能科技有限公司 | System and method for evaluating chip quality under nondestructive testing |
| CN114882027B (en) * | 2022-07-08 | 2022-09-06 | 南通浩盛汽车科技有限公司 | Electronic equipment chip pin defect detection method and system |
| CN115308222B (en) * | 2022-07-11 | 2024-02-09 | 江苏汤谷智能科技有限公司 | System and method for identifying poor chip appearance based on machine vision |
| CN115439452B (en) * | 2022-09-13 | 2023-04-11 | 杭州凯智莆电子有限公司 | Capacitance product detection and evaluation system based on data analysis |
| CN115423814B (en) * | 2022-11-07 | 2023-03-24 | 江西兆驰半导体有限公司 | Chip origin positioning method and device, readable storage medium and electronic equipment |
| CN116051564B (en) * | 2023-04-02 | 2023-06-16 | 广东仁懋电子有限公司 | Chip packaging defect detection method and system |
| CN116698753B (en) * | 2023-07-25 | 2024-03-26 | 广州纳动半导体设备有限公司 | Mini-LED panel defect detection equipment and method based on machine vision |
| CN117161624B (en) * | 2023-10-18 | 2024-04-26 | 中设智能装备（苏州）有限公司 | Intelligent welding detection device and control system |
| CN118037628A (en) * | 2023-12-25 | 2024-05-14 | 中船重工鹏力（南京）智能装备系统有限公司 | IC pin defect detection method based on image processing |
| CN117686516B (en) * | 2024-01-29 | 2024-05-10 | 江苏优众微纳半导体科技有限公司 | Automatic chip appearance defect detection system based on machine vision |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226106A (en) * | 2013-03-15 | 2013-07-31 | 江南大学 | Machine vision based bearing dust cap quality monitoring system |
| WO2018086299A1 (en) * | 2016-11-11 | 2018-05-17 | 广东电网有限责任公司清远供电局 | Image processing-based insulator defect detection method and system |
| CN108982508A (en) * | 2018-05-23 | 2018-12-11 | 江苏农林职业技术学院 | A kind of plastic-sealed body IC chip defect inspection method based on feature templates matching and deep learning |
-
2019
- 2019-01-22 CN CN201910061382.1A patent/CN109785316B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226106A (en) * | 2013-03-15 | 2013-07-31 | 江南大学 | Machine vision based bearing dust cap quality monitoring system |
| WO2018086299A1 (en) * | 2016-11-11 | 2018-05-17 | 广东电网有限责任公司清远供电局 | Image processing-based insulator defect detection method and system |
| CN108982508A (en) * | 2018-05-23 | 2018-12-11 | 江苏农林职业技术学院 | A kind of plastic-sealed body IC chip defect inspection method based on feature templates matching and deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109785316A (en) | 2019-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109785316B (en) | Method for detecting apparent defects of chip | |
| CN111474184B (en) | AOI character defect detection method and device based on industrial machine vision | |
| CN106570511B (en) | Keyboard character defect online detection system and method based on computer vision | |
| CN111650220B (en) | Vision-based image-text defect detection method | |
| CN104680519B (en) | Seven-piece puzzle recognition methods based on profile and color | |
| CN104200210B (en) | A kind of registration number character dividing method based on component | |
| CN109724990A (en) | The quick positioning and detection method in coding region in a kind of packing box label | |
| Zhong et al. | Blob analyzation-based template matching algorithm for LED chip localization | |
| CN107767379B (en) | PCB label printing quality detection method | |
| CN111915704A (en) | Apple hierarchical identification method based on deep learning | |
| CN107389693B (en) | Automatic detection method for defects of printed matter based on machine vision | |
| CN113109348B (en) | Paddle image transfer printing defect identification method based on machine vision | |
| CN110866903A (en) | Ping-pong ball identification method based on Hough circle transformation technology | |
| CN110047063B (en) | Material drop detection method, device, equipment and storage medium | |
| CN106501272B (en) | Machine vision soldering tin positioning detection system | |
| CN112304957A (en) | Machine vision-based intelligent detection method and system for appearance defects | |
| CN112861654B (en) | Machine vision-based famous tea picking point position information acquisition method | |
| CN113205099A (en) | Visual detection system and method for material-removing characters on surface of nickel-plated metal | |
| CN106203439B (en) | The homing vector landing concept of unmanned plane based on marker multiple features fusion | |
| CN111476804A (en) | Method, device and equipment for efficiently segmenting carrier roller image and storage medium | |
| CN110674812A (en) | Civil license plate positioning and character segmentation method facing complex background | |
| CN114241438B (en) | Traffic signal lamp rapid and accurate identification method based on priori information | |
| CN110288040B (en) | Image similarity judging method and device based on topology verification | |
| CN112419225B (en) | SOP type chip detection method and system based on pin segmentation | |
| CN116523916B (en) | Product surface defect detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |