CN110992326A

CN110992326A - A fast tilt correction method for QFN chip pin image

Info

Publication number: CN110992326A
Application number: CN201911182587.1A
Authority: CN
Inventors: 巢渊; 周伟; 刘文汇; 唐寒冰; 李龑; 李兴成
Original assignee: Jiangsu University of Technology
Current assignee: Jiangsu University of Technology
Priority date: 2019-11-27
Filing date: 2019-11-27
Publication date: 2020-04-10
Anticipated expiration: 2039-11-27
Also published as: CN110992326B

Abstract

The invention discloses a quick tilt correction method for a pin image of a QFN chip, comprising the following steps: (1) collecting a pin image of a QFN chip, filtering and binarizing it; (2) extracting the center pad of the chip image by using a polygon approximation method contour; (3) propose an improved Harris corner detection algorithm to obtain contour vertices; (4) use the least squares method to fit a straight line to the farthest vertex, and use the straight line as the angle to identify the direction; (5) use the image centroid coordinates For the center of rotation, quickly correct the chip pin image and remove white edges. The correction method provides a certain theoretical basis for faster and more accurate correction of QFN chips, and improves the visual inspection efficiency of QFN package defects.

Description

QFN chip pin image rapid inclination correction method

Technical Field

The invention belongs to the field of image processing algorithm design, and provides a method for improving a Harris corner detection algorithm and designing a quick inclination correction method for a QFN chip pin image by combining a polygon approximation method.

Background

QFN (Quad Flat No-lead Package) is a leadless Package, which is square or rectangular, and uses the middle bonding pad at the bottom of the Package to conduct heat, and the periphery of the Package surrounding the middle bonding pad is provided with a conductive bonding pad for realizing electrical connection. The QFN chip has certain size error in the production process, so a margin is left when a tray bearing opening is manufactured, but the chip is inclined when being placed into the tray bearing opening, and the visual detection of the packaging quality of the chip is influenced.

Meanwhile, because the size of the chip is small, the manual detection and identification mode with low accuracy can not meet the requirements of current chip production, and the problem that the development of an effective technology capable of quickly and accurately correcting the chip image is in urgent need is solved at present. In the prior art, research on a QFN chip mainly focuses on the aspects of QFN chip structure improvement, manufacturing process, appearance detection and QFN chip defect detection, and documents and patents related to a QFN chip image inclination rapid correction method are not found at present, so that a method for rapidly correcting an inclination of a QFN chip pin image is designed, the defect of the existing research on the aspect is filled, and the method is particularly important for improving the QFN package defect visual detection efficiency.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides an improved Harris angular point detection algorithm, and a quick inclination correction method for a QFN chip pin image is designed by combining a polygon approximation method, so that the speed is higher and the efficiency is obviously improved compared with the traditional algorithm.

The technical scheme of the invention is as follows: a method for fast inclination correction of QFN chip pin images comprises the following steps:

1. preprocessing a chip pin image acquired by an industrial personal computer:

1.1 image filtering:

in order to remove noise and reduce image distortion, a 5 × 5 gaussian filter is convolved with an image to smooth the image and reduce the obvious noise effect on an edge detector, and in image processing, a two-dimensional gaussian function is often used for filtering, and the calculation formula is as follows:

wherein G (x, y) is a two-dimensional Gaussian function, (x, y) is a point coordinate, sigma is a standard deviation, A is a normalization coefficient, and the sum of different weights is one;

1.2. and (3) binarization processing:

carrying out binarization processing on the image by adopting a fixed threshold method, wherein a calculation formula is as follows:

wherein f (x, y) represents a distribution function of image pixel values, g (x, y) represents a distribution function of pixel values after threshold segmentation, and a fixed threshold T is 145;

2. extracting the target contour by using a polygon approximation method, which specifically comprises the following steps:

2.1 edge detection is carried out on the chip pin image:

adopting Canny operator edge detection to obtain edge contour information of a chip pin image;

2.2, extracting a target contour by using a polygon approximation method, namely the contour of a central bonding pad of a chip pin image:

extracting the outline of a central bonding pad of a chip pin image by a polygon approximation method, and filtering out the rest outline parts, thereby simplifying the subsequent image processing link; then, a point which is farthest away from the line segment is searched from the target contour, the point is added into the approximated new contour, namely, a triangle formed by connecting the three points is used as the contour; finally, starting from any one side of the triangle, repeating the previous step, adding the point with the farthest distance into the new contour, and continuously iterating until the output precision requirement is met;

3. an improved Harris corner detection algorithm is provided, and a target contour vertex is obtained, and the method specifically comprises the following steps:

3.1. determining a selection threshold, and acquiring a target contour corner:

in general, when the difference between the pixel gray levels of the two black and white images is less than 10% -15% of the maximum pixel gray level, the human eye is hard to distinguish, so the threshold N is selected to extract the corner points of the target contour of the image, and the formula is as follows:

N＝255×12％≈30

3.2. extracting corner points of the target contour:

extracting and storing all angular points, and sequentially reading three points M in the angular points_a-n、M_a、M_a+nThree points form a template consisting of three elements, where M is defined by the three points_aAs the template center, the subscript of the point represents the serial number of the corner point in all the corner points, all the corner points are traversed, and from the initial value, M is_aDetermining as a real-time operating point, and taking two points with the distance of n between the front and rear serial numbers, M_aRespectively form two edges with other two points in the template, and the included angle formed by the two edges is taken as M_aCorner point response value of point, from point M_aAnd point M_a-nDistance determination edge L₁Point M_aAnd point M_a+nIs determined by the distance of₂Point M_a-nAnd point M_a+nIs determined by the distance of₃Trilateral M can be obtained according to the cosine theorem_aThe angle of the point is calculated as follows:

judging whether to reserve the angular point or not through an included angle formed by two edges, namely an angular point response value, setting g as the angular point response value, and if the point M is the point M, judging whether to reserve the angular point or not_aIf the number is more than or equal to g, the corner points are saved as the needed corner points; otherwise, removing;

3.3. eliminating adjacent angular points, and keeping contour vertexes:

after the contour Corner corners are extracted, other corners may exist around the contour Corner corners, in order to eliminate the phenomenon, adjacent corners are removed, the remaining points are taken as contour Corner vertices, the image height is H, the image width is W, Corner (x, y) indicates whether a Corner exists at the image (x, y), when the Corner (x, y) is 1, (x, y) has a Corner, and mxm (m >1) is the size of a matrix with (x, y) as the center, then:

wherein, m is not less than x and not more than H, m is not less than y and not more than W, count refers to the corner point of the matrix range with (x, y) as the center, all the adjacent corner points of the matrix range with (x, y) as the center are removed, and the remaining corner points at the corners of the target contour are reserved as the vertexes, so as to facilitate the subsequent image correction processing;

4. fitting a straight line by a least square method:

performing linear fitting on two vertexes of the longest edge of the target contour by using a least square method to serve as angle identification directions of the chip pin image;

5. according to the quick correction chip of image centroid and remove the white edge, specifically include:

5.1. taking the centroid of the image as the rotation center:

in order to accurately correct the image, the center position of the image is determined by using a centroid method, and a centroid coordinate O point is used as the rotation center of the image;

5.2. and acquiring an image inclination angle and correcting the chip pin image.

Further, in the step 4, two vertex coordinates are respectively defined as p (x, y) and q (x, y), and the least square method fitting straight line calculation formula is as follows:

y＝ax+b

a and b are the slope and intercept, respectively, of the linear equation, then:

wherein

Respectively the mean values of the abscissa and ordinate of the vertexes p, q,

further, in step 5.1, the upper left corner of the image is set as the start point coordinate (0, 0), the lower right corner is set as the end point coordinate (m, n), and the centroid formula of the image is as follows:

wherein (x)₀，y₀) Is the centroid coordinate, m, n are the number of rows and columns of the image respectively (m, n are both integers greater than or equal to 2), f (x, y) is the gray value of the image at point (x, y).

Further, in step 5.2, the chip image inclination angle α can be obtained by the following calculation formula:

the horizontal offset x of the pin image can be obtained as follows:

x＝p_x-h_x

the pin image tilt angle α is:

wherein l is the vertical distance from the point p to the point O, OA is the extension line of the centroid coordinate to the positive direction of the x axis, and h (x, y) is the intersection coordinate of pq and OA.

The invention has the beneficial effects that:

the quick inclination correction method for the QFN chip pin image, disclosed by the invention, provides a certain theoretical basis for correcting the QFN chip more quickly and accurately, and improves the visual detection efficiency of QFN package defects.

Drawings

FIG. 1a is a QFN chip original image, FIG. 1b is a Gaussian filter image, and FIG. 1c is a binary image;

FIG. 2a is a Canny edge detection image, and FIG. 2b is a polygon approximation contour image;

FIG. 3 is a corner image acquired from a threshold;

FIG. 4 is a diagram illustrating the extraction of the vertices of the contour of an image object;

FIG. 5 is a straight line fitting method using least square method to select two vertices of the longest profile;

FIG. 6 is a schematic view of angular misalignment;

FIG. 7a is the image after the rotation correction, and FIG. 7b is the image after the white border is removed;

FIG. 8 is a schematic diagram of calibration labeling after angle detection by Hough transform algorithm;

FIG. 9 is a schematic diagram of calibration labeling after detecting an angle by a minimum external moment method;

FIG. 10 is a schematic diagram of calibration labeling after detecting angles based on an improved Harris corner detection algorithm;

FIG. 11 is a table of angle deviation data statistics for three algorithms;

FIG. 12 is a statistical table of operational time data for three algorithms;

FIG. 13 is a flow chart of a method for rapidly correcting the inclination of the pin image of the QFN chip.

Detailed Description

The following examples further illustrate the present invention but are not to be construed as limiting the invention. Modifications and substitutions to methods, procedures, or conditions of the invention may be made without departing from the spirit of the invention.

In order to improve the visual inspection efficiency of QFN package defects, the present embodiment discloses a method for rapidly tilting and correcting a QFN chip pin image, where fig. 1a is an original image, and fig. 1a is taken as an explanatory image of the present embodiment, and a specific correction process includes the following steps:

(1) preprocessing a chip pin image acquired by an industrial personal computer:

(1.1) image filtering:

to remove noise and reduce image distortion, a 5 × 5 gaussian filter is convolved with the image (see fig. 1b) to smooth the image and reduce the apparent noise effect on the edge detector. In image processing, a two-dimensional gaussian function is often used for filtering, and the calculation formula is as follows:

wherein G (x, y) is a two-dimensional Gaussian function, (x, y) is a point coordinate, sigma is a standard deviation, and A is a normalization coefficient, so that the sum of different weights is one.

(1.2) binarization processing:

the image is binarized by using a fixed threshold method (as shown in fig. 1c), and the calculation formula is as follows:

where f (x, y) represents a distribution function of image pixel values, g (x, y) represents a distribution function of pixel values after threshold segmentation, and a fixed threshold T is 145;

(2) extracting the target contour by using a polygon approximation method, which specifically comprises the following steps:

(2.1) carrying out edge detection on the chip pin image:

adopting Canny operator edge detection to obtain edge contour information of the chip pin image, as shown in FIG. 2 a;

(2.2) extracting a target contour by using a polygon approximation method, namely extracting a contour of a central bonding pad of a chip pin image:

the outline of the central bonding pad of the chip pin image is extracted by a polygon approximation method (as shown in fig. 2b), and the rest outline parts are filtered, so that the subsequent image processing is simplified. The polygon approximation method is to pick out two farthest points from the target contour and connect the points; then, a point which is farthest away from the line segment is searched from the target contour, the point is added into the approximated new contour, namely, a triangle formed by connecting the three points is used as the contour; and finally, starting from any one side of the triangle, repeating the previous step, adding the point with the farthest distance into the new contour, and continuously iterating until the output precision requirement is met.

(3) An improved Harris corner detection algorithm is provided, and a target contour vertex is obtained, and the method specifically comprises the following steps:

(3.1) determining a selection threshold, and acquiring a target contour corner:

in general, when the difference between the pixel gray levels of the two black and white images is less than 10% -15% of the maximum pixel gray level, the human eye is hard to distinguish, so a threshold N is selected to extract the corner points of the target contour of the image (as shown in fig. 3), and the formula is as follows:

N＝255×12％≈30

(3.2) extracting corner points of the target contour:

extracting and storing all angular points, and sequentially reading three points M in the angular points_a-n、M_a、M_a+nThree points form a template consisting of three elements, where M is defined by the three points_aAs the template center, the subscript of a point represents the number of the corner point among all the corner points. Traversing all corner points, starting from an initial value, M_aDetermining as a real-time operating point, and taking two points with the distance of n between the front and rear serial numbers, M_aRespectively form two edges with other two points in the template, and the included angle formed by the two edges is taken as M_aThe corner response value of the point. From point M_aAnd point M_a-nDistance determination edge L₁Point M_aAnd point M_a+nIs determined by the distance of₂Point M_a-nAnd point M_a+nIs determined by the distance of₃Trilateral M can be obtained according to the cosine theorem_aThe angle of the point is calculated as follows:

and judging whether the angular point is reserved or not according to an included angle formed by the two edges, namely an angular point response value. Let g be the angular response value, if point M_aIf the number is more than or equal to g, the corner points are saved as the needed corner points; otherwise, removing.

(3.3) eliminating adjacent angular points, and keeping contour vertexes:

after the corner points of the contour are extracted, other corner points may exist around the corner points, in order to eliminate the phenomenon, adjacent corner points are removed, and the remaining points are taken as the vertexes of the contour corners. Assuming that the image height is H, the image width is W, and Corner (x, y) indicates whether there is a Corner at the image (x, y), and when the Corner is 1 at the image (x, y), the Corner is at the image (x, y), and m × m (m >1) is the size of the matrix centered on the image (x, y), then:

wherein m is not less than x and not more than H, m is not less than y and not more than W, count refers to the corner points of the matrix range with (x, y) as the center, all the neighboring corner points of the matrix range with (x, y) as the center are removed, and the remaining corner points at the corners of the target contour are reserved as the vertexes (as shown in FIG. 4), so as to facilitate the subsequent image correction processing.

(4) Fitting a straight line by a least square method:

and performing linear fitting on two vertexes of the longest edge of the target contour by using a least square method to serve as the angle identification direction of the chip pin image, as shown in fig. 5.

(5) According to the quick correction chip of image centroid and remove the white edge, specifically include:

(5.1) taking the centroid of the image as the rotation center:

for accurately correcting the image, the center position of the image is determined by using a centroid method, and a centroid coordinate O point is used as the rotation center of the image.

(5.2) obtaining an image inclination angle, correcting a chip pin image:

when the QFN chip is packaged into a carrier tape, angular deviation which is difficult to distinguish by naked eyes exists, in order to calculate a deviation angle, an improved Harris corner detection algorithm is proposed, a polygonal approximation contour is combined, a least square method is utilized to perform straight line fitting on two vertexes of a longest side, at the moment, a chip inclination angle α and a chip horizontal direction deviation value x have a right triangle relationship, as shown in fig. 6, the chip is corrected according to an inclination angle α (as shown in fig. 7a), and white edges existing in a chip pin image after rotation correction are removed (as shown in fig. 7 b).

In the step (4), two vertex coordinates are respectively set as p (x, y) and q (x, y), and the least square method fitting straight line calculation formula is as follows:

y＝ax+b

wherein

Respectively the mean values of the abscissa and ordinate of the vertexes p, q,

in the step (5.1), the upper left corner of the image is set as the start point coordinate (0, 0), the lower right corner is set as the end point coordinate (m, n), and the centroid formula of the image is as follows:

In step (5.2), the chip image inclination angle α is obtained from the following calculation formula:

the horizontal offset x of the pin image can be obtained as follows:

x＝p_x-h_x

the pin image tilt angle α is:

Fig. 13 is a flowchart of a QFN chip pin image fast skew correction method, which is experimentally verified and compared as follows:

(1) the correction method disclosed in the embodiment is compared with the correction precision of the traditional Hough transformation and the minimum second-order moment method:

in the embodiment, the memory is 4GB, the processor is AMD A10-7300 radio R6, 10 computer computers 4C +6G @1.9GHz operating system, and the Visual Studio version is 2013. Selecting 10 different QFN chip pin images as experimental objects, running a program in the same environment, and comparing the correction method disclosed by the embodiment with the traditional Hough transformation and minimum external moment method. Fig. 8, 9, and 10 are schematic diagrams of calibration labeling after detecting angles by the three algorithms. And carrying out angle detection and verification by using the corrected chip central bonding pad, wherein the gray frame line is the minimum external rectangle of the pentagonal part of the chip obtained by different algorithms, and the black frame line is the ideal external rectangle of the manual marking of the pentagonal part of the chip. It can be easily seen that there is a certain angular deviation between fig. 8 and 9. The schematic diagram of the circumscribed rectangle marked after being corrected by the correction method disclosed by the embodiment is shown in fig. 10, and the marked gray frame line is approximately overlapped with the artificially marked black frame line, so that the inclination angle obtained by the correction method disclosed by the embodiment is more accurate. Fig. 11 lists the tilt angles of the chip detected by three algorithms.

(2) The correction method disclosed by the embodiment is compared with the correction time of the traditional Hough transformation and the minimum second-order moment method:

in order to detect the running time difference between the correction method disclosed by the embodiment and the traditional Hough transformation and minimum external moment method, the running times of 10 different QFN chip pin images are respectively compared in an experiment. Fig. 12 lists the run times of the three algorithms. Taking the graph number 5 as an example, the conventional Hough transform correction time is 357ms, the minimum external moment method correction time is 116ms, and the correction method disclosed in this embodiment completes the chip image correction process only in 18 ms. Compared with the traditional Hough transformation, the running time of the correction method disclosed by the embodiment is only 1/34, and the running time of the correction method disclosed by the embodiment is only 1/10 compared with the minimum external moment method. Therefore, the QFN chip image fast skew correction method proposed in this embodiment is not only highly accurate, but also significantly reduces the running time, and has higher computational efficiency.

In summary, the method for rapidly correcting the inclination of the pin image of the QFN chip provided by the embodiment of the present invention is faster and more efficient than the conventional algorithm, and can be used in the correction link of the QFN chip, so as to provide a clear and accurate image for the inclination correction of the chip and the detection of the appearance defect of the chip, and improve the visual detection efficiency of the QFN package defect.

The foregoing illustrates and describes the principles, general features, and advantages of the present invention. However, the above description is only an example of the present invention, the technical features of the present invention are not limited thereto, and any other embodiments that can be obtained by those skilled in the art without departing from the technical solution of the present invention should be covered by the claims of the present invention.

Claims

1. a method for quick tilt correction for QFN chip pin image, is characterized in that, comprises the following steps:

(1) Preprocess the chip pin image collected by the industrial computer:

1.1) Image filtering:

In order to remove noise and reduce image distortion, a 5×5 Gaussian filter is used to convolve the image. In image processing, a two-dimensional Gaussian function is used for filtering. The calculation formula is as follows:

Among them, G(x, y) is the two-dimensional Gaussian function, (x, y) is the point coordinate, σ is the standard deviation, and A is the normalization coefficient, so that the sum of different weights is one;

1.2) Binarization processing:

The fixed threshold method is used to binarize the image, and the calculation formula is as follows:

Among them, f(x, y) represents the distribution function of image pixel values, g(x, y) represents the pixel value distribution function after threshold segmentation, and the fixed threshold T=145;

(2) Extract the target contour by using the polygon approximation method, which specifically includes:

2.1) Edge detection on the chip pin image:

Using Canny operator edge detection, the edge contour information of the chip pin image is obtained;

2.2) Use the polygon approximation method to extract the target contour:

The contour of the center pad of the chip pin image is extracted by the polygon approximation method, and the remaining contour parts are filtered out. The polygon approximation method is to pick out the two farthest points from the target contour and connect them; then find an offline segment from the target contour. The farthest point is added to the new contour after approximation, that is, the triangle formed by connecting the three points is used as the contour; finally, any side of the triangle is selected to start, and the previous step is repeated to add the farthest point to the new contour. , and iterate continuously until the accuracy requirements of the output are met;

(3) An improved Harris corner detection algorithm is proposed to obtain the target contour vertices, including:

3.1) Determine the selection threshold and obtain the corner points of the target contour:

Under normal circumstances, when the difference between the pixel gray levels of two black and white images is less than 10% to 15% of the maximum pixel gray value, it is difficult for the human eye to distinguish. Therefore, the threshold N is selected to extract the corner points of the target contour of the image. The formula is as follows:

N=255×12%≈30

3.2) Extract the corner points of the target contour:

Extract and save all the corner points, read the three points _Man , M _a , and M _a+n in the corner points in order, the three points constitute a template composed of three elements, among which the three points take M _a as the template center , the subscript of the point represents the sequence number of the corner point in all the corner points, traverse all the corner points, starting from the initial value, M _a is determined as a real-time operation point, take the two points whose front and rear numbers are separated by n, and M _a are respectively the same as The other two points in the template form two sides, the angle formed by the two sides is used as the corner response value of point _{Ma, and the distance between point Ma and point M an} _determines edge L ₁ , point _Ma and point M _a ₊ The distance of _n determines the side L ₂ , the distance between the point _Man and the point M _a _+n determines the side L ₃ , and the angle of the point Ma can be obtained from the three sides according to the cosine law. The calculation formula is as follows:

The angle formed by the two sides, that is, the response value of the corner point, is used to determine whether to retain the corner point. Let g be the response value of the corner point. If the point M _a ≥ g, save it as the required corner point; otherwise, remove it;

3.3) Eliminate adjacent corners and retain contour vertices:

After the contour corner points are extracted, there may be other corner points around. To eliminate this phenomenon, the adjacent corner points are removed, and the remaining points are taken as the contour corner vertices. Set the image height as H and the image width as W, Corner(x, y) indicates whether there is a corner point at the image (x, y), when Corner(x, y)=1, there is a corner point at (x, y), m×m (m>1 ) is the size of the matrix centered at (x, y), then:

Among them, m≤x≤H, m≤y≤W, count refers to the corner points of the matrix range with (x, y) as the center, and all the adjacent corner points of the matrix range with (x, y) as the center are removed and reserved The remaining corner points at the corners of the target contour are used as vertices to facilitate subsequent image correction processing;

(4) Fitting a straight line by the least squares method:

Using the least squares method, the two vertices of the longest side of the target contour are fitted with a straight line as the angle identification direction of the chip pin image;

(5) Quickly correct the chip and remove the white edge according to the image centroid, including:

5.1) Take the image centroid as the rotation center:

In order to correct the image accurately, the centroid method is used to determine the center position of the image, and the centroid coordinate O point is used as the rotation center of the image;

5.2) Obtain the tilt angle of the image and correct the chip pin image.

2. a kind of method for quick tilt correction for QFN chip pin image as claimed in claim 1, is characterized in that, in described step 4, set two vertex coordinates to be respectively p(x, y), q( x, y), the least squares fitting straight line calculation formula is as follows:

y=ax+b

a and b are the slope and intercept of the straight line equation, respectively, then:

in,

are the mean values of the abscissa and ordinate of vertices p and q, respectively,

3. a kind of method for quick tilt correction for QFN chip pin image as claimed in claim 2, is characterized in that, in described step 5.1, the upper left corner of the image is set as starting point coordinates (0,0), the lower right corner Set as the end point coordinates (m, n), the formula of the image centroid is as follows:

Among them, (x ₀ , y ₀ ) is the centroid coordinate, m and n are the number of rows and columns of the image respectively (m, n are both integers greater than or equal to 2), f(x, y) is the image at the point ( gray value at x, y).

4. a kind of method for quick tilt correction for QFN chip pin image as claimed in claim 3, is characterized in that, in described step 5.2, chip image tilt angle α can be obtained by following calculation formula:

The offset value x of the pin image in the horizontal direction can be obtained:

x=p _x -h _x

Then the pin image tilt angle α is:

Among them, l is the vertical distance from point p to point O, OA is the extension of the centroid coordinate to the positive direction of the x-axis, and h(x, y) is the coordinate of the intersection of pq and OA.