JP3256372B2 - Image recognition device and image recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition apparatus and an image recognition method suitable for automatic mounting of a ball grid array type electronic device.

[0002]

2. Description of the Related Art As electrodes used in electronic devices such as integrated circuits, there are known electrodes in which a large number of small spherical solder balls are arranged in a two-dimensional array on the bottom surface of an electronic device.
A package using such electrodes is called a ball grid array (hereinafter, referred to as BGA). When mounting a BGA on a printed board, the BGA is arranged at a predetermined position on the printed board, and the printed board is reflow-heated. Thereby, the solder balls are melted, and the electronic device and the printed board are joined.

[0003]

By the way, when automatically mounting a BGA package on a printed circuit board, it is sufficient to hold the package with a holding portion of a surface mounter or the like and transport it to a predetermined position on the board. In some cases, an error occurs in the posture (position or angle) when the package is gripped. If a mounting operation is performed with an error, a product defect naturally occurs. If the error can be accurately grasped, it is possible to control the attitude of the package. However, conventionally, there is no technique capable of accurately grasping the attitude of the package. The present invention has been made in view of the above circumstances, and has as its object to provide an image recognition device and an image recognition method capable of accurately grasping the attitude of a BGA package.

[0004]

According to the first aspect of the present invention, when image data obtained by imaging the bottom surface of a package of a ball grid array type is supplied, a predetermined value is obtained. detection means for detecting a plurality of small-shaped image from among the image data having an area of less than a predetermined maximum value a to the minimum value or more, that of the plurality of small-shaped image
For each of them, calculate the intercepts of the four straight lines passing through the image.
Out of the plurality of small-grained images based on the section
Four-corner extraction means for extracting an image to be four corners from the four corners;
A first straight line connecting the two images in the image, and a
And a second straight line connecting two images other than the two images
Calculating means for calculating the size of each of the plurality of small-grain images.
And passes through a straight line having the same inclination as the first straight line.
Groups the images that are adjacent to each other
In each of the plurality of small-grain images,
Pass straight lines with the same slope as the two straight lines, and the straight lines
Grouping means for grouping images adjacent to each other,
For each of the groups, the direct
Calculate the line and calculate the average
And inclination calculating means for calculating the inclination of the package .

[0005] Further, in the configuration according to claim 2,
2. The image recognition apparatus according to claim 1, wherein the inclination is calculated.
Means for calculating a straight line passing through the images in the same group.
Or the square of the distance between an image in the same group and a straight line
Is characterized by calculating a straight line that minimizes the average of .

[0006] In the structure of the third aspect,
2. The image recognition device according to claim 1, wherein the plurality of small
Further comprising a center of gravity calculating means for calculating the center of gravity of the granular image,
The center-of-gravity calculating means is configured to calculate the package calculated by the inclination calculating means.
Longitudinal direction of the plurality of small-grained images based on the inclination of the cage
And obtaining the lateral direction, each of the plurality of small-grained images
With respect to the position of the image in the vertical direction and
Two positions symmetrical to the horizontal direction with respect to the two positions
Judge whether the image exists and whether it exists
For example, the center of gravity of the above four positions is obtained, and the average value of the obtained center of gravity is obtained.
Is calculated as the center of gravity of the package . Further, in the configuration described in claim 4, the button
Image the bottom surface of the
From the image data of the package,
A plurality of small-grained images having an area smaller than a predetermined maximum value
And for each of the plurality of small-grain images,
Calculate the intercepts of four types of straight lines passing through the image and calculate the intercept
Images at four corners from among the plurality of small-grained images based on
And a first straight line connecting two of the four corner images
And two images other than the two images in the four corner images
Calculating a second straight line connecting the plurality of small-grain images;
Each passes through a straight line with the same slope as the first straight line
And group the images whose straight lines are adjacent to each other.
For each of the plurality of small-grained images.
Then, a straight line having the same inclination as the second straight line is passed,
Grouping images where straight lines are adjacent to each other,
Calculate a straight line that passes through the images in the same group for the loop
And calculating the average value of the calculated inclinations of the straight lines in the package.
It is characterized in that it is calculated as the inclination of.

[0007]

According to the arrangement of the first, second and third aspects, when the detection means is supplied with image data obtained by imaging the bottom surface of the ball grid array type package, the detection means does not exceed a predetermined minimum value. the Te plurality of small-shaped image having an area of less than a predetermined maximum value detected from the said image data. Such an image corresponds to a solder ball of a ball grid array type package. Then, the inclination of the package is obtained from the inclination of the straight line connecting these images.

[0008] Also, the plurality of straight lines each are formed by connecting a plurality of images, from the average slope of the straight line determining the inclination of the package, can be determined more accurately the inclination of the package.

Further, in the configuration according to the fourth aspect ,
The bottom surface of the ball grid array type package is imaged, and from the image data of the package, a plurality of small-grain images having an area equal to or larger than a predetermined minimum value and equal to or smaller than a predetermined maximum value are detected. Then, the inclination of the package is obtained from the inclination of the straight line connecting these images.

[0010]

Embodiment A. Configuration examples below, with reference to the drawings, an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a camera, which picks up an image of a bottom surface of a package held by a holding portion or the like of a surface mounter and outputs the image as image data. This image data is supplied to the A / D converter 2
, And is stored in the image memory 3. Reference numeral 4 denotes a D / A converter, which converts image data in the image memory 3 into an analog signal and outputs the analog signal. A monitor TV 5 displays the image data supplied from the D / A converter 4. Reference numeral 6 denotes a CPU, which controls a surface mounter (not shown) via an external interface 7 based on a processing program described later.

B. Next, the operation of this embodiment will be described. First, when the package is gripped by the gripper of the surface mounter, the package is transported forward of the imaging lens of the camera 1. In the camera 1, the solder balls arranged on the bottom surface of the package are imaged, and the image data is stored in the image memory 3. Here, the image of each solder ball is a small-grained image arranged, for example, as a black dot shown in FIG. Next, C
In the PU 6, the program shown in FIGS.

First, in step SP1, the outline of each captured image is specified by a well-known boundary line tracking process. Details of such technology are disclosed in, for example,
27. The "image" here
The term includes not only an image of a solder ball but also an image of dust adhering to a package, a flaw of a package, and the like. Then, when the process proceeds to step SP2, the area and the center of gravity of each image are calculated. Next, in the processing of steps SP3 to SP7, an image of the solder ball is detected from each object. That is, since the allowable range of dimensions of the solder balls is known, is the image having an area within a permissible range (a predetermined area S ₁ or more and S ₂ or less) is considered to be an image of the solder balls.

Next, in steps SP8 to SP12, the following values are calculated for each solder ball image. y _L1 : Y-cut side of a straight line with a slope of “0.5” passing through the center of gravity of the solder ball image (hereinafter, referred to as solder ball point) x _L2 : X-cut side of a straight line with a slope of “2.0” passing through the solder ball point y _L3 : Y of a straight line with a slope “−0.5” passing through the solder ball point
Cut side x _L4 : Y of straight line with slope "-2.0" passing through the solder ball point
In FIG. 4, the value y for a certain solder ball point is shown.
_The manner in which _L1 , _xL2 , _yL3 , and _xL4 are determined is illustrated.

Further, in steps SP8 to SP12, the following solder ball points B0 to B7 are specified. B0: The value y _L3 solder balls points have minimum B1: values y _L1 solder balls points have minimum B2: The value y _L1 solder ball point is the maximum B3: value y _L3 solder ball point is the maximum B4: Solder ball point with minimum value x _L4 B5: Solder ball point with minimum value x _L2 B6: Solder ball point with maximum value x _L2 B7: Solder ball point with maximum value x _L4 Solder ball point B0 How B7 is determined depends on the arrangement of the solder balls. An example is shown in FIG.
It is shown in (b).

Next, when the process proceeds to step SP13,
The values (slope) h1, h2, v1, v2, ah and av are obtained based on the following equation. Where h1 is the solder ball point B
The slope of the straight line connecting 0 and B1, h2 is the solder ball point B2, B
The inclination of the straight line connecting the three points, v1 is the inclination of the straight line connecting the solder ball points B4 and B5, and v2 is the inclination of the straight line connecting the solder ball points B6 and B7. Ah is the average value of the slopes h1 and h2, and av is the average value of the slopes v1 and v2. In the following formula, the X coordinate and the Y coordinate of the solder ball points B0 to B7 are indicated by adding "x" or "y" after "B0" to "B7".

[0016]

H1 = (B1y-B0y) / (B1x-B0x) h2 = (B3y-B2y) / (B3x-B2x) v1 = (B5x-B4x) / (B5y-B4y) v2 = (B7x-B6x) / (B7y-B6y) ah = (h1 + h2) / 2 av = (v1 + v2) / 2

Next, when the process proceeds to step SP14,
Each solder ball point (the coordinates (x _N, and y _N)), the value based on the following formula bh, bv is calculated.

[Number 2] _{bh = y N -ah · x N} bv = x N -av · y N

That is, the value bh is the Y-cut side of the straight line having the slope ah passing through the solder ball point, and the value bv is the X-cut side of the straight line having the slope av passing the solder ball point. In FIG. 6, for a certain solder ball point, the value b
h and bv are shown on the coordinates. Next, the processing proceeds to step SP1.
Proceeding to 5, the solder ball points are sorted based on the X-cut side bv. Next, in the processing of steps SP16 to SP21, the X cut sides bv are close to each other (the X cut sides bv
Are within a predetermined range). The solder ball points are grouped. For example, the solder balls points aligned along a straight line L ₅ in FIG. 6, it will belong to the same group.

Next, when the process proceeds to step SP22,
The solder ball points are sorted based on the Y-cut side bh. Then, in the processing of steps SP23 to SP28, the solder ball points adjacent to each other at the Y-cut side bh are grouped. FIG. 7 shows how the solder ball points are grouped based on the Y-cut side bh. Next, step S
In P29 and P30, a straight line is obtained for each group based on the least squares method. That is, the straight line is determined so that the average of the square values of the distances between the solder ball points in the same group and the straight line is minimized.

Here, the equation of the above-mentioned straight line is expressed by the X-intersection bv
For a group based on,

[Number 3] _{x = va i · y-vb} i ( but i = 1,2, ····, k, k is the number of groups based on the X Setsuhen bv)

For the group based on the Y-intersection bh, the equation of the straight line is as follows.

Equation 4] _{y = ha j · x-hb} j ( where j = 1,2, ····, m, m is the number of groups based on Y Setsuhen bh)

In FIG. 8, the slopes va _i and ha _j
Is illustrated. Next, when the process proceeds to step SP31, the average value a of the slopes va _i and ha _j is obtained based on the following equation. Then, this average value a is used as the inclination of the BGA package.

(Equation 5)

Next, in the processing of steps SP32 to SP35, the center of gravity of the solder ball point is obtained as follows. First, in FIG.
(I, j), and the following processing is performed on the upper left position B (1, 1). That is, B (1,1), a position B (k, 1) vertically symmetric with respect to B (1,1), and B (1, m ) And B (k, m) (in this case, there are four solder balls as can be seen from the upper left of FIG. 9), the above four solder balls are required to determine the center of gravity of the BGA. The X and Y coordinates of the center of gravity of the ball are integrated.

Similarly, all the solder balls B belonging to the range of i = “1−m / 2 + 1” and j = “1−k / 2 + 1”
For (i, j), B (i, j), a position B (k-i + 1, j) that is vertically symmetric with respect to B (i, j), and a horizontal position between these two positions. B (i, m-j + 1) and B (k-i +
If the solder ball exists at (1, m−j + 1), the X and Y coordinates of the center of gravity of the four solder balls are integrated.

However, when m is an odd number, there may be two symmetrical positions (see the upper right and the right center in FIG. 9). In this case, the center of gravity is integrated for the two. . Also,
Even when k is an odd number, there are two symmetrical positions (see the lower left and lower center in FIG. 9). Further, in this case, if m is also an odd number, there is one symmetric position (see the lower right of FIG. 9). In such a case, the above processing is performed only on two or one solder ball.

When the processing is completed for all the solder balls by the above processing, an average value is obtained by dividing each integrated result of the X and Y coordinates by the integrated number, and this is set as the BGA center of gravity position. . Next, the process proceeds to step SP
In step 36, the obtained inclinations and the center of gravity of all the solder ball points are output via the external interface 7. The processing in the image recognition device of the present embodiment ends here.
In an external surface mounter or the like, the attitude of the package is corrected based on the output inclination and the center of gravity.

C. Modifications The present invention is not limited to the above-described embodiments, and various modifications are possible, for example, as follows. The BGA package to be subjected to the posture detection is not limited to the above-described one. For example, as shown in FIG. 10, the present invention can be applied to a package in which solder balls are arranged in various patterns. Further, it goes without saying that the pitch of the solder balls may be different in the vertical and horizontal directions.

[0028] In the above embodiment has been outputs an average value a gradient va _i and ha _j as the slope of the BGA package, and outputs any one of tilt va _i and ha _j as the slope of the BGA package Is also good.

In the processing of steps SP15 to SP28, the solder ball points are grouped, but the number of solder ball points that should belong to each group should be known in advance according to the type of BGA package. Therefore, if the number of solder ball points actually belonging to each group is different from the known number, the BGA
It is highly possible that a defect exists in the package itself. Therefore, the failure of the BGA package may be detected by detecting such a state.

[0030]

As described above, according to the image recognizing apparatus and the image recognizing method of the present invention, a plurality of small-grain images having an area not less than a predetermined minimum value and not more than a predetermined maximum value are soldered. Since the ball is detected and the inclination of the package is obtained from the inclination of the straight line connecting these images, the attitude of the BGA package can be accurately grasped.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an embodiment.

FIG. 2 is a flowchart of a processing program according to an embodiment.

FIG. 3 is a flowchart of a processing program according to one embodiment.

FIG. 4 is an operation explanatory diagram of one embodiment.

FIG. 5 is an operation explanatory diagram of one embodiment.

FIG. 6 is an operation explanatory diagram of one embodiment.

FIG. 7 is an operation explanatory diagram of one embodiment.

FIG. 8 is an operation explanatory diagram of one embodiment.

FIG. 9 is an operation explanatory diagram of one embodiment.

FIG. 10 is a bottom view of various BGA packages.

[Explanation of symbols]

 6 CPU (detection means, calculation means)

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06T 1/00 G06T ⁷ /00-7/60 H05K 13/04 G01B 11/00-11/30

Claims (4)

(57) [Claims] When image data obtained by imaging the bottom surface of a ball grid array type package is supplied, a plurality of small particle images having an area not less than a predetermined minimum value and not more than a predetermined maximum value are formed. 4 through a detecting means for detecting from among the image data, for each of the plurality of small-shaped image, the image
Calculate the intercept of the straight line of the type of inclination, based on the intercept
Extracting four corner images from the plurality of small-grain images
Four corner extracting means, a first straight line connecting two of the four corner images,
A second straight line connecting two images other than the two images in the corner images
Calculating means for calculating a line and the first straight line for each of the plurality of small-grain images.
Pass a straight line with the same slope as the line, and the straight lines
The contacting images are grouped together and the plurality of small
For each of the granular images, the same inclination as the second straight line is used.
Pass through the straight line, and the straight lines
Grouping means for grouping images, and for each of the groups,
Calculate the line and calculate the average
An image recognizing device, comprising: an inclination calculating means for calculating an inclination of the package . 2. The method according to claim 1, wherein said inclination calculating means includes :
When calculating a straight line passing through the images, the images in the same group
And the straight line that minimizes the average of the squared values of the distances between
The image recognition apparatus according to claim 1, wherein the calculation is performed . 3. The image recognition device according to claim 2 , wherein said image recognition device includes a center of gravity calculating means for calculating a center of gravity of said plurality of small-grain images.
Further, the center of gravity calculating means has a package inclination calculated by the inclination calculating means.
The vertical and horizontal directions of the plurality of small-grain images are obtained, and for each of the plurality of small-grain images, the position of the image is obtained.
And symmetrically in the vertical direction, and for these two positions
Whether an image exists at two positions symmetrical with respect to the horizontal direction
And if it is determined that it exists, the above four positions
Find the center of gravity, and calculate the average value of the found center of gravity for the package.
2. The method according to claim 1, wherein the center of gravity is calculated.
Image recognition device. 4. A ball grid array type package.
Take an image of the bottom surface, and from the image data of the package,
A plurality of small-grained
Detecting an image and passing through each of the plurality of small-grain images through the image;
Calculate the intercept of the straight line of the type of inclination, based on the intercept
An image forming four corners is extracted from the plurality of small-grained images, and a first straight line connecting two images among the four corner images,
A second straight line connecting two images other than the two images in the corner images
And calculating a line for each of said plurality of small-grain images.
Pass a straight line with the same slope as the line, and the straight lines
The contacting images are grouped together and the plurality of small
For each of the granular images, the same inclination as the second straight line is used.
Pass through the straight line, and the straight lines
Are grouped, and for each of the groups, the
Calculate the line and calculate the average
Image recognition, which is calculated as the inclination of the package.
Knowledge method.