JPH08147468A - Position recognizing method for bag by image processing - Google Patents

Abstract

PURPOSE: To recognize the position of the BGA sucked to a suction nozzle accurately at all times irrelevantly to the color of the substrate of the BGA, whether or not there is a pattern on it, etc. CONSTITUTION: The reverse surface of the BGA sucked to the suction nozzle is lit with scattered light to pick up its image, parts which are closed to at least four sides of the image are directly filtered horizontally and vertically to find the center coordinates of each solder ball candidate whose lightness exceeds a set level, and the point where the center coordinates found by the filtering in both the directions nearly match each other is extracted as the center point coordinates of each solder ball. The respective center point coordinates are searched from the outside to the inside of the image to extract only the center point coordinates of respective solder balls at the periphery of the BGA, and the tilt and center of the BGA are found from the extracted center point coordinates of the solder balls.

Description

Detailed Description of the Invention

[0001]

[Industrial application] BGA in chip mounters
The position of the BGA that is picked up by the suction nozzle by image processing in order to correct the position after mounting the (Ball Grid Array) with the suction nozzle and mount it at a predetermined position on the printed circuit board. Regarding recognition method.

[0002]

2. Description of the Related Art Conventionally, a QFP (Qu
Most were ad flat packages. This is a package with many lead terminals protruding from all four sides. To increase the number of input / output terminals without changing the external dimensions, the terminal pitch must be narrowed, but this is already the limit. Has reached Further, since the lead terminals of the fine pitch QFP are thin and easily deformed, various problems occur in the mounting process.

For this reason, a new multi-terminal package called BGA has been developed in recent years and has come to be used as an LSI package for personal computers, portable communication devices and the like. This BGA has a semiconductor integrated circuit 12 bonded on a base substrate 11 and sealed with a mold resin 13 as shown in side views and bottom views of FIGS. This is a surface-mounted LSI package in which a large number of solder balls (spherical solders) 14 are arranged in a two-dimensional array as terminals.

Therefore, since this BGA1 is connected to the circuit on the printed circuit board by a large number of solder balls, the QFP
Since the size of the solder balls is smaller than that of the solder balls, and the solder balls that are terminals are not deformed, even if the mounting density is increased, no serious problem occurs in the mounting process.

In order to mount such a BGA on a printed circuit board by a chip mounter, as in the case of mounting the QFP, the deviation and inclination of the suction position of the BGA sucked by the suction nozzle from the specified position are corrected. Then, in order to accurately mount it on a predetermined position on the printed circuit board, it is necessary to recognize the position (including the inclination) of the BGA sucked by the suction nozzle.

Therefore, in the conventional chip mounter, for example, as shown in FIGS. 12 and 13, a lighting device 4 in which a plurality of LEDs (light emitting diodes) 3 are two-dimensionally arranged on the lower surface of the BGA 1 sucked by the suction nozzle 2 is provided. The illumination light by the light emission of is illuminated by the scattered light that has passed through the scattering plate 5, and the image by the reflected light is captured by the CCD camera 6 through the central opening 5a of the scattering plate 5 and the central opening 4a of the illuminating device 4. Reference numeral 7 is a black plate which is arranged behind the BGA 1 so as to penetrate the suction nozzle 2.

In this way, the lower surface of the BGA 1 adsorbed by the adsorption nozzle 2 is brought into a high contrast state by the black plate 7 of the back and the reflected illumination by the scattered light. Then, the image data picked up by the CCD camera 6 is taken into the image processing unit 8, and the BGA is processed by the image processing.
1 position recognition is performed.

A conventional BG by the image processing unit 8
An example of the position recognition processing of A will be described according to the flowchart of FIG. 14 and also with reference to FIGS. CCD
Image data captured by the camera 6 (multi-tone image data as shown in FIG. 15) is taken in, binarized as shown in FIG. 16, and stored in the image memory as binary image data.

As shown in FIG. 17, the image data is searched from the outside to an edge (a point of the white pixel which is in contact with a black pixel: indicated by a small circle e in the figure), and each detected edge is isolated white. The regions W are grouped. The coordinates of the center point O of the solder ball are obtained by averaging the detected coordinates of each edge for each group.

Then, the solder balls (center point coordinates) in the column closest to the outer periphery of the BGA image are extracted, and the slope of a straight line obtained by the least square method from each center point (coordinates) group of the solder balls, that is, BGA. Find the slope of. Further, the center of the BGA is obtained from the average of the coordinates of the respective center points of the solder balls along the outer circumference thereof.

[0011]

However, in such a conventional BGA position recognition method by image processing,
Image processing is performed by converting multi-valued image data captured from a CCD camera into binary image data, but if the base (substrate) around the BGA solder ball is whitish color like ceramic, it was taken. Image data is shown in Figure 1.
As shown in 8. As shown in FIG. 19 by partially enlarging this, the image K of the substrate around the solder ball is bright, and the image S of the solder ball is bright at the central portion a, but the peripheral portion thereof shows the illumination light and its reflection angle. Due to the relationship, there is a tendency to become a donut-shaped dark portion b. In this case, the luminance distribution (light map) along the line A passing through the centers of the images of the solder balls in one row is as shown in FIG.

Therefore, when this image data is binarized and edges are detected, it becomes as shown by a small circle e in FIG. 21, and if the edge of the binarized image data is simply detected, the image S of the solder ball and that However, there is a problem in that the images other than the above cannot be distinguished and may be erroneously recognized.

In addition, even if the underlying substrate of BGA is a dark color,
If a bright pattern exists there, the captured image data is as shown in FIG. FIG. 23 is an enlarged view of a part including the image P of the bright pattern, and the luminance distribution (light map) along the line B is shown.
Is as shown in FIG. Therefore, when this image data is binarized and edges are detected, it becomes as shown by a small circle e in FIG. 25. Even if the edges of the binarized image data are simply detected, the image S of the solder ball and other images are not detected. The images (bright pattern images P, etc.) cannot be distinguished, resulting in erroneous recognition.

The present invention has been made in view of the above problems, and as described above, the BGA substrate has a bright color, the peripheral portion of the solder ball has a donut-shaped dark image, and the BGA substrate has a dark image. Even if there is a bright pattern on the upper side, the object is to always be able to accurately recognize the position of the BGA sucked by the suction nozzle.

[0015]

In order to achieve the above object, the BGA position recognition method by image processing of the present invention is a BGA which is sucked by a suction nozzle of a chip mounter as shown in FIGS. 12 and 13. The lower surface of the image is illuminated with scattered light to be imaged, at least the portion near the outer periphery of the image is scanned to find the coordinates of the center point of each solder ball candidate, and the calculated solder ball candidate point is located on the outer periphery of the image. In addition, it is characterized in that a point which is bright in the central part and dark in the peripheral part is extracted as the center point coordinates of each solder ball, and the inclination and center of the BGA are obtained from the extracted center point coordinates of the solder ball.

Alternatively, a portion of at least four sides of an image of the lower surface of the BGA is filtered in the horizontal direction to obtain the center point coordinates of each solder ball candidate having a brightness equal to or higher than a set level, and in the vertical direction. Also performs filter lining to find the center point coordinates of each solder ball candidate whose brightness is above the set level, and extracts the points where the center point coordinates found by filtering in both directions are approximately the same as the center point coordinates of each solder ball. Then, the inclination and center of the BGA can be obtained from the extracted center point coordinates of the solder ball.

In this BGA position recognizing method, at least a portion close to four sides of an image of the lower surface of the BGA is horizontally filtered to obtain center point coordinates of each solder ball candidate, and vertically filtered. Then, when the center point coordinates of each solder ball candidate are obtained, the center point coordinates obtained for each scanning line in the filtering in each direction are grouped for each solder ball candidate and averaged to obtain each solder ball candidate. It is better to find the coordinates of the center point of the candidate.

Further, when filtering the horizontal and vertical directions, it is considered that a bright portion of the solder ball in the image of the lower surface of the BGA is out of focus within a certain range, and template matching is performed for the solder ball. It is desirable to use a filter having a characteristic that strongly reacts with the image portion.

[0019]

If the BGA position recognition method by image processing of the present invention is implemented, at least a portion of the image of the lower surface of the BGA near the outer periphery is scanned to obtain the coordinates of the center point of each solder ball candidate, and the obtained coordinates are obtained. The points that are on the outer periphery of the image from the candidate points of the solder ball, and the central part is bright and the peripheral part is dark are extracted as the coordinates of the central point of each solder ball, so it is almost impossible to extract the parts other than the solder ball. The inclination and center of the BGA can be accurately obtained from the coordinates of the center point of the solder ball.

Further, by filtering the picked-up multi-valued image data in the horizontal and vertical directions, the coordinates of each central point of the isolated area having the brightness equal to or higher than the set level are determined as the solder ball candidates, and the filtering is performed in both directions. By extracting only the coordinates of the points where the calculated center point coordinates are approximately the same as the center point coordinates of the solder ball, even if there is a bright pattern other than the solder ball in the base area, it will not be detected incorrectly Only the center point coordinates of the solder ball can be obtained.

As described above, after filtering the image data and extracting the portion where the solder ball is present as a ball candidate, the filter characteristic is appropriately selected, preferably the image portion of the solder ball by template matching. By using a filter having a characteristic that strongly reacts to, even if there is some fluctuation or unevenness of the illumination, it is possible to always perform accurate position recognition without being affected by it. Further, even if the material or color of the substrate of the BGA changes, by changing the characteristics of the filter accordingly, accurate position recognition can be performed without changing the image processing algorithm.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of the present invention will be specifically described below with reference to the drawings. The hardware configuration of the BGA position recognition unit of the chip mounter for carrying out the present invention is as shown in FIGS.
The image processing algorithm for recognizing the position of the BGA in the image processing unit 8 is different from that shown in FIG. BG in this image processing unit 8
Image processing for recognizing the position A will be described with reference to the flowchart of FIG.

First, the CCD camera 6 shown in FIGS. 12 and 13 captures the BGA 1 adsorbed by the adsorption nozzle 2 at substantially the center, and multi-valued (multi-gradation) image data, for example, FIG. 15, FIG. 18, the image data as shown in FIG. 22 is fetched and stored in the image memory. Then, horizontal and vertical filtering processes are performed, and then scanning processes for determining the center points of the solder balls (images) are performed. In this embodiment, these processes can be performed efficiently only in the necessary portions. In order to achieve this, as shown in FIG. 2, four windows 10a are provided for the image 1i of BGA1.
-10d is set.

The four windows 10a-10d are
Even if the image 1i of the BGA1 is displaced or tilted to some extent, it is set to cover the inside of the four sides of the top, bottom, left, and right with a required width, and filtering processing and soldering are performed only on the image data in each window Ball (statue)
The scanning process is performed to find the center point of. The arrow H in FIG. 2 indicates the horizontal filtering direction, and the arrow V indicates the vertical filtering direction. FIG.
In the example shown in FIG. 2, the four windows 10a to 10d have no overlapping portion, but the window 10
It is also possible to extend c and 10d in the vertical direction so that the horizontal window and the vertical window overlap at the four corners.

Therefore, first, the window 10 on the upper side of FIG.
The horizontal filtering process is performed on the image data in a. The filter used for this filtering is designed to have a characteristic of strongly reacting to the image of the solder ball (particularly the diameter portion) by template matching, which will be described later. Then, as shown in FIG. 3, the center point (midpoint between edges) Ch of the ball candidate S '(isolated area whose brightness is equal to or higher than a set level) in one line of each horizontal scanning is detected.

While this process is repeated until the window of one side (the window 10a of the upper side at the beginning) is completed, the center points Ch found for each scanning line are grouped for each ball candidate S'as shown in FIG. To do. That is, the center points Ch that are continuous in the vertical direction that is substantially orthogonal to the filtering direction are grouped. The result of the grouping is also shown in FIG. Each of these center points is detected as an X, Y coordinate value.

Then, a group in which the number of center points of each group is within a certain range is extracted. At this time, the number of continuous lines is counted, and the number, that is, the number of center points of the group is larger than the number of pixels (for example, 6) exceeding the diameter of the solder ball, or extremely small (for example, 1). This is because it is determined that the point is not a point in the image of the solder ball and is removed.

Then, the average of the coordinates of the center point Ch of each extracted group is obtained. As a result, as shown on the right side of FIG. 3, the coordinates (xh, yh) of the central point Ch ₀ on the substantially horizontal diameter of the ball candidate S ′ are obtained. The coordinates of each central point Ch of this group are (x1, y1), (x
2, y2), (x3, y3), xh = (x1 + x2 + x3) / 3 yh = (y1 + y2 + y3) / 3

Next, after vertically filtering the image data in the same window 10a as described above, as shown in FIG. 4, the center point of the ball candidate S'in one line of each vertical scan. Detect Cv.

While this process is repeated for one side window (the upper side window 10a first), the center points Cv found for each scanning line are grouped for each ball candidate S'as shown in FIG. To do. That is, the center points Cv that are continuous in the horizontal direction, which is a direction substantially orthogonal to the filtering direction, are grouped. The result of the grouping is also shown in FIG. Each of these center points is detected as an X, Y coordinate value.

Then, a group in which the number of center points of each group is within a certain range is extracted, and the average of the coordinates of the center point Cv is calculated for each extracted group. As a result, as shown on the right side of FIG. 4, the coordinates (xv, yv) of the center point Cv ₀ on the substantially vertical diameter of the candidate ball S ′ are obtained in the same manner as in the horizontal direction described above.

Thereafter, the difference between the average coordinates (xh, yh) based on the horizontal filtering obtained in step A of FIG. 1 and the average coordinates (xv, yv) obtained from the vertical filtering obtained in step B, that is, in FIG. Only pairs in which the distance d between the coordinates (xh, yh) of the point Ch ₀ and the coordinates (xv, yv) of the point Cv ₀ shown below are equal to or less than a fixed value (a considerably small value) are extracted. This is because the image of the solder ball should be circular, so if there is an image of the solder ball, there should be almost no difference between the coordinates obtained from these two directions. Only the central coordinates can be extracted.

To find the coordinates of the center point of each solder ball (image) from the extracted coordinates of each pair, either one of the pairs of average coordinates is selected or the average of both coordinate values is taken. Good. Further, the center point coordinates of each solder ball in the window 10a thus obtained are searched from the outside to the inside of the image to extract the center point coordinates of the solder ball on the outer periphery of the BGA.

The processing from such horizontal filtering is performed by the windows 10a to 10d on each side shown in FIG.
When each image data in the image data is sequentially executed and the processing for the four side windows is completed, the process proceeds to the next step. By the processing so far, the central point coordinate groups of the solder balls (images) on the outer circumferences of the four sides of the BGA are obtained.
Determine the slope and center of GA. That is, similar to the conventional processing method shown in FIG. 14, the BGA inclination is calculated from the center points (coordinates) of the solder balls by the method of least squares, and the BGA inclination is calculated from the center points (coordinates) of the solder balls. The center coordinates of the BGA can be obtained from the average of the center point coordinates.

Here, the filtering will be described. For example, when the BGA substrate has a bright color, the signal level of the line A passing through each center of the image of the solder balls in one row of image data as shown in FIGS. 18 and 19 is as shown in FIG. Changes to. The high peak of this waveform is the center of the solder ball image, the slightly lower peak is the substrate portion, and the valley is the peripheral portion of the solder ball image. In this way, the image of the solder ball has a bright central part, but a donut-shaped dark part in the peripheral part.

By filtering this image data, it is possible to obtain a signal waveform in which a peak appears only in the central portion of the image of the solder ball as shown in FIG. 8B. Thereby, the subsequent process of extracting the image of the solder ball and obtaining the coordinates of the center point thereof can be performed easily and reliably.

Although a one-dimensional filter is used for this filtering, it is preferable to use an electronic filter whose characteristics can be arbitrarily changed. As a characteristic, a bright portion of the solder ball in the captured image is blurred in a certain range. Therefore, it is preferable to have a characteristic that strongly reacts to the image portion of the solder ball by template matching.

FIG. 9 shows an example of the characteristic curve of the Gaussian filter. (A) has the characteristic of equation (1) and (b) has the characteristic of equation (2). In these formulas, σ1 = solder ball diameter / 4, σ2 = solder ball diameter / 2, which shows the blur of the image of the solder ball. The target solder ball is the blur of (a). The difference between f ₁ and f ₂ is considered to be a blur between the blurs in (b), and the difference between f ₁ and f ₂ is used as one solder ball filter. In fact, f ₁
And fold the difference of f ₂ into the array of solder balls,
The filter is made up of several solder balls and then the stability is made more and more characteristic. Since the difference between f ₁ and f ₂ and the array of solder balls are used as a filter, data other than the solder balls can be removed.

[0039]

[Equation 1]

[0040]

[Equation 2]

It is advisable to prepare a plurality of such one-dimensional filters according to the color of the BGA substrate for which the position is to be recognized and the size of the solder balls, and switch and use them. In the above embodiment, the filtering process, the extraction of the solder ball candidates, and the process of obtaining the center coordinates thereof are performed only for the image data in the window shown in FIG. 2 to improve the processing efficiency. Even if the same processing is performed on the entire data,
Of course, the position of BGA can be accurately recognized.

[0042]

As described above, according to the position recognition method of the present invention, the position (including the inclination) of the BGA adsorbed by the adsorption nozzle of the chip mounter indicates the bright color of the BGA substrate such as ceramic. Even if there is a bright pattern other than the solder ball on the substrate, it can always be recognized accurately. Further, by appropriately selecting the filter characteristic when filtering the image data, even if there is some variation or unevenness in the illumination, accurate position recognition can be performed without being affected by it.

[Brief description of drawings]

FIG. 1 is a flow chart of image processing for recognizing a position of a BGA in an image processing unit according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a four-sided window used in this embodiment.

FIG. 3 is an explanatory diagram of a process of obtaining a center coordinate of a solder ball candidate and an average coordinate of a group for each scanning line based on horizontal filtering.

FIG. 4 is an explanatory diagram of a process of obtaining a center coordinate of a solder ball candidate and an average coordinate of a group for each scanning line based on vertical filtering.

FIG. 5 is an explanatory diagram of a plurality of center coordinates found in each solder ball candidate based on horizontal filtering and grouping thereof.

FIG. 6 is an explanatory diagram of a plurality of center coordinates found in each solder ball candidate based on vertical filtering and grouping thereof.

FIG. 7 is an explanatory diagram of a distance between center point coordinates of solder ball candidates obtained from a horizontal direction and a vertical direction.

FIG. 8 is a diagram showing a signal level distribution of image data for explaining a filtering process according to the embodiment of the present invention.

FIG. 9 is a diagram for similarly explaining the characteristics of the filter.

FIG. 10 is a side view showing an example of a BGA.

FIG. 11 is a bottom view of the same.

FIG. 12 is a front view showing a schematic configuration of a BGA position recognition unit in the chip mounter.

FIG. 13 is a perspective view of the same as seen from diagonally above.

FIG. 14 is a flowchart showing an example of conventional BGA position recognition processing in the image processing unit 8 of FIGS. 12 and 13;

FIG. 15 is a diagram showing an example of image data captured by the CCD camera 6 of FIGS. 12 and 13.

16 is a diagram showing an example of binary image data obtained by binarizing the image data of FIG.

17 is an explanatory diagram of edge detection of the binary image data shown in FIG.

FIG. 18 is a diagram showing captured image data when the BGA substrate has a whitish color such as ceramic.

FIG. 19 is a partially enlarged view of the same.

20 is a diagram showing a luminance distribution along a line A in FIG.

FIG. 21 is an explanatory diagram of edge detection of the binary image data.

FIG. 22 is a diagram showing captured image data in the case where a bright pattern exists on the substrate of BGA.

FIG. 23 is a partially enlarged view of the same.

FIG. 24 is a diagram showing a luminance distribution along line B in FIG. 23.

FIG. 25 is an explanatory diagram of edge detection of the binary image data.

[Explanation of symbols]

 1: BGA 1i: BGA image 2: Adsorption nozzle 3: LED 4: Lighting device 5: Scattering plate 6: CCD camera 7: Black plate 8: Image processing unit 10a to 10d: Window 11: Substrate 12: Semiconductor integrated circuit 13 : Mold resin 14: Solder ball

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G01B 11/26 H G06T 1/00 H05K 13/08 Q 8315-4E G06F 15/64 325 F

Claims (4)

[Claims] 1. A surface mount type LSI in which a large number of solder balls are arranged in a two-dimensional array in a chip mounter.
BGA, which is a package, is adsorbed by the adsorption nozzle and the B
A method of recognizing the position of the GA by image processing, wherein the lower surface of the BGA adsorbed by the adsorption nozzle is illuminated with scattered light to be imaged, and at least a portion near the outer periphery of the image is scanned to determine each solder ball. The coordinates of the center points of the candidates are calculated, and points that are on the outer periphery of the image and are bright in the central part and dark in the peripheral parts are extracted as the coordinates of the center points of the respective solder balls by calculation from the calculated solder ball candidate points. BGA by image processing characterized by obtaining the inclination and center of the BGA from the coordinates of the center point of the solder ball
Position recognition method. 2. A surface mount type LSI in which a large number of solder balls are arranged in a two-dimensional array in a chip mounter.
BGA, which is a package, is adsorbed by the adsorption nozzle and the B
A method of recognizing the position of GA by image processing, illuminating the lower surface of BGA adsorbed by the adsorption nozzle with scattered light to capture an image, and horizontally filtering at least a portion close to four sides of the image. , The center point coordinates of each solder ball candidate whose brightness is above the set level are found, and the center point coordinates of each solder ball candidate whose brightness is above the set level are found by filtering in the vertical direction as well. Image processing characterized in that points at which the coordinates of the center points obtained by filtering are substantially the same are extracted as the coordinates of the center points of the solder balls, and the inclination and center of the BGA are calculated from the coordinates of the center points of the extracted solder balls. By BGA
Position recognition method. 3. The BGA position recognition method according to claim 2, wherein a portion near at least four sides of the captured image is horizontally filtered to obtain center point coordinates of each solder ball candidate, and vertically. When filtering the center point coordinates of each solder ball candidate by filtering, the center point coordinates found for each scanning line in the filtering in each direction are grouped for each solder ball candidate and averaged. A BGA position recognition method by image processing, characterized in that the coordinates of the center point of each solder ball candidate are obtained. 4. The image processing B according to claim 2 or 3.
In the position recognition method of GA, when filtering in the horizontal and vertical directions, it is considered that a bright portion of the solder ball in the captured image is out of focus within a certain range, and the image portion of the solder ball is subjected to template matching. A method of recognizing a position of a BGA by image processing, which comprises using a filter having a characteristic of strongly reacting to.