JPH11281325A - Image recognition system and image recognition method using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately recognize the presence of a defective mark caused from dummy bonding. SOLUTION: A picked up color image forming predetermined region of an inferior by a color CCD camera 1 is separated into three elements of red, green and blue by a RGB decoder 3, and a spectral reflectance is calculated by wavelength-analyzing an image separated into each element, and then an image expressing a specific characteristic space being similar to a characteristic space of the inferior mark is extracted by a color extraction apparatus 4 from this factor. An extracted image is converted into a white image to area-count white pixels as objects after thresholding by an area counter 5, and then, the presence of the inferior mark is judged by comparing the area-counted quantity with reference quantity of white pixels precalculated based upon the characteristic of the defective mark.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition system and an image recognition method using the same, and more particularly, to an image recognition system for detecting the presence or absence of a defective mark in a semiconductor package assembling process and an image recognition method using the same. About.

[0002]

2. Description of the Related Art In the process of assembling a semiconductor package,
If an assembly failure occurs in the die bonding process or wire bonding process, instead of removing it every time a failure occurs, attach dummy bonding indicating the failure and collectively attach it before the next process such as resin sealing. A production method is employed in which the productivity is improved by removing the components.

[0003] A defective product detection system according to a conventional technique in a semiconductor package assembling process will be described with reference to the drawings. In the following drawings, the same portions are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

FIG. 7 is a schematic plan view showing an example of a lead frame in a semiconductor package assembling step.

As shown in FIG. 1, semiconductor chips C1 are arranged at predetermined intervals on a lead frame 11 having a long shape.
To C3 are mounted and fixed on the lead frame 11 through a die bonding process. The electrode pads 21 on each semiconductor chip are connected to metal wiring (not shown) on the lead frame 11 by a wire bonding process.
And a wire 22. If there is a bonding failure in each of the die bonding and wire bonding steps, dummy bonding as a failure mark is made in a marginal space on the lead frame 11 to clearly indicate that the failure has occurred. In the example shown in FIG. 7, dummy bonding is performed on the lower left area a1 to a3 of the paper around the semiconductor chip. In the example shown in the figure, the semiconductor chip C1 has no bonding failure and therefore no dummy bonding has been performed. However, the semiconductor chips C2 and C3 have misalignment during die bonding and poor connection during wire bonding. , Defective marks Fm1 and Fm2 are dummy-bonded to a2 and a3, respectively.

FIG. 8 shows an example of a defective product detection system according to the prior art. The defective product detection system 90 shown in FIG. 6 includes a region A in the longitudinal direction including regions a1 to a3 to which the defective mark is attached, among the regions on the lead frame 11.
(See FIG. 7), an illumination device 2 for irradiating the area A, a monochrome CCD camera 91 installed almost directly above the area A, and imaging the area A, and a binary image signal of the area A imaged by the CCD camera. And an area counter 95 for performing a conversion process. The lead frame 11 is conveyed in the direction of the arrow in FIG. 1, whereby the area A to be imaged by the CCD camera moves. The pilot holes 12 (see FIG. 7) formed at both ends in the short direction of the lead frame 11 are pin holes that are also used for transporting the lead frame 11. It is also used to set a basic position reference,
The pilot hole 12 is also imaged.

The operation of the defective product detection system 90 will be described together with the principle of its detection with reference to the flowchart of FIG.

First, the monochrome CCD camera 91 captures an image of the area A to be subjected to dummy bonding, which moves with the transport of the lead frame 11, and supplies an image signal to the area counter 95 (step S500).

Next, the area counter 95, which has received the image signal of the area A from the CCD camera, binarizes the image signal (step S510). As a result, when no defective mark is bonded, all the light emitted from the illumination device 2 is specularly reflected, so that an image signal consisting of only white pixels is obtained. In the captured image in the case where the defect mark is formed, since the defective mark formed of a metal such as gold (Au) has a dome shape (see FIG. 8), the irradiation light from the illumination device 2 causes irregular reflection, Black pixels will be mixed. Therefore, only the number B of black pixels is counted by the area counter 95 for the image signal thus binarized (step S520), and the amount of black pixels that would be mixed in when dummy bonding is performed is determined in advance. If the value is set as a value S ′ and the above-described number B of black pixels is compared with the reference value S ′ for each frame (step S530), the presence or absence of a defective mark can be detected.

That is, if the quantity B of the black pixels does not reach the reference value S ', it is determined that no defective mark Fm has been formed in the area A by die bonding or the like, and the frame remains unchanged. It is carried out to the process (Step S540). On the other hand, when the number B of the black pixels is equal to or more than the reference value S ′, it is determined that the defective mark Fm is formed in the area A by die bonding or the like,
The frame is removed as a bad frame (Step S550). The position of the defective mark is specified using the pilot hole 12 described above.

Since at least two defective marks Fm1 and Fm2 are formed on the lead frame 11 shown in FIG. 7, the number B of black pixels included in the binarized image signal is equal to or larger than the reference value S '. And this lead frame 1
1 is removed as a bad frame.

As described above, the defective product detection system according to the prior art utilizes a difference in the number of black pixels contained in an image signal binarized by specular reflection / diffuse reflection of light. This has the advantage that a defective mark can be detected.

[0013]

However, the above-mentioned prior art has the following drawbacks because it depends on the characteristics of the three-dimensional shape of the dummy bonding in the region to be imaged.

That is, in the above-mentioned prior art, the irregular reflection of the irradiation light caused by the dome-shaped three-dimensional shape of the dummy bonding is used to determine the number of black pixels in the black and white image. For this reason, there is a disadvantage that it cannot be distinguished from a black pixel mixed in a monochrome image. For example, when the surface of the lead frame 11 is in a bad state, such as when the surface of the lead frame 11 has irregularities due to scratches or the like, irregular reflection occurs due to the irregularities. , And various other irregular reflections are generated from an object such as a frame holder under the lead frame 11 and a heater block, and these form black pixels of a binarized image signal.

FIG. 10 shows a specific example of the case where irregular reflection occurs due to such various factors. FIG. 10 shows a black and white CC
9 shows a black-and-white image 100 after an image signal captured by a D camera 91 has been binarized by an area counter 95. As shown in the figure, a large number of black pixels are irregularly arranged in the black and white image 1 due to a number of factors other than the dummy bonding in the imaging region.
Appears on 00.

Accordingly, in the prior art, erroneous detection may occur when there is a large amount of irregular reflection by an object other than the above-described dummy bonding. In this case, there is a problem in that although there is no defective mark on the lead frame 11, it is removed as a defective frame.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to provide a high-precision image recognition system capable of accurately identifying the presence or absence of a defective mark due to dummy bonding without erroneous detection. An object of the present invention is to provide an image recognition method using this.

[0018]

The present invention solves the above problems by the following means. That is, the present invention (Claim 1)
According to the present invention, there is provided a color imaging unit that captures a color image of a subject, a spectral reflectance analysis unit that analyzes three components of hue, saturation, and brightness of the color image of the subject and calculates a spectral reflectance based on the three components. A storage unit for storing a reference spectral reflectance based on the three elements of the object to be recognized, and comparing the spectral reflectance of the subject with the reference spectral reflectance to determine whether the subject includes the object. An image recognition system including a determination unit for determining whether the image recognition is performed.

The judging means extracts an image area substantially corresponding to a reference spectral reflectance of the object from the color image of the object, and when the size of the image area is within a predetermined range, Is preferably provided with the above object.

Further, according to the present invention (claim 3), a color image pickup means for picking up a color image of a subject, and the color image of the subject picked up by the color image pickup means are converted into three primary colors of red, green and blue Color separation means for outputting a first image signal for each color, and analyzing the three components of the hue, saturation and lightness of the color image based on the first image signal, and performing spectral analysis based on the three components. Spectral reflectance analyzing means for calculating the reflectance, storage means for storing a reference spectral reflectance based on the three elements of the object to be recognized, and substantially corresponding to the reference spectral reflectance of the object from the color image of the subject Image extracting means for extracting an image area to be output and outputting a second image signal, white pixel converting means for binarizing the three elements of the second image signal, and calculating the number of white pixels Area calculating means; Calculation result by the area calculation means the subject when falling within a predetermined range image recognition system and a determining means and being provided with the above-mentioned object is provided.

In the image recognition system according to the present invention, the subject is a semiconductor package during an assembling process, and the object is attached to a surface of the semiconductor package when the semiconductor package has a defective assembly. It is preferable that the mark is a defective mark.

Further, according to the present invention (claim 5), an image pickup process of picking up a color image of a subject, and three components of hue, saturation and lightness of the color image of the subject are analyzed, and spectral analysis based on the three components is performed. A spectral reflectance analysis process for calculating the reflectance, and an image area substantially corresponding to a reference spectral reflectance based on the three elements of the object to be recognized, which is stored in advance in a storage unit, is extracted from the color image of the subject. An image recognition method is provided that includes an image extraction step and a determination step of determining that the subject has the object when the size of the image area is within a predetermined range.

Further, according to the present invention (claim 6), an image pickup process of picking up a color image of a subject, and the color image is separated into three primary colors of red, green and blue to form a first image for each color. A color separation process of outputting a signal, and three of hue, saturation and lightness of the color image based on the first image signal.
A spectral reflectance analyzing step of analyzing the elements and calculating a spectral reflectance based on the spectral reflectance; and a spectral reflectance substantially corresponding to the reference spectral reflectance based on the three elements of the object to be recognized from the first image signal. An image extraction step of extracting a second image signal having a ratio, a white pixel conversion step of forming a white image by binarizing the three elements of the second image signal, and a white pixel conversion step included in the white image. An area calculating step of calculating the number of white pixels, and a determining step of determining that the subject has the predetermined object when the calculated area obtained in the area calculating step is within a predetermined range. An image recognition method comprising:

In the image recognition method according to the present invention, the subject is a semiconductor package in an assembling process, and the object is attached to a surface of the semiconductor package when the semiconductor package has a defective assembly. It is preferable that the mark is a defective mark.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of an image recognition method according to the present invention will be described.

The present invention utilizes the fact that an object has a unique spectral reflectance for each material and each surface shape. That is, the object is irradiated with light, the reflected light is extracted and decomposed into three elements of hue, saturation, and lightness, and the spectral reflectance is calculated based on the three elements. The fact that the spectral reflectance is distributed within a predetermined range is used.

FIG. 4 shows the calculated spectral reflectances of blue rubber, red fiber, and red plastic, each having a certain shape. 4, the horizontal axis indicates the wavelength of the reflected light, and the vertical axis indicates the spectral reflectance. As shown in the figure, red fibers and plastics have high spectral reflectance in the range of about 780 nm to about 640 nm, while blue rubber has high spectral reflectance in the range of about 570 nm to about 380 nm. Have. Even for the same red color, the fiber and the plastic show different distribution states. As described above, each object forms a unique feature space including the ratio and level of three elements of hue, saturation, and brightness, and the reflected light of each object has a unique spectral reflectance based on this feature space. It is determined. For example, as shown in FIG. 4, even if the hue is the same, if the surface shape is different, the saturation and brightness are different, so that the feature space is different and the spectral reflectance is different.

Therefore, if the object to be detected has a specific feature space that can be clearly distinguished from surrounding objects, the spectral reflectance is calculated from the entire captured image, and the level of only a specific element is extracted. After that, a white image conversion is performed, and then a binarization process is performed, the number of white pixels only is counted, and the presence / absence of the target substance is determined by comparison with the number of white pixels that the target substance should have. be able to.

A lead frame 1 which is a problem to be solved by the present invention
1 in the detection of the defective mark on the lead frame 11
Is formed by silver plating, whereas the defective mark to be detected is formed by gold. Therefore, their spectral reflectances are distinctly different.

FIG. 5 shows the distribution of the spectral reflectance in the same shape of silver and gold. As shown in FIG. 5, gold and silver are about 5
It can be seen that the distribution is almost line-symmetric with respect to the region of 90 nm. Further, as described above, the defective mark has a dome-like shape, so that the feature space is clearly different from that of the lead frame 11 having a substantially flat shape.

Therefore, if the number of reference white pixels is prepared in advance through white image conversion and binarization processing based on the spectral reflectance of the defective mark by the dummy bonding, it is extracted by the above-described method. By comparing the number of white pixels having a specific feature space with the number of white pixels, it is possible to recognize the presence or absence of a defective mark in the target image.

Hereinafter, some embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic front view showing an embodiment of the image recognition system according to the present invention.

As shown in the figure, an image recognition system 10 according to the present embodiment includes an illumination device 2 for irradiating light to an image pickup area A shown in FIG. 7, and a color CCD camera for picking up a color image of the area A. 1 and R for separating the color image captured by the color CCD camera 1 into three elements of red, green, and blue.
A GB decoder 3, a color extraction device 4 for calculating a spectral reflectance by wavelength analysis of an image separated into each element, and then extracting a specific characteristic space, and a white pixel of a defective mark calculated in advance. It has a memory 6 (not shown) storing the quantity, converts the image extracted by the color extraction device 4 into a white image, performs a binarization process, counts the area of white pixels, An area counter 5 for comparing the count result with the number of defective mark white pixels stored in the memory 6.
And

The operation of the image recognition system 10 will be described as an embodiment of an image recognition method according to the present invention with reference to the flowcharts of FIGS.

First, as shown in FIG. 2, a color image of the area A for forming dummy bonding is captured by the color CCD camera 1 (step S100).

Next, the RGB decoder 3 decomposes the color image of the area A into image signals for each of three elements of red, green and blue (step S110).

Next, the color extraction device 4 performs wavelength analysis on the image signal for each color supplied from the RGB decoder 3 to calculate the spectral reflectance of each captured image (step S120), and then detects the image. An image substantially corresponding to the spectral reflectance distribution in a predetermined range of the feature space of the defective mark is extracted and supplied to the area counter 5 (step S1).
30).

Next, the extracted image is converted into a white image by the area counter 5 (step S140), and further, a binarization process is performed (step S150).

Next, as shown in FIG. 3, for the white image that has been subjected to the binarization processing, only the white pixels are counted by the area counter 5 to calculate the number W of white pixels (step S1).
60).

Next, the area counter 5 compares the number W of white pixels in the captured image with a reference value S calculated in advance (step S170). The reference value S is a numerical value obtained by binarizing the white image of the defective mark to be detected and calculating the number of white pixels. Whether the number W of white pixels in the captured image is equal to or greater than the reference value S Thereby, it can be determined whether or not the defective mark exists in the captured image.

That is, when W is smaller than the reference value S, it is determined that there is no defective mark due to the dummy bonding in the captured image, and the lead frame 11 corresponding to the captured image is directly processed by resin sealing or the like. It is carried out to the process (Step S180).

On the other hand, when W is equal to or larger than the reference value S,
It is determined that a defective mark due to dummy bonding exists in the captured image, and the lead frame 11 corresponding to the captured image is removed as a defective frame (step S190).

According to the image recognition method of this embodiment, 2
FIG. 6 shows a specific example of the white image after the binarization process.

FIG. 6 shows a white image inverted for black and white for convenience.

FIG. 6 shows a specific example of two white images. As shown in FIG. 6, FIG. 6A shows a black elliptical shape 15 substantially at the lower center of the image, and the elliptical shape 15 is a white pixel when a defective mark exists in the captured image. Is an aggregate of On the other hand, in FIG. 6B, there is no white pixel indicating a specific feature space. That is, FIG.
(B) shows a white image when no defective mark exists in the captured image.

As described above, according to the present invention, since the difference in spectral reflectance based on the feature space of an object is used, it is possible to recognize a detection target object from a detection target region with high accuracy using a simple configuration. A possible image recognition system is provided.
As a result, it is possible to improve the product yield corresponding to the occurrence of erroneous detection in the related art.

Further, according to the present invention, since the difference in spectral reflectance based on the feature space of an object is used, an image recognition method capable of recognizing a detection target object from a detection target region with high accuracy is provided. You. Also, because of its high recognition accuracy, the efficiency of detecting bad marks is greatly improved,
As a result, the production capacity of the entire production line can be improved.

Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment.
Various changes can be made without departing from the scope of the invention. In the above-described embodiment, the form in which the principle of the present invention is applied to detection of a defective mark in a semiconductor package assembling process has been described. For example, it can be applied to other technical fields.

[0049]

As described in detail above, the present invention has the following effects. That is, according to the image recognition system of the present invention, the spectral reflectance analyzing means for calculating the spectral reflectance based on the three elements of the hue, lightness, and saturation of the color image of the subject; Since storage means for storing the reference spectral reflectance based on the three factors and determination means for comparing the spectral reflectance of the subject with the reference spectral reflectance are provided, the target object can be recognized with high accuracy. it can.

When the image recognition system according to the present invention is applied to the detection of a defective mark in a semiconductor package assembling process, an image recognition system capable of detecting a defective mark dummy-bonded on a lead frame with high accuracy. A system is provided. As a result, it is possible to improve the product yield corresponding to the occurrence of erroneous detection in the related art.

According to the image recognition method of the present invention, the difference between the spectral reflectances based on the above three factors in the color images of the subject and the object to be recognized is used.
The target object can be recognized from the subject with high accuracy.

Further, when the image recognition method according to the present invention is applied to a method for detecting a defective mark in a semiconductor package assembling process, it becomes possible to detect the defective mark with high accuracy and high efficiency. As a result, the yield of products can be improved, and the production capacity of the entire production line can be improved.

[Brief description of the drawings]

FIG. 1 is a schematic front view showing an embodiment of an image recognition system according to the present invention.

FIG. 2 is a flowchart illustrating an embodiment of an image recognition method according to the present invention.

FIG. 3 is a flowchart illustrating an embodiment of an image recognition method according to the present invention.

FIG. 4 is a characteristic diagram showing a spectral reflectance distribution of blue rubber, red fiber, and red plastic.

FIG. 5 is a characteristic diagram showing a distribution of spectral reflectances of silver and gold.

FIG. 6 is a specific example of a white image after binarization processing obtained by the embodiment of the image recognition method according to the present invention. That is, FIG. 6A shows a white image when a defective mark exists in the captured image, and FIG. 6B shows a white image when no defective mark exists in the captured image.

FIG. 7 is a schematic plan view showing an example of a lead frame in a semiconductor package assembling step.

FIG. 8 is a front view showing an example of a defective product detection system according to the related art.

9 is a flowchart showing the operation of the defective product detection system shown in FIG.

FIG. 10 is a specific example of a black-and-white image when irregular reflection occurs due to factors other than dummy bonding.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Color CCD camera 2 Lighting device 3 RGB decoder 4 Color extraction device 5 Area counter 6 Memory 10 One embodiment of the image recognition system according to the present invention 11 Lead frame 12 Pilot hole 21 Electrode pad 22 Wire 90 Defective product detection by conventional technology Example of system 91 Monochrome CCD camera 95 Area counter a1 to a3 Dummy bonding formation area C1 to C3 Semiconductor chip Fm1, Fm2 Defective mark by dummy bonding

Continued on the front page (51) Int.Cl. ⁶ Identification code FI H01L 23/12 G06F 15/70 310 H01L 23/12 W

Claims (7)

[Claims] 1. A color imaging means for capturing a color image of a subject, and a spectral reflectance analyzing means for analyzing three elements of hue, saturation and lightness of the color image of the subject and calculating a spectral reflectance based on the three elements. Storage means for storing a reference spectral reflectance based on the three elements of the object to be recognized; and comparing the spectral reflectance of the subject with the reference spectral reflectance to determine whether the subject includes the object. An image recognition system comprising: a determination unit configured to determine whether the image is recognized. 2. The method according to claim 1, wherein the determining unit extracts, from the color image of the subject, an image region substantially corresponding to a reference spectral reflectance of the object, and when the size of the image region is included in a predetermined range, The image recognition system according to claim 1, wherein it is determined that the subject has the object. 3. A color image pickup means for picking up a color image of a subject, and a color image of the subject picked up by the color image pickup means is separated into three primary colors of red, green and blue, and a first image for each color is provided. A color separation unit that outputs a signal; a hue of the color image based on the first image signal;
A spectral reflectance analyzing unit that analyzes three components of saturation and lightness and calculates a spectral reflectance based on the three components; a storage unit that stores a reference spectral reflectance based on the three components of an object to be recognized; Image extracting means for extracting an image region substantially corresponding to the reference spectral reflectance of the object from the color image of the object and outputting a second image signal; and binarizing the three elements of the second image signal White pixel converting means, area calculating means for calculating the number of white pixels, and determining means for determining that the subject has the object when a calculation result by the area calculating means falls within a predetermined range. An image recognition system comprising: 4. The semiconductor device according to claim 1, wherein the object is a semiconductor package in an assembling process, and the object is a defect mark attached to a surface of the semiconductor package when the semiconductor package has an assembly defect. The image recognition system according to claim 3. 5. An imaging process of capturing a color image of a subject, a spectral reflectance analysis process of analyzing three components of hue, saturation, and lightness of the color image of the subject and calculating a spectral reflectance based on the three components. An image extraction step of extracting, from a color image of the subject, an image area substantially corresponding to a reference spectral reflectance based on the three elements of the object to be recognized, which is stored in advance in a storage unit; Determining that the subject is provided with the object when is within a predetermined range. 6. An image capturing step of capturing a color image of a subject, a color separating step of separating the color image into three primary colors of red, green and blue and outputting a first image signal for each color; Hue of the color image based on the image signal
Analyzing the three components of saturation and lightness and calculating the spectral reflectance based on the three components; and analyzing the three components of the object to be recognized from the first image signal.
An image extracting step of extracting a second image signal having a spectral reflectance substantially corresponding to the reference spectral reflectance based on the element; and forming a white image by binarizing the three elements of the second image signal. A white pixel conversion step of calculating the number of white pixels included in the white image, and an area calculation step of calculating the number of white pixels included in the white image. And a determining step of determining that the object is provided. 7. The semiconductor device according to claim 1, wherein the object is a semiconductor package during an assembling process, and the object is a defect mark attached to a surface of the semiconductor package when the semiconductor package has a defective assembly. The image recognition method according to claim 6.