JP4459715B2 - Image recognition method, image recognition apparatus, and mounting apparatus - Google Patents

Description

  The present invention relates to an image recognition method, an image recognition apparatus, and a mounting apparatus that are suitable for mounting electronic components on a substrate, for example.

  For example, when an electronic component is mounted on a substrate, first, an alignment mark as an inspection image provided on the electronic component is positioned so as to match the alignment mark provided on the substrate, and then the electronic component Is mounted on the substrate.

  In order to align the alignment mark provided on the electronic component and the substrate, the alignment mark provided on the electronic component and the alignment mark provided on the substrate are respectively imaged with an imaging camera, The position of the electronic component and the substrate is recognized based on the imaging signal of the alignment mark, and both the electronic component and the substrate are positioned based on the recognition. Such a prior art is shown in Patent Document 1.

Recently, for example, TCP (Tape Carrier Package) and COF (Chip on Film) are often used as electronic components, for example, as electronic devices become smaller and thinner. These electronic components are formed by punching out a thin and long resin-made tape in which semiconductor chips are die-bonded at predetermined intervals into a predetermined shape. And the said alignment mark is formed in the location where the said tape is pierced, for example by gold plating.
JP-A-3-46244

  By the way, when an electronic component is punched from a long and thin tape, stress is applied to the tape at that time, and the punched portion of the tape may be deformed. In that case, a part of the alignment mark formed on the tape may be deformed or damaged.

  When an alignment mark that is deformed or damaged is picked up by an imaging camera, light may be irregularly reflected at the deformed portion or light may not be reflected at the damaged portion. As a result, the alignment mark may not be recognized by the imaging camera. In such a case, the electronic component may be determined as a defective product.

  In other words, only the part where the alignment mark is formed is deformed or scratched, and even if the semiconductor chip is not defective, the imaging result by the imaging camera is defective. The parts were discarded, and as a result, the yield was reduced.

  It is an object of the present invention to provide an image recognition method, an image recognition apparatus, and a mounting apparatus that can surely recognize an inspection image even if a part of the inspection image is missing or does not appear.

The present invention is an image recognition method for determining whether or not an inspection image matches a reference image,
Dividing the inspection image and the reference image into a plurality of corresponding areas, and
Multiple areas of the inspection image are added sequentially, the area of the inspection image is compared with the corresponding area of the reference image, and if there is a mismatch between the area of the inspection image and the reference image during the comparison process, the mismatch Removing the area, adding a new area and continuing the comparison,
Detecting a ratio of the number of times of non-coincidence with respect to the number of comparisons between the area of the inspection image and the reference image, and determining whether the inspection image matches the reference image when the ratio is equal to or less than a reference value; An image recognition method characterized by comprising:

The present invention is an image recognition apparatus that determines whether or not an inspection image matches a reference image,
Imaging means for imaging the inspection image;
An inspection image memory for storing an imaging signal from the imaging means;
A reference image memory for storing a preset reference image;
A control means for dividing the inspection image and the reference image stored in each image memory into a plurality of areas and sequentially outputting them,
The areas of each image output from the control means are sequentially added and compared. If a mismatch occurs between the inspection image and the reference image in the comparison process, the mismatch area is removed and a new area is added. A comparison unit that continues the comparison,
A determination unit that detects a ratio of the number of times of non-coincidence with respect to the number of comparisons between areas of the inspection image and the reference image, and determines whether the inspection image matches the reference image when the ratio is equal to or less than a reference value And an image recognition apparatus characterized by comprising:

  According to the present invention, even if a part of the inspection image is missing or does not appear, since the inspection image is divided into a plurality of areas and compared with a preset reference image, a part of the inspection image is abnormal. Even if it exists, it becomes possible to recognize this test | inspection image, without receiving the influence of the part.

Embodiments of the present invention will be described below with reference to the drawings.
1 to 4 show a first embodiment of the present invention.
FIG. 2 is an explanatory diagram showing a state in which a plurality of TCPs 1 as electronic components are mounted at a predetermined interval on a peripheral portion of a liquid crystal display panel 2 as a substrate. The TCP 1 is punched from the carrier tape 3 and then mounted on the peripheral portion of the liquid crystal display panel 2 via the anisotropic conductive tape 4 by a mounting device (not shown).

  When the TCP 1 is mounted on the liquid crystal display panel 2, the alignment mark 6 (shown in FIG. 3) formed on the TCP 1 is imaged by an imaging camera 7 (shown in FIG. 1) as imaging means, The position of TCP1 is recognized.

On the other hand, the liquid crystal display panel 2 is provided with an alignment mark (not shown) at a position where the TCP 1 is mounted. The alignment mark is imaged by the imaging camera 7 or another imaging camera, and the liquid crystal display is performed by the imaging signal. The position of panel 2 is recognized.
Then, after the TCP 1 is positioned with respect to the liquid crystal display panel 2 based on the recognized positional information of the TCP 1 and the liquid crystal display panel 2, the TCP 1 is mounted.

  The alignment mark 6 formed on the TCP 1 has, for example, a ring shape as shown in FIG. 3, and the alignment mark 6 is deformed or scratched when the TCP 1 is punched from the carrier tape 3. An abnormal part 6a may occur. If the abnormal part 6a occurs, the alignment mark 6 may not be reliably recognized by the imaging camera 7, and in this case, the TCP 1 may be determined as a defective product.

  Therefore, even if there is an abnormal portion 6a in a part of the alignment mark 6 formed on the TCP 1, the alignment mark 6 can be recognized by the image recognition device 11 shown in FIG. The image recognition device 11 includes an image processing device 12. The image processing apparatus 12 includes an A / D converter 13 that converts an imaging signal from the imaging camera 7 into a digital signal. The digital signal converted by the A / D converter 13, that is, the inspection image X (shown in FIG. 4A) corresponding to the imaged alignment mark 6 is stored in the inspection image memory 14.

  The image processing apparatus 12 is provided with a reference image memory 15. In this reference image memory 15, a reference image Y (shown in FIG. 4B) corresponding to the normal state, that is, the shape of the alignment mark without the abnormal portion 6a is stored as a digital signal.

  A control device 16 is connected to the image processing device 12 so that the reference image memory 15 can store the digital signal of the alignment mark in a normal state, that is, the reference image memory Y. It has become.

  The inspection image X and the reference image Y stored as digital signals in the inspection image memory 14 and the reference image memory 15 are output to the comparison unit 17 where they are compared as will be described later. The comparison result in the comparison unit 17 is determined by the determination unit 18 and displayed on the monitor 19.

  The comparison unit 17 compares the inspection image X with the reference image Y based on a command from the control device 16. That is, the entire areas of the inspection image X and the reference image Y are divided into first to ninth areas, which are nine areas corresponding to each other as shown in FIGS. The areas of the images X and Y are X1 to X9 and Y1 to Y9. It is assumed that the fourth area X4 and the eighth area X8 of the inspection image X are images corresponding to the part where the abnormal portion 6a of the alignment mark 6 is generated, and these areas X4 and X8 are crossed. Shown with a line.

  Then, the control device 16 sequentially inputs the areas divided from the inspection image memory 14 and the reference image memory 15 to the comparison unit 17 as follows. First, the first area X1 of the inspection image X is compared with the first area Y1 of the reference image. If it is recognized that these areas X1 and Y1 match, then the second areas X2 and Y2 are added to the first areas X1 and Y1, and the first and second areas of the images X and Y are added. The areas (X1 + X2) and (Y1 + Y2) are compared. If it is recognized that they match, the third areas X3 and Y3 are added to it, and (X1 + X2 + X3) and (Y1 + Y2 + Y3) are compared.

  If the images up to the third area of the images X and Y match, the fourth areas X4 and Y4 are added and compared. Here, since the fourth area X4 of the inspection image X is an area corresponding to the abnormal portion 6a of the alignment mark 6, addition of the first to fourth areas obtained by adding the inspection image X and the reference image Y is performed. The images (X1 + X2 + X3 + X4) and (Y1 + Y2 + Y3 + Y4) do not match.

  If the inspection image X and the reference image Y up to the added fourth area do not match, the fourth images X4 and Y4 are removed (masked) from the comparison target, and the first to third images are changed to the fifth. The images X5 and Y5 are added and compared. That is, the area (X1 + X2 + X3 + X5) to which the inspection image X is added is compared with the area (Y1 + Y2 + Y3 + Y5) to which the reference image Y is added.

If the comparisons match, X6 and Y6 are added and compared, and if they match, X7 and Y7 are added and compared.
Next, X8 and Y8 are added and compared, and the eighth area X8 of the inspection image X corresponds to the abnormal portion 6 a of the alignment mark 6. Therefore, the image of the area (X1 + X2 + X3 + X5 + X6 + X7 + X8) to which the inspection image X is added and the area (Y1 + Y2 + Y3 + Y5 + Y6 + Y7 + Y8) to which the reference image Y is added do not match. If they do not match, the eighth images X8 and Y8 are removed from the comparison target, and the next ninth images X9 and Y9 are added together to add the inspection image X area (Y1 + Y2 + Y3 + Y5 + Y6 + Y7 + Y9) and the reference image Y area ( Y1 + Y2 + Y3 + Y5 + Y6 + Y7 + Y9).

  The result of the comparison is input to the determination unit 18. The determination unit 18 calculates a ratio between the number of times of matching and the number of times of mismatching with respect to the total number of comparisons, and if the ratio of mismatching is a predetermined value or less, for example, 30% or less, the inspection image X is aligned. The mark 6 is determined.

  Therefore, based on the determination result, the position information of the alignment mark 6 imaged by the imaging camera 7 is input to the mounting apparatus, and the TCP 1 is positioned and mounted with respect to the liquid crystal display panel 2 based on the position information. become.

  In this way, the inspection image X based on the imaging signal of the alignment mark 6 from the imaging camera 7 is divided into a plurality of areas, while the reference image Y corresponding to the alignment mark 6 is a plurality corresponding to the inspection image X. Are compared, and the areas of each image are compared while being sequentially added. If a mismatch occurs, the area in which the mismatch has occurred is removed from the comparison target, the next area is added, and the comparison is continued. Finally, the inspection image X is determined by the ratio of the number of mismatches to the number of comparisons. It is determined whether the image is the image of the alignment mark 6 or not.

  Therefore, even if there is an abnormal portion 6a in a part of the alignment mark 6, it is possible to prevent the TCP 1 from being determined to be defective because the alignment mark 6 cannot be recognized by the abnormal portion 6a. In addition, since it is determined whether or not the inspection image X is the alignment mark 6 based on the ratio of mismatch with respect to the number of comparisons of each area between the inspection image X and the reference image Y, the reliability of the determination result depends on the ratio. Can increase the sex.

  Further, the divided areas of the inspection image X and the reference image Y are not compared one by one, but are compared while being sequentially added, and if an inconsistent area appears in the middle, the area is compared. The remaining areas were added and compared. Therefore, the inspection image X close to the alignment mark 6 is finally compared with the reference image Y, which can also improve the recognition accuracy.

  5 to 8 show a second embodiment of the present invention. The configuration is the same as that of the image recognition device 11 used in the second embodiment, but the control by the control device 16 and the shape of the alignment mark and the way of division are different from those of the first embodiment.

  As shown in FIG. 5, the alignment mark 60 of this embodiment is circular, and is provided on the substrate 1A, not an electronic component, for example. The alignment mark 60 provided on the substrate 1A is hardly deformed as in the case of providing it on the TCP 1, but an abnormal portion 60a may be generated due to scratches or adhesion of dust.

  The inspection image L corresponding to the alignment mark 60 and the reference image M corresponding to the alignment mark 60 without the abnormal portion 60a are each divided into first to fourth areas as shown in FIG. Then, the comparison unit 17 performs comparison as will be described later. The first to fourth areas of the divided images L and M are denoted by L1 to L4 and M1 to M4.

  If the alignment mark 60 has an abnormal portion 60a, the abnormal portion 60a is displayed in the corresponding area on the inspection image L. In this embodiment, it is assumed that the abnormal portion 60a is displayed in the image of the fourth area L4, and the area L4 is indicated by a cross line.

  In the comparison unit 17, first of the four divided areas of the inspection image L stored in the inspection image memory 14 and the reference image M stored in the reference image memory 15 according to a command from the control device 16. In the first stage, as shown in FIG. 7A, the images (L2, L3, L4) and (M2) of the remaining second to fourth areas excluding the first areas L1, M1 of the images L, M. , M3, M4) are compared. In this case, the image to be compared includes the fourth area L4 of the inspection image L. Therefore, since the inspection image L and the reference image M do not match, the inspection image L cannot be recognized.

  In the second stage, as shown in FIG. 7B, among the inspection image L and the reference image M, the three areas of the images (L1, L3, L4) excluding the second areas L2, M2 and (M1, M3, M4) are compared. Also in this case, since the inspection image L includes the fourth area L4, the inspection image L does not coincide with the reference image M, and therefore the inspection image L cannot be recognized.

  In the third stage, as shown in FIG. 7C, among the inspection image L and the reference image M, the images of three areas (L1, L2, L4) excluding the third areas L3, M3 and (M1, M2, M4) are compared. Also in this case, since the fourth area L4 of the inspection image L is included, the inspection image L does not match the reference image M and cannot be recognized.

  In the fourth stage, as shown in FIG. 7D, among the inspection image L and the reference image M, images (L1, L2, L3) and (M1, M2, M3). In this case, since the fourth area L4 corresponding to the abnormal portion 60a of the inspection image L is not included, the inspection image L matches the reference image M. Thereby, the reference image L can be recognized.

  Therefore, the determination unit 18 determines that the inspection image L is the image of the alignment mark 60 based on the recognition result of the abnormal image 60a, that is, the inspection image excluding the fourth areas L4 and M4 and the reference image. The position of the base alignment mark 60 is detected, and the substrate is positioned based on the position information.

  FIG. 8 shows a flowchart for recognizing the inspection image L of the second embodiment. In S1, the inspection image L and the reference image M are each divided into N areas. In S2, among the N images of the images L and M, (N-1) area images obtained by sequentially removing one area at the corresponding position are compared.

  In S3, for each comparison performed in S2, it is determined whether or not the images in the (N−1) areas of the inspection image L and the reference image M match, that is, whether or not the inspection image L has been recognized. The If recognized, the recognition is successful as indicated by S4, and the substrate 1A is positioned based on the inspection image L.

  If the inspection image L cannot be recognized in S3, the number of comparisons is checked as shown in S5. If the number is N or less, the process returns to S2 and the comparison is repeated. When the comparison was performed N times, the inspection image L could not be recognized even when all the (N-1) areas except for one area were sequentially compared among the N areas. Therefore, the recognition fails as shown in S6.

  That is, when there is an abnormal portion 60a in a part of the alignment mark 60 corresponding to the inspection image L, it can be recognized by performing a maximum of N comparisons. However, the degree of abnormality of the alignment mark 60 is If the size extends over each area or the inspection image L is different from the alignment mark 60, the recognition fails.

  The present invention is not limited to the above-described embodiments. For example, the shape of the alignment mark is not limited to the ring shape or the circular shape, and may be other shapes. Further, in the first embodiment, the case where the electronic component is positioned is described, and in the second embodiment, the case where the substrate is positioned is described. However, the first embodiment is applied to the positioning of the substrate. The second embodiment can also be applied to positioning of electronic components.

  Moreover, the number of division | segmentation with a test | inspection image and a reference | standard image is not limited, What is necessary is just plural. Further, the shapes of the inspection image and the reference image are not limited to a ring shape or a circular shape, and may be other shapes such as a rectangular shape or a cross shape.

  In the first embodiment and the second embodiment, first, the positioning marks are collectively imaged by the imaging camera. As a result, when the positioning marks cannot be recognized, the image recognition according to the present invention is applied next. You may do it.

1 is a block diagram of an image recognition apparatus showing a first embodiment of the present invention. The perspective view which shows liquid crystal display panel and TCP mounted in this liquid crystal display panel. The figure which shows the alignment mark in which the abnormal part produced in part. The figure which divided | segmented the test | inspection image and the reference | standard image into each of nine areas. The figure of the alignment mark in which the abnormal part which shows 2nd Embodiment of this invention produced. The figure which divided | segmented the test | inspection image and the reference image into four areas, respectively. The figure which shows the order which compares the test | inspection image divided | segmented into four areas, and a reference | standard image. The flowchart which shows the procedure of a comparison with a test | inspection image and a reference | standard image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... TCP (electronic component) 2 ... Liquid crystal display panel (board | substrate) 6,60 ... Position alignment mark, 7 ... Imaging camera (imaging means), 11 ... Image recognition apparatus, 12 ... Image processing apparatus, 14 ... Inspection image memory , 15... Reference image memory, 16... Control device, 17.

Claims (3)

An image recognition method for determining whether or not an inspection image matches a reference image,
Dividing the inspection image and the reference image into a plurality of corresponding areas, and
Multiple areas of the inspection image are added sequentially, the area of the inspection image is compared with the corresponding area of the reference image, and if there is a mismatch between the area of the inspection image and the reference image during the comparison process, the mismatch Removing the area, adding a new area and continuing the comparison,
Detecting a ratio of the number of times of non-coincidence with respect to the number of comparisons between the area of the inspection image and the reference image, and determining whether the inspection image matches the reference image when the ratio is equal to or less than a reference value; An image recognition method comprising: An image recognition device that determines whether an inspection image matches a reference image,
Imaging means for imaging the inspection image;
An inspection image memory for storing an imaging signal from the imaging means;
A reference image memory for storing a preset reference image;
A control means for dividing the inspection image and the reference image stored in each image memory into a plurality of areas and sequentially outputting them,
The areas of each image output from the control means are sequentially added and compared. If a mismatch occurs between the inspection image and the reference image in the comparison process, the mismatch area is removed and a new area is added. A comparison unit that continues the comparison,
A determination unit that detects a ratio of the number of times of non-coincidence with respect to the number of comparisons between areas of the inspection image and the reference image, and determines whether the inspection image matches the reference image when the ratio is equal to or less than a reference value And an image recognition apparatus. Mounting that takes an inspection image formed on an electronic component, determines whether the inspection image matches a reference image stored in an image recognition device, and mounts the electronic component on a substrate based on the determination result A device,
The mounting apparatus according to claim 2, wherein the image recognition apparatus has the configuration described in claim 2.

Images