JP2000213921A - Device and method for recognising image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance recognition precision and an operation rate for a device. SOLUTION: An image of a recognition mark of a recognition object is uptaken by a camera 11, a resulting image data is stored in an image storing part 30a, the image data is pattern-matched with one of plural reference patterns stored preliminarily in a pattern storing part 30b and different respectively in views of the recognition mark so as to find a matching rate, and the quality of the found matching rate is determined. When the matching rate is rejected, the pattern matching hereinbefore is conducted in order with respect to other reference patterns, and a position of the recognition mark is detected based on a collated result with respect to the reference pattern having a prescribed value or more of matching rate. A recognition error caused by difference in view of the same object is reduced, thereby enhances recognition precision.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognizing apparatus and an image recognizing method for recognizing an image by capturing an image of an object to be recognized.

[0002]

2. Description of the Related Art In the field of manufacturing electronic components and equipment, a method of capturing an object to be recognized as an image and performing image recognition for the purpose of inspection, position detection, and the like is often used. As one of the image recognition techniques, pattern matching is widely used. In this pattern matching, a typical image of the recognition target is stored in advance as a reference pattern, the image of the recognition target acquired by the camera is compared with the reference pattern, and a matching ratio is obtained to obtain a matching ratio. It is to determine whether or not there is. This pattern matching is widely applied to position detection of a substrate or an electronic component, and a recognition mark for position detection is formed on the substrate or the electronic component.

[0003]

However, when a recognition mark formed on a flexible substrate such as a flexible substrate is an object to be recognized, there are the following difficulties in image recognition. First, in order to obtain a good image of the recognition target, illumination around the recognition target is performed, but when the surface on which the recognition mark is formed is bent, the direction in which the illumination light is reflected is Depending on the state of bending, a large difference in brightness occurs on an image obtained by imaging with a camera. For this reason, when the obtained image data is subjected to pattern matching with a reference pattern, the matching rate may be low even though the mark is originally a correct recognition mark, and a recognition error may occur.

[0004] Further, even if the recognition mark itself is the same, if there is a difference in the background around the recognition mark on the image, or further, if the surface of the recognition target object changes over time (for example, A similar recognition error may also occur in the case of oxidation of the metal surface. As described above, in the conventional image recognition device, even if the recognition mark itself is the same, the recognition accuracy is low due to a recognition error caused by a different appearance on a captured image, and adjustment of illumination or the like is performed each time. This is one of the factors that lowers the operation rate of the apparatus.

Accordingly, an object of the present invention is to provide an image recognition apparatus and an image recognition method capable of improving recognition accuracy and improving the operation rate of the apparatus.

[0006]

According to a first aspect of the present invention, there is provided an image recognition apparatus comprising: a camera for capturing an image of an object to be recognized; an image storage unit for storing image data acquired by the camera; A pattern storage unit that stores a plurality of reference patterns different from each other, an image processing unit that recognizes the recognition target by performing pattern matching of the stored image data with the plurality of reference patterns, A determination unit for performing a pass / fail determination of the matching rate determined by the unit and sequentially instructing pattern matching with another reference pattern if the matching rate is rejected.

According to a second aspect of the present invention, the image of the object to be recognized is captured by a camera, the acquired image data is stored in an image storage unit, and the appearance of the object to be recognized previously stored in a pattern storage unit is stored. Is obtained by performing pattern matching between one of a plurality of reference patterns different from each other and the image data to obtain a matching ratio, and determining whether the obtained matching ratio is acceptable. If the matching ratio is rejected, the pattern with another reference pattern is sequentially determined. Perform matching.

According to the invention described in each claim, a plurality of reference patterns having different appearances for the same object to be recognized are stored in advance, and if a recognition error occurs by pattern matching with one reference pattern. By sequentially performing pattern matching with another reference pattern, it is possible to reduce a recognition error caused by a mere difference in appearance of the same recognition target.

[0009]

Embodiments of the present invention will now be described with reference to the drawings. 1 is a side view of a bonding apparatus according to an embodiment of the present invention, FIG. 2 is a plan view of the same substrate and sub-substrate, FIGS. 3, 4, and 5 are partial cross-sectional views of the bonding apparatus, and FIG. FIG. 7 is a block diagram of a control system of the bonding device, FIG. 7 is an image diagram of the reference pattern, FIG. 8 is a flowchart showing an operation of the bonding device, and FIG. 9 is a flowchart of the image recognition process.

First, the structure of a bonding apparatus in which the image recognition apparatus of the present invention is incorporated will be described with reference to FIG. In FIG. 1, a base 1 is provided with a first XYθ table 2. The first XYθ table 2 is an X table 2A,
It is configured by stacking a Y table 2B and a θ table 2C. The main board holding table 3 is mounted on the θ table 2C, and the main board holding table 3 is
Is absorbed and held.

At the end opposite to the first XY table 2 on the base 1, a second XYθ table 5 is provided. The second XYθ table 5 is configured by stacking an X table 5A, a Y table 5B and a θ table 5C, and a sub board holding table 6 is mounted on the θ table 5C. The sub-board holding table 6 sucks and holds the sub-board 7 to which the flexible board 8 is connected in advance.

First XYθ table 2 and second XYθ table
By driving the θ table 5, the end of the flexible substrate 8 can be aligned with the edge of the main substrate 4. An imaging section 9 is provided between the first XYθ table 2 and the second XYθ table 5. Base 1
A camera 11 is mounted on a camera XY table 10 as a camera moving means provided above. A lens barrel 14 provided in the camera 11 is provided with a coaxial illumination unit 12 and a ring illumination unit 13. By driving the camera XY table 10, the camera 11 moves in the horizontal direction, and the lower surface of the flexible substrate 8 is illuminated by the coaxial illumination unit 12 and the ring illumination unit 13. Is imaged.

An XY table 15 is provided above the main board 4. XY table 15 is X table 15A
And a Y table 15B, and a moving block 16 is mounted on the lower surface of the X table 15A. A crimping head 17 having a crimping tool 18 is fixed to the moving block 16, and a lower end of a rod-shaped bracket erected on the moving block 16 is provided with a transmitted light unit 19.
Is installed. When the transmission illumination unit 19 is positioned above the edge of the main substrate 4 and illuminates the main substrate 4 from above, and the camera 11 captures an image from below the main substrate 4, the recognition mark on the upper surface of the main substrate 4 is captured. . The flexible substrate 8 is bonded to the main substrate 4 by pressing the pressing tool 18 against the end of the flexible substrate 8 in a state where the end of the flexible substrate 8 is aligned with the edge of the main substrate 4.

Next, the details of the bonding portion between the main substrate 4 and the flexible substrate 8 will be described with reference to FIG. In FIG. 2, a large number of terminal electrodes 20 are provided at the edge of the main substrate 4.
Are formed in a row to form a terminal row 20A. On both sides of each terminal row 20A, the A of the first recognition mark 22, which is a recognition object for position recognition of the main board 4, is provided.
A point and a B point are provided. On the upper surface of the main substrate 4,
A tape-shaped anisotropic conductive material 21 is attached along the edge. Here, the anisotropic conductive material 21 should be originally adhered only to the area of the terminal row 20A, but a method is adopted in which a continuous tape is applied collectively to the plurality of terminal rows 20A at the time of application. Therefore, the anisotropic conductive material 21 is not stuck on the first recognition mark 22 shown at the top in FIG. 2, and the anisotropic conductive material 21 is placed on the first recognition mark 22 at other positions. Is affixed.

On the lower surface of the flexible substrate 8, the main substrate 4
Similarly, the terminal row 23A is constituted by a large number of terminal electrodes 23 corresponding to the arrangement pitch of the terminal row 20A.
On the lower surface of the flexible board 8 on both sides of the terminal row 23A, a second recognition mark 24, which is a recognition target for position recognition of the flexible board 8, is formed.

By recognizing the first recognition mark 22 and the second recognition mark 24, the positions of the main board 4 and the flexible board 8 are recognized. Therefore, the first recognition mark 22 and the second recognition mark 24
By recognizing this, the amount of displacement between the terminal rows 20A and 23A can be obtained, and by driving the second XYθ table 5 based on the amount of displacement, the terminal rows 23A are connected to the terminal rows 20A. Accurate alignment can be achieved.

Next, the imaging section 9 will be described with reference to FIG. In FIG. 3, the camera 11 mounted on the XY table 10 has a lens barrel 14. A coaxial illumination unit 12 is provided on a side surface of the lens barrel 14, and when the coaxial illumination unit 12 emits light, the illumination light is reflected upward by the half mirror 12a, and an object located above the camera 11 is coaxial. Illuminate from the direction (arrow a). A ring illumination unit 13 is attached to the upper end of the lens barrel 14 so as to surround the lens barrel 14 in a ring shape. The target above the camera 11 is illuminated obliquely from below (arrow b) by causing the ring illumination unit 13 to emit light.

With the imaging unit 9 positioned at the imaging position A shown in FIG. 3 and the coaxial illumination unit 12 and the ring illumination unit 13 emitting light, the camera 11 is moved to the second position on the lower surface of the flexible substrate 8.
The recognition mark 24 is imaged. In addition, the image pickup unit 9 is positioned at the image pickup position B, and the main board 4 is imaged by the camera 11 in a state where the transmissive illumination unit 19 is positioned above the image pickup position B. The first recognition mark 22 (FIG. 2) is imaged. In FIG. 3, although the flexible substrate 8 overlaps the main substrate 4 at the imaging position B, the flexible substrate 8 does not exist at the position of the first recognition mark 22, and thus does not hinder the transmitted illumination.

Next, the configuration of the control system of the bonding apparatus will be described with reference to FIG. In FIG.
Is connected to the image processing unit 30. Image processing unit 30
Performs pattern matching of the acquired image data with a plurality of reference patterns having different appearances of the recognition marks stored in advance, thereby obtaining the first recognition marks 22.
And the second recognition mark 24 is recognized to detect the position.

The image processing unit 30 includes an image storage unit 30a
And a pattern storage unit 30b. The image storage unit 30a stores image data obtained by the camera 11, and the pattern storage unit 30b stores the plurality of reference patterns.

The coaxial illumination unit 12, the ring illumination unit 13, and the transmission illumination unit 19 are connected to an illumination controller 31. The lighting controller 31 controls ON-OFF and brightness of each lighting unit according to a command from a control unit described later. The control unit 32 is a CPU, which controls the operation of the entire bonding apparatus, performs position correction calculation, and the like, and performs a matching rate pass / fail determination process as a determination unit. That is, it is determined whether or not the matching ratio obtained by the image processing unit 30 is equal to or greater than a predetermined value. The storage unit 33 stores position coordinate data and position correction data of the detected recognition mark.

First XYθ table 2, second XYθ table 5, XY table 15, camera XY table 10
Is driven by the motor drive unit 34. Monitor 35
Displays a notification such as a recognition error display and an operation / input screen. The operation / input unit 36 is a keyboard, and performs an input operation of a general operation command or a new reference pattern.

6, the camera XY table 10, camera 11, image processing unit 30, ring illumination unit 13, coaxial illumination unit 12, transmission illumination unit 19, illumination controller 31, control unit 32, and storage unit. 33 constitutes an image recognition device.

This bonding apparatus is configured as described above, and the operation will be described below with reference to the flow chart of FIG. First, in FIG. 1, the main substrate 4 is placed on the main substrate holding table 3, and the camera 11 captures an image of points A and B of the first recognition mark 22 of the main substrate 4. At this time, as shown in FIGS. 4 and 5, the transmission illumination unit 19 is positioned on the first recognition mark 22 to be imaged, and the camera 11 is positioned below the first recognition mark 22. The points A and B of the first recognition mark 22 in a state where the transmitted illumination light is irradiated
Points are sequentially imaged (ST1).

FIG. 4 shows a state in which the anisotropic conductive material 21 is attached on the first recognition mark 22, and FIG. 5 shows a state in which the anisotropic conductive material 21 is not attached. FIG.
FIG. 4 corresponds to the first recognition mark 22 at the uppermost portion shown in FIG. 2, and FIG. 4 corresponds to the other first recognition mark 22. As described above, even when the first recognition mark 22 that is the same recognition target is imaged under the same illumination condition, the obtained image depends on whether the anisotropic conductive material 21 is attached or not. Are observed with different lightness.

Next, based on the image data obtained by the imaging, the recognition process of the points A and B of the first recognition mark 22 is performed (ST2), and the positional deviation of the main substrate 4 is obtained from the recognition result (ST3). ). Then, the first XYθ table 2 is driven to correct the position of the main substrate 4 based on the position shift detection result (ST4).

Thereafter, the second XYθ table 5 is driven so that the end of the flexible substrate 8 is overlapped with the edge of the main substrate 4, and the sub substrate 7 is provisionally positioned with respect to the main substrate 4. Next, as shown in FIG. 3, the camera XY table 10 is driven to position the camera 11 below the second recognition mark 24 on the flexible substrate 8, and the point A of the second recognition mark 24,
The point B is imaged (ST6). Next, the obtained image data is recognized (ST7), and the position of the second recognition mark 24 is detected. Then, the positional deviation of the flexible substrate 8 with respect to the main substrate 4 is obtained (ST8), and based on this positional deviation detection result, the second XYθ table 5 is driven to align the sub substrate 7 with the main substrate 4 (ST9). . Thereafter, the XY table 15 is driven to move the pressure bonding head 17 over the edge of the main substrate 4, and the pressure bonding tool 18 is lowered to press the flexible substrate 8 against the main substrate 4. Thereby, the flexible substrate 8 is bonded to the main substrate 4 via the anisotropic conductive material 21 (ST10).

Next, the recognition process (ST7) in the above flow will be described with reference to FIG. Here, a plurality of types (here, six types) of the reference patterns of the second recognition marks 24, which are objects, are stored in advance in the pattern storage unit 30a.
These reference patterns are sequentially subjected to pattern matching with image data acquired by imaging. The use of a plurality of reference patterns in this manner is because the flexible substrate 8 is easily bent and the tendency of the bending does not have a rule, so that the images taken by the individual flexible substrates 8 are different and a single reference pattern is used. This is because recognition errors frequently occur when only the above is used.

In this embodiment, six types of reference patterns as shown in FIG. 7 are used. FIG. 7A shows an image of the second recognition mark 24 in a state where the flexible substrate 8 is kept horizontal without substantially bending. On the other hand, FIGS. 7B, 7C, 7D, and 7E show shadows in the left, right, upper, and lower portions of the image, respectively, due to the bending direction of the flexible substrate 8. The state is shown.
FIG. 7F shows that although the flexible substrate 8 itself does not bend and no shadow is generated on the image of the second recognition mark 24, the surface of the second recognition mark 24 is deteriorated by oxidation or the like, and An example is shown in which an image having a different brightness due to a change in the reflection state is obtained. In this way, a plurality of types of reference patterns that can be normally assumed are prepared in advance, and are sequentially subjected to pattern matching, so that the recognition marks for the camera 11 can be recognized despite the fact that the recognition marks themselves are normal. It is possible to prevent a situation in which a recognition error occurs due to a different appearance.

In FIG. 9, first, index Nos. Assigned to a plurality of reference patterns are set. (I) is set to 1 (ST11). Next, the reference pattern N
o. (I) is compared with the image of the point A of the second recognition mark 24 by pattern matching (ST12). Then, the control unit 32 determines whether the matching ratio is acceptable or not, that is, whether the obtained matching ratio is 80% or more (ST13). Here, if the matching rate is less than 80%, it is determined to be rejected, and pattern matching is performed using another reference pattern according to a command from the control unit 32. That is, the index No. It is determined whether or not (i) is less than 6 (ST14).
That is, if there is another unused reference pattern, 1 is added to (i) (ST15), and the process returns to ST12 to set a new index No. Pattern matching is performed using the reference pattern of. Then, in ST14, the index No. If is 6, the process proceeds to ST23 described below, and recognition error processing is performed. If the matching rate is 80% or more in ST13, the position of the point A of the second recognition mark 24 specified by the reference pattern is stored (ST16).

Next, for the point B of the second recognition mark 24, the above-described recognition processing for the point A (ST11 to ST11) is performed.
As in 16), the recognition processing in ST17 to ST22 is performed. Then, NO is determined in ST14 and ST20,
That is, if it is determined that there is no reference pattern that provides a matching rate of 80% or more, recognition error processing is performed (ST23). Here, the monitor 35 is notified of the recognition error, and the operator receives the notification and adjusts the lighting condition and registers a new reference pattern.

In this embodiment, the second recognition mark 24
Although only a plurality of reference patterns are described for the first recognition mark 22, a plurality of reference patterns are similarly used for the first recognition mark 22. As mentioned above,
This is because the appearance of an image captured under transmitted illumination differs depending on whether or not the anisotropic conductive material 21 is attached to the upper surface of the recognition mark 22. Therefore, a plurality of reference patterns corresponding to this appearance are prepared in advance, and pattern matching is sequentially performed with these reference patterns, whereby a recognition error can be reduced.

The present invention is not limited to the above-described embodiment, and provides different recognition results due to different appearances on an image such as lighting conditions and background conditions while maintaining the same identity. It can be applied to various recognition targets.

[0034]

According to the present invention, a plurality of reference patterns having different appearances for the same object are stored in advance, and if a recognition error occurs, pattern matching is sequentially performed with another reference pattern. Therefore, it is possible to reduce recognition errors caused by the fact that the same object simply looks different, improve recognition accuracy, and improve the operation rate of the image recognition device.

[Brief description of the drawings]

FIG. 1 is a side view of a bonding apparatus according to an embodiment of the present invention.

FIG. 2 is a plan view of a substrate and a sub-substrate according to an embodiment of the present invention.

FIG. 3 is a partial cross-sectional view of the bonding apparatus according to one embodiment of the present invention;

FIG. 4 is a partial sectional view of the bonding apparatus according to one embodiment of the present invention;

FIG. 5 is a partial sectional view of the bonding apparatus according to the embodiment of the present invention;

FIG. 6 is a block diagram of a control system of the bonding apparatus according to the embodiment of the present invention;

FIG. 7 is an image diagram of a reference pattern according to an embodiment of the present invention.

FIG. 8 is a flowchart showing the operation of the bonding apparatus according to the embodiment of the present invention;

FIG. 9 is a flowchart of an image recognition process according to an embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 4 main board 8 flexible board 9 imaging section 11 camera 22 first recognition mark 24 second recognition mark 30 image processing section 30a image storage section 30b pattern storage section 32 control section 33 storage section

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 2F065 AA03 AA20 BB27 CC25 DD00 EE00 FF02 FF04 FF42 FF61 GG17 HH12 HH13 HH14 HH15 JJ03 JJ09 LL00 PP11 PP12 PP13 QQ23 QQ24 QQ26 QQ38 RR09 5B007 AA03 DA11

Claims (2)

[Claims] 1. A camera for capturing an image of an object to be recognized, an image storage unit for storing image data acquired by the camera, and a pattern storage for storing a plurality of reference patterns each having a different appearance of the object to be recognized. Unit, an image processing unit for recognizing the recognition target by performing pattern matching of the stored image data with the plurality of reference patterns, and performing a pass / fail determination of a matching rate determined by the image processing unit. An image recognition apparatus comprising: a determination unit that sequentially instructs pattern matching with another reference pattern if the rate is unacceptable. 2. An image of a recognition target is captured by a camera, and the obtained image data is stored in an image storage unit, and a plurality of reference patterns stored in a pattern storage unit in advance, each of which has a different appearance of the recognition target. And the image data is subjected to pattern matching to obtain a matching ratio,
An image recognition method, comprising: determining whether the obtained matching ratio is acceptable or not, and if the matching ratio is rejected, sequentially performing pattern matching with another reference pattern.