JP4015423B2 - Image recognition apparatus and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image recognition apparatus and a program for recognizing a nearby mark having a high recognition rate and adding a relative coordinate of a target to be known to the recognized position coordinate to obtain a recognition result.
[0002]
[Prior art]
Conventionally, it has been necessary to accurately control a mechanism for automatic check (inspection) of a circuit board, mounting of components, and the like. In general, it is difficult to obtain sufficient accuracy for the mechanism itself. For this reason, a method for improving the final positioning accuracy by capturing a mark with a camera attached to a robot hand or the like, estimating the position coordinates by image recognition, and correcting the mechanical control based on the information. Was used.
[0003]
Such image recognition requires processing for determining which part of an image corresponds to which part of another image. Such processing is called pattern matching and includes matching between images and matching between figures. Furthermore, it is a matching problem to check where a pattern corresponding to a specific object exists in the image and recognize the object. In particular, the pattern of the object is represented as an image, and the operation for obtaining the position of the object by examining the similarity between the pattern and each part of the object image is called template matching, and is used in image processing. It is the most basic and often used technique for matching.
[0004]
FIG. 10 is an explanatory diagram of a conventional example. FIG. 10 illustrates a conventional template matching, in which a template (1) and an image A are provided. In the template matching, a template (1) representing an object to be detected is overlapped at a point (i, j) in the image A, and the similarity between the template (1) and a partial pattern of the overlapping image is determined. This is a method of measuring and setting the value as a probability that the object exists at the point (i, j). In order to obtain the position of the object, this operation is performed on all the points of the image, and the position where the similarity is maximum is obtained.
[0005]
[Problems to be solved by the invention]
In the conventional apparatus, it is difficult to recognize an object having a low recognition rate by image recognition.
[0006]
The present invention does not directly recognize an object with a low recognition rate, but recognizes a nearby mark with a high recognition rate, and adds the relative coordinates of the target that the user wants to know to the recognized position coordinates, The purpose is to greatly improve the recognition rate by obtaining the recognition result.
[0007]
[Means for Solving the Problems]
FIG. 1 is an explanatory diagram of an image recognition apparatus according to the present invention. In FIG. 1, 1a is a recognition object, 2 is a camera, 4 is a probe, 10 is a control unit, 11a is drive means, 13a is image recognition means, and 16a is storage means.
[0008]
The present invention has the following means in order to solve the conventional problems.
[0009]
(1): a camera 2 that captures the recognition target object 1a, a template of a part with a high recognition rate in a mounted component on the board near the recognition target object 1a of the image of the camera 2, and the image of the camera 2 A storage unit 16a that stores position data of the recognition object 1a; and an image recognition unit 13a that performs pattern matching on the image of the camera 2 and the template. The image recognition unit 13a Pattern matching for determining which position corresponds to the template is performed, the recognition position of the template image is determined, and the difference vector up to the recognition object 1a determined from the position data is added to the determined recognition position. For this reason, the recognition rate of the recognition target object with a low recognition rate can be improved.
[0010]
(2): In the image recognition apparatus according to (1), a plurality of templates having a high recognition rate among mounted components on a board near the recognition target object 1a of the camera image and recognition in the image of the camera 2 Storage means 16a for storing the position data of the object 1a, and the image recognition means 13a includes the template in a position in the image of the camera 2 determined based on the position data and a certain range around the position. The pattern matching matching value is obtained at each moving position, and the matching value obtained in the same manner is added for each template, and the position where the added matching value is the highest is recognized for each template image. As a position, a difference vector to the recognition object 1a obtained from the position data is added to the recognition position. For this reason, the recognition rate of the recognition target object with a low recognition rate can be improved more.
[0011]
(3) Storage means for storing a template of a part with a high recognition rate in a mounted component on the board near the recognition object 1a of the image of the camera 2 and position data of the recognition object 1a in the image of the camera 2 16a and pattern matching for determining which position of the image of the camera 2 corresponds to the template, the recognition position of the template image is determined, and the recognition object obtained from the position data at the determined recognition position A program for causing a computer to function as image recognition means for adding difference vectors up to 1a. For this reason, by installing this program in a computer, it is possible to easily provide an image recognition apparatus that can greatly improve the recognition rate of a recognition object having a low recognition rate.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
The present invention does not directly recognize the originally recognized object, but recognizes a nearby relative position that is clear and easy to recognize, and sets the difference vector of the position coordinate of the originally recognized object as the position coordinate of the recognition result. The position coordinates of the object (part) to be obtained by addition are determined.
[0013]
(1): Description of Image Recognition Device FIG. 1 is an explanatory diagram of an image recognition device. In FIG. 1, a recognition object 1a is a target lead or the like of a printed board on which an electronic component is mounted. The camera 2 is attached to a robot hand or the like and obtains an image of the recognition object 1a. The probe 4 is attached to a robot hand or the like and inspects the recognition object 1a. The control unit 10 controls the entire image recognition apparatus. The driving unit 11a includes four driving units such as an X axis, a Y axis, a Z axis, and a Z axis rotation (θ3) for driving the robot hand. The image recognition means 13a performs processing such as pattern matching. The storage means 16a stores light / dark (light / dark) data such as CAD data (position data) of the object 1 and a template.
[0014]
For example, when it is desired to recognize a part of the lead of the IC, the recognition rate decreases because the lead of the IC is a repetition of the same pattern. For this reason, a VIA (via), test pad (TP), recognition mark (RM), IC alignment mark, etc. with a high recognition rate near the object to be recognized are recognized. Then, the recognized position coordinates (recognized VIA (via), test pad, center coordinates of the position coordinates of landmarks, etc.) and the difference vector on CAD (computer design) data of the coordinate value of the target lead are added. The coordinates of the target lead are obtained. The CAD data is data indicating what kind of mark (size), IC, pad, IC lead, etc. are present on a printed circuit board or the like. Hereinafter, an image recognition apparatus applied to an automatic substrate check apparatus will be described.
[0015]
(2): Explanation of automatic substrate check {circle around (1)} Description of automatic substrate check device configuration FIG. 2 is an explanatory diagram of a substrate automatic check device configuration. In FIG. 2, the substrate automatic check device includes a printed circuit board 1, a camera 2, a hand (robot hand) 3, a probe 4, a probe pin 5, a stage 6, a motor x, a motor y, a motor z, and a motor θ3. is there. The printed circuit board 1 is a check target on which electronic components are mounted. The camera 2 is attached to the hand 3 and obtains an image of the printed circuit board 1. The hand 3 is driven by a motor for automatic check of the printed circuit board. The probe 4 is attached to the hand 3 to check the printed circuit board 1. The probe pin 5 makes a check by contacting with a contact point of the printed circuit board 1. The stage 6 is a base on which the check device is mounted. The motor x is a driving unit that moves the hand 3 in the X-axis direction. The motor y is a driving unit that moves the hand 3 in the Y-axis (perpendicular to the paper surface) direction. The motor z is a driving unit that moves the hand 3 in the Z-axis direction. The motor θ3 is a driving means for rotating the hand 3 in the Z-axis direction.
[0016]
In FIG. 2, for the sake of simplicity, an example is shown in which one hand 3 is provided, but usually a plurality of (generally, about four) hands 3 are mounted and different points on the printed circuit board 1 are measured simultaneously. It will be.
[0017]
{Circle over (2)} Description of Control Blocks of Automatic Board Checking Device FIG. 3 is an explanatory diagram of control blocks of the automatic board checking device. The control block of the automatic substrate check apparatus includes a control unit 10, a motor group driving unit 11, a camera control unit 12, an image recognition unit 13, a correction amount calculation unit 14, a measurement unit 15, and a measurement point data storage unit 16. .
[0018]
The control unit 10 performs overall control. The motor group driving unit 11 shows driving units for all the motors necessary to move one hand. For example, it consists of four motor drive units for X-axis, Y-axis, Z-axis and Z-axis rotation (θ3). The camera control unit 12 controls the camera 2 that captures an image of the printed circuit board 1. The image recognition unit 13 is image recognition means for performing image recognition by performing pattern matching between a camera image and a template. The correction amount calculation unit 14 calculates the correction amount of the hand from the image recognition result. The measuring unit 15 measures a voltage, an electric waveform, etc. with a digital sampling oscilloscope, for example. The measurement point data storage unit 16 is a set of coordinate values of places for measuring by contacting the measurement points of the printed circuit board (measurement object) 1 with the probe pins 5 or a set of data that can be converted into coordinate values ( CAD data of the printed circuit board 1) and the like are stored.
[0019]
(3): Explanation of the case where Z-axis rotation (θ3) is required In a simple case, the θ3 motor is omitted, and contact at any point is possible with only the three motors of X, Y, and Z axes. is there. However, considering the following case, a motor of θ3 is required.
[0020]
1) If there is an error around the θ3 axis when the printed circuit board 1 is mounted, the X and Y axes of the printed circuit board 1 and the X and Y axes of the camera image are aligned by rotating the hand by θ3 (in this case, image conversion is performed) (It can be said that the motor of θ3 is not indispensable because it can cope with rotation correction).
[0021]
2) When contacting with the probe pin 5 tilted to avoid an obstacle on the printed circuit board 1. In this case, when the direction inclined in advance is the same as the direction with the obstacle, the obstacle can be avoided by performing θ3 rotation.
[0022]
FIG. 4 is an explanatory diagram for inclining the probe pin, FIG. 4 (a) is an explanation when the probe pin is not inclined, and FIG. 4 (b) is an explanation when the probe pin is inclined and rotated. In FIG. 4A, when a large component 7 such as a hard disk is mounted on the printed circuit board 1, if an attempt is made to contact the probe 4 without tilting the point to be contacted, a part of the probe 4 interferes. Therefore, as shown in FIG. 4B, the hand 3 is rotated by θ3 with the probe pin 5 tilted in advance, so that interference between the large component 7 and the probe 4 occurs as in the left and right probes 4 in the figure. Can be prevented.
[0023]
3) When performing simultaneous measurement with a plurality of hands, if the contact points are close to each other, there is a risk of the hands colliding with each other. In this case, the collision may be avoided by rotating the hand.
[0024]
4) When the hand has two pins, a probe pin and a ground pin. If the tip of the probe pin coincides with the θ3 axis, both the probe pin and the ground pin can be contacted correctly if the ground pin is moved by a predetermined amount from the probe pin and the θ3 motor is moved correctly.
[0025]
(3): Explanation when VIA (via) is used FIG. 5 is an explanatory diagram when VIA is used. In FIG. 5, an image A and a template (1) are provided. The image A is an image of the camera of the printed circuit board 1 and shows an IC image having leads around it and images of a plurality of VIAs. The IC image has leads (a part of the leads are omitted) around it, and these leads are connected to a pad on the printed board. The VIA has a VIA hole (a hole (shown by a black circle) for connecting the layers of the multilayer printed board) in the center. Template {circle around (1)} is data (here, VIA pattern data) with a high recognition rate near the target object stored in advance for use in pattern matching.
[0026]
(Explanation for obtaining target lead coordinates)
From the CAD data stored in the storage unit of the image recognition apparatus, the relative positions of the leads of the plurality of VIAs and ICs can be accurately obtained. For this reason, the coordinates of the target lead are obtained by adding the difference vector on the CAD data of the center coordinate of the VIA and the coordinate value of the target lead to the position coordinates of the VIA obtained by the pattern matching method to the recognition result coordinates.
[0027]
Pattern matching is performed by moving the template forward, backward, left, and right from the position where the pattern seems to match if there is no error because the approximate position of the image portion corresponding to the template is known in advance. FIG. 6 is a processing flow for obtaining the coordinates of the target lead. Hereinafter, a description will be given according to the processes S1 to S4 of FIG.
[0028]
S1: The VIA pattern data having a high recognition rate close to the target lead is set as the light and dark data of the template (1) in the storage means of the image recognition unit 13 or the like.
[0029]
S2: The image recognizing unit 13 sets the template {circle around (1)} at the position where the pattern is considered to match if there is no error in the portion corresponding to the template {circle around (1)} and the circuit board image template {circle around (1)} of the camera image. Pattern matching is performed for all positions that are finely moved within a certain range to obtain a similarity (create a similarity matrix). Note that the range in which the template (1) is moved is a range that covers the positional deviation error. The unit to be moved is one pixel (pixel) unit.
[0030]
S3: The image recognizing unit 13 recognizes the position where the similarity is the highest (the position of the similarity matrix) as an accurate VIA image position.
[0031]
S4: Next, the difference vector (X, Y component) on the CAD data of the center coordinate of VIA and the coordinate value of the target lead is added to the position coordinate (recognition result coordinate) of VIA to the recognition result coordinate. Find the coordinates.
[0032]
(4): Explanation in the case of using a plurality of templates for pattern matching FIG. 7 is an explanatory diagram in the case of using two templates. In FIG. 7, an image A and templates (1) and (2) are provided. An image A is an image of a camera, and shows an IC image a and a VIA (via) b having leads (a part of leads are omitted) in the periphery. Templates {circle around (1)} and {circle around (2)} are data stored in advance for use in pattern matching (here, VIA and partial data of the lead pattern). In FIG. 7, templates {circle around (1)} and {circle around (2)} are shown at the same time, but are usually stored separately in storage means such as the image recognition unit 13.
[0033]
(Explanation for obtaining target lead coordinates)
In pattern matching, the approximate position of the image portion corresponding to the template is known in advance from the CAD data. Therefore, if there is no positioning error on the printed circuit board, the pattern is assumed to match, and the template is moved forward, backward, left, and right from that position. Go to and do. FIG. 8 is a processing flow for obtaining the coordinates of the target lead. In the following, description will be given according to processing S11 to S15 of FIG.
[0034]
S11: Brightness / darkness data of templates (1) and (2) having a high recognition rate close to the target lead is set in the storage means of the image recognition unit 13 or the like.
[0035]
S12: First, the image recognizing unit 13 sets the pattern to match if there is no error in the portion corresponding to the template (1) for VIA and the template (1) for the circuit board image of the camera image, and to the front, rear, left and right. Pattern matching is performed on all the positions where the template (1) has been finely moved within a certain range, and the similarity is obtained (similarity matrix is created). Note that the range in which the template (1) is moved is a range that covers the positional deviation error. The unit to be moved is one pixel (pixel) unit.
[0036]
S13: Next, the image recognizing unit 13 uses the lead pattern template (2) to move by one pixel similarly to the template (1) and perform pattern matching to obtain a similarity (template (2)). Create the similarity matrix used). This moving range is performed in the same range as the template (1) (centering on the position where the pattern matches if there is no error).
[0037]
S14: Next, the image recognizing unit 13 adds the similarity obtained using the template (1) and the similarity obtained using the template (2) at each position (same moving distance) ( A similarity matrix created using the templates (1) and (2) is added), and the difference vector between the position on the matrix where the added similarity is the highest and the center of the matrix is an error. Correction is performed only for the error.
[0038]
S15: The image recognizing unit 13 adds the difference vector to the target lead obtained from the CAD data to the center coordinates of the image corresponding to the recognized template {circle around (1)} (or {circle around (2)}), finds the coordinates of the target lead, and accurately It is recognized as a correct target lead position.
[0039]
FIG. 9 is an explanatory view of similarity addition, FIG. 9 (a) is a description of the similarity matrix of template (1), FIG. 9 (b) is a description of the similarity matrix of template (2), and FIG. ) Is a description of the similarity matrix addition of template (1) and template (2).
[0040]
The sum of the similarity matrix obtained in template (1) in FIG. 9 (a) and the similarity matrix obtained in template (2) in FIG. 9 (b) is the similarity matrix in FIG. 9 (c). It is. For example, since the central similarity in FIGS. 9A and 9B is “18” and “20”, respectively, the central similarity in FIG. 9C is “38”.
[0041]
In FIG. 9C, “51” has the highest degree of similarity, and this position is diagonally right above the center by one pixel. This position can be recognized as an accurate image position, and this displacement can be corrected to accurately position the probe or the like.
[0042]
In this way, since the reliability is low with only one recognition result, the recognition performance can be improved by recognizing a plurality of objects with a high recognition rate at one time whose mutual positional relationship is known by CAD data or the like. It can be greatly improved. That is, in order to improve the recognition result, the relative coordinates of the position coordinates of the plurality of recognition results are made to coincide with the relative coordinates (the actual position) known in advance.
[0043]
It should be noted that the template (1) and the template (2) can reduce the recognition error (because the direction in which the error is likely to appear is different) when a combination of completely different patterns is used. In addition, a large template combining the templates (1) and (2) is not practical because many programs are required for saving (the number of templates increases) and the pattern matching processing amount increases. .
[0044]
In the embodiment described above, one or two templates are used. However, the use of three or more templates can further improve the image position recognition result. In addition, although it has been described that the pixel moves by one pixel, pattern matching can be performed by moving a certain distance (especially in the case of analog). Furthermore, although similarity is used for pattern matching, distance can also be used. On the contrary to the similarity, the distance becomes smaller (shorter) as the matching degree is higher.
[0045]
(5): Description of program installation control unit 10, motor group drive unit 11, drive unit 11a, camera control unit 12, image recognition unit 13, image recognition unit 13a, correction amount calculation unit 14, measurement unit 15, measurement point data The storage unit 16, the storage unit 16a, and the like can be configured by a program, and are executed by a main control unit (CPU) and stored in the main memory. This program is generally processed by a computer. This computer is composed of hardware such as an input device as input means such as a main control unit, main memory, file device, display device, and keyboard.
[0046]
The program of the present invention is installed on this computer. In this installation, these programs are stored in a portable recording (storage) medium such as a floppy disk or a magneto-optical disk, and a drive device for accessing the recording medium provided in the computer is used. Alternatively, it is installed in a file device provided in the computer via a network such as a LAN. Then, the program steps necessary for processing are read from the file device into the main memory and executed by the main control unit.
[0047]
【The invention's effect】
As described above, the present invention has the following effects.
[0048]
(1): by the image recognition unit performs pattern matching to determine the what the position of the camera image corresponds to the template of high recognition rate portion, determines the recognized position of the template image, the recognition position that the determined Since the difference vector to the recognition target obtained from the position data is added to the position data, the recognition rate of the recognition target having a low recognition rate can be improved.
[0049]
(2): A template having a plurality of parts having a high recognition rate in the vicinity of the recognition target object of the camera image is provided, and the recognition rate is determined at and around the position of the camera image determined based on the position data by the image recognition means. Pattern matching is performed with a high part template to obtain a matching value, and further, the matching value at the same position obtained for each template is added, and the position with the highest matching value is set as the recognition position of each template image. Since the difference vector to the recognition target obtained from the position data is added to the recognition position, the recognition rate of the recognition target having a low recognition rate can be further improved.
[0050]
(3) Storage means for storing a template of a part having a high recognition rate among mounted components on a board near a recognition target object of a camera image, and position data of the recognition target object in the camera image, and the camera Pattern matching to determine which position of the image corresponds to the template, determine the recognition position of the template image, and add the difference vector to the recognition object obtained from the position data to the determined recognition position Since the image recognition means is a program for causing a computer to function, an image recognition apparatus that can improve the recognition rate of a recognition object having a low recognition rate can be easily provided by installing this program in the computer. can do.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram of an image recognition apparatus according to the present invention.
FIG. 2 is an explanatory diagram of a substrate automatic check device configuration according to the embodiment.
FIG. 3 is an explanatory diagram of a control block of the automatic substrate checking apparatus according to the embodiment.
FIG. 4 is an explanatory diagram for inclining a probe pin in the embodiment.
FIG. 5 is an explanatory diagram in the case of using the VIA in the embodiment.
FIG. 6 is a processing flow for obtaining the coordinates of a target lead in the embodiment.
FIG. 7 is an explanatory diagram when two templates are used in the embodiment.
FIG. 8 is a processing flow for obtaining target lead coordinates in the embodiment;
FIG. 9 is an explanatory diagram of similarity addition in the embodiment.
FIG. 10 is an explanatory diagram of a conventional example.
[Explanation of symbols]
1a Recognition object 2 Camera 4 Probe 10 Control unit 11a Drive means 13a Image recognition means 16a Storage means

Claims (3)

A camera that captures the recognition object;
A storage means for storing a template of a part having a high recognition rate in a mounting component on a substrate near a recognition target object of the camera image, and position data of the recognition target object in the camera image;
Image recognition means for performing pattern matching with the image of the camera and the template;
The image recognition means performs pattern matching to determine which position of the image of the camera corresponds to the template, determines a recognition position of the template image, and the recognition obtained from the position data at the determined recognition position An image recognition apparatus characterized by adding a difference vector to an object. A plurality of templates having a high recognition rate in a mounting component on a board near a recognition object of the camera image, and storage means for storing position data of the recognition object in the camera image;
The image recognition means moves the template within a certain range around the position in the image of the camera determined based on the position data, and obtains a matching value of the pattern matching at each moving position, The matching value obtained in the same manner for each template is added, the position where the added matching value is the highest is set as the recognition position of each template image, and the difference to the recognition object obtained from the position data at the recognition position The image recognition apparatus according to claim 1, wherein vectors are added. A storage means for storing a template of a part having a high recognition rate in a mounting component on a board near a recognition target object of a camera image and position data of the recognition target object in the camera image;
Perform pattern matching to determine which position of the camera image corresponds to the template, determine a recognition position of the template image, and obtain a difference vector to the recognition object obtained from the position data at the determined recognition position A program for causing a computer to function as an image recognition means for adding.

Images