JP3575165B2 - Calibration jig for image recognition device and calibration method for image recognition device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention is used in an image capturing apparatus that recognizes an object by the image capturing apparatus, and performs calibration by comparing the positional relationship between the coordinates when the object is moved and the coordinates of the image obtained by the image capturing apparatus. The present invention relates to a calibration jig used and a calibration method.
[0002]
[Prior art]
In an image recognition device that captures an object with an image capturing device such as a camera and measures the position, shape, size, or angle of the object, the following calibration (calibration) is performed before the measurement. Work). This calibration work is to measure in advance how many pixels of the camera correspond to the actual object, how many times the camera is tilted with respect to the reference coordinate axis, and the like. Is calibrated by comparing the coordinates when the is moved with the positional relationship between the coordinates of the image of the object obtained by the camera.
[0003]
FIG. 18 shows an example of a commonly used image processing apparatus.
This image recognition device includes an XY table 1, a camera 2, a ring light 3, a servo controller 4, an image processing unit 5, and the like. The XY table 1 is movable in the X and Y directions. The ring light 3 is an illuminator that illuminates the XY table 1, and the camera 2 captures and recognizes an image of the mark jig 6 set on the XY table 1.
The conventional mark jig 6 of this type has a disk-shaped metal part 7 and a mark 8, and the metal part 7 is positioned so as to fit into the hole 1 a of the XY table 1.
By appropriately moving the mark jig 6 in the X and Y directions using the XY table 1, the position of the mark jig 6 moves with respect to the camera 2. The position of is captured as an image. The relationship between the coordinate axes of the camera 2 and the reference coordinate axes (in the X and Y directions) of the XY table 1 is obtained based on the captured images of the mark jig 6 at a plurality of locations.
[0004]
[Problems to be solved by the invention]
However, using the mark jig 6 having such a structure has the following problem.
The mark jig 6 has a structure as shown in FIGS. The metal portion 7 of the mark jig 6 has a black plated portion 7a and a mark 8 on the upper surface side, and a white plastic member 7b is disposed as a background in the center portion. A black plated mark 8 is provided at the center of the white plastic 7b for the background.
However, when such a structure is adopted, as shown in FIG. 20, even if the metal portion 7 is machined and the mark 8 is machined so as to be as round as possible by machining, the crush 8a occurs. . Also, if a white plastic 7b for the background is to be drilled by machining, burrs 7d are generated.
As a result, as shown in FIG. 23, the outline of the mark 8 is blurred, and there is a problem that the image of the mark cannot be obtained as a substantially perfect mark by the camera. I can't.
[0005]
Another conventional mark jig as shown in FIGS. 21 and 22 has the following problem. In this mark jig 6, a metal portion (positioning portion) 7 has a black-plated portion 7a on the upper surface, and a white mark 8e is affixed to the center thereof. The mark 8e is a printing mark. In this case, it is difficult to print the printing mark 8e on the center line of the metal portion 7 so that the center of the printing mark 8e is aligned with the center of the metal portion 7.
As described above, when the conventional mark jig 6 is used as a calibration mark for a high-magnification camera, it is difficult to secure a mark jig with improved processing accuracy. Very expensive.
An object of the present invention is to provide a calibration jig for an image recognition device and a calibration method for an image recognition device that can solve the above problems, can be manufactured at low cost, and can easily calibrate the correspondence between pixels and an object. The purpose is.
[0006]
[Means for Solving the Problems]
The object described above is used in an image recognition device that recognizes an object in the present invention, and performs calibration by comparing the positional relationship between the coordinates when the object is moved and the coordinates of the image obtained by the image capturing device. This is achieved by a calibration jig for an image recognition device including a positioning member having a concave portion and a substantially spherical mark body held in the concave portion of the positioning member.
In the present invention, when the image recognition device recognizes the target object, when calibration is performed by comparing the positional relationship between the coordinates when the target object is moved and the coordinates of the image obtained by the image capturing device, the concave portion of the positioning member is used. , A substantially spherical mark body is held. Thus, the image processing apparatus can easily perform the calibration work of the correspondence between the pixels and the target object by capturing and recognizing only the image of the true spherical mark body.
[0007]
The mark body has a metallic luster, and the peripheral portion defining the concave portion of the positioning member is white, and if the central portion of the mark body is reflected on the projection surface, light is reflected around the mark body. The image of the part can be clearly obtained, and the proofreading operation can be performed more reliably.
In this case, if the size of the recess is smaller than the diameter of the mark body, the presence of the recess can be ignored since the recess is hidden by the mark body.
The focus of the image pickup device is adjusted to the maximum outer diameter portion of the mark body so that the concave portion of the positioning member is hidden from the maximum outer diameter portion of the mark body, so that the presence of the concave portion of the positioning member can be ignored. .
By changing the mark body from one having metallic luster to white, the background of the mark body can be more clearly captured.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The embodiments described below are preferred specific examples of the present invention, and therefore, various technically preferable limitations are given. However, the scope of the present invention particularly limits the present invention in the following description. It is not limited to these forms unless otherwise stated.
FIG. 1 shows an example of an image recognition device provided with a calibration jig for the image recognition device of the present invention.
The image recognition device 10 includes a camera 11, a ring light 12, a table moving unit 20, a servo controller 30, an image processing unit 40, a calibration jig 50, and the like.
[0009]
As the camera 11, for example, a CCD (Charge Coupled Device) camera or the like can be adopted, and the lens barrel 11a is directed vertically downward. The ring light 12 is, for example, a ring-shaped fluorescent lamp that is turned on by a power supply 12a. The ring light 12 is located around the lens barrel 11 a of the camera 11. The ring light 12 irradiates light to the surface of the table 21 of the table moving means 20.
The table moving means 20 is means for appropriately moving and positioning the table 21 in the X and Y directions. The table moving means 20 has a table 21 and bases 22 and 23. The base 22 includes an X-axis motor 24 and a ball screw 25. The ball screw 25 meshes with a nut (not shown) provided on the base 23. When the ball screw 25 rotates by the driving of the motor 24, the ball screw 25 moves the base 23 along the linear motion guide 26. It can be positioned by moving in the direction.
[0010]
Similarly, the base 23 has a motor 27 for the Y direction and a ball screw 28. The ball screw 28 meshes with a nut (not shown) provided below the table 21. When the ball screw 28 is rotated by the driving of the motor 27, the table 21 moves in the Y direction along the linear motion guide 29. It can be moved and positioned.
In this way, the table 21 can be moved and positioned appropriately in the X and Y directions.
[0011]
The servo controller 30 controls the driving of the motors 24 and 27 described above. The servo controller 30, the image processing unit 30, and the power supply 12a are controlled by the control unit 100.
The image processing unit 40 is a device that processes an image of the calibration jig 50 on the table 21 captured by the camera 11.
The servo controller 30 sends the XY coordinate position of the table to the image processing unit 40.
[0012]
Next, referring to FIG. 2, the image recognition device 10 can be used, for example, as a mounting device for an electronic component PA. The mounting apparatus of FIG. 2 includes the above-described table moving unit 20, and the table moving unit 20 is set on the base 31. For example, a printed wiring board OB can be detachably set on the table 21 as an object. The head 32 can mount the component PA on the printed wiring board OB. The head 32 holds the component PA, for example, by vacuum suction, and the head 32 can be moved and positioned in the Z direction by the moving operation means 33 according to a command from the control unit 100.
[0013]
Next, the structure of the calibration jig 50 of FIGS. 1 and 2 will be described.
The calibration jig 50 includes a positioning member 51 and a substantially spherical mark body 52. First, as shown in FIGS. 1, 3 and 4, the positioning member 51 has a fitting portion 51a and a flange portion 53. The fitting portion 51a fits into the hole 21a of the table 21 in FIG. That is, the inner diameter of the hole 21a substantially matches the outer diameter of the fitting portion 51a. The outer diameter of the flange portion 53 is larger than the outer diameter of the fitting portion 51a, and the background member 54 is fitted and attached to the central portion thereof. The background member 54 is, for example, white plastic, and methacryl can be used as the plastic.
[0014]
On the other hand, the flange portion 53 and the fitting portion 51a are integrated, and are made of, for example, metal, and carbon steel S45C for mechanical structure can be used as the type of metal.
A concave portion 56 is formed at the center of the background member 54. The inner diameter D1 of the concave portion 56 in FIG. 3 is set smaller than the diameter L of the maximum outer diameter portion of the mark body 52 described above.
[0015]
As shown in FIG. 4, the upper surface 53a of the flange portion 53 is plated, for example, black. Therefore, the background member 54 is made of white plastic, and the upper surface 53a is plated with black, so that the ring-shaped outer contour of the background member 54 is clearly exposed by color coding in black and white.
The substantially spherical mark body 52 is placed in the recess 56 at the center of the background member 54. As the mark body 52, for example, a steel ball can be used. As this steel ball, for example, a commonly used carbon steel steel ball or stainless steel steel ball can be used. This type of steel ball has a high degree of roundness of, for example, 3 μm, whereas the roundness of a conventionally used printing mark body is only about 20 μm at most.
As this type of steel ball, for example, a steel ball for a ball bearing can be used practically.
[0016]
As shown in FIG. 3, the inner diameter D1 of the concave portion 56 is set to be smaller than the diameter L of the maximum outer diameter portion of the mark body 52 because the camera 11 of FIG. This is to prevent the concave portion 56 from being hidden behind the maximum outer diameter portion 52R of the mark body 52 and being photographed.
The reason why the background member 54 is white is that the camera 11 can capture the background member 54 in white, the outer peripheral portion 52g of the mark body 52 seen from above is specularly reflected, and the illumination light of the ring light 12 is reflected by the camera 11 By not returning to the side, the outer peripheral portion 52g of the mark body 52 can be imaged black by the camera as shown in FIG. When the calibration work is performed in this manner, the mark body 52 appears black, the surrounding background member 54 appears white, and the image of the mark body 52 that appears black on a white background can be captured with a clean circular outline as shown in FIG. become.
[0017]
More specifically, as shown in FIG. 6, the light LT from the ring light 12 is reflected in a region R1 centered on the central axis CL of the mark body 52, and the reflected light LT returns to the camera 11 side. However, the surrounding region R2 is further reflected obliquely in the oblique direction, and the light LT reflected on the further outer region R3 (corresponding to the maximum outer diameter portion 52R) is reflected laterally. Therefore, as shown in FIG. 5, the camera 11 captures the center portion of the mark body 52 in white, and captures the black color toward the periphery.
Then, the maximum outer diameter portion 52R of the mark body 52 can make a substantially circular contour emerge with strong contrast with respect to the white of the background member 54.
In order to obtain such a result, it is preferable that the mark body 52 has metallic luster like a normal steel ball for a bearing.
[0018]
Next, an example of a calibration method using such a calibration jig 50 will be described.
7 to 9 show the k-l coordinate system J of the image IM to be captured by the camera 11 and the xy coordinate system H of the table 21 in FIG. The coordinate system J and the coordinate system H are inclined by an arbitrary angle θ. That is, an example is shown in which the direction of the camera 11 and the direction of the table 21 do not match.
FIG. 11 shows the coordinate system J of the camera, the coordinate system H of the table 21, and the coordinate system (unit pixel) k-1 of the camera 11. In FIG. 11, the coordinate system H and the coordinate system J have a unit of mm, and the coordinate system k-1 of the unit pixel is a coordinate system obtained by expanding and contracting the α-β coordinate system J along the α and β coordinate axes. is there.
[0019]
In FIG. 11, OR indicates the origin coordinates (x0, y0) of the camera. Θ indicates the inclination angle between the coordinate system H and the coordinate system J.
Description of each element shown in FIG. 11 is shown in FIG. 12, a vector Xi is a vector representing the position of the mark body 52 in the coordinate system H of the table 21. The vector Xa is a vector representing the origin coordinate OR (x0, y0) of the camera 11 in the coordinate system H of the table 21.
The vector Xb is a vector that represents from the origin coordinate OR (x0, y0) of the camera 11 to the position coordinates (Xi, Yi) of the mark body 52 in the coordinate system H of the table 21. The vector P is a vector representing the vector Xb in the coordinate system J of the camera 11. Therefore, the vector Xb is equal to the vector P.
The vector R is a vector representing the vector P in the pixel coordinate system k-1 of the camera 11.
[0020]
FIG. 13 shows the relationship between the vector Xi and the vector R. The vector Xi can be represented by (xa, ya), α, β, xp, yp, and the like.
FIG. 14 shows a vector Xi of the coordinates of the first, second, and third points of each mark body 52 when captured by the camera 11, and also shows a vector R. Other vectors Xa and Xb can be similarly expressed.
The coordinates Xi1 of the first mark body 52 are shown in FIG. 7, the coordinates Xi2 of the second mark body are shown in FIG. 8, and the coordinates Xi3 of the third mark body are shown in FIG. ing. Vector Xi1, vector Xi2, and vector Xi3 are at linearly independent positions.
[0021]
Here, as shown in FIG. 15, since xa, ya, S, t, and θ are fixed values, an equation as shown in FIG. 15 is established. Note that θ is the tilt angle as described above, and t is the magnification of how many mm corresponds to one pixel in the β direction of the camera image.
It is.
S is a magnification of how many mm corresponds to one pixel in the α direction of the camera image.
[0022]
FIG. 16 shows a coordinate transformation matrix obtained by adding the expressions (6) and (7) of FIG.
17 is obtained by substituting the coordinate transformation matrix of FIG. 16 into the equation (1) of FIG. 15 to obtain the origin coordinate OR (xa, ya) of the camera, and obtaining the equation (FIG. 10) of the coordinate transformation matrix of FIG. As a result of finding the origin coordinates OR of the camera 11, it is shown that arbitrary pixel coordinates on the camera are converted into the coordinate system H of the table 21 by the equation (0) in FIG.
[0023]
Next, with reference to FIGS. 7 to 9, the camera 11 detects the positions of the first mark body 52, the second mark body 52, and the third mark body 52, and a coordinate transformation matrix ( Expression (0) and the origin coordinate OR of the camera 11 are obtained, and an arbitrary pixel coordinate system (k-1) of the camera is converted into an XY table coordinate system H by expression (0).
In steps S1 to S9 shown in FIG. 10, the servo motor 30 controls the motors 24 and 27 of the table moving means 20, and the image processing unit 40 determines the center position of the mark body 52 of the calibration jig 50 captured by the camera 11. Ask.
[0024]
In step S1 of FIG. 10, the camera 11 is moved to a position where the mark body 52 appears. Thereby, the camera 11 takes in the coordinates P1 (vector Xi1) (xi1, yi1) of the first point of the mark body 52 in FIG. 7 (step S2).
Then, in step S3 in FIG. 10, the image processing unit 40 obtains the center position of the mark body 52 at the first coordinate P1 by image processing. This corresponds to the pixel (x1p, y1p) on the camera 11.
[0025]
Next, the table 21 is moved, and the camera 11 captures the coordinates P2 (vector Xi2) (xi2, yi2) of the second point of the mark body 52 as shown in FIG. 8 (step S5). Then, the image processing unit 40 obtains the center position of the second mark body 52 by image processing. The center position of the mark body 52 corresponds to the pixel (x2p, y2p) (step S6).
[0026]
Next, as shown in FIG. 9, the table 21 is moved, and the coordinates P3 (vector Xi3) (xi3, yi3) of the third point of the mark body 52 are captured as an image (step S7). Then, the center position of the mark body 52 is obtained by image processing (step S9). The center position of the mark body 52 corresponds to the pixel (x3p, y3p).
[0027]
Then, as shown in step S10, if the processing of the equations (0) to (11) in FIGS. 15 to 17 is performed, one pixel (pixel) of the camera is displayed on the table 21 which is the actual object. Calibration (calibration) can be performed before measurement by image recognition of the object to determine how many mm the camera 11 corresponds to or how many times the camera 11 is tilted as represented by θ as shown in FIG. In other words, it is possible to find out, for example, what mm of the printed wiring board the pixel is going to place on the table 21.
[0028]
In the present invention, the substantially spherical mark body 52 can be used as a mark close to a perfect circle at a low price, for example, by using a commercially available high-precision steel ball with respect to the concave portion of the positioning member. . Thereby, the calibration accuracy at the time of image recognition can be improved.
Further, since the centering of the positioning member 51 and the substantially spherical mark body 52 in FIG. 3 is determined by the hole position of the concave portion 56 of the background member 54 opened by machining, it can be suppressed to the minimum deviation. In addition, the calibration accuracy at the time of image recognition can be improved.
[0029]
In a mark formed by machining as in the prior art, the calibration accuracy was reduced due to loosening of an edge portion having burrs around the mark. However, a calibration jig 50 as shown in FIG. The use eliminates such a problem.
The background member 54 is behind the mark body 52 when viewed from the camera 11 side, and the upper surface of the background member 54 which is the background surface is behind (below) the focal height FPP of the camera 11. In addition, the camera 11 does not focus on dust and flaws attached to the background member 54, so that an image with less noise than a conventional jig can be obtained, and the calibration accuracy can be improved. That is, it is preferable that the focal height FPP of the camera 11 be located at the maximum outer diameter portion 52R of the mark body 52.
[0030]
The present invention is not limited to the above embodiment. For example, the background member 54 in FIG. 3 is made of white plastic, is embedded in the flange portion 53, and makes use of the fact that the outer peripheral portion of the mark body 52 appears black (see FIG. 5). The image recognition and calibration are performed using the circular mark 52 as a mark. However, the present invention is not limited to this. If the outer periphery of the mark body 52 also shines due to disturbance light, the mark body may be plated with black.
Instead of embedding a white plastic as the background member 54, a white paint may be applied to the central portion of the upper surface of the flange portion 53 (the peripheral portion of the concave portion 56). In the case where the background member 54 is plated with a dark color, the contrast of the circular mark may be obtained by painting the mark body 52 with white or a color that appears brightly on the camera image. When the background is made black, it is conceivable to use a high-precision white plastic ball without using a mark body.
The mark body 52 in FIG. 3 can be prevented from coming off the recess 56 by bonding the corner of the recess 56 with an adhesive.
[0031]
As described above, in an image recognition device that captures an object with an image of a camera and obtains information on its shape, size, and position and angle by image processing, one pixel is actually stored on the object such as a table. It is possible to calibrate with a high degree of accuracy before the object measurement how many mm the camera corresponds to and how many times the camera is tilted with respect to the reference coordinate axis.
[0032]
【The invention's effect】
As described above, according to the present invention, it can be manufactured at low cost, and the calibration of the correspondence between the pixel and the object is easy.
[Brief description of the drawings]
FIG. 1 is a perspective view showing an example of an image recognition device having a preferred embodiment of a calibration jig of the image recognition device of the present invention.
FIG. 2 is a perspective view showing an example in which the image recognition device of FIG. 1 is used for an electronic component device.
FIG. 3 is a side view having a partial cross section showing the calibration jig of FIG. 1;
FIG. 4 is a plan view of the calibration jig of FIG. 3;
FIG. 5 is a view showing a state of reflected light from a mark body of the calibration jig of FIGS. 3 and 4;
FIG. 6 is a side view showing a state of reflected light from a mark body.
FIG. 7 is a view showing a state where coordinates of a first point are captured by a camera.
FIG. 8 is a diagram showing a state where coordinates of a second point are captured by a camera.
FIG. 9 is a diagram showing a state in which coordinates of a third point are captured by a camera.
FIG. 10 is a diagram showing an example of image processing when the coordinates of the first to third points are captured.
FIG. 11 is a diagram showing a relationship among a table coordinate system, a camera coordinate system, and a camera coordinate system (unit pixel).
FIG. 12 is a view for explaining each element in FIG. 11;
FIG. 13 is a diagram showing a relationship between a vector Xi and a vector R;
FIG. 14 is a diagram showing vectors Xi1 to Xi3 and a vector R of coordinates of a first point to a third point.
FIG. 15 is a diagram illustrating a relationship between coordinates of first to third points;
FIG. 16 is a diagram showing a coordinate transformation matrix.
FIG. 17 is a diagram showing a state in which arbitrary pixel coordinates on a camera are obtained from a coordinate transformation matrix and camera origin coordinates.
FIG. 18 is a perspective view showing an example of a conventional image recognition device.
19 is a plan view showing an example of a mark body used in the conventional image recognition device of FIG.
FIG. 20 is a side view of the mark in FIG. 19;
FIG. 21 is a plan view of another conventional mark.
FIG. 22 is a side view of the mark in FIG. 21;
FIG. 23 is a diagram showing the unclearness of the outline of a conventional mark.
[Explanation of symbols]
11 camera, 12 ring light (illuminating means), 20 table moving means, 21 table (object), 50 calibration jig, 51 positioning member 52 ..Mark bodies, 56 recesses

Claims (6)

Used in an image recognition device that recognizes an object, a calibration jig used when calibrating by comparing the positional relationship between the coordinates when the object is moved and the image obtained by the image capturing device,
A positioning member having a recess,
A substantially spherical mark body held in the concave portion of the positioning member,
A calibration jig for an image recognition device, comprising: 2. The calibration for an image recognition apparatus according to claim 1, wherein the mark body has a metallic luster, a peripheral portion defining a concave portion of the positioning member is white, and a substantially central portion of the mark body reflects light as viewed on a projection surface. jig. The positioning member is attached to a moving means for holding and moving the object, the size of the recess is smaller than the diameter of the mark body, and the center of the mark body coincides with the center axis of the positioning means. 2. A calibration jig for an image recognition device according to claim 1. Used in an image recognition device that recognizes an object, a calibration method for calibrating by comparing the positional relationship between the coordinates when the object is moved and the coordinates of the image obtained by the image recognition device,
Hold the positioning member on the object,
A substantially spherical mark body is held in the concave portion of the positioning member, and the center of the substantially spherical mark body is aligned with the central axis of the concave portion of the positioning member,
A calibration method for an image recognition device, comprising: correcting a positional relationship between coordinates indicating a moving direction of an object and coordinates of an image obtained by the image recognition device based on reflected light of a mark body. 5. The calibration method for an image recognition device according to claim 4, wherein the focus of the image pickup device is set to the maximum outer diameter portion of the mark body, and the concave portion of the positioning member is hidden by the maximum outer diameter portion of the mark body. 6. The calibration method for an image recognition device according to claim 5, wherein the surface behind the mark body is a white surface of the positioning member.

Images