JP6031368B2 - Correlation positioning method with workpiece - Google Patents

Description

  The present invention relates to a correlation positioning method with a workpiece, and more particularly, to a correlation positioning method with a workpiece using a camera.

  Conventionally, in a factory or the like, a worker carries a workpiece manually or semi-automatically. For example, when a work machined to a predetermined shape (for example, a car body work) is transported from the original work place to a destination (for example, inside the car body being assembled on the assembly line), the work stand is a relay point. May go through. In such a field, measures such as introducing a robot are taken in order to speed up the transfer work and increase the size of the target work. In this case, since a new facility requires a large amount of money, a contrivance has been made to reduce the introduction cost by improving the existing facility.

  When it is assumed that a workpiece is placed on a work stand that is a relay point in workpiece conveyance, it is ideal to automate all workpiece conveyance from the original workpiece storage location to the destination. However, in terms of cost, either the process until the work is placed at the relay point (hereinafter referred to as the first process) or the process of transporting the work from the relay point to the destination (hereinafter referred to as the second process). On the other hand, it is preferable to presuppose maintenance of the current transport method and diversion of the current equipment.

  For example, in the second step, assuming that the work efficiency is improved by taking out the workpiece placed on the work stand by the robot, the robot can securely hold the workpiece by, for example, the first step. Therefore, it is necessary to position the work with high accuracy relative to the work stand. However, if the workpiece is not strictly positioned in the current first process, the current conveyance method cannot be maintained.

  As another method for the robot to securely hold the workpiece placed on the work stand, if the workpiece position varies on the work stand in order to maintain the current transfer method, It is conceivable to detect the position / posture of the workpiece placed on it with a sensor (camera or laser displacement meter), pass the positional deviation information to the robot, and correct the teaching position data before taking out the workpiece. .

  A technique for measuring the position / posture of a workpiece by detecting the position of a hole on the workpiece with a sensor (camera or laser displacement meter) is described in Patent Document 1. However, the technique described in Patent Document 1 is a technique for determining whether or not a manufactured work itself is a non-defective product by determining how much the manufactured work itself is deviated from a design original drawing.

JP 2006-308500 A

  However, if there is a large variation in the work position on the work stand in order to maintain the current transport method from the original work position to the work stand, in the image (camera image) taken by the camera, The object to be detected appears in a position away from the camera axis. Therefore, when an inexpensive and widely used camera is used for position detection, high-precision detection cannot be performed. The reason is that with an inexpensive camera, it is possible to detect the position of the camera image with high accuracy near the center of the lens of the camera because the image is not distorted. It is.

  For this reason, if the position of workpieces varies greatly on the work stand, special and expensive accessories such as non-generic cameras, camera lenses, and lighting equipment must be used for position detection unless special measures are taken. Will occur.

  Also, when measuring the position / posture of the workpiece by measuring the position of the hole on the workpiece, if the workpiece position variation is large, the laser displacement meter emits light based on the position determined by the camera image at that time There is a problem that the aim of the irradiation position of the laser beam is lost, and the laser beam passes through the hole of the workpiece and the reflected beam cannot be detected.

  Therefore, an object of the present invention is to solve the above-described problems and provide a correlation positioning method with a workpiece that can detect the position of the workpiece with high accuracy even when an inexpensive camera is used.

  In order to solve the above problems, the correlation positioning method with a workpiece according to the present invention detects the position and orientation of the workpiece from at least three measurement target points set in advance on the workpiece, and the workpiece and its workpiece A correlation positioning method with a workpiece for determining a correlation position with a gripping means, wherein a robot jig equipped with a position measurement means including a camera and a laser displacement meter is provided for each measurement target point set on the workpiece. The camera is fixed to the camera reference measurement position by moving based on the teaching position data that teaches the reference position of the measurement target point, and the position of the measurement target point is measured from the camera image captured by the camera. A first value which is a converted value obtained by converting a difference between the first measurement step, the position of the measurement target point measured in the first measurement step, and the lens center position of the camera into a real space distance. A measurement position correction step of fixing the camera to a camera correction measurement position in which the robot jig is moved by the amount of deviation, and a position of the measurement target point from a camera image captured by the camera at the camera correction measurement position The laser displacement meter is fixed to a laser correction measurement position where the laser reference measurement position is corrected by the amount of the first positional deviation by moving the robot jig again with a second measurement step for measuring again, and the laser displacement Measuring a depth position of the measurement target point with a meter, and detecting a positional deviation from a reference depth position of the measurement target point, and including all the measurement target points set in the workpiece The difference between the first displacement, the position of the measurement target point measured in the second measurement step, and the position of the lens center of the camera was converted into a real space distance. The position and orientation of the workpiece are detected from the second positional deviation that is the calculated value and the positional deviation of the depth position, and an amount of deviation of the workpiece from the reference position of the measurement target point is detected with respect to the workpiece. A position data correction step for correcting the teaching position data is included.

  According to such a procedure, in the correlation positioning method with the workpiece, for each measurement target point set on the workpiece, in the first measurement step, at the camera reference measurement position where the camera is moved based on the teaching position data, Based on the camera image, the amount of deviation of the measurement point from the reference in the image plane is detected once. Then, in the measurement position correction step, the robot jig is moved by the amount of the first position shift obtained by converting the shift amount into the camera position shift, and in the second measurement step, the same measurement target point is re-photographed. Then, in the depth position deviation detection step, a position deviation from the reference depth position of the measurement target point is detected. Further, when the three-dimensional positional deviation is obtained for all measurement target points, the teaching position data is corrected in the position data correcting step. In the second measurement step, the amount of deviation from the reference in the image plane of the measurement target point can be detected again based on the camera image taken again. Therefore, when an inexpensive camera is used, even if there is an error in the positional deviation measured for the first time, the error can be eliminated by adding the information for the positional deviation measured for the second time. Therefore, the correlation positioning method with the workpiece of the present invention can detect the position of the workpiece with high accuracy even if an inexpensive camera is used.

  In the correlation positioning method with a workpiece according to the present invention, the workpiece is provided with at least three through holes used when the gripping means is gripped. It is preferable that the measurement target point is set for.

  According to such a procedure, the correlation positioning method with the workpiece sets the measurement target point in the through hole for the workpiece in which the through hole is drilled, so that a specific mark is previously set on the workpiece to measure the position. There is no need to turn on. In addition, since the laser reference measurement position is corrected by the amount of the first positional deviation, it is possible to prevent the irradiation position of the laser beam emitted from the laser displacement meter from being untargeted, and thus there is no concern that the reflected laser beam cannot be detected. .

  According to the present invention, in order to measure the position of the workpiece, the position of the camera is moved and the same measurement target point on the workpiece is photographed twice. Therefore, when an inexpensive camera is used, the first photographing result is obtained. Even if there is an error, the error can be eliminated if the second shooting result is also used. Therefore, the correlation positioning method with the workpiece of the present invention can detect the position of the workpiece with high accuracy even if an inexpensive camera is used. For this reason, even if the position of the placed work varies, the gripping means can easily grip the work by using the data obtained by correcting the teaching position data in the next step.

1 is a block diagram schematically showing a configuration of a robot system according to the present invention. It is a figure which shows typically a mode that the position of a workpiece | work is measured in the robot system which concerns on this invention. It is a figure which shows an example of a workpiece | work, Comprising: (a) is a side view, (b) has shown the front view. It is a figure which shows an example of the workpiece | work mounted on the work stand, Comprising: (a) has shown the state mounted in the reference position, (b) has shown the state mounted | wed and shifted | deviated from the reference position. It is explanatory drawing of position shift of the measurement target point of a workpiece | work, Comprising: (a) is a measurement target point, (b) is a camera measurement range, (c) has shown the state without a position shift. 4A and 4B are schematic diagrams of camera images taken with different camera lenses. FIG. 5A shows a case where a camera lens with a small distortion is used, and FIG. 5B shows a case where a camera lens with a large distortion is used. It is explanatory drawing which shows typically the procedure which measures the position of the measuring object point of a workpiece | work in the robot system which concerns on this invention. It is a flowchart which shows typically the procedure of the correlation positioning method with the workpiece | work which concerns on this invention.

  DESCRIPTION OF EMBODIMENTS Embodiments (referred to as embodiments) for carrying out the present invention will be described in detail with reference to the drawings. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation.

[1. Overview of robot system configuration]
Here, an outline of a configuration of a robot system for carrying out the correlation positioning method with the workpiece according to the present invention will be described with reference to FIG.
In the correlation positioning method with the workpiece according to the present invention, the position of the workpiece 100 from at least three measurement target points 110 set in advance in the workpiece 100 as a pre-process for providing the workpiece 100 to the gripping means (not shown). And a method of detecting a posture and determining a correlation position between the workpiece 100 and the gripping means.
For this purpose, as shown in FIG. 1, the robot system 1 mainly includes a robot 2, a robot control device 3, a teaching data input device 4, a robot jig 5, a camera 7 as a position measuring means 6, and A laser displacement meter 8 and a position correction control device 10 are provided.

  Details of each component will be described later. The robot controller 3 and the position correction controller 10 are, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), an input / output interface, and the like. It has.

  The robot system 1 uses the robot 2 to move the position measurement means 6 to the taught reference measurement position, and from the movement destination, a measurement target point 110 set in advance on the workpiece 100 placed at a predetermined location. The position of the workpiece 100 is measured, and a positional deviation from a reference value of a predetermined position / posture of the workpiece 100 is detected.

  In the robot system 1, the amount of deviation from the reference in the xy direction of the image of the measurement target point 110 is detected once based on the camera image at the destination of the position measuring means 6 including the camera 7, and the detected amount of deviation is actually realized. A second deviation amount is detected from a camera image obtained by re-imaging the same measurement target point 110 after moving the position measuring means 6 including the camera 7 by the amount of the positional deviation converted into a spatial distance. Then, the distance to the measurement target point 110 is measured at the measurement position moved from the reference measurement position of the laser displacement meter 8 by the first position shift, and the depth position shift is determined as a shift from the reference in the z direction. Is detected.

[2. work]
Here, a workpiece to be positioned in the present invention will be described.
In the correlation positioning method with the workpiece of this embodiment, the use, shape, material, and size of the workpiece are not particularly limited. For convenience of explanation, a specific example of the workpiece will be described with reference to FIGS. FIG. 2 is a diagram schematically showing how the position of a workpiece is measured in the robot system according to the present invention. 3A and 3B are diagrams illustrating an example of a workpiece, in which FIG. 3A is a side view and FIG. 3B is a front view. The z axis shown in FIG. 3A is orthogonal to the xy plane shown in FIG.

  As an example, a workpiece 100 shown in FIG. 2 is a workpiece used for a vehicle body, and a galvanized steel sheet processed into a predetermined shape has a side of about several tens of centimeters. Further, in order to describe the position in space for the workpiece 100 shown in FIG. 2, the spatial coordinates (xyz coordinates) shown in FIG. 3 are used. Note that the spatial coordinates (xyz coordinates) shown in FIG. 3 are coordinates for explanation when focusing on one specific measurement target point 110.

  As shown in FIG. 2, the robot 2 is disposed in the vicinity of the workpiece 100, and a robot jig 5 is attached to the arm on the distal end side of the robot 2. The robot jig 5 is provided with a camera 7 and a laser displacement meter 8 in parallel.

  The side on which the workpiece 100 is photographed is referred to as the upper side for the workpiece (positive direction of the z-axis: see FIG. 3A). The workpiece 100 includes a housing 101 having an opening on the upper side, and a horizontally long belt-like member (hereinafter referred to as beams 102 and 103) provided on the opening side of the housing 101 (upper side for the workpiece). Are provided at predetermined intervals in the direction (see FIG. 3B). The beams 102 and 103 protrude outward from the outline of the casing 101 of the workpiece 100 on both sides, and through-holes (hereinafter referred to as holes) are formed in the protruding portions. These holes are assumed to be used when a workpiece-dedicated robot (not shown) grips the workpiece 100. In the present embodiment, three of these four holes are set as measurement target points 110.

As shown in FIG. 2, the work 100 is placed on a table-like work stand 200. The work stand 200 serves as a relay point in the case where the work stand 200 is transported from the original work place to the inside of the vehicle body being assembled on the assembly line that is the destination of the work transport destination.
On the surface of the work stand 200 that comes into contact with the workpiece 100, two support portions 201 and 202 that support the workpiece 100 on both the left and right sides (the x-axis direction: see FIG. 3B) are provided. The support parts 201 and 202 hold and support the beams 102 and 103 of the workpiece 100 when the workpiece 100 is placed obliquely.

FIG. 4A shows an ideal placement state in which the workpiece 100 is positioned with high accuracy with respect to the workpiece stand 200. Further, FIG. 4B shows a mounting state when the position of the work 100 has variations on the work stand 200.
As shown in FIG. 4A, reference positions P1, P2, and P3 in the three-dimensional space for the three measurement target points 110 on the work stand 200 are set.
For example, the reference position P1 (X ₁ , Y ₁ , Z ₁ ) shown in FIG. 4A indicates the center position of the first measurement target point 110 for measuring the position of the workpiece 100. On the other hand, in the placement state shown in FIG. 4B, the center position of the first measurement target point 110 of the workpiece 100 is represented by P′1.

The measurement timings of the three measurement target points 110 shown in FIG. 4A are different. Thus, the coordinate system can be considered independently.
In the coordinate system unique to the first measurement target point 110, the reference position P1 is the origin. Therefore, it can be expressed as P1 = (0, 0, 0), P′1 = (Δx ₁ , Δy ₁ , Δz ₁ ). Similarly, in the coordinate system unique to the second measurement target point 110, the reference position P2 is the origin. Therefore, it can also be expressed as P2 = (0, 0, 0), P′2 = (Δx ₂ , Δy ₂ , Δz ₂ ). Similarly, in the coordinate system unique to the third measurement target point 110, the reference position P3 is the origin. Therefore, it can also be expressed as P3 = (0, 0, 0), P′3 = (Δx ₃ , Δy ₃ , Δz ₃ ).

From another point of view, in order to comprehensively grasp the positions of three measurement target points 110 having different measurement timings, the position / orientation of the workpiece 100 is described in spatial coordinates having an origin on the robot 2 side. can do.
For example, the value of (Z ₁ ) at the reference position P 1 (X ₁ , Y ₁ , Z ₁ ) shown in FIG. 4A is 0 in the coordinate system unique to the first measurement target point 110, but the robot 2 In a spatial coordinate system with an origin on the side, it represents a depth value or distance. Hereinafter, when referring to the depth position, the position of the origin of the xy plane shown in FIG. 3B and the z axis shown in FIG. A coordinate system is assumed which is provided on the displacement meter 8 and in which the direction of the z-axis is reversed from the direction shown in FIG.

  Here, the origin on the robot 2 side can be set, for example, in the robot jig 5, the camera 7, the laser displacement meter 8, or the like. Further, the robot control device 3 can perform position designation by a spatial coordinate system having an origin on the robot 2 side using the teaching position data or the like. The robot control device 3 may use a world coordinate system. Note that the space coordinate system having the origin on the robot 2 side and the space coordinate system having the origin in the workpiece can be mutually converted by predetermined coordinate conversion. These conversion processes can be performed by the robot control device 3, for example.

[3. Details of robot system configuration]
Returning to FIG. 1, details of the configuration of the robot system 1 will be described.

(Robot 2)
The robot 2 is a general industrial robot, for example, a multi-axis multi-joint general-purpose robot. A robot jig 5 is attached to the arm on the distal end side of the robot 2.
When the structure of the robot 2 is a structure including a base on which the joint turns from below, a lower arm on which the joint rotates, an upper arm on which the joint rotates, and a hand having a joint with three degrees of freedom. The robot jig 5 is attached to the hand. The robot 2 can move the robot jig 5 by moving each joint with a motor. In general industrial general-purpose robots, one having a repeated positioning accuracy of about ± 0.1 mm to ± 1 mm is known depending on the specifications, and it is preferable to use one having a performance equivalent to that.

(Robot controller 3)
The robot control device 3 is connected to the robot 2, and operates the robot 2 based on an input command such as a route to the measurement position of the robot 2 or a command stored in advance for measuring the position coordinates of the workpiece 100. And control the posture.
In the present embodiment, the robot control device 3 is configured to receive teaching data for controlling the operation and posture of the robot 2 from the teaching data input device 4.

  In the present embodiment, when the robot control device 3 receives from the position correction control device 10 a shift amount (for example, the number of pixels) that is a difference in position on the camera image of the work 100 actually placed on the work stand 200. Then, a coordinate conversion process is performed to convert the positional deviation on the xy plane, which is a real space. As a result, the robot control device 3 generates teaching data corrected by reflecting the positional deviation of the workpiece 100 actually placed on the teaching data received from the teaching data input device 4. Store in memory. This correction data is used, for example, when the workpiece 100 that is actually placed and whose position / orientation is measured is gripped by a workpiece gripping robot (not shown). The number of pixels indicating the length of the predetermined area in the camera image and the distance in the real space can be handled according to the reference position on which the workpiece 100 is placed, the reference measurement position by the camera 7, the camera measurement range, and the like. It is.

(Teaching data input device 4)
The teaching data input device 4 uses the robot control device 3 to transmit data such as a target movement destination position (reference measurement position) when a predetermined reference point moves in the robot jig 5 attached to the arm tip of the robot 2. It is a device to input to. The teaching data input device 4 may be a personal computer, for example, or may be a pendant for inputting teaching data to a control device for an industrial robot, for example.

(Robot jig 5)
The robot jig 5 is provided with a camera 7 and a laser displacement meter 8 as position measuring means 6. In this embodiment, the camera 7 and the laser displacement meter 8 are fixed to the robot jig 5 so that the camera 7 and the laser displacement meter 8 are juxtaposed in the horizontal direction and face the workpiece 100 in the same direction.

  Since the reference measurement position input from the teaching data input device 4 is a position for a predetermined reference point in the robot jig 5, it may be designated as the reference measurement position for the camera 7, or the laser displacement meter 8 It may be designated as a reference measurement position for use. In the present embodiment, both of them are designated. Further, since the position of the laser displacement meter 8 is fixed with respect to the camera 7, only the reference measurement position for the camera 7 is designated, and the laser displacement meter 8 is determined from the relative positional relationship according to the posture of the robot jig 5. A reference measurement position for use may be obtained as necessary.

  In the present embodiment, the robot jig 5 is configured by combining rod-shaped members. The structure of the robot jig 5 is not particularly limited as long as the position measuring means 6 can be fixed and can be attached to the arm tip of the robot 2. The robot jig 5 is not limited in its material as long as it has strength to support the position measuring means 6, and may be made of metal or alloy, for example.

(Position measuring means 6)
The position measurement means 6 measures the x- and y-direction positions of the two-dimensional plane among the xyz three-dimensional positions of the measurement target point 110 with the camera 7 and laser-displaces the z-direction position orthogonal to the xy plane. A total of 8 is measured.
Note that an image sensor is constituted by the camera 7 and image processing means 11 described later. The position of the subject in the xy two-dimensional space is measured based on the camera image (two-dimensional xy plane image) photographed by the image sensor.

(Camera 7)
The camera 7 is for measuring the position of the measurement target point 110 in the x direction and the y direction. This camera 7 is such that the optical axis of the camera 7 passes through the center of the measurement target point 110 on the workpiece 100 when the robot jig 5 is moved to the taught reference measurement position (reference measurement position for the image sensor). It is fixed to the robot jig 5 in the direction of the camera 7. Here, the reference measurement position for the image sensor is set in a space close to the measurement target point 110 of the workpiece 100 in order to take a close-up of the workpiece 100.

  When the workpiece 100 is taken close-up, the camera 7 is photographed within 1 m from the workpiece 100, preferably at a distance of about several tens of cm to 50 cm. The camera 7 is not particularly limited as long as it can close-up the workpiece 100. The camera lens may be a wide-angle lens, a wide-angle lens, or the like, and the camera lens configuration is not particularly limited, and an inexpensive camera lens that causes distortion in the peripheral portion of the camera image may be used.

  Further, it is preferable to illuminate the workpiece 100 with a separate illumination device during camera shooting. This illumination device only needs to be able to partially illuminate the area around the measurement target point 110 of the workpiece 100 at the time of photographing, and does not have to be a dedicated light for camera photography, and may be a general illumination device.

(Laser displacement meter 8)
The laser displacement meter 8 measures the position (depth position) in the z direction of the measurement target point 110. This laser displacement meter 8 is a predetermined distance from the peripheral edge of the hole which is the measurement target point 110 on the workpiece 100 when the robot jig 5 is moved to the taught reference measurement position (reference measurement position for the laser displacement meter). It is fixed to the robot jig 5 in such a direction that the laser beam can be irradiated to the separated position.
Here, the reference measurement position for the laser displacement meter is set on a space close to the measurement target point 110 of the workpiece 100. Although depending on the specifications of the laser displacement meter 8 to be used, in order to detect the depth distance accurately, the laser displacement meter 8 is separated from the workpiece 100 by about 10 to 50 cm and irradiated with laser light. As the laser displacement meter 8, a general device that can measure the distance to the object with laser light can be used.

(Position correction controller 10)
The position correction control device 10 performs control for correcting teaching data indicating a position set in advance for the movement of the robot 2.
As shown in FIG. 1, the position correction control device 10 includes an image processing unit 11, a laser control unit 12, a storage unit 13, an output unit 14, and an arithmetic processing unit 15.

<Image processing means 11>
The image processing unit 11 processes a camera image (a two-dimensional xy plane image) photographed by the camera 7 and measures the position of the subject on the two-dimensional image.
The method of detecting the measurement target point 110 (hole) from the camera image is, for example, binarizing the camera image according to the brightness, labeling the image object in the binarized image, and extracting the feature quantity related to the size and shape. The image processing method can be performed by a conventionally known image processing method such as a filtering process for selecting an object that matches the measurement target point 110 to be detected.

FIG. 5A is an enlarged view of the measurement target point 110 provided on the beam 102 (103) in the workpiece 100, and FIG. 5B schematically illustrates the camera measurement range 70 of the camera image as a rectangle. FIG. In the camera measurement range 70, O indicates the lens center of the camera 7.
As shown in FIG. 5A, the measurement target point 110 is a circular hole of a predetermined size, and there is no point having a characteristic shape in the vicinity thereof.
Therefore, as shown in FIG. 5B, in the camera measurement range 70 of the camera image obtained by close-up of the workpiece 100, the measurement target point (hole) can be easily detected as indicated by a circular virtual line. it can. The center position of the measurement target point (hole) indicates the center coordinates of the circular region of the measurement target point 110 on the image.
FIG. 5C is a diagram schematically showing a state where the center of the measurement target point 110 (hole) and the lens center O of the camera 7 coincide with each other (a state without deviation).
The image processing means 11 detects the difference between the position of the measurement target point 110 and the position of the lens center O of the camera 7 as a shift amount on the two-dimensional image (shift amount on the camera image).

In the present embodiment, the image processing means 11 per one measurement target point 110, the amount of deviation on the camera image taken at the reference measurement position for the image sensor, and on the camera image taken at the correction position described later. The amount of deviation is detected.
The position corrected by adding the sum of the distances in the real space to the reference measurement position (reference measurement position for the image sensor) taught by the robot jig 5 is about the measurement target point 110. The positions in the x-axis direction and the y-axis direction in the three-dimensional space are a part of the corrected teaching position data for the robot dedicated to gripping a workpiece (not shown). The position in the z-axis direction for the measurement target point 110 will be described later.

Here, the principle of correcting the teaching position data in the x-axis direction and the y-axis direction in the three-dimensional space for the measurement target point 110 will be described in two specific examples with reference to FIGS. 6 and 7, respectively. .
<< Specific Example 1 >>
First, referring to FIG. FIG. 6A schematically shows an image (hereinafter referred to as an image a) obtained by close-up of a predetermined subject using a camera lens that constitutes an image sensor together with the image processing means 11 using an expensive and low distortion camera lens. FIG. FIG. 6B schematically shows an image (hereinafter referred to as an image b) obtained by close-up of a predetermined subject using an inexpensive and highly distorted camera lens on the camera 7 that constitutes the image sensor together with the image processing means 11. FIG. In these examples, it is assumed that images a and b are taken from an indoor table and a box on the room under similar conditions.

  As shown in FIG. 6B, in the image b, the distortion increases as the distance from the center of the lens increases. Assuming that a subject (for example, the tip of a table leg) in the periphery of the image a is a measurement target point, the amount of deviation from the center on the camera image is, for example, (x-axis direction: +440 pixels, y (Axial direction: −660 pixels).

  On the other hand, assuming that the subject (for example, the tip of a table leg) in the periphery of the image b is the measurement target point, the amount of deviation from the center on the camera image is, for example, (x-axis direction: +400 pixels, y-axis direction: -540 pixels).

  In other words, if the amount of deviation from the center of the subject (for example, the tip of the table leg) in the image a is correct, for example, the subject (for example, the table leg in the image b) is correct. The amount of deviation from the center of (1) was estimated lower than the correct answer.

  Therefore, according to this specific example 1, in order to obtain the correct amount of deviation, the amount of deviation (number of pixels) estimated in the image a is equivalent to the converted value obtained by converting the real space distance to x in the three-dimensional space. It can be seen that it is necessary to move the camera 7 itself in the axial direction and the y-axis direction and take a picture again at the correction position. In addition, in the image obtained by re-shooting at the correction position, a position shift reflecting the shift amount estimated lower than the correct answer in the first shooting should have occurred. Therefore, the correct deviation amount is obtained by adding the converted value obtained by converting the deviation amount detected at the correction position to the real space distance to the distance (converted value) in the real space corresponding to the deviation amount detected for the first time. The distance (converted value) in the real space corresponding to is obtained.

≪Specific example 2≫
FIG. 7 is an explanatory diagram schematically showing a procedure for measuring the position of the measurement target point 110 of the workpiece 100 in the robot system according to the present invention. FIG. 7A is a diagram schematically showing a state in which the camera is photographed at the reference measurement position for the image sensor in order to measure the position of one measurement target point 110 of the workpiece 100. In this first shooting, in the camera image, the lens center O of the camera 7 and the center of the hole of the measurement target point 110 are greatly separated, and the measurement target point 110 is in a peripheral portion near the end of the camera measurement range 71. Exists. Here, the shift amount on the camera image can be expressed as, for example, (x-axis direction: −500 pixels, y-axis direction: +600 pixels).

When the difference between the lens center O of the camera 7 and the center of the hole of the measurement target point 110 on the camera image shown in FIG. 7A is converted into a real space distance, for example, a difference in the x-axis direction (distance Δx _11). ) Is, for example, −10 mm, and the converted value is obtained such that the difference in the y-axis direction (distance Δy ₁₁ ) is, for example, +12 mm. In this case, the position Ca of the lens center of the camera 7 projected onto the xy plane in real space and the position Ha of the center of the hole of the measurement target point 110 are shown in FIG.

As shown in FIG. 7 (b), when a reference position Ca, position Ha is a spaced in the negative direction of the x-axis by a distance [Delta] x _11, spaced apart in the positive direction of the y-axis by a distance [Delta] y ₁₁ position is there.
Here, the xy coordinates of the lens center of the camera 7 when moving to the reference measurement position for the image sensor are, for example, at the reference position P1 (X ₁ , Y ₁ , Z ₁ ) of the workpiece 100 shown in FIG. It is set to be (X ₁ , Y ₁ ). Therefore, when there is no positional deviation as shown in FIG. 5 (c), the distance [Delta] x ₁₁ and distance [Delta] y ₁₁ becomes zero.

On the other hand, when there is a displacement as in the example shown in FIG. 7B, the camera 7 provided on the robot jig 5 is moved a distance Δx ₁₁ from the reference measurement position for the image sensor in the negative direction of the x axis. And the measurement position is corrected by moving it by a distance Δy _{11 in} the positive direction of the y-axis.

FIGS. 7C and 7D are drawings of the second shooting performed after the camera 7 is moved, and correspond to FIGS. 7A and 7B. Is omitted as appropriate.
As shown in FIG. 7C, in the second shooting performed at the corrected measurement position, in the camera image, the lens center O of the camera 7 and the center of the measurement target point 110 are more than in the first shooting. The measurement target point 110 exists in a portion near the lens center O in the camera measurement range 72. Here, the shift amount on the camera image can be expressed as (x-axis direction: -150 pixels, y-axis direction: +200 pixels), for example.

When the difference between the lens center O of the camera 7 and the center of the hole of the measurement target point 110 on the camera image shown in FIG. 7C is converted into a real space distance, for example, a difference in the x-axis direction (distance Δx _12). ) Is, for example, −3 mm, and the difference in the y-axis direction (distance Δy ₁₂ ) is, for example, +4 mm. In this case, the lens center position Cb projected onto the xy plane in real space and the hole center position Hb of the measurement target point 110 are shown in FIG.

As shown in FIG. 7D, with reference to the position Cb, the position Hb is still separated by a distance Δx _{12 in} the negative x-axis direction and separated by a distance Δy _{12 in} the positive y-axis direction. Yes.
FIG. 7D also shows the position Ca of the lens center of the camera 7 at the time of the first shooting. As shown in the drawing, the lens center position Cb is different from the lens center position Ca at the time of the first photographing. On the other hand, the position Hb of the center of the hole is the same as the position Ha of the center of the hole at the time of the first photographing (not moving). That is, the distance between the position Ca and the position Cb in FIG. 7D and the distance between the position Ca and the position Ha in FIG. 7B are the same. However, the distance between the position Ca and the position Cb in FIG. 7D and the distance between the position Ca and the position Ha in FIG. 7B are drawn differently. This is because the camera image shown in FIG. If the camera image shown in FIG. 7A reflects the correct position, the converted value in the space shown in FIG. 7B is correct. Similarly, assuming that the camera image shown in FIG. 7C reflects the correct position, the converted value in the space shown in FIG. 7D is correct.

Therefore, from FIG. 7B and FIG. 7D, the x axis in the real space between the position Hb (= position Ha) of the measurement target point 110 and the position Ca of the lens center of the camera 7 at the time of the first shooting. It can be seen that the positional deviation in the direction is Δx ₁ in the following equation (1), and the positional deviation in the y-axis direction is represented by Δy ₁ in the following equation (2).
Δx ₁ = Δx ₁₁ + Δx ₁₂ (1)
Δy ₁ = Δy ₁₁ + Δy ₁₂ (2)

In the numerical example of the distance in the above-described real space, the correct position shift can be expressed as the following Expression (3) and Expression (4).
(−10 mm) + (− 3 mm) = − 13 mm Formula (3)
(+12 mm) + (+ 4 mm) = + 16 mm Formula (4)
In other words, the position that should be determined as (−13 mm, +16 mm) by one detection from the camera image has been determined as the positional deviation is (−10 mm, +12 mm) only by the first measurement. This corresponds to the fact that the measurement target point 110 was captured in the peripheral portion of the camera image at the time of the first shooting, and therefore the amount of deviation on the camera image was estimated lower than the correct answer.
In the present embodiment, coordinate conversion processing for converting a shift amount (for example, the number of pixels) on the camera image into a positional shift on the xy plane that is a real space is performed on the robot control device 3 side. Note that the arithmetic processing means 15 of the position correction control device 10 may be configured to perform instead.

  The description of each means other than the image processing means 11 of the position correction control apparatus 10 shown in FIG.

<Laser control means 12>
The laser control means 12 controls the operation of the laser displacement meter 8. The laser control unit 12 controls the laser displacement meter 8 to emit laser light at a predetermined timing, or amplifies the detection signal of the laser displacement meter 8 and outputs it to the output unit 14. Settings such as the timing of irradiating the laser control means 12 with the laser beam can be made from the arithmetic processing means 15. In this case, when an operator inputs a command or the like from an input device such as a mouse or a keyboard (not shown) in advance, setting data is input to the laser control unit 12 via the arithmetic processing unit 15 and laser irradiation timing and the like are set in advance. The The laser displacement meter 8 and the laser control means 12 can be configured as a set in which a known sensor head and an amplifier unit are connected by a cable.

  The laser control unit 12 moves the reference measurement position for the laser displacement meter by the distance measured in the real space corresponding to the displacement detected first by the image processing unit 11 at the laser measurement position. Control to irradiate. At this time, the detection signal detected by the laser displacement meter 8 indicates the distance from the laser displacement meter 8 to the measurement target point 110. This distance corresponds to the position in the z-axis direction with respect to the measurement target point 110.

<Storage means 13>
The storage unit 13 stores various data and operation programs used for processing of the position correction control device 10. The storage means 13 is composed of, for example, a general hard disk or memory. The storage unit 13 may be disposed inside the unit of the position correction control device 10 or may be an external storage device. A plurality of storage devices may be distributed as the storage unit 13.

<Output means 14>
The output unit 14 is a predetermined output interface for outputting a deviation amount as a processing result of the image processing unit 11 and a detection signal of the laser displacement meter 8 to the robot control device 3.
The output unit 14 outputs the deviation amount and the detection signal of the laser displacement meter 8 to the robot control device 3 at a predetermined timing under the control of the arithmetic processing unit 15.

<Calculation processing means 15>
The arithmetic processing unit 15 controls each part of the position correction control device 10 and includes, for example, a CPU and a memory. The arithmetic processing means 15 can also be comprised by a personal computer etc., for example. The arithmetic processing unit 15 sequentially performs position measurement processing on at least three measurement target points 110 provided on the workpiece 100 by a predetermined algorithm described later.

[4. Flow of processing in the robot system]
A processing flow in the robot system 1 will be described with reference to FIG. 8 (refer to FIG. 1 as appropriate). FIG. 8 is a flowchart schematically showing the procedure of the correlation positioning method with the workpiece according to the present invention.
First, in step S1, the arithmetic processing means 15 of the position correction control device 10 initializes an identifier of the i-th measurement target point 110 (hereinafter referred to as measurement target point i) (i = 1). . Then, the following steps S <b> 2 to S <b> 10 are sequentially performed for each measurement target point set on the workpiece 100.

<First measurement process>
In step S2, the robot controller 3 moves to the reference measurement position (first measurement position) for the image sensor in which the robot jig 5 equipped with the position measuring means 6 is brought close to the measurement target point i based on teaching. Move and fix once there.
In step S3, the position of the measurement target point i is measured by the image sensor while being fixed at the reference measurement position for the image sensor, and the first shift amount from the lens center O of the camera 7 is detected.
In step S <b _> 4, the position correction control device 10 outputs a detection signal (first shift amount x _i1 , y _i1 ) of the image sensor to the robot control device 3. The detection signal of the image sensor corresponds to the distance on the camera image.

<Measurement position correction process>
In step S5, the robot control device 3 performs processing for converting the distance on the camera image into the distance on the real space, stores the value, and sets the reference measurement position for the image sensor according to the value. Correction is made for the first position deviation, and the robot jig 5 is moved to the second measurement position after correction. That is, the camera 7 is moved from the reference measurement position to the camera correction measurement position by moving the robot jig 5.

<Second measurement process>
In step S6, with the camera correction measurement position (second measurement position) fixed, the position of the measurement target point i is measured again by the image sensor, and the lens center O of the camera 7 is used as a reference from the camera image. The second deviation amount of the measured measurement point i is detected.
In step S <b _> 7, the position correction control device 10 outputs a detection signal (second displacement amount x _i2 , y _i2 ) of the image sensor to the robot control device 3. The detection signal of the image sensor corresponds to the distance on the camera image. The robot controller 3 also performs processing for converting the distance on the camera image into the distance on the real space for the second shift amount, and stores the value.

<Depth displacement detection process>
In step S8, the robot controller 3 reads the distance in the real space calculated by the conversion process in step S5, and moves the robot jig 5 to set the reference measurement position for the laser displacement meter 8 to 1. The laser displacement meter 8 is moved to the laser measurement position corrected by the amount of the positional deviation for the second time.
In step S9, the depth position of the measurement target point i is measured by the laser displacement meter 8 while being fixed at the corrected laser measurement position, and the displacement of the measurement target point i from the reference depth position is detected. Here, the reference depth position of the measurement target point i represents, for example, (Z ₁ ) at the reference position P1 (X ₁ , Y ₁ , Z ₁ ) of the workpiece 100 shown in FIG.
In step S <b> 10, the position correction control device 10 outputs a detection signal (depth position shift z _i ) of the laser displacement meter 8 to the robot control device 3.

<Robot controller data management>
For example, the robot controller 3 obtains, for example, the first (i = 1) measurement point, the first positional deviation obtained by the conversion processing in step S4 and the second positional deviation obtained by the conversion processing in step S7. The result of adding and is stored. In other words, the robot control device 3 obtains (Δx ₁ , Δy ₁ ) by performing the calculations of the above-described equations (1) and (2). In addition, the robot control device 3 stores the detection signal z _i of the laser displacement meter 8 acquired in step S10. Then, this Δz ₁ is stored in association with (Δx ₁ , Δy ₁ ) as a positional deviation (Δx ₁ , Δy ₁ , Δz ₁ ) of the measurement target point i in the three-dimensional space.

  In step S11, the arithmetic processing means 15 of the position correction control apparatus 10 determines whether or not all the measurement target points i have been selected. When there is an unprocessed measurement target point i (step S11: No), in step S12, the arithmetic processing unit 15 adds 1 to the current value of i, and returns to step S11 to continue the processing. On the other hand, when all the measurement target points i are selected (step S11: Yes), the end of the process is notified to the robot control device 3.

≪Position data correction process≫
In step S <b> 13, the robot control device 3 determines the first positional deviation, the second positional deviation, and the depth positional deviation obtained for all the measurement target points 110 set on the workpiece 100. The teaching position data for the workpiece 100 is corrected by the amount.

For example, the position / posture of the workpiece 100 is described in the space coordinates having the origin on the robot 2 side, and the teaching position data for the reference position P1 shown in FIG. 4A is (X ₁ , Y ₁ , Z ₁ ). If the positional deviation of the three-dimensional space detected by the mounted workpiece 100 shown in FIG. 4B is (Δx ₁ , Δy ₁ , Δz ₁ ), the first measurement target point The corrected teaching position data for teaching the center position P′1 of 110 is (X ₁ + Δx ₁ , Y ₁ + Δy ₁ , Z ₁ + Δz ₁ ).
Similarly, the teaching position data for the reference position P2 shown in FIG. 4A is represented by (X ₂ , Y ₂ , Z ₂ ), and the detected displacement in the three-dimensional space is (Δx ₂ , Δy ₂ , Δz ₂ ), the corrected teaching position data for teaching the center position P ′ 2 of the second measurement target point 110 is (X ₂ + Δx ₂ , Y ₂ + Δy ₂ , Z ₂ + Δz _2). )
Similarly, the teaching position data for the reference position P3 shown in FIG. 4A is represented by (X ₃ , Y ₃ , Z ₃ ), and the detected displacement in the three-dimensional space is (Δx ₃ , Δy ₃ , Δz ₃ ), the corrected teaching position data for teaching the center position P ′ 3 of the third measurement target point 110 is (X ₃ + Δx ₃ , Y ₃ + Δy ₃ , Z ₃ + Δz _3). )

[5. Example of variation in position of workpiece 100]
For example, when the diameter of the hole which is the measurement target point 110 of the workpiece 100 is about several centimeters, the variation in the position of the hole of the workpiece 100 placed on the workpiece stand 200 is in the x, y, and z directions. Assume that there is a maximum of several centimeters for each.
In this case, in order for a work holding robot (not shown) to securely hold the work 100 placed on the work stand 200, normally, the taught reference positions P1, P2, P3 (FIG. 4A) are used. It is thought that it is necessary to place the workpiece 100 at a position that is positioned with a high accuracy of several mm or less and without variation.

  On the other hand, according to the correlation positioning method with the workpiece according to the embodiment of the present invention, when the position of the workpiece 100 varies on the workpiece stand 200, the position of the workpiece 100 actually placed is placed. The posture can be detected by the position measuring means 6 (camera 7 and laser displacement meter 8), and the positional deviation information can be transferred to the robot controller 3 to correct the teaching position data.

  Therefore, after replacing the reference positions P1, P2, and P3 with the corrected reference positions P′1, P′2, and P′3 (see FIG. 4B), the work position is transferred to a work gripping robot (not shown). Can be taught. Therefore, when it is assumed that the workpiece 100 is transported to the destination via the work stand 200 at the relay point, the equipment is improved so that the robot from the relay point to the destination is transported by a work holding robot not shown. At the time, in the process of placing the workpiece 100 on the work stand 200 at the relay point, the conveyance work can be carried out as it is without satisfying strict positioning requirements such as several mm or less from the reference positions P1, P2, P3. Is possible.

  As described above, the correlation positioning method with the workpiece according to the embodiment of the present invention can detect the position of the workpiece with high accuracy even if an inexpensive camera is used. Further, it is possible to reduce the cost required for a configuration newly introduced in order to perform correlation positioning with the workpiece. In addition, if it is possible to use an existing camera that is close to hand because it is difficult to obtain a camera with the performance to be used, there is no need to purchase a new camera. By this method, the position of the workpiece can be detected with high accuracy. Therefore, even if an expensive camera is used, the position of the workpiece can be detected with high accuracy by the same algorithm.

  As mentioned above, although preferable embodiment of the correlation positioning method with the workpiece | work of this invention was described, this invention is not limited to above-described embodiment. For example, in the embodiment described above, the robot control device 3 performs the process of adding the horizontal direction and the vertical direction for the two detection signals of the image sensor. However, the arithmetic processing unit 15 of the position correction control device 10 is described. It is good also to do in. In this case, the arithmetic processing means 15 adds after performing the process which converts the distance on an image into the distance on real space.

  In the position correction control apparatus 10, the image processing unit 11, the laser control unit 12, the storage unit 13, the output unit 14, and the arithmetic processing unit 15 have been described as being incorporated in one unit. May be. Alternatively, although the position correction control device 10 is provided separately from the robot control device 3, the position correction control device 10 is not limited to this and may be incorporated in the robot control device 3.

In the above embodiment, the workpiece 100 shown in FIGS. 2 and 3 has been described as the workpiece to be positioned. However, the material, shape, size, and the like of the workpiece are not limited to this.
Although the number of measurement target points is three, it may be four or more.
Although the measurement target point is set at the position of the hole at the end of the workpiece, it may be set at an arbitrary position on the workpiece.
Although FIG. 2 illustrates a work stand 200 corresponding to one work 100 as the work stand, the work stand may be configured so that a plurality of works can be placed in an aligned state at a time.
Even in these modified examples, similar effects can be obtained.

  In the embodiment, the second shift amount is detected from the camera image re-photographed after moving the camera 7 by the amount of the first position shift indicating the converted value of the shift amount detected using the camera 7 into the real space. However, the third shift amount is detected from the camera image taken again after the camera 7 is further moved by the second position shift indicating the converted value of the second shift amount into the real space. Alternatively, the shift amount may be detected four or more times in the same manner. In this case, the detection can be repeated until the detected deviation amount becomes smaller than a predetermined threshold value.

  In the embodiment, the depth position shift is detected at the measurement position moved from the reference measurement position of the laser displacement meter 8 by the first position shift. However, the first position shift and the second position are further detected. You may make it detect depth position shift | offset | difference in the measurement position which moved the laser displacement meter 8 by the part which added the shift | offset | difference.

DESCRIPTION OF SYMBOLS 1 Robot system 2 Robot 3 Robot control apparatus 4 Teaching data input apparatus 5 Robot jig 6 Position measuring means 7 Camera 8 Laser displacement meter 10 Position correction control apparatus 11 Image processing means 12 Laser control means 13 Storage means 14 Output means 15 Calculation processing means

Claims (2)

A correlation positioning method for a workpiece, wherein a position and orientation of the workpiece is detected from at least three measurement target points set in advance on the workpiece, and a correlation position between the workpiece and its gripping means is determined.
For each measurement target point set for the workpiece,
By moving a robot jig equipped with a position measuring means including a camera and a laser displacement meter based on teaching position data that teaches a reference position of the measurement target point, the camera is fixed to the camera reference measurement position. A first measurement step of measuring a position of the measurement target point from a camera image photographed by the camera;
The robot jig is moved by an amount corresponding to a first positional deviation, which is a converted value obtained by converting the difference between the position of the measurement target point measured in the first measurement step and the lens center position of the camera into a real space distance. A measurement position correction step of fixing the camera to the camera correction measurement position;
A second measurement step of measuring again the position of the measurement target point from a camera image taken by the camera at the camera correction measurement position;
By moving the robot jig, the laser displacement meter is fixed to a laser correction measurement position in which the laser reference measurement position is corrected by the amount of the first positional deviation, and the depth position of the measurement target point is measured by the laser displacement meter. And a depth position deviation detection step of detecting a position deviation from a reference depth position of the measurement target point,
The difference between the first positional deviation obtained for all the measurement target points set on the workpiece, the position of the measurement target point measured in the second measurement step, and the position of the lens center of the camera is obtained. The position and orientation of the workpiece are detected from the second positional deviation, which is a converted value converted into a spatial distance, and the positional deviation of the depth position, and the amount of deviation from the reference position of the measurement target point of the workpiece is detected. And a position data correcting step for correcting the teaching position data for the work, and a correlation positioning method with respect to the work.   The workpiece has at least three through-holes used when the gripping means grips the workpiece, and the measurement target point is set for the through-hole drilled in the workpiece. The correlation positioning method with the workpiece according to claim 1.

Images