JP3175623B2 - Robot control device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device mounted on a hand of a robot which can be moved to an arbitrary position in a predetermined space and controlled to an arbitrary posture, and the object to be imaged is controlled by controlling the robot. The present invention relates to a robot control device that captures an image from a predetermined position using a device.

[0002]

2. Description of the Related Art In a robot with a visual hand attached to a hand such as a television camera or the like, when an image is accurately taken without a stop operation during the operation of the robot, Japanese Patent Publication No. 7-46 is disclosed.
As described in Japanese Patent Publication No. 288, a robot position is stored for each sampling time, and a position shift and an imaging timing error of a teaching point, which is an imaging target point, are calculated and corrected using this position data. I have to. For the correction, a method has been attempted in which position shift correction data is added to an imaging target point (= robot operating point) to pass the target point. As for the imaging timing error, it is determined after which sampling time of the TV camera the actual route passes through the imaging target point, and the time from the sampling time to the passage of the imaging point is calculated and stored, and the playback time is calculated. There is known a method of issuing an imaging start command to an image processing apparatus based on a sampling time and a time error stored during operation.

[0003]

By the way, in recent years, there has been a demand to view various portions of an object having a complicated three-dimensional shape with a television camera mounted on a robot. In this case, 6
It is conceivable that a television camera is attached to a hand of an axis robot or the like and an image of an object is captured by controlling the robot. In such a configuration, position correction of the robot and correction of an imaging timing error as shown in a conventional example are performed. Just do not know where the TV camera is pointing. For this reason, the same image as at the time of teaching cannot be obtained,
As a result, there is a problem that it is not possible to accurately measure an object using a television camera.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to image an object to be imaged at a desired imaging position and imaging posture by an imaging device without stopping a robot equipped with the imaging device. It is an object of the present invention to provide a robot control device capable of performing the following.

[0005]

According to the first aspect of the present invention, the control unit controls the imaging device mounted on the hand of the robot to pass through the imaging target point position stored in the imaging target point position storage unit. The robot is operated so that the posture of the imaging device at the imaging target point position becomes the target posture stored in the imaging target point posture storage means.

However, when the robot is operated at a high speed, the imaging device is not located at the imaging target point position at the time of sampling by the position detecting means, and the imaging device does not assume the target posture at the imaging target point position.

Therefore, the control means locates the imaging device at the imaging target point position stored in the imaging target point position storage means at the detection timing of the position detection means, and at the imaging target point, the imaging device detects the imaging target point. The robot is operated so as to have the posture stored in the point posture storage means.

The control means drives the imaging device at a predetermined timing under such a control state. This makes it possible to image the imaging target substantially at the imaging target point position and at the substantially desired posture.

According to the second aspect of the present invention, the control means corrects the position so that the imaging device is located at the imaging target point position at the time of sampling by the position detection means, and at the same time the imaging device is at the target attitude at the imaging target point position. The robot is operated with the posture corrected so that Thereby, the imaging device is located at or near the imaging target point position at the time of sampling of the position detection means,
At the imaging target point position, the target posture becomes a posture or a posture close to the posture.

The control means drives the imaging device when the position detection means detects that the imaging device is located at or near the imaging target point position. Thereby, the imaging device can image the imaging target object substantially at the imaging target point position and at the substantially target posture.

According to the third aspect of the present invention, the operation target point position setting means sets a plurality of operation target point positions before and after the imaging target point position along the passage of the imaging device, and stores the operation target point position. The means stores the operation target point position.

The control means controls the robot such that the imaging device passes through the operation target point position stored in the operation target point position storage means. At this time, although the robot operates so as to pass through each operation target point position, the imaging device does not normally pass through the operation target point position but passes near the operation target point position.

Therefore, the control means corrects the position of the operation target point by an amount of displacement between the position when the imaging device is located near the imaging target point position and the imaging target point position at the time of sampling by the position detection means. . Thus, the imaging device can be accurately positioned at the imaging target point position at the time of sampling by the position detection means.

According to the fourth aspect of the present invention, when the imaging device moves around the object to be imaged by the operation of the robot,
Since the intersection point where the plane along the direction in which the imaging device passes through the imaging target point position and the circle centered on the imaging direction at the imaging target point position intersects is approximately the same distance from the imaging target point position, the imaging target The point position is located near a locus of a curve that moves the imaging device so as to pass through the imaging target point position.

Therefore, the control means corrects the position of the operation target point stored in the operation target point position storage means by the amount of positional deviation. As a result, the locus of the curve on which the imaging device moves moves by the amount of displacement, and the imaging timing of the imaging device can be accurately matched with the imaging target point position.

According to the fifth aspect of the present invention, the control means controls the attitude of the imaging device at the operation target point position to be the target attitude at the imaging target point position. At this time,
When the attitude of the imaging device is changed at a high speed, the imaging device does not become the target attitude due to the influence of the front and rear attitudes.

In this case, since the imaging target point position is located in the vicinity of the operation target point position, the imaging device operates by the amount of the positional deviation between the attitude when the imaging device is located in the vicinity of the imaging target point position and the target attitude. By correcting the posture at the target point position, the posture when the imaging device is located near the imaging target point position can be set as the target posture.

Therefore, when the position detecting means detects that the imaging device is located near the imaging target point position, the control means determines the attitude of the imaging device at that time and the target attitude at the imaging target point. By correcting the posture when the imaging device is located at the operation target point by the amount of the deviation, the posture at the imaging target point can be accurately adjusted to the target posture.

According to the sixth aspect of the present invention, the surface along the direction in which the imaging device passes near the imaging target point position intersects the circle having the imaging direction at the imaging target point as the operation target point position. When the intersection point is set, the imaging target point position is located at the center of the operation target point position. Therefore, by correcting the target posture at the operation target point position, the orientation of the imaging device at the imaging target point can be accurately determined. The target posture can be set well.

According to the seventh aspect of the present invention, the spatial region setting means is configured such that the imaging device is located at least once at the time of sampling by the position detection means.
Is set at each imaging target position . Then, the space area storage means stores the space area data set by the space area setting means.

The control means drives the imaging device when the position detection means detects that the imaging device is located in the area indicated by the spatial area data stored in the spatial area storage means. This makes it possible to accurately drive the imaging device at the timing when the imaging device is located at or near the imaging target point.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a robot device with a visual sensor will be described below with reference to the drawings. FIG. 2 schematically shows the entire configuration. This figure 2
, The six-axis robot 1 is controlled by the robot controller 2, and the manipulator 3 has
A shutter camera 4 composed of a CD camera (corresponding to the imaging device of the present invention) is mounted. This shutter camera 4
Is configured to output an imaging signal to the image processing device 5 at a timing when a trigger is given from the image processing device 5.

The image processing device 5 recognizes the shape of the inspection object 6 (corresponding to the imaging object of the present invention) by performing image processing based on an imaging signal from the shutter camera 4. The teach pendant 7 is for instructing the robot controller 2 to set the shutter camera 4 at a desired imaging position and a desired imaging posture.

FIG. 1 is a block diagram showing the overall configuration.
In FIG. 1, the robot controller 2 includes a main controller 8
(Corresponding to the control means, the position detection means, the operation target point position setting means, the operation target point attitude setting means, and the space area setting means of the present invention), and the imaging target point storage section 9 (the imaging target of the present invention). Point position storage means, imaging target point / posture storage means), robot operation target point storage unit 10 (equivalent to operation target point position storage means, operation target point / posture storage means of the present invention), imaging point position / posture detection area A storage unit 11 (corresponding to a detection area storage unit of the present invention), a shift amount storage unit 12, and a detection storage unit 13 are provided.

On the other hand, the 6-axis robot 1 controls the position and the attitude of the shutter camera 4 by operating a predetermined manipulator 3 by a plurality of servomotors 14 in response to a command from the robot controller 2. I have. In this case, the position of the manipulator 3 driven by the servomotor 14 is detected by the encoder 15 (corresponding to the position detecting means of the present invention).

The image processing device 5 issues a trigger to the shutter camera 4 in response to a command from the main control unit 8 of the robot control device 2, and performs image processing based on an image signal input from the shutter camera 4. By executing, the inspection object 6 is recognized.

Next, the operation of the above configuration will be described. This system is divided into a teaching mode, a learning mode, and a playback mode as shown in FIG. 3, and the shutter camera 4 is executed by sequentially executing those modes.
The accuracy of the image pickup position and the image pickup posture is improved.

First, the teaching pendant 7 teaches the imaging target point position Pi and the imaging posture Si of the shutter camera 4 to the robot control device 2 with the teaching mode set for the robot control device 2.

FIG. 4 shows the teaching mode.
In FIG. 4, the robot controller 2 sequentially stores the taught imaging target point position Pi and the imaging posture Si taught (S401).

In this case, as shown in FIG. 7, the imaging target point position Pi is determined by a vector Pi = (Xi, Yi, Zi) (where i
= 1, 2,..., N). That is, the imaging target point immediately before the imaging target point Pi can be represented by Pi-1, and the imaging target point immediately after the imaging target point Pi can be represented by Pi + 1.

On the other hand, the imaging posture Si can be expressed by the following equation.

(Equation 1)

In this equation 1, the vector Six represents the unit vector xmi in the image plane X direction of the shutter camera 4 in the base coordinate system, the vector Siy represents the unit vector ymi in the image plane Y direction in the base coordinate system, and the vector Siz Represents the camera viewing direction unit vector zmi in the base coordinate system. XAi is the X component of the vector xmi in the base coordinate system, YAi is the Y component of the vector xmi in the base coordinate system, ZAi is the Z component of the vector xmi in the base coordinate system, X
Bi is the X component of the vector ymi in the base coordinate system, YBi is the Y component of the vector ymi in the base coordinate system, ZBi is the Z component of the vector ymi in the base coordinate system, and Xci is X in the base coordinate system of the vector zmi. The component Yci represents the Y component of the vector zmi in the base coordinate system, and Zci represents the Z component of the vector zmi in the base coordinate system.

Then, when the teaching operation by the operator is completed, the robot controller 2 (S402: YE
S) As shown in FIG. 7, the vectors Mi, 1, Mi, 2, which are the robot operation target point position data, near the robot operation target point position data Pi, and the posture data at those robot operation target point positions. Vectors Si, 1 and S
i, 2 is automatically generated and stored in the robot operation target point storage unit 10 (S403).

In this case, the robot operating point target point Mi, 1,
Mi, 2 is automatically generated as two points having the same posture at the same distance Li about the camera imaging target point position in the direction perpendicular to the camera viewing direction, for example.

That is, Pi is the target position of the camera image to be corrected, Si, z is the line-of-sight unit vector, Pi-1 is the position of the immediately preceding camera image, and Pi is the position of the next camera image target point. +1 and the vector S
Consider a circle 2Li with a radius Li (the size is determined by the robot operation speed) perpendicular to i, z. Also, Pi, Pi-1 and Pi +
A normal unit vector of the plane Hi including 1 is defined as a vector Ni. The line of intersection of the circle 2Li and the plane Hi is the path through which the robot's hand, and thus the shutter camera 4 passes.

Here, assuming that a parallel unit vector of an intersection line between the circle 2Li and the plane Hi is li, ◎

(Equation 2) Here, in equation (2), when the plane including the circle 2LI and the plane Hi are parallel to each other, the intersection of the two planes is not obtained, and therefore, the planes parallel to the line segments Pi-1 and Pi + 1 are not obtained. This means that the intersection of the straight line and the circle 2Li is set as the passage route of the robot. When the normal unit vector Ni is set in the direction shown in FIG. 7, the vector li becomes the direction shown in FIG.
Mi, 1 = Pi−Li · li, but when the normal unit vector Ni is set in the opposite direction to the direction shown in FIG. 7, the vector 11 also becomes the opposite direction to the direction shown in FIG.
Mi, 1 = Pi + Li · li. The same applies to the robot operation target point Mi, 2.

Further, Si, 1 = Si, 2 = Si is set.
That is, the imaging postures Si, 1, Si, 2 at the robot operation target point positions Mi, 1, Mi, 2 set as described above are taken as the imaging postures Si set in the teaching mode.

Further, as shown in FIG. 8, the distance Ki (Ki = Vi) for moving the shutter camera 4 to the control sampling time Ts with respect to the traveling direction of the six-axis robot 1 around each imaging target point Pi. Ts mm: Vi is the robot operation speed) and sets the robot position / posture detection area Di in a space area where the robot trajectory error (r1 mm) or more is perpendicular to the traveling direction of the 6-axis robot 1, and the robot controller 2 Imaging point position / posture detection area storage unit 11
(S404).

Subsequently, a learning mode is set for the robot controller 2. FIG. 5 shows the learning mode. In FIG. 5, the robot controller 2 calculates a robot operation interpolation target value by calculating interpolation data for each sampling time based on a robot operation target point position controlled under the same conditions as a playback mode described later ( S501).

That is, in the learning mode described above,
Although the operation target point position is set as the position where the robot passes, the robot operation interpolation target value is set so that the robot passes smoothly at or near the operation target point position as the target path where the actual robot operates. It is.

Then, when the robot is operated, the robot controller 2 operates the manipulator 3 so that the robot passes the position of the motion interpolation target value (S502).
In the operation state of the robot, the position information Rij and the posture information Rsi, j (j = 1, 2,...) Of the hand of the manipulator 3 of the six-axis robot 1 are output to the output from the encoder 15 as shown in FIG. Read in order based on (S503) At this time, the hand of the manipulator 3 is controlled so as to smoothly pass through the operation target point positions Mi, 1 and Mi, 2 shown in FIG. 8, so that the intersection line A as shown in FIG. It passes through the vicinity of the imaging target point position Pi.

Here, the robot controller 2 compares the sequentially read position information Ri, J with the data Di stored in the imaging target point position / posture detection area storage unit 11 (S
504) If it is determined that the area Di includes the position information Ri, j (S504: YES), the current position information Ri, J and the current attitude information Rsi, j are stored in the detection storage unit 1.
3 (S506).

That is, in FIG. 8, the robot controller 2 reads the position information Ri, 1, Ri, 2, Ri, 3 in this order.
When it is determined that the subsequent position information Ri, 4 is included in the area Di, the current position information Ri, 4 and the current attitude information Rsi, 4 expressed by the following equations are stored in the detection storage unit 13.

[0044]

(Equation 3)

Subsequently, when the above-described series of robot operations is completed (S.506: YES), the detection storage unit 13
Is compared with the camera target position information Pi stored in the target image storage unit 9,
Imaging target point positional shift amount Hi (ΔXi, ΔYi, ΔZi)
Is calculated as follows.

[0046]

ΔXi = Xri−Xi, ΔYi = Yri−Yi, Δ
Zi = Zri-Zi

Further, the current posture information Rsi stored in the detection storage unit 13 and the imaging posture information Si stored in the imaging target point storage unit 9 are compared, and the posture deviation amounts Hs (ΔAi, ΔBi
, ΔCi) are calculated as follows.

[0048]

(Equation 5) In the equation (5), Mi is an attitude conversion matrix, which indicates the amount of attitude shift of the current attitude information Rsi with respect to the imaging attitude information Si. Where ΔAi is around the X axis and ΔBi is Y
.DELTA.Ci about the axis represents a rotational deviation about the Z axis. Equation 5 indicates that the direction of the shutter camera 4 is first rotated about the X axis by the shift amount ΔAi, then rotated only around the Y axis by the shift amount ΔBi, and further rotated about the Z axis by the shift amount ΔCi. It means to do.

Here, Expression 5 indicates that the posture S is obtained when the posture of the camera imaging target point after teaching is Si and the detected posture after actually operating the robot is Rsi, 4.
The matrix Mi representing the rotational movement from i to Rsi, 4 is Rsi, 4 = Si · Mi, where ΔAi is the rotation around the X axis, ΔBi is the rotation around the Y axis, and ΔCi is the rotation around the Z axis. . In this case, Mi = Si ⁻¹ · Rsi, 4, and Si,
Since Rsi, 4 is known, Mi and thus ΔAi, ΔB
i, ΔCi can be calculated.

Then, the robot control device 2 stores the imaging target point position deviation amount Hi and the posture deviation amount Hs in the deviation amount storage unit 1.
2 is stored. When the learning mode is completed as described above, the playback mode is set for the robot controller 2.

FIG. 6 shows the playback mode.
In FIG. 6, the robot control device 2 stores the shift amount storage unit 12 from the position information Mi, 1 and Mi, 2 of the robot operation target point storage unit 10 corresponding to each imaging target point position information Pi.
(S601).

Also, the posture deviation Hsi stored in the deviation amount storage unit 12 is corrected from the posture data Si, 1 and Si, 2 so as to be reduced as shown in the following equation (S602).

[Si, 1 '] = [Si, 2'] = [Si] [Mi '] In this equation 6, [Mi'] is [Delta] Ai of [Mi],
ΔBi and ΔCi are replaced with -ΔAi, -ΔBi and -ΔCi.

The robot operation target point position information Mi, 1 ', Mi, 2' and attitude information S obtained as described above are obtained.
With i, 1 'and Si, 2' as new target values, control is performed in the same manner as in the learning mode (S603 to S607).

In this case, the difference from the learning mode is that the current position information Ri, j of the six-axis robot is compared with the data Di of the imaging point position / posture detection area, and when it enters this area Di (S607: YES). The other point is that an imaging start command is output to the image processing apparatus 5 (S608). As a result, the image processing device 5 outputs a trigger to the shutter camera 4 and the shutter camera 4 outputs the video signal at that time accordingly. The inspection object 6 can be recognized by inputting a signal and performing image processing.

According to the above configuration, the robot controller 2
Execute the learning mode to correct the position of the imaging target point taught in the teaching mode so that the robot passes at the sampling time, and to correct the position so that the imaging target point position becomes the imaging target posture. Therefore, compared to the conventional technology that only corrects the position so as to become the imaging target position, the position and orientation error of the robot can be corrected, and the imaging timing is adjusted to the sampling timing without requiring a new function. Can be achieved. Accordingly, accurate imaging can be realized without stopping operation during operation of the robot, and inspection can be speeded up.

The present invention is not limited to the above embodiment, but can be modified or expanded as follows. In the present embodiment, the learning mode is described as a mode different from the teaching mode and the playback mode. However, the learning mode may be performed during the playback mode.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an overall configuration according to an embodiment of the present invention.

FIG. 2 is a diagram showing an arrangement relationship between a six-axis robot and an inspection object.

FIG. 3 is a flowchart showing the order of operation modes of the robot control device.

FIG. 4 is a flowchart showing a teaching mode of the robot control device.

FIG. 5 is a flowchart showing a learning mode of the robot control device.

FIG. 6 is a flowchart showing a playback mode of the robot control device.

FIG. 7 is a diagram showing the principle of a method of generating an operation target point by the robot control device.

FIG. 8 is a diagram showing a relationship between a position of an imaging target point of a robot and an actual passing position.

FIG. 9 is a diagram illustrating a relationship between a target position of an imaging target of a robot and a passing position after position correction.

[Explanation of symbols]

1 is a 6-axis robot, 2 is a robot controller, 3 is a manipulator, 4 is a shutter camera (imaging device), 5 is an image processing device, 6 is an inspection object (imaging object), 8 is a main control unit (control means) , Position detecting means, operation target point position setting means, operation target point attitude setting means, space area setting means), 9
Is an imaging target point storage unit (imaging target point position storage means, imaging target point / posture storage means), 10 is a robot operation target point storage unit (operation target point position storage means, operation target point / posture storage means), and 11 is an imaging point It is a position / posture detection area storage unit (spatial area storage means).

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-8-313225 (JP, A) JP-A-8-108386 (JP, A) JP-A-9-47991 (JP, A) 336523 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) B25J 3/00-3/10 B25J 9/10-9/22 B25J 13/00-13/08 B25J 19/02 -19/06 H04N 7/18 G01B 11/24

Claims (7)

(57) [Claims] An imaging device is mounted on a hand of a robot which can move to an arbitrary position in a predetermined space and can be controlled to an arbitrary posture, and an image of an object to be imaged is captured from the predetermined position by the imaging device by controlling the robot. An imaging target point storage means for storing positions of a plurality of imaging target points set around the imaging target; and an imaging target for storing an attitude of the imaging device for each imaging target point. Point attitude storage means, position detection means for detecting the position of the imaging device at each sampling time in the operation state of the robot, and the imaging device is stored in the imaging target point position storage means at the detection timing of the position detection means. The imaging device is positioned so as to be the imaging target point position, and at the imaging target point, the image capturing device has the attitude stored in the imaging target point attitude storage means. The robot controller being characterized in that with said control means for driving the imaging device robot while operating at a predetermined timing. 2. The control means corrects the position of the imaging device at the detection timing of the position detection means so as to be positioned at the imaging target point position stored in the imaging target point position storage means, and corrects the imaging target point. While operating the robot in a state where the imaging device is posture corrected so as to be the posture stored in the imaging target point posture storage means,
In the operation state, when the position detection unit detects that the imaging device is located at the imaging target point position stored in the imaging target point position storage unit or at a position near the imaging target point position, the imaging device is driven. Claim 1.
The control device of the described robot. 3. An operation target position setting unit that sets a plurality of operation target point positions before and after the imaging target point position stored in the imaging target point position storage unit along a passage route of the imaging device; Operation target point position storage means for storing a plurality of operation target point positions set by the target point position setting means, wherein the control means is configured to control the operation target stored in the operation target point position storage means by the imaging device. When the robot is operated so as to pass through a point position, and when the position detecting unit detects that the imaging device is positioned near the imaging target point position in accordance with the operation, the imaging device at that time is detected. A state in which the operation target point position stored in the operation target point position storage unit is corrected by a positional deviation amount between the detected position of the image and the imaging target point position stored in the imaging target point position storage unit. Claim 2, characterized in that to operate the robot
The control device of the described robot. 4. The operation target point position setting means includes a surface along a direction in which the imaging device passes near the imaging target point position and a circle centered on the imaging direction of the imaging device at the imaging target point position. 4. The control device for a robot according to claim 3, wherein an intersection point between the two is set as an operation target point position. 5. An operation target point attitude setting means for setting an attitude of the imaging device at an operation target point position stored in the operation target point position storage means, and an attitude set by the operation target point attitude setting means. An operation target point attitude storage means for storing, wherein the control means stores the attitude of the imaging device at each operation target point position stored in the operation target point position storage means in the imaging target point attitude storage means. When the robot is operated so as to assume the posture, and when the position detecting means detects that the imaging device is located near the imaging target point position, the posture of the imaging device and the imaging target point position at that time are detected. 4. The robot is operated in a state in which the posture stored in the operation target point posture storage means has been corrected by the amount of the posture deviation from the target posture in the step (c).
Or the control device for a robot according to 4. 6. The operation target point attitude setting means, wherein the plane along the direction in which the imaging device passes near the imaging target point position intersects a circle centered on the imaging direction at the imaging target point position. The control device for a robot according to claim 5, wherein an intersection point of the movement is set as an operation target point position. 7. The apparatus according to claim 1, wherein said imaging device is positioned at least once at a sampling time by said position detecting means.
The spatial area including the various imaging target point positions is
A space area setting means for setting the space area data set by the space area setting means, and the control means controls the position detection means to cause the imaging device to store the space area data. 7. The robot control device according to claim 1, wherein the imaging device is driven when it is detected that the image pickup device is located in an area indicated by the spatial area data stored in the means.