JP2003305676A - Control method and control device for mobile robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a control method and a control device for a mobile robot that can shorten working time. <P>SOLUTION: A work 8 is disposed on a working table 7, and work of gripping the work 8 by an arm 3 of the mobile robot 1 is executed. In this case, a hand part 3b of the arm 3 is provided with a CCD camera 6. The work 8 is photographed in the state of the arm 3 being positioned in a relative target position immediately before gripping the work 8 by the hand part 3b, and the image is stored as a reference image. When gripping the work 8 by the arm 3, the arm 3 is controlled during the movement of the mobile robot 1 so that the image photographed by the CCD camera 6 coincides with the reference image, and when it is within a prescribed error, the work 8 is gripped by the hand part 3b. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a control method and a control device for a mobile robot, which feedback-controls the position and orientation of an arm based on the data of an image taken by an imaging means provided on the arm.

[0002]

In a mobile robot in which the robot is mounted on a mobile carriage, the actual work is started in a state where the robot is stopped at a predetermined work position, but the position accuracy when stopped is poor. Therefore, if you did not stop exactly at the proper stop position when you stopped at the working position,
The position of the mobile robot is corrected so as to be the regular stop position, and the actual work is started when the position correction is completed. Although various methods have been proposed as such a correction method, as an example, a marker is provided on a workbench, the marker is photographed by a camera provided on a mobile robot, and the position of the photographed marker is preset. When it coincides with the position, it is determined that it is located at the regular work position.

However, the work of correcting the mobile robot from the stop position to the normal position based on the image taken by the camera is extremely troublesome, and therefore, the tact time from the stop of the mobile robot to the start of the actual work. Has the drawback of being long.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a control method and a control device for a mobile robot capable of reducing the working time.

[0005]

According to the first aspect of the present invention, the work object is photographed by the photographing means in a state where the arm of the mobile robot is positioned at the target position relative to the work object, and the photographed image is taken. It is stored as a reference image.

Then, at the time of actual work, in the running state of the mobile robot, the visual feedback control is performed to control the arm to the relative target position so that the image picked up by the image pickup means and the reference image coincide with each other by the visual feedback control. When the arm can be controlled to the relative target position regardless of the traveling state of the mobile robot, the actual work is started. Accordingly, when the actual work can be performed on the work object while the mobile robot is traveling, the tact time until the actual work is performed can be shortened.

Further, since the actual work is started when the arm can be controlled to the relative target position while the mobile robot is stopped, even if the stop position of the mobile robot deviates from the regular stop position, Since it is possible to perform actual work on the work object without correcting the stop position,
The tact time from the stop of the mobile robot to the execution of actual work can be shortened accordingly.

According to the second aspect of the present invention, when the work object is located in the image taken by the image taking means while the mobile robot is traveling, deceleration for stopping the mobile robot at the work position is started. Therefore, since the visual feedback control is executed, it is possible to significantly reduce the work time as compared with the case of executing the actual work after stopping at the work position.

According to the third aspect of the invention, when the work object is located in the image taken by the image taking means, the visual feedback control is executed while the mobile robot is decelerated for the actual work, and the actual work is performed. When it finishes, it accelerates the mobile robot, so it is possible to execute visual feedback control at low speed while moving the mobile robot at high speed.
The working time can be shortened.

According to the fourth aspect of the present invention, prior to executing the actual work, the work target is photographed by the photographing means in a state where the arm of the mobile robot is positioned at the target position relative to the work target, The captured image is stored in the storage means as a reference image. Then, during the actual work, the control means controls the arm to the relative target position by visual feedback control in the running state of the mobile robot so that the image captured by the image capturing means and the reference image stored in the storage means coincide with each other. . At this time, the control means starts the actual work when the arm can be controlled to the relative target position irrespective of the traveling state of the mobile robot, so that the tact time until the actual work is executed can be shortened.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION (First Embodiment) A first embodiment of the present invention will be described below with reference to FIGS. In the present embodiment, the case where the present invention is applied to a mobile robot having an arm mounted on a moving carriage is taken as an example.

FIG. 1 shows an external configuration of a mobile robot 1. The mobile robot 1 is constructed by mounting a vertical articulated (6-axis) type arm 3 on a moving carriage 2, for example. . A plurality of work equipments (work stations) are arranged along the traveling path of the mobile robot 1. The mobile robot 1 sequentially travels in front of each work equipment while traveling on a preset traveling path, and the arm 3 Various work such as work delivery, work assembly, machining, and inspection is performed.

Here, a guide line (not shown) made of a magnetic tape for guiding the mobile robot 1 along the traveling route is laid along the traveling route on the traveling route.
On the other hand, a guide sensor (not shown) for detecting the guideline is provided at the bottom of the moving carriage 2 at the center of the front and the rear, respectively, and the detection signal of the guide sensor is controlled by the control sensor. It is adapted to be input to the mobile robot controller 4 as means.

The mobile robot controller 4 is composed mainly of a CPU, a ROM and a RAM storing a control program.
(Corresponding to a storage means), map data indicating a layout of a traveling path, a work position, etc. is stored, and a work execution program is input. It is designed to control the vehicle so that it will follow the guidelines. The mobile robot controller 4 also controls the arm 3 based on a work execution program stored in the ROM.

The mobile robot 1 comprises a visual device 5 and a CCD camera 6 as a photographing means in addition to the mobile robot controller 4 described above. The visual device 5 and the CCD camera 6 are for visual feedback control (hereinafter referred to as visual servo), and
The D camera 6 is attached to the wrist portion 3 a of the arm 3. Further, the visual device 5 is mainly provided with a CPU, and is provided with a ROM and a RAM that store a program for position and orientation control, and is also provided with an interface circuit for inputting an image signal from the CCD camera 6, an image processing circuit, an image memory, and the like. It is configured.

The visual device 5 is adapted to give a control operation signal (position correction amount) to the mobile robot controller 4 based on image processing. In this case, the visual device 5
Registration data of the reference image obtained by the teaching operation executed in advance is stored in the image memory. In this teaching operation, for example, a manual operation using a teaching pendant (not shown) moves the arm 3 to a teaching target position for the work 2 (for example, a position and posture immediately before gripping the work by the hand portion 3b), and at this position, the CCD camera The work is photographed by 6, and the photographed image is registered (stored) in the visual device 5 as a reference image.

Here, as shown in FIG. 1, a work 8 as an object supplied at a predetermined position and posture is set on a work table 7 installed at a work position. A plurality of feature points (specific edges, holes, pasted markers, etc.) are set on the work 8 for visual recognition described later.

FIG. 2 shows an example of registration data of a reference image obtained by photographing the work 8 with the CCD camera 6. The registration data is set with the positional relationship between the reference point and the characteristic point in the field of view of the CCD camera 6 as the characteristic amount. That is, an arbitrary reference point P0 is set on the visual field, and the work 2
Characteristic points on the shape of are defined as, for example, characteristic points P1 to P5. Then, on the captured image, the reference point P0
And a vector V1 formed by connecting each of the feature points P1 to P5
The data of V5 and the data of the angles φ1 to φ5 formed by the adjacent vectors V1 to V5 are calculated, and this feature amount data is used as the registration data of the reference image.

In the actual work, the visual device 5
Is the work 8 with the CCD camera 6 in any position and orientation.
Is photographed, the photographed image data is processed by the visual device 5, and feature amounts (data of V1 to V5, angles φ1 to φ5) on the current image are similarly obtained as current data (feedback signal), and the current The position correction amount is adjusted so as to reduce the error (deviation) to 0 (for example) by comparing the data with the registration data of the reference image, that is, the image by the CCD camera 6 looks the same as at the time of teaching. It is calculated and output to the mobile robot controller 4 as a control operation signal. It should be noted that the position correction amount is determined by the wrist portion 3a of the arm 3.
Is converted into a velocity vector that gives the motion velocity of the above, and is given to the mobile robot controller 4.

The mobile robot controller 4 repeats the control of the position and orientation of the arm 3 based on the given position correction amount, and when the error converges to 0 or becomes a predetermined value or less by this control, that is, the arm 3 When the target position reaches the teaching target position, the gripping operation by the hand portion 3b is started. Therefore, the visual device 5 functions as a reference image storage unit, and the mobile robot controller 4 functions as a control unit that controls the operation of the hand unit 3b.

In the present embodiment, the mobile robot 1
When the workpiece 8 is gripped by the hand portion 3b at the working position, the tact time until the actual work can be started is shortened as follows. That is, in the teaching process,
First, in order to register the above-mentioned reference image, the mobile robot 1 is moved to a relative target position immediately before gripping the work 8 with the arm 3 by a manual operation of the teaching pendant while the mobile robot 1 is stopped at a predetermined work position. The work 8 is photographed at this relative target position, and the feature amount of the work 8 on the image at that time is stored in the visual device 5 as registration data of the reference image.

Further, when the work 8 is assembled to the assembly target, the arm 3 is moved to a relative target position which is a position immediately before the work 8 is assembled to the assembly target, and at this relative target position. The assembling body is photographed, and the feature amount of the work 8 on the image at that time is stored in the visual device 5 as registration data of the reference image. As described above, when the registration of the reference image is completed, the actual work by the mobile robot 1 can be executed.

FIG. 3 shows a flowchart of the operation of the mobile robot 1 in which the work 8 is held by the arm 3. The operation of the mobile robot 1 is characterized in that the actual work can be executed without stopping the mobile robot 1. In FIG. 3, the mobile robot controller 4 starts the movement of the moving carriage 2 (S101). At this time, the visual device 5 captures the image captured by the CCD camera 6 (S102) and determines whether the work 8 can be detected in the captured image (S103). In this case, the work 8 is detected even while the mobile robot 1 is moving, and when the work 8 is detected, the arm 3 is controlled by the visual servo so as to reach the relative target position (S10).
4). Then, when the arm 3 can be controlled to the relative target position, the actual work is executed (S105: YES). That is, the gripping operation for the work 8 by the hand portion 3b is executed (S106).

Here, after the time when the arm 3 reaches the relative target position, the control of the arm 3 is executed based on the prediction. That is, since the hand unit 3b is on the verge of gripping the work 8, the gripping operation can be performed by predicting the speed vector to which the gripping motion is applied while maintaining the speed vector of the hand unit 3b at that time. Can be done.

By storing the images until the work 8 is gripped as continuous reference images, the work 8 can be gripped by the arm 3 even while the mobile robot 1 is running. In order to perform such an operation, a position that is an image captured at a position before gripping and at a position where the hand portion 3b does not collide is stored as a relative target position, and such a relative target position is stored. The work 8 can be gripped by controlling the arm 3 so that

By the way, when the arm 3 cannot be controlled to the relative target position before the mobile robot 1 stops at the work position, the arm 3 is controlled to the relative target position by the visual servo while the mobile robot 1 is stopped. However, the positioning accuracy of the stop position of the mobile robot 1 is about several mm, which is extremely poor in the position accuracy for gripping or assembling the work 8, but it is possible to move by executing the above-mentioned visual servo. It is possible to position the arm 3 with respect to the work 8 with high accuracy without executing the position correction of the robot 1.

In summary, in the visual servo, CCD
Since the arm 3 is controlled using the image information sequentially obtained from the camera 6, even when the positional relationship between the arm 3 and the work 8 is relatively moving, such as when the mobile robot 1 is running. , The arm 3 can be controlled to a relative target position based on the work 8. Further, since the final position accuracy when gripping the work 8 depends only on the relative position accuracy of the arm 3 with respect to the work 8, even if the position accuracy of the mobile robot 1 is not sufficient, high positioning accuracy is achieved. After the actual work can be performed with the arm 3 in the state where the work 8 is gripped by the hand portion 3b, the arm 3 is assembled based on the image just before the work 8 is assembled to the assembly body as described above. It can be attached to the body, and even in this case, it can be executed even while the mobile robot 1 is running by the visual servo.

According to such an embodiment, even when the mobile robot 1 is in a traveling state, when the work 8 can be detected by the CCD camera 6, the relative movement of the arm 3 on the verge of gripping the work 8 by the visual servo. It is characterized in that the actual work is executed when the target position is controlled so that the relative target position can be controlled. Therefore, unlike the conventional example in which the mobile robot is stopped and then the actual work is started, the actual work can be executed while the mobile robot 1 is traveling, and the tact time until the actual work is significantly shortened. You can

Moreover, while the mobile robot 1 is running, the arm 3 cannot be controlled by the visual servo so as to reach the relative target position on the verge of gripping the work 8, and the arm 3 is eventually stopped while the mobile robot 1 is stopped. Even when the mobile robot 1 can be controlled to the relative target position, the position correction of the mobile robot 1 is not necessary. Therefore, unlike the conventional one in which the position correction is necessary after the mobile robot is stopped, the mobile robot 1 is stopped. The takt time from the start to the actual work can be greatly reduced.

(Second Embodiment) Next, a second embodiment of the present invention will be described with reference to FIGS. The second embodiment is characterized by performing visual servo by utilizing the fact that the mobile robot 1 starts deceleration in order to stop at the work position.

In this embodiment, the mobile robot 1 is
As shown in FIG. 4, an example is shown in which the work 11 gripped by the arm 3 is installed on the installation jig 12 placed on the workbench 7, and in order to execute the installation work,
With the arm 11 controlling the workpiece 11 to the relative target position immediately before installation on the installation jig 12, the CCD camera 6
And the photographed image is stored as a reference image.

Now, in FIG. 5 showing the installation work of the work 11 by the mobile robot controller 4, when the installation jig 12 can be detected in the image taken by the CCD camera 6 (S203: YES), the mobile robot 1 is stopped. To start the deceleration (S204), when the arm 3 can be controlled to the relative target position by the visual servo (S204).
206: YES), the work 11 is installed on the installation jig 12.

In this case, the timing at which the mobile robot 1 starts decelerating is executed with priority over the original deceleration timing preset in the mobile robot 1. Further, the operation of installing the work 11 on the installation jig 12 is executed regardless of whether the mobile robot 1 is running or in a stopped state.

According to such an embodiment, when the work 11 is recognized by the CCD camera 6, the deceleration for stopping the mobile robot 1 is started, so that even when the mobile robot 1 is decelerated. The arm 3 can be controlled, and the working time can be greatly shortened as compared with the conventional example in which the mobile robot stops and then the actual work is performed.

Moreover, the deceleration of the mobile robot 1 is started immediately when the installation jig 12 is detected without waiting for the normal deceleration timing. It is possible to further reduce the tact time until the actual work.

(Third Embodiment) Next, a third embodiment of the present invention will be described with reference to FIG. The third embodiment is characterized in that the speed of the mobile robot 1 is reduced when performing the visual servo.

In the first embodiment, when the mobile robot 1 is moving at high speed, it may be difficult to hold the work 8 by the visual servo due to the relationship with the response speed. Therefore, in the present embodiment, as shown in FIG. 6 showing the operation of the mobile robot controller 4, when the work 8 is recognized (S303: YES), deceleration for work is executed (S304), and the visual servo is performed. Arm 3 by
Is controlled to the relative target position (S306: YE
S) After executing the actual work (S308), the mobile robot 1 is accelerated to move to the next work point at an optimum speed.

According to such an embodiment, the speed of the mobile robot 1 is switched between when the mobile robot 1 moves to the work position and when the arm 3 operates, so that the mobile robot 1 travels at high speed. While doing so, the work 8 can be reliably gripped, and a work without waste can be realized.

The present invention is not limited to the embodiments described above and shown in the drawings, and the following expansions and modifications are possible. The CCD camera 6 is not limited to the wrist unit 3a and may be attached to another arm unit. The form of the arm is not limited to the vertical articulated type. It can be widely applied not only when gripping or assembling a workpiece to an assembly object but also when delivering an object to the other side, and the other side may be a workbench on which the object is placed or a pallet or the like. May be.

[Brief description of drawings]

FIG. 1 is an overall schematic configuration diagram according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of a storage form of a captured image of a work.

FIG. 3 is a flowchart showing an actual work operation of the mobile robot controller.

FIG. 4 is a view corresponding to FIG. 1 showing a second embodiment of the present invention.

FIG. 5 is a view corresponding to FIG.

FIG. 6 is a view corresponding to FIG. 3 showing a third embodiment of the present invention.

[Explanation of symbols]

1 is a mobile robot, 2 is a moving carriage, 3 is an arm, 3a is a hand unit, 4 is a robot controller (control means), 5
Is a visual device (storage means), 6 is a CCD camera (imaging means), 8 is a work, 11 is a work, and 12 is an installation jig.

Continued front page    (72) Inventor Takashi Naito             Aichi Prefecture Nagachite Town Aichi District             Ground 1 Toyota Central Research Institute Co., Ltd. (72) Inventor Hideki Nomura             Aichi Prefecture Nagachite Town Aichi District             Ground 1 Toyota Central Research Institute Co., Ltd. F term (reference) 3C007 AS01 BS12 CS08 KS03 KS07                       KT01 KT05 LT06 MT02 WA16                 5H301 AA02 AA10 BB05 BB14 CC04                       CC07 CC08 CC10 DD02 EE31                       GG09 JJ01 JJ06

Claims (4)

[Claims] 1. A photographing means attached to an arm of a mobile robot, photographs a work object such as a work in advance, stores the photographed object as a reference image, and photographs the photographed by the photographing means while the mobile robot is running. Visual feedback control for controlling the position and orientation of the arm is executed at a relative target position where the image of the work object and the reference image are in the same state, and the arm is relatively moved regardless of the running state of the mobile robot. A control method for a mobile robot, characterized in that an actual work is started when the robot can be controlled to a target position. 2. When the work object is located in the image taken by the imaging means, deceleration for stopping the mobile robot at the work position is started, and then the visual feedback control is executed. The method for controlling a mobile robot according to claim 1, wherein: 3. The visual feedback control is executed in a state in which the mobile robot is decelerated when the work object is located in the image taken by the imaging means, and when the actual work is finished, the mobile robot is executed. The method for controlling a mobile robot according to claim 1, further comprising accelerating. 4. A mobile robot, a photographing means attached to an arm of the mobile robot, a storage means for storing an image of a work target such as a work previously photographed by the photographing means as a reference image, and the mobile robot. Visual feedback control for controlling the position and orientation of the arm to a relative target position in which the image of the work target imaged by the imaging unit and the reference image stored in the storage unit are in the same state in the traveling state And a control means for starting an actual work when the arm can be controlled to the relative target position regardless of the traveling state of the mobile robot.