JPH07244512A - Method for informing position and time of target to robot and tracking method - Google Patents

Abstract

PURPOSE:To execute accurate tracking by informing a robot of accurate time obtaining a picture of a target by a visual sensor together with the positional information of the target. CONSTITUTION:A work W is detected by a limit sensor 4 and transmitted to a picture processor 11. A CPU transmits the detected contents to a robot controller 20. The controller 20 starts a visual clock 5 after the lapse of proper time, and simultaneously, clears the count value of a counting register in a pulse coder 3. Immediately after the clear, photographing by a camera 12 is executed and the LED display density strings of the work W and the clock 5 are fetched as picture signals. The CPU analyzes these pictures, finds out the position of the work W and the time T0 concerned (point of time starting the opening of an electronic shutter) and transmits the position and time T0 to the controller 20. The controller 20 starts tracking utilizing the photographing time T0 and the work position data at the timing when the work W reaches a position appropriate for starting its tracking after starting the robot 30.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention recognizes a robot work object (hereinafter, simply referred to as "object") moving by a transfer means by a visual sensor and determines the position of the object by an industrial robot ( (Hereinafter, simply referred to as "robot") to make it useful for robot work, and in particular, a method of accurately notifying the robot of the time corresponding to the position of the object and tracking the robot by using this method. Regarding how to do it. As a specific example of performing tracking, for example, using the line tracking function that operates the robot while correcting the robot position according to the position of the work transferred on the line conveyor, There is an application that holds and picks up.

[0002]

2. Description of the Related Art When a robot is used to perform an operation on an object, if the object is stationary during the operation, the robot is controlled by a normal teaching / playback method or an offline teaching method. You can do it. Further, when the position of the target object varies, a method of measuring the position of the target object with a visual sensor and correcting the robot operation based on the measurement result is adopted.

However, in a process line or the like of a factory, there are many cases where it is desired to perform robot work while moving a work by a conveyor for reasons such as work efficiency. For example, the work (various parts, container-shaped material of plastic processed products, etc.) placed on the moving conveyor is grasped and picked up from the conveyor, and processing such as painting, marking, liquid injection, and mechanical processing is performed. This is the case when a robot system is applied to work.

In such a case, the desired work cannot be performed by reproducing the position of the robot taught under the condition that the work is stationary. Therefore, the tracking according to the movement path of the work is performed. It becomes necessary to perform the program regeneration operation.

By the way, in order to perform tracking, it is necessary to know at which position the object is located at each point of time including the robot operation. If the transfer speed of the object is slow, the required tracking accuracy is low, or the position or orientation of the object in the width direction of the conveyor does not vary, it is easy to place it at the appropriate reference position facing the transfer route. The arrival of the object is detected by various sensors (mechanical contact, light transmission or light reflection, etc.), and the movement amount of the conveyor is measured using a pulse coder from the detection time point as the starting point, and each time point thereafter. The method of calculating the object position in can be adopted.

However, when the transfer speed of the object is relatively high, the required tracking accuracy is high, or the position or orientation of the object in the width direction of the conveyor varies, this method is insufficient. The camera means (CCD camera, etc.) attached to the visual sensor is arranged at an appropriate fixed position facing the transfer path, and the sensor (mechanical contact, light transmission or light reflection, etc.) is used to detect the object. A method is adopted in which the object is photographed at an appropriate timing after the arrival is detected, the moving amount of the conveyor is measured using a pulse coder from the photographing time point, and the object position at each time point thereafter is calculated. Has been done.

[0007]

However, even if such a system is adopted, it is difficult to accurately specify the time when the object was photographed. This is mainly for the following reasons. Since a CCD camera or the like used for a visual sensor usually has an electronic shutter function, the time required for photographing itself (light detection time by the CCD element) is 100.
It is not difficult to set it to 1/0 or less, but there is a restriction on the timing when the electronic shutter opens (for example, about 1 /
In general, there is a timing at which photography can be performed every 60 seconds), and there is generally no means for accurately specifying the timing when the electronic shutter is opened.

Therefore, when the image acquisition time point is specified from the output time point of the photographing command (synchronization signal) issued from the control unit of the visual sensor, the time point at which the image obtained for the object is obtained is considerable. Duration (eg,
It is inevitable that an error of about 1/60 second) will occur. For example, if the conveyor speed is 600 mm / sec,
There is a possibility that a measurement error of 600/60 = 10 mm will occur. If the electronic shutter speed is 1/1000 second, the image flow (which appears as image disturbance in the interlace system) is evaluated to be 1 mm or less, and the range is well balanced with the normal robot positioning accuracy. Becomes

In addition to the problem of error due to the above factors, there is also a problem that the position of the object cannot be known unless the clock means is shared between the visual sensor and the robot using the visual sensor. .

Therefore, an object of the present invention is to provide a method for informing a robot of the time at which an image of an object is acquired together with the position information of the object with an accuracy commensurate with the shutter performance of the camera means attached to the visual sensor, Another object of the present invention is to provide a method for performing tracking of a robot based on the information on the time and the object position. Furthermore, the present invention also contemplates providing the above method which does not necessarily require the visual sensor and the robot to share a clock means.

[0011]

The present invention has a basic structure for solving the above-mentioned problems, in which "a visual clock means is arranged in the vicinity of a transfer means for transferring an object and the transfer means is used. The object and the visual clock being transferred by the transfer means by using a visual sensor having a camera means arranged at a position where both the object and the visual clock means being transferred can be included in the visual field. An image in which both means are included in the field of view is acquired, and by analyzing the image, a signal indicating the position of the object and a signal indicating the time when the image is acquired are generated in the visual sensor, and the object is detected. A signal indicating the position of the object and a signal indicating the time at the time of image acquisition to robot control means linked to the visual sensor, and notifying the position and time of the object to the robot. In which proposed a way. "

Further, the present invention provides a tracking method using the position / time information obtained by the above method, "a visual clock means is arranged in the vicinity of a transfer means for transferring an object, and the object is transferred by the transfer means. The object and the visual clock means being transferred by the transfer means using a visual sensor having a camera means arranged at a position where both the object and the visual clock means inside can be included in the visual field. A signal representing the position of the target object and a signal representing the time at which the image is acquired are generated in the visual sensor by acquiring an image in which both the fields of view are captured, and the signals are analyzed. A tracking method for transmitting to a robot control device that cooperates with the visual sensor, and using the transmitted information to execute robot control that follows the movement of the object. In the robot control device, the robot controller obtains a transfer distance of the transfer means starting from a time point when the clock means is in a reference time state before the image acquisition time, and a signal representing the transfer distance, the transmitted object. A position of the object during robot operation is sequentially obtained based on a signal representing the position of the object and a signal representing the image acquisition time, and the robot operation is controlled according to the obtained object position. The above-mentioned tracking method as a feature ”is proposed.

[0013]

According to the present invention, by capturing the image of the visual clock means at the same time as the image of the object, the time when the image of the object is acquired can be accurately determined without strictly controlling the photographing time by the camera attached to the visual sensor. Can be specified. The shooting timing may be set within a range in which the object does not deviate from the field of view of the camera, and it is not necessary for the visual sensor and the robot controller using the visual sensor to share the clock.

That is, highly accurate time management including the start of the visual clock is performed by the robot controller alone, and the visual sensor is released from strict time management, captured at an appropriate timing, and taken in. It suffices to perform only the role of analyzing the image and identifying the position of the object at the time of shooting and the display content of the visual clock at that time.

The robot controller obtains the transfer distance of the transfer means based on the transfer speed of the transfer means starting from the time when the visual timepiece is in the reference time state before the time of executing the photographing, and a signal representing the transfer distance. The position of the object during robot operation is sequentially obtained based on the signal indicating the position of the object and the signal indicating the image acquisition time, and the robot operation is controlled according to the obtained object position. By doing so, the robot can take a tracking trajectory.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing an example of the overall arrangement for carrying out the method of the present invention. In the figure, reference numeral 1 denotes a linear transfer conveyor, which is driven by a conveyor drive unit 2 which is controlled as an additional axis of a robot 30 described later. The rotation amount of the drive motor 2 included in the conveyor drive unit 2 is output by the pulse coder 3 in the form of a pulse train. Reference numeral 4 is a limit sensor for detecting a work W (hereinafter, the object is represented by the work W) placed on the conveyor 4 and transferred at a fixed position.
The position detection accuracy of the limit sensor 4 does not need to be so high, and an appropriate type such as a mechanical type or an optical type can be used. As soon as the limit sensor 4 detects the arrival of the work W (broken line position), a work arrival detection signal is output.

Reference numeral 10 represents a visual sensor including an image processing device 11 and a camera 12 such as a CCD camera, and reference numeral 121 represents a field of view of the camera 12. Further, reference numeral 5 is a visual clock means (hereinafter, simply referred to as “visual clock”) having a function of displaying time in a form recognizable by the visual sensor 10, and the robot controller 2
It is connected to 0. An important requirement for the arrangement position of the visual timepiece 5 is that when the work W (solid line position) is photographed by the camera 12, both the work W and the visual timepiece 5 have a visual field 121.
Being selected to fit within. The configurations and functions of the visual timepiece 5 and the image processing device 11 will be described later.

Next, reference numeral 20 is a robot controller for controlling the robot 30, which includes a CPU, a program memory,
An ordinary type having a data memory, an axis controller, a servo control circuit, etc. can be used. The robot controller 20 is
It is connected to the image processing device 11 of the visual sensor 10 via a general-purpose signal interface (not shown).

As described above, the drive unit of the conveyor 1 is controlled as an additional axis of the robot 30, and the rotation amount of the drive motor is set in the memory of the robot controller 20 as the output pulse number of the pulse coder 3. It is counted by the counter register.

FIG. 2 is a view exemplifying the visual timepiece 5, and as shown in the figure, a predetermined number (here, 10) of LEDs 52 and LEs for controlling the turning on / off thereof.
It is composed of a D drive circuit 51. The LED drive circuit includes a clock generator, a frequency dividing circuit, and a binary counter, and the binary counter is the robot controller 20.
When the visual clock start signal is received from the clock generator, the clock signal from the clock generator is divided by an appropriate division ratio and counting of known period pulses is started. The size of the known period pulse interval is preferably set to be approximately the same as the operation time of the electronic shutter of the camera 12. In the figure, the count value 01101000
The state of 01 is illustrated. When the binary counter receives the reset signal from the robot controller 20, the count value is cleared to 0000000000, and the 0 count state (reference time state) is held until the next visual clock start signal is received.

The count value of the binary counter may be converted into a decimal count value, and the time may be displayed by turning on / off three digits × 10 = 30 LEDs, for example. Also,
The robot controller 2 receives the pulse signal counted by the counter.
It is also possible to send from the CPU within 0.

Next, the configuration and function of the image processing apparatus 11 will be outlined. FIG. 3 is a block diagram showing an essential part of the image processing apparatus 11 shown in FIG. 1 and the elements connected to the image processing apparatus 11. The image processing apparatus 11 has a CPU (central processing unit) 101 composed of a microprocessor, and C
A camera interface 102, a frame memory 103, an image processor 104, a program memory 105, a monitor interface 106, a data memory 107, and a general-purpose signal interface 108 are connected to the PU 101 via a bus 109.

A camera 12 is connected to the camera interface 102, and when a photographing command (synchronization signal) is sent via the camera interface 10, the camera 12 is connected.
Electronic shutter function set to (the shutter speed is
Shooting is performed for one thousandth of a second), and the video signal is stored in the frame memory 103 in the form of a grayscale signal via the camera interface 102.

The video signals of the work W and the visual clock 5 stored in the frame memory 103 are stored in the program memory 10
5, the position of the work W (for example, the position of one or more specific corners) analyzed by the image processor 104 according to the analysis program stored in FIG. The measured elapsed time) is required. LED lighting as shown in FIG.
Since the software that identifies the time from the unlit image determines the brightness at a limited fixed position, it can be made extremely simple.

Reference numeral 40 is a monitor interface 106.
Is a monitor (CRT or the like) connected to, and is used for calling and displaying the image signal stored in the frame memory 103. The data memory 107 has an area for storing various setting values related to the system and the CPU 102.
It includes an area used for temporary storage of data necessary for various processes executed by. General-purpose signal interface 10
The reference numeral 8 is connected to the robot controller 20 and the limit sensor 4 so that various commands and data can be exchanged.

The processing procedure for carrying out the method of the present invention will be described below with reference to FIG.
This will be described with reference to the flowcharts (1) and (2). The program memory 10 of the image processing apparatus 11
5, the data memory 107 and the program memory and data memory (not shown) in the robot controller 20
It is assumed that a program for executing the processes described in these flowcharts and related setting data have been stored in advance.

Here, as the trajectory of the robot 30, proper timing (limit sensor 4) after the position and time data of the work W is received by the robot controller 20.
After a set time has elapsed from the output of the work arrival signal by (1), a movement from the standby position P0 to the appropriately set fixed point Q as the initial target position is started, and the work W is approached while performing the position correction by the tracking operation. Standby position P
It is assumed that a fixed point W0 on the work W (moving) is reached at a position relatively close to 0.

Each of the flowcharts in FIGS. 4A and 4B represents the processing contents by the CPU 101 of the image processing apparatus and the CPU (not shown) in the robot controller 20. When both processes are started by an appropriate external signal, first, the CPU of the robot controller 20 issues a command to reset the visual timepiece 5, and all the LEDs 52 are turned off (step M1). Then, the robot controller 20 obtains the speed V of the conveyor 1 from the pulse output frequency of the pulse coder 3 (step M2), and also informs the image processing device (step M3). In the present embodiment, the motor of the conveyor drive unit 2 is controlled as an additional axis of the robot 30, so a speed command value can be used instead.

The image processing apparatus 11, which has acquired the data of the conveyor speed V in step S1, immediately starts the limit sensor 4
To wait for the arrival detection signal of the work W from (step S2). When the work W arrives at the position indicated by the broken line in FIG. 1, the limit sensor 4 detects this and the arrival of the work is transmitted to the image processing apparatus 11. The CPU 101 immediately transmits this to the robot controller 20 (step S3).

From the limit sensor detection position of the work W,
The time required to enter the visual field 121 of the camera 12 is set to T1, and the image processing apparatus 11 is ready to wait for the time T1 to elapse (step S4). Time T1 is from the position detected by the conveyor speed V and the limit sensor 4 to the visual field 12
It can be obtained from the distance L1 up to 1 (see FIG. 1), but it does not require exact calculation.

On the other hand, in the robot controller 20, the time T1−ΔT has elapsed since the work arrival signal was received in step M4.
After the lapse of (ΔT is an appropriately set slight time) (step M5), a start command is sent to the visual timepiece 5 and the visual timepiece 5 is started.
Start the time display by turning on the LED 52 of the
At the same time, the count value of the count register of the pulse coder 3 is cleared (step M6). This synchronization can be performed with extremely high time accuracy.

On the other hand, on the side of the image processing apparatus 11, since the time T1 elapses immediately after this (approximately after ΔT elapses), a photographing command is sent to the camera 12 to acquire an image using the electronic shutter function (step S5). . By photographing at this point, the work W and the LED display light and dark columns of the visual timepiece 5 are captured as image signals. CPU 101
Analyzes the image using the image processor 104 to obtain the position of the work W and the time T0 at that time (that is, when the electronic shutter is opened), and transmits it to the robot controller 20 (step S6). . This completes the role of the visual sensor 10.

Now, in the robot controller 20, immediately after the visual timepiece 5 is started, the time count for waiting the time T2 is started (step M7). This time T2 is a time required for the work W to move from a shooting position (solid line position) to a position suitable for the robot 30 to start moving by a distance L2 (illustrated in FIG. 1). This waiting time T
Since 2 has a considerable tolerance, no strict calculation is required (it can be handled by the tracking function described later).

The robot controller 20 waits for transmission of the data of the work position and the time T0 obtained in step S6, and shortly after the work W is photographed in the visual field 121,
This is received (step M8). The precision of this time T0 (the elapsed time from the start of the visual timepiece 5) is extremely high, and it can be evaluated with an error (for example, one thousandth of a second) that is comparable to the opening time of the electronic shutter. Further, also with respect to the position of the work W, it is possible to expect an accuracy of about the moving distance in the opening time of the electronic shutter. Conveyor speed 600mm, electronic shutter open time 10
If it is 1/00 second, the error will be about 0.6 mm,
It is a value that is sufficiently commensurate with the position control accuracy of the robot.

The robot controller 20 which has acquired the data of the work W at the time T0 indicates that the work W is the robot 3
Wait for the elapse of time T2 to wait for a position suitable for starting 0 to start tracking (step M9). Then, when the time T2 has elapsed, the calculation of the interpolation position is started (step M10), and the interpolation position is corrected using the data received in step M8. The correction contents are as follows.

Position vector data representing the n-th interpolation position before correction calculated from the trajectory toward the initial target position Q is represented by <qn>. Further, the position vector data of the work position obtained in step M8 is set as <X>. The elapsed time from the start of the visual timepiece 5 to the calculation of the n-th interpolation position is tn, and the movement amount (think of it as a vector amount; calculated from the pulse coder count value) during that period is <dn>.

Then, the elapsed time from the shooting time to the calculation of the n-th interpolation position measured is tn-T0. Therefore, the conveyor movement amount from the time of photographing to the time of calculating the n-th interpolation position is <dn> / (tn-T0).
Therefore, if the position of the work W during the n-th interpolation position calculation is <Xn>, then <Xn> = <X> + <dn> /
(Tn-T0). Deviation <Xn between this value and <qn>
>-<Qn> is the correction amount required for tracking.

Such correction calculation is performed sequentially,
Robot movement with the corrected interpolation position as the target position
The process is repeated until the robot 30 reaches the fixed point W0 on the work W (steps M10 to M1).
3). After the robot 30 reaches the fixed point W0 on the work W, a necessary operation (grasping of the work W or the like; not shown) is performed and the process is ended. The work W shown by hatching in FIG. 1 is an example of the position of the robot 30 when it reaches the fixed point W0, and the trajectory R1 is a teaching trajectory and R2 is a tracking trajectory. It is a thing.

In the above description, the conveyor 1 is described as a linear transfer type. However, even in the case of an arc transfer conveyor, a slight correction is added to the tracking correction calculation.
It will also be apparent from the above description that the method of the present invention can be applied to arc tracking.

[0040]

According to the present invention, there is provided a method of notifying the robot of the time when an image of an object is acquired together with the position information of the object with an accuracy commensurate with the shutter performance of the camera means attached to the visual sensor. . Further, the position of the object can be known even if the clock means is not shared between the visual sensor and the robot using the visual sensor.

By using the information provided by this notification method, it is possible to perform accurate tracking of the robot work on the work or the like transferred on the conveyor.

[Brief description of drawings]

FIG. 1 is a block diagram of an essential part showing an example of the overall arrangement when carrying out the method of the present invention.

FIG. 2 is a diagram exemplifying a display form of a visual timepiece used in the present invention.

FIG. 3 is a block diagram showing a main part of the image processing apparatus shown in FIG. 1 and elements connected to the image processing apparatus.

4A and 4B are schematic diagrams of processing contents by a CPU of an image processing apparatus and a CPU in a robot control apparatus, respectively, for an embodiment.

[Explanation of symbols]

 1 Conveyor 2 Conveyor drive unit 3 Pulse coder 4 Limit sensor 5 Visual clock 10 Visual sensor 11 Image processing device 12 Camera 20 Robot controller 30 Robot W work W0 Fixed point on work

Continuation of front page (51) Int.Cl. ⁶ Identification number Office reference number FI technical display location G05D 3/12 L 7609-3H 305 S 7609-3H G06T 1/00

Claims (2)

[Claims] 1. A position where visual clock means is arranged in the vicinity of a transfer means for transferring an object, and both the object and the visual clock means being transferred by the transfer means can be included in the visual field. An image in which both the object being transferred by the transfer means and the visual clock means are in the field of view is obtained by using a visual sensor having a camera means arranged in the field, and the object is analyzed by analyzing the image. A signal representing the position of an object and a signal representing the time of the image acquisition time are generated in the visual sensor, and the signal representing the position of the object and the signal representing the time of the image acquisition time are linked to the visual sensor. A method for informing a robot of the position and time of an object, characterized by transmitting the information to a robot control means. 2. A position where visual clock means is arranged in the vicinity of a transfer means for transferring an object, and both the object and the visual clock means being transferred by the transfer means can be included in the visual field. An image in which both the object being transferred by the transfer means and the visual clock means are in the field of view is obtained by using a visual sensor having a camera means arranged in the field, and the object is analyzed by analyzing the image. A signal representing the position of an object and a signal representing the time at the time of image acquisition are generated in the visual sensor, the signals are transmitted to a robot controller linked to the visual sensor, and the transmitted information is used. In the tracking method for executing the robot control following the movement of the object, the robot control device uses the clock means as a reference time before the image acquisition time. The transfer distance of the transfer means is obtained starting from a time point in a state, and at the time of the robot operation based on the signal representing the transfer distance, the signal representing the position of the transmitted object and the signal representing the image acquisition time. The position of the object is sequentially obtained,
The tracking method, wherein the robot operation is controlled according to the obtained object position.