JP2004001122A - Picking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a picking device which can carry out picking of a moving workpiece. <P>SOLUTION: The location and position of the workpiece 2 are correctly recognized by a primary detecting operation by using a first stereo camera 6, and then the amounts of displacement of the location and position of the workpiece 2 are calculated by repeatedly operating a second stereo camera 7 while a hand 4 is moved to trace the location of the workpiece 2 on the move. Further, the amounts of displacement of the location and position are corrected, and then a desired location of the final picking operation and a position of the hand 4 are controlled. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an improvement in a picking device that grasps a workpiece by grasping the position and orientation of the workpiece, and particularly relates to an improvement for improving accuracy and processing speed related to detection of the position and orientation.
[0002]
[Prior art]
As a technique for improving the accuracy of grasping the position and orientation of a workpiece by using two camera systems, for example, a robot device disclosed in Japanese Patent Application Laid-Open No. 2000-288974 has already been proposed.
[0003]
The robot device disclosed in JP-A-2000-288974 is provided with a first visual sensor composed of a two-dimensional visual sensor and a second visual sensor composed of a two-dimensional visual sensor or a three-dimensional visual sensor at the tip of the wrist of the robot. The rough position and posture of the work are detected by one visual sensor, the wrist of the robot is moved to a position near the work position obtained by the first visual sensor, and the second visual sensor is operated to operate the robot. The position and the posture are strictly measured, and the picking operation by the industrial robot is performed.
[0004]
[Problems to be solved by the invention]
However, in the robot apparatus of Japanese Patent Application Laid-Open No. 2000-288974, since the first visual sensor is constituted by a two-dimensional visual sensor, it is necessary to perform highly accurate recognition by an initial-stage detection operation using the first visual sensor. As a result, it is difficult to effectively narrow down an area to be photographed in a precise measurement operation performed using the second visual sensor, and the time required for capturing images and performing arithmetic processing is difficult. The adverse effect such as an increase in the size of the image occurs.
[0005]
In addition, as a result of an increase in the time required for capturing images and performing arithmetic processing, it is practically impossible to detect the position and orientation of a moving workpiece and perform a picking operation.
[0006]
[Object of the invention]
Accordingly, an object of the present invention is to solve the above-mentioned drawbacks of the prior art, reduce the time required for image capture and arithmetic processing, and recognize the position and orientation of a work with high speed and high accuracy. It is another object of the present invention to provide a picking device that can handle picking of a moving workpiece.
[0007]
[Means for Solving the Problems]
The present invention relates to an articulated industrial robot having a hand for gripping a work, a camera for grasping the position and orientation of the work, and a picking device including a computer system for driving and controlling the industrial robot and the camera. In order to achieve the above object, in particular,
While the first stereo camera having a large parallax is fixedly installed near the mounting position of the work, the second stereo camera having a relatively small parallax is arranged near the hand in the industrial robot,
In the computer system, position and orientation calculating means for determining the position and orientation of the work based on the image captured by the first stereo camera,
The hand is moved to the vicinity of the work position calculated by the position and orientation calculating means, the second stereo camera is operated to capture an image, and the actual position of the work relative to the position and orientation of the work determined by the position and attitude calculating means is calculated. A shift amount calculating means for obtaining a shift amount between the position and the actual posture;
Position and orientation for obtaining the final work position and orientation based on the position and orientation of the work determined by the position and orientation calculation means and the actual position and actual position shift of the work determined by the shift amount calculation means Correction means;
A picking operation control means for controlling the target position of the picking operation and the posture of the hand based on the position and posture of the work obtained by the position and posture correcting means is provided.
[0008]
In the above configuration, first, the first stereo camera installed near the work placement position captures an image around the work, and the position and orientation calculation means analyzes the image to determine the position and orientation of the work. . Since the first stereo camera is fixedly installed independently of the industrial robot, it is possible to take a large parallax between the cameras constituting the stereo camera, and the initial stereo camera using the first stereo camera is used. The accuracy of recognition of the position and orientation of the work in the step detection operation is improved.
Next, the shift amount calculating means moves the hand to the vicinity of the work position calculated by the position and attitude calculating means, activates the second stereo camera to capture an image, and is obtained by the position and attitude calculating means. The amount of deviation between the actual position and the actual posture of the work with respect to the position and the posture of the work is obtained. Since the recognition accuracy of the position and orientation of the work by the first stereo camera is high, the second stereo camera can reliably capture the work only by narrowing down the area to be photographed and photographing. In other words, by reducing the capacity of the image data to be analyzed, the time required for calculating the amount of deviation between the actual position and the actual posture of the work is reduced, and high-speed arithmetic processing is realized. become.
Since the second stereo camera is mounted near the hand on the industrial robot, the parallax between the cameras that make up the stereo camera cannot be as large as the first stereo camera, but when the camera is close to the work Since the work can be photographed, the deviation of the position and the posture of the work can be detected with sufficient accuracy.
Then, based on the position and orientation of the work determined by the position and orientation calculating means and the actual position and actual position of the work determined by the shift amount calculating means, the position and orientation correcting means determines the final position and orientation of the workpiece. The posture is obtained, and the picking operation control means controls the target position of the picking operation and the posture of the hand based on the position and posture of the work obtained by the position and posture correction means.
[0009]
Also, a multi-joint type industrial robot having a conveyor for transporting a workpiece, a hand for gripping the workpiece, a camera for grasping the position and orientation of the workpiece, and a computer system for driving and controlling the industrial robot and the camera. In the case of a picking device with
While fixedly installing a first stereo camera having a large parallax for photographing a work conveyed on a conveyor at a predetermined position, a second stereo camera having a relatively small parallax near a hand in an industrial robot is provided. Deploy,
The computer system, a position and orientation calculating means for determining the position and orientation of the work at the predetermined position based on an image captured by a first stereo camera,
Tracking operation control means for determining the current position of the work based on the work position and the conveyor speed and elapsed time determined by the position and orientation calculation means, and moving the hand following the current position of the work;
During the operation of the tracking operation control means, the second stereo camera is repeatedly operated to take an image in a coordinate system which is translated together with the hand and the conveyor, and the position of the work with respect to the position and posture of the work determined by the position and posture calculation means is taken. Deviation amount calculating means for calculating the deviation amount between the actual position and the actual posture;
Based on the position and orientation of the work determined by the position and orientation calculation means and the actual position and actual position shift of the work determined by the shift amount calculation means, the final position in the coordinate system that translates with the hand and the conveyor. Position and orientation correction means for determining the position and orientation of a typical work,
A picking operation control means for controlling the target position of the picking operation and the posture of the hand based on the position and posture of the work obtained by the position and posture correcting means is provided.
[0010]
When such a configuration is applied, the tracking operation control means obtains the current position of the work based on the work position obtained by the position and orientation calculation means, the conveyor speed and the elapsed time, and causes the current position of the work to follow. Since the hand is moved, even if the work moves, the relative positional relationship between the work and the hand is always kept constant unless the position and posture of the work on the conveyor change. .
Therefore, there is no change in the positional relationship between the hand and the work due to the time lag from the start of shooting by the first stereo camera to the drive control of the hand by the picking operation control means. As in the case where the camera calculates the deviation between the actual position and the actual posture of the workpiece and controls the target position of the picking operation and the posture of the hand, a reliable picking operation can be performed.
Also, if a change occurs in the relative positional relationship between the work and the second stereo camera, this means that the position or posture of the work on the conveyor has changed due to collapse of the load, etc. The position / posture correction unit corrects the target position of the picking operation and the posture of the hand according to the amount of deviation of the posture, so that the picking operation can be reliably performed even if the work conveyed by the conveyor collapses. It becomes possible.
In addition, the shift amount calculating means repeatedly operates the second stereo camera while the tracking operation control means is operating, captures an image in a coordinate system which is translated along with the hand and the conveyor, and obtains the actual position and actual posture of the work. Even if the position or posture of the work on the conveyor changes due to collapse of load, etc., the position and posture of the work detected in the immediately preceding process are initialized. It is possible to execute the calculation processing of the shift amount as the position, and it is possible to greatly reduce the time required for calculating the shift amount of the actual position and the actual posture of the work. Also, even if a large change occurs as a result in the position and orientation of the workpiece on the conveyor, the amount of displacement of the position and orientation during one shooting cycle is not large, so that the position and orientation may change discontinuously. The deviation amount of the actual position and the actual posture of the work can be tracked without causing an error due to an increase in the load of the data search.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic diagram showing a simplified configuration of a picking device 1 according to an embodiment to which the present invention is applied.
[0012]
The picking device 1 of this embodiment includes a conveyor 3 for transporting a work 2, an articulated industrial robot 5 including a hand 4 for holding the work 2, and a position and a posture of the work 2. It comprises a first stereo camera 6 and a second stereo camera 7, and a computer system 8 that drives and controls the industrial robot 5 and the first and second stereo cameras 6, 7.
[0013]
The first stereo camera 6 captures the workpiece 2 when the workpiece 2 conveyed on the conveyor 3 passes through a predetermined position. The first stereo camera 6 has three cameras 6a, 6b, 6c, and is fixedly installed on one side of the conveyor 3. The first stereo camera 6 captures an image in accordance with a command from a numerical controller 9 constituting a part of the computer system 8, and transfers data of a captured image to the numerical controller 9.
[0014]
The second stereo camera 7 is also composed of three cameras 7a, 7b, 7c like the first stereo camera 6, but is mounted near the hand 4 on the industrial robot 5 and moves integrally with the hand 4. Due to this, the parallax between the cameras 7a, 7b and 7c is relatively small, and the cameras 7a, 7b and 7c are compactly arranged as shown in FIG.
[0015]
The computer system 8 is composed of a robot controller 10 attached to the industrial robot 5 and a general-purpose numerical controller 9, of which the robot controller 10 is exclusively used for drive control of the industrial robot 5 and for numerical control. The device 9 is used to realize additional functions such as image processing and the like, and is connected so that data can be transferred between them.
[0016]
In this embodiment, it is assumed that a commercially available robot controller 10 is used without any modification, and a numerical controller 9 for realizing additional functions such as image processing is added to the robot controller 10 to add a computer system 8 However, in the case where the robot controller 10 is to be redesigned from the beginning including the implementation of additional functions such as image processing, the single robot controller 10 itself can be used as the computer system 8. It is. Conversely, the drive control function of the industrial robot 5 may be implemented in the numerical control device 9, and the single numerical control device 9 may be a computer system 8.
[0017]
As described above, there is no particular limitation on the configuration of the computer system 8, but here, as an example, a structure in a case where the computer system 8 is configured by the robot controller 10 and the numerical controller 9 will be briefly described.
[0018]
As shown in FIG. 3, a robot control device 10 constituting a part of the computer system 8 includes a ROM 11 storing a basic control program, and a non-volatile memory 12 storing an operation program created by a user. A CPU 13 for executing various arithmetic processes based on an operation program stored in the nonvolatile memory 12 and a control program stored in the ROM 12, a RAM 14 used for temporarily storing data of the arithmetic process, and the like; And an axis control circuit 15 that drives and controls servomotors (not shown) of each axis of the industrial robot 5 in accordance with a command from the CPU 13.
[0019]
The axis control circuit 15 drives the servomotor of each axis based on the movement command given from the CPU 13 to control the position and posture, the moving speed, and the driving torque of the hand 4 of the industrial robot 5.
[0020]
The manual data input device 16 with a display device is used for editing operation programs and the like, and the input / output interface 17 is used for data transfer between the robot control device 10 and the numerical control device 9.
[0021]
As shown in FIG. 4, the numerical controller 9 includes a ROM 18 storing a system program, a non-volatile memory 19 storing an application program and the like used for image analysis, a CPU 20 for arithmetic processing, , A manual data input device 22 with a display device, an input / output interface 23, and a frame memory 24 for image capture. The frame memory 24 may be constituted by a part of the RAM 21.
[0022]
The input / output interface 23 is used to transfer data by connecting the numerical controller 9 and the robot controller 10, and image data captured by three cameras 6 a, 6 b, and 6 c of the first stereo camera 6. Also, image data captured by the three cameras 7a, 7b, 7c of the second stereo camera 7 is sent to the frame memory 24 via the input / output interface 23.
[0023]
FIG. 10 shows an example of a data array in the frame memory 24. In this embodiment, three sets of pixel data of 640 dots × 480 dots are stored at a density of 256 gradations. Shall be determined appropriately in accordance with
[0024]
Further, as shown in FIG. 1, a fixed position detection sensor 25 for detecting that the work 2 conveyed by the conveyor 3 has reached a predetermined photographing position is provided near the conveyor 3. The signal from the fixed position detection sensor 25 is also input to the CPU 20 of the numerical controller 9 via the input / output interface 23.
[0025]
In the non-volatile memory 19, some three-dimensional shape data corresponding to works 2 having various shapes that may be handled are registered in advance. FIG. 11 shows an example of the three-dimensional shape data stored in the nonvolatile memory 19.
[0026]
FIGS. 5 to 7 are flowcharts showing an outline of picking control executed by the CPU 20 of the numerical controller 9. FIGS. 8 to 9 show the CPU 13 of the robot controller 10 receiving a command from the numerical controller 9. It is the flowchart which showed the outline of the pulse distribution process performed.
[0027]
Next, with reference to FIGS. 5 to 9, the processing operation of the CPU 20 as the position / orientation calculating means, the deviation amount calculating means, the position / orientation correcting means, the processing operation of the CPU 20 and the CPU 13 as the tracking operation control means, and The processing operation of the CPU 13 as picking operation control means will be described.
[0028]
The CPU 20 that has started the picking control first determines whether or not the fixed position detection signal from the fixed position detection sensor 25 has been input, that is, whether or not the workpiece 2 conveyed by the conveyor 3 has reached a predetermined shooting position. Is determined (step a1), and if the fixed position detection signal is not detected, the standby state for waiting for the input of the fixed position detection signal is held.
[0029]
When the work 2 reaches the predetermined shooting position and the fixed position detection signal from the fixed position detection sensor 25 is confirmed, the CPU 20 restarts the elapsed time measurement timer t and moves the work 2 to the predetermined shooting position. The measurement of the elapsed time from the arrival is started (step a2), and the first stereo camera 6 is operated to photograph the work 2 at the predetermined photographing position and capture images from the cameras 6a, 6b, 6c. (Step a3) These image data are stored in the frame memory 24 (Step a4). FIG. 12 shows an example of the captured image.
[0030]
Next, the CPU 20 functioning as the position and orientation calculation means starts an application program used for image analysis, analyzes the image data of the work 2 stored in the frame memory 24, and obtains three-dimensional information (step a5). After comparing the three-dimensional information with a plurality of three-dimensional shape data registered in the non-volatile memory 19 to specify the shape of the work 2 currently being worked on (step a6), The positions X, Y, Z of the work 2 and the postures α, β, γ of the work 2 at the time of execution of step a3 are obtained (step a7). An example of three-dimensional information obtained by analyzing the image data of the work 2 stored in the frame memory 24 is shown in FIG. 13, and the three-dimensional information of the work 2 and the three-dimensional shape registered in the nonvolatile memory 19 are shown. FIG. 14 shows an example of collation with data.
[0031]
However, the positions X, Y, and Z of the work 2 are coordinate systems based on the fixed first stereo camera 6, and the directions of the respective coordinate systems are as shown in FIG. In the postures α, β, and γ of the work 2, α is the inclination of the posture around the X axis, β is the inclination of the posture around the Y axis, and γ is the inclination of the posture around the Z axis.
[0032]
As described above, since the first stereo camera 6 is fixedly installed on the side of the conveyor 3 independently of the industrial robot 5, the parallax between the cameras 6a, 6b, 6c constituting the stereo camera 6 is provided. And the accuracy of the position and orientation recognition of the work 2 in the detection operation using the first stereo camera 6 can be greatly improved.
[0033]
Next, the CPU 20, which functions as a part of the tracking operation control means, transfers the conveyor speed V (set value) of the conveyor 3, the current value of the elapsed time measurement timer t, and the photographing time, that is, the time of execution of the step a3. Based on the position X on the X-axis coordinate, the current position X on the X-axis coordinate of the work 2 is updated and obtained (step a8).
[0034]
Then, the CPU 20 accesses the robot controller 10 via the input / output interface 23 and reads the current position x of the hand 4 from the current position storage register of each axis provided in the axis control circuit 15. _n , Y _n , Z _n Is read (step a9), and the current position x of the hand 4 is determined from the elements of the current position X, Y, and Z of the work 2 _n , Y _n , Z _n , And for the Z-axis component, the offset amount z ₀ Is added to the hand 4 and the current position x of the hand 4 _n , Y _n , Z _n From the current position X, Y, Z of work 2 ₀ The movement amounts Dx, Dy, and Dz required to move the object upward only are obtained (step a10). Then, Dx, Dy, and Dz are transmitted to the robot controller 10 as incremental movement commands (step a11).
[0035]
Further, the CPU 20 controls the distribution pulse amount D per unit time required when feeding the hand 4 in the X-axis direction at a speed corresponding to the transport speed V of the conveyor 3. ₀ The value of x is obtained, and this value is transmitted to the robot controller 10 (step a12).
[0036]
Next, the CPU 20 initializes the value of the number-of-times-of-imaging counter C to zero (step a13), accesses the robot controller 10 via the input / output interface 23, and provides the robot controller 10 with the axis control circuit 15 for distribution processing. The value of the position deviation storage register of each axis is read (step a14), and it is determined whether or not the value is within the range of the in-position width (step a15).
[0037]
At this stage immediately after transmitting the incremental movement command to the robot controller 10 in the process of step a11, the positional deviation of each axis is increasing, and the value does not fall within the range of the in-position width. Steps a14 to a15 are repeatedly executed to wait for the value of the position deviation storage register of each axis to fall within the in-position width range.
[0038]
On the other hand, the incremental movement commands Dx, Dy, Dz transmitted to the robot control device 10 in the process of step a11 are detected by the CPU 13 of the robot control device 10 in the process of step b1. The movement commands Dx, Dy, Dz are stored in the position deviation storage registers E of the respective axes. _r x, E _r y, E _r Set to z (step b2). Further, the distribution pulse amount D transmitted from the numerical controller 9 ₀ After reading the value of x (step b3) and storing this value (step b4), a torque limit is set in the torque limit circuit of the axis control circuit 15 (step b5), and a known pulse distribution process is started (step b5). Step b6).
[0039]
In the pulse distribution processing of step b6, the position deviation storage register E of each axis _r x, E _r y, E _r The incremental movement commands Dx, Dy, Dz stored in z are converted into minute movement commands ΔDx, ΔDy, ΔDz per unit time at a ratio of Dx: Dy: Dz to the servo motors of each axis as movement commands for each pulse distribution cycle. The sending process is repeated, and each time the position deviation storage register E for each axis is _r x, E _r y, E _r The distributed pulse amounts ΔDx, ΔDy, ΔDz (feedback pulses from the pulse coder) are subtracted from z.
[0040]
Each time the pulse distribution processing for one cycle is completed in step b6, the CPU 13 functioning as a part of the tracking operation control means stores the position deviation storage register E for the X axis. _r distribution pulse D to x ₀ The value of x is added (step b7).
[0041]
Next, the CPU 13 determines whether or not the correction commands dx, dy, dz, Dα, Dβ, and Dγ from the numerical controller 9 have been input (step b8), and determines whether the picking command or the place command from the numerical controller 9 has been input. It is determined whether or not an input has been made (step b10). At this stage, since no command has been input, the CPU 13 functioning as a part of the tracking operation control means executes step b6 to step b8 and step b8. Only the processing of b10 is repeatedly executed.
[0042]
First, at the stage immediately after the start of the repetitive processing of step b6 and step b7, the position deviation storage register E of each axis is set. _r x, E _r y, E _r Since the value of z is substantially Dx, Dy, Dz and the positional deviation is sufficiently large, the hand 4 is driven and controlled with a strong force within a range not exceeding the torque limit, that is, at a high speed, and the work 2 moving with the conveyor 3 Follow that upward z ₀ Is moved to the position by linear interpolation. At this time, the work 2 moves in the X-axis direction due to the conveyance of the conveyor 3, but the processing of step b7 causes the X-axis position deviation storage register E _r The value of x is D for each distribution cycle ₀ x, so that even if the position deviation of Dx, Dy, Dz is eliminated, the position deviation storage register E _r x is D ₀ x stays constantly, and this position deviation D ₀ By the operation of the axis control circuit 15 for eliminating x, the hand 4 is sent at the moving speed V in the X-axis direction in the same manner as the work 2 on the conveyor 3.
[0043]
Thus, the position deviation storage register E _r x, E _r y, E _r The position deviation of z is almost eliminated, and the hand 4 ₀ And the position deviation storage register E _r Position deviation D staying at x ₀ When the hand 4 moves in the X-axis direction while maintaining a fixed positional relationship with the work 2 only by x, the value of the position deviation storage register of each axis falls within the range of the in-position width, and the numerical controller The CPU 20 on the ninth side detects this in the processing of step a15.
[0044]
Then, the CPU 20 functioning as a shift amount calculating means operates the second stereo camera 7 which moves while maintaining a fixed positional relationship with the work 2, takes an image of the work 2 and takes in images from the cameras 7 a, 7 b, 7 c. (Step a16) These image data are stored in the frame memory 24 (Step a17).
[0045]
Next, the CPU 20 functioning as a shift amount calculating means starts an application program used for analyzing the image, and for example, sets the image of the work 2 as the first search area near the center of the image stored in the frame memory 24. A search is performed to determine the amount of deviation (Δx, Δy, Δz) of the position of the work 2 in each axis direction and the position of each axis based on the posture (α, β, γ) of the work 2 obtained in the process of step a7. The amount of deviation (Δα, Δβ, Δγ) of the posture of the work 2 is determined (step a18).
[0046]
In this embodiment, by the pulse distribution process shown in FIG. 8, the position z (X, Y, Z) of the work 2 obtained in the process of step a7 is used as a reference, and ₀ Is basically controlled so as to move the hand 4 to the position of the second stereo camera 7 on the assumption that the offset amount of the second stereo camera 7 with respect to the hand 4 is subtracted. (0, 0, 0) coincides with the position (X, Y, Z) of the work 2 obtained in the process of step a7. However, the value of X here is a variable value in the transport process, and substantially, X = X _{(Initial value of step a7)} + V · t. In other words, the coordinate system of the second stereo camera 7 is a coordinate system that translates in the X-axis direction together with the hand 4 and the conveyor 3, and assuming that the offset amount of the stereo camera 7 with respect to the hand 4 is deducted, As long as the work 2 does not move relative to the conveyor 3 due to collapse or the like, the work 2 is always captured at the center of the image of the second stereo camera 7. Therefore, after the photographing by the first stereo camera 6 is completed, the values of (Δx, Δy, Δz) and (Δα, Δβ, Δγ) are basically zero as long as the position of the work 2 with respect to the conveyor 3 does not change. It is.
[0047]
On the other hand, when the work 2 moves relative to the conveyor 3 or the posture is lost due to collapse of the load or the like in the middle, the deviation amount is determined by the processing of step a18 to (Δx, Δy, Δz) or (Δα). , Δβ, Δγ). In this case, after storing the values of (Δx, Δy, Δz) and (Δα, Δβ, Δγ) as the next search area of the work 2 (step a19), the CPU 20 increments the value of the number-of-photographing-times counter C by one. Then, it is determined whether or not the number of repetitions C of photographing has reached the set number N (step a20).
[0048]
If the value of the number-of-photographs accumulation counter C has not reached the set number N, the CPU 20 functioning as a deviation amount calculating means repeatedly executes the processes of step a16 to step a21 in the same manner as described above. 2 and the deviation amounts (Δα, Δβ, Δγ) of the postures around the respective axes are obtained, and these values are used as a new search area of the work 2. Update and remember.
[0049]
During this time, the work 2 may move relative to the conveyor 3 or the posture may collapse due to collapse of the load or the like. However, the amount of deviation of the position and posture during one photographing cycle is not large, and Since the image of the work 2 is searched in the vicinity of the work 2 in the next processing cycle based on the displacement amount of the position and posture of the work 2 stored in the processing cycle (see step a19) (see step a18), The deviation of the actual position and actual posture of the work 2 can be accurately tracked without causing an error due to an increase in the data search load due to the discontinuous change in the position and posture of the work 2.
[0050]
In addition, since the position and orientation of the work 2 are highly accurately recognized by the first stereo camera 6, the work 2 can be accurately placed in the field of view at the time of the first shooting by the second stereo camera 7, and Since the search area of the image data can be narrowed down, it is useful for shortening the overall processing speed.
[0051]
Although the parallax of the second stereo camera 7 is relatively small as compared with the parallax of the first stereo camera 6, it is possible to take an image sufficiently close to the work 2, so that the work 2 can be photographed with sufficient accuracy. Can be detected.
[0052]
Then, when it is detected by the determination processing in step a21 that the value of the number-of-photographing-times integration counter C has reached the set number N, the CPU 20 functioning as the position / orientation correcting means determines the displacement amount of the position finally obtained, that is, , The deviation amounts Δx, Δy, Δz of the final position of the work 2 in the coordinate system which is translated in the X-axis direction together with the hand 4 and the conveyor 3 are stored as incremental movement commands dx, dy, dz for position deviation correction. , The upward offset z for the correction amount in the Z-axis direction. ₀ Correction value -z for eliminating ₀ Is further added, and the deviation amounts Δα, Δβ, and Δγ of the posture finally obtained are added to α, β, and γ obtained in the process of step a7, and the absolute postures α, β, and γ of the workpiece 2 are added. (Step a22).
[0053]
Next, the CPU 20 functioning as the position / posture correction means accesses the robot controller 10 via the input / output interface 23, and reads the current posture α of the hand 4 from the current posture storage register of each axis provided in the axis control circuit 15. _n , Β _n , Γ _n Is read (step a23), and the current attitude α of the hand 4 is determined from the elements of the axes α, β, and γ required for the hand 4 to grip the workpiece 2. _n , Β _n , Γ _n The value of the element of each axis is reduced, and the hand 4 is moved to the current posture α. _n , Β _n , Γ _n , The posture change amounts Dα, Dβ, and Dγ required to change the postures to the postures α, β, and γ suitable for gripping the work 2 are obtained (step a24), and the incremental movement commands dx and dy for positional deviation correction are obtained. , Dz and incremental posture change commands Dα, Dβ, Dγ for posture control of the hand 4 to the robot controller 10 (step a25).
[0054]
Next, the CPU 20 accesses the robot controller 10 via the input / output interface 23 and reads the values of the position deviation storage register and the posture deviation storage register of each axis for distribution processing provided in the axis control circuit 15 ( In step a26), it is determined whether or not the value falls within the range of the in-position width (step a27). Immediately after transmitting the incremental movement command and the incremental posture change command to the robot controller 10 in the process of step a25, the position deviation and the posture deviation of each axis are increasing, and the values fall within the range of the in-position width. Therefore, the CPU 20 repeatedly executes the processing from step a26 to step a27 and waits until the values of the position deviation storage register and the attitude deviation storage register of each axis fall within the in-position width range. .
[0055]
On the other hand, the incremental movement commands dx, dy, dz and the incremental posture change commands Dα, Dβ, Dγ transmitted to the robot controller 10 in the process of step a25 are detected by the CPU 13 of the robot controller 10 in the process of step b8. Upon detecting this, the CPU 13 stores the incremental movement commands dx, dy, dz in the position deviation storage registers E of the respective axes. _r x, E _r y, E _r z, and an attitude deviation storage register E for each axis. _r α, E _r β, E _r The incremental posture change commands Dα, Dβ, and Dγ are set to γ (step b9).
[0056]
Then, the position deviation storage register E of each axis updated in this manner is used. _r x, E _r y, E _r z and attitude deviation storage register E for each axis _r α, E _r β, E _r Based on the value of γ, the CPU 13 functioning as the picking operation control means repeatedly executes only the processing of steps b6 to b8 and b10 in the same manner as described above.
[0057]
As a result, in the pulse distribution process of step b6, the process of applying the speed V to the hand 4 in the X-axis direction is superimposed, and the dx, dy, dz is performed in each of the X, Y, and Z axis directions. And the process of changing (rotating) the attitude of the hand 4 by Dα, Dβ, and Dγ around the X, Y, and Z axes is performed in parallel. .
[0058]
Then, the position deviation storage register E _r x, E _r y, E _r The position deviation of z is almost eliminated, the hand 4 reaches a position where the work 2 can be gripped, and the posture deviation storage register E _r α, E _r β, E _r When the posture deviation of γ is eliminated and the hand 4 reaches a posture where the front and back of the work 2 can be gripped from the normal direction, the hand 4 moves to the position deviation D described above. ₀ Due to the action of x, it moves only in the X-axis direction together with the work 2 at the speed V while maintaining this state.
[0059]
On the other hand, the CPU 20 of the numerical controller 9 determines the position deviation storage register E by the determination processing of step a27. _r x, E _r y, E _r z and attitude deviation storage register E _r α, E _r β, E _r It detects that the deviation of γ has entered the in-position width, and sends a picking command and a place command to the robot controller 10 (step a28).
[0060]
Then, the CPU 13 that has detected these commands in the processing of step b10 closes the hand 4 to grip the work 2 (step b11), clears the error registers of each axis (step b12), and sets The hand 4 is moved to the present work transfer position, the work 2 is placed (step b13), a completion signal is transmitted to the numerical controller 9 (step b14), and a standby state waiting for an original position return command from the numerical controller 9 is performed. (Step b15).
[0061]
After detecting the completion signal in the process of step a29, the CPU 20 outputs a current position return command to the robot controller 10 (step a30), and then waits for a fixed position detection signal from the fixed position detection sensor 25 in an initial standby state. (Step a1), and upon detecting the home position return instruction, the CPU 13 returns the hand 4 to the approach point of the initial setting position (step b16), and thereafter, the incremental movement instructions Dx and Dy from the numerical controller 9. , Dz is returned to the initial standby state (step b1).
[0062]
As described above, as one embodiment, the case where the hand 2 is gripped while the hand 4 is synchronously moved with respect to the work 2 conveyed by the conveyor 3 has been described. 3 is stopped, the picking device in which the placement position of the work 2 is substantially constant is changed by changing various set values. The processing of the CPU 20 of the numerical control device 9 shown in FIG. 7 and the processing of the CPU 13 of the robot control device 10 shown in FIGS. 8 to 9 can be applied as they are.
[0063]
Specifically, the variable V in step a8 and the variable D in step a12 ₀ The value of x may be set to zero, and the value of N serving as a criterion may be set to 1 in step a21.
[0064]
In this case, the shift amount calculating means moves the hand 4 to the vicinity of the work position photographed by the first stereo camera 6, activates the second stereo camera 7 only once, and sets the initial position and initial posture of the work 2. (N = 1), and the tracking operation control means is practically inactive (V = D). ₀ x = 0).
[0065]
In addition, when the hand 4 is synchronously moved with respect to the workpiece 2 in the embodiment described first, the second stereo camera 7 is moved in various directions to change the photographing conditions, so that the position and the posture of the whole are changed. It is also possible to improve the detection accuracy of the displacement. As a method of changing the position and the attitude of the second stereo camera 7, a method of adjusting the feeding operation of the hand 4 and a method of changing the position and the position of the second stereo camera 7 between the hand 4 and the second stereo camera 7 are described. A method of mounting an actuator for changing the posture can be considered.
[0066]
Regarding the former, basically, an appropriate movement command (offset command) for forcibly moving the hand 4 during the tracking operation in the pulse distribution processing as shown in step b6 to step 10 in FIG. Can be achieved. Naturally, a change in the position of the hand 4 causes a change in the appearance of the work 2 by the second stereo camera 7, that is, a change in the position and posture. In this case, the deviation of the position and posture itself corrects the position and posture of the hand 4. Since it functions in the same manner as the amount, no problem in drive control occurs. For example, if the hand 4 is moved from the normal position by 5 cm in the + Y direction by the offset command and the image is taken by the second stereo camera 7, as a result, the work 2 moves in the −Y direction from the initial position. Since it is determined that the position is shifted by 5 cm, the value of Δy becomes −5 cm, and as a result, the hand 4 is forcibly moved by 5 cm from the normal position in the + Y direction by the offset command. The position is corrected by 5 cm in the -Y direction from the position thus set, and finally the position is returned to the optimum position for picking.
[0067]
When the position and orientation of the second stereo camera 7 with respect to the hand 4 are changed using the actuator, the hand 4 is moved to the position (X, Y, Z) of the work 2 determined in the process of step a7. Then, by subtracting the offset amount of the second stereo camera 7 with respect to the hand 4, the coordinate origin of the second stereo camera 7 substantially coincides with the coordinate position (X, Y, Z) of the first stereo camera 6. As in the case, it is possible to take measures by subtracting the offset amount of the second stereo camera 7 with respect to the hand 4 according to the operation state of the actuator.
[0068]
【The invention's effect】
The picking device of the present invention configures the first stereo camera by first and fixedly installing the first stereo camera used for detecting the position and orientation of the work independently of the industrial robot. The parallax between the cameras to be taken is large, so that the recognition accuracy of the position and orientation of the workpiece in the initial stage of the detection operation using the first stereo camera is improved, and the second stereo camera is an area to be photographed. It is necessary to calculate the amount of deviation between the actual position and actual posture of the work because it is possible to reliably shoot the work only by narrowing down the image and reducing the amount of image data to be analyzed. Required time is shortened, and high-speed arithmetic processing is realized.
The second stereo camera needs to be mounted near the hand on the industrial robot, and has a compact configuration that shortens the parallax between the cameras.However, since the work can be taken close to the work, it is sufficient. It is possible to detect the deviation of the position and the posture of the work with high accuracy.
[0069]
In addition, the second stereo camera is repeatedly operated while moving the hand while following the moving position of the work to obtain the deviation of the position and posture of the work, so that the position of the work on the conveyor due to collapse of the load or the like. Even if a change occurs in the position or posture, the target position of the picking operation or the posture of the hand can be appropriately corrected in accordance with the amount of deviation of the position or posture, and a reliable picking operation can be performed.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a simplified configuration of a picking device according to an embodiment to which the present invention is applied.
FIG. 2 is a simplified perspective view showing the appearance of a second stereo camera.
FIG. 3 is a functional block diagram showing a main part of a robot control device constituting a part of the computer system.
FIG. 4 is a functional block diagram showing a main part of a numerical control device constituting a part of the computer system.
FIG. 5 is a flowchart showing an outline of picking control executed by a CPU of the numerical controller.
FIG. 6 is a continuation of the flowchart showing an outline of picking control.
FIG. 7 is a continuation of the flowchart showing an outline of picking control.
FIG. 8 is a flowchart schematically showing a pulse distribution process executed by a CPU of the robot control device.
FIG. 9 is a continuation of the flowchart showing an outline of the pulse distribution processing.
FIG. 10 is a conceptual diagram showing an example of a data array in a frame memory.
FIG. 11 is a conceptual diagram showing an example of three-dimensional shape data stored in a nonvolatile memory.
FIG. 12 is a conceptual diagram illustrating an example of an image captured by a first stereo camera.
FIG. 13 is a conceptual diagram showing an example of three-dimensional information obtained by analyzing image data of a work.
FIG. 14 is a conceptual diagram showing an example of collation between three-dimensional information of a work and three-dimensional shape data registered in a nonvolatile memory.
[Explanation of symbols]
1 Picking device
2 Work
3 conveyor
4 hands
5 Industrial robots
6 first stereo camera
6a, 6b, 6c camera
7 Second stereo camera
7a, 7b, 7c camera
8 Computer system
9 Numerical control unit (part of computer system)
10 Robot controller (part of computer system)
11 ROM
12 Non-volatile memory
13 CPU (part of tracking operation control means, picking operation control means)
14 RAM
15 axis control circuit
16 Manual data input device with display
17 Input / output interface
18 ROM
19 Non-volatile memory
20 CPU (part of position and orientation calculation means, deviation amount calculation means, position and orientation correction means, tracking operation control means)
21 RAM
22 Manual data input device with display
23 I / O interface
24 frame memory
25 Fixed position detection sensor

Claims (2)

An articulated industrial robot having a hand for gripping a work, a camera for grasping the position and orientation of the work, and a picking device including a computer system for driving and controlling the industrial robot and camera,
While a first stereo camera having a large parallax is fixedly installed near the mounting position of the work, a second stereo camera having a relatively small parallax is arranged near the hand in the industrial robot. ,
In the computer system, position and orientation calculating means for determining the position and orientation of the work based on an image captured by the first stereo camera,
The hand is moved to the vicinity of the work position obtained by the position / posture calculation means, the second stereo camera is operated to capture an image, and the work corresponding to the position and posture of the work obtained by the position / posture calculation means is obtained. Deviation amount calculating means for calculating the deviation amount of the actual position and the actual posture of
Position and orientation for obtaining the final work position and orientation based on the position and orientation of the work determined by the position and orientation calculation means and the actual position and actual position shift of the work determined by the shift amount calculation means Correction means;
A picking operation control means for controlling a target position of the picking operation and a posture of the hand based on the position and posture of the work obtained by the position and posture correction means. A conveyor for transporting the workpiece, an articulated industrial robot having a hand for gripping the workpiece, a camera for grasping the position and orientation of the workpiece, and a computer system for driving and controlling the industrial robot and camera In a picking device provided with
While a first stereo camera having a large parallax for photographing a workpiece conveyed on the conveyor at a predetermined position is fixedly installed, a second parallax is relatively small near the hand in the industrial robot. Deploy a stereo camera,
In the computer system, position and orientation calculating means for determining the position and orientation of the work at the predetermined position based on an image captured by the first stereo camera,
Tracking operation control means for determining the current position of the work based on the work position and the conveyor speed and elapsed time of the work determined by the position and orientation calculation means, and moving the hand following the current position of the work. ,
During the operation of the tracking operation control means, the second stereo camera is repeatedly operated to capture an image in a coordinate system which is translated together with the hand and the conveyor, and the position and orientation of the workpiece determined by the position and orientation calculation means are determined. A shift amount calculating means for calculating a shift amount between the actual position and the actual posture of the work;
Based on the position and orientation of the work determined by the position and orientation calculation means and the actual position and actual position shift of the work determined by the shift amount calculation means, the final position in the coordinate system that translates with the hand and the conveyor. Position and orientation correction means for determining the position and orientation of a typical work,
A picking operation control means for controlling a target position of the picking operation and a posture of the hand based on the position and posture of the work obtained by the position and posture correction means.

Images