WO2023032400A1 - Automatic transport device, and system - Google Patents

Abstract

This system comprises: an automatic transport device comprising a robot (25) with a stereo camera (31), a transport device (35) for moving the robot (25) to a work position, and a control device (40) for controlling the robot (25); and a machine tool (10) as an industrial machine. The control device (40), when performing a teaching operation, captures an image of a structure of the machine tool (10) with the stereo camera (31), and generates reference data composed of three-dimensional point cloud position data of the structure; and, when actually operating the robot (25), captures an image of the structure with the stereo camera (31), generates current data composed of three-dimensional point cloud position data of the structure, estimates, on the basis of the obtained current data and reference data, a position error amount and a rotation error amount of the stereo camera (31) between a current posture of the robot (25) and a posture during the teaching operation, and corrects the posture of the robot (25) on the basis of the estimated position error amount and rotation error amount.

Description

Automatic transport device and system

The present disclosure includes an industrial machine such as a machine tool that processes a workpiece, a robot that performs work on the industrial machine, a transport device equipped with the robot and capable of moving to a work position set with respect to the industrial machine, and the robot. It relates to a system provided with a control device for controlling the

Conventionally, the system disclosed in JP-A-2017-132002 (Patent Document 1 below) is known as an example of the above system. In this system, an unmanned guided vehicle equipped with a robot moves to a working position set for a machine tool, which is one type of industrial machine. work is performed.

In such a system, a single robot moved by an unmanned guided vehicle can carry out operations such as attaching and detaching workpieces to and from a plurality of machine tools. Since the degree of freedom in the layout of the machine tool is increased compared to the case of arranging them, the layout of the machine tool can be set to a layout that can further improve production efficiency. In addition, compared to conventional systems in which robots are fixed, one robot can work on a larger number of machine tools, so equipment costs can be reduced.

On the other hand, since the unmanned guided vehicle is self-propelled using wheels, it cannot be said that the positioning accuracy of stopping at the above working position is necessarily high. For this reason, in order for the robot to perform accurate work on the machine tool, the robot attitude when the automatic guided vehicle is positioned at the working position and the robot attitude set at the time of teaching, which is the reference for control, are required. It is necessary to compare the posture with a reference posture, detect the amount of error, and correct the working posture of the robot according to the amount of error.

As a technique for correcting the posture of such a robot, a position correction method disclosed in Japanese Patent Application Laid-Open No. 2016-221622 (Patent Document 2 below) is conventionally known. Specifically, in this position correction method, a visual target consisting of two calibration markers is arranged on the outer surface of the machine tool, and the visual target is imaged by a camera provided on the movable part of the robot. Based on the captured image and the position and orientation of the camera, the relative positional relationship between the robot and the machine tool is measured, and the working posture of the robot is corrected based on the measured positional relationship. .

Japanese Patent Application Laid-Open No. 2017-132002 JP 2016-221622 A

However, in the conventional position correction method described above, in order to perform correction, it is necessary to prepare two special calibration markers, which are special components, and to perform troublesome preparatory work of arranging them on the outer surface of the machine tool. Therefore, there is a problem that a system for correcting the posture of the robot cannot be easily constructed.

Furthermore, in the position correction method disclosed in Patent Document 2, the two calibration markers are captured by cameras, respectively. There is a problem that the production efficiency in the system is low.

Also, as the camera, a so-called stereo camera having two cameras is known, and it would be convenient if the posture of the robot could be corrected using such a known stereo camera.

Therefore, the present invention provides an automatic transport device and system described in the claims.

According to the present invention, an industrial machine in which a robot actually works is configured, or a structure provided in this industrial machine is imaged by a stereo camera, and the posture of the robot is corrected based on the obtained image. Therefore, there is no need to prepare a special component such as a calibration marker for correcting the posture of the robot as in the conventional art, and the troublesome preparation of arranging this on the outer surface of the machine tool is eliminated. no need to do any work. Therefore, it is possible to easily construct a system for correcting the posture of the robot. In addition, in the present invention, it is possible to correct the posture of the robot by taking an image of a single structure, so that the operation time of the robot for taking an image can be shortened compared to the conventional art. , the production efficiency in the system can be increased compared to the conventional system.

1 is a plan view showing a schematic configuration of a system according to one embodiment of the present invention; FIG. 1 is a block diagram showing the configuration of a system according to this embodiment; FIG. It is the perspective view which showed the automatic guided vehicle and robot which concern on this embodiment. 1 is a perspective view showing a spindle and a chuck, which are structures constituting the machine tool according to the present embodiment; FIG. FIG. 4 is an explanatory diagram for explaining an imaging posture of a robot; FIG. 11 is an explanatory diagram for explaining processing for calculating a correction amount for correcting the posture of the robot;

Specific embodiments of the present invention will be described below with reference to the drawings.

As shown in FIGS. 1 and 2, this system 1 includes a machine tool 10 as an industrial machine, a material stocker 20 and a product stocker 21, an automatic guided vehicle 35 as a transport device, and a robot mounted on the automatic guided vehicle 35. 25, and a control device 40 that controls the robot 25 and the automatic guided vehicle 35. Among them, the automatic guided vehicle 35, the robot 25, and the controller 40 constitute an automatic guided vehicle. Also, the robot 25 is provided with a stereo camera 31 . In addition, in this example, the machine tool 10 and the material stocker 20 and the product stocker 21 provided around the machine tool 10 are exemplified as the industrial machine, but the industrial machine to which the present invention is applicable is not limited to this. , cleaning devices, cleaning devices, robots other than the robot 25 in this example, and all machines used industrially. Further, in this example, a known horizontal NC (numerical control) lathe is exemplified as the machine tool 10, but the machine tool is, of course, not limited to this. In addition to machining centers, all conventionally known machine tools such as multi-tasking machines equipped with a tool spindle and a work spindle are included.

As described above, the machine tool 10 of this example is a conventionally known horizontal NC (numerical control) lathe, and as shown in FIG. The workpiece W can be turned by using an appropriate tool. 4 shows only the main spindle 11 and the chuck 12 as components (structures) of the machine tool 10, but it goes without saying that the machine tool 10 is a carriage that is disposed within the machining area. , In addition to components such as a tool post and tools, other components known in the field such as a door arranged outside the processing area, a handle attached to the door, a control panel and an indicator light be able to.

The material stocker 20 is arranged on the left side of the machine tool 10 in FIG. The product stocker 21 is arranged on the right side of the machine tool 10 in FIG. .

As shown in FIG. 1, the automatic guided vehicle 35 has the robot 25 mounted on its upper surface, that is, a mounting surface 36, and is provided with an operation panel 37 that can be carried by an operator. The operation panel 37 includes an input/output unit for inputting/outputting data, an operation unit for manually operating the automatic guided vehicle 35 and the robot 25, and a display capable of displaying a screen.

Further, the automatic guided vehicle 35 is equipped with a sensor capable of recognizing its own position in the factory (for example, a distance measurement sensor using laser light), and under the control of the control device 40, the machine tool 10 , the machine tool 10, the material stocker 20 and the product stocker 21. through each working position.

As shown in FIGS. 1 and 3, the robot 25 of this example is an articulated robot having three arms, a first arm 26, a second arm 27 and a third arm 28. A hand 29 as an end effector (action part) is attached to the tip of the camera, and two cameras 31a and 31b are attached via a support bar 30. As shown in FIG. These two cameras 31a and 31b are so-called stereo cameras 31 arranged so that their imaging optical axes intersect.

It should be noted that the aspect of the robot applicable to the present invention is not limited to the aspect of the robot 25 of this example, and the robot includes (i) a camera and (ii) a hand portion for gripping a workpiece or a tool. , (iii) a second arm portion movably connected to the hand portion, and (iv) a first arm portion movably connected to the second arm portion. When compared with the robot 25 of this embodiment, the hand portion corresponds to the hand 29, the second arm portion corresponds to the joint portion rotatably (movably) coupled to the second arm 27, and the first arm portion corresponds to a joint portion rotatably (movably) coupled to the first arm 26 . In addition, the third arm 28 of the robot of the present embodiment and the joint portion that is rotatably or advanceably retractable (movable) may be understood to correspond to the second arm portion. That is, although the robot 25 of this example has three arms, the robot may have at least two arms.

As shown in FIG. 2, the control device 40 includes an operation program storage unit 41, a movement position storage unit 42, an operation attitude storage unit 43, a map information storage unit 44, a reference data storage unit 45, a manual operation control unit 46, an automatic It is composed of an operation control unit 47 , a map information generation unit 48 , a position recognition unit 49 , a point cloud data calculation unit 50 , a correction amount calculation unit 51 and an input/output interface 52 . The control device 40 is connected to the machine tool 10 , material stocker 20 , product stocker 21 , robot 25 , stereo camera 31 , automatic guided vehicle 35 and operation panel 37 via this input/output interface 52 . Note that the control device 40 is not limited to this aspect. The control device 40 may have at least a control section for controlling the position of the hand 30 of the robot 25, and the other storage section and the like may be provided by another device.

The control device 40 of this example is composed of a computer including a CPU, a RAM, a ROM, etc., and includes the manual operation control unit 46, the automatic operation control unit 47, the map information generation unit 48, the position recognition unit 49, and the point cloud data calculation unit. 50, the correction amount calculation unit 51, and the input/output interface 52 are realized by a computer program, and execute processing described later. The motion program storage unit 41, movement position storage unit 42, motion posture storage unit 43, map information storage unit 44, and reference data storage unit 45 are configured from appropriate storage media such as RAM. In this example, the control device 40 is attached to the automatic guided vehicle 35 and is connected to the machine tool 10, the material stocker 20 and the product stocker 21 by suitable communication means, and the robot 25, the stereo camera 31, the automatic guided vehicle 35 and the operator. It is connected to the board 37 by wire or wirelessly. However, the control device 40 is not limited to such a mode, and the control device 40 may be arranged at an appropriate position other than the automatic guided vehicle 35 . In this case, the control device 40 is appropriately connected to each section by communication means.

The manual operation control unit 46 is a functional unit that operates the unmanned guided vehicle 35, the robot 25, and the stereo camera 31 according to operation signals input from the operation panel 37 by the operator. That is, the operator can manually operate the automatic guided vehicle 35 , the robot 25 and the stereo camera 31 using the operation panel 37 under the control of the manual operation control section 46 .

The operation program storage unit 41 operates the automatic guided vehicle 35 when generating an automatic driving program for automatically driving the automatic guided vehicle 35 and the robot 25 during production, and map information in the factory, which will be described later. This is a functional unit that stores a map generation program for The automatic driving program and the map generating program are input from, for example, an input/output unit provided on the operation panel 37 and stored in the operation program storage unit 41 .

The automatic operation program includes command codes relating to the movement position as a target position to which the automatic guided vehicle 35 moves, the movement speed, and the orientation of the automatic guided vehicle 35. and a command code for operating the stereo camera 31 are included. The map generation program includes command codes for causing the automatic guided vehicle 35 to travel all over the factory without a track so that the map information generation unit 48 can generate map information.

The map information storage unit 44 is a functional unit that stores map information including arrangement information of machines, devices, equipment, etc. (devices, etc.) arranged in the factory where the automatic guided vehicle 35 travels. It is generated by the map information generator 48 .

The map information generation unit 48 causes the automatic guided vehicle 35 to travel according to the map generation program stored in the operation program storage unit 41 under the control of the automatic driving control unit 47 of the control device 40, which will be described later in detail. At that time, the space information in the factory is acquired from the distance data detected by the sensor, and the planar shape of the equipment etc. arranged in the factory is recognized, for example, the planar shape of the equipment etc. registered in advance Based on this, the positions, plane shapes, etc. (layout information) of the specific devices arranged in the factory, in this example, the machine tool 10, the material stocker 20 and the product stocker 21 are recognized. Then, the map information generating unit 48 stores the obtained spatial information and arrangement information of the devices in the map information storage unit 44 as map information of the factory.

The position recognition unit 49 is a function unit that recognizes the position of the automatic guided vehicle 35 in the factory based on the distance data detected by the sensor and the map information in the factory stored in the map information storage unit 44. Based on the position of the automatic guided vehicle 35 recognized by the position recognition section 49 , the operation of the automatic guided vehicle 35 is controlled by the automatic operation control section 47 .

The movement position storage unit 42 is a movement position as a specific target position to which the automatic guided vehicle 35 moves, and is a functional unit that stores a specific movement position corresponding to the command code in the operation program. These movement positions include the work positions set for the machine tool 10, the material stocker 20, and the product stocker 21 described above. This movement position is determined by, for example, manually operating the automatic guided vehicle 35 using the operation panel 37 under the control of the manual operation control unit 46 to move it to each target position, and then performing the position recognition. It is set by the operation of storing the position data recognized by the unit 49 in the movement position storage unit 42 . This operation is called a so-called teaching operation.

The motion posture storage unit 43 stores data relating to motion postures (motion postures) of the robot 25 that sequentially change as the robot 25 moves in a predetermined order, corresponding to command codes in the motion program. is a functional unit that stores the The data relating to the motion posture is obtained when the robot 25 is manually operated by teaching operation using the operation panel 37 under the control of the manual operation control unit 46 to take each target posture. , the rotation angle data of each joint (motor) of the robot 25 in each posture, and this rotation angle data is stored in the motion posture storage unit 43 as data relating to the motion posture.

Specific motion postures of the robot 25 are set in the material stocker 20, the machine tool 10 and the product stocker 21 respectively. For example, in the material stocker 20, the work start posture (take-out start posture) when work is started in the material stocker 20, the unprocessed work W stored in the material stocker 20 is gripped by the hand 29, and the material stocker Each work posture (each take-out posture) for taking out from 20 and the posture when taking out is completed (take-out completion posture, which is the same posture as the take-out start posture in this example) are set as take-out motion postures.

Also, in the machine tool 10, a work take-out motion posture for taking out the machined work W' from the machine tool 10 and a work mounting motion posture for attaching the pre-machined work W to the machine tool 10 are set.

Specifically, in the work take-out operation posture, for example, the work start posture before entering the machine tool 10, the hand 29 and the stereo camera 31 are moved into the machining area of the machine tool 10, and the stereo camera 31 chucks the workpiece. 12 (imaging posture) (see FIG. 5), the posture in which the hand 29 is opposed to the processed work W′ gripped by the chuck 12 (retrieval preparation posture), and the hand 29 is moved to the chuck 12 side. and a posture (gripping posture) in which the hand 29 grips the machined work W′ held by the chuck 12, and a posture in which the hand 29 is separated from the chuck 12 and the machined work W′ is removed from the chuck 12 (grasping posture). Outer posture), and a posture in which the hand 29 and the camera 31 are removed from the machine tool 10 (work completion posture) are set.

In addition, in the work mounting posture, for example, the work start posture before entering the machine tool 10, the hand 29 and the stereo camera 31 are moved into the machining area of the machine tool 10, and the stereo camera 31 images the chuck 12. posture (imaging posture) (see FIG. 5), posture in which the unprocessed work W gripped by the hand 29 is opposed to the chuck 12 of the machine tool 10 (mounting preparation posture), and the hand 29 is moved to the chuck 12 side. The chuck 12 can grip the workpiece W before machining (mounting posture), the hand 29 is separated from the chuck 12 (separation posture), and the hand 29 and the camera 31 are removed from the machine tool 10. Each posture of the posture (work completion posture) is set. Although FIG. 5 illustrates a state in which the stereo camera 31 faces the chuck 12, the imaging posture is not limited to this. It can be a posture of being. A two-dot chain line shown in FIG. 5 is the field of view of the stereo camera 31 .

In the product stocker 21, the work start posture (storage start posture) when work is started in the product stocker 21, each work posture for storing the processed work W′ gripped by the hand 29 in the product stocker 21 ( storage posture) and the posture when the storage is completed (storage completion posture, which is the same posture as the storage start posture in this example) are set as the storage operation posture.

The automatic operation control unit 47 uses either the automatic operation program or the map generation program stored in the operation program storage unit 41, and operates the automatic guided vehicle 35, the robot 25, and the camera 31 according to the program. Department. At that time, the data stored in the movement position storage section 42 and the motion posture storage section 43 are used as necessary.

Based on the two images captured by the cameras 31a and 31b, the point cloud data calculation unit 50 calculates the focal length of each camera 31a and 31b, the distance between the cameras 31a and 31b, and the distance between the two cameras 31a and 31b. From the parallax, the position in the three-dimensional space of the camera coordinate system set for the stereo camera 31 is calculated for each element obtained by dividing the object to be imaged into a predetermined size. Each of the above elements is recognized as a point group forming an object to be imaged, and the position data thereof is the position data of the point group in the three-dimensional space of the camera coordinate system (three-dimensional point group position data).

In this example, the point cloud data calculation unit 50 detects the stereo image when the automatic guided vehicle 35 is at the work position set with respect to the machine tool 10 and the robot 25 takes the above-described imaging posture. Based on two images of the chuck 12 captured by the camera 31, three-dimensional point cloud position data of the chuck 12 in the camera coordinate system is calculated from these images.

The reference data storage unit 45 captures images with the stereo camera 31 when the unmanned guided vehicle 35 is at the working position set with respect to the machine tool 10 and the robot 25 is in the imaging posture during teaching operation. It is a functional unit that stores the three-dimensional point cloud position data of the chuck 12 calculated by the point cloud data calculation unit 50 based on the image of the chuck 12 obtained as reference data.

In addition to the reference data, the reference data storage unit 45 also stores three-dimensional point group position data obtained from CAD data of the chuck 12 , which is an object coordinate system set for the chuck 12 . 3D point cloud position data in the coordinate system is stored. It should be noted that this object coordinate system is defined by, for example, two orthogonal x-axis and y-axis set in a plane orthogonal to the axis of the chuck 12, and three orthogonal axes of the z-axis orthogonal to these x-axis and y-axis. can be defined.

When the robot 25 is automatically operated in accordance with the automatic operation program stored in the operation program storage unit 41 under the control of the automatic operation control unit 47, the correction amount calculation unit 51 determines whether the robot 25 is When the chuck 12 is imaged by the stereo camera 31 in the imaging posture and the current three-dimensional point cloud position data of the chuck 12 is calculated by the point cloud data calculation unit 50, this current three-dimensional point cloud position data is obtained. and the reference data (three-dimensional point cloud position data of the chuck 12 calculated during the teaching operation) stored in the reference data storage unit 45, the current posture of the robot 25 and the posture during the teaching operation. The amount of positional error and the amount of rotational error of the stereo camera 31 in between are estimated, and based on each estimated amount of error, the amount of correction for the action portion in the working posture is calculated.

Specifically, the correction amount calculation unit 51 calculates the following based on the current three-dimensional point cloud position data of the chuck 12 obtained during automatic operation and the reference data stored in the reference data storage unit 45. By executing processing, the amount of positional error of the stereo camera 31 between the current posture of the robot 25 and the posture during the teaching operation, which is set within a plane orthogonal to the axis of the chuck 12, is determined. Estimate the positional error amount of the stereo camera 31 in the x-axis and y-axis and the z-axis direction orthogonal to these x-axis and y-axis, and the rotation error amount of the stereo camera 31 around the x-axis, y-axis and z-axis. At the same time, each working posture of the robot 25 is corrected based on the estimated position error amount and rotation error amount.

(Preprocessing)
First, as pre-processing, the correction amount calculation unit 51 sets the chuck 12 from the camera coordinate system corresponding to the stereo camera 31 based on the reference data obtained during the teaching operation. coordinate transformation matrix for transforming to the object coordinate system, which is the coordinate system

to get Note that the camera coordinate system can be defined, for example, by orthogonal three axes that are set centering on an intermediate position between the imaging elements in a plane that includes the imaging elements (for example, CMOS sensors) of the cameras 31a and 31b. .

Specifically, this coordinate transformation matrix

can be calculated by the following procedure. That is, first, the three-dimensional point cloud position data in the object coordinate system of the chuck 12 obtained from the CAD data stored in the reference data storage unit 45, the internal parameters of the stereo camera 31 (for example, the focal length of the camera), and Coordinate transformation matrix from camera coordinate system to object coordinate system related to CAD data based on external parameters (distance between cameras 31a and 31b, parallax, etc.)

to get

Next, the correction amount calculation unit 51 stores the three-dimensional point cloud position data of the chuck 12 in the object coordinate system in the object coordinate system of the chuck 12 obtained from the CAD data stored in the reference data storage unit 45 as well. Using the reference data, which is the three-dimensional point cloud position data of the chuck 12 in the camera coordinate system during the teaching operation, for example, these are applied to the RANSAC algorithm (global alignment) and the ICP algorithm (local alignment) Accordingly, by superimposing the three-dimensional point cloud position data related to the CAD data on the reference data, the coordinate transformation matrix from the object coordinate system related to the CAD data to the object coordinate system related to the chuck 12 at the time of teaching operation

to get As conceptually shown in FIG. 6, the figure indicated by the dashed line is the figure related to the three-dimensional point group position data in the object coordinate system of the chuck 12 obtained from the CAD data, and the figure indicated by the solid line is obtained during the teaching operation. , the above coordinate transformation matrix

is obtained.

Next, the coordinate transformation matrix obtained as above

, and coordinate transformation matrix

, according to Equation 1 below, the coordinate transformation matrix

Calculate
(Formula 1)

As described above, the coordinate transformation matrix

After calculating the correction amount calculation unit 51, the calculated coordinate transformation matrix

and a camera position in the camera coordinate system, which is a camera position at the time of imaging in the teaching operation

and based on the camera position during the teaching operation in the object coordinate system

is calculated by Equation 2 below.
(Formula 2)

(Camera position calculation processing during teaching operation)
Next, the correction amount calculation unit 51 calculates the camera position in the robot coordinate system during the teaching operation.

is calculated by Equation 3 below. The robot coordinate system is a three-dimensional coordinate system set for the control device 40 to control the robot 25, and is defined by three orthogonal axes with origins set at appropriate positions.
(Formula 3)

Subsequently, the correction amount calculation unit 51 calculates the coordinate transformation matrix for converting from the camera coordinate system during the teaching operation to the robot coordinate system during the teaching operation.

is calculated by Equation 4 below.
(Formula 4)

here,

Rotation matrix elements of

Rotation angles around x, y and z axes based on

,

,

Calculate

still,

is a coordinate transformation matrix for transforming from the object coordinate system to the robot coordinate system at the time of teaching operation. For example, the coordinate transformation matrix for converting from the object coordinate system to the camera coordinate system during teaching operation

and a coordinate transformation matrix for transforming from the camera coordinate system to the robot coordinate system at the time of teaching operation

can be obtained by Equation 5 below.
(Formula 5)

(Camera position calculation processing during automatic driving)
Next, the correction amount calculation unit 51 calculates the above based on the current three-dimensional point cloud position data in the camera coordinate system of the chuck 12 calculated by the point cloud data calculation unit 50 during automatic operation (during actual operation). A coordinate transformation matrix for transforming from the camera coordinate system to the object coordinate system in the same way as

is calculated according to Equation 6 below.
(Formula 6)

Note that the coordinate transformation matrix from the camera coordinate system to the object coordinate system at the time of teaching operation

is the three-dimensional point cloud position data in the object coordinate system of the chuck 12 at the time of the teaching operation, the internal parameters of the stereo camera 31 (for example, the focal length of the camera, etc.), which are stored in the reference data storage unit 45, as described above ) and external parameters (distance between cameras 31a and 31b, parallax, etc.).

In addition, the coordinate transformation matrix from the object coordinate system during teaching operation to the object coordinate system during automatic operation

are three-dimensional point cloud position data of the chuck 12 in the camera coordinate system during the teaching operation stored in the reference data storage unit 45, and calculated by the point cloud data calculation unit 50 during automatic operation. Using the 3D point cloud position data of the chuck 12, for example, according to the RANSAC algorithm (global alignment) and ICP algorithm (local alignment), the 3D point cloud position data during teaching operation is automatically operated. It can be calculated by performing a process of superimposing it on the three-dimensional point cloud position data of time. Referring to FIG. 6, the figure indicated by the dashed line is the figure related to the three-dimensional point group position data of the chuck 12 in the object coordinate system during teaching operation, and the figure indicated by the solid line is the object coordinate system during automatic operation. , the coordinate transformation matrix

is obtained.

As described above, the coordinate transformation matrix

After calculating the correction amount calculation unit 51, the calculated coordinate transformation matrix

and the camera position in the camera coordinate system, which is the camera position at the time of imaging in automatic driving

and based on the camera position during automatic driving in the object coordinate system

is calculated by Equation 7 below.
(Formula 7)

Next, the correction amount calculation unit 51 calculates the current camera position in the object coordinate system during automatic driving.

is calculated by the following formula 8, and the current camera position in the robot coordinate system during automatic operation

is calculated by Equation 9 below.
(Formula 8)

(Formula 9)

Subsequently, the correction amount calculation unit 51 calculates the coordinate transformation matrix for transforming the current camera coordinate system to the robot coordinate system at the time of the teaching operation.

is calculated by Equation 10 below.
(Formula 10)

here,

Rotation matrix elements of

Rotation angles around x, y and z axes based on

,

,

Calculate

(Error amount calculation process)
Next, the correction amount calculation unit 51 calculates the camera angle during the teaching operation in the coordinate system during the teaching operation calculated as described above.

,

,

and the current camera angle in the coordinate system during teaching operation

,

,

Rotational errors Δrx, Δry, and Δrz about the x-axis, y-axis, and z-axis are calculated by calculating these differences based on and.
however,

Next, based on the rotation errors Δrx, Δry, and Δrz calculated as described above, the correction amount calculation unit 51 calculates the rotation matrix between the robot coordinate system at the time of the teaching operation and the current robot coordinate system.

That is, the rotation error amount is calculated by the following formula 11, and the translation matrix from the robot coordinate system at the time of the teaching operation to the current robot coordinate system

That is, the position error amount is calculated by Equation 12 below.
(Formula 11)

(Formula 12)

(Correction amount calculation process)
Next, based on the error amount calculated as described above, the correction amount calculation unit 51 calculates a correction amount for correcting this error amount.

is calculated by Equation 13 below.
(Formula 13)

Then, the automatic driving control unit 47 determines the position of the hand 29 in the subsequent motion posture of the robot 25 based on the correction amount calculated by the correction amount calculation unit 51 .

is corrected according to Equation 14 below.
(Formula 14)

According to the system 1 of this example having the above configuration, unmanned automatic production is executed as follows.

That is, under the control of the automatic operation control unit 47 of the control device 40, the automatic operation program stored in the operation program storage unit 41 is executed, and according to this automatic operation program, for example, the automatic guided vehicle 35 and Robot 25 operates as follows.

First, the unmanned guided vehicle 35 moves to the work position set with respect to the machine tool 10, and the robot 25 assumes the work start posture for the above-described work take-out operation. At this time, it is assumed that the machine tool 10 has completed the predetermined machining and the door cover is open so that the robot 25 can enter the machining area.

Next, the robot 25 shifts to the imaging posture and images the chuck 12 with the stereo camera 31 . When the chuck 12 is imaged by the stereo camera 31 in this manner, the point cloud data calculation unit 50 calculates three-dimensional point cloud position data of the chuck 12, and the correction amount calculation unit 51 calculates , based on the three-dimensional point cloud position data and the reference data stored in the reference data storage unit 45, the imaging posture of the robot 25 at the time of the teaching operation and the current imaging posture are calculated according to Equations 11 and 12 above. Based on each estimated error amount, the correction amount for the subsequent work take-out motion posture of the robot 25 is calculated according to Equation 13 described above.

Based on the correction amounts calculated by the correction amount calculation unit 51, the automatic operation control unit 47 determines the subsequent work take-out operation postures, that is, the above-described take-out preparation posture, gripping posture, removal posture, and work completion posture. The position of the hand 29 at is corrected according to Equation 14, and the machined workpiece W' gripped by the chuck 12 of the machine tool 10 is gripped by the hand 29 and taken out from the machine tool 10. The chuck 12 is opened by transmitting a chuck opening command from the automatic operation control unit 47 to the machine tool 10 after causing the robot 25 to take the gripping posture.

Next, the automatic operation control unit 47 moves the unmanned guided vehicle 35 to the work position set with respect to the product stocker 21, and instructs the robot 25 to perform the storage start posture when starting work in the product stocker 21. , each storage attitude for storing the machined workpiece gripped by the hand 29 in the product stocker 21 and the storage completion attitude when the storage is completed. Store in

Next, the automatic operation control unit 47 moves the unmanned guided vehicle 35 to the work position set with respect to the material stocker 20, and instructs the robot 25 to take out a take-out start posture when starting work in the material stocker 20, The unprocessed work stored in the material stocker 20 is grasped by the hand 29, and the hand 29 is caused to sequentially take each take-out posture for taking out the work from the material stocker 20 and the take-out completion posture when the take-out is completed, and the work is processed by the hand 29. Grip the front workpiece.

Next, the automatic operation control unit 47 again moves the unmanned guided vehicle 35 to the work position set with respect to the machine tool 10, and causes the robot 25 to assume the work start posture for the work mounting operation described above. Next, the robot 25 is shifted to the imaging posture, and the chuck 12 is imaged by the stereo camera 31 . When the chuck 12 is imaged by the stereo camera 31 in this manner, the point cloud data calculation unit 50 calculates three-dimensional point cloud position data of the chuck 12, and the correction amount calculation unit 51 calculates , based on the three-dimensional point cloud position data and the reference data stored in the reference data storage unit 45, the imaging posture of the robot 25 at the time of the teaching operation and the current imaging posture are calculated according to Equations 11 and 12 above. Based on each estimated error amount, the correction amount for the subsequent work take-out motion posture of the robot 25 is calculated according to Equation 13 described above.

After that, the automatic operation control unit 47 determines the subsequent work mounting posture of the robot 25, that is, the above-described mounting preparation posture, mounting posture, separation posture, and After correcting the position of the hand 29 in the work completion posture according to Equation 14, the robot 25 attaches the unprocessed workpiece W gripped by the hand 29 to the chuck 12 of the machine tool 10, and then moves out of the machine. Let After that, the automatic operation control unit 47 transmits a machining start command to the machine tool 10 to cause the machine tool 10 to perform the machining operation. After the robot 25 takes the mounting posture, the automatic operation control unit 47 transmits a chuck close command to the machine tool 10, whereby the chuck 12 is closed and the workpiece W to be machined is gripped by the chuck 12. be done.

By repeating the above, system 1 of this example continuously executes unmanned automatic production.

Thus, in the system 1 of this example, the stereo camera 31 images the chuck 12, which is a structure that constitutes the machine tool 10 on which the robot 25 actually works. Since the posture is corrected, there is no need to prepare a special component such as a calibration marker in order to correct the posture of the robot 25 as in the conventional art, and this can be installed outside the machine tool 10. There is no need to perform troublesome preparatory work for arranging on the surface. Therefore, a system for correcting the posture of the robot 25 can be easily constructed. In addition, in the system 1 of the present example, it is possible to correct the posture of the robot 25 by capturing an image of the chuck 12, which is a single structure. This can be done by motion, and as a result, the operating time of the robot 25 for imaging can be shortened compared to the conventional art, and as a result, the production efficiency in the system 1 can be increased compared to the conventional art. can.

Among the operations performed by the robot 25 of this example, the operation on the chuck 12 requires the most accurate operation. Since the working posture of the robot 25 is corrected based on the image obtained by the above operation, the working posture can be corrected accurately. However, the work can be performed accurately.

Since the unmanned guided vehicle 35 is configured to move by the motion of the wheels with a relatively high degree of freedom, the mounting surface on which the robot 25 is mounted is likely to tilt with respect to the floor surface, and the robot to be mounted can be tilted with respect to the floor surface. 25, in other words, according to a change in the position of the center of gravity of the robot 25, the inclination is likely to change. Therefore, when the workpieces W and W′ described above are attached and detached, the robot 25 causes the hand 29 and the stereo camera 31 to enter the processing area of the machine tool 10 so that the hand 29 is far away from the automatic guided vehicle 35. When the hand 29 is out of the working area of the machine tool 10, the hand 29 does not overhang the automatic guided vehicle 35, or the overhang is small even if it does. is larger than the slope of

Therefore, as in the conventional position correction technique described above, visual targets, which are calibration markers, are provided on the outer surface of machine tool 10, and robot 25 is positioned outside the machining area of machine tool 10. In the aspect of acquiring the posture correction amount, the posture correction amount does not accurately reflect the posture error amount of the robot 25 when the hand 29 of the robot 25 is in the machining area of the machine tool 10. The posture of the robot 25 cannot be corrected accurately.

If the posture of the robot 25 when attaching and detaching the works W and W' cannot be corrected accurately, the hand 29 of the robot 25 cannot be accurately positioned with respect to the chuck 12. For example, the chuck 12 is a collet chuck or the like, and when the movement allowance (stroke) of the gripping part is extremely small, that is, when the clearance between the workpieces W, W' and the collet chuck is extremely small, the workpiece W is moved to the collet chuck. , W′ cannot be securely gripped. If the attachment and detachment of the workpieces W and W' cannot be reliably performed, the operation rate of the system 1 will decrease, and unmanned production with good production efficiency cannot be realized.

In the system 1 of the present example, as described above, an image obtained by imaging the chuck 12 for which accurate work is required by the stereo camera 31, in other words, is similar to the posture when accurate work is required. Since the working posture of the robot 25 is corrected based on the image obtained when the posture is taken, the working posture can be corrected accurately, thereby allowing the robot 25 to operate with high accuracy. can be performed with high accuracy even for work that requires In addition, since the robot 25 performs work with high precision in this way, the system 1 operates at a high operating rate without causing unnecessary interruptions, and as a result, the system 1 is highly reliable. , it is possible to achieve unmanned production with high production efficiency.

Although one embodiment of the present invention has been described above, specific aspects that the present invention can take are not limited to this.

For example, in the above example, as the most preferable mode, the chuck 12, which requires the most accurate work in the work of the robot 25, is set as a structure to be imaged by the stereo camera 31, but it is limited to such a mode. Instead, other structures that make up the machine tool 10, such as a tool post, a tool, a carriage, etc. that are also arranged in the machining area, may be imaged according to the part where the robot 25 works. . Alternatively, when such high correction accuracy is not required, for example, a door disposed outside the processing area, a handle attached to the door, a structure such as an operation panel or an indicator light, and the like may be imaged. This aspect also has the effect that a system for correcting the posture of the robot 25 can be easily constructed without preparing special components or performing troublesome preparatory work. be done. In addition, since it is possible to correct the posture of the robot 25 by capturing an image of a single structure, it is possible to perform this in a single operation when capturing an image of the structure with the stereo camera 31. As a result, the operation time of the robot 25 for imaging can be shortened as compared with the conventional art, and as a result, the production efficiency in the system 1 can be improved as compared with the conventional art. When the imaging target is a structure provided outside the machining area, the imaging attitude is the attitude before the hand 29 and the stereo camera 31 enter the machining area of the machine tool 10, that is, the hand When the 29 and the stereo camera 31 are outside the machining area of the machine tool 10, they are in a posture for imaging the imaging target.

In addition, in the above-described embodiment, in the operation of the robot 25 with respect to the machine tool 10, a mode of correcting the working posture is adopted. , the working posture of the robot 25 may be corrected in the same manner as described above. Further, as described above, the industrial machine according to the present invention includes all machines used industrially, such as a measuring device, a cleaning device, a washing device, and robots other than the robot 25 of this example. When the operator 25 works on this industrial machine, it is possible to correct its working posture in the manner described above.

Further, in the above-described embodiment, an aspect using the automatic guided vehicle 35 was exemplified, but the present invention is not limited to this. It may be a configured transport device. Then, a robot 25 is mounted on the transport device, and the transport device is manually transported to the working position of the machine tool 10, and the robot 25 attaches and detaches the work to and from the machine tool 10. Also good.

Again, the above description of the embodiment is illustrative in all respects and is not restrictive. Modifications and modifications are possible for those skilled in the art. The scope of the invention is indicated by the claims rather than the above-described embodiments. Furthermore, the scope of the present invention includes modifications from the embodiments within the scope of claims and equivalents.

1 system 10 machine tool 11 spindle 12 chuck 20 material stocker 21 product stocker 25 robot 29 hand 31 stereo camera 35 unmanned guided vehicle 37 operation panel 40 control device 41 operation program storage unit 42 movement position storage unit 43 operation posture storage unit 44 map information Storage unit 45 Reference data storage unit 46 Manual operation control unit 47 Automatic operation control unit 48 Map information generation unit 49 Position recognition unit 50 Point cloud data calculation unit 51 Correction amount calculation unit 52 Input/output interface W Pre-machining work W' Processed work

Claims (5)

1. A stereo camera having two cameras that capture images, and a robot that has an action unit that acts on an object and performs work on an industrial machine;
   a transport device equipped with the robot and configured to be movable to a work position set with respect to the industrial machine;
   Controlling the robot according to an operation program including preset operation commands, and imaging the robot with a work start posture, a structure constituting the industrial machine, or a structure provided in the industrial machine with the stereo camera. a posture, a control device configured to sequentially take one or more working postures for causing the action portion to act on the object, and to control the conveying device;
   The work start posture, the imaging posture and the work posture are set in advance by teaching the robot, wherein
   The control device is
   During the teaching operation, the structure is imaged by the stereo camera while the robot is shifted to the imaging posture, and based on the obtained image, the structure in the camera coordinate system set for the stereo camera is determined. Execute a process of generating reference data consisting of three-dimensional point cloud position data of the structure,
   When actually operating the robot according to the operation program, the robot is moved from the work start posture to the imaging posture in a state in which the transport device has moved to the work position, and then the stereo camera performs the An image of a structure is captured, and based on the obtained image, processing is performed to generate current data consisting of three-dimensional point cloud position data of the structure in the camera coordinate system, and the obtained current data and the reference are executed. Based on the data, the amount of error in the position of the stereo camera between the current posture of the robot and the posture at the time of the teaching operation, which is a predetermined first axis and a second axis direction orthogonal to each other, and the Estimate each position error amount of the stereo camera in a third axis direction perpendicular to the first axis and the second axis, and each rotation error amount of the stereo camera about the first axis, the second axis and the third axis. and correcting the position of the working portion in the working posture based on each estimated positional error amount and each rotational error amount.
2. industrial machinery;
   a stereo camera having two cameras that capture images, and a robot that has an action unit that acts on an object and performs work on the industrial machine;
   a transport device equipped with the robot and configured to be movable to a work position set with respect to the industrial machine;
   According to an operation program including preset operation commands, the robot is given a work start posture, an imaging posture for imaging a structure constituting the industrial machine or provided on the industrial machine with the stereo camera, and the object. A control device configured to sequentially take one or more working postures for causing the action part to act on,
   A system in which the work start posture, the imaging posture, and the work posture are set in advance by teaching the robot,
   The control device is
   During the teaching operation, the structure is imaged by the stereo camera while the robot is shifted to the imaging posture, and based on the obtained image, the structure in the camera coordinate system set for the stereo camera is determined. Execute a process of generating reference data consisting of three-dimensional point cloud position data of the structure,
   When actually operating the robot according to the operation program, the robot is moved from the work start posture to the imaging posture in a state in which the transport device has moved to the work position, and then the stereo camera performs the An image of a structure is captured, and based on the obtained image, processing is performed to generate current data consisting of three-dimensional point cloud position data of the structure in the camera coordinate system, and the obtained current data and the reference are executed. Based on the data, the amount of error in the position of the stereo camera between the current posture of the robot and the posture at the time of the teaching operation, which is a predetermined first axis and a second axis direction orthogonal to each other, and the Estimate each position error amount of the stereo camera in a third axis direction perpendicular to the first axis and the second axis, and each rotation error amount of the stereo camera about the first axis, the second axis and the third axis. and correcting the position of the working portion in the working posture based on each estimated positional error amount and each rotational error amount.
3. The control device is
   A camera position at the time of imaging by the teaching operation in a robot coordinate system set for the robot based on the reference data, in the robot coordinate system at the time of the teaching operation
   

   

   a process of calculating
   Based on the current data obtained during the actual movement, the camera position at the time of imaging by the actual movement in the robot coordinate system during the teaching operation
   

   

   a process of calculating
   The position of the camera in the robot coordinate system during the teaching operation
   

   

   , and the camera position
   

   

   based on the position error amount of the stereo camera in the first, second and third axis directions in the robot coordinate system during the actual operation, and the positional error amount around the first, second and third axes of the stereo camera a process of estimating the amount of rotation error;
   3. The system according to claim 2, further configured to correct the position of said working portion in said working posture based on said estimated position error amount and rotation error amount.
4. The control device is
   The position of the camera in the robot coordinate system during the teaching operation
   

   

   , and the camera position
   

   

   After calculating
   Camera angle during teaching operation in the coordinate system during teaching operation
   

   

   ,
   

   

   ,
   

   

   and the current camera angle in the coordinate system during teaching operation
   

   

   ,
   

   

   ,
   

   

   Rotational errors around the x-, y-, and z-axes, where the first axis is the x-axis, the second axis is the y-axis, and the third axis is the z-axis, are calculated by calculating these differences based on The amounts Δrx, Δry, and Δrz are calculated, and based on the calculated rotation error amounts Δrx, Δry, and Δrz, the robot coordinates during the teaching operation are calculated as the rotation error amount of the stereo camera in the robot coordinate system during the actual operation. Rotation matrix between system and robot coordinate system during operation
   

   

   a process of calculating
   A translation matrix from the robot coordinate system during the teaching operation to the robot coordinate system during the actual operation as the positional error amount of the stereo camera in the robot coordinate system during the actual operation
   

   

   a process of calculating
   the rotation matrix
   

   

   and parallel matrix
   

   

   based on the correction amount
   

   

   a process of calculating
   Correction amount
   

   

   the corrected position of the working part in the working posture based on
   

   

   4. The system of claim 3, wherein the system is configured to:
5. The controller controls the rotation matrix
   

   

   , a parallel matrix
   

   

   ,Correction amount
   

   

   and correction position
   

   

   5. The system according to claim 4, which is configured to calculate by the following formula.
   

   

   

   

   

   however,
   

   

   is the position of the action part during the teaching operation in the robot coordinate system during the teaching operation.
   

Images