JP2012125871A - Robot control setting support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for supporting robot motion teaching and image processing therefor in consideration of constraints on robot motion, without requiring a CAD model or work content model for a target workpiece.SOLUTION: The robot control setting support device performs interactive processing with a teaching operator. A two-dimensional image and three-dimensional data processing unit performs image processing for determining a target position and attitude of a robot corresponding to the position and attitude of a target workpiece, according to an input, on a displayed two-dimensional image and three-dimensional data related to the target workpiece. A constraint calculation and evaluation processing unit performs calculation and evaluation of constraints on motion of the robot operated according to the target position and attitude of the robot as a result of the image processing, and if no approval input is received, performs the image processing for determining the target position and attitude according to a new input and performs calculation and evaluation of robot motion constraints according to a new target position and attitude till an approval input is received. A robot motion monitoring unit records a robot motion procedure under approved conditions. A processing procedure analysis and reconfiguration unit displays a series of recorded processing contents, performs integration and elimination of redundant processing according to an input and then rerecords the processing contents.

Description

  The present invention relates to a robot control setting support apparatus for supporting robot operation teaching and quick start-up of an image processing procedure in a robot system including a three-dimensional measurement apparatus.

  Conventional teaching of a three-dimensional work path to a robot and teaching of a processing procedure of three-dimensional image processing require specialized knowledge and many work procedures. For example, in the following Patent Document 1, a setting operation of an image processing procedure is simplified by performing a simulation to determine a measurement condition for an object. Further, in the following Patent Document 2, in order to simplify the work teaching of the robot and the setting process of the image processing processing procedure, a work template and a recognition model prepared in advance are called and modified and used. Yes.

Japanese Patent No. 4266946 JP 2009-214212 A

However, all of the conventional techniques require that a CAD model and a work content template must be prepared in advance, and cannot be used for a recognition object or work content that should be newly handled. There was a problem in the rapid start-up of the image processing procedure.
Further, when operating with an actual machine, various restrictions on gripping the target workpiece with the hand of the robot hand are not taken into consideration, and there has been a problem that desired performance cannot be sufficiently realized when the work is realized. For example, problems such as collision avoidance on the route to the assembly or alignment position, operation time to achieve the objective, maintenance of stable gripping during transportation are not considered, and performance at the time of work implementation is not sufficient was there. For this reason, since it takes time to adjust the teaching for realizing the desired robot operation, there is a problem that it is difficult to quickly start up.

  The present invention has been made to solve the above-described problems, and does not require a CAD model of a target work or a template of work contents in advance, and further considers restrictions on robot operation when actually operating. It is an object of the present invention to obtain a robot control setting support device that quickly performs robot operation teaching and image processing at that time.

  The present invention is a robot control setting support device that includes a display device and an input device that support robot operation teaching to an arm type robot having a hand that holds a target workpiece, and that performs interactive processing with a teaching worker, A three-dimensional measuring apparatus that measures a two-dimensional image and three-dimensional data indicating a distance at each two-dimensional point, and a two-dimensional image and a three-dimensional data that measure a two-dimensional image of a measurement region including a target workpiece by the three-dimensional measuring apparatus The three-dimensional image / three-dimensional data measuring means, and the measured two-dimensional image and three-dimensional data are displayed on the display device, and the robot corresponding to the position and orientation of the target workpiece is displayed according to the input to the input device of the teaching worker for the display. Performs image processing to determine the target position and orientation, displays the image processing result including the target position and orientation of the robot, and receives the consent input from the input device. A two-dimensional image / three-dimensional data processing unit including a two-dimensional image / three-dimensional data processing means to be stored in an image processing content data storage unit, and an operation of the robot operated based on the target position / posture of the robot as a result of the image processing The result of calculation and evaluation of the constraint condition is displayed on the display device. When the consent input is received from the input device and the consent input cannot be received, the above 2 is performed according to the new input to the teaching worker's input device. The image processing for determining the target position and orientation is performed by the three-dimensional image / three-dimensional data processing means, and the result of calculating and evaluating the constraint condition regarding the robot operation based on the target position and orientation of the new robot is displayed on the display device. The constraint condition calculation / evaluation processing unit that displays and repeats the process until it receives an acceptance input from the input device, and is operated under the conditions approved by the constraint condition calculation / evaluation processing unit. The robot operation monitor unit that inputs the robot execution procedure data from the robot as robot operation procedure data and records it in the robot operation procedure data storage unit, and the series of images recorded in the image processing content data storage unit and the robot operation procedure data storage unit The processing contents are displayed on the display device in accordance with the instruction input to the input device by the teaching worker, the redundant processing is integrated and the erasable processing is deleted in accordance with the input instruction from the teaching worker, the processing procedure is reconfigured and re-executed. A robot control setting support device comprising: a processing procedure analysis / reconfiguration processing unit that stores configured processing content as a reconstruction processing record in each storage unit after confirmation input to the input device is there.

  In the present invention, a CAD model of a target work and a template of work contents are not required in advance, and robot operation teaching and image processing at that time are quickly performed in consideration of restrictions on the operation of the robot in actual operation. A robot control setting support device can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram of the robot control setting assistance apparatus by Embodiment 1 of this invention containing a robot. It is a figure which shows an example of a structure of the periphery of a robot. It is an operation | movement flowchart which concerns on the robot control setting assistance apparatus by Embodiment 1 of this invention. It is a figure for demonstrating operation | movement of the 2-dimensional image and 3-dimensional data processing means of FIG. It is a figure which shows the more detailed structure of the constraint condition calculation and evaluation process part of FIG. It is a block diagram of the robot control setting assistance apparatus by Embodiment 2 of this invention containing a robot.

  Hereinafter, a robot control setting support apparatus according to the present invention will be described with reference to the drawings according to each embodiment. In each embodiment, the same or corresponding parts are denoted by the same reference numerals, and redundant description is omitted.

Embodiment 1 FIG.
1 is a diagram showing a configuration of a robot control setting support apparatus according to Embodiment 1 of the present invention including a robot. FIG. 2 is a diagram showing an example of the configuration around the robot. 1 and 2, an information processing apparatus 130 configured by a computer, which is a main component of the robot control setting support apparatus, is connected to the arm type robot 104 and performs control setting support. The arm type robot 104 has, for example, a hand 103 that grips the target workpiece 201, and is connected to a robot operation panel 105 that performs operation input to the robot.

  The information processing apparatus 130 includes a two-dimensional image / three-dimensional data processing unit 131 including a two-dimensional image / three-dimensional data measurement unit 131a and a two-dimensional image / three-dimensional data processing unit 131b, which are represented by functional blocks, and a constraint condition calculation. An evaluation processing unit 132, a robot operation monitoring unit 133, and a processing procedure analysis / reconfiguration processing unit 134 are included. Further, the information processing device 130 moves to a three-dimensional measurement device 101 that measures two-dimensional images and three-dimensional data that is distance data at each two-dimensional point, a display device 106 that displays a series of processing results, and an information processing device 130. An input device 107 for performing an operation input, a three-dimensional robot simulator 108 for checking necessary constraint conditions related to movement of the robot, which will be described later, and calculating an evaluation value, and the like, a measurement data storage unit 121, an image processing content data storage unit 122, A storage unit for storing each data of the robot operation procedure data storage unit 123 and the constraint condition calculation / evaluation processing data storage unit 124 is connected.

  FIG. 3 shows an example of an operation flowchart according to the robot control setting support apparatus of FIG. The following operations are performed interactively between the robot control setting support device and the teaching worker. First, the two-dimensional image / three-dimensional data measuring unit 131a uses the three-dimensional measuring apparatus 101 to measure the two-dimensional image of the measurement region 202 (see FIG. 2) including the target work 201 and the above-described two-dimensional points. The three-dimensional data as distance data is measured (S101), and the two-dimensional image and the three-dimensional data are recorded in the measurement data storage unit 121 (S102).

  Next, the two-dimensional image / three-dimensional data processing means 131b reads the two-dimensional image and the three-dimensional data and displays them on the display device 106 such as a computer display (S103). When the teaching operator instructs / inputs a measurement instruction and a processing area necessary for determining the position / posture of the target workpiece and the target position / posture of the robot with respect to the displayed data from the input device 107 (S104), Image processing corresponding to the measurement instruction and the processing area is performed (S105). Image processing performed at this time includes the following. In the three-dimensional data measurement processing, basic calculation processing for three-dimensional data such as average distance value, nearest / farst distance value calculation, and average normal direction calculation can be selected and executed. Through these calculation processes, it is possible to calculate the three-dimensional target position and posture and obtain the target position and posture for the robot. In the two-dimensional image measurement processing, the region center of gravity position, the center line of the parallel contour portion, region division on the same plane, template matching (a two-dimensional image pattern of the target workpiece is stored in any storage unit in advance), etc. Basic image processing for the two-dimensional image is selected and executed. Through these image processes, it is possible to calculate the three-dimensional target position and orientation in the target work surface and obtain the target position and orientation to the robot.

  Next, the two-dimensional image / three-dimensional data processing means 131b displays an image processing result including the target position and orientation of the robot with respect to the position and orientation of the target work on the display device 106, and inputs (consent) confirmation (acceptance) of the teaching worker for this. The processing result acceptance is confirmed (S106), and the image processing content is temporarily recorded in the image processing content data storage unit 122 (S107).

  Then, in order to move the robot to the target position and orientation displayed on the display device 106, the teaching worker inputs a robot operation using the robot operation panel 105 (S108). As the operation instruction to the robot operation panel 105, the robot target position / posture value described above is used.

  Prior to this operation, the constraint condition calculation / evaluation processing unit 132 uses the three-dimensional robot simulator 108 to check necessary constraint conditions regarding the movement of the robot and to calculate an evaluation value. For example, the calculation and evaluation of constraints such as the presence or absence of interference with the surroundings when performing robot operations can be performed by approximating the object to be interfered in the work area as a set of many cylinders and spheres. This can be done based on the distance from the central axis of each axis. Furthermore, it is easy to obtain the path operation time calculation of items related to the operation time evaluation such as the length of the path through the three-dimensional robot simulator 108. Furthermore, the calculation and evaluation of the gripping state of the target workpiece of the robot hand can be obtained from the measured target workpiece position and orientation and the relative position of the hand, and the center of gravity position estimated from the three-dimensional shape and the target gripping center of the hand An evaluation index such as increasing the evaluation value can be used as the relative distance is shorter. These constraint conditions and evaluation results are displayed on the display device 106, for example, and confirmed (accepted) by the teaching operator for confirmation (S110). If there is a problem here, the process returns to step S104, the calculation process of the target position and orientation of the robot is executed again, and the process is repeated until an appropriate result is obtained.

  Next, the robot operation content based on the robot operation input in step S108 when confirmed (accepted) in step S110 is confirmed by the teaching operator on the robot operation panel 105 (S111), and the robot operation is executed (S112). . The robot operation monitor unit 133 inputs the robot execution content from the robot as robot operation procedure data and records it in the robot operation procedure data storage unit 123 (S113).

  Next, the processing procedure analysis / reconstruction processing unit 134 converts a series of processing contents recorded in the measurement data storage unit 121, the image processing content data storage unit 122, and the robot operation procedure data storage unit 123 into similar robot operation paths. The result of automatically analyzing the work content that is considered redundant through the degree comparison is displayed on the display device 106, and in accordance with an instruction input by the teaching worker, the redundant processes are integrated, the deleteable processes are deleted, and the processing procedure is performed. Reconfiguration is performed (S114). At this time, image processing contents related to the robot operation are deleted as necessary. Alternatively, the determination of the redundancy processing may be performed by the teaching worker himself and an instruction for integration or deletion may be input. The reconstructed processing content is input to the input device 107 by a teaching worker after confirmation (acceptance) input (S115), and the reconstruction processing is recorded in each storage unit (S116).

  Note that the operations in steps S101 to S116 are for one operation of the robot. When the above processing is performed for a plurality of operations, the processing from step S101 to step S116 is repeated for the number of times. Further, for example, the processing from step S114 to step S116 regarding the processing procedure analysis / reconfiguration processing unit 134 may be performed in batch after the processing from step S101 to step S113 is performed a plurality of times.

  The above information processing is performed by the information processing apparatus 130. Note that the display device 106 and the input device 107 may be configured by a dedicated robot operation terminal provided with a display with a touch panel. As a result, the teaching time can be shortened by performing a more intuitive operation quickly.

  By combining the above-described three-dimensional measurement data processing and the two-dimensional image processing result related to the two-dimensional image / three-dimensional data processing means 131b, for example, scale correction based on distance values, extraction of a planar region, calculation of a representative normal in the region The target position / posture value of the robot is determined.

  FIG. 4 shows an example of processing that combines the three-dimensional measurement data processing and the two-dimensional image processing results. First, FIG. 4 is roughly divided into four figure areas (a) to (d). In each figure area, a two-dimensional gray image and a corresponding distance image are displayed in a small frame. . Here, the distance image is obtained by converting the difference in distance into a shaded display, and a close portion is brightly displayed in the drawing. FIG. 4 shows a state in which the process proceeds from (a) to (d) in the upper left, upper right, lower left, and lower right, where (a) is an initial measurement state. In FIG. 5B, the processing area instruction is indicated by a thick frame, but the interface configuration is such that the area shape can be instructed and input with a mouse or the like as the input device 107 on the screen as the display device 106. It is an example. Of course, it is also possible to input instructions more intuitively from a teaching device provided with a display with a touch panel or the like. Calculate the average distance and normal direction from the distance value distribution in the instruction area at this time, and generate an operation instruction to the robot so that the visual line direction and instruction distance along the normal direction with respect to the processing area. Robot operation is executed after the teaching operator confirms. As a result of this operation, the robot is positioned in a posture and a target distance facing the target work, and only the horizontal movement and rotation deviation in the plane parallel to the work surface remains. A measurement result image shown in FIG. 4C is obtained by performing measurement again after this operation.

  For this measurement result image, information serving as a reference for calculating the in-plane angle, for example, the position of a small circle around the great circle below the target workpiece is detected using the two-dimensional image processing provided by the system. The in-plane angle is calculated using the direction defined from the relative position of each other and the relative angle of the vertical line on the screen. The result of detecting the contour direction in the longitudinal direction of the target workpiece using the image processing means (two-dimensional image / three-dimensional data processing means 131b) provided in addition to this and calculating the relative angle with the vertical line on the screen Etc. can be used.

  The interface is configured so that the image processing content, processing area, definition of the robot operation using the processing result, and the like necessary for these processes can be input. If the processing result is insufficient, it is possible to return to the previous step and retry the processing. When the processing result is confirmed, it is determined whether or not to perform the robot operation and the robot is operated. As a result of this operation, the amount of displacement of the horizontal movement and rotation in the plane parallel to the target work surface is corrected, and the final target position / posture before the gripping operation is obtained. FIG. 4 (d) shows the result of the object being observed in a posture along the top and bottom of the screen. After that, if the robot grasps the robot to be gripped from the measurement result and confirms that the target workpiece is gripped, the basic teaching work is completed.

  A more detailed configuration of the constraint condition calculation / evaluation processing unit 132 is shown in FIG. In FIG. 5, the constraint condition calculation / evaluation processing unit 132 includes an interference condition calculation / evaluation processing unit 132a, a route operation time calculation / display processing unit 132b, and a gripping state calculation / evaluation processing unit 132c. The constraint condition calculation / evaluation processing unit 132 may include at least one of these means.

  By calculating an evaluation index related to the robot path when performing the above-described series of operations, it is possible to realize robot teaching satisfying desired performance. Various evaluation indices can be set at this time. The interference condition calculation / evaluation processing means 132a checks for the presence or absence of interference. The interference condition calculation / evaluation processing means 132a uses the 3D robot simulator 108 to check the interference between the 3D measurement result in the work area, the robot hand (hand), and the path of the intermediate axis. To check. The presence / absence of interference is determined, and if it is a path where interference occurs, it is displayed on the display device 106 to prompt the teaching worker to input an avoidance point, and the teaching worker performs a check again after setting the avoidance point. To do.

  The 3D measurement result in the work area is obtained by using the 2D image and 3D data measurement area 202 as the work area, and the robot hand (hand) path is obtained as motion data from the robot 104 side.

  The route operation time calculation / display processing unit 132b evaluates the route operation time, calculates a series of accumulated operation times, and the display device 106 indicates the difference between the calculated estimated time and the target time. Shown in From this result, the teaching worker sets a route that can shorten (or extend) the processing time, performs the evaluation again, and shortens (or extends) the processing time.

  The operation time can be calculated using, for example, the three-dimensional robot simulator 108. The target time can be stored in advance as constraint condition calculation / evaluation reference data in the constraint condition calculation / evaluation processing data storage unit 124, or can be input by a teaching worker on the spot.

  The gripping state calculation / evaluation processing unit 132c evaluates a stable gripping index, and the center of gravity position and the gripping position of the target work area estimated from the measurement result of the two-dimensional image / three-dimensional data processing unit 131 of the target work 201. The gripping stability is evaluated based on the distance and the result is presented to the teaching worker on the display device 106, and the teaching worker inputs a gripping point correction instruction.

  For example, a ratio between the center of gravity position and the distance between the gripping position with respect to the length or width of the target workpiece is stored in advance in the constraint condition calculation / evaluation processing data storage unit 124 as a gripping stability evaluation reference value, and the evaluation reference Judgment and evaluation by comparison with values.

  Through the above processing, the processing procedure analysis / reconstruction processing unit 134 performs redundant processing on a series of processing contents such as robot operations recorded in the image processing content data storage unit 122, the robot operation procedure data storage unit 123, and the like. Integration, deletion of deletable processes, etc. are performed and reconfiguration is performed. At this time, interactive integration processing by the teaching worker can also be performed. The reconfigured processing content is finally recorded in each storage unit.

  In the processing procedure analysis / reconstruction processing unit 134, for example, an operation part of the same route in the robot operation procedure data in the robot operation procedure data storage unit 123 is extracted and displayed on the display device 106 for the teaching worker, Examination of deletion is urged, and the robot operation procedure data is changed in accordance with an instruction input to the display device 106 of the teaching worker. Accordingly, the corresponding part of the image processing content data in the image processing content data storage unit 122 is changed or deleted.

As described above, according to the present invention, the basic image processing procedure prepared in advance in the robot control setting support device and the robot operation are sequentially combined and executed by the teaching worker while confirming the contents displayed on the screen. Image processing procedure can be realized relatively easily, and by recording the result in the image processing content data storage unit, it is not necessary to prepare a CAD model or a work content template in advance. The robot can be taught the necessary image processing content.
In addition, the robot operation history at this time is recorded in the robot operation procedure data storage unit, and the robot trajectory from which redundant elements are eliminated by using the processing procedure analysis / reconfiguration processing unit that analyzes and reconfigures the processing procedure Can be realized at the same time.
In addition, by providing a constraint condition calculation / evaluation processing unit that evaluates motion constraint conditions such as presence / absence of interference, motion time, and gripping stability, the desired performance will be met or close to that immediately after robot teaching. Thus, the adjustment time after teaching can be shortened, and quick start-up can be realized.

Embodiment 2. FIG.
FIG. 6 is a diagram showing the configuration of the robot control setting support apparatus according to the first embodiment of the present invention including a robot. In the robot control setting support apparatus of FIG. 6, an external case reference unit 135 is provided in the information processing apparatus 130. The external case reference unit 135 stores one or a plurality of external case examples that store the execution history of the robot control settings of other bases connected to the external network NT by the communication unit 109 that can communicate via the external network NT. Connected to device 140. Thereby, it is possible to update the processing contents in the two-dimensional image / three-dimensional data processing unit 131 to a desired one. Also, in the processing in the constraint condition calculation / evaluation processing unit 132 and the processing procedure analysis / reconstruction processing unit 134, appropriate processing based on more cases and latest cases is performed by referring to external cases. be able to.

  With the above configuration, by providing an external case reference section that refers to the implementation history of multiple locations, it is possible to refer to past cases in a wide range as well as the relevant locations. An appropriate reconfiguration of the processing procedure based on the above is performed. As a result, the teaching time can be further shortened by using data of past cases.

  103 hand, 104 arm type robot, 105 robot operation panel, 106 display device, 107 input device, 108 three-dimensional robot simulator, 109 communication unit, 121 measurement data storage unit, 122 image processing content data storage unit, 123 robot operation procedure data Storage unit 124 Data storage unit for constraint condition calculation / evaluation processing 130 Information processing device 131 Two-dimensional image / three-dimensional data processing unit 131a Two-dimensional image / three-dimensional data measuring unit 131b Two-dimensional image / three-dimensional data Processing means, 132 Restriction condition calculation / evaluation processing section, 132a Interference condition calculation / evaluation processing means, 132b Path movement time calculation / display processing means, 132c Grasping state calculation / evaluation processing means, 133 Robot motion monitoring section, 134 Processing procedure analysis・ Reconfiguration processor, 135 external Reference unit, 140 external case repository, 201 target work, 202 measurement area, NT external network.

Claims (3)

A robot control setting support device that includes a display device and an input device for supporting robot operation teaching to an arm type robot having a hand that holds a target workpiece, and that performs interactive processing with a teaching worker,
A three-dimensional measuring device that measures three-dimensional data indicating a two-dimensional image and a distance at each two-dimensional point;
A two-dimensional image / three-dimensional data measuring means for measuring a two-dimensional image and three-dimensional data of a measurement region including a target workpiece by the three-dimensional measuring device, and displaying the measured two-dimensional image and three-dimensional data on the display device. In accordance with the input to the input device of the teaching worker for the display, the image processing for determining the target position and orientation of the robot corresponding to the position and orientation of the target work is performed, and the image processing result including the target position and orientation of the robot is displayed. A two-dimensional image / three-dimensional data processing unit including a two-dimensional image / three-dimensional data processing unit for storing an image processing result received from the input device in the image processing content data storage unit;
The result of the calculation and evaluation of the constraint condition regarding the operation of the robot operated based on the target position and orientation of the robot as the image processing result is displayed on the display device, and the consent input is received from the input device. In this case, in accordance with a new input to the input device of the teaching worker, the two-dimensional image / three-dimensional data processing means performs image processing for determining the target position and orientation, and sets the target position and orientation of the new robot. A constraint condition calculation / evaluation processing unit that displays the result of the calculation and evaluation of the constraint condition on the operation of the robot based on the display device, and repeats until the consent input is received from the input device;
A robot operation monitoring unit that inputs the robot execution contents operated under the conditions approved by the constraint condition calculation / evaluation processing unit from the robot as robot operation procedure data and records them in the robot operation procedure data storage unit;
A series of processing contents recorded in the image processing content data storage unit and the robot operation procedure data storage unit are displayed on a display device in accordance with an input instruction to the input device by the teaching worker, and redundant processing is performed in accordance with the input instruction by the teaching worker. Processing procedure analysis / removal of deleteable processes, reconfiguration of the processing procedure, and storage of the reconfigured processing content as a reconfiguration processing record in each storage unit after confirmation input to the input device A reconstruction processing unit;
A robot control setting support device comprising: A communication unit capable of communicating via an external network with an external case storage device at an external base that stores the execution history of robot control settings;
An external case reference unit that inputs an implementation history of the external case storage device via the external network by the communication unit;
The robot control setting support device according to claim 1, further comprising:   The robot control setting support device according to claim 1 or 2, wherein the display device and the input device comprise a dedicated robot operation terminal provided with a display with a touch panel.

Images