JP2015112654A - Control apparatus, robot, teaching data generation method, and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device for generating teaching data to execute a stable gripping action in a mode understandable for a user, a robot, a teaching data generation method and a robot system.SOLUTION: A control apparatus 100 comprises: an image recognition part 110 for accepting an imaging picture image having imaged a work-piece and a marker; and a processing part 120 for generating the robot teaching data. The processing part 120 generates first teaching data of the robot by the imaging picture image accepted by the image recognition part 110, and generates second teaching data of the robot by causing a gripping part 319 of the robot to grip the work-piece.

Description

  The present invention relates to a control device, a robot, a teaching data generation method, a robot system, and the like.

  In order to cause a robot to perform a desired operation, a method using teaching by a user is widely known. Specifically, direct teaching that teaches robot operation by a user (teacher) moving a robot arm or the like by hand, teaching playback that teaches robot operation by operating an operation unit such as a teaching pendant, Alternatively, off-line teaching or the like in which teaching is performed using a simulation on a computer without using a real robot is known.

  For example, Patent Document 1 discloses a technique for detecting the position of a workpiece based on the force received by the robot when the gripping portion (hand) of the robot comes into contact with the workpiece and performing robot control using the detected position of the workpiece. It is disclosed.

  Patent Document 2 discloses a method of accepting designation of an elementary shape model corresponding to a workpiece gripping part based on an image and selecting a grasping pattern for appropriately grasping the workpiece based on the designated elementary shape model. Has been.

  Patent Document 3 discloses a method of teaching a gripping position and posture by designating a plurality of gripping points of a workpiece based on an image and performing vector calculation using a vector or the like representing the gripping point.

JP 2009-220248 A JP 2009-214212 A Japanese Patent Laid-Open No. 11-58279

  In Patent Document 1, a hand is controlled based on an image, and a force is detected each time to detect a workpiece position and a subsequent workpiece gripping operation. For this reason, the gripping method changes with each movement, and it is difficult to cause the robot to perform a stable gripping movement.

  In Patent Document 2, a gripping operation is taught using an original shape model without using an actual robot. In Patent Document 3, a vector calculation using a gripping point determined based on an image without using an actual robot. In this way, the gripping operation is taught. Therefore, in any of the methods, it is difficult to cause the robot to perform a stable gripping operation because the workpiece is not gripped by using an actual gripping part during teaching.

  One aspect of the present invention includes an image recognition unit that receives a captured image obtained by capturing a workpiece and a marker, and a processing unit that generates robot teaching data, and the processing unit receives the image recognition unit. The present invention relates to a control device that generates first teaching data of the robot based on a captured image and causes the gripping unit of the robot to grip the workpiece to generate second teaching data of the robot.

  In one aspect of the present invention, the first teaching data is generated using the captured image, and the second teaching data is generated by causing the gripping unit to actually grip the workpiece. Therefore, since offline teaching and direct teaching can be used in an appropriate combination, teaching by the user can be facilitated, robot operation based on teaching can be stabilized, and the like.

  In the aspect of the invention, the processing unit may generate the second teaching data of the robot by causing the gripping unit to grip the workpiece based on the first teaching data.

  Thereby, since the first teaching data can be used for the gripping operation of the workpiece by the gripping unit, it is possible to easily generate the second teaching data.

  In one embodiment of the present invention, the processing unit acquires force sensor information output by a force sensor provided in the robot by causing the grip unit to grip the workpiece, and the force sensor Based on the sensor information, a grip position in a vertical direction with respect to the work table may be determined, and the second teaching data including the determined grip position may be generated.

  Accordingly, it is possible to determine a grip position in the vertical direction of the work table by using the force sensor information.

  In the aspect of the invention, the processing unit may acquire torque limit information of a motor that drives the robot by causing the gripping unit to grip the work, and based on the torque limit information, The opening amount information of the grip portion in a state where the grip portion is held may be determined, and the second teaching data including the determined opening amount information may be generated.

  Thereby, it is possible to determine the opening amount information of the gripping part using the torque limit information.

  In the aspect of the invention, the image recognition unit performs an image recognition process for recognizing the marker, and the processing unit performs the first teaching based on a result of the image recognition process for the marker. Data may be generated.

  Accordingly, it is possible to generate the first teaching data based on the result of the image recognition process for recognizing the marker.

  Moreover, in one aspect of the present invention, the image recognition unit performs the image recognition process for recognizing the plurality of markers arranged around the workpiece that is a gripping target, and the processing unit includes the plurality of the markers. The first teaching data may be generated based on the result of the image recognition process for the.

  As a result, it is possible to use a plurality of markers as objects of image recognition processing.

  In the aspect of the invention, the image recognition unit may be a first marker that is the marker indicating the first direction and the marker that indicates the second direction intersecting the first direction. The image recognition processing for recognizing two markers is performed, and the processing unit determines a temporary gripping position based on a position where the first direction and the second direction intersect, and the first direction In addition, a temporary gripping posture may be determined based on at least one of the second directions, and the first teaching data including the temporary gripping position and the temporary gripping posture may be generated.

  As a result, it is possible to use the first and second markers that define the direction as the targets of the image recognition processing, respectively.

In one embodiment of the present invention, when the workpiece having first to Nth (N is an integer of 2 or more) surfaces is a gripping target, the processing unit is configured to perform the i-th (i is 1) of the workpiece. ≦ satisfy i ≦ N faces integer) as a processing target surface, the by gripping a workpiece to said grip portion, said first i second _i is the second teaching data plane and said processed surface of The teaching data is generated, the j-th surface of the workpiece (j is an integer satisfying 1 ≦ j ≦ N, j ≠ i) is defined as the processing target surface, and the gripping unit grips the workpiece, Second _j teaching data, which is the second teaching data with the j plane as the processing target plane, may be generated.

  Thereby, it is possible to generate a plurality of second teaching data by performing processing on different surfaces of a given workpiece.

In one aspect of the present invention, when the workpiece is arranged in a position and orientation in which the i-th surface of the workpiece is directed vertically upward of a work table, the processing unit is configured to perform the i-th operation of the workpiece. The second _i teaching data may be generated by making the gripping part grip the work with the surface to be processed as the processing target surface.

  Thus, it is possible to generate second teaching data for an appropriate surface according to the position and orientation of the placed workpiece.

  In one aspect of the present invention, the image recognition unit performs an object recognition process for recognizing the position and orientation of the workpiece, and the processing unit performs a final position and orientation of the workpiece acquired by the object recognition process. The second teaching data including an actual gripping position and a final gripping posture may be generated.

  This makes it possible to generate second teaching data based on the result of the object recognition process.

  In the aspect of the invention, the image recognition unit may perform the object recognition process after causing the gripping unit to grip the workpiece based on the first teaching data.

  This makes it possible to perform object recognition processing at a more desirable timing.

  In the aspect of the invention, the processing unit generates the first teaching data including the temporary gripping position and the temporary gripping posture based on the captured image, and determines the temporary gripping position and the temporary gripping posture. As a target, the second teaching data including the final gripping position and the final gripping posture may be generated by causing the gripping unit to grip the workpiece.

  Thus, the final gripping position and gripping posture can be generated as the second teaching data by causing the gripping part to grip the workpiece using the temporary gripping position and the temporary gripping posture.

  Another aspect of the present invention includes a gripping unit that grips a workpiece, and a control device that operates the gripping unit, and the control device captures the first teaching data by imaging the workpiece and the marker. , And the second teaching data is generated by causing the gripper to grip the workpiece.

  In another aspect of the present invention, the first teaching data is generated using the captured image, and the second teaching data is generated by causing the gripping unit to actually grip the workpiece. Therefore, since offline teaching and direct teaching can be used in an appropriate combination, teaching by the user can be facilitated, robot operation based on teaching can be stabilized, and the like.

  According to another aspect of the present invention, a first teaching data of a robot is generated by imaging a workpiece and a marker, and a second teaching of the robot is caused by gripping the workpiece by a gripping portion of the robot. Generating data, and a teaching data generation method.

  In another aspect of the present invention, the first teaching data is generated by imaging the workpiece and the marker, and the second teaching data is generated by causing the gripping unit to actually grip the workpiece. Therefore, since offline teaching and direct teaching can be used in an appropriate combination, teaching by the user can be facilitated, robot operation based on teaching can be stabilized, and the like.

  Another aspect of the present invention includes a robot including a gripping unit that grips a workpiece, a control device that operates the robot, and a marker. The control device images the workpiece and the marker. And a robot system that generates first teaching data of the robot and causes the gripping unit to grip the workpiece to generate second teaching data of the robot.

  In another aspect of the present invention, the first teaching data is generated using the captured image, and the second teaching data is generated by causing the gripping unit to actually grip the workpiece. Therefore, since offline teaching and direct teaching can be used in an appropriate combination, teaching by the user can be facilitated, robot operation based on teaching can be stabilized, and the like.

  As described above, according to some aspects of the present invention, it is possible to provide a control device, a robot, a teaching data generation method, a robot system, and the like that generate teaching data for performing a stable gripping operation in a form that is easy for a user to understand. it can.

The structural example of the control apparatus which concerns on this embodiment. The detailed structural example of the control apparatus which concerns on this embodiment. 2 is a configuration example of a robot according to the present embodiment. The other structural example of the robot which concerns on this embodiment. FIG. 5A and FIG. 5B are explanatory diagrams of the position and orientation of the gripping portion. FIG. 6A and FIG. 6B are explanatory diagrams of a gripping operation targeting a temporary gripping position / posture. FIGS. 7A to 7F are diagrams for explaining the correspondence between the gripping operation and the arrangement of the workpieces on the work table. 8A to 8D show examples of marker arrangement. Explanatory drawing of the image recognition process using a captured image. 10A and 10B show examples of arrangement of force sensors. FIG. 11A to FIG. 11C are other configuration examples of the grip portion and explanatory diagrams of the grip depth. The example of the input screen used by this embodiment. The flowchart explaining the process of this embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. First, the method of this embodiment will be described. Conventionally, in order to cause a robot to perform a desired operation, a technique using teaching by a user is widely known. Specifically, as described above, direct teaching, teaching playback, offline teaching, and the like are known.

  However, in recent years, robots have been used in a very wide area. For this reason, it is assumed that there are many cases where a user who teaches a robot does not have sufficient expertise about the robot. In other words, it is desirable that teaching to the robot be performed in an easy-to-understand form so that even a user with insufficient expertise can execute it. In that respect, teaching playback using a teaching pendant is not preferable in that the teaching user needs to know the robot and to master the usage of the teaching pendant.

  In offline teaching, an easy-to-understand input interface is presented to the user, and the robot operation can be automatically generated on the system side based on information input from the user, thereby facilitating teaching. Various methods are conceivable as to what kind of input is received from the user and how it is processed on the system side. For example, the methods of Patent Document 2 and Patent Document 3 described above are disclosed.

  However, in offline teaching, the robot and the workpiece to be grasped are not actually used. Therefore, the robot operation taught by off-line teaching may not actually realize a stable gripping operation. For example, even if it is easy to reproduce the shape of the robot's gripping part (hand in the narrow sense) and the shape of the workpiece with a 3D model, it is possible to reproduce information on the weight balance of the workpiece, surface friction, etc. It is not realistic considering the processing load. Therefore, even if the gripping operation can be created by off-line teaching, when the gripping operation is actually performed, the workpiece may not be gripped in a balanced manner and may be dropped. In addition, if the user input is simplified from the viewpoint of user-friendliness, or the processing based on user input is simplified from the viewpoint of reducing the processing load on the system side, the necessary information compared to actual robots and workpieces. Is missing. For this reason, there is a great possibility that it is important to consider the possibility that the robot motion obtained as a result of off-line teaching may not be a stable motion in the actual machine.

  Therefore, the present applicant proposes a technique for teaching a robot to grasp a gripping operation in an easy-to-understand form for a user and using an actual robot or workpiece. At that time, the approach in the gripping operation is easy to generate automatically, for example, as shown in FIG. Note that the approach here represents an operation of moving the gripping portion of the robot to the position of the workpiece to be gripped. In the present embodiment, a gripping position and a gripping posture representing the position and posture of a gripping part in a state of gripping a workpiece (which are collectively referred to as a gripping position / posture) are provisionally determined, and the gripping operation is actually performed by the above approach. To do. Then, using the sensing information obtained during the gripping operation, a final gripping position / posture is obtained, and teaching data including the gripping position / posture is generated.

  Specifically, as illustrated in FIG. 1, the control device 100 according to the present embodiment corresponds to an image recognition unit 110 that receives a captured image obtained by capturing a workpiece and a marker, and a robot (a robot main body 300 illustrated in FIG. 3 described later). ) Includes processing unit 120 that generates teaching data. Then, the processing unit 120 generates first teaching data of the robot based on the captured image received by the image recognition unit 110, and causes the gripping unit (corresponding to the hand shown in FIG. 3 described later) of the robot 319 to grip the workpiece. Second teaching data of the robot is generated.

  At this time, the processing unit 120 may generate the second teaching data of the robot by causing the gripping unit 319 to grip the workpiece based on the first teaching data. Specifically, the processing unit 120 generates first teaching data including a temporary gripping position and a temporary gripping posture based on the captured image, and sets the work to the gripping unit 319 with the temporary gripping position and the temporary gripping posture as targets. The second teaching data including the final gripping position and the final gripping posture may be generated.

  Here, the gripping position represents the position of the gripping part in the state of gripping the work, and the gripping posture represents the posture of the gripping part in the state of gripping the work. In addition, since the gripping part is an object having a size and often has a variable structure, there are various ways of defining the position of the gripping part. In the present embodiment, the end point EP of the arm provided with the grip portion may be used as the position of the grip portion. For example, as shown in FIG. 5A, when a gripping portion is provided at the end point EP at the tip of the arm, the position of the gripping portion may be expressed by the position of the end point EP. For example, when an xyz 3-axis coordinate system is set with reference to the robot, the gripping position can be expressed by the coordinate values (x, y, z) of the EP. Further, the posture of the gripping portion may be expressed by rotation with respect to a given reference posture. For example, as shown in FIG. 5B, when the rotation about each axis of xyz is u, v, w, the posture of a given gripping part can be expressed by (u, v, w). The final gripping position and gripping posture (the output result) included in the teaching data is information expressed by (x, y, z, u, v, w).

  Further, the temporary gripping position and the temporary gripping posture are provisional gripping position and postures acquired by the image recognition process. In the present embodiment, with the temporary gripping position and the temporary gripping posture as targets, the gripping operation by the above-described approach that is automatically determined is performed. Since the gripping operation is only for generating final teaching data, the temporary gripping position used for the gripping operation does not need to be detailed.

  For example, as shown in FIG. 6B, a line L1 in the first direction determined by the marker M1 and a line L2 in the second direction determined by the marker M2 are defined on the workbench, and M1 , M2 is recognized by the processing unit 120 through the image recognition process. In this case, the temporary gripping position is, for example, the intersection of L1 and L2 (x0, y0), and the temporary gripping posture is two fingers (claws) of the gripping portion on L2, as shown in FIG. 6B. A lined posture (actually, for example, expressed by uvw as described above) may be used. Assuming that the gripping operation using the temporary gripping position and posture is realized only by movement in the z-axis direction without changing the position on the xy plane and the posture of the gripping portion as shown in FIG. Even if the z coordinate value of the temporary gripping position is indefinite, a temporary gripping operation is possible. Specifically, the gripping unit is set at a given position at (x0, y0, z0) in a posture that matches the temporary gripping posture, and the gripping unit is moved in the positive z-axis direction without changing the posture from there. Move vertically (downward). Since the z-coordinate value of the temporary gripping position is indefinite, the end point of the movement is unknown, but here it is only necessary to move to a position where the gripping part and the workpiece come into contact and stop when the contact is detected.

  The temporary gripping position / posture can be easily acquired based on the image recognition processing, and information necessary for the approach shown in FIG. 6A can be easily acquired because the approach operation is simple. That is, a temporary gripping operation can be realized using information that can be easily acquired. Since the gripping operation can be realized using the actual machine, information necessary for the final teaching data (second teaching data) may be acquired and output from the information obtained during this operation. In the present embodiment, second teaching data is generated based on sensing information during a gripping operation. For example, by providing the grip portion with a force sensor, the above-described contact between the grip portion and the workpiece may be detected, and the grip position in the z direction may be determined based on the detected contact. Alternatively, instead of simple contact, the gripping position may be determined from the position of the gripping part when a given force F is applied. Further, in the gripping operation, an operation for determining the opening amount (closing amount) of the gripping portion is performed as in the operation of closing the finger. Therefore, it is conceivable that torque limit information of the motor that drives the finger is acquired as sensing information, and opening amount information indicating how much the finger should be closed is included in the second teaching data.

  As described above, according to the method of the present embodiment, it is possible to generate the first teaching data based on the marker recognition process and the second teaching data by operating the actual robot. Since the first teaching data is generated using the marker, it is possible to teach the gripping operation to the robot in a form that is easy for the user to understand. In the example using the markers M1 and M2 as shown in FIG. 6B, the lines L1 and L2 are displayed on the work table so that the user can understand. Then, the user is notified in advance that "the robot performs a gripping operation using fingers arranged in the L2 direction with the intersection of L1 and L2 as a target". By doing so, the user can teach a desired gripping operation only by placing the workpiece desired to be gripped by the robot on the workbench in the position and orientation to be gripped.

  For example, as shown in FIG. 7A, if the user wants to hold a rectangular parallelepiped work so that the surface A2 and the surface A3 are sandwiched between fingers from the surface A1 side, the user can The workpiece may be arranged in such a position that A2 and A3 intersect L2 with the surface A1 facing upward.

  Similarly, as shown in FIG. 7C, if the user wants to hold A5 and A6 from the A4 side, which is the back side of A1, with a finger, the user can use the surface as shown in FIG. A work may be arranged in a position and orientation where A5 and A6 cross L2 with A4 facing upward. It is also possible to set the gripping position toward the end of the workpiece. For example, as shown in FIG. 7 (E), if it is desired to grasp the positions near A5 of A2 and A3 from the A1 side with fingers, as shown in FIG. What is necessary is just to arrange | position a workpiece | work on a work bench in the state which shifted the gravity center compared with (B).

  In addition, even when various workpieces are gripped by various gripping methods, the work by the user is easy because it is simply arranged on the work table in consideration of the position and orientation of the workpiece. Note that the work placement by the user is not necessarily strict, and it is sufficient if the work is placed in a rough position and orientation. Of course, if it is necessary to determine the gripping position and orientation with very high accuracy, it is also necessary for the user to place the workpiece with high accuracy. However, in the present embodiment, importance is attached to the fact that teaching by the user is easy, and it is only necessary to generate a gripping operation in accordance with the user's intention as teaching data. I don't expect it.

  Furthermore, since the second teaching data is generated by operating the actual robot, unlike the offline teaching, when the gripping operation is executed using the generated teaching data, the workpiece can be stably gripped. It is possible to suppress the possibility of being unable to. In particular, in the example in which the first teaching data is used for the gripping operation for generating the second teaching data, the final teaching data (second teaching data) uses information at the time of the actually performed gripping operation. Since it is generated, the robot is stably operated, and since the first teaching data is used for the gripping operation, it is easy to execute the gripping operation. That is, the advantages of off-line teaching and direct teaching can be combined, and it is possible to stabilize the operation based on the generated teaching data while enabling teaching in an easy form for the user.

  Hereinafter, detailed configuration examples of the control device 100 and the robot (robot system) according to the present embodiment will be described, and then the processing of the present embodiment will be specifically described. Finally, the processing flow of the present embodiment will be described using a flowchart.

2. System Configuration Example FIG. 2 shows a detailed system configuration example of the control device 100 according to the present embodiment. However, the control device 100 is not limited to the configuration in FIG. 2, and various modifications such as omitting some of these components or adding other components are possible.

  As shown in FIG. 2, the control device 100 includes an image recognition unit 110, a processing unit 120, and a robot control unit 130, and the processing unit 120 includes a temporary gripping position / posture detection unit 121 and a sensing information acquisition unit 123. And a teaching data generation unit 125.

  The image recognition unit 110 performs image recognition processing. Here, the image recognition process may be a marker recognition process, for example. Further, in generating the teaching data, it is necessary to perform processing for recognizing the position and orientation of the workpiece in the coordinate system set in the robot. Therefore, the image recognition unit 110 may perform object recognition processing for recognizing the position and orientation of the workpiece as image recognition processing. Details of the marker recognition process and the object recognition process will be described later.

  The processing unit 120 performs various processes based on the result of the image recognition process and the acquired sensing information. The function of the processing unit 120 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program, and the like.

  The robot control unit 130 controls the robot body 300 based on the information generated by the processing of the processing unit 120. Specifically, it controls the motor that drives the arm 310 and the grip portion 319 (hand) of the robot body 300. In addition, while performing control of a motor, you may perform together the process which acquires torque limit information based on the drive result of a motor.

  The temporary gripping position / posture detection unit 121 detects a temporary gripping position / posture (a temporary gripping position and a temporary gripping posture) based on a result of image recognition processing (marker recognition processing in a narrow sense). The detected temporary gripping position / posture is output to the robot controller 130, and robot control is performed to perform a gripping operation targeting the temporary position / posture.

  The sensing information acquisition unit 123 performs processing for acquiring sensing information detected in the gripping operation with the temporary gripping position / posture as a target. As described above, the sensing information may be sensor information from the force sensor 350 or torque limit information.

  The teaching data generation unit 125 generates second teaching data including the final gripping position and orientation based on the sensing information acquired by the sensing information acquisition unit 123. An example of detailed processing in the teaching data generation unit 125 will be described later.

  Further, as shown in FIG. 3, the method of the present embodiment includes a gripping unit 319 that grips a workpiece and a control device 100 that operates the gripping unit 319, and the control device 100 captures an image of the workpiece and the marker. Thus, the first teaching data can be generated and the gripping unit 319 can grip the work to generate the second teaching data. With the configuration of FIG. 3, the robot according to the present embodiment includes a control device 100 and a robot body 300.

  The robot main body 300 includes an arm 310 and a grip portion 319 (an end effector in a broad sense) provided at the tip of the arm 310 or the like.

  As a result, a robot (robot system) that generates teaching data related to a gripping operation using an actual machine and controls the robot body 300 to actually perform the gripping operation based on the generated teaching data is realized by an easy user input. It becomes possible to do. In this robot system, the robot body 300 and the control device 100 may be provided separately as shown in FIG.

  Note that the configuration example of the robot system according to the present embodiment is not limited to FIG. For example, as shown in FIG. 4, the robot system may include a robot main body 300 and a base unit unit 400.

  The robot body 300 may be a double-arm robot, and includes a first arm 310 and a second arm 320 in addition to portions corresponding to a head and a torso. In FIG. 4, the first arm 310 includes joints 311 and 313 and frames 315 and 317 provided between the joints. The same applies to the second arm 320, but is not limited thereto. Further, end effectors 319 are provided at the tips of the first arm 310 and the second arm 320. Although FIG. 4 shows an example of a double-arm robot having two arms, the robot of this embodiment may have three or more arms.

  The base unit part 400 is provided at the lower part of the robot body 300 and supports the robot body 300. In the example of FIG. 4, the base unit 400 is provided with wheels and the like so that the entire robot can move. However, the base unit 400 may be configured to be fixed to a floor surface or the like without having wheels or the like. In the robot system of FIG. 4, the robot main body 300 and the control device 100 are integrally configured by storing the control device 100 in the base unit 400.

  In addition, the method of the present embodiment includes a robot including a gripping unit 319 that grips a workpiece, a control device 100 that operates the robot, and a marker. The control device 100 captures the workpiece and the marker to capture the first of the robot. The present invention can be applied to a robot system that generates one teaching data and causes the gripping unit 319 to grip a workpiece to generate second teaching data of the robot. Here, the robot, the control device 100, and the marker are shown as configurations included in the robot system, but other configurations may be added. For example, the robot system of the present embodiment may include an imaging unit that captures a captured image.

3. Specific Processing Example Next, a specific example of processing according to the present embodiment will be described. Hereinafter, a marker recognition process will be described as a specific example of the image recognition process, a temporary gripping position / posture will be determined from the result of the marker recognition process, and a gripping operation targeting the temporary gripping position / posture will be described. Further, the second teaching data generation process using the sensing information obtained during the gripping operation will be described, and finally a modified example will be described.

3.1 Marker Placement and Marker Recognition Processing First, an example of marker placement in the work space and an example of image recognition processing (marker recognition processing) in the image recognition unit 110 will be described. The image recognition processing in this embodiment is performed to determine a temporary gripping position / posture that is a target of the gripping operation. That is, the image to be captured by the image recognition process may be any image that can determine the temporary gripping position and orientation. In this embodiment, for example, a marker is used.

  That is, in the present embodiment, the image recognition unit 110 performs an image recognition process for recognizing a marker, and the processing unit 120 performs a temporary gripping position and a temporary gripping based on a result of an image recognition process (marker recognition process) for the marker. Determine posture.

  This makes it possible to determine the temporary gripping position and orientation using a marker that can be easily recognized by the image recognition process. In this embodiment, since it is assumed that the work is placed by the user's hand at a position corresponding to the temporary gripping position, the gripping operation desired by the user is taught if it is not appropriate to grasp the temporary gripping position and posture by the robot. Can not do it. In other words, it is necessary to grasp the temporary gripping position / posture by the robot (for example, processing for expressing the temporary gripping position / posture using the robot coordinate system) with sufficient accuracy, and use a marker that facilitates image recognition processing at that point. It is useful.

  When using a marker, it is necessary to determine the temporary gripping position and orientation by recognizing the marker as described above. If only the temporary gripping position is specified, a marker that defines a point is sufficient, but in order to specify the temporary gripping posture, it is necessary to use a marker that further defines the direction.

  For example, when a marker having a symmetry that does not change the shape pattern even when rotated at an arbitrary angle, one point on the work space (one point on the worktable plane in a narrow sense) can be obtained by using one of the markers. Can be identified. However, in that case, the temporary gripping position can be determined as the position of the marker, or a position shifted from the marker by (x, y) in the robot coordinate system, but the temporary gripping posture cannot be specified.

  Accordingly, the image recognition unit 110 performs an image recognition process for recognizing a plurality of markers arranged around the workpiece to be grasped, and the processing unit 120 temporarily determines the result of the image recognition process for the plurality of markers. The gripping position and the temporary gripping posture may be determined.

  For example, two markers M3 and M4 may be arranged as shown in FIG. In this case, since the direction can be defined by the line L3 connecting M3 and M4, for example, the temporary gripping position can be specified as the position of M3 and the temporary gripping posture can be specified as a posture in which fingers are arranged on L3.

  Alternatively, three or more similar markers may be arranged. For example, if four markers M5 to M8 are arranged as shown in FIG. 8B, a line L4 can be defined from M5 and M6, and a line L5 can be defined from M7 and M8. Therefore, it is possible to determine the temporary gripping posture from at least one of L4 and L5 using the intersection of L4 and L5 as the temporary gripping position.

  The image recognition unit 110 also recognizes a first marker that is a marker indicating the first direction and a second marker that is a marker indicating the second direction that intersects the first direction. The processing unit 120 determines a temporary gripping position based on a position where the first direction and the second direction intersect, and based on at least one of the first direction and the second direction, The gripping posture may be determined.

  For example, a symmetric marker having the same shape pattern when rotated 180 degrees may be used as the first and second markers. As shown in FIG. 8C, the markers M9 and M10 can detect not only the position where the marker exists but also the direction in which the marker is arranged by detecting the pattern of the marker. Therefore, by recognizing the marker M9, the line L6 in a given direction can be specified together with the position of M9. Similarly, by recognizing the marker M10, the line L7 in a given direction can be specified together with the position of M10. In this case, using two markers, two directions can be defined as in FIG. 8B, and the temporary gripping position and orientation can be specified. In this case, since one marker can specify one direction, the number of markers used can be reduced as compared with FIG.

  In addition, if the marker does not have rotational symmetry, the position and direction of the marker can be uniquely specified by recognizing the shape pattern of the marker. Therefore, for example, when considering two directions of a marker reference direction and a direction orthogonal to the reference direction, the two directions can be specified by recognizing one marker. That is, as shown in FIG. 8D, it is also possible to determine the temporary gripping position / posture similarly to FIGS. 8B and 8C using one marker.

  However, the more information that can be recognized from one marker, the more complicated the shape and pattern of the marker. For example, a marker that only needs to know a position such as M3 to M8 can be made very simple, but a marker that recognizes even a fine direction such as M11 needs to have a shape pattern without symmetry. For this reason, it may be considered that the use of a complicated marker reduces the marker recognition accuracy or requires the marker itself to be increased in size. In other words, it can be said that by using many markers, the size of each marker may be reduced or the detection accuracy may be improved. As described above, in this embodiment, various markers can be used in various arrangements. However, in consideration of the number of markers that can be arranged, the accuracy of image recognition processing, and the like, an appropriate marker type and the number of markers. It is desirable to determine.

  The image recognition unit 110 performs image recognition processing (marker recognition processing) on a captured image obtained by capturing the work space. For example, it is assumed that the captured image IM is the image illustrated in FIG. 9 and a work table on which markers are arranged is captured on a part of IM. In that case, the image recognition unit 110 performs marker recognition processing for IM and recognizes the position of the marker on the image.

  The result of the image recognition process represents the position of the marker on the image. For example, in the example of FIG. 9, when the upper left point (pixel) of the image is used as a reference, the marker is m0 pixel on the right, one on the position of n0 pixel below, m1 pixel on the right, and below Information that one is detected at the position of the n1 pixel is acquired. Since the information indicating the marker position cannot be used for robot control as it is, the temporary gripping position / posture detection unit 121 of the processing unit 120 converts the marker position into the robot coordinate system and specifies the temporary gripping position / posture. Do.

  For example, when the position of the imaging unit that captures a captured image and the imaging direction are known in the robot coordinate system, the position on the image of the captured image and the coordinate value in the robot coordinate system can be directly converted. That is, since the coordinate conversion matrix between the coordinate system representing the position on the image (the coordinate system defined by the mn axis in the example of FIG. 9) and the robot coordinate system is known, the result of the image recognition process By performing the coordinate conversion process, the marker position can be acquired in the robot coordinate system. If the marker position and the direction (line) determined by the marker can be acquired in the robot coordinate system, the temporary gripping position and orientation can be easily determined from the intersection of two lines, the direction along one line, etc. Can be determined.

  Further, when the position of the imaging unit and the imaging direction are unknown in the robot coordinate system, it is necessary to obtain a coordinate transformation matrix or the like using some known information. For example, when the robot is placed in the work space, the positional relationship between the robot and the work table is fixed. In this case, the position of the work table is acquired as information that can be expressed in the robot coordinate system. Therefore, the process of recognizing the work table from the captured image IM is performed together, and a conversion matrix and the like used for the coordinate conversion process can be obtained from the position of the work table in IM and the position of the work table in the robot coordinate system. Subsequent processing is the same as when the coordinate transformation processing is known.

3.2 Gripping Operation Targeting the Temporary Gripping Position / Posture and Generation of Second Teaching Data Next, the processing unit 120 executes a gripping operation targeted for the temporary gripping position / posture. Specifically, the robot controller 130 is instructed to control the robot to lower the gripper in the same posture as the temporary gripping posture from the upper side of the temporary gripping position to the lower side of the vertical direction with the temporary gripping position and posture as a target. do it. At this time, in the approach from above, since the finger structure of the gripping part is unlikely to collide with an obstacle (workbench, robot, other work that is not the work target), Open it. In this way, it is possible to suppress the possibility that the finger will collide with the upper surface of the workpiece to be grasped.

  As shown in FIG. 9, when the captured image IM is captured from vertically above the workbench, the temporary gripping position acquired by the image recognition processing for the captured image becomes a position in the xy plane and is in the z-axis direction. The position is indefinite. However, in this embodiment, as described above, since the approach of the gripping operation is performed only by the movement in the z-axis direction, the gripping operation can be realized by actually performing the approach and stopping the gripping portion at a position where it contacts the workpiece. . That is, it is not necessary to strictly determine the temporary gripping position before the gripping operation.

  In other words, in order to determine the stop position of the approach, it is necessary to detect the contact between the grip portion and the workpiece by some method. Therefore, in the present embodiment, as shown in FIG. 10A, a force sensor 350 may be provided in the grip portion 319 of the robot body 300.

  In this case, the sensing information acquired in the gripping operation is force sensor information output by the force sensor 350 provided in the robot. As shown in FIG. 10B, when the grip portion 319 comes into contact with the workpiece, the workpiece is pressed against the force sensor 350, and thus some force is detected. In this embodiment, when a force is detected by the force sensor 350, the gripping approach is stopped.

  After the approach is stopped, the workpiece is fixed (gripped) using the structure of the gripping portion. For example, when the gripping portion has a finger-shaped variable portion as shown in FIG. 5 and the like, the workpiece is gripped by closing the finger. In addition, if the gripping portion has a variable portion that can be opened and closed, an operation of closing the variable portion in the open state may be performed.

  The closing amount (opening amount) of the variable portion of the gripping portion is not determined in advance. Therefore, in the present embodiment, similar to the approach stop position, the gripping operation is tried and the closing amount is determined from the result. Specifically, a torque limiter or a similar structure may be provided in the motor that drives the variable portion, and information indicating that may be obtained when an overload exceeding a predetermined value occurs.

  In this case, the sensing information acquired in the gripping operation is torque limit information of the motor that drives the robot. When shifting from the state in which the gripping portion is opened as shown in FIG. 10 (A) to the state in which the gripping portion is closed as shown in FIG. 10 (B), the motor is driven to change the variable portion. Generate torque in the direction of closing. However, in a state where the variable portion and the workpiece are in contact and the workpiece is gripped with sufficient strength, a high load is generated if an attempt is made to generate torque in the direction of closing the variable portion. In this state, it can be determined that the variable part is excessively closed. Therefore, if the value at that time is used as the closing amount of the variable part, the work to be taught can be gripped stably (with sufficient force). Is possible.

  That is, in this embodiment, information that could not be determined before the start of the gripping operation is determined using the sensing information acquired during the gripping operation. And the grasping operation was realized using the actual machine by adding the information determined by the sensing information to the information on the temporary grasping position and orientation and the information on the approach (here, the operation of moving down in the z-axis direction). It will be. If a gripping operation using an actual machine is realized, the gripping operation here can be made reproducible by using the information at that time as the second teaching data.

  Therefore, the processing unit 120 determines a grip position in the vertical direction with respect to the work table based on the force sensor information, and generates second teaching data including the determined grip position. Here, the gripping position in the vertical direction with respect to the work table is the position in the z-axis direction in FIG. 6 and the like, and is information for specifying the approach distance in the gripping operation.

  Further, the processing unit 120 determines the opening amount information of the gripping portion in the state where the workpiece is gripped based on the torque limit information, and generates second teaching data including the determined opening amount information. Here, the opening amount information of the gripping part is information indicating how much the variable part is opened (closed) when the gripping part has a variable part that opens and closes. When the variable part is configured by one or a plurality of joints, the opening amount information may be expressed by a joint angle of each joint.

  This makes it possible to use sensing information to determine information necessary for the gripping operation, such as the approach stop position and the finger structure closing state, and realize the gripping operation using the temporary gripping position and orientation. It becomes possible. Further, the information acquired based on the sensing information can be used as the second teaching data as it is to reproduce the gripping operation using the temporary gripping position / posture.

  For example, the robot parameters (for example, the values of the joint angles of each joint included in the robot) in the gripping operation using the temporary gripping position and posture are stored one by one, and the robot control according to the robot parameters is performed, thereby performing the gripping operation. May be reproduced.

  Alternatively, a parameter abstracted compared to the joint angle or the like may be generated as the second teaching data in order to improve the flexibility of the gripping operation. The second teaching data here is information including, for example, final gripping position, gripping posture, and opening amount information of the gripping part. For the final gripping position / posture, the position / posture of the gripping part at the end of the approach in the gripping operation using the temporary gripping position / posture may be used, and the opening amount information may be information determined by the torque limit information. .

  In actual robot control using the second teaching data, the gripping unit is set vertically above the final gripping position in a final gripping posture, and from that state, the final gripping position and posture are determined. The grip part is lowered (approached) in the z-axis direction until After the approach is completed, the gripping operation is completed by closing the movable part of the gripper by the amount specified by the opening amount information.

  Note that the method of the present embodiment creates teaching data that enables a stable gripping operation by actually gripping a workpiece using a real robot and using the result. That is, the teaching data generated by the method of the present embodiment should be information that defines the relationship with the workpiece. On the other hand, in the above description, the temporary gripping position / posture and the final gripping position / posture are information indicating the target in the gripping unit control of the robot, and do not indicate the position of the workpiece. In other words, as it is, only the teaching data that “the gripping operation is performed for the given position as a target” has been created, and it means that the workpiece is not placed at the “given position” in the same position and orientation as at the time of teaching. There will be no.

  Therefore, in this embodiment, the image recognition unit 110 performs object recognition processing for recognizing the position and orientation of the workpiece as image recognition processing, and the processing unit 120 determines the position and orientation of the workpiece acquired by the object recognition processing, and finally Teaching data (second teaching data in a narrow sense) including a proper gripping position and a final gripping posture is generated.

  Here, the object recognition process itself can be realized by a widely known method such as template matching using the workpiece model data as a template, and thus detailed description thereof is omitted.

  This makes it possible to recognize the position and orientation of the workpiece on the image. Further, by performing the coordinate conversion process in the same manner as the marker recognition process, the position and orientation of the workpiece can be acquired by the robot coordinate system. Therefore, even when teaching data targeting a given gripping position / posture is obtained, the gripping position / posture can be understood as a relative relationship with the workpiece. For example, teaching data “perform a gripping operation with (x, y, z, u, v, w) as targets in the robot coordinate system” is referred to as “perform a gripping operation with a given point on the workpiece as a target”. Can be transformed into teaching data. Therefore, even when the position of the workpiece changes, the gripping operation can be appropriately changed according to the change of the workpiece position, and teaching data that can flexibly cope with many situations can be generated. .

  Further, the image recognition unit 110 may perform the object recognition process after causing the robot to perform a gripping operation of gripping the workpiece by the gripping unit 319 of the robot with the temporary gripping position and the temporary gripping posture as targets.

  In the present embodiment, it is not hindered to perform the object recognition process before the gripping operation targeting the temporary gripping position / posture. However, depending on the arrangement of the workpiece by the user, the weight balance of the workpiece itself, the configuration of the gripping portion 319 of the robot, the position and orientation of the workpiece may change during the gripping operation. For example, when gripping a workpiece with two fingers, it is difficult to think of the movement of the workpiece if the direction of the force of the left finger and the direction of the force of the right finger are on the same straight line and opposite directions. However, when the work is disposed obliquely or the gripping part itself is not so rigid, the force by each finger is not on the same straight line, and there is a possibility of generating a moment for rotating the work. The stable gripping operation in the present embodiment is realized by reproducing the final state after the gripping operation is completed. That is, it can be said that it is preferable to use the state after the workpiece is gripped by the grip portion 319 by the gripping operation as teaching data, not the state where the workpiece is placed by the user. In other words, by performing the object recognition processing after the gripping operation with the temporary gripping position / posture as a target, even when the workpiece is moved by the gripping operation, teaching data that realizes a stable gripping operation can be generated. It becomes possible.

3.3 Modifications As described above, in this embodiment, in order to easily realize the gripping operation using the temporary gripping position and orientation, the approach in the gripping operation is simple and simple as shown in FIG. It is assumed that automatic generation is easy. This makes it difficult to perform a gripping operation on a given workpiece from a plurality of approach directions.

  For example, as described above with reference to FIGS. 7A to 7F, there is a scene where one workpiece is desired to be gripped from the direction of the surface A1, and another scene is desired to be gripped from the direction of the surface A4. is there. Alternatively, there is a scene where it is desired to change the contact points on the surfaces A2 and A3 even when gripping from the direction of A1. In this embodiment, since the approach direction at the time of teaching is constant, the gripping operation from various directions is taught by changing the position and orientation on the workpiece side.

Specifically, when a workpiece having first to Nth (N is an integer equal to or greater than 2) surfaces is to be grasped, the processing unit 120 satisfies the i-th of the workpiece (i satisfies 1 ≦ i ≦ N). (Integer) surface as a processing target surface, the robot performs a gripping operation targeting the temporary gripping position and the temporary gripping posture, and the i-th surface is defined as the processing target surface based on sensing information detected in the gripping operation. The second _i teaching data, which is the second teaching data, is generated. Similarly, the processing unit 120 uses the second _{j surface} (j is an integer satisfying 1 ≦ j ≦ N, j ≠ i) as a processing target surface, and performs a gripping operation with a temporary gripping position and a temporary gripping posture as targets. And the second _j teaching data, which is the second teaching data having the j th surface as the processing target surface, is generated based on the sensing information detected in the gripping operation.

  This makes it possible to teach a gripping operation from a different surface direction of a given workpiece. The processing target surface here refers to the direction of the approach point of the gripping unit 319 (corresponding to the upper position in FIG. 6A) of the gripping unit 319 among a plurality of surfaces of the workpiece. Represents the facing surface. In FIG. 7A, the surface A1 is the processing target surface, and in FIG. 7C, the surface A4 is the processing target surface. As shown in FIGS. 7A and 7C, even when it is desired to teach a gripping operation from a plurality of directions, the teaching data is appropriately generated by repeating the above processing for each surface. It becomes possible to do.

At this time, when the workpiece is arranged in a position and orientation in which the i-th surface of the workpiece faces the vertically upward direction of the work table, the processing unit 120 uses the i-th surface of the workpiece as a processing target surface and performs a temporary holding position. Then, the robot may perform a gripping operation targeting the temporary gripping posture, and the second _i teaching data may be generated based on the sensing information detected in the gripping operation.

As a result, it is possible to generate appropriate teaching data according to the position and orientation of the placed workpiece. In the present embodiment, it is considered to reduce as much as possible the processing load for generating an approach motion on the system side when realizing a gripping operation targeting the temporary gripping position and orientation. That is, the diversity of approach motions is not high, but teaching data that realizes various gripping motions can be generated by changing the position and orientation of the workpiece. Since the work placement is left to the user, the processing load on the control device 100 side does not increase. In addition, since the user only places the work at a predetermined position with the processing target surface facing upward, the work placement is not a heavy burden. In the present embodiment, as shown in FIG. 6 (A), since the approach is made from the vertically upward direction, the i-th surface of the workpiece is directed vertically upward. That is, in a broad sense, when the workpiece is arranged in a position and orientation in which the i-th surface of the workpiece faces the direction of the approach point of the gripping unit, the processing unit 120 uses the i-th surface of the workpiece as a processing target surface. The robot may perform a gripping operation targeting the temporary gripping position and the temporary gripping posture, and the second _i teaching data may be generated based on sensing information detected in the gripping operation.

  In addition, the user may input some gripping parameters regarding the gripping operation to be taught. For example, the user may specify the depth of gripping. Here, the gripping depth is information representing the contact position between the movable part of the gripping part and the workpiece. For example, as shown in FIG. 7A, when gripping the work by pressing the fingers of the gripping part against the surfaces A2 and A3, the contact positions of the fingers A2, A3 and the finger are close to the work table. Alternatively, it may be held with the contact position close to the surface A1. In the present embodiment, the grip state at a position far from the processing target surface (position close to the work table) is expressed as “deep grip”, and the grip state at a position close to the processing target surface is expressed as “shallow grip”. To do.

  As described above, in the present embodiment, the position where the gripping portion and the workpiece come into contact is set as the gripping position. Therefore, it is difficult to adjust the gripping depth when the relative position between the finger and the part in contact with the workpiece, such as the gripping portion shown in FIG. However, as shown in FIG. 11A, if the part 3193 is movable, the relative position between the part 3193 and the fingers 3191 and 3192 can be changed. Therefore, if the part 3193 is pushed out as shown in FIG. 11B, the gripping of the workpiece by the fingers 3191 and 3192 can be shallow. On the other hand, when the part 3193 is pulled as shown in FIG. 11C, the workpieces 3191 and 3192 can be deeply gripped.

  In other words, with the configuration of the gripping portion as shown in FIG. 11A, it is possible to receive information on the gripping depth from the user and generate teaching data that realizes a gripping operation corresponding to the information. Become. In the configuration of FIG. 11A, control may be performed to pull the part 3193 when instructed to deepen the grip and to push out the part 3193 when instructed to shallowly grip. If the teaching data is generated by performing a gripping operation targeting the temporary gripping position and orientation after the control of the part 3193, the teaching data reflects the user's intention regarding the gripping depth.

  Further, the user may specify the strength of gripping. As described above, the strength of gripping is determined by the torque limit information of the motor that drives the variable portion of the gripper (for example, fingers 3191 and 3192). At that time, the degree of load as a limit can be changed by setting. For example, if it is determined that the load is a limit even with a relatively small load, the force applied to the workpiece by the variable unit is relatively small, and if the load is not relatively large, the limit is not determined. The force applied to the workpiece is relatively large. That is, it is possible to generate teaching data that realizes a gripping operation corresponding to a user instruction by receiving information on the gripping strength from the user and setting a torque limit based on the information.

  Further, the screen shown in FIG. 12 or the like may be used as the above-described teaching of the present embodiment and the user interface for setting the gripping parameters. In the example of FIG. 12, as shown in B <b> 1, by displaying a slider, it is possible to input a gripping depth (grip depth) among gripping parameters. Further, as shown in B2, by displaying radio buttons, it is possible to input grip strength (grip strength) among grip parameters.

  In B3, a plurality of buttons for instructing the robot to perform necessary operations are displayed. The home button is a button for giving an instruction to return the robot to the home position. When the robot arm 310 or the like is driven, the arm 310 or the like appears in the captured image IM, and the accuracy of the marker recognition process or the object recognition process may be reduced. The home position represents the position (and posture) of the robot so that at least the robot parts are not reflected in the captured image. It is assumed that the home button is pressed before the marker recognition process or the object recognition process. The return button is a button for returning the operation of the robot or the control device 100 to the previous state.

  The marker recognition button is a button for instructing marker recognition processing, and the object recognition button is a button for instructing object recognition processing. The grip button is a button for instructing the start of a gripping operation with the temporary gripping position / posture as a target. When the grip button is pressed, a gripping operation corresponding to the grip depth and strength instructed by the interface of B1 is executed.

  The registration button is a button for registering teaching data generated based on the gripping operation by pressing the gripping button in the database. In such an interface, when the gripping operation executed by pressing the grip button is not a desired motion, the user does not have to press the registration button. By doing so, it is possible to prevent storing teaching data that does not match the user's intention.

  In addition, as shown to B4, you may provide an interface which performs the input regarding a finger direction. For example, in FIG. 11A and the like, the number of fingers of the gripping part is two, but may be four or the like. In that case, each finger is placed at the position of the apex of the quadrangle, and the two fingers are used to pinch in the first direction, and the remaining two are used to cross the first direction (in a narrow sense). Is sandwiched in the second direction. The interface of B4 is used to instruct how to use the finger. When the check box for no finger direction gripping is checked, a gripping operation is performed in which only one direction is sandwiched without using two of the four fingers. Let Also, when the check box after the finger direction is checked, a gripping operation is performed in which there is a time difference between pinching with two fingers and pinching with the remaining two fingers.

  In addition, the input interface in the method of this embodiment is not limited to FIG. 12, and various deformation | transformation implementation is possible.

4). Process Flow FIG. 13 is a flowchart for explaining the process of this embodiment. When this process is started, a marker recognition process is first performed (S101). The details of the marker recognition process are as described above, and the process of S101 may be performed by, for example, pressing the marker recognition button in FIG. 12 as a trigger.

  Next, a gripping position / orientation is specified based on the result of the marker recognition process (S102), and a gripping operation is performed with the specified gripping position / orientation as a target (S103). In the process of S103, sensing information is acquired as described above, and the approach stop position and the finger opening amount are determined.

  The position and orientation of the workpiece at the end of the gripping operation in S103 are recognized by object recognition processing (S104), and teaching data is generated based on the processing results in S103 and S104 (S105). If the generated teaching data is appropriate, the teaching data is registered in the database (S106).

  Thereafter, it is determined whether the teaching has been completed (S107). If not, the process returns to S101 to continue the processing. Note that if the relative position between the robot and the workbench does not change, it is not necessary to perform the processing of S101 and S102 again. Therefore, if No in S107, the process may return to S103. If Yes in S107, the process ends.

  Note that the control device 100 or the like of the present embodiment may realize part or most of the processing by a program. In this case, the control device 100 according to the present embodiment is realized by a processor such as a CPU executing a program. Specifically, a program stored in a non-temporary information storage medium is read, and a processor such as a CPU executes the read program. Here, the information storage medium (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes according to the present embodiment based on a program (data) stored in the information storage medium. That is, in the information storage medium, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit) Is memorized.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configuration and operation of the control device 100 and the like are not limited to those described in the present embodiment, and various modifications can be made.

EP end point, IM captured image, L1-L7 line,
M1 to M11 markers, 100 control device, 110 image recognition unit, 120 processing unit,
121 temporary gripping position and orientation detection unit, 123 sensing information acquisition unit,
125 teaching data generation unit, 130 robot control unit, 300 robot body,
310 arm, 311, 313 joint, 315, 317 frame, 319 gripping part, 320 second arm, 350 force sensor, 400 base unit part,
3191, 3192 fingers, 3193 parts,

Claims (15)

An image recognition unit that accepts a captured image obtained by imaging the workpiece and the marker;
A processing unit for generating robot teaching data;
Including
The processor is
Generating first teaching data of the robot by the captured image received by the image recognition unit;
A control device, wherein the robot is configured to generate the second teaching data of the robot by gripping the workpiece. In claim 1,
The processor is
A control apparatus that causes the gripper to grip the workpiece based on the first teaching data and generates the second teaching data of the robot. In claim 1 or 2,
The processor is
By causing the gripping part to grip the workpiece, the force sensor information output by a force sensor provided in the robot is obtained,
A control device that determines a grip position in a vertical direction with respect to a work table based on the force sensor information, and generates the second teaching data including the determined grip position. In any one of Claims 1 thru | or 3,
The processor is
By causing the gripping part to grip the work, obtain torque limit information of a motor that drives the robot,
Based on the torque limit information, the opening amount information of the gripping portion in a state where the workpiece is gripped is determined, and the second teaching data including the determined opening amount information is generated. . In any one of Claims 1 thru | or 4,
The image recognition unit
Perform image recognition processing to recognize the marker,
The processor is
The control device, wherein the first teaching data is generated based on a result of the image recognition processing for the marker. In claim 5,
The image recognition unit
Performing the image recognition processing for recognizing a plurality of the markers arranged around the workpiece to be grasped;
The processor is
The control device, wherein the first teaching data is generated based on a result of the image recognition processing for a plurality of the markers. In claim 6,
The image recognition unit
Performing the image recognition processing for recognizing a first marker that is the marker indicating a first direction and a second marker that is the marker indicating a second direction intersecting the first direction;
The processor is
A temporary gripping position is determined based on a position where the first direction and the second direction intersect, and a temporary gripping posture is determined based on at least one of the first direction and the second direction. ,
The control apparatus that generates the first teaching data including the temporary gripping position and the temporary gripping posture. In any one of Claims 1 thru | or 7,
When the workpiece having the first to Nth surfaces (N is an integer of 2 or more) is a gripping target,
The processor is
The i-th surface of the workpiece (i is an integer satisfying 1 ≦ i ≦ N) is a processing target surface, the gripping part is gripped by the workpiece, and the i-th surface is the processing target surface. generating the teaching data of the 2 _i is two teaching data,
Using the jth surface of the workpiece (j is an integer satisfying 1 ≦ j ≦ N, j ≠ i) as the processing target surface, the gripping portion is caused to grip the workpiece, and the jth surface is the processing target. A control apparatus for generating second _j teaching data which is the second teaching data as a plane. In claim 8,
When the workpiece is arranged in a position and orientation in which the i-th surface of the workpiece is directed vertically upward of a work table,
The processor is
The control device, wherein the i-th surface of the workpiece is the processing target surface, and the gripping unit grips the workpiece to generate the second _i- teach data. In any one of Claims 1 thru | or 9,
The image recognition unit
Perform object recognition processing to recognize the position and orientation of the workpiece,
The processor is
A control apparatus that generates the second teaching data including a position and orientation of the workpiece, a final gripping position, and a final gripping posture acquired by the object recognition processing. In claim 10,
The image recognition unit
A control apparatus that performs the object recognition process after the gripping unit grips the workpiece based on the first teaching data. In any one of Claims 1 thru | or 11,
The processor is
Based on the captured image, generate the first teaching data including a temporary gripping position and a temporary gripping posture,
The second teaching data including the final gripping position and the final gripping posture are generated by causing the gripping part to grip the workpiece with the temporary gripping position and the temporary gripping posture as targets. Control device. A gripping part for gripping the workpiece;
A control device for operating the gripping part;
Including
The controller is
Imaging the workpiece and the marker to generate first teaching data;
A robot characterized in that the gripping unit grips the workpiece to generate second teaching data. Imaging a workpiece and a marker to generate first teaching data of the robot;
And generating a second teaching data of the robot by causing the gripping portion of the robot to grip the workpiece. A robot having a gripping part for gripping a workpiece;
A control device for operating the robot;
A marker, and
The controller is
Imaging the workpiece and the marker to generate first teaching data of the robot;
A robot system, wherein the gripping unit grips the workpiece to generate second teaching data of the robot.

Images