JP2016221647A - Grip control device, grip control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism for reducing the possibility of the occurrence of a failure such as falling or breaking of a gripped workpiece and another workpiece when the gripped workpiece is taken out from a box or the like.SOLUTION: A grip control device comprises: recognition units 112, 113 each recognizing grip position information on a workpiece to be gripped and surrounding information on the workpiece to be gripped on the basis of workpiece images of a plurality of workpieces obtained by an imaging device 130 and workpiece information which is held in a workpiece information holding unit 118 and which is about the workpiece to be gripped; a grip method determination unit 115 determining a grip method for gripping the workpiece to be gripped on the basis of the grip position information, the surrounding information, and grip form information which is held in a grip-form-information holding unit 119 and which is about a grip form of a hand 121; and a robot control unit 116 controlling the hand 121 of a robot 120 to grip the workpiece to be gripped on the basis of the grip method determined by the grip method determination unit 115.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a gripping control device and a gripping control method for controlling gripping by a robot hand when gripping and taking out a workpiece to be gripped by a robot hand from a plurality of workpieces, and the gripping control method. The present invention relates to a program for causing a computer to execute.

  2. Description of the Related Art Conventionally, there has been proposed a technique for controlling a robot so that the workpiece can be held in a stable state when holding a workpiece with a hand attached to the robot (for example, see Patent Document 1 below). Specifically, Patent Document 1 below describes a technique for controlling a robot hand so that the number of contact points between the workpiece and the robot hand increases.

JP 2007-125653 A JP2011-22991A

  However, in the technique described in Patent Document 1, for example, even when a workpiece to be grasped interferes with another workpiece in a workpiece supplied in a stacked state, the contact point with the robot hand is more The workpiece to be grasped is grasped so as to increase. Therefore, when the gripped work is taken out from the box or the like, the force based on the pick-up operation acts on and interferes with the other work, resulting in problems such as dropping or breakage of the gripped work and the other work. there is a possibility.

  The present invention has been made in view of such problems, and reduces the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces when taking out the gripped workpiece from a box or the like. The purpose is to provide a mechanism.

The gripping control device of the present invention is a gripping control device that controls gripping by the robot hand when gripping and taking out the workpiece to be gripped by a robot hand from a plurality of workpieces. A workpiece information holding unit that holds workpiece information related to the workpiece, a gripping mode information holding unit that holds gripping mode information related to a gripping mode in the robot hand, and workpieces in the plurality of workpieces obtained by the imaging device Based on the image and the workpiece information, a recognition unit that recognizes gripping position information related to a gripping position in the gripping target workpiece and surrounding information related to a surrounding situation in the gripping target workpiece, the gripping position information, Determination of a gripping method for determining a gripping method when gripping the workpiece to be gripped based on surrounding information and the gripping form information When, and a on the basis of the holding method holding method determined by the determination unit, a control unit for controlling the grasping by the hand of the robot with respect to the gripping target work.
The present invention also includes a grip control method using the above-described grip control device, and a program for causing a computer to execute the grip control method.

  According to the present invention, when a gripped work is taken out from a box or the like, it is possible to reduce a possibility that a problem such as dropping or breakage of the gripped work and another work occurs.

It is a schematic diagram which shows an example of schematic structure of the holding | grip control system which concerns on the 1st Embodiment of this invention. FIG. 2 is a schematic diagram illustrating an example in which the robot hand takes out a workpiece to be grasped from a box based on a grasping method determined by the grasping control device illustrated in FIG. 1 according to the first embodiment of the present invention. It is a block diagram which shows an example of an internal structure of the holding | grip control apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the 1st Embodiment of this invention and shows an example of the item contained in the holding | maintenance method of the workpiece | work information hold | maintained at the workpiece | work information holding | maintenance part shown in FIG. 3, and workpiece | work information. It is a figure which shows an example of the holding method of the holding | grip form information hold | maintained at the holding | grip form information holding part shown in FIG. 3, and the item contained in holding | grip form information. It is a figure for demonstrating the grip form information hold | maintained at the grip form information holding part shown in FIG. It is a schematic diagram for demonstrating the recognition process of the holding position information by the holding position recognition part shown in FIG. It is a schematic diagram for demonstrating the recognition process of the surrounding information by the environment recognition part shown in FIG. It is a schematic diagram for demonstrating the evaluation process of each holding method by the holding method evaluation part shown in FIG. It is a flowchart which shows an example of the process sequence of the grip control method by the grip control apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the 2nd Embodiment of this invention and shows an example of the item contained in the holding | maintenance method of the workpiece | work information hold | maintained at the workpiece | work information holding | maintenance part shown in FIG. 3, and workpiece | work information. It is a schematic diagram for demonstrating the workpiece | work information shown in FIG. FIG. 10 is a schematic diagram for explaining a processing content of an environment recognition unit and a processing method of a gripping method evaluation unit shown in FIG. 3 according to a third embodiment of the present invention. It is a figure which shows the 4th Embodiment of this invention and shows an example of the holding | maintenance method of the workpiece | work information hold | maintained at the workpiece | work information holding | maintenance part shown in FIG. 3, and the item contained in workpiece | work information. FIG. 10 is a schematic diagram for explaining a processing content of a gripping method evaluation unit shown in FIG. 3 according to a fourth embodiment of the present invention. FIG. 10 is a schematic diagram for explaining a processing content of an environment recognition unit and a processing method of a gripping method evaluation unit shown in FIG. 3 according to a fifth embodiment of the present invention. It is the characteristic view which showed the 5th Embodiment of this invention and recorded the force which the other workpiece | work which is a surrounding obstruction acts on the workpiece | work of a holding | grip object in time series.

  Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described.

  In the present embodiment, based on the grip position information related to the grip position of the grip target work, the peripheral information related to the surrounding condition of the grip target work, and the grip form information related to the grip form of the robot hand, Determine the gripping method for gripping the workpiece. As a result, when the gripped work is taken out from the box or the like, the influence of the gripped work on other workpieces can be reduced, so that a problem such as dropping or breakage of the gripped workpiece and other workpieces may occur. Can be reduced.

  FIG. 1 is a schematic diagram illustrating an example of a schematic configuration of a grip control system 100 according to the first embodiment of the present invention. As shown in FIG. 1, the grip control system 100 includes a grip control device 110, a robot 120, and an imaging device 130.

  The gripping control device 110 is a device for controlling gripping by the hand 121 of the robot 120 when gripping the workpiece 201 to be gripped by the hand 121 of the robot 120 from the bulky workpiece 200 and taking it out of the box 210. is there.

  The robot 120 is a device that performs a work of gripping the workpiece 201 to be gripped from the separated workpieces 200 and taking it out of the box 210 with the hand 121 based on the control of the grip control device 110. In the work area of the robot 120, there are a work 200 stacked in bulk and a box 210 for storing the work 200.

  The imaging device 130 is a device that takes an image of the workpiece 200 stacked in the box 210 based on the control of the grip control device 110 and generates a stacked workpiece image.

Next, an operation flow in the grip control system 100 shown in FIG. 1 will be described.
First, the grip control device 110, based on the loosely stacked workpiece image captured by the imaging device 130 and the workpiece information regarding the grip target workpiece 201, grip position information regarding the grip position on the grip target workpiece 201, and Then, the surrounding information related to the surrounding state of the workpiece 201 to be grasped is recognized.

  Thereafter, the grip control device 110 grips the gripping target workpiece 201 based on the grip position information described above, the surrounding information described above, and the grip pattern information related to the grip pattern in the hand 121 of the robot 120. To decide.

Thereafter, the grip control device 110 controls the robot 120 and the hand 121 based on the determined gripping method of the workpiece 201 to be gripped, and controls gripping of the gripped workpiece 201 by the hand 121 of the robot 120. In response to this, the robot 120 grips the workpiece 201 to be gripped with the hand 121 and takes out the workpiece 201 to be gripped from the box 210.
In this embodiment, an example in which a workpiece to be grasped is taken out from the workpieces stacked in bulk will be described. However, in the present invention, even when a plurality of aligned workpieces are taken out, the workpieces interfere with each other. If there is a possibility to do, it is applicable.

Next, an example in which the workpiece 201 to be gripped is taken out from the box 210 by the determined gripping method will be described with reference to FIG.
FIG. 2 shows a first embodiment of the present invention, and is a schematic diagram showing an example in which the robot hand 121 takes out the workpiece 201 to be gripped from the box 210 based on the gripping method determined by the gripping control device 110 shown in FIG. FIG. 2A, 2 </ b> B, and 2 </ b> C illustrate a process in which the robot hand 121 takes out the workpiece 201 to be grasped from the box 210 in time series.

  As shown in FIG. 2A, the workpiece 201 to be gripped among the stacked workpieces 200 is gripped by the hand 121. In addition, among the stacked workpieces 200, another workpiece 301 that is a peripheral workpiece with respect to the workpiece 201 to be gripped is stored in the box 210 in a state of interfering with the workpiece 201 to be gripped.

  Subsequently, the hand 121 moves in the Z direction of the coordinate system 302 and tries to take out the gripped workpiece 201 to be gripped from the box 210. Here, when the hand 121 is moved in the Z direction in a state where the gripping target workpiece 201 is not relatively movable with respect to the hand 121, the other workpiece 301 is lifted by the gripping target workpiece 201, and the box Drop off from 210. However, in the example illustrated in FIG. 2, the gripping control device 110 selects a gripping method that allows the workpiece 201 to be gripped to move relative to the hand 121.

  Therefore, at the time of FIG. 2B, the workpiece 201 to be gripped moves relative to the hand 121 by the weight of the other workpiece 301.

  As a result, the workpiece 201 to be grasped can be taken out of the box 210 without dropping the other workpiece 301 from the box 210 at the time of FIG.

Next, the internal configuration of the grip control device 110 will be described with reference to FIG.
FIG. 3 is a block diagram illustrating an example of an internal configuration of the grip control device 110 according to the first embodiment of the present invention. In addition to the grip control device 110, FIG. 3 illustrates the robot 120, its hand 121, and the imaging device 130 shown in FIG. Further, FIG. 3 does not show the workpiece 200 and the box 210 stacked in bulk shown in FIG.

  As illustrated in FIG. 3, the grip control device 110 includes a control unit 111 and a holding unit 117. As shown in FIG. 3, the control unit 111 includes a gripping position recognition unit 112, an environment recognition unit 113, a gripping method evaluation unit 114, a gripping method determination unit 115, and a robot control unit 116. Further, as shown in FIG. 3, the holding unit 117 includes a work information holding unit 118 and a gripping form information holding unit 119.

  The workpiece information holding unit 118 holds workpiece information related to the stacked workpieces 200 (including the workpiece 201 to be grasped). The grip form information holding unit 119 holds grip form information related to the grip form when the hand 121 of the robot 120 is gripped.

  The gripping position recognition unit 112 is based on the loosely stacked workpiece image captured by the imaging device 130 and the workpiece information held in the workpiece information holding unit 118, and the gripping position information related to the gripping position of the workpiece 201 to be gripped. Recognize The environment recognizing unit 113 recognizes the gripping target based on the stacked workpiece image captured by the imaging device 130, the gripping position information recognized by the gripping position recognizing unit 112, and the work information held in the work information holding unit 118. Recognize surrounding information related to the surrounding situation of the work 201. The gripping position recognition unit 112 and the environment recognition unit 113 constitute a recognition unit in the present invention.

  The gripping method evaluation unit 114 includes a plurality of gripping position information recognized by the gripping position recognition unit 112, surrounding information recognized by the environment recognition unit 113, and gripping form information held in the gripping form information holding unit 119. Each gripping method in the gripping method is evaluated. Specifically, in the present embodiment, the gripping method evaluation unit 114 determines each gripping method in a plurality of gripping methods based on the above-described gripping position information, the above-described surrounding information, and the moving direction of the workpiece in the above-described gripping form information. Perform an evaluation.

  The gripping method determination unit 115 performs gripping based on gripping position information recognized by the gripping position recognition unit 112, surrounding information recognized by the environment recognition unit 113, and gripping form information held in the gripping form information holding unit 119. A gripping method for gripping the target workpiece 201 is determined. Specifically, the gripping method determination unit 115 determines a gripping method for gripping the workpiece 201 to be gripped, according to the evaluation value of each gripping method evaluated by the gripping method evaluation unit 114.

  The robot control unit 116 controls the robot 120 and the hand 121 based on the gripping method determined by the gripping method determination unit 115, and controls gripping of the workpiece 201 to be gripped by the hand 121 of the robot 120.

  Next, each internal configuration of the grip control device 110 shown in FIG. 3 will be described in detail.

First, with reference to FIG. 4, a work information holding method held in the work information holding unit 118 and items included in the work information will be described.
FIG. 4 is a diagram illustrating an example of items included in the work information and the work information holding method held in the work information holding unit 118 shown in FIG. 3 according to the first embodiment of this invention.

  As an example of a work information holding method by the work information holding unit 118, the work information is held in an XML (Extensible Markup Language) format as shown in FIG. FIG. 4 shows the workpiece information of the workpiece 201 to be grasped.

  In the work information held in the work information holding unit 118, the work information of one work is described in an item enclosed by a tag <object>. Further, as items included in the tag <object>, a tag <name>, a tag <shape>, and a tag <size> are shown.

  The tag <name> describes name information in the work.

  Also, the tag <shape> describes information on the shape of the workpiece. That is, information on the shape of the work 201 to be grasped is described in the work information tag <shape>. In this embodiment, since the workpiece 201 to be grasped is a rectangular parallelepiped, the shape information is box.

  The tag <size> describes dimension information on the workpiece. Specifically, in the tag <xyz> included in the tag <size>, the dimensions of each side of the rectangular parallelepiped described above are described.

Next, with reference to FIG. 5 and FIG. 6, a holding method of holding form information held by the holding form information holding unit 119 and items included in the holding form information will be described.
FIG. 5 is a diagram illustrating an example of a holding method of holding form information held in the holding form information holding unit 119 shown in FIG. 3 and items included in the holding form information. FIG. 6 is a diagram for explaining the grip form information held in the grip form information holding unit 119 shown in FIG.

First, the definition of the grip form in the present invention will be described.
The gripping form refers to the position / posture of the hand 121 of the robot 120 with respect to the work 201 to be gripped when the hand 201 of the robot 120 is gripped, and the moving direction of the hand 121 of the robot 120.

  As an example of a holding method of holding form information held by the holding form information holding unit 119, the holding form information is held in an XML format as shown in FIG.

  In the grip form information held in the grip form information holding unit 119, one grip form is described in an item enclosed by a tag <grasp>. Furthermore, items included in the tag <grasp> include a tag <name>, a tag <object name>, a tag <xyz>, a tag <rpy>, and a tag <move xyz>.

  The tag <name> describes name information in the gripping form.

  The tag <object name> describes name information in the workpiece 201 to be grasped. In the example illustrated in FIG. 5, work 1 representing the workpiece 201 to be grasped is described.

  The tag <xyz> and the tag <rpy> include the position / posture of the hand coordinate system (right hand system) based on the work coordinate system (right hand system) of the workpiece 201 to be grasped described in <object name>. Information is described. Specifically, the tag <xyz> represents the position of the hand coordinate system based on the work coordinate system, and the tag <rpy> represents the posture of the hand coordinate system based on the work coordinate system. In the example shown in FIG. 5, the tag <xyz> describes three numbers separated by spaces, each corresponding to XYZ of the work coordinate system, and the unit is m. In the example illustrated in FIG. 5, the tag <rpy> also includes three numbers separated by spaces, each representing roll, pitch, and yaw, and the unit is radians.

  Also, in the tag <move xyz>, information on the moving direction of the workpiece with respect to the workpiece coordinate system in the workpiece 201 to be grasped described in <object name> is described. In the example shown in FIG. 5, the movable direction of the workpiece of grasp1 is the Y-axis direction, and the movable direction of the workpiece of grasp2 is the X-axis direction.

  Here, FIG. 6A is a diagram showing the information on the grace1 in FIG. 5, and FIG. 6B is a diagram showing the information on the grace2 in FIG. 5. In FIG. 6, a work coordinate system 601 and a hand coordinate system 602 are defined, and an arrow 603 and an arrow 604 indicate the movable direction of the work, respectively.

Next, the gripping position recognition unit 112 shown in FIG. 3 will be described with reference to FIG.
FIG. 7 is a schematic diagram for explaining recognition processing of gripping position information by the gripping position recognition unit 112 shown in FIG.

  The gripping control device 110 shown in FIG. 7 (the gripping position recognition unit 112 in FIG. 3) is based on the stacked workpiece image captured by the imaging device 130 and the workpiece information held in the workpiece information holding unit 118. The gripping position information related to the gripping position of the workpiece 201 to be gripped is recognized. Here, as a method for recognizing the position / orientation of an object, many methods have been conventionally proposed. For example, the method described in Patent Document 2 may be used.

  Subsequently, the gripping position recognition unit 112 illustrated in FIG. 3 determines whether the robot 120 can generate a path for moving the hand 121 to a grippable position set on the workpiece 201 to be gripped. Here, as a method for calculating a route, RRT (Rapidly expanding Random Trees), which is a general method for searching for a route of a robot, can be used. At this time, the reason for using RRT is that even a complicated route can be searched for in a short time. When the robot 120 can generate a path for moving the hand 121 to the grippable position set for the workpiece 201 to be gripped, the gripping position recognition unit 112 determines the position where the gripperable workpiece 201 can be gripped. 701 is recognized as gripping position information. Then, the grip position recognition unit 112 outputs the grip position information to the environment recognition unit 113 and the grip method evaluation unit 114.

Next, the environment recognition unit 113 shown in FIG. 3 will be described with reference to FIG.
FIG. 8 is a schematic diagram for explaining surrounding information recognition processing by the environment recognition unit 113 shown in FIG. 3.

  When the environment recognition unit 113 takes out the workpiece 201 to be gripped recognized by the gripping position recognition unit 112, the environment recognition unit 113 recognizes an obstacle (such as a surrounding workpiece) around which the workpiece 201 to be gripped is placed. Hereinafter, a method for recognizing surrounding obstacles (such as surrounding workpieces) on which the workpiece 201 to be grasped is placed will be described with reference to FIG.

  FIG. 8 shows an example of a loosely stacked workpiece image obtained by imaging the workpieces 200 stacked in the box 210 by the imaging device 130. In this embodiment, the environment recognition unit 113 holds a bottom surface 801 of the box 210 in advance and captures a bottom image captured before the work 201 to be grasped and the other works 802, 803, 804, and 805 are placed in the box 210. To do. Then, the environment recognition unit 113 calculates the difference between the bulkwork workpiece image and the bottom image. Then, the environment recognition unit 113 masks the part of the workpiece 201 to be gripped whose position and orientation are recognized by the gripping position recognition unit 112 on the basis of the difference image, so that the obstacle around the workpiece 201 to be gripped Recognize arrangement information of other works 802 to 805 which are objects. That is, in the present embodiment, the environment recognition unit 113 recognizes arrangement information such as surrounding workpieces in the workpiece 201 to be grasped as surrounding information related to the surrounding situation of the workpiece 201 to be grasped.

  Next, the environment recognition unit 113 calculates a contact point 806 between the workpiece 201 to be grasped and other workpieces 802, 803, 804, and 805 that are surrounding obstacles. Here, the contact point 806 between the workpiece 201 to be gripped in FIG. 8 and the other workpieces 802 to 805 which are obstacles in the vicinity is a representative point, and the point of the same shape shown in FIG. This is a contact point between the workpiece 201 to be gripped and other workpieces 802 to 805 as surrounding obstacles. In the present embodiment, the environment recognition unit 113 is a contact point between the workpiece 201 to be held and other workpieces 802 to 805 that are surrounding obstacles as the surrounding information regarding the surrounding situation of the workpiece 201 to be grasped. It is also possible to apply a mode for recognizing information 806.

Next, the gripping method evaluation unit 114 shown in FIG. 3 will be described with reference to FIG.
FIG. 9 is a schematic diagram for explaining evaluation processing of each gripping method by the gripping method evaluation unit 114 shown in FIG.

  As described above, the gripping method evaluation unit 114 includes the gripping position information recognized by the gripping position recognition unit 112, the surrounding information recognized by the environment recognition unit 113, and the gripping form information held in the gripping form information holding unit 119. Based on the above, each gripping method in a plurality of gripping methods is evaluated. Here, in the present invention, the gripping method means a combination of a gripping position and a gripping form.

  FIG. 9A and FIG. 9B are each a stacked workpiece image obtained by imaging the workpiece 200 stacked in the box 210 using the imaging device 130. 9A and 9B show a state in which the two grip forms input from the grip form information holding unit 119 are applied to the grip position information recognized by the grip position recognition unit 112, respectively. Is shown.

  Moreover, FIG.9 (c) and FIG.9 (d) are the figures which show the evaluation method of the holding | grip method in FIG.9 (a) and FIG.9 (b), respectively. In FIG. 9C and FIG. 9D, a contact point 806 is a contact point between the workpiece 201 to be gripped calculated by the environment recognition unit 113 and another workpiece that is an obstacle around. 9C and 9D show an evaluation range 901 and an evaluation range 902 that are set when an evaluation is performed by the grasping method evaluation unit 114, respectively. , Defined for each gripping method. In addition, in the example shown in FIGS. 9C and 9D, the evaluation range 901 and the evaluation range 902 are the gripping target workpiece 201 that exists in the movable direction of the gripping target workpiece 201 for each gripping method. Is set to the face.

  Next, the gripping method evaluation unit 114 sets the reciprocal of the sum of the numbers of contact points 806 belonging to the evaluation range 901 and the evaluation range 902 as the evaluation value of the gripping method related to the evaluation range. Specifically, the evaluation value V is expressed by the following equation (1). In the following equation (1), “p” is the number of contact points 806 in the evaluation range.

  Here, the evaluation value in the case of FIG. 9C is V = 1/2, and the evaluation value in the case of FIG. 9D is V = 1/5. That is, the evaluation value is higher when the number of contact points 806 existing in the evaluation range 901 and the evaluation range 902 is smaller.

  Then, the gripping method evaluation unit 114 outputs the calculated evaluation value of each gripping method to the gripping method determination unit 115.

  Then, the gripping method determination unit 115 determines a gripping method based on the evaluation value input from the gripping method evaluation unit 114. Specifically, in the present embodiment, the gripping method determination unit 115 determines the gripping method having a high evaluation value input from the gripping method evaluation unit 114 as the gripping method when actually gripping the workpiece 201 to be gripped. To do. In the example shown in FIG. 9, the gripping method of FIG. 9A is determined as a gripping method when actually gripping the workpiece 201 to be gripped.

  Then, the gripping method determination unit 115 outputs information on the determined gripping method to the robot control unit 116.

  Then, the robot control unit 116 controls the robot 120 and the hand 121 based on the gripping method input from the gripping method determination unit 115, and controls gripping of the workpiece 201 to be gripped by the hand 121 of the robot 120.

Next, the processing procedure of the grip control method by the grip control device 110 according to the present embodiment will be described with reference to FIG.
FIG. 10 is a flowchart illustrating an example of a processing procedure of a grip control method by the grip control device 110 according to the first embodiment of the present invention.

  First, in step S <b> 1001, the gripping position recognition unit 112 recognizes the workpiece 201 to be gripped based on the stacked workpiece image input from the imaging device 130 and the workpiece information held in the workpiece information holding unit 118. To do.

  Subsequently, in step S <b> 1002, the gripping position recognition unit 112 determines that the hand 121 is a gripping target based on the stacked workpiece image input from the imaging device 130 and the workpiece information held in the workpiece information holding unit 118. A position where the workpiece 201 can be gripped is recognized. In other words, the gripping position recognition unit 112 recognizes gripping position information related to the gripping position of the workpiece 201 to be gripped.

  Subsequently, in step S <b> 1003, the environment recognition unit 113 is based on the stacked workpiece image input from the imaging device 130, the grip position information recognized in step S <b> 1002, and the workpiece information held in the workpiece information holding unit 118. Then, the surrounding information related to the surrounding state of the workpiece 201 to be grasped is recognized.

  Subsequently, in step S1004, the gripping method evaluation unit 114 is based on the gripping position information recognized in step S1002, the surrounding information recognized in step S1003, and the gripping form information held in the gripping form information holding part 119. Evaluation of each gripping method in a plurality of gripping methods is performed, and the above-described evaluation value is calculated for each gripping method.

  In step S1005, the gripping method determination unit 115 first determines the gripping method having the highest evaluation value among the gripping methods evaluated in step S114 as the gripping method when actually gripping the workpiece 201 to be gripped. (Gripping method determination step). Next, the robot control unit 116 controls the robot 120 and the hand 121 based on the gripping method input from the gripping method determination unit 115, and the gripping method input from the gripping method determination unit 115 to the hand 121 of the robot 120. To grip the workpiece 201 to be gripped.

  Subsequently, in step S <b> 1006, the control unit 111 determines whether or not the conveyance of the workpiece 201 to be grasped by the hand 121 of the robot 120 has been completed.

As a result of the determination in step S1006, when the conveyance of the workpiece 201 to be grasped by the hand 121 of the robot 120 is not completed (S1006 / NO), the process proceeds to step S1007.
In step S1007, the robot control unit 116 controls the robot 120 and the hand 121, and performs a transfer process of the workpiece 201 to be grasped. Thereafter, the process returns to step S1006.

  On the other hand, as a result of the determination in step S1006, when the conveyance of the workpiece 201 to be grasped by the hand 121 of the robot 120 is completed (S1006 / YES), the processing of the flowchart in FIG.

In this embodiment, gripping position information related to the gripping position of the workpiece 201 to be gripped, surrounding information related to the surrounding state of the workpiece 201 to be gripped, and gripping mode information related to the gripping form of the hand 121 of the robot 120 (for example, The gripping method for gripping the workpiece 201 to be gripped is determined based on the information on the moving direction of the workpiece 201 to be gripped in the gripping form of the hand 121.
According to this configuration, when the gripped work is taken out from the box or the like, the influence of the gripped work on other workpieces can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  In the above-described first embodiment, the description has been given of the form including the information on the shape of the stacked workpiece 200 (including the workpiece 201 to be grasped) as the workpiece information held in the workpiece information holding unit 118. In the second embodiment, as the work information held in the work information holding unit 118, in addition to the shape information described above, the slipperiness information, the breakability information, the entanglement in the workpiece 201 to be gripped, and the like. Information on ease and information on flexibility are included. Then, by using the above-described workpiece information held in the workpiece information holding unit 118, a gripping method for gripping the workpiece 201 to be gripped is determined, so that the gripped workpiece and other workpieces are dropped or damaged. It is possible to reduce the possibility of occurrence of defects.

  The schematic configuration of the grip control system according to the second embodiment is the same as the schematic configuration of the grip control system 100 according to the first embodiment shown in FIG. The internal configuration of the grip control device according to the second embodiment is the same as the internal configuration of the grip control device 110 according to the first embodiment shown in FIG.

  The internal configuration of the grip control device 110 according to the second embodiment is as shown in FIG. 3, but the first embodiment differs from the content of the work information held in the work information holding unit 118 and the gripping. Since the processing contents of the method evaluation unit 114 are different, the different parts will be described below. Further, description of other parts other than the different parts will be omitted.

The work information holding method held in the work information holding unit 118 according to the present embodiment and items included in the work information will be described with reference to FIGS. 11 and 12.
FIG. 11 is a diagram illustrating an example of items included in the work information and the work information holding method held in the work information holding unit 118 shown in FIG. 3 according to the second embodiment of this invention. FIG. 12 is a schematic diagram for explaining the work information shown in FIG.

  As shown in FIG. 11, the work information holding unit 118 of the present embodiment includes a tag <plane>, a tag <slippersiness>, in addition to the work information items in the first embodiment described with reference to FIG. Work information including items of a tag <flimcesses>, a tag <complementess>, and a tag <flexibleness> is held.

The tag <plane> is a tag representing one surface constituting the shape of the workpiece 201 to be grasped. This tag <plane> will be described with reference to FIG.
Surfaces 1201, 1202, 1203, and 1204 illustrated in FIG. 12 are surfaces constituting the workpiece 201 to be grasped, respectively. The planes 1201, 1202, 1203, and 1204 illustrated in FIG. 12 correspond to the plane_1, plane_2, plane_3, and plane_4 of the tag <plane> illustrated in FIG. 11, respectively. Further, the tag <plane> has a tag representing the state of the surface of the workpiece 201 to be grasped, as shown in FIG. The state of the workpiece surface that each tag means will be described below.

  The tag <slippersiness> indicates slipperiness information, the tag <flimsiness> indicates fragility information, the tag <complementess> indicates entanglement information, and the tag <flexibleness> indicates flexibility information . The value set for each tag will be described later in the description of the gripping method evaluation unit 114.

Next, the gripping method evaluation unit 114 in this embodiment will be described.
The gripping method evaluation unit 114 of the present embodiment, for the evaluation value calculation in the gripping method evaluation unit 114 of the first embodiment, information on the ease of slipping, information on the fragility of the workpiece 201 to be gripped, It further takes into account the information on the ease of entanglement and the information on flexibility. Specifically, the evaluation value V is expressed by the following equation (2).

  This evaluation value V is calculated for each gripping method. In this equation (2), “slp” is a coefficient representing slipperiness, “flm” is a coefficient representing breakability, “com” is a coefficient representing ease of entanglement, and “flx” is a coefficient representing flexibility. is there. Also, in this equation (2), the variable followed by v after each coefficient name is the value given to the tag <slippersiness>, tag <flimnessness>, tag <complementess>, and tag <flexibleness> in FIG. It is. This can set slipperiness, fragility, entanglement, and flexibility for each surface of the workpiece 201 to be gripped. In the equation (2), the variable in which n follows each coefficient name is the number of contact points described with reference to FIG. In the equation (2), “other” is the number of contact points with respect to a surface where no tag representing the surface state exists.

  In the present embodiment, as the work information held in the work information holding unit 118, in addition to the shape information described in the first embodiment, the slipperiness information, the breakability information, and the entanglement information And information on flexibility. Then, by using the above-described work information held in the work information holding unit 118 to determine a gripping method when gripping the work to be gripped, a problem such as dropping or breakage of the gripped work and other work Can reduce the possibility of occurrence.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  In the first embodiment described above, as the surrounding information recognized by the environment recognition unit 113, the arrangement information of the surrounding workpieces in the workpiece 201 to be grasped and the information on the contact points between the holding workpiece 201 and the surrounding workpieces, etc. It was a form to apply. In the third embodiment, as the surrounding information recognized by the environment recognition unit 113, the gripping target work 201 is detected using information on the force acting on the gripping target work 201 from the surrounding work or the hand 121 of the robot 120. This is a form in which information on force generated by movement is applied. Then, by using the above-described surrounding information recognized by the environment recognition unit 113, a gripping method for gripping the gripping workpiece 201 is determined, so that the gripped workpiece can be moved when the gripping target workpiece 201 is taken out. By doing so, the influence of the gripped workpiece on other workpieces can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

  The schematic configuration of the grip control system according to the third embodiment is the same as the schematic configuration of the grip control system 100 according to the first embodiment shown in FIG. The internal configuration of the grip control device according to the third embodiment is the same as the internal configuration of the grip control device 110 according to the first embodiment shown in FIG.

  The internal configuration of the grip control device 110 according to the third embodiment is as shown in FIG. 3, but the first embodiment differs from the processing content of the environment recognition unit 113 and the processing of the gripping method evaluation unit 114. Since the contents are different, the different parts will be described below. Further, description of other parts other than the different parts will be omitted.

The processing content of the environment recognition unit 113 and the processing content of the gripping method evaluation unit 114 according to this embodiment will be described with reference to FIG.
FIG. 13 is a schematic diagram for explaining the processing content of the environment recognition unit 113 and the processing content of the grasping method evaluation unit 114 shown in FIG. 3 according to the third embodiment of the present invention. Here, FIG. 13A and FIG. 13B each show a state where the hand 121 of the robot 120 is gripping the workpiece 201 to be gripped by different gripping methods.

  13A and 13B, the environment recognition unit 113 of this embodiment recognizes another workpiece 804 that is an obstacle around the workpiece 201 to be grasped. Furthermore, the environment recognition unit 113 of this embodiment calculates and recognizes information on the force 1301 generated at the contact point 806 as ambient information.

  Next, a method for calculating an evaluation value by the gripping method evaluation unit 114 of this embodiment will be described. Specifically, in the present embodiment, the gripping method evaluation unit 114 calculates the evaluation value V using the following expression (3). The evaluation value V expressed by the equation (3) indicates the reciprocal of the force generated in the hand 121 by the force acting on the workpiece 201 to be grasped from the surrounding workpiece.

  In this equation (3), “free” is a coefficient determined by the degree of freedom of each gripping method in FIGS. 13 (a) and 13 (b). In FIG. 13A, a force is applied in the direction in which the workpiece 201 to be gripped moves, so the value of free is a value lower than 1. For example, in the gripping method of FIG. 13A, the value of free is set to 0.4. In FIG. 13B, the force is applied in a direction different from the direction in which the workpiece 201 to be gripped moves, so the value of free is 1. In the equation (3), “r” is from contact points 1304 and 1305 to the hand 121 to the contact point 806 in the gripping method of the hand 121 in FIGS. 13A and 13B. Is the length of In the formula (3), “F” is a force F generated at the contact point 806. In the equation (3), “θ” is an angle formed by r and a direction perpendicular to the direction of the force F.

  Further, in the present embodiment, by actually moving the hand 121 minutely as in the directions 1306 and 1307 shown in FIG. A force may be applied to the workpiece 201 and the generated force may be measured and evaluated. That is, in this case, the environment recognizing unit 113 recognizes force information generated by moving the workpiece 201 to be grasped using the hand 121 of the robot 120 as surrounding information. When this mode is adopted, assuming that the force generated when the hand 121 is slightly operated is F, the gripping method evaluation unit 114 calculates the evaluation value V using the following equation (4).

  In this embodiment, as the surrounding information recognized by the environment recognition unit 113, the gripping target workpiece 201 is moved using the information on the force acting on the gripping target workpiece 201 from the surrounding workpiece or the hand 121 of the robot 120. The information on the force generated by this is applied. Then, by determining the gripping method when gripping the workpiece 201 to be gripped using this surrounding information, the gripped workpiece is moved when the gripped workpiece 201 is taken out, so that the gripped workpiece can be The influence on the workpiece can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.

  In the above-described first embodiment, the description has been given of the form including the information on the shape of the stacked workpiece 200 (including the workpiece 201 to be grasped) as the workpiece information held in the workpiece information holding unit 118. In the fourth embodiment, the workpiece information held in the workpiece information holding unit 118 includes information on the priority of the gripping method in the workpiece 201 to be gripped. Then, a gripping method for gripping the workpiece 201 to be gripped is determined using the above-described workpiece information held in the workpiece information holding unit 118. As a result, the gripped work moves when the gripped work is taken out, so that the influence of the gripped work on other works can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

  The schematic configuration of the grip control system according to the fourth embodiment is the same as the schematic configuration of the grip control system 100 according to the first embodiment shown in FIG. The internal configuration of the grip control device according to the fourth embodiment is the same as the internal configuration of the grip control device 110 according to the first embodiment shown in FIG.

  The internal configuration of the grip control device 110 according to the fourth embodiment is as shown in FIG. 3. The first embodiment is different from the first embodiment in the contents of work information held in the work information holding unit 118 and gripping. Since the processing contents of the method evaluation unit 114 are different, the different parts will be described below. Further, description of other parts other than the different parts will be omitted.

First, the work information holding method held in the work information holding unit 118 of this embodiment and items included in the work information will be described with reference to FIG.
FIG. 14 shows a fourth embodiment of the present invention, and is a diagram showing an example of a work information holding method held in the work information holding unit 118 shown in FIG. 3 and items included in the work information.

  Unlike the work information in the first embodiment described with reference to FIG. 4, the work information in the work information holding unit 118 of the present embodiment represents a three-dimensional shape in the tag <shape> illustrated in FIG. 14. The file name of the file saved in the data format is specified. By specifying the shape of the workpiece in this way, the workpiece information of the workpiece having an arbitrary shape can be held.

  In addition, as shown in FIG. 14, the work information holding unit 118 of this embodiment holds work information including an item of the tag <priority>. This tag <priority> indicates the priority information of the gripping method. In the example shown in FIG. 14, the smaller the value indicated, the higher the priority of the gripping method.

  Further, as shown in FIG. 14, the tag <priority> includes a tag <xyz> indicating position information that can be gripped and a tag <grasp_name> indicating gripping form information. At this time, the position of the work coordinate system is described in the tag <xyz>, and the name given to the gripping form information held by the gripping form information holding unit 119 is written in the tag <grasp_name>.

Next, the gripping method evaluation unit 114 of this embodiment will be described with reference to FIG.
FIG. 15 is a schematic diagram for explaining the processing content of the grasping method evaluation unit 114 shown in FIG. 3 according to the fourth embodiment of the present invention. Here, FIGS. 15A and 15B show a state in which the hand 121 of the robot 120 is gripping the workpiece 1501 to be gripped by different gripping methods.

  In addition, in the gripping methods of FIGS. 15A and 15B, the priority information is held by the work information holding unit 118, respectively. In the work information shown in FIG. 14, the work information holding unit 118 sets the priority of the gripping method shown in FIG. 15B to 2 and sets the priority of the gripping method shown in FIG. That is, in this example, the gripping method shown in FIG. 15A is set to have a higher priority than the gripping method shown in FIG.

Here, the reason why the holding method shown in FIG. 15A has higher priority than the holding method shown in FIG.
The workpiece 1501 to be grasped is L-shaped as shown in FIG. When a force is generated by another workpiece on the L-shaped protrusion 1502 of the workpiece 1501 to be grasped, the workpiece 1501 to be grasped can be moved by the grasping method shown in FIG. On the other hand, with the gripping method shown in FIG. 15B, the workpiece 1501 to be gripped cannot be moved. That is, when a force is applied to a gripping target workpiece 1501 in a certain direction, the hand 121 of the robot 120 grips the gripping target workpiece 1501 so that the gripping target workpiece 1501 can operate in the direction of the force. To do. By gripping in this way, a force applied to the workpiece 1501 to be gripped can be released, and the influence on surrounding objects (including surrounding workpieces) can be reduced.

  In the present embodiment, the gripping method evaluation unit 114 assigns a high evaluation value to the gripping method having a high priority and outputs it to the gripping method determination unit 115.

  In the present embodiment, the workpiece information held in the workpiece information holding unit 118 includes information on the priority of the gripping method in the workpiece 201 to be gripped. Then, using the above-described work information held in the work information holding unit 118, the gripping method evaluation unit 114 evaluates the gripping method, and determines the gripping method when gripping the workpiece to be gripped based on the evaluation result. Like to do. Thereby, when the workpiece gripped when the workpiece 1501 to be gripped is taken out, the influence of the gripped workpiece on other workpieces can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described.

  In the first embodiment described above, as the surrounding information recognized by the environment recognition unit 113, the arrangement information of the surrounding workpieces in the workpiece 201 to be grasped and the information on the contact points between the holding workpiece 201 and the surrounding workpieces, etc. It was a form to apply. In the fifth embodiment, as the surrounding information recognized by the environment recognition unit 113, information on the force acting on the gripping target workpiece 201 from the surrounding workpiece is applied to the gripping target workpiece 201 when the gripping target workpiece 201 is taken out after gripping. It is a form. Then, the gripping method determination unit 115 determines a gripping method when gripping the workpiece 201 to be gripped, using the above-described surrounding information recognized by the environment recognition unit 113. Then, the robot controller 116 determines the position and orientation of the hand 121 so that the resistance becomes the smallest with respect to the direction of the force applied by the hand 121 to the gripping work 201 after the hand 121 grips the gripping work 201. To control. Thereby, when taking out the workpiece 201 to be gripped, the influence of the workpiece 201 to be gripped on other workpieces can be reduced. As a result, it is possible to reduce the possibility of problems such as dropping or breakage of the gripped workpiece and other workpieces.

  The schematic configuration of the grip control system according to the fifth embodiment is the same as the schematic configuration of the grip control system 100 according to the first embodiment shown in FIG. Further, the internal configuration of the grip control device according to the fifth embodiment is the same as the internal configuration of the grip control device 110 according to the first embodiment shown in FIG.

  The internal configuration of the grip control device 110 according to the fifth embodiment is as shown in FIG. 3, but the first embodiment differs from the processing content of the environment recognition unit 113 and the processing of the gripping method evaluation unit 114. Since the contents are different, the different parts will be described below. Further, description of other parts other than the different parts will be omitted.

The processing content of the environment recognition unit 113 and the processing content of the gripping method evaluation unit 114 according to the present embodiment will be described with reference to FIG.
FIG. 16 is a schematic diagram for explaining the processing content of the environment recognition unit 113 and the processing content of the gripping method evaluation unit 114 shown in FIG. 3 according to the fifth embodiment of the present invention. Here, FIG. 16A and FIG. 16B each show a state in which the hand 121 of the robot 120 is gripping the workpiece 201 to be gripped by different gripping methods.

  In FIG. 16A and FIG. 16B, the environment recognition unit 113 according to the present embodiment recognizes another workpiece 1605 that is an obstacle around the workpiece 201 to be grasped. Furthermore, the environment recognition unit 113 according to the present embodiment outputs information on the force acting from another work 1605 that is an obstacle around the work 201 to be grasped when the work 201 to be grasped is taken out after grasping. Calculate and recognize as information.

The processing content of this embodiment is demonstrated concretely.
The transport path 1601 and the transport path 1602 in FIG. 16A and the transport path 1603 and the transport path 1604 in FIG. 16B are respectively the gripping method in FIG. 16A and the transport path in FIG. This is a conveyance path by the hand 121 set in advance for the gripping method.

  Here, the environment recognizing unit 113 simulates the operations of the transport path 1601, the transport path 1602, the transport path 1603, and the transport path 1604, and chronologically represents the force that the surrounding obstacles generated at that time act on the workpiece 201 to be gripped. To calculate. In the present embodiment, the environment recognition unit 113 calculates the force that the surrounding obstacles act on the workpiece 201 to be gripped in time series. For example, the gripping method evaluation unit 114 calculates the force. It may be.

  FIG. 17 shows a fifth embodiment of the present invention, and is a characteristic diagram in which forces acting on a workpiece 201 to be grasped by other workpieces 1605 as surrounding obstacles are recorded in time series. Here, in FIG. 17, the horizontal axis indicates time (seconds), and the vertical axis indicates force (Newton). In FIG. 17, a graph 1701 shows the characteristics of the transport path 1601 in FIG. 16A, a graph 1702 shows the characteristics of the transport path 1602 in FIG. 16A, and a graph 1703 shows the transport of FIG. The characteristic of the path 1603 is shown, and the graph 1704 shows the characteristic of the transport path 1604 in FIG.

  And the grasping method evaluation part 114 of this embodiment calculates the evaluation value V using the following (5) Formula. That is, the evaluation value V is the reciprocal of the maximum value of the time-series force value x.

  In the example illustrated in FIG. 17, the maximum force value 1705 in the graph 1704 is the minimum, and the evaluation value is the highest. That is, in this embodiment, when a force is applied in a certain direction in the transport path after gripping the workpiece 201 to be gripped, the gripping target workpiece 201 can be moved in the direction of the force. The work 201 is gripped. In addition, by gripping in this way, the force applied to the gripping workpiece 201 can be released in the transport path after gripping the gripping workpiece 201, and other workpieces 1605 can be released. The effect on it can be reduced.

  In the present embodiment, as the surrounding information recognized by the environment recognition unit 113, information on the force acting from the surrounding workpiece on the gripping workpiece 201 is applied when the gripping workpiece 201 is taken out after gripping. . The surrounding information is used to determine a gripping method for gripping the workpiece 201 to be gripped. Then, by controlling the position and posture of the hand so that the resistance is the smallest with respect to the direction of the force applied to the workpiece 201 to be gripped after gripping, the gripped workpiece and other workpieces can be removed. It is possible to reduce the possibility of a problem such as breakage.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
This program and a computer-readable storage medium storing the program are included in the present invention.

  Note that the above-described embodiments of the present invention are merely examples of implementation in practicing the present invention, and the technical scope of the present invention should not be construed as being limited thereto. It is. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 100 Grasping control apparatus 110, 111 Control part, 112 Grasping position recognition part, 113 Environment recognition part, 114 Grasping method evaluation part, 115 Grasping method determination part, 116 Robot control part, 117 Holding part, 118 Work information holding part, 119 Gripping Form information holding unit

Claims (17)

A gripping control device for controlling gripping by the robot hand when gripping and taking out a workpiece to be gripped by a robot hand from a plurality of workpieces,
A workpiece information holding unit that holds workpiece information related to the workpiece to be grasped;
A gripping form information holding unit for holding gripping form information related to a gripping form in the robot hand;
Based on the workpiece images of the plurality of workpieces and the workpiece information obtained by the imaging device, the gripping position information related to the gripping position of the gripping target workpiece and the surrounding information related to the surrounding situation of the gripping target workpiece A recognition unit that recognizes
A gripping method determination unit that determines a gripping method when gripping the workpiece to be gripped based on the gripping position information, the surrounding information, and the gripping form information;
A gripping control apparatus comprising: a control unit configured to control gripping of the workpiece to be gripped by the robot hand based on the gripping method determined by the gripping method determination unit. The recognition unit
A gripping position recognition unit that recognizes the gripping position information based on the workpiece image and the workpiece information;
The environment recognition unit for recognizing the surrounding information based on the work image, the grip position information recognized by the grip position recognition unit, and the work information. Item 2. The grip control device according to Item 1. A gripping method evaluation unit that evaluates each gripping method in a plurality of gripping methods based on the gripping position information, the surrounding information, and the gripping form information;
The gripping method determination unit determines a gripping method for gripping the workpiece to be gripped according to an evaluation value of each gripping method evaluated by the gripping method evaluation unit. Or the holding | grip control apparatus of 2.   The grip control apparatus according to claim 3, wherein the gripping method evaluation unit evaluates each gripping method based on a moving direction of a workpiece in the gripping position information, the surrounding information, and the gripping form information. .   The grip control apparatus according to claim 1, wherein the workpiece information is information including information on a shape of the workpiece to be gripped.   The grip control device according to claim 1, wherein the workpiece information is information including information on ease of slipping in the workpiece to be gripped.   The grip control apparatus according to claim 1, wherein the workpiece information is information including information on fragility of the workpiece to be gripped.   The grip control apparatus according to claim 1, wherein the workpiece information is information including information on the ease of entanglement in the workpiece to be gripped.   The grip control apparatus according to claim 1, wherein the workpiece information is information including information on flexibility of the workpiece to be gripped.   The grip control apparatus according to claim 1, wherein the workpiece information is information including information on a priority of a gripping method in the workpiece to be gripped.   The gripping control apparatus according to any one of claims 1 to 10, wherein the surrounding information is arrangement information of surrounding workpieces in the workpiece to be gripped.   The gripping control apparatus according to any one of claims 1 to 10, wherein the surrounding information is information on a point in contact with a surrounding workpiece in the workpiece to be gripped.   The gripping control apparatus according to any one of claims 1 to 10, wherein the surrounding information is information on a force acting from a surrounding workpiece on the workpiece to be gripped.   The grip control apparatus according to claim 1, wherein the surrounding information is information on a force generated by moving the workpiece to be gripped using a hand of the robot.   11. The surrounding information is information on a force acting on the workpiece to be gripped from the surrounding workpiece when the workpiece to be gripped is taken out after gripping. The grip control device according to the item. Work information for controlling gripping by the robot hand when gripping and taking out the work to be gripped by a robot hand from a plurality of workpieces, and holding work information related to the gripped work A gripping control method by a gripping control device including a holding part and a gripping form information holding part that holds gripping form information related to a gripping form in the robot hand,
Based on the workpiece images of the plurality of workpieces and the workpiece information obtained by the imaging device, the gripping position information related to the gripping position of the gripping target workpiece and the surrounding information related to the surrounding situation of the gripping target workpiece A recognition step for recognizing
A gripping method determination step of determining a gripping method when gripping the workpiece to be gripped based on the gripping position information, the surrounding information, and the gripping form information;
And a control step of controlling gripping of the workpiece to be gripped by the robot hand based on the gripping method determined in the gripping method determination step. Work information for controlling gripping by the robot hand when gripping and taking out the work to be gripped by a robot hand from a plurality of workpieces, and holding work information related to the gripped work A program for causing a computer to execute a gripping control method by a gripping control device including a holding part and a gripping form information holding part that holds gripping form information related to a gripping form in the hand of the robot,
Based on the workpiece images of the plurality of workpieces and the workpiece information obtained by the imaging device, the gripping position information related to the gripping position of the gripping target workpiece and the surrounding information related to the surrounding situation of the gripping target workpiece A recognition step for recognizing
A gripping method determination step of determining a gripping method when gripping the workpiece to be gripped based on the gripping position information, the surrounding information, and the gripping form information;
A program for causing a computer to execute a control step of controlling gripping of the workpiece to be gripped by the robot hand based on the gripping method determined in the gripping method determination step.