JP2019181573A - Picking device and method for the same - Google Patents

Abstract

To provide a picking device which can automatically recognize and pick a picking object as a holding object at a distribution site.SOLUTION: A picking device includes: a camera which generates three-dimensional point group information of a plurality of picking objects as a whole loaded in a cardboard box; area determination means (a step S7, a step S12) which determines an area of an individual picking object by finding out a common three-dimensional shape from the plurality of picking objects as a whole based on the generated three-dimensional point group information; positional posture output means (a step S12) which outputs a three-dimensional positional posture of the picking object as a holding object based on the three-dimensional point group information generated by the camera belonging to the area of the picking object determined by the area determination means (the step S7, the step S12); and a robot which holds and transfers the picking object as the holding object based on the three-dimensional positional posture output by the positional posture output means (the step S12).SELECTED DRAWING: Figure 2

Description

  The present invention relates to a picking apparatus and a method thereof.

  In recent years, in the logistics industry, automation of operations such as sorting, loading, and unloading in warehouses has been demanded, and various automated systems have been introduced. As this automation system, various automation techniques using image processing techniques are used. As such a technique, for example, a technique is known in which position information of a picking object is recognized by image processing, and the picking object is gripped and moved using a robot hand based on the recognized position information (for example, Patent Document 1).

JP 2004-188562 A

  However, when using the technique as described above, the three-dimensional shape of the picking object must be created in advance as a database. On the physical distribution site, the number of types of picking objects is thousands to hundreds of thousands. Therefore, there is a problem that it is very difficult to prepare such a huge number of three-dimensional shapes in advance as a database. Therefore, there has been a problem that automation has not yet been realized at the distribution site.

  In view of the above problems, an object of the present invention is to provide a picking apparatus and method for automatically recognizing and picking a picking object to be grasped at a physical distribution site.

  The object of the present invention is achieved by the following means. In addition, although the code | symbol in a parenthesis attaches the referential mark of embodiment mentioned later, this invention is not limited to this.

The picking device according to the first aspect of the present invention includes a plurality of picking objects (for example, the picking object Wa shown in FIG. 1) loaded in a predetermined container (for example, the cardboard box CA shown in FIG. 1) ( W) 3D point cloud information generating means (for example, the camera 4 shown in FIG. 1) for generating 3D point cloud information;
Based on the three-dimensional point group information generated by the three-dimensional point group information generating means (for example, the camera 4 shown in FIG. 1), the plurality of picking objects (for example, the picking object Wa shown in FIG. 1). A region determining means (for example, step S7, step S12 shown in FIG. 2) that finds a common three-dimensional shape from the whole (W) and determines the region of each picking target (for example, picking object Wa shown in FIG. 1). )When,
The three-dimensional point cloud information generating means (for example, belonging to the area of the picking object (for example, picking object Wa shown in FIG. 1) determined by the area determining means (for example, step S7, step S12 shown in FIG. 2)) For example, based on the three-dimensional point cloud information generated by the camera 4) shown in FIG. 1, the position for outputting the three-dimensional position and orientation of the picking target (for example, the picking target Wa shown in FIG. 1) to be grasped Posture output means (for example, step S12 shown in FIG. 2);
Based on the three-dimensional position and orientation output by the position and orientation output means (for example, step S12 shown in FIG. 2), the picking target (for example, the picking target Wa shown in FIG. 1) to be gripped is gripped. And a transfer robot (for example, the robot 2 shown in FIG. 1).

On the other hand, the picking method according to the invention of claim 2 is the picking object (for example, picking object Wa shown in FIG. 1) loaded in a predetermined container (for example, cardboard box CA shown in FIG. 1). Generating the whole (W) three-dimensional point cloud information (see, for example, FIG. 3C);
Based on the generated three-dimensional point cloud information, a common three-dimensional shape is found from the whole (W) of the plurality of picking objects (for example, picking object Wa shown in FIG. 1), and individual picking objects (for example, , A region of the picking object Wa shown in FIG. 1 is determined (for example, step S7 and step S12 shown in FIG. 2),
Based on the generated three-dimensional point cloud information belonging to the area of the determined picking object (for example, picking object Wa shown in FIG. 1), the picking object (for example, the picking object shown in FIG. 1) to be grasped Output the three-dimensional position and orientation of the object Wa) (for example, step S12 shown in FIG. 2);
Based on the output three-dimensional position and orientation, a robot (for example, the robot 2 shown in FIG. 1) is used to grip and move the picking target object (for example, the picking target object Wa shown in FIG. 1). It is characterized by being placed.

  According to a third aspect of the present invention, in the picking method according to the second aspect, an accurate size of the picking object to be grasped (for example, the picking object Wa shown in FIG. 1) is input in advance. It is not necessary.

  Furthermore, according to the invention of claim 4, in the picking method according to claim 2 or 3, based on the generated 3D point cloud information, a discontinuous 3D point cloud in the depth direction (Z direction). By finding information, a common three-dimensional shape is found from the whole (W) of the plurality of picking objects (for example, picking object Wa shown in FIG. 1), and individual picking objects (for example, picking shown in FIG. 1) are found. It is characterized by determining the area of the object Wa).

  Still further, according to the invention of claim 5, in the picking method according to any one of claims 2 to 4, based on the generated three-dimensional point cloud information, in the depth direction (Z direction). Segmentation is performed by finding discontinuous 3D point cloud information (for example, step S4 shown in FIG. 2), and the similarity of the 3D point cloud information between the segmentations is examined using a predetermined algorithm (for example, FIG. Step S5 and Step S8) shown in FIG. 2 are used to find a common three-dimensional shape from the whole (W) of the plurality of picking objects (for example, picking object Wa shown in FIG. 1), and to select individual picking objects (for example, The region of the picking object Wa) shown in FIG. 1 is determined.

  On the other hand, according to a sixth aspect of the present invention, in the picking method according to any one of the second to fifth aspects, the determined picking object (for example, the picking object Wa shown in FIG. 1) is included. A minimum box-shaped body (for example, the minimum rectangular parallelepiped R shown in FIG. 6) is set, and the length (for example, FIG. 6) of the set minimum box-shaped body (for example, the minimum rectangular parallelepiped R shown in FIG. 6) is set. (L) shown in (a), length in the width direction (for example, vertical T shown in FIG. 6 (a)), and height (for example, height H shown in FIG. 6 (a)). It is characterized by.

  According to a seventh aspect of the invention, in the picking method according to the sixth aspect, a length (for example, the longitudinal direction) of the measured minimum box-shaped body (for example, the smallest rectangular parallelepiped R shown in FIG. 6) is measured. The horizontal L) shown in FIG. 6A, the length in the width direction (for example, the vertical T shown in FIG. 6A), and the height (for example, the height H shown in FIG. 6A) are output. It is characterized by becoming.

  On the other hand, according to the invention of claim 8, in the picking method according to any one of claims 2 to 7, a robot (for example, the robot 2 shown in FIG. 1) based on the output three-dimensional position and orientation. When the picking object to be grasped (for example, the picking object Wa shown in FIG. 1) is gripped, the picking object to be grasped (for example, the picking object Wa shown in FIG. 1) is used. It is characterized by measuring weight.

  According to a ninth aspect of the present invention, in the picking method according to the eighth aspect, based on the weight of the measured picking target (for example, the picking target Wa shown in FIG. 1), Using a robot (for example, the robot 2 shown in FIG. 1), the acceleration at the time of transferring the picking target (for example, the picking target Wa shown in FIG. 1) to be grasped is controlled. It is said.

  On the other hand, according to the invention of claim 10, in the picking method according to claim 6, the measured length (for example, the size of horizontal L, vertical T, and height H shown in FIG. 6A). Based on the above, the robot (for example, the robot 2 shown in FIG. 1) is used to set and control a path for transferring the picking target object (for example, the picking target object Wa shown in FIG. 1) to be gripped. It is characterized by.

  According to the invention of claim 11, in the picking method according to claim 6, the measured length (for example, the minimum rectangular parallelepiped R shown in FIG. 6) in the longitudinal direction (for example, the minimum box-like body R). The gripping tool (for example, the robot hand 22 shown in FIG. 1) is used at least using the width L in FIG. 6 (a) and the length in the width direction (for example, the vertical T shown in FIG. 6 (a)). It is characterized by being selected.

  Next, effects of the present invention will be described with reference numerals in the drawings. In addition, although the code | symbol in a parenthesis attaches the referential mark of embodiment mentioned later, this invention is not limited to this.

  According to the invention which concerns on Claim 1 or Claim 2, the picking target object used as the grasping object can be automatically recognized and picked in the physical distribution site. As a result, it is not necessary to prepare all the three-dimensional shapes of picking objects of thousands to hundreds of thousands in advance as a database, and automation can be realized. According to the invention of claim 3, it is not necessary to accurately input the size of the picking object to be grasped.

  Furthermore, as a method of finding a common three-dimensional shape from the whole (W) of a plurality of picking objects (for example, the picking object Wa shown in FIG. 1), as in the invention according to claim 4, the depth direction (Z direction) The method of finding discontinuous three-dimensional point cloud information of (3) is preferable, and more preferably, as in the invention according to claim 5, segmentation is performed by finding discontinuous three-dimensional point cloud information in the depth direction (Z direction). (For example, step S4 shown in FIG. 2), and the similarity of the three-dimensional point cloud information between the segmentations is examined using a predetermined algorithm (for example, step S5 and step S8 shown in FIG. 2).

  On the other hand, according to the sixth aspect of the invention, the minimum box-like body (for example, the smallest rectangular parallelepiped R shown in FIG. 6) containing the determined picking object (for example, the picking object Wa shown in FIG. 1) is provided. The length of the set minimum box-like body (for example, the smallest rectangular parallelepiped R shown in FIG. 6) in the longitudinal direction (for example, the lateral L shown in FIG. 6A) and the length in the width direction (for example, Since the vertical T) and the height (for example, the height H illustrated in FIG. 6A) are measured, the minimum box-shaped body (for example, the minimum rectangular parallelepiped R illustrated in FIG. 6) is measured. Can be set automatically and measured automatically.

  Moreover, according to the invention which concerns on Claim 7, the length (for example, horizontal L shown to Fig.6 (a)) of the measured minimum box-like body (for example, the minimum rectangular parallelepiped R shown in FIG. 6) in the longitudinal direction) And the length in the width direction (for example, the vertical T shown in FIG. 6A) and the height (for example, the height H shown in FIG. 6A) are output accurately. The picking object Wa) shown in FIG. 1 can be transferred.

  On the other hand, according to the invention according to claim 8, based on the output three-dimensional position and orientation, using a robot (for example, the robot 2 illustrated in FIG. 1), a picking target (for example, illustrated in FIG. 1) to be grasped. When the picking object Wa) is gripped, the weight of the picking object (for example, the picking object Wa shown in FIG. 1) as the gripping object is measured, so that the picking object (for example, FIG. The weight of the picking object Wa) shown can be measured.

  According to the ninth aspect of the present invention, the robot (for example, the robot 2 shown in FIG. 1) is moved based on the weight of the measured picking target (for example, the picking object Wa shown in FIG. 1). Since the acceleration at the time of transferring the picking target object (for example, the picking target object Wa shown in FIG. 1) is controlled using the picking target object (for example, shown in FIG. 1). It is possible to reduce a situation in which the picking object Wa) is dropped while being transferred to a predetermined location (for example, a conveyor).

  On the other hand, according to the invention of claim 10, based on the measured length (for example, the size of horizontal L, vertical T, and height H shown in FIG. 6A), the robot (for example, FIG. The robot 2) shown in FIG. 1 is used to set and control a path for transferring a picking object to be grasped (for example, the picking object Wa shown in FIG. 1). For example, the picking object Wa) shown in FIG. 1 can be transferred to a predetermined location (for example, a conveyor).

  According to the invention of claim 11, the measured length of the smallest box (for example, the smallest rectangular parallelepiped R shown in FIG. 6) in the longitudinal direction (for example, the lateral L shown in FIG. 6A) and Since the gripping tool (for example, the robot hand 22 illustrated in FIG. 1) is selected using at least the length in the width direction (for example, the vertical T illustrated in FIG. 6A), an optimal gripping tool is selected. Can be selected automatically.

1 is a schematic overall view of a picking apparatus according to an embodiment of the present invention. It is a flowchart figure which shows the control procedure of the picking apparatus which concerns on the same embodiment. (A) is the perspective view which looked at the picking target object concerning the embodiment from the upper part, (b) was provided with the three-dimensional coordinates of the X direction, the Y direction, and the Z direction in the picking target object shown in (a). The perspective view which shows the state as which the three-dimensional point cloud information was represented, (c) is equipped with the three-dimensional coordinate of a X direction, a Y direction, and a Z direction in the whole several picking target object loaded adjacently. It is a perspective view which shows the state by which the three-dimensional point cloud information represented. (A) is explanatory drawing which shows the state which cut out area | regions A and B from the three-dimensional point cloud information shown in FIG.3 (c), (b) is an area | region from the three-dimensional point cloud information shown in FIG.3 (c). Explanatory drawing which shows the state which cut out C, (c) is explanatory drawing which shows the state which cut out the area | region D from the three-dimensional point cloud information shown in FIG.3 (c). FIG. 4 is an explanatory diagram showing a state where a grippable region group is determined based on the three-dimensional point group information shown in FIG. (A) is explanatory drawing explaining having set the minimum rectangular parallelepiped which includes the determined single item point group, (b) is the state which match | combined the minimum rectangular parallelepiped shown to (a) for every grippable area group It is explanatory drawing which shows.

  Hereinafter, an embodiment of a picking apparatus according to the present invention will be specifically described with reference to the drawings. In addition, in the following description, when showing the direction of up, down, left and right, it means up, down, left and right when viewed from the front of the figure.

  As shown in FIG. 1, the picking device 1 includes a robot 2, a robot control device 3 that controls the robot 2, a camera 4, and an image processing device 5. Note that the symbol W indicates the entire plurality of picking objects Wa loaded in the cardboard box CA.

  The robot 2 is attached to the base 20 fixed to an installation surface such as a floor or a wall, a robot arm unit 21 whose base end is rotatably connected to the base 20, and a tip of the robot arm unit 21. And a robot hand 22 that can grip the object Wa by suction or pinching. The robot hand 22 is provided with a weight sensor such as a strain gauge for measuring the weight of the picking object Wa when the picking object Wa is gripped. 3 is output. Moreover, it replaces with a weight sensor, the electric current required for the motor drive of the robot arm part 21 with which the base end was rotatably connected can also be detected, and the weight of the picking target object Wa can also be estimated from the detection information. . In other words, the difference between the torque of the unloaded motor when the robot arm unit 21 whose base end is rotatably connected in the same posture is stopped and the torque of the motor when the picking object Wa is gripped is calculated. It is only necessary to estimate the weight of the picking object Wa. The estimated weight information is output to the robot control device 3 as is the case with the weight sensor measurement information.

  On the other hand, the robot control device 3 controls the operation of the robot 2 based on data output from the image processing device 5 or data output from the robot 2.

  The camera 4 can acquire three-dimensional point group information indicating the three-dimensional coordinates of the plurality of picking objects Wa whole W loaded in the cardboard box CA.

  The image processing apparatus 5 can output predetermined data to the outside of the image processing apparatus 5 and an input unit 51 that can input predetermined data to the image processing apparatus 5 from the outside with a CPU 50, a mouse, a keyboard, a touch panel, or the like. An output unit 52, a ROM 53 composed of a writable flash ROM or the like storing a predetermined program, a RAM 54 functioning as a work area or a buffer memory, a display unit 55 composed of an LCD (Liquid Crystal Display), etc. It consists of

  Thus, when using the picking apparatus as described above, the operator uses the input unit 51 of the image processing apparatus 5 shown in FIG. 1 to instruct to start the program stored in the ROM 53 shown in FIG. To do. As a result, the CPU 50 (see FIG. 1) of the image processing apparatus 5 performs processing as shown in FIG. Hereinafter, a description will be given with reference to FIG. Note that the processing content of the program shown in FIG. 2 is merely an example, and the present invention is not limited to this.

First, the CPU 50 (see FIG. 1) acquires three-dimensional point group information indicating the three-dimensional coordinates of the entire plurality of picking objects Wa loaded in the cardboard box CA acquired by the camera 4 (step). S1). More specifically, when the picking object Wa is a container made of a push (push-down) type pump dispenser as shown in FIG. 3 (a), the camera 4 uses a cardboard as shown in FIG. When the entire picking object Wa loaded in the box CA is imaged from above, the 3D point cloud information indicating the three-dimensional coordinates of the picking object Wa acquired by the camera 4 is shown in FIG. ), As a plurality of three-dimensional point groups TG represented by three-dimensional coordinates in the X direction, Y direction, and Z direction, spaced apart from the upper side of the pump dispenser of the picking object Wa, the lower side of the pump dispenser, That is, it is expressed as a discontinuous thing. However, the three-dimensional point cloud information indicating the entire plurality of picking objects Wa loaded in the cardboard box CA has three-dimensional coordinates in the X, Y, and Z directions as shown in FIG. It is expressed as a plurality of three-dimensional point groups TG expressed by Each of the plurality of three-dimensional point groups TG includes three-dimensional coordinates (X _n , Y _n , Z _n ) (where n ≧ 1) in the X direction, the Y direction, and the Z direction. In addition, the CPU 50 (see FIG. 1) temporarily stores in the RAM 54 three-dimensional point cloud information indicating the entire plurality of picking objects Wa loaded in the cardboard box CA.

Next, the CPU 50 (see FIG. 1) reads out the three-dimensional point group information indicating the three-dimensional coordinates of the entire plurality of picking objects Wa loaded in the cardboard box CA stored in the RAM 54, in the depth direction. As shown in FIG. 4, regions A, B, and C are obtained from this three-dimensional point cloud information on the basis of a depth edge where the change of A exceeds a certain amount or the change of the normal direction exceeds a certain value. , D are cut out (step S2). More specifically, the CPU 50 (see FIG. 1) determines that the camera 50 determines the camera from the three-dimensional point group information indicating the three-dimensional coordinates of the entire picking object Wa loaded in the cardboard box CA stored in the RAM 54. 4 is a point group that is determined to have a short distance from 4, and the positions of adjacent points (X _n , Y _n , Z _n ) (an integer of n ≧ 1) are within a predetermined fixed distance. Yes, an area that can be regarded as a lump is cut out as area A as shown in FIG. Next, the CPU 50 (see FIG. 1) is an area where it can be determined that the distance in the Z direction is away from the area A, and the positions (X _n , Y _n , Z _n ) (n ≧ 1) of adjacent points. An area that is within a predetermined fixed distance and can be regarded as a lump is cut out as area B as shown in FIG. Next, the CPU 50 (see FIG. 1) is an area where it can be determined that the distance in the Y direction is away from the area A, and the positions (X _n , Y _n , Z _n ) (n ≧ 1) of adjacent points. An area that is within a predetermined fixed distance and can be regarded as a lump is cut out as an area C as shown in FIG. Next, the CPU 50 (see FIG. 1) is an area where it can be determined that the distance in the X direction is away from the area A, and the positions (X _n , Y _n , Z _n ) (n ≧ 1) of adjacent points. An area that is within a predetermined fixed distance and can be regarded as a lump is cut out as an area D as shown in FIG. Thus, in this way, the CPU 50 (see FIG. 1) allows the three-dimensional point group indicating the three-dimensional coordinates of the entire plurality of picking objects Wa loaded in the cardboard box CA read from the RAM 54. The areas A, B, C, and D are cut out from the information.

  Next, the CPU 50 (see FIG. 1) determines a grippable area group from the cut areas A, B, C, and D (step S3). More specifically, the CPU 50 (see FIG. 1) determines each area from the three-dimensional point cloud information of the extracted areas A, B, C, and D in order to select an upper part necessary for gripping the picking object Wa. A region group in which the position in the Z direction of each of A, B, C, and D is within a predetermined fixed range and the region A, B, C, and D is above is selected. As a result, the region B is excluded, and the grippable region groups HG1 to HG15 as shown in FIG. 5 are determined.

  Next, the CPU 50 (see FIG. 1) selects a grippable area group close to the reference point from the determined grippable area groups HG1 to HG15 and sets it as a temporary single item point group (step S4). More specifically, if this reference point is set in advance, for example, with the upper left corner shown in FIG. 5 as the reference point O, the CPU 50 (see FIG. 1) can obtain the three-dimensional point group information belonging to the grippable region groups HG1 to HG15. Therefore, it can be calculated that the grippable region group having the shortest distance from the reference point O is the grippable region group HG1. Thereby, CPU50 (refer FIG. 1) determines this holdable area | region group HG1 as a temporary single item group.

  Next, the CPU 50 (see FIG. 1) calculates the similarity between the determined provisional single item point group (gripable region group HG1) and the other grippable region groups HG2 to HG15 (step S5). More specifically, as a method for calculating the similarity, an algorithm composed of a conventionally known method, for example, a technique of OSADA et al. (Shape Distributions, ACM Transactions on Graphics, Vol. 21, No. 4, October 2002, Pages 807-832). .)), The CPU 50 (see FIG. 1) calculates the similarity between the temporary single item point group (gripable region group HG1) and the other graspable region groups HG2 to HG15. Specifically, since the technique of OSADA et al. Calculates the degree of difference, the CPU 50 (see FIG. 1) determines whether the temporary single item point group (gripable area group HG1) and another grippable area group HG2 The degree of difference is calculated, and the degree of similarity is calculated as 1-difference. Similarly, the degree of difference between the temporary single item point group (gripable area group HG1) and another graspable area group HG3 is calculated. , 1−difference, the similarity is calculated, and similarly, the dissimilarity between the temporary single item point group (gripable region group HG1) and the other grippable region group HG4 is calculated. Thus, the similarity is calculated, and so on.

  Next, the CPU 50 (see FIG. 1) calculates whether or not the ratio of the calculated similarity as a whole is a predetermined amount (for example, 80%) (step S6). If there is a predetermined amount (step S6: YES), the CPU 50 (see FIG. 1) determines the determined temporary single item point group (gripable region group HG1) as a single item point group (gripable region group HG1) (step S7). ).

  On the other hand, if there is no predetermined amount (step S6: NO), it is assumed that a plurality of picking objects Wa loaded in the cardboard box CA shown in FIG. 1) confirms the calculated degree of difference (step S8). At this time, the degree of difference is, for example, two or more point groups of 0.09 or less (for example, the degree of difference between the provisional single item point group (gripable region group HG1) and the other graspable region group HG2, temporary single item) If the degree of difference between the point group (gripable region group HG1) and other grippable region group HG3 is 0.09 or less, there are two or more groups of the grippable region group HG2 and the grippable region group HG3. If so (step S8: YES), the CPU 50 (see FIG. 1) determines the determined temporary single item point group (gripable region group HG1) as a single item point group (gripable region group HG1) ( Step S7).

  On the other hand, for example, if there are not two or more point groups having a degree of difference of 0.09 or less (step S8: NO), the CPU 50 (see FIG. 1) causes the display unit 55 (see FIG. 1) to display a warning. (Step S9), and the processing of the program ends.

  Next, the CPU 50 (see FIG. 1) sets a minimum rectangular parallelepiped as shown in FIG. 6A that includes the single item point group (gripable region group HG1) (step S10). More specifically, as shown in FIG. 6A, the single item point group (gripable region group HG1) includes a three-dimensional point group TG indicating three-dimensional coordinates. From the information, the CPU 50 (see FIG. 1) can set the minimum rectangular parallelepiped R as shown in FIG. Thereby, CPU50 (refer FIG. 1) can measure the size of the vertical T, horizontal L, and height H of this rectangular parallelepiped R, as shown to Fig.6 (a). Thereby, the minimum rectangular parallelepiped R as shown in FIG. 6A that includes the single item point group (gripable region group HG1) can be automatically set and automatically measured.

  Next, the CPU 50 (see FIG. 1) selects and outputs the robot hand 22 (see FIG. 1) corresponding to the area of the upper surface Ra (see FIG. 6 (a)) of the set minimum rectangular parallelepiped R (step S11). ). More specifically, the CPU 50 (see FIG. 1) calculates the area of the upper surface Ra (see FIG. 6A) of the set minimum rectangular parallelepiped R from the size of the measured vertical T and horizontal L of the minimum rectangular parallelepiped R. calculate. If the calculated area is an area that can be sucked, information that prompts the user to select the suction hand as the robot hand 22 (see FIG. 1) is sent to the robot controller 3 via the output unit 52 (see FIG. 1). Output. As a result, the robot control device 3 controls the robot 2 (see FIG. 1) and causes the robot 2 to select the suction hand as the robot hand 22 (see FIG. 1) using a tool changer (not shown).

  On the other hand, if the calculated area is not an adsorbable area, the CPU 50 (see FIG. 1) outputs information for prompting the user to select the sandwiching hand as the robot hand 22 (see FIG. 1). Is output to the robot controller 3. As a result, the robot control device 3 controls the robot 2 (see FIG. 1) and causes the robot 2 to select the pinching hand as the robot hand 22 (see FIG. 1) using a tool changer (not shown).

  In this way, the optimum hand can be automatically selected. In addition, what is necessary is just to preset the reference | standard of whether it is an area which can be attracted | sucked. Further, when a suction hand or a pinching hand is selected as the robot hand 22 (see FIG. 1), the size of the suction hand or the pinching hand may be selected. For example, when a suction hand is selected, if the calculated area exceeds the first reference, information for prompting the user to select a larger suction hand is controlled via the output unit 52 (see FIG. 1). Information that prompts the user to select a medium suction hand is output to the apparatus 3 via the output unit 52 (see FIG. 1) if the first reference is not exceeded but the second reference is exceeded. Information that prompts the user to select a smaller suction hand is output to the control device 3 if the first and second standards are not exceeded but the third standard is exceeded. And output to the robot control device 3 via this.

  Next, as shown in FIG. 6B, the CPU 50 (see FIG. 1) matches the set minimum rectangular parallelepiped R for each grippable region group HG2 to HG15. That is, since there is a three-dimensional point group TG indicating three-dimensional coordinates for each grippable region group HG2 to HG15, the CPU 50 (see FIG. 1) uses the information of the three-dimensional point group TG to determine the grippable region group HG2. The direction of the set minimum rectangular parallelepiped R can be matched for each ˜HG15. Then, the CPU 50 (see FIG. 1) adjusts the direction of the minimum rectangular parallelepiped R set for each grippable region group HG2 to HG15, that is, the longitudinal direction (lateral direction) and the width direction (vertical direction) of the minimum rectangular parallelepiped R. Direction) and calculate the position and orientation. Then, the CPU 50 (see FIG. 1) outputs the calculated position and orientation to the robot control device 3 via the output unit 52 (see FIG. 1) (step S12), and the program processing ends. At this time, the measured size of the minimum rectangular parallelepiped R is T, L, and H. Thereby, as will be described in detail below, the picking object Wa (see FIG. 1) can be accurately transferred.

  That is, the robot control device 3 controls the robot 2 (see FIG. 1) based on the output vertical T, horizontal L size, and height H, and thus the robot hand of the robot 2 (see FIG. 1). The picking object Wa (see FIG. 1) gripped at 22 (see FIG. 1) is transferred to a predetermined location (for example, a conveyor). More specifically, the robot control device 3 considers the length and width of a predetermined location (for example, a conveyor) from the vertical T and horizontal L sizes (information on the length and width of the predetermined location (for example, conveyor)) The path is set when the picking object Wa (see FIG. 1) held by the robot hand 22 (see FIG. 1) is transferred. Further, the robot control device 3 considers the height of a predetermined location (for example, a conveyor) from the height H, a clearance for safely placing the picking target object Wa (the predetermined location (for example, a conveyor) ) Is stored in advance), and a route is set for transferring the picking object Wa (see FIG. 1) held by the robot hand 22 (see FIG. 1). Thus, the robot 2 (see FIG. 1) is controlled based on the route setting set in this way, and the picking target gripped by the robot hand 22 (see FIG. 1) of the robot 2 (see FIG. 1). The object Wa (see FIG. 1) is transferred to a predetermined location (for example, a conveyor). Thereby, the picking target object Wa (refer FIG. 1) can be safely transferred to a predetermined location (for example, conveyor).

  The robot hand 22 (see FIG. 1) is provided with a weight sensor such as a strain gauge that measures the weight of the picking object Wa when the picking object Wa is gripped. Is output to the robot controller 3. Thereby, the weight of picking target object Wa can be measured automatically.

  On the other hand, the robot controller 3 receives this, and if the measured weight is larger than the preset weight, the picking object Wa (see FIG. 1) held by the robot hand 22 (see FIG. 1). 1) is set lower than the preset acceleration, and if the measured weight is smaller than the preset weight, the robot hand 22 (see FIG. 1) holds it. If the picking target object Wa (see FIG. 1) to be transferred is set to a higher acceleration than the preset acceleration, and the measured weight is the same as the preset weight, The acceleration at the time of transferring the picking object Wa (see FIG. 1) held by the robot hand 22 (see FIG. 1) is set to a preset acceleration.

  Thus, the picking object controlled by the robot hand 22 (see FIG. 1) of the robot 2 (see FIG. 1) is controlled by controlling the robot 2 (see FIG. 1) based on the set acceleration. Wa (see FIG. 1) is transferred to a predetermined location (for example, a conveyor). As a result, it is possible to reduce a situation in which the picked object Wa (see FIG. 1) that is gripped is dropped while being transferred to a predetermined location (for example, a conveyor). In place of the weight sensor, the current required for driving the motor of the robot arm unit 21 whose base end is rotatably connected is detected, and the weight of the picking object Wa is estimated from the detected information. The same processing can be performed.

  Thus, according to the present embodiment described above, it is possible to automatically recognize and pick a picking object to be grasped at a distribution site. As a result, it is not necessary to prepare all the three-dimensional shapes of picking objects of thousands to hundreds of thousands in advance as a database, and automation can be realized.

  In other words, conventionally, since it is necessary to register all three-dimensional shapes of picking objects of several thousand to several hundred thousand in advance in the database, the size of the picking object to be grasped is accurately registered. There is a need to. On the other hand, according to the present embodiment, since a picking target object to be grasped can be automatically recognized and picked, such an accurate registration becomes unnecessary. Moreover, even if the size of the picking target object to be grasped is input (registered), accurate input (registration) is not required, and an outline (for example, a reference for the vertical / horizontal height of the picking target object or a vertical / horizontal height) It is only necessary to input (register) a shape such as a numerical range that can be obtained.

  The contents illustrated in the present embodiment are merely examples, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims.

  For example, in the present embodiment, the corrugated cardboard box CA has been illustrated as loading a plurality of picking objects Wa. However, the present invention is not limited to this and may be loaded in any container such as a container.

  Moreover, in this embodiment, although the example which can acquire the three-dimensional point group information which shows the three-dimensional coordinate of the picking target object Wa whole W with the camera 4 was shown, you may enable it to acquire a two-dimensional image.

  In the present embodiment, an example is shown in which the camera 4 can acquire 3D point group information indicating the 3D coordinates of the entire picking object Wa. However, the present invention is not limited thereto, and the camera 4 has only an imaging function. The CPU 50 (see FIG. 1) of the image processing apparatus 5 (see FIG. 1) may generate three-dimensional point group information indicating the three-dimensional coordinates of the entire picking object Wa.

  On the other hand, in the present embodiment, an example in which the camera 4 is provided separately has been described, but the present invention is not limited thereto, and the camera 4 may be incorporated in the robot 2. Moreover, although the example which provides the robot control apparatus 3 and the image processing apparatus 5 separately was shown, you may integrate the robot control apparatus 3 and the image processing apparatus 5. FIG.

1 Picking device 2 Robot 3 Robot control device 4 Camera (3D point cloud information generating means)
5 Image processing device 50 CPU
CA cardboard box (predetermined container)
W Whole (of multiple picking objects) Wa Picking object R Minimum rectangular parallelepiped (minimum box-shaped body)
L side (longitudinal direction of the smallest box)
T length (width direction of the smallest box-shaped body)
H Height (minimum box height)

Claims (11)

3D point cloud information generating means for generating 3D point cloud information of the entire plurality of picking objects loaded in a predetermined container;
Based on the three-dimensional point cloud information generated by the three-dimensional point cloud information generation means, a common three-dimensional shape is found from the entire plurality of picking objects, and an area determination for determining an area of each picking object Means,
Based on the 3D point cloud information generated by the 3D point cloud information generation means belonging to the area of the picking target object determined by the area determination means, the 3D position and orientation of the picking target object to be grasped are determined. Position and orientation output means for outputting;
A picking apparatus comprising: a robot that grips and transfers a picking target object to be gripped based on the three-dimensional position / posture output by the position / posture output means. Generate three-dimensional point cloud information of the entire plurality of picking objects loaded in a predetermined container,
Based on the generated 3D point cloud information, find a common 3D shape from the entire plurality of picking objects, determine the area of each picking object,
Based on the generated 3D point cloud information belonging to the determined area of the picking object, output the 3D position and orientation of the picking object to be grasped,
A picking method in which a picking object to be grasped is grasped and transferred using a robot based on the output three-dimensional position and orientation.   The picking method according to claim 2, wherein an accurate size of the picking object to be grasped is not required to be input in advance.   Based on the generated three-dimensional point cloud information, by finding discontinuous three-dimensional point cloud information in the depth direction, a common three-dimensional shape is found from the whole of the plurality of picking objects, and individual picking objects The picking method according to claim 2 or 3, wherein the area is determined.   Based on the generated three-dimensional point cloud information, segmentation is performed by finding discontinuous three-dimensional point cloud information in the depth direction, and the similarity of the three-dimensional point cloud information between the segmentations is determined using a predetermined algorithm. The picking method according to claim 2, wherein a common three-dimensional shape is found from all of the plurality of picking objects, and an area of each picking object is determined.   The minimum box-shaped body containing the determined picking object is set, and the length in the longitudinal direction, the length in the width direction, and the height of the set minimum box-shaped body are measured. Item 2. The picking method according to item 1.   The picking method according to claim 6, wherein the measured length in the longitudinal direction, the length in the width direction, and the height of the minimum box-shaped body are output.   The weight of the picking target as the gripping target is measured when the picking target as the gripping target is gripped using a robot based on the output three-dimensional position and orientation. The picking method according to claim 1.   The picking method according to claim 8, wherein an acceleration at the time of transferring the picking target to be gripped is controlled using the robot based on the measured weight of the picking target to be gripped.   The picking method according to claim 6, wherein, based on the measured length, a robot is used to set and control a path for transferring the picking object to be grasped. The picking method according to claim 6, wherein a gripping tool is selected using at least the measured length in the longitudinal direction and the length in the width direction of the minimum box-shaped body.

Images