JP7191569B2 - gripping device - Google Patents

Description

The present invention relates to a gripping device for picking up workpieces.

2. Description of the Related Art In machining or assembly work, a workpiece, which is a target object, is often automatically picked up by a robot, an assembly device, or the like and set in a processing device or assembly housing or the like. When picking up, it is necessary to recognize the shape and posture of the workpiece and control the pickup arm. For example, the workpiece information processing apparatus is used for posture detection, shape recognition, and the like of workpieces required for picking up workpieces by robots, assembling apparatuses, and the like.

Conventionally, image processing is used for posture detection and shape recognition of workpieces. In a work information processing apparatus, a camera photographs a work, the posture of the work is detected by pattern matching or the like, and the pickup arm is controlled based on the detection result to position the hand in a grippable posture.

Japanese Patent Laying-Open Nos. 2015-213973, 2017-047505, 2016-221647, 10-332333 and 11-066321 disclose a gripping device for picking up a workpiece. ing.

JP 2015-213973 A JP 2017-047505 A JP 2016-221647 A JP-A-10-332333 JP-A-11-066321

Regarding the attitude control of the hand, in Japanese Patent Application Laid-Open No. 2015-213973 (Patent Document 1), interference between the hand, the positioning mechanism of the hand, and a container in which the work is stored is checked before the gripping operation, and the detected work is detected. A method has been proposed for determining whether or not an item can be picked. It checks for mechanical interference before the gripping operation, and is a function necessary for picking up work.

In addition, the gripping device disclosed in Japanese Patent Laying-Open No. 2017-047505 (Patent Document 2) can determine whether gripping is impossible even if there is an obstacle in the process of the hand passing from the approach point of the hand to the gripping point. First, it learns past gripping success/failure information and the state of existence of obstacles in the hand passage area during the gripping motion by machine learning (support vector machine), and determines whether the gripping success or failure using the learning results before the gripping motion. , the grasping motion is performed based on the determination result. This device also determines whether or not a gripping operation due to mechanical interference can be performed before the gripping operation, as in Japanese Patent Application Laid-Open No. 2015-213973 (Patent Document 1).

Japanese Patent Laying-Open No. 2016-221647 (Patent Document 3) proposes a method of selecting a gripping form from a plurality of pieces of gripping form information held in an apparatus in consideration of the state of surrounding workpieces at a gripping position. ing. As the evaluation value for selecting the gripping form, the evaluation value is calculated based on the number of contact points between the gripping target and surrounding workpieces, and information on the slipperiness, breakage, and tangling of the target workpiece at the contact points. We propose a method of selecting the grasping form that maximizes the evaluation value.

Japanese Patent Application Laid-Open No. 2017-047505 (Patent Document 2) is superior to Japanese Patent Application Laid-Open No. 2015-213973 (Patent Document 1) in that the accuracy of success/failure determination can be improved by learning based on the success/failure information of past gripping motions. However, there is a problem that learning requires trial of motions and it takes time to learn. In addition, as in Japanese Patent Application Laid-Open No. 2016-221647 (Patent Document 3), it is necessary to determine the characteristics in advance, and it may take time to prepare for learning.

The present invention is intended to solve such problems, and its object is to complete the learning for selecting an appropriate gripping position in a relatively short period of time in the work of picking up bulk workpieces. The object is to provide a gripping device that is possible.

A gripping device according to the present disclosure includes an imaging device for observing the surface of a workpiece, a three-dimensional sensor for measuring coordinate data of points on the surface of the workpiece, a hand for gripping the workpiece, and a grippable posture. and a workpiece information processing device that detects the optimum gripping position of a workpiece using an imaging device and a three-dimensional sensor and controls the positioning mechanism and the hand. The workpiece information processing device determines a workpiece gripping position based on coordinate data acquired from a three-dimensional sensor or image data acquired from an imaging device, and includes a positioning mechanism and a hand so that the hand grips the workpiece based on the gripping position. is operated, the hand is photographed by the imaging device, and based on the obtained image data, it is determined whether or not the workpiece is actually gripped, and the relationship between the determination result and the gripping position is accumulated to determine whether gripping is successful at the gripping position. Then, based on the success rate of gripping, the gripping position to be adopted in subsequent gripping operations is determined.

Preferably, the workpiece information processing device includes a workpiece detection unit that detects the position of the workpiece to be gripped using coordinate data acquired from the three-dimensional sensor or image data acquired from the photographing device, and the position of the workpiece detected by the workpiece detection unit. A gripping position candidate detection unit that determines interference between objects surrounding the workpiece and the hand based on the position of the workpiece and detects one or more gripping position candidates, and among the gripping position candidates extracted by the gripping position candidate detection unit a gripping position selection unit that selects the optimum gripping position from the optimal gripping position, a command output unit that outputs a positioning command or a gripping/releasing command to the positioning mechanism or hand based on the optimal gripping position, and an actual and a gripping motion creation failure determination unit that determines whether or not the workpiece can be gripped by image processing. The gripping position candidate detection unit obtains a gripping success rate for the gripping position candidates based on machine learning results using data obtained from the three-dimensional sensor or the imaging device and determination results of the gripping motion creation failure determination unit. The gripping position selection unit selects the candidate with the highest gripping success rate as the optimum gripping position.

More preferably, the gripping position candidate detection unit uses coordinate data acquired from a three-dimensional sensor or image data acquired from an imaging device as learning data for machine learning, a gripping position when a gripping motion is executed, and a gripping motion creation failure determination unit. The grasping success rate is obtained using the determination result of .

More preferably, when a predetermined N is a natural number, the number of gripping motions for which the gripping position candidate detection unit does not use the learning result for calculating the gripping success rate is N times. The gripping position selection unit selects the point closest to the center of gravity position on the image of the workpiece from the gripping position candidates up to the Nth gripping, and after the Nth gripping is exceeded, the gripping position selection unit selects The candidate with the highest gripping success rate is selected as the optimum gripping position.

More preferably, the gripping position candidate detection unit uses the coordinate data acquired from the three-dimensional sensor or the image data acquired from the photographing device while the gripping device is in operation, and the determination result of the gripping movement creation/non-creation determination unit, and sequentially calculates the results of machine learning. Update.

More preferably, the gripping position candidate detection unit resets at least a portion of the machine learning result every predetermined number of gripping motions.

Preferably, the workpiece information processing apparatus further includes a vibration table on which the container containing the workpiece is placed, and the workpiece information processing apparatus, in the next gripping operation, prior to gripping by the hand, based on the image of the workpiece in the container captured by the imaging device. If the gripping success rate corresponding to the gripping position determined by the above is lower than the judgment value, the position of the workpiece is changed by the shaking table.

Preferably, the photographing device includes a first camera for photographing the placement location where the work is placed before it is gripped by the hand, and a second camera for photographing the placement location where the work is to be placed after it is gripped by the hand. including. The workpiece information processing device determines the placement position of the workpiece based on the image data acquired from the second camera, determines whether the placement position matches the reference position, and calculates the placement success rate based on the determination result. . In the next gripping operation, the workpiece information processing apparatus captures an image of the workpiece placed on the placement location with the first camera before gripping with the hand, and grips the workpiece corresponding to the gripping position determined based on the captured image. Based on the success rate and placement success rate, it is determined whether or not to adopt the gripping position.

According to the present invention, it is possible to complete the learning for selecting an appropriate gripping position in a relatively short period of time in the work of picking up bulk workpieces.

1 is a diagram showing a configuration of a grasping device according to Embodiment 1; FIG. 4 is a front view showing the shape of a hand; FIG. FIG. 4 is a side view showing the shape of a hand; 2 is a diagram showing an internal configuration of a work information processing device 6; FIG. FIG. 4 is a diagram for explaining how pattern matching is performed; FIG. 4 is a diagram for explaining information detected during pattern matching; FIG. FIG. 7 is a view of the workpiece W viewed from the + direction of the Z axis in FIG. 6; FIG. 7 is a diagram for explaining a process executed by a gripping position candidate detection unit 63; This is a first example of indicating the grippable position of the workpiece W. FIG. This is a second example of indicating the grippable position of the workpiece W. FIG. FIG. 4 is a diagram showing an arrangement example of a situation of a workpiece W having a plurality of grippable positions and surrounding workpieces. FIG. 10 is a diagram showing an image detected when gripping is successful; FIG. 10 is a diagram showing an image detected when gripping fails; 4 is a flow chart showing processing performed by the work information processing apparatus; FIG. 10 is a diagram showing the configuration of a gripping device according to Embodiment 2; 9 is a flowchart for explaining processing executed by the work information processing apparatus according to the second embodiment; FIG. 12 is a diagram showing the configuration of a gripping device according to Embodiment 3; 10 is a flowchart for explaining processing executed by the work information processing apparatus according to the third embodiment; It is a figure which shows the image detected at the time of installation success. It is a figure which shows the image detected at the time of installation failure.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings below, the same reference numbers are given to the same or corresponding parts, and the description thereof will not be repeated.

[Embodiment 1]
In the following embodiments, a case of gripping cylindrical workpieces in a bulk state will be described. The gripping device of the present embodiment selects the optimum gripping position for inserting the hand while avoiding interference with the surrounding workpieces when detecting the gripping position of a randomly stacked workpiece.

(Configuration of gripping device)
FIG. 1 is a diagram showing the configuration of a grasping device according to Embodiment 1. FIG. Referring to FIG. 1, a gripping device 1 includes a camera 2 for observing the surface of a workpiece W, a three-dimensional sensor 3 for measuring XYZ coordinate data of each point on the workpiece surface, and a An optimum gripping position is determined using the hand 4, a positioning mechanism 5 that positions the hand 4 in a posture capable of gripping the workpiece W, image data R, G, B from the camera 2, and XYZ coordinate data D from the three-dimensional sensor 3. and a work information processing device 6 for controlling the positioning mechanism 5 and the hand 4 .

The workpiece information processing device 6 determines the gripping position of the workpiece based on the coordinate data acquired from the three-dimensional sensor 3 or the image data acquired from the imaging device 2 so that the hand 4 grips the workpiece based on the gripping position. The positioning mechanism 5 and the hand 4 are operated, the hand 4 is photographed by the photographing device 2, it is determined whether or not the workpiece is actually gripped based on the acquired image data, and the relationship between the determination result and the gripping position is determined. By accumulating, the gripping success rate at the gripping position is calculated, and based on the gripping success rate, the gripping position to be adopted in subsequent gripping operations is determined.

In this embodiment, an example using an articulated robot as the positioning mechanism 5 is shown. In addition, a horizontal multi-joint type in which an arm moves in the horizontal direction, an orthogonal type robot configured by three orthogonal slide axes, or the like can be selected as the positioning mechanism 5 .

FIG. 2 is a front view showing the shape of the hand. FIG. 3 is a side view showing the shape of the hand. The hand 4 grips the work by sandwiching the work with two fingers 41 and 42 .

Referring to FIG. 1 again, two cameras (not shown) are mounted inside the three-dimensional sensor 3 to measure the three-dimensional coordinates of the surface of the object by triangulation. If the surface of the workpiece W does not have a pattern that serves as a measurement mark, the points on the images of the two cameras cannot be associated, so the workpiece W is irradiated with slit-shaped light or dot-shaped light from the light source. . This light source may be mounted inside the three-dimensional sensor 3 or may be arranged above the camera 2 . Moreover, the three-dimensional sensor 3 is not particularly limited to a stereo three-dimensional sensor, and is not particularly limited as long as it is a sensor that measures three-dimensional coordinate data by a light section method, a phase shift method, or the like.

(Internal configuration of workpiece information processing device)
FIG. 4 is a diagram showing the internal configuration of the work information processing device 6. As shown in FIG. The workpiece information processing device 6 includes an image input unit 61, a workpiece detection unit 62, a gripping position candidate detection unit 63, a gripping position selection unit 64, a gripping motion creation failure determination unit 65, a data storage unit 67, and a command output. a portion 66;

The image input unit 61 inputs XYZ coordinate data from the three-dimensional sensor 3 and image data from the camera 2 . The workpiece detection unit 62 uses the XYZ coordinate data acquired from the three-dimensional sensor 3 or the image data acquired from the camera 2 to detect the position of the workpiece to be gripped. The gripping position candidate detection unit 63 determines interference between the surrounding objects of the workpiece and the hand 4 based on the position of the workpiece detected by the workpiece detecting unit 62, and detects one or more gripping position candidates. The gripping position selection unit 64 selects the optimum gripping position that maximizes the predicted success rate from the gripping position candidates extracted by the gripping position candidate detection unit 63 . The grip motion creation failure determination unit 65 determines whether or not the work is actually gripped based on the image data acquired from the camera 2 by image processing.

The data storage unit 67 stores the XYZ coordinate data from the three-dimensional sensor 3, the image data acquired from the camera 2, the coordinate values of the gripping position candidates, the result of determining the success or failure of the gripping motion, and the like. The command output unit 66 outputs a positioning command or a gripping/releasing command to the positioning mechanism 5 or the hand 4 based on the optimum gripping position.

The gripping position candidate detection unit 63 obtains a gripping success rate from the result of machine learning using the data acquired from the three-dimensional sensor 3 or the imaging device 2 and the determination result of the gripping movement creation failure determination unit 65 for the gripping position candidate. The candidate with the highest success rate is selected as the optimal gripping position.
(Processing procedure in workpiece information processing device)
The processing of each part of the work information processing device 6 will be described in detail below.

The workpiece detection unit 62 detects a gripping target using the XYZ coordinate data D acquired from the three-dimensional sensor 3 . In this embodiment, an example in which pattern matching is used to detect a grasped target is shown.

FIG. 5 is a diagram for explaining how pattern matching is performed. An image of the workpiece W is stored as a template T in the data storage unit 67 in advance. The work detection unit 62 reads out the template T when detecting a gripping target, superimposes it on the image data from the camera 2 and the XYZ coordinate data from the three-dimensional sensor 3, and performs pattern matching to search for a work having the most matching posture. .

FIG. 6 is a diagram for explaining information detected during pattern matching. In pattern matching, the center coordinates (ox, oy, oz) of the workpiece W and the rotation angles (α, β, γ) around each axis are detected.

Next, processing executed by the grip position candidate detection unit 63 will be described. FIG. 7 is a view of the workpiece W viewed from the + direction of the Z axis in FIG. FIG. 7 shows the positional relationship between the workpiece W to be gripped, four surrounding workpieces, and two fingers 41 and 42 of the hand 4 . 7, the hand 4 is positioned such that the midpoint of the coordinates of the two fingers 41 and 42 of the hand 4 coincides with the center of gravity O of the workpiece W. As shown in FIG. In the example of FIG. 7, the workpiece W and the two fingers 41 and 42 of the hand 4 do not interfere. The gripping position candidate detection unit 63 determines that the workpiece W and the two fingers 41 and 42 of the hand 4 do not interfere with each other as shown in FIG. 7 as grippable.

FIG. 8 is a diagram for explaining the processing executed by the gripping position candidate detection unit 63. As shown in FIG. The gripping position candidate detection unit 63 predicts the gripping position candidates shown in FIG. 7 and the success rate of the gripping motion using the neural network shown in FIG. The neural network receives the XYZ coordinate data D acquired from the three-dimensional sensor 3 and the images R, G, and B acquired from the camera 2 as inputs, and predicts the XYZ coordinate values of gripping position candidates and the gripping success rate.

The gripping position candidate detection unit 63 also performs learning processing for a neural network. Back propagation (backpropagation method) is used for learning. Prepare the following teacher data for learning.

The workpiece W is photographed by the camera 2 from the + direction of the Z axis in FIG. A person indicates a position of the work W where the hand 4 does not interfere. FIG. 9 shows a first example of indication of the grippable position of the workpiece W. FIG. FIG. 10 shows a second example of indicating the grippable position of the workpiece W. FIG. A rectangle as shown in FIGS. 9 and 10 is designated by a human and used as training data of grippable positions.

The central coordinates X, Y and the depth direction coordinate Z of the rectangle shown in FIGS. Also, the maximum value of 100 is given as the initial value of the success rate of the instructed gripping position. The success rate is a value in the range of 0-100.

It takes time to try grasping and learning. To address this issue, in the present embodiment, a certain amount of learning is done before the motion is attempted.

For example, a workpiece is photographed in advance, a human judges and designates a grippable part in the photographed image, and learning is performed in advance using the photographed image and the part indicated by the human as teacher data. By using this learning result, it is possible to reduce the number of actual motion trials.

In addition, by adopting the above method, it is no longer necessary to determine the characteristics in advance as in Japanese Patent Application Laid-Open No. 2016-221647 (Patent Document 3), and the characteristic information necessary for determining whether or not the grip can be grasped is acquired by learning. can do.

FIG. 11 is a diagram showing an arrangement example of a situation of a workpiece W having a plurality of grippable positions and surrounding workpieces. Assuming a situation such as that shown in FIG. 11, a label indicating the priority of gripping is assigned to the gripping position. Here, for example, A has the highest priority, and three levels of priority, B and C, are provided in order. The priority is determined and assigned by the rectangle designator.

As described above, the central coordinates (X, Y), the depth coordinate Z, the vertical and horizontal size of the rectangle, the initial value of the success rate, and the label are assigned to each rectangle. These pieces of information, the images R, G, and B of the workpiece W, and the XYZ coordinate data D obtained from the three-dimensional sensor 3 are used as training data for neural network learning. The gripping position candidate detection unit 63 detects gripping position candidates using this learning result as an initial value.

Next, processing executed by the gripping position selection unit 64 and the gripping motion creation failure determination unit 65 will be described. The gripping position selection unit 64 selects a candidate having a high priority and a maximum prediction success rate of gripping motion among the gripping position candidates detected by the gripping position candidate detection unit 63 .

The success or failure of the gripping motion is determined by the gripping motion creation/failure determining unit 65 . After the actual gripping operation, the hand 4 is photographed by the camera 2, and the work in the hand 4 is detected by image processing.

FIG. 12 is a diagram showing an image detected when gripping is successful. Comparing the image of the hand 4 that has not gripped the workpiece and the image of the hand 4 that has successfully gripped the workpiece W results in a mismatch. If there is such a mismatching portion, it is determined that the grip has been successful.

FIG. 13 is a diagram showing an image detected when gripping fails. When the image of the hand 4 not gripping the workpiece and the image of the hand 4 failing to grip the workpiece W are compared, the two images match each other, so no difference image is generated. When there is no inconsistent part in this way, it is determined that gripping has failed.

As shown in FIGS. 12 and 13 , the grip motion creation failure determination unit 65 compares an image of the hand 4 in a non-gripping state captured in advance and stored in the data storage unit 67 with an image after the grip motion. Then, if the two images match, it is determined as undetected, and if they do not match, it is determined as detected.

The gripping position candidate detection unit 63 performs learning even during the trial of the gripping motion, and updates the learning result. The gripping position candidate detection unit 63 detects the coordinate data acquired from the three-dimensional sensor 3 or the image data acquired from the imaging device as learning data for machine learning, the gripping position when the gripping motion is performed, and the gripping motion creation failure determination unit 65. The judgment result is used.

In the case of non-detection in the grasping movement creation failure determination unit 65, the success rate S of the grasping movement predicted by the neural network is substituted into the following equation to obtain the updated success rate S'.
S'=α×S+β
Here, (α, β) are called update coefficients. Each value is arbitrary, but here (α, β)=(0.99, −0.1). An optimum value determined empirically by trial learning may be used.

During the gripping motion trial, the success rate S′ updated as described above, the central coordinates (X, Y) predicted by the neural network, the depth coordinate Z, the vertical and horizontal size and label of the rectangle, and the gripping position candidate detection unit 63 Learning is performed using the images R, G, B and data used for detection, and the image data D of the three-dimensional sensor 3 .

The gripping position candidate detection unit 63 uses the coordinate data acquired from the three-dimensional sensor 3 or the image data acquired from the photographing device 2 while the gripping device 1 is in operation, and the determination result of the gripping motion creation/non-creation determination unit 65, and sequentially obtains the results of machine learning. to update.

As described above, by performing learning based on the gripping motion results for each gripping motion, gripped positions with a low success rate are excluded, and only gripped positions with a high success rate are selected. As a result, gripping failures can be reduced, and the efficiency of the take-out operation can be improved.

Further, in the present embodiment, an example is shown in which the sequential learning is performed while the gripping device is in operation. In some cases, it may be preferable to reset the learning results at regular intervals.

In such a case, the learning result may be reset every M gripping motions set in advance. The gripping position candidate detection unit 63 resets the machine learning result for each predetermined number of gripping operations. After resetting, the same learning is performed again each time the grasping motion is performed, and when the number of times of learning reaches M, the learning result is reset. Also, M does not have to be a fixed value, and may be changed at each reset according to changes in the surrounding environment, the hand 4, the workpiece, and the like.

Also, in the automatic learning during the gripping motion, a human may periodically check the recognition result and create teacher data.

In this case, an image stored in the data storage unit 67 is referred to, and a person designates a rectangle as shown in FIGS. 9 and 10, for example. Alternatively, a human may directly specify the updated success rate S'. This makes it possible to further shorten the learning time.

A series of processes described above will be described with reference to a flowchart. FIG. 14 is a flow chart showing processing performed by the work information processing apparatus. The process shown in FIG. 14 is carried out after the human gives the training data of the gripping positions described in FIGS. 9 and 10 and causes the neural network to learn to some extent. The work information processing device 6 acquires images from the camera 2 and the three-dimensional sensor 3 (S1). Subsequently, the workpiece information processing device 6 detects a gripping target from the acquired image (S2), detects gripping position candidates, and predicts a gripping success rate of the gripping position candidates (S3).

Then, the workpiece information processing device 6 selects a gripping position candidate with the highest priority and the highest gripping success rate from among the gripping position candidates (S4), and performs a gripping operation using the positioning mechanism 5 and the hand 4 (S5). ).

Preferably, when N is a natural number, N is set in advance as the number of gripping motions in which the learning result is not used for the calculation of the gripping success rate in step S3. After the number of times exceeds N, the gripping success rate is calculated using machine learning results in step S3, and the candidate with the maximum gripping success rate is set as the optimum gripping position in step S4. You may choose.

After executing the gripping motion, the work information processing device 6 captures an image of the hand 4 with the camera 2 (S6), and confirms whether the gripping motion was successful or not (S7). Then, the workpiece information processing device 6 saves the image, the gripping position coordinates, and whether or not the gripping motion is to be created (S8), and executes the neural network learning process (S9).

Further, when the load of the learning process is high, it may take time to calculate the gripping success rate. In the present embodiment, an example in which learning is performed by the gripping position candidate detection unit 63 of the work information processing device 6 has been described, but there is no particular need to limit it to this, and the learning process of the gripping position candidate detection unit 63 can be performed by the work information processing. A dedicated learning device for the device 6 may be used. In this case, for example, the workpiece information processing device 6 and the learning device are connected by serial communication, I/O, etc., and the coordinate values of gripping position candidates used for learning, the XYZ coordinate data D of the three-dimensional sensor, the image data R of the camera, The work information processing device 6 transmits the success/failure determination result of G, B, and the gripping motion to the learning device. The learning device performs learning based on the received data and transmits the learning result to the work information processing device 6 . The workpiece information processing device 6 calculates the gripping success rate using the new learning result received from the learning device.

In this embodiment, back propagation (error backpropagation method) is used for learning, but other than this, supervised learning methods such as support vector machines, autoencoders (self-encoders), Unsupervised learning methods such as the k-means method, principal component analysis, and machine learning methods combining them may also be used.

As described above, according to the gripping apparatus according to the first embodiment, by using the workpiece image actually photographed by the three-dimensional sensor or camera and the actual gripping operation result for learning, it is possible to determine the dimensions and surface texture of the workpiece. Variation, the current state of the hand, etc. can be reflected in the learning, and a drop in the gripping success rate can be reduced. In addition, since it is possible to reduce the decrease in gripping success rate, it is possible to shorten the tact time of the work.

[Embodiment 2]
In the first embodiment, the work with the highest gripping success rate is selected from among the works in the current arrangement. However, the hand 4 may not be inserted depending on the positional relationship with the adjacent work. In the second embodiment, the grasping work is assisted in such a case.

FIG. 15 is a diagram showing a configuration of a grasping device according to Embodiment 2. FIG. Referring to FIG. 15, gripping apparatus 1A includes, in addition to the structure of gripping apparatus 1 of FIG. 1, a vibration table 8 on which a container (work tray 7) containing a workpiece is placed. The vibrating table 8 is provided with a vibrating body. Further, the gripping device 1A includes a work information processing device 6A instead of the work information processing device 6. As shown in FIG. The camera 2, the three-dimensional sensor 3, the hand 4, and the positioning mechanism 5 are the same as in the case of FIG. 1, so the description will not be repeated.

When the gripping success rate of the gripping position candidate is lower than the threshold value, the workpiece information processing device 6A vibrates the shaking table 8 to try to improve the gripping success rate.

In the next gripping operation of the workpiece information processing device 6A, the gripping success rate corresponding to the gripping position determined based on the image of the workpiece in the container photographed by the photographing device before gripping by the hand is lower than the judgment value. In this case, the position of the work is changed by the vibration table 8 .

FIG. 16 is a flowchart for explaining processing executed by the work information processing apparatus according to the second embodiment. The processing of the flowchart of FIG. 16 differs from the processing described with reference to FIG. 14 in that steps S51 and S52 are executed after step S4.

The workpiece information processing device 6A acquires an image (S1), detects a gripping target (S2), predicts the gripping success rate of gripping position candidates (S3), and selects a gripping position candidate with the highest priority and highest gripping success rate. Each process (S4) is sequentially performed in the same manner as in FIG.

Then, the work information processing device 6A determines whether or not the predicted success rate S of the gripping position candidate is higher than a certain threshold value T in step S51.

If the predicted success rate S of the gripping position candidate does not reach the threshold value T (NO in S51), vibrate the shaking table 8 in the direction of the arrow for a certain period of time (S52), and then execute steps S1 to S4 again. , and attempt to detect gripping position candidates that give a success rate S that exceeds the threshold T.

If the predicted success rate S of the gripping position candidate is higher than the threshold value T (YES in S51), steps S5 to S9 are executed as in the first embodiment.

If a gripping position candidate with an improved success rate is not detected even after repeating the process of step S52 several times, for example, the hand 4 may be caused to shift the workpiece.

The gripping device according to the second embodiment has the same effect as the gripping device according to the first embodiment, and can improve the gripping success rate when the gripping success rate is not high due to poor placement of the workpiece.

[Embodiment 3]
In Embodiment 1, learning was performed so as to increase the gripping success rate. However, it is necessary for the gripping device to not only grip the workpiece, but also to place the gripped workpiece on a workbench or the like for processing in the next process. Depending on the gripping position of the hand 4, the work may not be placed at the optimum position for the work in the next process. In the third embodiment, learning is performed in consideration of the placement motion as well as the gripping motion.

FIG. 17 is a diagram showing a configuration of a grasping device according to Embodiment 3. FIG. Referring to FIG. 17, a gripping device 1B has a camera 2A and a camera 2B as imaging devices in the configuration of the gripping device 1 of FIG. Camera 2A is a camera similar to camera 2 in FIG. The camera 2B photographs the position of the work placed on the work table 10 for the next process. The gripping device 1B also includes a work information processing device 6B in place of the work information processing device 6. As shown in FIG. The camera 2, three-dimensional sensor 3, hand 4, and positioning mechanism 5 are the same as in FIG.

That is, as shown in FIG. 17, the photographing device 2 includes a first camera 2A for photographing the placing place where the work is placed before being gripped by the hand, and a member on which the work is placed after being gripped by the hand. and a second camera 2B that captures the . The workpiece information processing device 6B determines the placement position of the workpiece based on the image data acquired from the second camera 2B, determines whether or not the placement position matches the reference position, and calculates the placement success rate based on the determination result. In the next gripping operation, the gripping success rate and placement corresponding to the gripping position determined based on the image of the workpiece placed on the placement location before gripping by the hand 4, which is captured by the first camera 2A. Adoption or rejection of the gripping position is determined based on the success rate.

FIG. 18 is a flowchart for explaining processing executed by the work information processing apparatus according to the third embodiment. The processing of the flowchart of FIG. 18 differs from the processing described with reference to FIG. 14 in that steps S101 to S103 are executed after step S7.

Work information processing device 6B sequentially performs the processes of steps S1 to S7 in the same manner as in FIG. Then, in step S101, the workpiece information processing apparatus 6B executes the operation of setting the gripped workpiece, takes an image of the installation position with the camera 2B in step S102, and confirms the success or failure of the installation operation in step S103.

Then, as in the first embodiment, the processes of steps S8 and S9 are executed. In the learning process of step S9, the installation success rate is considered in addition to the grasping success rate.

FIG. 19 is a diagram showing an image detected when installation is successful. Comparing the image in which the work is placed at the target position and the image in which the work is placed on the work table 10 after being actually gripped by the hand 4, they match. If there is no inconsistent part in this way, it is determined that the installation has been successful.

FIG. 20 is a diagram showing an image detected when installation fails. When comparing the image of the work placed at the target position and the image of the work actually held by the hand 4 and placed on the work table 10, a discrepancy occurs. If there is such a mismatch, it is determined that the installation has failed.

As shown in FIGS. 19 and 20, the work information processing device 6B compares the image of the work placed at the target position that has been photographed in advance and stored in the data storage unit 67 with the image of the work after the setting operation. If the two images match, it is determined to be successful, and if they do not match, it is determined to be a failure.

The gripping device according to the third embodiment has the same effects as the gripping device according to the first embodiment, and can improve the installation success rate even when the installation success rate is not high due to poor workpiece gripping positions. .

It should be noted that a gripping device obtained by combining Embodiments 2 and 3 may be used.
The embodiments disclosed this time should be considered as examples and not restrictive in all respects. The scope of the present invention is indicated by the scope of the claims rather than the description of the above-described embodiments, and is intended to include all modifications within the scope and meaning equivalent to the scope of the claims.

1, 1A, 1B gripping device, 2, 2A, 2B camera, 3 three-dimensional sensor, 4 hand, 5 positioning mechanism, 6, 6A, 6B work information processing device, 7 work tray, 8 shaking table, 10 work workbench, 41, 42 fingers, 61 image input unit, 62 workpiece detection unit, 63 gripping position candidate detection unit, 64 gripping position selection unit, 65 gripping movement creation/non-creation determination unit, 66 command output unit, 67 data storage unit.

Claims (6)

an imaging device for observing the surface of the workpiece;
a three-dimensional sensor that measures coordinate data of points on the surface of the workpiece;
a hand for gripping the workpiece;
a positioning mechanism that positions the hand in a graspable posture;
a work information processing device that detects an optimum gripping position of the work using the photographing device and the three-dimensional sensor, and controls the positioning mechanism and the hand;
The workpiece information processing device determines a gripping position of the workpiece based on coordinate data acquired from the three-dimensional sensor or image data acquired from the imaging device, and the hand grips the workpiece based on the gripping position. The positioning mechanism and the hand are operated so that the hand is photographed by the photographing device, and it is determined whether or not the workpiece is actually gripped based on the acquired image data, and the determination result and the grip calculating a gripping success rate at the gripping position by accumulating the relationship with the position, determining a gripping position to be adopted in subsequent gripping operations based on the gripping success rate ;
The work information processing device includes:
a workpiece detection unit that detects the position of the workpiece to be gripped using the coordinate data acquired from the three-dimensional sensor or the image data acquired from the imaging device;
a gripping position candidate detection unit that determines interference between an object surrounding the workpiece and the hand based on the position of the workpiece detected by the workpiece detection unit, and detects one or more gripping position candidates;
a gripping position selection unit that selects the optimum gripping position from among the gripping position candidates extracted by the gripping position candidate detection unit;
a command output unit that outputs a positioning command or a gripping/releasing command to the positioning mechanism or the hand based on the optimum gripping position;
a grasping motion creation failure determination unit that determines whether or not the workpiece is actually grasped based on image data acquired from the photographing device by image processing;
The gripping position candidate detection unit obtains the gripping success rate for the gripping position candidates based on machine learning results using data acquired from the three-dimensional sensor or the imaging device and determination results of the gripping movement creation failure determination unit. ,
The gripping position selection unit selects a candidate that maximizes the gripping success rate as the optimal gripping position,
The gripping position candidate detection unit resets at least a part of the machine learning result every predetermined number of gripping operations . The gripping position candidate detection unit includes, as learning data for machine learning, coordinate data acquired from the three-dimensional sensor or image data acquired from the photographing device, a gripping position when a gripping motion is executed, and the gripping motion creation failure determination unit. 2. The grasping device according to claim 1 , wherein the grasping success rate is obtained using the determination result of . When a predetermined N is a natural number, the number of gripping motions for which the gripping position candidate detecting unit does not use the learning result for calculating the gripping success rate is N times,
The gripping position selection unit selects a point closest to the center of gravity position on the image of the workpiece from among the gripping position candidates up to the Nth gripping, and after the Nth gripping is exceeded, the gripping position selection unit uses the machine learning result. 2. The gripping device according to claim 1 , wherein the obtained candidate with the highest gripping success rate is selected as the optimum gripping position. The gripping position candidate detection unit sequentially performs the machine learning using the coordinate data acquired from the three-dimensional sensor or the image data acquired from the photographing device during the operation of the gripping device and the determination result of the gripping movement creation/non-creation determination unit. 2. The gripping device of claim 1 , wherein the results are updated. further comprising a vibrating table on which a container containing the work is placed,
In the next gripping operation, the workpiece information processing device determines the gripping success rate corresponding to the gripping position determined based on the image of the workpiece in the container captured by the photographing device before gripping by the hand. 2. The gripping device according to claim 1, wherein the shaking table changes the position of the workpiece when the value is lower than the value. The imaging device is
a first camera that captures a place where the workpiece is placed before being gripped by the hand;
a second camera that captures an arrangement location where the work is to be arranged after being gripped by the hand;
The work information processing device determines a placement position of the work in the placement location based on the image data acquired from the second camera, determines whether the placement position matches a reference position, and determines whether or not the placement position matches a reference position. Calculate the placement success rate based on
In the next gripping operation, the workpiece information processing apparatus captures an image of the workpiece placed on the placement location with the first camera before gripping with the hand, and determines a gripping position based on the captured image. 2. The gripping device according to claim 1, wherein adoption or rejection of said gripping position is determined based on said gripping success rate and said placement success rate corresponding to .

Images