CN115246124A

CN115246124A - Object grabbing method and device, electronic equipment and storage medium

Info

Publication number: CN115246124A
Application number: CN202111483974.6A
Authority: CN
Inventors: 朱溪女; 丁有爽; 邵天兰
Original assignee: Mech Mind Robotics Technologies Co Ltd
Current assignee: Mech Mind Robotics Technologies Co Ltd
Priority date: 2021-12-07
Filing date: 2021-12-07
Publication date: 2022-10-28

Abstract

The invention provides an object grabbing method, which comprises the following steps: acquiring a first object pose of a first target object, wherein the first target object comprises an object determined from N objects to be grabbed; calculating pose distances between the first object pose and at least one second object pose of M second object poses, wherein the M second object poses comprise object poses of M failed grabbing objects of the N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N; and stopping grabbing the first target object under the condition that any one of the calculated pose distances is smaller than a first distance threshold. The invention also provides an object grabbing device, electronic equipment and a storage medium.

Description

Object grabbing method and device, electronic equipment and storage medium

Technical Field

The invention relates to the field of intelligent logistics, in particular to an object grabbing method and device, electronic equipment and a storage medium.

Background

With the development of automation technology, work is gradually performed by automated machines instead of human beings in some scenes. For example, a robot arm is used to grasp an object and then move the object to a target position, instead of manually carrying the object.

Generally, when the mechanical arm grabs an object, there may be a situation of grabbing failure, and the result calculated by the control algorithm indicates that the object which is currently grabbed in failure has the highest priority, and the grabbing path is planned successfully. The robot arm may continuously repeat attempts to grasp after a grasping failure. If the object which fails to be grabbed currently does not have the possibility of being grabbed successfully, the repeated grabbing mode of the mechanical arm can cause the consumption of time and resources, and the overall grabbing efficiency is reduced.

Disclosure of Invention

In view of the above problems, the present invention provides an object grasping method, an apparatus, an electronic device, and a storage medium, which can improve the overall grasping efficiency.

One aspect of the embodiments of the present invention provides an object grasping method, including: acquiring a first object pose of a first target object, wherein the first target object comprises an object determined from N objects to be grabbed; calculating pose distances between the first object pose and at least one second object pose of M second object poses, wherein the M second object poses comprise object poses of M failed grabbing objects of the N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N; and stopping grabbing the first target object under the condition that any one of the calculated pose distances is smaller than a first distance threshold.

According to an embodiment of the invention, the method further comprises: and grabbing the first target object under the condition that any one of the pose distances obtained through calculation is larger than or equal to a first distance threshold value.

According to an embodiment of the invention, said grabbing said first target object comprises: grabbing the first target object based on a first grab pose of the first target object, wherein the first grab pose is the same as or different from the first object pose.

According to an embodiment of the present invention, the grasping the first target object based on the first grasping pose of the first target object includes: controlling a sucker to suck the first target object at the first grabbing pose; or the clamping jaw is controlled to clamp the first target object in the first grabbing pose.

According to an embodiment of the present invention, the M second object poses are recorded in a failure record queue, and in a case where the M second object poses are not recorded in the failure record queue before the acquiring of the first object pose of the first target object, the method further includes: acquiring a third object pose of a second target object and a second grabbing pose, wherein the third object pose is the same as or different from the second grabbing pose; grabbing the second target object based on the second grabbing pose; and under the condition that the second target object fails to be grabbed, the second target object is a failed grabbing object, and the third object pose is used as the second object pose and written into the failure record queue.

According to an embodiment of the invention, the method further comprises: and under the condition that the first target object fails to be grabbed, the first target object is a failed grabbing object, and the first object pose is written into the failure record queue as the second object pose.

According to an embodiment of the invention, the method further comprises: setting the maximum number of records of the failed record queue to be less than or equal to N-1; before writing the first object pose as the second object pose into the failure record queue, further comprising: and if M is equal to N-1, deleting the second object pose recorded earliest in the M second object poses from the failure record queue.

According to an embodiment of the present invention, the method further includes determining the first target object, specifically including: s objects to be determined are determined from the N objects to be grabbed, wherein S is an integer larger than or equal to 1, and S is smaller than or equal to N; determining the grabbing priority of S to-be-determined target objects; and taking the object with the highest priority in the S undetermined target objects as the first target object.

According to an embodiment of the invention, the method further comprises: and writing the object pose of each object in the S pending target objects into a priority queue based on the sequence of the grabbing priorities, wherein the priority queue is used for determining the grabbing sequence of the S pending target objects.

According to an embodiment of the present invention, the stopping of grabbing the first target object in the case where any one of the pose distances obtained by the calculation is smaller than a first distance threshold value includes: and deleting the first object pose from the priority queue under the condition that any calculated pose distance is smaller than a second distance threshold, wherein the second distance threshold is smaller than the first distance threshold.

According to an embodiment of the present invention, the stopping of grabbing the first target object in the case where any one of the pose distances obtained by the calculation is smaller than a first distance threshold further includes: and when any calculated pose distance is greater than or equal to a second distance threshold and smaller than the first distance threshold, reducing the priority order of the first object pose in the priority queue.

Another aspect of an embodiment of the present invention provides an object grasping apparatus, including: the system comprises a pose acquisition module, a pose acquisition module and a pose acquisition module, wherein the pose acquisition module is used for acquiring a first object pose of a first target object, and the first target object comprises an object determined from N objects to be grabbed; a pose distance module, configured to calculate a pose distance between the first object pose and at least one of M second object poses, where the M second object poses include object poses of M failed grabbing objects of the N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N; and the object grabbing module is used for stopping grabbing the first target object under the condition that any one of the pose distances obtained through calculation is smaller than a first distance threshold value.

Another aspect of an embodiment of the present invention provides an electronic device, including: one or more processors; a storage device to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method as described above.

Yet another aspect of embodiments of the present invention provides a computer-readable storage medium having stored thereon executable instructions, which when executed by a processor, cause the processor to perform the method as described above.

Compared with the mode that grabbing is continuously and repeatedly attempted after the mechanical arm fails to grab the object in the related art, the method and the device can firstly acquire the first object pose of the first target object, then use the object poses of M objects with grabbing failures as reference, and determine that the first target object may fail to grab under the condition that the pose distance between the first object pose and at least one second object pose in the M second object poses is smaller than the first distance threshold, so that grabbing is stopped, time and resource consumption caused by repeated grabbing is avoided, and the overall grabbing efficiency is improved.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention, which proceeds with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates a flow chart of a method of object grabbing, according to an embodiment of the present invention;

FIG. 2 schematically illustrates a flow diagram for writing a third object pose to a failure record queue according to an embodiment of the invention;

FIG. 3 schematically illustrates a flow diagram for writing a first object pose to a failure record queue according to an embodiment of the invention;

FIG. 4 schematically illustrates a flow chart for determining a first target object according to an embodiment of the invention;

FIG. 5 is a schematic diagram that schematically illustrates a priority queue and a failure record queue, in accordance with an embodiment of the present invention;

FIG. 6 schematically illustrates a flow chart of an object grasping method according to another embodiment of the present invention;

fig. 7 is a block diagram schematically showing the structure of an object grasping apparatus according to an embodiment of the present invention;

FIG. 8 shows a schematic structural diagram of a computing device according to an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Fig. 1 schematically shows a flow chart of an object grasping method according to an embodiment of the present invention.

As shown in fig. 1, the object grasping method of the embodiment includes operations S110 to S150.

In operation S110, a first object pose of a first target object is obtained, where the first target object includes an object determined from N objects to be grabbed.

For example, the point cloud data of N objects to be grabbed are obtained by taking a picture or scanning with a laser radar. And then, processing the point cloud data by adopting a target detection algorithm (such as a classification algorithm or a template matching algorithm) to determine a first target object. Wherein the first object pose may be obtained from the point cloud data.

In operation S120, a pose distance between the first object pose and at least one of M second object poses is calculated, where the M second object poses include object poses of M failed grabbing objects among N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N.

For example, the object pose can be represented by a rotational translation matrix [ x, y, z, α, β, γ ], where x, y, z represents the displacement of three degrees of freedom and α, β, γ represents the spatial rotation of three degrees of freedom. The pose distance may be a euclidean distance between calculating a matrix representing the poses of the first object and a matrix passing any of the poses of the second object. The first object pose may include a first object position and a first object pose, and is used to represent the position and orientation of the object.

The object which is failed to be grabbed refers to that the objects are grabbed historically, but the grabbing is not successful and still exists among the N objects to be grabbed. The gripping failure may mean that the gripper makes contact with an object to be gripped and applies an external force, and a normal gripping operation is performed, but the object is not displaced or rotated, or only slightly displaced or rotated, and does not leave the current position. The grabbing failure may also mean that an object to be grabbed is not grabbed from the position to be grabbed to the target position.

In operation S130, it is determined whether any of the calculated pose distances is smaller than a first distance threshold. If yes, operation S140 is performed. If not, operation S150 is performed.

In operation S140, in a case where any one of the calculated pose distances is smaller than the first distance threshold, the grasping of the first target object is stopped.

The first distance threshold D1 may be used to characterize a first magnitude of movement of the object. Different first distance thresholds D1 may be defined for different object types. For example, for an object with a small volume, grabbing may be attempted when the movement amplitude is likely to be small. For large objects, it is possible that the grabbing may be attempted only after a large movement amplitude has been generated.

In some embodiments, different types of objects may be preset with different first distance thresholds D1 corresponding to one another. For example, when a first target object is acquired, the type of the first target object may be determined, then a corresponding first distance threshold D1 is obtained based on the type, and finally each pose distance is compared with the first distance threshold D1. The object type can be classified according to volume, length, height and the like, and can also be specific object name, type and model. It should be noted that the object to be grabbed referred to in the present invention may be a regular object, that is, an object whose shape can be expressed by a formula, such as a rectangle, a circle, etc. Or an irregular object.

In the related art, for example, when the robot arm sucks the object through the suction cup, if the suction cup cannot be vacuumized due to poor surface conditions of the sucked area of the object, the suction may fail. When a robot arm grips an object by a gripper, for example, a heavy metal part such as an automobile crankshaft, some of the crankshafts may be tightly clamped due to the stacked crankshafts, and may not grip the object although they can be gripped successfully. From the view of the vision calculation result, the grabbing priority of the objects with failed grabbing is higher (for example, the objects at the high position are grabbed first and then the objects at the low position are grabbed), so that the objects are grabbed preferentially at each time. From the viewpoint of motion, the grab planning is successful, but the actual grab fails. When the above situation occurs, the robot always repeatedly grabs the same object, resulting in a decrease in overall grabbing efficiency, which consumes time and grabbing resources.

Compared with the mode that grabbing is continuously and repeatedly attempted after the mechanical arm fails to grab the object in the related art, the method can firstly acquire the first object pose of the first target object, then take the object poses of M objects with grabbing failures as reference, and determine that the first target object is likely to fail to grab under the condition that the pose distance between the first object pose and at least one second object pose in the M second object poses is smaller than a first distance threshold value, so that grabbing is stopped, time and resource consumption caused by repeated grabbing is avoided, and the overall grabbing efficiency is improved.

In operation S150, in a case where any one of the calculated pose distances is greater than or equal to the first distance threshold, the first target object is grasped.

When the pose distance is greater than or equal to the first distance threshold D1, it may be determined that the first target object has not attempted to be grabbed, or although grabbing fails, the moving amplitude meets the condition, and grabbing may be attempted again. Therefore, the pose distance between the first object pose and each second object is calculated and compared with the first distance threshold. On one hand, objects which are once grabbed and failed and have no change in object pose can be filtered. On the other hand, no matter the object is not grabbed or the object which can be grabbed repeatedly can be grabbed, the grabbing operation can be carried out, and omission is avoided. Thereby improving the single grabbing success rate and the overall grabbing efficiency.

According to an embodiment of the present invention, grabbing the first target object in operation S150 includes: and grabbing the first target object based on a first grabbing pose of the first target object, wherein the first grabbing pose is the same as or different from the first object pose.

The object pose and the grab pose are, for example, inherent attributes of the object. The pose of the object can be defined at any position, such as at the center of the object or not, above the object or outside the object. After being defined, the object pose is uniquely determined for the object. In practical applications, where the object pose is defined, the object pose can be set by software (such as Mech-Vision software), and after the crankshaft model is taken, mech-Vision can edit the object pose on the loaded crankshaft.

The grabbing poses can be determined according to the camera or radar position, the grabbing mode of the mechanical arm and other factors, and the number of the grabbing poses can be one or more. Taking the crankshaft as an example, generally, the front part, the middle part and the tail part of the crankshaft are respectively provided with a groove for grabbing, and then grabbing poses are defined at the three grabbing positions. Because the object pose can be set according to the actual condition, the object pose and the grabbing pose can be the same or different. The grab pose may be acquired simultaneously with the object pose, or may be acquired after or before the object pose. It should be noted that the front and rear portions of the crankshaft can be flexibly defined according to the actual coordinate system, with the middle portion being located between the front and rear portions.

Therefore, after the first target object is grasped, the mechanical arm is controlled to grasp the first target object according to the first grasping pose, for example, when the first target object is a crankshaft, the first grasping pose can be at the position of the middle groove, and then the mechanical arm grasps the middle groove of the crankshaft.

In an alternative embodiment, grabbing the first target object based on the first grab pose of the first target object includes controlling the suction cups to suck the first target object in the first grab pose.

The sucking disc can be used for sucking objects such as plate-shaped objects, box bodies (such as containers), light objects (such as small and medium-sized express boxes) and the like.

Another optional embodiment is that grabbing the first target object based on the first grab pose of the first target object includes controlling the gripping jaws to grip the first target object in the first grab pose.

For regular or irregular objects with grooves therein adapted for gripping, gripping may be performed using matching jaws.

In some embodiments, the end of the robotic arm may be equipped with multiple types of grippers, such as suction cups and jaws at the same time. The specific type of object may be determined by a classification algorithm and then an adapted fixture selected based on the type. Image recognition may also be performed on the surface of the object to determine that a portion of the area on the object is suitable for use with a fixture of some kind. If the recess of the crankshaft can be detected, the suitability of the clamping jaw for use can be determined.

FIG. 2 schematically illustrates a flow diagram for writing a third object pose to a failure record queue according to an embodiment of the invention. And M second object poses are recorded in the failure record queue.

As shown in fig. 2, before operation S110 is performed, for example, before the first object pose of the first target object is acquired, in a case where M second object poses are not recorded in the failure record queue, operations S210 to S230 may be performed.

In operation S210, a third object pose of the second target object and a second capture pose are obtained, where the third object pose is the same as or different from the second capture pose.

In operation S220, the second target object is grabbed based on the second grabbing pose.

In operation S230, in a case where the grabbing of the second target object fails, the second target object is a failed object, and the third object pose is written as the second object pose in the failure record queue.

A queue here refers to a data structure, a special linear table. And the failure record queue is used for storing the object pose of the failed object. For example, when no data is recorded in the failure record queue, the calculation of the pose distance is meaningless, so that the third object pose and the second capture pose of the second target object are directly obtained. And if the grabbing is successful, continuously and directly grabbing the next target object. And if the grabbing fails, writing the pose of the third object into a failure record queue, calculating the pose distance when the next target object is grabbed, and comparing the pose distance with the first distance threshold.

FIG. 3 schematically illustrates a flow diagram for writing a first object pose to a failure record queue according to an embodiment of the invention.

After performing operation S150, as shown in fig. 3, operations S310 to S340 may also be performed.

In operation S310, it is determined whether the first target object is successfully grasped. If yes, the operation is finished. If not, operation S320 is performed.

In operation S320, it is determined whether the number M of current records in the failed record queue is equal to N-1. If yes, operation S330 is performed. If not, operation S340 is performed.

According to an embodiment of the present invention, the maximum number of records of the failed record queue may be set to be less than or equal to N-1 before operation S320 is performed.

Taking the crankshaft as an example again, N crankshafts to be grabbed are arranged in one logistics box, and after grabbing is finished, the next logistics box is replaced to grab the crankshafts. If the maximum record number of the failed record queue is not limited, the object poses of the failed grabbing crankshafts of other logistics boxes are possibly recorded in the queue under the condition that the mechanical arm continuously runs. In fact, when the crankshaft in the current logistics box is grabbed, the object poses of the crankshafts in other logistics boxes are meaningless, in other words, the comparison result between the calculated pose distance and the first distance threshold value is meaningless. Therefore, the maximum number of records may be set to be less than or equal to N.

In some embodiments, the recorded data in the failure record queue may be emptied based on the completion of the grabbing of the N objects to be grabbed, without setting the maximum number of records.

In operation S330, if M is equal to N-1, the second object pose that was recorded earliest among the M second object poses is deleted from the failed record queue.

Taking N as 3 for example, there are 3 objects to be grabbed. If the maximum record number is 3,3, and the objects to be grabbed are all failed to be grabbed, the position and posture distance between the object position and posture of each object to be grabbed and the object position and posture in the failure record queue is possibly smaller than the first distance threshold value, the mechanical arm possibly stops grabbing any object to be grabbed, and therefore grabbing work cannot be carried out.

If the maximum record number is 2,3 objects to be grabbed are all failed to be grabbed, an extruded queue can be provided. Specifically, if three objects a, B, and C are grabbed, and each time of planning can be successfully grabbed, but each grabbing fails, if the maximum number of records is set to 2, the actual effect is: and recording the object pose of A after the grabbing failure A. And recording the object poses of A and B after the grabbing fails to be performed on the object B. And after the grabbing fails to reach the position C, the A is out of the queue, and the object poses of the B and the C are recorded. At which point a can attempt to grab. Therefore, external force can be continuously applied to the object through the mechanical arm, and the grabbing success rate is improved by changing the position or the posture of the object.

In operation S340, in the case where the grabbing of the first target object fails, the first target object is a failed object, and the first object pose is written into the failure record queue as the second object pose.

According to the embodiment of the invention, when one object fails to be grabbed, the object pose of the object can be written into the failure record queue to be used as the reference for grabbing the object subsequently. For example, if the first target object is an object that fails to be grabbed, the pose distance is calculated next time before the grabbing plan, and the pose distance is compared with the first distance threshold to determine whether grabbing is possible. The situation that the first target object does not move and the second grabbing still fails is avoided.

Fig. 4 schematically shows a flow chart for determining a first target object according to an embodiment of the invention.

As shown in fig. 4, determining the first target object in this embodiment may include operations S410 to S430.

In operation S410, S pending target objects are determined from the N objects to be grabbed, where S is an integer greater than or equal to 1, and S is less than or equal to N.

In operation S420, the grasping priorities of the S pending target objects are determined.

In operation S430, the object with the highest priority among the S pending target objects is taken as the first target object.

For example, when N crankshafts are grabbed in one logistics box, the crankshafts are stacked. And photographing the N crankshafts to obtain the point cloud data of the crankshafts on the uppermost layer. Firstly, target detection is carried out on a plurality of crankshafts on the uppermost layer, the crankshafts are used as S to-be-determined target objects, and S is smaller than or equal to the number of the crankshafts on the uppermost layer. Then, further calculation is performed to determine one of the objects as a first target object. For example, the object with the highest grabbing success rate may be set as the first target object with the highest priority, and the objects may be prioritized according to the grabbing success rate (for example only).

According to the embodiment of the invention, based on the sequencing of the grabbing priorities, the object pose of each object in the S objects to be targeted is written into the priority queue, wherein the priority queue is used for determining the grabbing sequence of the S objects to be targeted.

An optional mode is that the object pose and the grabbing pose of each object in the S objects to be targeted are written into the priority queue correspondingly. The object pose is used for determining the object, and the grabbing pose is used for grabbing the object. And when the grabbing fails, writing the pose of the object into a failure record queue. And the priority sequence of the pose of each object in the priority queue is the grabbing sequence. The mechanical arm can be controlled to grab according to the grabbing pose corresponding to the object pose based on the grabbing sequence.

FIG. 5 is a schematic diagram that schematically illustrates a priority queue and a failure record queue, in accordance with an embodiment of the present invention.

As shown in FIG. 5, a first object pose, and S-1 pending target object poses can be included in the priority queue. The failure record queue comprises second object poses 1-M. The robotic arm performs the grasping in the order of priority in fig. 5. Since the priority of the first object pose is highest, the first target object is grabbed first. Before the grabbing planning is performed, the poses of the first object may be compared with the poses of the M second objects in the failure record queue one by one. As further described below in conjunction with fig. 6.

Fig. 6 schematically shows a flow chart of an object grabbing method according to another embodiment of the present invention.

As shown in fig. 6, the object grasping method of this embodiment may include operations S110 and S150, and may further include operations S610 to S660.

In operation S110, a first object pose of a first target object is acquired.

In operation S610, a pose distance between the first object pose and the ith object pose is calculated. Wherein i has an initial value of 1,i less than or equal to M.

In operation S620, it is determined whether a pose distance between the first object pose and the ith object pose is less than a second distance threshold. If so, operation S630 is performed. If not, operation S640 is performed. Wherein the second distance threshold D2 is smaller than the first distance threshold D1.

The second distance threshold D2 may be used to characterize a second magnitude of movement of the object. For different object types, different second distance thresholds D2 may be defined. In some embodiments, different second distance thresholds D2 may be preset for different types of objects, one for one. For example, when a second target object is acquired, the type of the second target object may be determined, then a corresponding second distance threshold D2 is obtained based on the type, and finally each pose distance is compared with the second distance threshold D2.

In operation S630, in case any one of the calculated pose distances is less than the second distance threshold, the first object pose is deleted from the priority queue.

Referring to fig. 5, the first object pose is deleted from the priority queue, that is, the priority of the pending target object pose 1 becomes the highest, and S-1 object poses are also recorded in the priority queue. Before each grabbing planning, S undetermined target objects may be determined and grabbed in sequence. The second distance threshold D2 has the effect of determining whether the first target object is feasible for attempting to grab. If the moving amplitude of the first target object is smaller than the second moving amplitude, the capturing success rate is considered to be very low, and the capturing priority should not be allocated, so that the capturing at the time is abandoned.

In operation S640, it is determined whether a pose distance between the first object pose and the ith object pose is less than a first distance threshold D1. If so, operation S650 is performed. If not, operation S660 is performed.

In operation S650, in a case where any one of the calculated pose distances is greater than or equal to the second distance threshold and less than the first distance threshold, the priority order of the first object pose in the priority queue is decreased.

When the pose distance is greater than or equal to the second distance threshold and less than the first distance threshold, it is considered that the pose of the first object may have failed to be captured once, but has a certain moving range, and at this time, the first target object may be affected by other objects, such as being pressed on the first target object. After grabbing other objects, trying to grab the first target object again will have a greater success rate of grabbing. Referring to fig. 5, the first object pose may be inserted between any two of the positions 1 to S-1 of the undetermined target object, for example, the first object pose may be inserted between the position 1 of the undetermined target object and the position 2 of the undetermined target object, that is, a priority order is turned down.

The number of priorities for lowering the pose of the first object can be determined according to the actual running condition. Taking the crankshaft as an example, it can be observed how many crankshafts are grasped, and the crankshafts in the logistics box can be loosened, so that the degradation sequence is set to the value. Because the crankshafts that can be ranked ahead are generally the more graspable crankshafts, their grasping priority is reduced because they are stuck too tightly to actually grasp. But when the crankshaft looses (i.e. the object posture changes), the crankshaft can be grasped first.

In some embodiments, referring to FIG. 5, if the pending target object poses 1-S-1 are both deleted from the priority queue due to a successful or failed grab. Even if the priority order of the first object pose is reduced to the pose S-1 of the undetermined target object, under the condition that only the first object pose exists in the priority queue, the pose distance between the first object pose and the ith second object pose is still larger than or equal to the second distance threshold and smaller than the first distance threshold, and if the priority of the first object pose cannot be reduced, the first object pose is deleted.

In operation S660, it is determined whether i is less than M. If yes, let i = i +1, and perform operation S610. If not, operation S150 is performed.

Referring to fig. 5, the first object pose is compared with each of the second object poses 1 to M to obtain a pose distance, and it is determined whether the first object pose can be grasped. After the fact that grabbing can be performed is determined, factors such as a grabbing pose, a grabbing path, a grabbing angle and a grabbing fixture are planned. For example, the pose of the first object is obtained first, after grabbing can be performed, the pose of the first object is obtained, and grabbing paths, angles, clamps and the like are determined, so that computing resources can be saved.

In operation S150, a first target object is grasped.

According to the embodiment of the invention, the stopping of grabbing the first target object can be divided into two measures of deletion in the priority queue or reduction of the priority order in the priority queue. The situation that the crankshaft is stuck very dead in some scenes and cannot be loosened in any way can be flexibly handled, and at the moment, the crankshaft is not tried any more as long as the crankshaft is tried to be grabbed but not grabbed. Or in other scenes, along with the grabbing of the crankshafts, the unhooked crankshafts can be loosened under the influence of grabbing, so that the attempted but unsuccessfully grabbed crankshafts are only subjected to priority reduction, and grabbing can be re-attempted after the grabbing is performed to a certain degree. Therefore, the grabbing device can be adapted to complex and changeable working conditions in the grabbing process, and the overall grabbing efficiency is improved.

Based on the object grabbing method, the invention further provides an object grabbing device. The apparatus will be described in detail below with reference to fig. 7.

Fig. 7 schematically shows a block diagram of the object grasping apparatus 700 according to the embodiment of the present invention.

As shown in fig. 7, the object grasping apparatus 700 of this embodiment includes a pose acquisition module 710, a pose distance module 720, and an object grasping module 730.

The pose acquisition module 710 may perform, for example, operation S110 for acquiring a first object pose of a first target object, wherein the first target object includes an object determined from the N objects to be grabbed.

The pose distance module 720 may perform operation S120, for example, for calculating a pose distance between a first object pose and at least one of M second object poses, where the M second object poses include object poses of M failed grab objects of N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N.

The object grasping module 730 may perform, for example, operation S140 for stopping grasping the first target object in the case where any one of the calculated pose distances is smaller than the first distance threshold.

The object grasping module 730 may further perform operation S150 for grasping the first target object in a case where any one of the calculated pose distances is greater than or equal to the first distance threshold. Wherein grabbing the first target object comprises: and grabbing the first target object based on a first grabbing pose of the first target object, wherein the first grabbing pose is the same as or different from the first object pose.

The object capture module 730 may also be configured to control the suction cups to capture the first target object in the first capture pose. Or the clamping jaw is controlled to clamp the first target object in the first grabbing pose.

The object grabbing device 700 may further include a failure record queue module, where M second object poses are recorded, and before the first object pose of the first target object is acquired, in a case where M second object poses are not recorded in the failure record queue, the failure record queue module is configured to: and acquiring a third object pose of the second target object and a second grabbing pose, wherein the third object pose is the same as or different from the second grabbing pose. And grabbing the second target object based on the second grabbing pose. And under the condition that the second target object fails to be grabbed, the second target object is a failed object to be grabbed, and the third object pose is written into a failure record queue as the second object pose.

The object grasping apparatus 700 may further include a first writing module configured to, in a case where the grasping of the first target object fails, write the first object pose as a second object pose into a failure record queue, where the first target object is a failed object. And also for setting the maximum number of records of the failed record queue to be less than or equal to N-1. Before writing the first object pose as the second object pose into the failure record queue, the method further comprises: and if the M is equal to N-1, deleting the second object pose recorded earliest in the M second object poses from the failure record queue.

The object grabbing device 700 may further include a target determining module, configured to determine S objects to be targeted from the N objects to be grabbed, where S is an integer greater than or equal to 1, and S is less than or equal to N. And determining the grabbing priority of S objects to be targeted. And taking the object with the highest priority in the S undetermined target objects as a first target object.

The object grabbing device 700 may further include a second writing module, configured to write the object pose of each of the S pending target objects into a priority queue based on the ranking of the grabbing priorities, where the priority queue is used to determine a grabbing order of the S pending target objects. And the processor is further used for deleting the first object pose from the priority queue under the condition that any pose distance obtained by calculation is smaller than a second distance threshold value, wherein the second distance threshold value is smaller than the first distance threshold value. And the priority sequence of the first object pose in the priority queue is reduced under the condition that any pose distance obtained by calculation is greater than or equal to the second distance threshold and smaller than the first distance threshold.

Compared with the method of continuously and repeatedly attempting to grab objects after the robot arm fails to grab the objects in the related art, the object grabbing device 700 in the embodiment of the invention can firstly acquire the first object pose of the first target object, and then, by taking the object poses of the M objects with grabbing failures as reference, under the condition that the pose distance between the first object pose and at least one second object pose of the M second object poses is smaller than the first distance threshold, it can be determined that the first target object may have grabbing failures, so that grabbing is stopped, time and resource consumption caused by repeated attempts are avoided, and the overall grabbing efficiency is improved.

Fig. 8 is a schematic structural diagram of a computing device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in fig. 8, the computing device may include: a processor (processor) 802, a Communications Interface 804, a memory 806, and a communication bus 808.

Wherein:

the processor 802, communication interface 804, and memory 806 communicate with one another via a communication bus 808.

A communication interface 804 for communicating with network elements of other devices, such as clients or other servers.

The processor 802 is configured to execute the program 810, and may specifically execute the relevant steps in the above-described embodiment of the object capture method.

In particular, the program 810 may include program code comprising computer operating instructions.

The processor 802 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 806 stores a program 810. The memory 806 may include high-speed RAM memory, and may also include non-volatile memory (nonvolatile memory), such as at least one disk memory.

The program 810 may be specifically configured to cause the processor 802 to perform an object grasping method in any of the method embodiments described above. For specific implementation of each step in the program 810, reference may be made to corresponding steps and corresponding descriptions in units in the above-described object capture embodiments, which are not described herein again. It is clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described again here

The present invention also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the present invention.

The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus.

Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the method of the invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment.

Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limited to the order of execution unless otherwise specified.

Claims

1. An object grasping method, comprising:

acquiring a first object pose of a first target object, wherein the first target object comprises an object determined from N objects to be grabbed;

calculating pose distances between the first object pose and at least one second object pose of M second object poses, wherein the M second object poses comprise object poses of M failed grabbing objects of the N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N;

and stopping grabbing the first target object under the condition that any one of the calculated pose distances is smaller than a first distance threshold.

2. The method of claim 1, wherein the method further comprises:

and grabbing the first target object under the condition that any one of the pose distances obtained through calculation is larger than or equal to a first distance threshold value.

3. The method of claim 2, wherein said grabbing the first target object comprises:

grabbing the first target object based on a first grab pose of the first target object, wherein the first grab pose is the same as or different from the first object pose.

4. The method of claim 3, wherein the grabbing the first target object based on the first grab pose of the first target object comprises:

controlling a sucker to suck the first target object at the first grabbing pose; or

And controlling the clamping jaw to clamp the first target object at the first grabbing pose.

5. The method of claim 2, wherein the M second object poses are recorded in a fail record queue, and wherein, in the event that the M second object poses are not recorded in the fail record queue prior to the acquiring the first object pose of the first target object, the method further comprises:

acquiring a third object pose of a second target object and a second grabbing pose, wherein the third object pose is the same as or different from the second grabbing pose;

grabbing the second target object based on the second grabbing pose;

and under the condition that the second target object fails to be grabbed, the second target object is a failed grabbing object, and the third object pose is used as the second object pose and written into the failure record queue.

6. The method of claim 5, wherein the method further comprises:

and under the condition that the first target object fails to be grabbed, the first target object is a failed grabbing object, and the first object pose is written into the failure record queue as the second object pose.

7. The method of claim 6, wherein the method further comprises:

setting the maximum number of records of the failed record queue to be less than or equal to N-1;

before writing the first object pose as the second object pose to the failure record queue, further comprising:

and if M is equal to N-1, deleting the second object pose recorded earliest in the M second object poses from the failure record queue.

8. The method according to claim 1, wherein the method further comprises determining the first target object, specifically comprising:

s objects to be determined are determined from the N objects to be grabbed, wherein S is an integer larger than or equal to 1, and S is smaller than or equal to N;

determining the grabbing priority of S to-be-determined target objects;

and taking the object with the highest priority in the S undetermined target objects as the first target object.

9. The method of claim 8, wherein the method further comprises:

and writing the object pose of each object in the S pending target objects into a priority queue based on the sequencing of the grabbing priorities, wherein the priority queue is used for determining the grabbing sequence of the S pending target objects.

10. The method according to claim 9, wherein the stopping of grabbing the first target object in the case where any one of the pose distances obtained by the calculation is smaller than a first distance threshold value comprises:

and deleting the first object pose from the priority queue under the condition that any calculated pose distance is smaller than a second distance threshold, wherein the second distance threshold is smaller than the first distance threshold.

11. The method according to claim 10, wherein the stopping of grabbing the first target object in the case where any one of the calculated pose distances is less than a first distance threshold further comprises:

and when any calculated pose distance is greater than or equal to a second distance threshold and smaller than the first distance threshold, reducing the priority order of the first object pose in the priority queue.

12. An object grasping device, comprising:

the system comprises a pose acquisition module, a pose acquisition module and a pose acquisition module, wherein the pose acquisition module is used for acquiring a first object pose of a first target object, and the first target object comprises an object determined from N objects to be grabbed;

a pose distance module, configured to calculate a pose distance between the first object pose and at least one of M second object poses, where the M second object poses include object poses of M failed grabbing objects of the N objects to be grabbed, M or N is an integer greater than or equal to 1, and M is less than or equal to N;

and the object grabbing module is used for stopping grabbing the first target object under the condition that any one of the pose distances obtained through calculation is smaller than a first distance threshold value.

13. An electronic device, comprising:

one or more processors;

a storage device to store one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-11.

14. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 11.