CN113524187B - Method and device for determining workpiece grabbing sequence, computer equipment and medium - Google Patents
Method and device for determining workpiece grabbing sequence, computer equipment and medium Download PDFInfo
- Publication number
- CN113524187B CN113524187B CN202110819471.5A CN202110819471A CN113524187B CN 113524187 B CN113524187 B CN 113524187B CN 202110819471 A CN202110819471 A CN 202110819471A CN 113524187 B CN113524187 B CN 113524187B
- Authority
- CN
- China
- Prior art keywords
- workpiece
- material frame
- point cloud
- point
- preset position
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
Abstract
The invention discloses a method and a device for determining a workpiece grabbing sequence, computer equipment and a medium, wherein the method comprises the following steps: identifying and obtaining a material frame point cloud and a plurality of workpiece point clouds; acquiring position information of a preset position point of the material frame point cloud and position information of a plurality of workpiece point clouds; and calculating the distance from each workpiece to the preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame, so that the workpieces close to the central position of the material frame can be preferentially grabbed, and the grabbing stability is ensured.
Description
Technical Field
The invention relates to the technical field of robots, in particular to a method and a device for determining a workpiece grabbing sequence, computer equipment and a medium.
Background
With the development of the era, 3D vision technology is increasingly used in industrial automation sorting scenarios for sorting of workpieces. The workpieces are recognized and then picked out according to a certain sequence, and the current method for determining the workpiece picking sequence is to arrange the detected workpieces from high to low according to the positions of the workpieces, and start to pick from the workpiece with the highest position during picking.
Although the prior art can solve most problems, due to the existence of the material frame, when a workpiece close to the corner of the material frame is grabbed, the mechanical arm is easy to collide with the material frame, so that the grabbing stability and reliability are affected, and when the texture of the workpiece is fragile, the workpiece is easy to damage.
Therefore, how to avoid grabbing the workpiece at the corner position and preferentially grab the workpiece close to the center position of the material frame becomes a problem to be solved urgently at present.
Disclosure of Invention
The invention aims to provide a method, a device, computer equipment and a medium for determining a workpiece grabbing sequence, which are used for solving the problem of poor workpiece grabbing stability caused by the fact that workpieces close to the center of a material frame cannot be grabbed preferentially in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for determining a workpiece gripping sequence, including the steps of:
identifying and obtaining a material frame point cloud and a plurality of workpiece point clouds;
acquiring position information of preset position points of the material frame point cloud and position information of a plurality of workpiece point clouds;
and calculating the distance from each workpiece to the preset position point of the material frame based on the position information of the preset position points of the material frame point cloud and the position information of a plurality of workpiece point clouds, and determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame.
Preferably, the position information of the preset position point of the frame point cloud and the position information of the workpiece point clouds comprise:
acquiring preset position points of the material frame point cloud and coordinates of a plurality of workpiece point clouds under a camera coordinate system;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system.
Further, the determining the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud according to the preset position point of the material frame point cloud and the coordinates of the workpiece point cloud under the camera coordinate system specifically includes:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under the material frame coordinate system; the material frame coordinate system is a coordinate system taking a preset position point of the material frame point cloud as a coordinate origin;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the material frame coordinate system.
Further, the step of converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under the camera coordinate system into the coordinates under the material frame coordinate system specifically includes:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system;
calculating a conversion relation between a world coordinate system and a material frame coordinate system according to the coordinates of the preset position point of the material frame point cloud in the world coordinate system and the coordinates of the preset position point of the material frame point cloud in the material frame coordinate system;
and calculating the coordinates of the workpiece point clouds in the material frame coordinate system according to the conversion relation between the world coordinate system and the material frame coordinate system.
Preferably, the step of calculating the distance from each workpiece to the preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, wherein the step of determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame specifically comprises the steps of:
judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse with a preset position point of the material frame as a center;
calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame.
Further, the length of the long axis of the ellipse is half of the length of the material frame, and the length of the short axis of the ellipse is half of the width of the material frame.
Further, the step of determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame specifically comprises:
and taking the workpiece with the minimum distance as the highest priority to grasp the workpieces according to the sequence from small to large from the distance from each workpiece in the elliptical area to the preset position point of the material frame.
In a second aspect, the present invention also provides an apparatus for determining a workpiece gripping sequence, including:
the point cloud identification module is used for identifying and acquiring a material frame point cloud and a plurality of workpiece point clouds;
the position acquisition module is used for acquiring position information of preset position points of the material frame point cloud and position information of the plurality of workpiece point clouds;
and the grabbing sequence determining module is used for calculating the distance from each workpiece to the preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the grabbing sequence of the workpieces based on the distance from each workpiece to the preset position point of the material frame.
Preferably, the position acquiring module specifically includes:
the coordinate acquisition unit is used for acquiring preset position points of the material frame point cloud and coordinates of the plurality of workpiece point clouds under a camera coordinate system;
and the position acquisition unit is used for determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system.
Further, the position acquiring unit specifically includes:
the coordinate conversion subunit is used for converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under a camera coordinate system into coordinates under the material frame coordinate system; the material frame coordinate system is a coordinate system taking a preset position point of the material frame point cloud as a coordinate origin;
and the position determining subunit is used for determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the material frame coordinate system.
Further, the coordinate transformation subunit is specifically configured to:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system;
calculating a conversion relation between a world coordinate system and a material frame coordinate system according to the coordinates of the preset position points of the material frame point cloud in the world coordinate system and the coordinates of the preset position points of the material frame point cloud in the material frame coordinate system;
and calculating the coordinates of the workpiece point clouds in the material frame coordinate system according to the conversion relation between the world coordinate system and the material frame coordinate system.
Further, the grasping order determining module specifically includes:
the position judging unit is used for judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse taking a preset position point of the material frame as a center;
the distance calculation unit is used for calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and the sequence determining unit is used for determining the workpiece grabbing sequence according to the distance relation between each workpiece in the elliptical area and a preset position point of the material frame.
Further, the length of the long axis of the ellipse is half of the length of the material frame, and the length of the short axis of the ellipse is half of the width of the material frame.
Further, the order determination unit is specifically configured to:
and taking the workpiece with the minimum distance as the highest priority to grasp the workpieces according to the sequence from small to large from the distance from each workpiece in the elliptical area to the preset position point of the material frame.
In a third aspect, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods when executing the computer program.
In a fourth aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described above.
Compared with the prior art, the method, the device, the computer equipment and the medium for determining the workpiece grabbing sequence provided by the invention have the advantages that the position information of the center of the material frame and the position information of each workpiece are determined firstly, the distance between each workpiece and the preset position point of the material frame is determined according to the position information of the preset position point of the material frame and the position information of each workpiece, and the workpiece grabbing sequence is sequenced according to the distance between each workpiece and the preset position point of the material frame, so that the reasonable grabbing sequence of the workpieces is determined.
Meanwhile, after the workpiece is grabbed according to the method disclosed by the invention, the workpiece at the corner of the material frame is grabbed finally, and other workpieces are grabbed, so that the possibility that the workpiece at the corner is shielded by other workpieces is reduced, and the grabbing stability is also effectively improved.
Drawings
FIG. 1 is a flowchart of one embodiment of a method for determining a workpiece capture sequence according to the present invention;
FIG. 2 is a schematic view of the placement of a material frame and a workpiece according to an embodiment of the present invention;
FIG. 3 is a schematic view of an embodiment of the distance between the center of the frame and the workpiece according to the present invention;
FIG. 4 is a schematic structural diagram illustrating an embodiment of an apparatus for determining a workpiece capture sequence according to the present invention;
fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of the invention in any way.
Like reference numerals refer to like elements throughout the specification. The expression "and/or" includes any and all combinations of one or more of the associated listed items. In the drawings, the thickness, size, and shape of an object have been slightly exaggerated for convenience of explanation. The figures are purely diagrammatic and not drawn to scale.
It will be further understood that the terms "comprises," "comprising," "includes," "including," "has," "includes" and/or "including," when used in this specification, specify the presence of stated features, steps, integers, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, integers, operations, elements, components, and/or groups thereof.
The terms "substantially", "about" and the like as used in the specification are used as terms of approximation and not as terms of degree, and are intended to account for inherent deviations in measured or calculated values that would be recognized by one of ordinary skill in the art.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Example one
As shown in fig. 1, the present invention discloses a method for determining a workpiece grabbing sequence, comprising the following steps:
s100, identifying and obtaining a material frame point cloud and a plurality of workpiece point clouds;
s200, acquiring position information of preset position points of the material frame point cloud and position information of a plurality of workpiece point clouds;
s300, calculating the distance from each workpiece to the preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame.
In the embodiment, the position information of the preset position point of the material frame and the position information of each workpiece are firstly determined, then the distance between each workpiece and the preset position point of the material frame is determined according to the position information of the preset position point of the material frame and the position information of each workpiece, and finally the workpiece grabbing sequence is sequenced according to the distance between each workpiece and the material frame, so that a reasonable workpiece grabbing sequence is determined. Meanwhile, after the workpiece is grabbed according to the method disclosed by the invention, the workpiece at the corner of the material frame is grabbed finally, and other workpieces are grabbed, so that the possibility that the workpiece at the corner is shielded by other workpieces is reduced, and the grabbing stability is also effectively improved.
In a preferred embodiment, in the step S100, a depth camera is used to photograph the workpiece to be sorted and the material frame at the same time, so as to obtain a plurality of point clouds; and then, identifying the point cloud based on a template matching technology, wherein the point cloud is identified by using a SeizetPickling software 3D template matching function in the embodiment. Preferably, in order to ensure the accuracy of point cloud matching, after the original point cloud is obtained, the original point cloud is further trimmed in this embodiment, so that the point cloud is more accurately identified. As shown in fig. 2, in an embodiment, a depth camera is used to photograph a material frame and workpieces in the material frame, and then the material frame and a plurality of workpieces, namely material frame, workpiece 1, workpiece 2, workpiece 3, workpiece 4, and workpiece 5, are identified from the randomly placed workpieces.
It should be noted that, the method for determining a workpiece grabbing sequence in the embodiment of the present invention is directed to a scenario in which a plurality of workpieces are recognized, and when only one workpiece is recognized, sorting is not required, and only the recognized workpiece is grabbed. In addition, due to the influence of the shooting environment, the shooting angle or other factors, when the depth camera is used for shooting and acquiring the original point cloud, the situation that the workpiece can not be identified may occur, at the moment, the workpiece point cloud is acquired again by adjusting the shooting angle or the lighting angle of the camera, and then the capturing sequence is determined.
In a preferred embodiment, the step S200 specifically includes:
acquiring preset position points of the material frame point cloud and coordinates of a plurality of workpiece point clouds under a camera coordinate system;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system.
In this embodiment, when the camera is used for shooting, the preset position point of the material frame point cloud and the X coordinates and the Y coordinates of the plurality of workpiece point clouds in the camera coordinate system can be directly obtained, and when the depth camera is used for shooting, the Z coordinate can be determined according to the depth value of the camera, so that the preset position point of the material frame point cloud and the coordinates of the plurality of workpiece point clouds in the camera coordinate system can be obtained. The preset position point is any point on the center line in the height direction of the material frame, and the center point of the material frame is preferred in the embodiment of the present invention.
In a preferred embodiment, the preset position point is a material frame center point.
In a preferred embodiment, the determining the position information of the preset position point of the material frame point cloud and the position information of the plurality of workpiece point clouds according to the preset position point of the material frame point cloud and the coordinates of the plurality of workpiece point clouds in the camera coordinate system specifically includes:
converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under a camera coordinate system into coordinates under the material frame coordinate system; the material frame coordinate system is a coordinate system taking a preset position point of the material frame point cloud as a coordinate origin;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the material frame coordinate system.
In this embodiment, the camera coordinate system is a three-dimensional rectangular coordinate system established with the focusing center of the camera as the origin and the optical axis as the Z axis, and since it is inconvenient to subsequently calculate the distance in the camera coordinate system, in order to conveniently obtain the real position information, the coordinates of the camera coordinate system need to be converted into the coordinates in the frame coordinate system, so as to conveniently calculate the real distance. And when the coordinates under the world coordinate system are obtained, the position information of the preset position points of the material frame point cloud and the position information of the plurality of workpiece point clouds can be determined. As shown in fig. 2, coordinates of the workpiece 1, the workpiece 2, the workpiece 3, the workpiece 4, and the workpiece 5 in a camera coordinate system are (x 1, y1, z 1), (x 2, y2, z 2), (x 3, y3, z 3), (x 4, y4, z 4), (x 5, y5, z 5), respectively, coordinates of a center of the material frame point cloud in the camera coordinate system are (x 6, y6, z 6), and then coordinates of preset position points of each workpiece point cloud and the material frame point cloud in the material frame coordinate system are obtained through coordinate system conversion.
It should be noted that, in order to facilitate the calculation, the position information and the various coordinate information of each workpiece point cloud described in the embodiment of the present invention are information of the centroid position of the workpiece point cloud, and in addition, in this embodiment, the centroid position of each identified workpiece point cloud is calculated by using the seietpicking software 3D template matching function as well.
In a preferred embodiment, the step of converting the center of the material frame point cloud and the coordinates of the plurality of workpiece point clouds in the camera coordinate system into the coordinates in the material frame coordinate system is specifically as follows:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system;
calculating a conversion relation between a world coordinate system and a material frame coordinate system according to the coordinates of the preset position point of the material frame point cloud in the world coordinate system and the coordinates of the preset position point of the material frame point cloud in the material frame coordinate system;
and calculating the coordinates of the workpiece point clouds in the material frame coordinate system according to the conversion relation between the world coordinate system and the material frame coordinate system.
In this embodiment, when the camera coordinate system and the frame coordinate system are converted, two times of coordinate conversion are required. Because the coordinates under the world coordinate system obtained after the first coordinate conversion brings trouble to the calculation of the distance, in order to further facilitate the calculation of the distance, the embodiment of the invention further performs the second coordinate conversion, and the purpose of the second coordinate conversion is to simplify the coordinate data, so that the calculation is more convenient, wherein the frame coordinate system is set by a user in a self-defined way. The conversion between the world coordinate system and the material frame coordinate system only has a translation process, and the coordinates of the preset position points of the material frame point cloud under the material frame coordinate system are preset, so that the translation relationship can be determined according to the coordinates of the preset position points of the material frame point cloud under the world coordinate system and the coordinates of the material frame coordinate system, and the coordinates of each workpiece point cloud are calculated. In specific implementation, the coordinates of each workpiece point cloud in the material frame coordinate system, namely (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), (X4, Y4, Z4), (X5, Y5, Z5), can be obtained by subtracting the coordinates of the preset position point of the material frame point cloud in the world coordinate system from the coordinates of each workpiece point cloud in the world coordinate system. Of course, in other embodiments, the coordinate system of the material frame may also select another origin, which is not limited in the present invention.
In a further embodiment, the step of converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds in the camera coordinate system into the coordinates in the world coordinate system specifically includes:
and acquiring an external parameter matrix of a camera, and converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system according to the external parameter matrix of the camera.
Specifically, the external reference matrix of the camera includes a rotation matrix R and a translation matrix T, and the world coordinate system is an absolute coordinate system of the system, and coordinates of all points determine respective positions with an origin of the world coordinate system. The positions of the cameras under the world coordinate system are different, and the origin points of the camera coordinate system are different, so that when the camera coordinate system and the world coordinate system are converted, the external reference matrix is determined according to the external reference matrix of the cameras, wherein the external reference matrix is determined when the cameras are calibrated, so that the conversion relation between the camera coordinate system and the world coordinate system can be determined by acquiring external parameters when the cameras are calibrated, and the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the world coordinate system are further acquired according to the conversion relation between the coordinate system and the world coordinate system, so that the distance can be conveniently calculated.
In a further embodiment, the step S300 specifically includes:
judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse taking a preset position point of the material frame as a center;
calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame.
In this embodiment, in order to avoid grabbing the workpiece close to the corners of the material frame, before the distance calculation, the workpiece at the corners is first screened out, so as to ensure the stability of grabbing. When the method is specifically implemented, workpieces in the elliptical area are sorted and grabbed by presetting the elliptical area, workpieces outside the elliptical area are automatically screened, and when no workpiece exists in the elliptical area, no workpiece is grabbed at this time, so that the grabbing stability is ensured.
In a preferred embodiment, the preset elliptical area is an ellipse centered on a preset position point of the frame, the length of the long axis of the ellipse is half of the length of the frame, the length of the short axis of the ellipse is half of the width of the frame, in other words, the length coefficient of the ellipse is set to fa, the width coefficient is set to fb, the length and the width of the frame are L, W respectively, the long axis a = L/2 and the short axis b = W/2 of the ellipse, then the relative coordinates of each workpiece point cloud and the preset position point are calculated according to the position information of the preset position point of the frame point cloud and the position information of each workpiece point cloud, whether each workpiece is in the elliptical area is judged through the relative coordinates, of course, for convenience of calculation, the coordinates of each workpiece point cloud under the camera coordinate system can be directly converted into the coordinates under the frame coordinate system, and then the calculation is performed, specifically, the coordinates of the workpiece under the camera coordinate system are X1 and Y1, the pose of the workpiece under the frame coordinate system is X2 and Y2, the absolute value of X2 and Y3, therefore, Y2 is | =,
when in use
When the workpiece is in the elliptical area, it should be noted that the elliptical area described in the embodiment of the present invention includes an elliptical side line;
when in use
And when the workpiece is in the elliptical area, namely the workpiece is at the corner of the material frame, the workpiece is directly screened out.
In a further embodiment, the method for calculating the distance from each workpiece in the elliptical area to the preset position point of the material frame specifically comprises:
and calculating the distance from each workpiece to the center of the material frame based on the preset position point of the material frame point cloud and the X coordinate and the Y coordinate of each workpiece point cloud.
In this embodiment, the Z coordinate of the workpiece point cloud is ignored, and the distance from each workpiece to the preset position point of the material frame is calculated through the X coordinate and the Y coordinate of each workpiece point cloud, wherein the distance calculation method specifically comprises the following steps:
x1 represents the abscissa of the workpiece point cloud, x2 represents the abscissa of the preset position point, y1 represents the ordinate of the workpiece point cloud, y2 represents the abscissa of the preset position point, and s represents the distance. And then determining a grabbing sequence according to the calculated size relationship of each distance, so that the workpieces close to the center of the material frame are preferentially grabbed.
In a further embodiment of the method according to the invention,
the step of determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame is as follows:
and taking the workpiece with the minimum distance as the highest priority to grab the workpieces from the workpieces in the oval area to the preset position points of the material frame in the order from small to large.
Specifically, the grabbing sequence is determined according to the distance, and the smaller the distance is, the closer the workpiece is to the center of the material frame, so that the distance sequence is the grabbing sequence of the workpieces, and the grabbing sequence of the workpieces closer to the corner of the material frame is the later, and the grabbing sequence of the workpieces closer to the center of the material frame is the earlier. In one embodiment, as shown in fig. 3, the calculated distances between the workpieces 1, 2, 3, 4, and 5 and the center of the material frame are L1, L2, L3, L4, and L5, respectively, and the distances are sorted from low to high according to the magnitude of the values, and if L1< L3< L4< L2< L5, it is determined that the workpiece is grabbed in the sequence of the workpieces 1, 3, 4, 2, and 5. It should be noted that, in the embodiment of the present invention, only one workpiece is output at a time, that is, only the workpiece with the smallest distance is captured, and then the photographing is performed again, and the distance sorting of the next round and the output of the workpieces to be captured are performed. When the distance between two workpieces is the same, one of the workpieces is selected to be grabbed.
For better understanding of the present invention, the following detailed description of the technical solution of the present invention is provided with reference to fig. 1 to 3 as a specific embodiment:
the method for determining the workpiece grabbing sequence comprises the following steps:
the method comprises the following steps: photographing the material frame and the workpiece in the material frame by a depth camera;
step two: recognizing and acquiring a material frame point cloud and a plurality of workpiece point clouds, when the material frame point cloud or the workpiece point cloud is not recognized, adjusting the shooting angle of the depth camera, executing the step one again, when only one workpiece point cloud is recognized, grabbing the recognized workpiece, otherwise, executing the step three;
as shown in fig. 2, 5 workpieces are recognized, i.e., workpiece 1, workpiece 2, workpiece 3, workpiece 4, and workpiece 5.
Step three: and acquiring the position information of the center of the material frame point cloud and the position information of the plurality of workpiece point clouds.
In the embodiment, firstly, coordinates of centers of 5 workpiece point clouds and material frame point clouds under a camera coordinate system are obtained through a depth camera, then the coordinates of the centers of the 5 workpiece point clouds and the material frame point clouds under the camera coordinate system are converted into coordinates under a world coordinate system according to an external reference matrix of the depth camera, then the coordinates of the world coordinate system and the material frame coordinate system are converted, the coordinates of the centers of the 5 workpiece point clouds and the material frame point clouds under the world coordinate system are converted into coordinates under a user coordinate system, and the origin of coordinates of the material frame coordinate system is the center of the material frame point clouds.
Step four: calculating the distance from each workpiece to the center of the material frame based on the position information of the center of the material frame point cloud and the position information of the workpiece point cloud, and determining the workpiece grabbing sequence based on the distance from each workpiece to the center of the material frame;
in this embodiment, based on the center of the material frame point cloud and the X coordinate and the Y coordinate of each workpiece point cloud in the user coordinate system, the distance from each workpiece to the center of the material frame is calculated, then the distances from each workpiece to the center of the material frame are sorted in the order from small to large, and the workpiece with the smallest distance is used as the highest priority to grab the workpiece.
Step five: and grabbing the workpieces according to the determined workpiece grabbing sequence.
In this embodiment, after the workpiece with the highest priority is grabbed, the first step is executed after the image is taken again.
Example two
As shown in fig. 4, the present invention also discloses a device 10 for determining a workpiece gripping sequence, comprising:
the point cloud identification module 11 is used for identifying and acquiring a material frame point cloud and a plurality of workpiece point clouds;
the position acquisition module 12 is used for acquiring position information of preset position points of the material frame point cloud and position information of a plurality of workpiece point clouds;
and the grabbing sequence determining module 13 is configured to calculate distances from each workpiece to preset position points of the material frame based on the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud, and determine a workpiece grabbing sequence based on the distances from each workpiece to the preset position points of the material frame.
In the embodiment, the position information of the preset position point of the material frame and the position information of each workpiece are firstly determined, then the distance between each workpiece and the preset position point of the material frame is determined according to the position information of the preset position point of the material frame and the position information of each workpiece, and finally the workpiece grabbing sequence is sequenced according to the distance between each workpiece and the material frame, so that a reasonable workpiece grabbing sequence is determined. Meanwhile, after the workpiece is grabbed according to the method disclosed by the invention, the workpiece at the corner of the material frame is grabbed finally, and other workpieces are grabbed, so that the possibility that the workpiece at the corner is shielded by other workpieces is reduced, and the grabbing stability is also effectively improved.
Preferably, the position obtaining module 12 specifically includes:
the coordinate acquisition unit is used for acquiring preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system;
and the position acquisition unit is used for determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system.
Further, the position acquiring unit specifically includes:
the coordinate conversion subunit is used for converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under a camera coordinate system into coordinates under the material frame coordinate system; the material frame coordinate system is a coordinate system taking a preset position point of the material frame point cloud as a coordinate origin;
and the position determining subunit is used for determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the material frame coordinate system.
Further, the coordinate transformation subunit is specifically configured to:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system;
calculating a conversion relation between a world coordinate system and a material frame coordinate system according to the coordinates of the preset position points of the material frame point cloud in the world coordinate system and the coordinates of the preset position points of the material frame point cloud in the material frame coordinate system;
and calculating the coordinates of the workpiece point clouds in the material frame coordinate system according to the conversion relation between the world coordinate system and the material frame coordinate system.
Further, the grabbing order determining module 13 specifically includes:
the position judging unit is used for judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse with a preset position point of the material frame as a center;
the distance calculation unit is used for calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and the sequence determining unit is used for determining the workpiece grabbing sequence according to the distance relation between each workpiece in the elliptical area and a preset position point of the material frame.
Further, the length of the long axis of the ellipse is half of the length of the material frame, and the length of the short axis of the ellipse is half of the width of the material frame.
Further, the order determination unit is specifically configured to:
and taking the workpiece with the minimum distance as the highest priority to grasp the workpieces according to the sequence from small to large from the distance from each workpiece in the elliptical area to the preset position point of the material frame.
EXAMPLE III
Fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack-mounted server, a blade server, a tower server, or a rack-mounted server (including an independent server or a server cluster formed by multiple servers) that can execute programs. The computer device 20 of the present embodiment includes at least, but is not limited to: a memory 21, a processor 22, which may be communicatively coupled to each other via a system bus, as shown in FIG. 5. It is noted that fig. 5 only shows the computer device 20 with components 21-22, but it is to be understood that not all shown components are required to be implemented, and that more or less components may alternatively be implemented.
In this embodiment, the memory 21 (i.e., the readable storage medium) includes a Flash memory, a hard disk, a multimedia Card, a Card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), and a programmable read-only memory (PROM), and the memory 21 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), and the like, which are provided on the computer device 20. Of course, the memory 21 may also include both internal and external storage devices of the computer device 20. In the present embodiment, the memory 21 is generally used for storing an operating system installed in the computer device 20 and various types of application software, such as program codes of the workpiece grasping order determining device in the method embodiment. Further, the memory 21 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 22 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 22 is typically used to control the overall operation of the computer device 20. In the present embodiment, the processor 22 is configured to execute the program code stored in the memory 21 or process data, for example, to execute the workpiece capture sequence determining apparatus 11, so as to implement the method for determining the workpiece capture sequence in the method embodiment.
Example four
The present application also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer-readable storage medium of the present embodiment is for storing program code of a workpiece capture order determination apparatus, which when executed by a processor, implements a method of determining a workpiece capture order in a method embodiment.
It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A method for determining a workpiece gripping sequence is characterized in that: the method comprises the following steps:
identifying and obtaining a material frame point cloud and a plurality of workpiece point clouds;
acquiring position information of a preset position point of the material frame point cloud and position information of a plurality of workpiece point clouds;
calculating the distance from each workpiece to a preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame;
the method comprises the following steps of calculating the distance from each workpiece to a preset position point of a material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the workpiece grabbing sequence based on the distance from each workpiece to the preset position point of the material frame, wherein the step of:
judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse with a preset position point of the material frame as a center;
calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame.
2. The method of determining the order of gripping workpieces according to claim 1, wherein: the position information of the preset position points of the material frame point cloud and the position information of the workpiece point clouds are obtained and specifically comprise the following steps:
acquiring preset position points of the material frame point cloud and coordinates of a plurality of workpiece point clouds under a camera coordinate system;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system.
3. The method of determining the order of gripping workpieces according to claim 2, wherein: the method comprises the following steps of determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under a camera coordinate system, wherein the position information specifically comprises the following steps:
converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under a camera coordinate system into coordinates under the material frame coordinate system; the material frame coordinate system is a coordinate system taking a preset position point of the material frame point cloud as a coordinate origin;
and determining the position information of the preset position points of the material frame point cloud and the position information of the workpiece point cloud according to the preset position points of the material frame point cloud and the coordinates of the workpiece point cloud under the material frame coordinate system.
4. The method of determining the order of gripping workpieces according to claim 3, wherein: the step of converting the preset position points of the material frame point cloud and the coordinates of the plurality of workpiece point clouds under the camera coordinate system into the coordinates under the material frame coordinate system specifically comprises the following steps:
converting preset position points of the material frame point cloud and coordinates of the workpiece point clouds under a camera coordinate system into coordinates under a world coordinate system;
calculating a conversion relation between a world coordinate system and a material frame coordinate system according to the coordinates of the preset position point of the material frame point cloud in the world coordinate system and the coordinates of the preset position point of the material frame point cloud in the material frame coordinate system;
and calculating the coordinates of the workpiece point clouds in the material frame coordinate system according to the conversion relation between the world coordinate system and the material frame coordinate system.
5. The method of determining the order of gripping workpieces according to claim 1, wherein: the length of the long axis of the ellipse is half of the length of the material frame, and the length of the short axis of the ellipse is half of the width of the material frame.
6. The method of determining the order of gripping workpieces according to claim 1, wherein: the step of determining the workpiece grabbing sequence according to the distance relationship between each workpiece in the elliptical area and a preset position point of the material frame is as follows:
and taking the workpiece with the minimum distance as the highest priority to grab the workpieces from the workpieces in the oval area to the preset position points of the material frame in the order from small to large.
7. An apparatus for determining a workpiece gripping sequence, characterized in that: the method comprises the following steps:
the point cloud identification module is used for identifying and acquiring a material frame point cloud and a plurality of workpiece point clouds;
the position acquisition module is used for acquiring the position information of preset position points of the material frame point cloud and the position information of a plurality of workpiece point clouds;
the grabbing sequence determining module is used for calculating the distance from each workpiece to a preset position point of the material frame based on the position information of the preset position point of the material frame point cloud and the position information of the workpiece point cloud, and determining the grabbing sequence of the workpieces based on the distance from each workpiece to the preset position point of the material frame;
the grasping order determining module specifically includes:
the position judging unit is used for judging whether each workpiece is in a preset elliptical area or not based on the position information of the preset position point of the material frame point cloud and the position information of each workpiece point cloud; the preset elliptical area is an ellipse with a preset position point of the material frame as a center;
the distance calculation unit is used for calculating the distance from each workpiece in the elliptical area to a preset position point of the material frame;
and the sequence determining unit is used for determining the workpiece grabbing sequence according to the distance relation between each workpiece in the elliptical area and a preset position point of the material frame.
8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein: the processor, when executing the computer program, realizes the steps of the method of any one of claims 1 to 6.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110819471.5A CN113524187B (en) | 2021-07-20 | 2021-07-20 | Method and device for determining workpiece grabbing sequence, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110819471.5A CN113524187B (en) | 2021-07-20 | 2021-07-20 | Method and device for determining workpiece grabbing sequence, computer equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113524187A CN113524187A (en) | 2021-10-22 |
| CN113524187B true CN113524187B (en) | 2022-12-13 |
Family
ID=78128976
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110819471.5A Active CN113524187B (en) | 2021-07-20 | 2021-07-20 | Method and device for determining workpiece grabbing sequence, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113524187B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113910235A (en) * | 2021-10-29 | 2022-01-11 | 珠海格力智能装备有限公司 | Collision detection method, device and equipment for robot to grab materials and storage medium |
| CN116843631B (en) * | 2023-06-20 | 2024-04-02 | 安徽工布智造工业科技有限公司 | 3D visual material separating method for non-standard part stacking in light steel industry |
| CN117428788B (en) * | 2023-12-13 | 2024-04-05 | 杭州海康机器人股份有限公司 | Equipment control method and device, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6328523B1 (en) * | 1998-04-21 | 2001-12-11 | Fanuc Ltd. | Apparatus for picking up articles using a vision controlled robot |
| CN101034418A (en) * | 2006-03-10 | 2007-09-12 | 发那科株式会社 | Device, program, recording medium and method for robot simulation |
| CN106737664A (en) * | 2016-11-25 | 2017-05-31 | 中国科学院自动化研究所 | Sort the Delta robot control methods and system of multiclass workpiece |
| CN110370268A (en) * | 2018-09-11 | 2019-10-25 | 天津京东深拓机器人科技有限公司 | The methods, devices and systems sorted in case |
| CN111558940A (en) * | 2020-05-27 | 2020-08-21 | 佛山隆深机器人有限公司 | Robot material frame grabbing planning and collision detection method |
| CN111754515A (en) * | 2019-12-17 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Method and device for sequential gripping of stacked articles |
| CN112464410A (en) * | 2020-12-02 | 2021-03-09 | 熵智科技(深圳)有限公司 | Method and device for determining workpiece grabbing sequence, computer equipment and medium |
-
2021
- 2021-07-20 CN CN202110819471.5A patent/CN113524187B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6328523B1 (en) * | 1998-04-21 | 2001-12-11 | Fanuc Ltd. | Apparatus for picking up articles using a vision controlled robot |
| CN101034418A (en) * | 2006-03-10 | 2007-09-12 | 发那科株式会社 | Device, program, recording medium and method for robot simulation |
| CN106737664A (en) * | 2016-11-25 | 2017-05-31 | 中国科学院自动化研究所 | Sort the Delta robot control methods and system of multiclass workpiece |
| CN110370268A (en) * | 2018-09-11 | 2019-10-25 | 天津京东深拓机器人科技有限公司 | The methods, devices and systems sorted in case |
| CN111754515A (en) * | 2019-12-17 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Method and device for sequential gripping of stacked articles |
| CN111558940A (en) * | 2020-05-27 | 2020-08-21 | 佛山隆深机器人有限公司 | Robot material frame grabbing planning and collision detection method |
| CN112464410A (en) * | 2020-12-02 | 2021-03-09 | 熵智科技(深圳)有限公司 | Method and device for determining workpiece grabbing sequence, computer equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113524187A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113524187B (en) | Method and device for determining workpiece grabbing sequence, computer equipment and medium | |
| CN111015655B (en) | Mechanical arm grabbing method and device, computer readable storage medium and robot | |
| CN108044627B (en) | Method and device for detecting grabbing position and mechanical arm | |
| CN109785317B (en) | Automatic pile up neatly truss robot's vision system | |
| US20210197389A1 (en) | Computer device and method for controlling robotic arm to grasp and place objects | |
| CN112837371A (en) | Object grabbing method and device based on 3D matching and computing equipment | |
| CN112828892B (en) | Workpiece grabbing method and device, computer equipment and storage medium | |
| CN112464410B (en) | Method and device for determining workpiece grabbing sequence, computer equipment and medium | |
| CN112847375B (en) | Workpiece grabbing method and device, computer equipment and storage medium | |
| CN109955244B (en) | Grabbing control method and device based on visual servo and robot | |
| CN110621150B (en) | Printed circuit board assembling method and related device | |
| CN111242240A (en) | Material detection method and device and terminal equipment | |
| CN114820781A (en) | Intelligent carrying method, device and system based on machine vision and storage medium | |
| CN114092428A (en) | Image data processing method, image data processing device, electronic equipment and storage medium | |
| CN114310892A (en) | Object grabbing method, device and equipment based on point cloud data collision detection | |
| JP6844803B1 (en) | Method and calculation system for object recognition or object registration based on image classification | |
| CN113538582A (en) | Method and device for determining workpiece grabbing sequence, computer equipment and medium | |
| CN113034526A (en) | Grabbing method, grabbing device and robot | |
| CN114734444B (en) | Target positioning method and device, electronic equipment and storage medium | |
| CN115810049A (en) | Marker-based pose determination method, device, equipment, medium and product | |
| JP6041710B2 (en) | Image recognition method | |
| CN114022342A (en) | Acquisition method and device for acquisition point information, electronic equipment and storage medium | |
| CN114022341A (en) | Acquisition method and device for acquisition point information, electronic equipment and storage medium | |
| CN114037595A (en) | Image data processing method, image data processing device, electronic equipment and storage medium | |
| EP4207068A1 (en) | Target object detection method and apparatus, and electronic device, storage medium and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |