CN112734932A - Strip-shaped object unstacking method, unstacking device and computer-readable storage medium - Google Patents
Strip-shaped object unstacking method, unstacking device and computer-readable storage medium Download PDFInfo
- Publication number
- CN112734932A CN112734932A CN202110004796.8A CN202110004796A CN112734932A CN 112734932 A CN112734932 A CN 112734932A CN 202110004796 A CN202110004796 A CN 202110004796A CN 112734932 A CN112734932 A CN 112734932A
- Authority
- CN
- China
- Prior art keywords
- target
- point cloud
- strip
- posture
- unstacking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for unstacking strip-shaped objects and a computer readable storage medium, and belongs to the technical field of three-dimensional machine vision recognition. According to the method, the stacking surface point cloud of the highest layer is extracted, the main direction and the posture of a long-strip target are estimated, the initial position of the target is estimated according to the estimated posture, then the strip-shaped object is segmented, the initial position component of the segmented nth object is corrected, the target position and the posture of the nth object are output, and the robot performs grabbing operation according to the target position and the posture. The invention carries out integral prediction estimation on the target, adopts a weighting estimation strategy to quickly and accurately estimate the posture and the position of the object, and carries out reasonable deviation correction on the position of the long-strip object, thereby being used in a complex industrial scene and effectively providing a high-performance and stable unstacking identification solution.
Description
Technical Field
The invention relates to the technical field of three-dimensional machine vision recognition, in particular to a method and a device for unstacking strip-shaped objects and a computer readable storage medium.
Background
The problem of positioning and attitude determination of strip-shaped objects is widely existed in the pile removing industry, such as the steel structure production industry, the aluminum product industry, the food raw material processing industry, the automobile assembly industry and the like. It features long strip shape, simple characteristics and high requirement for positioning and grabbing. Because the object is long and has simple characteristics, the recognized position and posture are easy to offset, so that the grabbing is unstable, and even grabbing interference is caused.
The following method is generally used based on a visual positioning unstacking method.
The first method is two-dimensional positioning and attitude determination, a monocular camera and an auxiliary laser range finder are required, and the method has the defects that accurate depth information cannot be provided, the positioning is not accurate for complex scenes, and the teaching and calibration are extremely complicated.
The second method is laser line scanning positioning and pose determination, the method needs to use a laser line scanning instrument to perform 3D scanning on a scene, acquire point cloud, and position and pose determination on a target according to the point cloud.
The third method is three-dimensional positioning and attitude determination of a stereo camera, and the method needs to use a 3D camera to carry out three-dimensional modeling on a scene, acquire a depth map and a gray/color image after registration, and then position and attitude determination are carried out on a target by utilizing a three-dimensional recognition technology. Specifically, the recognition technology of the 3D camera generally processes in two steps:
the model making stage comprises the following specific steps:
making a 3D model of the target, such as STL/STEP;
making the identified model from the 3D model;
identifying a position pose of the target, the steps generally comprising:
acquiring a depth map and an aligned image;
and identifying the target in the scene by using the identification model according to the input depth map and the input image.
The model-based recognition technology generally adopts the characteristics of a digital-analog model to establish a recognition model, and the method generally has larger deviation of recognized positions and postures for the problems of simple characteristics and long strip shape of an object. The reason for this is that the features provided by the digital model cannot be effectively extracted in the actual environment, and the features of the non-digital model cannot be effectively distinguished, such as the long wooden board or the gridded outline on the cutting board, which are similar to the features of the long grabbed object, but the features of the surface of the long grabbed object are not well expressed, so that the two are easily identified by mistake. Therefore, the existing visual positioning unstacking methods have the problems of low object identification performance, unstable identification and the like.
Disclosure of Invention
In view of the above, the present invention aims to provide a method, an apparatus and a computer readable storage medium for unstacking strip-shaped objects, which can solve the problems of large position offset and posture offset in the identification of strip-shaped objects, thereby providing a high-performance and stable solution for unstacking identification.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a first aspect of the invention provides a method for unstacking strip-shaped objects, comprising the following steps:
extracting a stack surface point cloud of the highest layer;
estimating the main direction and the attitude of the target long strip;
dividing the target, and estimating the initial position of the target;
correcting the initial position component of the nth object;
outputting the target position and the attitude of the nth object;
and controlling the robot to grab according to the target position and the posture of the object.
In some embodiments, the extracting the highest layer of stacked point clouds step comprises:
projecting all the point clouds on a z-axis, and counting a projection distribution histogram of the point clouds;
and cutting a layer of point cloud set S _ top with the highest stack surface.
In some embodiments, the step of cutting out a layer of point cloud set S _ top with the highest stack face comprises:
estimating an upper bound top _ z _ min and a lower bound top _ z _ max of a z value corresponding to the top point cloud according to the projection distribution histogram z _ hist; wherein the value range of the upper bound top _ z _ min and the lower bound top _ z _ max is 10000-;
screening each point cloud, and recording the point cloud with the z value larger than top _ z _ min and smaller than top _ z _ max into a layer of point cloud data set S _ top with the highest stack surface;
and filtering the noise points of the top-level point cloud set.
In some embodiments, the estimating a preliminary location of the target step comprises:
projecting the point cloud set S _ top to an image, and calculating a minimum circumscribed rectangular frame BBox;
in an interested area BBox on the gray level image, estimating a straight line set formed by a long-strip object, estimating a main pixel level direction by using SVD according to the straight line set, and filtering a noise direction;
calculating the coordinates of the end point of the main direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv;
mapping the pixel coordinate S _ center to a 3D camera coordinate S _ cam _ center ═ xs zs ] T, and mapping the pixel coordinate E _ center to a 3D camera coordinate E _ cam _ center ═ xe ye ] T, according to the camera parameters;
calculating a target long-strip main direction X _ direct ═ [ xe-xs, ye-ys, ze-zs ] T; the Y-axis direction of the target posture is obtained by cross-multiplying Z _ direct by X _ direct, namely Y _ direct is Z _ direct multiplied by X _ direct; the Z-axis direction of the target pose, Z _ direct ═ xz yz zz ] T;
the target pose may be denoted as R _ rot ═ X _ direct Y _ direct Z _ direct.
In some embodiments, estimating the preliminary location of the target comprises:
projecting the point cloud set S _ top onto an XY plane to obtain a plane point set P _ top, projecting all points of the plane point set P _ top onto a Y _ direct axis, and searching two end points of which the gradients at the left end and the right end are greater than a threshold value YDegThr on the Y _ direct axis;
and (4) dividing the long blocks to obtain the initial position of the target.
In some embodiments, the step of dividing the bar and obtaining the initial position of the target comprises:
assuming that the layer contains N elongated objects, the initial center position [ cx cy cz ] T of the nth object is:
cx_n＝lx+(rx-lx)*(2n+1)/(2N)
cy_n＝ly+(ry-ly)*(2n+1)/(2N)
cz_n＝lz+(rz-lz)*(2n+1)/(2N)；
n is a positive integer greater than 1 and greater than N.
In some embodiments, the step of rectifying the initial position component of the nth object includes:
calculating an external rectangular frame nBox of the nth object, and extracting point cloud in the nBox, and recording the point cloud as N _ top;
projecting all points of N _ top to an X _ direct axis, searching two end points of which the gradients of the left end and the right end are larger than a threshold XDegThr on the X _ direct axis, and marking as nY _ left ═ nly nlz ] T, and nY _ right ═ nrx nry z ] T; wherein the value range of the threshold XDegThr is 10-100;
the deviation correction initial position component [ cx cy cz ] T is:
cx_n＝(nrx+nlx)/2；
cy_n＝cy_n+(nrx+nlx)/2-cx_n
cz_n＝cz_n+(nrx+nlx)/2-cx_n。
the second aspect of the invention also provides a device for unstacking the strip of objects, which executes the method for unstacking the strip of objects, and comprises a cloud extraction module, a direction posture estimation module, a target position estimation module, a position deviation correction module, a position posture output module and a target grabbing module;
the point cloud extraction module is used for extracting the buttress point cloud of the highest layer;
the direction and attitude estimation module is used for estimating the main direction and the attitude of the target long strip;
the target position estimation module is used for estimating the preliminary position of the target;
the position deviation rectifying module is used for rectifying the initial position component of the nth object;
the position and posture output module is used for outputting the target position and posture of the nth object;
and a target grabbing module; and controlling the robot to grab according to the position and posture information of the bar-shaped target.
In some embodiments, the point cloud extraction module comprises a point cloud projection unit and a cut stack point cloud unit;
the point cloud projection unit is used for projecting all point clouds onto a z axis and counting a projection distribution histogram of the point clouds;
and the cutting stack surface point cloud unit is used for cutting a layer of point cloud set S _ top with the highest stack surface.
In some embodiments, the direction pose estimation module includes a center point pixel calculation unit, a direction vector calculation unit, a main direction calculation unit, a pixel coordinate calculation unit, and a target pose acquisition unit;
the center point pixel calculation unit is used for projecting the point cloud set S _ top onto an image and calculating a minimum circumscribed rectangular frame BBox; the pixel coordinate of the central point is S _ center ═ us vs ] T; wherein us represents the x coordinate value of the image horizontal axis, and vs represents the y coordinate system of the image vertical axis;
the direction vector calculation unit is used for estimating a straight line set formed by a long object in an interested area BBox on a gray level image, estimating a main pixel level direction by using SVD according to the straight line set, filtering a noise direction, and recording a main direction vector as d ═ du dv ] T; wherein du represents the component value in the x direction of the horizontal axis, dv represents the component value in the y direction of the vertical axis, and the length value of the d vector is 1;
the main direction calculating unit is used for calculating the terminal point coordinates of the main direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv; wherein ue represents the abscissa value of the terminal point, and ve represents the ordinate value of the terminal point;
the pixel coordinate calculation unit is used for mapping a pixel coordinate S _ center to a 3D camera coordinate S _ cam _ center ═ xs zs ] T and mapping a pixel coordinate E _ center to a 3D camera coordinate E _ cam _ center ═ xe ye ] T according to the camera parameters;
and the target posture acquisition unit is used for calculating the posture of the target.
The present application also provides a computer-readable storage medium comprising a processor, a computer-readable storage medium and a computer program stored on the computer-readable storage medium, which computer program, when executed by the processor, performs the steps of the method as described above.
According to the method for unstacking the strip-shaped objects, the unstacking device and the computer storage medium provided by the embodiment of the invention, the strip-shaped objects are segmented by extracting the stack surface point cloud of the highest layer and estimating the main direction and the posture of the strip-shaped object according to the estimated posture, the initial position of the segmented nth object is corrected, the target position and the posture of the nth object are output, and the robot performs grabbing operation according to the target position and the posture. The invention carries out integral prediction estimation on the target, adopts a weighting estimation strategy to quickly and accurately estimate the posture and the position of the object, and carries out reasonable deviation correction on the position of the long-strip object, thereby being used in a complex industrial scene and effectively providing a high-performance and stable unstacking identification solution.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of a method of unstacking strip-shaped objects according to the invention;
fig. 2A is a flowchart of a method for extracting a cloud of stacking points of a highest layer in a method for unstacking strip-shaped objects according to an embodiment of the present invention;
fig. 2B is a schematic diagram of the method for unstacking strip-shaped objects according to the embodiment of the present invention, in which all point clouds are projected onto a z-axis and a projection distribution histogram of the point clouds is calculated;
FIG. 3 is a flowchart of a method for estimating a main direction and a posture of a target strip in a method for unstacking strip-shaped objects according to an embodiment of the invention;
FIG. 4 is a flowchart of a method for estimating a preliminary location of an object for a method of destacking strip-shaped objects according to an embodiment of the invention;
FIG. 5 is a flowchart of a method for correcting the deviation of an initial position component according to the method for unstacking strip-shaped objects of the embodiment of the invention;
FIG. 6 is a block diagram of an embodiment of an apparatus for de-stacking strip-shaped objects according to the present invention;
fig. 7 is a block diagram showing another embodiment of the apparatus for unstacking strip-shaped objects according to the embodiment of the invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in itself. Thus, "module", "component" or "unit" may be used mixedly.
The terminal may be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, and a fixed terminal such as a Digital TV, a desktop computer, and the like.
The first embodiment is as follows:
the invention provides a method for unstacking strip-shaped objects, which refers to fig. 1, fig. 2A and fig. 2B, and comprises the following steps:
and S10, extracting the buttress point cloud of the highest layer.
The method specifically comprises the following steps:
s101, projecting all the point clouds to a z axis, and counting a projection distribution histogram of the point clouds.
Specifically, assuming that the point cloud is in millimeters, the range of the z-axis is [0, z _ max ], and the z-value is an integer. The histogram of the projection distribution is represented by an array z _ hist [ z _ max +1], each element value of the initial array z _ hist [ z _ max +1] is 0, and the statistical process is as follows: and traversing all the point clouds, and performing projection statistics on the points with the Z-axis coordinate value of zi corresponding to the projection distribution histogram Z _ hist [ zi ] ═ Z _ hist [ zi ] +1 of each point. As shown in fig. 2B below.
And S102, cutting a layer of point cloud set S _ top with the highest stack surface.
The method specifically comprises the following steps: firstly, estimating an upper bound top _ z _ min and a lower bound top _ z _ max of a z value corresponding to the top point cloud according to a projection distribution histogram z _ hist, wherein the value range of the upper bound top _ z _ min and the lower bound top _ z _ max is 10000-. Then, screening each point cloud, and recording the point cloud with the z value larger than top _ z _ min and smaller than top _ z _ max into a layer of point cloud data set S _ top with the highest stack surface. And finally, filtering the noise points of the top-layer point cloud set, wherein a plurality of filtering methods are not repeated herein, and the filtered point cloud set is still marked as S _ top.
And S12, estimating the main direction and the posture of the target long strip.
Referring to fig. 1 and fig. 3, the method specifically includes the following steps:
and S121, projecting the point cloud set S _ top onto the image, and calculating a minimum circumscribed rectangular frame BBox, wherein the pixel coordinate of a center point is S _ center ═ us vs ] T, wherein us represents the x coordinate value of the horizontal axis of the image, and vs represents the y coordinate system of the vertical axis of the image.
S122, in the region of interest BBox on the grayscale image, estimating a straight line set formed by the long object, and according to the straight line set, estimating a main pixel and a direction by using SVD, filtering a noise direction, and keeping a main direction vector as d ═ du dv ] T, as shown in fig. 3, where du represents a component value in the x direction of the horizontal axis, dv represents a component value in the y direction of the vertical axis, and a length value of the d vector is 1.
S123, calculating the terminal point coordinates of the main direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv; where ue represents the abscissa value of the terminal point and ve represents the ordinate value of the terminal point.
S124, according to the camera parameters, mapping the pixel coordinate S _ center to the 3D camera coordinate S _ cam _ center ═ xs zs ] T, and mapping the pixel coordinate E _ center to the 3D camera coordinate E _ cam _ center ═ xe ye ] T.
S125, a target long-bar main direction X _ direct ═ [ xe-xs, ye-ys, ze-zs ] T is calculated, this direction is taken as the X-axis direction of the target attitude, X _ direct is normalized, and it is still denoted as X _ direct.
S126, from the point cloud set, a vertical upward direction, which is denoted as Z _ direct ═ xz yz zz ] T, can be easily obtained by using SVD (Singular Value Decomposition) Decomposition, and this direction is taken as the Z-axis direction of the target pose.
And S127, calculating the Y-axis direction of the target attitude, and obtaining the Y _ direction from the X _ direction after Z _ direction cross multiplication normalization, namely Y _ direction is Z _ direction multiplied by X _ direction.
The target pose may be denoted as R _ rot ═ X _ direct Y _ direct Z _ direct ];
and S14, estimating the preliminary position of the target.
As shown in fig. 4, the method specifically includes the following steps:
s141, projecting the point cloud set S _ top onto an XY plane to obtain a plane point set P _ top, projecting all points of the plane point set P _ top onto a Y _ direct axis, searching two end points on the Y _ direct axis, where the gradient of the left and right ends of each end point is greater than a threshold value YDegThr (the value is defaulted to 10, and the value range is 10-100), and recording the two end points as Y _ left ═ lxly lz ] T and Y _ right ═ rx ry rz ] T, where Y _ left ═ lxly lz ] T is a coordinate value of the point cloud searched at the left end of the threshold value YDegThr, Y ═ rx ry rz ] T is a coordinate value threshold value of the point cloud searched at the right end of the threshold value YDegThr, and the larger ydthr is, the more severe the gradient change is, the segmentation is more accurate, and vice versa the rougher is.
And S142, dividing the long blocks to obtain the initial position of the target. Assuming that the layer contains N elongated objects, the initial center position [ cx cy cz ] T of the nth object is:
cx_n＝lx+(rx-lx)*(2n+1)/(2N)
cy_n＝ly+(ry-ly)*(2n+1)/(2N)
cz_n＝lz+(rz-lz)*(2n+1)/(2N)
n is a positive integer greater than 1 and less than N.
And S16, correcting the initial position component [ cx cy cz ] T of the nth object.
As shown in fig. 5, the method specifically includes the following steps:
s161, calculating a circumscribed rectangle box nBox of the nth object, and extracting point cloud in the nBox, and recording the point cloud as N _ top.
S162, projecting all points of N _ top to an X _ direct axis, searching two end points of which the gradients at the left end and the right end are larger than a threshold XDegThr (the value is default to 10 and the value range is 10-100) on the X _ direct axis, and marking as nY _ left ═ nlynlz ] T and nY _ right ═ nrx nrz ] T.
S163, the deviation rectifying initial position component [ cx cy cz ] T is as follows:
cx_n＝(nrx+nlx)/2；
cy_n＝cy_n+(nrx+nlx)/2-cx_n
cz_n＝cz_n+(nrx+nlx)/2-cx_n；
and S18, outputting the target position and the attitude of the nth object, and respectively recording the target position and the attitude as a position translation [ cx cy cz ] T and a rotation matrix R _ rot ═ X _ direct Y _ direct Z _ direct ]. And displaying the position and the posture of the target workpiece on an interface.
And S20, formulating the type of the robot, confirming an Euler coordinate system of the robot, and converting workpiece coordinates to control the robot to grab according to the position and posture of the target.
Specifically, the type of the robot is selected according to an application scene and customer requirements, and an euler coordinate system of the robot is preset by a robot manufacturer.
According to the bar-shaped object unstacking method, the stacking surface point cloud of the highest layer is extracted, the main direction and the posture of a long-bar target are estimated, the initial position of the target is estimated according to the estimated posture, then the bar-shaped object is segmented, the initial position component of the segmented nth object is corrected, the target position and the posture of the nth object are output, and the robot grabs the target position and the posture. The invention carries out integral prediction estimation on the target, adopts a weighting estimation strategy to quickly and accurately estimate the posture and the position of the object, and carries out reasonable deviation correction on the position of the long-strip object, thereby being used in a complex industrial scene and effectively providing a high-performance and stable unstacking identification solution.
Compared with the prior art, the method for unstacking the strip-shaped objects has the advantages that: the attitude estimation is accurate, the positioning speed is high, the robustness characteristic is high, the threshold judgment condition is not required to be set, the method is suitable for unstacking of objects under various environmental states, and the universality is high.
Example two:
the strip-shaped object unstacking device adopts the strip-shaped object unstacking method in the embodiment I to complete the unstacking of strip-shaped objects, is accurate in posture estimation, high in positioning speed and high in robustness, and is suitable for unstacking of objects in various environment states.
Referring to fig. 6 and 7, the device 30 for unstacking strip-shaped objects according to the embodiment of the present invention includes a point cloud extracting module 301, a direction and orientation estimating module 302, a target position estimating module 303, a position correcting module 304, a position and orientation outputting module 305, and a target grabbing module 306.
The point cloud extraction module 301 is configured to extract a buttress point cloud of a highest layer;
a direction and posture estimation module 302 for estimating the main direction and posture of the target strip;
a target position estimation module 303, configured to estimate a preliminary position of the target;
the position deviation rectifying module 304 is used for rectifying the initial position component [ cx cy cz ] T of the nth object;
a position and posture output module 305 for outputting a target position and posture of the nth object;
and a target grabbing module 306; and controlling the robot to grab according to the position and posture information of the bar-shaped target.
Specifically, the point cloud extraction module 301 further includes a point cloud projection unit 3011 and a cut-and-stack point cloud unit 3012.
The point cloud projection unit 3011 is configured to project all point clouds onto the z-axis, and count a projection distribution histogram of the point clouds;
specifically, assuming that the point cloud is in millimeters, the range of the z-axis is [0, z _ max ], and the z-value is an integer. The histogram of the projection distribution is represented by an array z _ hist [ z _ max +1], each element value of the initial array z _ hist [ z _ max +1] is 0, and the statistical process is as follows: and traversing all the point clouds, and performing projection statistics on the points with the Z-axis coordinate value of zi corresponding to the projection distribution histogram Z _ hist [ zi ] ═ Z _ hist [ zi ] +1 of each point.
And the cut stack surface point cloud unit 3012 is configured to cut a layer of point cloud set S _ top with the highest stack surface.
Specifically, an upper bound top _ z _ min and a lower bound top _ z _ max of the z value corresponding to the top point cloud are estimated according to the projection distribution histogram z _ hist. Then, screening each point cloud, and recording the point cloud with the z value larger than top _ z _ min and smaller than top _ z _ max into a layer of point cloud data set S _ top with the highest stack surface. And finally, filtering the noise points of the top-layer point cloud set, wherein a plurality of filtering methods are not repeated herein, and the filtered point cloud set is still marked as S _ top.
The direction posture estimation module 302 includes a cardiac pixel calculation unit 3021, a direction vector calculation unit 3022, a main direction calculation unit 3023, a pixel coordinate calculation unit 3024, and a target posture acquisition unit 3025.
Specifically, the center point pixel calculation unit 3021 is configured to project the point cloud set S _ top onto the image, and calculate a minimum bounding rectangle BBox, where the center point pixel coordinate is S _ center ═ us vs ] T.
The direction vector calculation unit 3022 is configured to estimate a straight line set formed by the long objects in the region of interest BBox on the grayscale image, estimate a main pixel level direction by using SVD according to the straight line set, filter a noise direction, and consider a main direction vector as d ═ du dv ] T.
A principal direction calculation unit 3023 for calculating coordinates of a terminal point of the principal direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv.
A pixel coordinate calculation unit 3024 configured to map the pixel coordinate S _ center to the 3D camera coordinate S _ cam _ center ═ xs zs ] T and map the pixel coordinate E _ center to the 3D camera coordinate E _ cam _ center ═ xe ye ] T according to the camera parameters.
A target posture acquiring unit 3025 for calculating the posture of the target.
Specifically, a target long-bar main direction X _ direct ═ [ xe-xs, ye-ys, ze-zs ] T is calculated, which is taken as the X-axis direction of the target attitude, X _ direct is normalized, and is still denoted as X _ direct;
the method is used for easily obtaining a vertical upward direction which is marked as Z _ direct ═ xz yz zz ] T by utilizing SVD (singular value decomposition) according to a point cloud set, and the direction is taken as a Z-axis direction of a target posture;
calculating the Y-axis direction of the target attitude, and obtaining the Y _ direct by cross multiplying the Z _ direct by the X _ direct, namely Y _ direct is Z _ direct multiplied by X _ direct;
finally, the target posture is obtained and can be recorded as R _ rot ═ X _ direct Y _ direct Z _ direct.
The target position estimation module 303 includes a plane point set obtaining unit 3031 and an initial position obtaining unit 3032;
the plane point set obtaining unit 3031 is configured to project the point cloud set S _ top onto an XY plane to obtain a plane point set P _ top, project all points of the plane point set P _ top onto a Y _ direct axis, and find two end points on the Y _ direct axis where the gradients of the left end and the right end are greater than a threshold value YDegThr.
And an initial position acquisition unit 3032 for dividing the bar block to acquire an initial position of the target.
The position deviation rectifying module 304 is configured to rectify an initial position component [ cx cy cz ] T of the nth object; the method specifically comprises the following steps:
and calculating an external rectangular box nBox of the nth object, and extracting the point cloud in the nBox, and recording as N _ top.
Projecting all points of N _ top to an X _ direct axis, searching two end points of which the left and right end gradients are larger than XDegThr (the value is default to 10 and the value range is 10-100) on the X _ direct axis, and marking as nY _ left ═ nlz ] T, nY _ right ═ nrx nry ] T.
The deviation [ cx cy cz ] T is:
cx_n＝(nrx+nlx)/2；
cy_n＝cy_n+(nrx+nlx)/2-cx_n
cz_n＝cz_n+(nrx+nlx)/2-cx_n。
the strip-shaped object unstacking device extracts the stacking surface point cloud of the highest layer through the point cloud extracting module 301, estimates the main direction and the posture of a long strip target through the direction posture estimating module 302, estimates the initial position of the target through the target position estimating module 303, then divides the strip-shaped object through the position deviation correcting module 304, corrects the initial position component of the nth object after division, outputs the target position and the posture of the nth object, and controls the robot to grasp the target position and the posture. The invention carries out integral prediction estimation on the target, adopts a weighting estimation strategy to quickly and accurately estimate the posture and the position of the object, and carries out reasonable deviation correction on the position of the long-strip object, thereby being used in a complex industrial scene and effectively providing a high-performance and stable unstacking identification solution.
Example three:
according to an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps in the above electronic price tag communication method, where the specific steps are as described in the first embodiment, and are not described herein again.
The memory in the present embodiment may be used to store software programs as well as various data. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the mobile phone, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
According to an example of this embodiment, all or part of the processes in the methods of the embodiments described above may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer-readable storage medium, and in this embodiment of the present invention, the program may be stored in the storage medium of a computer system and executed by at least one processor in the computer system, so as to implement the processes including the embodiments of the methods described above. The storage medium includes, but is not limited to, a magnetic disk, a flash disk, an optical disk, a Read-Only Memory (ROM), and the like.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (11)
1. A method for unstacking strip-shaped objects is characterized by comprising the following steps:
extracting a stack surface point cloud of the highest layer;
estimating the main direction and the attitude of the target long strip;
dividing the target, and estimating the initial position of the target;
correcting the initial position component of the nth object;
outputting the target position and the attitude of the nth object;
and controlling the robot to grab according to the target position and the posture of the object.
2. The method of unstacking strip-shaped objects according to claim 1, wherein the step of extracting the cloud of stacking points of the highest level comprises:
projecting all the point clouds on a z-axis, and counting a projection distribution histogram of the point clouds;
and cutting a layer of point cloud set S _ top with the highest stack surface.
3. The method for unstacking strip-shaped objects according to claim 2, wherein the step of cutting out a layer of point cloud set S _ top with the highest stacking surface comprises the following steps:
estimating an upper bound top _ z _ min and a lower bound top _ z _ max of a z value corresponding to the top point cloud according to the projection distribution histogram z _ hist; wherein the value range of the upper bound top _ z _ min and the lower bound top _ z _ max is 10000-;
screening each point cloud, and recording the point cloud with the z value larger than top _ z _ min and smaller than top _ z _ max into a layer of point cloud data set S _ top with the highest stack surface;
and filtering the noise points of the top-level point cloud set.
4. The method of unstacking strip-shaped objects according to claim 1, characterized in that the step of estimating the preliminary position of the target comprises:
projecting the point cloud set S _ top to an image, and calculating a minimum circumscribed rectangular frame BBox;
in an interested area BBox on the gray level image, estimating a straight line set formed by a long-strip object, estimating a main pixel level direction by using SVD according to the straight line set, and filtering a noise direction;
calculating the coordinates of the end point of the main direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv;
mapping the pixel coordinate S _ center to a 3D camera coordinate S _ cam _ center ═ xs zs ] T, and mapping the pixel coordinate E _ center to a 3D camera coordinate E _ cam _ center ═ xe ye ] T, according to the camera parameters;
calculating a target long-strip main direction X _ direct ═ [ xe-xs, ye-ys, ze-zs ] T; the Y-axis direction of the target posture is obtained by cross-multiplying Z _ direct by X _ direct, namely Y _ direct is Z _ direct multiplied by X _ direct; the Z-axis direction of the target pose, Z _ direct ═ xz yz zz ] T;
the target pose may be denoted as R _ rot ═ X _ directY _ directZ _ direct.
5. The method of unstacking strip-shaped objects according to claim 1, characterized in that the step of estimating the preliminary position of the target comprises:
projecting the point cloud set S _ top onto an XY plane to obtain a plane point set P _ top, projecting all points of the plane point set P _ top onto a Y _ direct axis, and searching two end points of which the gradients at the left end and the right end are greater than a threshold value YDegThr on the Y _ direct axis;
and (4) dividing the long blocks to obtain the initial position of the target.
6. The method of unstacking strip-shaped objects as recited in claim 5, wherein the step of dividing the strip to obtain the initial position of the target comprises:
assuming that the layer contains N elongated objects, the initial center position [ cx cy cz ] T of the nth object is:
cx_n＝lx+(rx-lx)*(2n+1)/(2N)
cy_n＝ly+(ry-ly)*(2n+1)/(2N)
cz_n＝lz+(rz-lz)*(2n+1)/(2N)；
n is a positive integer greater than 1 and greater than N.
7. The method of unstacking strip-shaped objects according to claim 1, wherein the step of correcting the initial position component of the nth object comprises:
calculating an external rectangular frame nBox of the nth object, and extracting point cloud in the nBox, and recording the point cloud as N _ top;
projecting all points of N _ top to an X _ direct axis, searching two end points of which the gradients of the left end and the right end are larger than a threshold XDegThr on the X _ direct axis, and marking as nY _ left ═ nly nlz ] T, and nY _ right ═ nrx nry z ] T; wherein the value range of the threshold XDegThr is 10-100;
the deviation correction initial position component [ cx cy cz ] T is:
cx_n＝(nrx+nlx)/2；
cy_n＝cy_n+(nrx+nlx)/2-cx_n
cz_n＝cz_n+(nrx+nlx)/2-cx_n。
8. a device for unstacking strip-shaped objects is characterized by comprising a cloud extraction module, a direction posture estimation module, a target position estimation module, a position deviation correction module, a position posture output module and a target grabbing module;
the point cloud extraction module is used for extracting the buttress point cloud of the highest layer;
the direction and attitude estimation module is used for estimating the main direction and the attitude of the target long strip;
the target position estimation module is used for estimating the preliminary position of the target;
the position deviation rectifying module is used for rectifying the initial position component of the nth object;
the position and posture output module is used for outputting the target position and posture of the nth object;
and a target grabbing module; and controlling the robot to grab according to the position and posture information of the bar-shaped target.
9. The device for unstacking strip-shaped objects according to claim 8, wherein the point cloud extraction module comprises a point cloud projection unit and a cut stack point cloud unit;
the point cloud projection unit is used for projecting all point clouds onto a z axis and counting a projection distribution histogram of the point clouds;
and the cutting stack surface point cloud unit is used for cutting a layer of point cloud set S _ top with the highest stack surface.
10. The apparatus according to claim 8, wherein the orientation posture estimation module includes a center point pixel calculation unit, a direction vector calculation unit, a principal direction calculation unit, a pixel coordinate calculation unit, and a target posture acquisition unit;
the center point pixel calculation unit is used for projecting the point cloud set S _ top onto an image and calculating a minimum circumscribed rectangular frame BBox; the pixel coordinate of the central point is S _ center ═ us vs ] T; wherein us represents the x coordinate value of the image horizontal axis, and vs represents the y coordinate system of the image vertical axis;
the direction vector calculation unit is used for estimating a straight line set formed by a long object in an interested area BBox on a gray level image, estimating a main pixel level direction by using SVD according to the straight line set, filtering a noise direction, and recording a main direction vector as d ═ du dv ] T; wherein du represents the component value in the x direction of the horizontal axis, dv represents the component value in the y direction of the vertical axis, and the length value of the d vector is 1;
the main direction calculating unit is used for calculating the terminal point coordinates of the main direction: e _ center ═ [ ue, ve ], where ue ═ us + du, ve ═ vs + dv; wherein ue represents the abscissa value of the terminal point, and ve represents the ordinate value of the terminal point;
the pixel coordinate calculation unit is used for mapping a pixel coordinate S _ center to a 3D camera coordinate S _ cam _ center ═ xs zs ] T and mapping a pixel coordinate E _ center to a 3D camera coordinate E _ cam _ center ═ xe ye ] T according to the camera parameters;
and the target posture acquisition unit is used for calculating the posture of the target.
11. A computer-readable storage medium, comprising a processor, a computer-readable storage medium, and a computer program stored on the computer-readable storage medium, which computer program, when executed by the processor, performs the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110004796.8A CN112734932A (en) | 2021-01-04 | 2021-01-04 | Strip-shaped object unstacking method, unstacking device and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110004796.8A CN112734932A (en) | 2021-01-04 | 2021-01-04 | Strip-shaped object unstacking method, unstacking device and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112734932A true CN112734932A (en) | 2021-04-30 |
Family
ID=75589629
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110004796.8A Pending CN112734932A (en) | 2021-01-04 | 2021-01-04 | Strip-shaped object unstacking method, unstacking device and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112734932A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109033989A (en) * | 2018-07-02 | 2018-12-18 | 深圳辰视智能科技有限公司 | Target identification method, device and storage medium based on three-dimensional point cloud |
| US20190389062A1 (en) * | 2018-06-26 | 2019-12-26 | Teradyne, Inc. | System and method for robotic bin picking |
| CN111598946A (en) * | 2020-03-16 | 2020-08-28 | 华南理工大学 | Object pose measuring method and device and storage medium |
| CN111754515A (en) * | 2019-12-17 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Method and device for sequential gripping of stacked articles |
-
2021
- 2021-01-04 CN CN202110004796.8A patent/CN112734932A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190389062A1 (en) * | 2018-06-26 | 2019-12-26 | Teradyne, Inc. | System and method for robotic bin picking |
| CN109033989A (en) * | 2018-07-02 | 2018-12-18 | 深圳辰视智能科技有限公司 | Target identification method, device and storage medium based on three-dimensional point cloud |
| CN111754515A (en) * | 2019-12-17 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Method and device for sequential gripping of stacked articles |
| CN111598946A (en) * | 2020-03-16 | 2020-08-28 | 华南理工大学 | Object pose measuring method and device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| XINJIAN FAN 等: "An Automatic Robot Unstacking System Based on Binocular Stereo Vision", 《PROCEEDINGS 2014 IEEE INTERNATIONAL CONFERENCE ON SECURITY, PATTERN ANALYSIS, AND CYBERNETICS (SPAC)》, pages 86 - 90 * |
| 曹雏清 等: "机器人自主抓取的三维点云基本形体简化算法", 《华中科技大学学报(自然科学版)》, vol. 48, no. 1, pages 13 - 19 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112070818B (en) | Robot disordered grabbing method and system based on machine vision and storage medium | |
| CN109801337B (en) | 6D pose estimation method based on instance segmentation network and iterative optimization | |
| CN110426051B (en) | Lane line drawing method and device and storage medium | |
| CN107610176B (en) | Pallet dynamic identification and positioning method, system and medium based on Kinect | |
| WO2021184757A1 (en) | Robot vision terminal positioning method and device, and computer-readable storage medium | |
| CN110648367A (en) | Geometric object positioning method based on multilayer depth and color visual information | |
| CN107953329B (en) | Object recognition and attitude estimation method and device and mechanical arm grabbing system | |
| CN111553949B (en) | Positioning and grabbing method for irregular workpiece based on single-frame RGB-D image deep learning | |
| CN112233181A (en) | 6D pose recognition method and device and computer storage medium | |
| CN109472820B (en) | Monocular RGB-D camera real-time face reconstruction method and device | |
| CN111260289A (en) | Micro unmanned aerial vehicle warehouse checking system and method based on visual navigation | |
| CN111476841B (en) | Point cloud and image-based identification and positioning method and system | |
| CN111815706A (en) | Visual identification method, device, equipment and medium for single-article unstacking | |
| CN112712589A (en) | Plant 3D modeling method and system based on laser radar and deep learning | |
| CN114331986A (en) | Dam crack identification and measurement method based on unmanned aerial vehicle vision | |
| CN113420735B (en) | Contour extraction method, device, equipment and storage medium | |
| CN111127556A (en) | Target object identification and pose estimation method and device based on 3D vision | |
| CN111275758B (en) | Hybrid 3D visual positioning method, device, computer equipment and storage medium | |
| CN111080685A (en) | Airplane sheet metal part three-dimensional reconstruction method and system based on multi-view stereoscopic vision | |
| CN114972489A (en) | Tray identification method and device, storage medium and intelligent forklift equipment | |
| CN113963129A (en) | Point cloud-based ship small component template matching and online identification method | |
| CN110634160B (en) | Method for constructing target three-dimensional key point extraction model and recognizing posture in two-dimensional graph | |
| CN116309882A (en) | Tray detection and positioning method and system for unmanned forklift application | |
| CN109872343B (en) | Weak texture object posture tracking method, system and device | |
| CN116529760A (en) | Grabbing control method, grabbing control device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |