CN113223189A - Method for repairing holes of three-dimensional point cloud model for grabbing object by mechanical arm and fitting regular body - Google Patents
Method for repairing holes of three-dimensional point cloud model for grabbing object by mechanical arm and fitting regular body Download PDFInfo
- Publication number
- CN113223189A CN113223189A CN202110601908.8A CN202110601908A CN113223189A CN 113223189 A CN113223189 A CN 113223189A CN 202110601908 A CN202110601908 A CN 202110601908A CN 113223189 A CN113223189 A CN 113223189A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- hole
- boundary
- cloud model
- holes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Abstract
The invention relates to a hole repairing and regular body fitting method of a three-dimensional point cloud model for grabbing an object by a mechanical arm, which comprises the following steps: (1) detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line; (2) segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary; (3) aiming at the hole boundary obtained in the step (2), filling the hole by combining body shape information by using a hole repairing and point cloud filling method to obtain a complete point cloud model; (4) and (4) aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting. The method adopted by the invention has the advantages of simple principle, clear thought, good point cloud hole repairing and regular body fitting effects, and can achieve the purposes of repairing the holes of the three-dimensional point cloud model of the object grabbed by the mechanical arm and fitting the regular body.
Description
Technical Field
The invention relates to a hole repairing and regular body fitting method of a three-dimensional point cloud model for grabbing an object by a mechanical arm, which is suitable for the field of repairing and applying the three-dimensional point cloud model.
Background
A three-dimensional point cloud is composed of a set of points in three-dimensional space, sometimes with attributes such as normals, colors, etc., in addition to containing positional information, and is often used to represent the geometry of three-dimensional objects and scenes. There are many methods for acquiring point clouds, such as scanning with a laser scanner or an optical scanner, or three-dimensional reconstruction of images. Due to the development of three-dimensional scanning technology and modeling technology, three-dimensional point clouds are popular in augmented reality, mobile mapping, gaming, 3D telepresence, historical relic scanning and 3D printing. Due to the limitations of the acquisition technology and the acquisition visual angle, various holes often appear in the point cloud model, and the complexity of the structure of the scanned object or the structural defects can also cause the holes of the point cloud. The occurrence of holes can reduce the quality of the point cloud model and influence the subsequent three-dimensional reconstruction work. Therefore, repairing the point cloud hole is an important issue.
For a three-dimensional point cloud model of a mechanical arm for grabbing an object, the hole is mainly caused by the shielding of a mechanical claw and the visual angle limitation during reconstruction. The shielding of the mechanical claw can cause the mechanical claw and the object to be modeled together in the reconstruction process, and a plurality of holes with regular shapes and medium sizes can be generated on the side surface of the object model after the mechanical claw is removed; the visual angle limitation during reconstruction can cause the bottom of the object to be incapable of imaging, so that the bottom of the object model has large-area loss. The missing surface of the bottom can also be regarded as a hole with a larger area, but the processing method should be different from the hole left by the mechanical claw. The quality of the point cloud model can be reduced due to the holes, and subsequent rule body fitting and other model processing work are affected, so that the hole repairing work for grabbing the three-dimensional point cloud model of the object by the mechanical arm is indispensable.
At present, three-dimensional point cloud model hole repairing technologies have made serial progress, and are mainly divided into a method based on a grid model, a method directly based on point cloud, a point cloud completion method based on deep learning, and the like. However, methods for grabbing a three-dimensional point cloud model of an object by a mechanical arm are few, and the current research difficulty mainly appears in the following aspects: 1) the holes are various in types and large in area; 2) the point cloud model is sparse, disordered and poor in quality; 3) the existing method based on the grid model has the advantages that the time complexity of the process of three-angle subdivision and grid drawing is high, the requirement on the quality of a point cloud model is high, the method cannot process large-area loss, and the method is not suitable for a mechanical arm to capture an object three-dimensional point cloud model under the subject; 4) according to the existing point cloud completion method based on deep learning, a large amount of data support is needed in the training process, the existing point cloud database is few, the three-dimensional point cloud database for grabbing objects by a mechanical arm is difficult to obtain due to the particularity of the three-dimensional point cloud database, and the existing point cloud model is high in noise, disordered and poor in quality, so that the method is not suitable for hole repair of the three-dimensional point cloud model for grabbing objects by the mechanical arm.
Regular body fitting work of the three-dimensional point cloud model is beneficial to simplifying the model, reducing error influence of three-dimensional reconstruction, and extracting key information of the model, such as mass center, size and the like, so that the accuracy of motion and attitude control of subsequent objects is improved. The use of a regular body instead of the object itself for the calculation is also more efficient and simpler than the use of the object itself directly. The shape of the object grabbed in the combination project is mostly a cuboid, and the cuboid bounding box is compared with the size information of the object which can be better reflected when the shape is simple, such as a sphere, a cylinder and the like, so that the regular body fitting of the three-dimensional point cloud model is carried out in the form of drawing the cuboid bounding box.
The AABB bounding box and the OBB bounding box are widely applied at present, wherein the AABB bounding box is generated simply by searching the maximum and minimum values under a space coordinate, but is always parallel to a coordinate axis, does not rotate along with an object, and cannot be accurately attached to the object; the OBB bounding box obtains the main shaft through PCA principal component analysis according to object surface point to can rotate along with the object, but because the arm snatchs the mixed and disorderly, the density of object three-dimensional point cloud model is uneven, can lead to the main shaft skew, thereby leads to the bounding box great error to appear. Therefore, a new bounding box calculation method is required according to the characteristics of the model.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method overcomes the defects of the prior art, provides a hole repairing and regular body fitting algorithm by aiming at a three-dimensional point cloud model of an object grabbed by a mechanical arm and combining the generation reason and uniqueness of holes and actual prior information of engineering, improves the quality of the model and provides an accurate regular body fitting result.
The technical scheme of the invention is as follows: a method for repairing holes of a three-dimensional point cloud model of an object grabbed by a mechanical arm and fitting a regular body comprises the following steps:
(1) detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line;
(2) segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary;
(3) aiming at the hole boundary obtained in the step (2), filling the hole by combining body shape information by using a hole repairing and point cloud filling method to obtain a complete point cloud model;
(4) and (4) aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting.
Further, in the step (1), holes existing in the three-dimensional point cloud model of the object grabbed by the mechanical arm are detected, and the method comprises the following steps:
firstly, filtering processing for removing outliers is carried out on the point cloud model, and the influence of the outliers on edge detection is reduced. Then, carrying out edge detection on the point cloud model, wherein the principle is as follows: and for each point in the point cloud, observing k adjacent points, solving the normal direction of the point by using a covariance analysis method, projecting the k adjacent points to a tangent plane, and judging whether the point is a hole boundary point according to a maximum angle measurement criterion. And then, connecting the boundary points by using a neighbor method to form a closed curve, and simply clustering the boundaries of different holes according to the distance and the vector deflection angle.
In the edge detection, for a certain point p, the most intuitive method for searching the k neighbor point set N (p) is to calculate the Euclidean distance between the point and all other points and sort the points, but the method has high space and time complexity and low efficiency when facing point clouds with large data volume, so that the point cloud data is divided by using a kd-Tree, wherein the kd-Tree is a data structure convenient for space point search, and the search process of the k neighbor points can be accelerated by limiting the search range. After obtaining N (p), a covariance matrix is established according to the following equation:
and the eigenvector corresponding to the minimum eigenvalue of the matrix E is the normal direction of p. And then projecting all the points of N (p) to a tangent plane determined by the normal direction to obtain a point set N '(p), wherein if p is an inner point, the points of N' (p) are uniformly distributed around p, and if p is a boundary point, no projection point exists in a certain angle range around p. Each projection point qiConnecting element N' (p) with p to obtain vector
And (4) solving the maximum included angle among the vectors as a criterion whether p is a boundary point, namely the maximum angle measurement criterion. As the hole boundary or the missing surface boundary is sharper than the point cloud outer contour, the point cloud contour point and the hole boundary point have difference in the maximum angle, and all the hole boundary points can be accurately screened out by adjusting the threshold of the angle.
And after the hole boundary points are obtained, connecting the boundary points to form a closed curve, and distinguishing boundary lines of different holes. Since boundary points belonging to the same hole are adjacent to each other in space, a k-nearest neighbor method can be used to connect the adjacent boundary points. If the distance between two holes is too close, the boundary points of two holes are mutually 'stuck' only depending on the distance as a criterion, so that the connecting trend is introduced as another criterion, namely for a certain point determined to belong to the boundary B1Has been determined to belong to the boundary B1The nearest neighbor point of the boundary point P' is Q, and the vector deflection angle is calculated as the criterion of the connection trend:
if θ is greater than a threshold, indicating that the boundary line direction changes greatly when the point P connects to the point Q, and there is a high possibility that the point P connects to another hole boundary, the point Q is discarded and another point P adjacent thereto is examined.
Further, in the step (2), the boundary line obtained in the step (1) is segmented and repaired based on the actual prior information of the project, so as to obtain a simple and closed hole boundary, and the method comprises the following steps:
the boundary line of the connecting holes in the step (1) is simply clustered, but if the holes in the point cloud model are originally nested with each other, the holes cannot be distinguished, and the closed boundary of each hole is obtained. In the three-dimensional point cloud of the object grabbed by the mechanical arm, the relative positions of three claws are fixed during grabbing by the mechanical claw, so that the situation mainly occurs in the mutual nesting of a bottom missing surface and a side claw hole. Since the subsequent hole repairing work depends on the complete and closed hole boundary line, and the bottom missing surface and the side surface claw hole need to be processed separately due to the larger difference, the work of dividing the boundary line is indispensable.
Because the angle is fixed during camera modeling, the missing surface generated due to visual angle limitation always appears at the bottom under a camera coordinate system, and therefore the bottom missing surface and the side claw hole are distinguished by means of the prior information. Firstly, pose correction is carried out on a point cloud model by utilizing attitude angle information provided by a camera, the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane, whether the point belongs to a missing surface boundary or a claw hole boundary can be judged according to the size of a z-axis coordinate of a boundary point, and a proper coordinate threshold value is determined by combining shape information of the model, so that the claw hole boundary and the missing surface boundary are accurately segmented.
Through the segmentation process, a plurality of non-closed boundary lines are generated, wherein the non-closed boundary lines comprise a missing face boundary and a plurality of claw hole boundaries, the missing face boundary and the claw hole boundaries have different degrees of missing at the segmentation position, the number of points contained in the missing face boundary is the largest, the shape is well reserved, the line is fitted by utilizing a cubic B spline curve, uniform interpolation is carried out on a missing area to repair the boundary line, and other claw hole boundaries are also repaired in the same way at the missing position. The closed pore boundary corresponding to each pore is obtained through the steps.
Further, in the step (3), for the hole boundary obtained in the step (2), the hole is filled with the body shape information by using a hole repairing and point cloud completing method, which includes:
for holes on the side surface, which are generated due to the shielding of the mechanical claws, the hole shape estimation method has the characteristics of regular shape, medium size, fixed position and the like, the geometric shape of the non-hole position on the side surface is complete, the shape of the whole side surface is fitted by using a Poisson surface reconstruction method, the core idea is that point cloud represents the position of the surface of an object, a normal vector represents the inner direction and the outer direction, and the estimation of the smooth surface of the object can be given by implicitly fitting an indication function determined by the object. The basic idea is as follows: obtaining integral relation of point cloud sampling points and an indication function through the gradient relation, obtaining a vector field of a point set by a partitioning method according to the integral relation, calculating approximation of the gradient field of the indication function to form a Poisson equation, then using matrix iteration to obtain an approximate solution according to the Poisson equation, and adopting a moving cube algorithm to extract an isosurface, thereby reconstructing an estimated continuous curved surface from the point cloud. And (3) after Poisson reconstruction, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, reserving a non-hole area of the original point cloud, and finally performing interpolation processing on a continuous curved surface in the filling area to obtain a filling point to complete hole filling.
For a missing surface which is positioned at the bottom and is generated due to the limitation of a reconstruction visual angle, the method has the characteristics of uncertain shape, large size, incapability of predicting the shape of a missing area by a neighborhood point and the like, and is not suitable for filling by the hole repairing method, so the method adopts the idea of point cloud completion for processing. For a regular body with symmetry captured in engineering, the missing surface is always parallel to a normal plane of the mechanical claw in the main direction, so that the missing area at the bottom can be repaired by a top area which is symmetrical with the missing area and is well modeled, and the method comprises the following specific steps of: firstly, a point cloud model is sectioned in a middle height area of the model along a normal plane of a mechanical claw main direction, the intercepted height is adjusted up and down, the height which can enable the area of the section area to be maximum is found, and the section under the height is used as a symmetrical plane of the whole point cloud model; and (3) then combining the missing surface boundary in the hole boundary generated in the step (2), drawing a bounding box to determine a filling area, filling all points on the symmetrical positions of the bounding box into the filling area, and completing the completion of the missing surface.
Further, in the step (3), aiming at the complete point cloud model obtained in the step (3), the pose is corrected by combining the point cloud shape, the minimum bounding box is drawn, and the regular body fitting is completed, wherein the method comprises the following steps:
firstly, further pose correction is carried out on the point cloud model. In the step (2), the pose correction is performed on the point cloud model once by using the attitude angle information provided by the camera, and the point cloud is rotated to the position where the bottom surface is parallel to the horizontal plane. After the point cloud holes are repaired, each characteristic surface of the point cloud model is more complete than that before the point cloud holes are repaired. And performing Randac plane extraction processing on the complete point cloud model, determining plane equations of the bottom surface and the side surface, and still rotating the point cloud model until the bottom surface is parallel to the horizontal plane, wherein the correction is more accurate than the first pose correction. And projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud around a z axis by 0-180 degrees, and searching a rotating angle which can enable the area of the enclosing frame to be minimum. And aligning the three axes of the point cloud model with the three axes of the world coordinate system through the two rotations.
And at the moment, an AABB bounding box thought is adopted, the maximum and minimum coordinates of the rotated point cloud model are searched along the three-axis direction of the world coordinate system, the bounding box is drawn, the bounding box of the original point cloud can be obtained through inverse rotation, and the regular body fitting is completed.
Compared with the prior art, the invention has the advantages that:
(1) aiming at the problem that a mechanical arm grabs an object three-dimensional point cloud model and the generation reasons and the characteristics of holes of the model are fully analyzed, the idea from hole searching and hole processing to hole repairing is adopted, the holes of the point cloud model are repaired on the basis of not damaging the original structure of the point cloud, and the point cloud model repairing method has a good repairing effect. The method has strong robustness and low time and space complexity.
(2) The method provided by the invention is used for grabbing the three-dimensional point cloud model of the object based on the mechanical arm, optimizing the bounding box drawing method by combining the prior knowledge of the actual engineering, providing a targeted regular body fitting method, drawing the tightly-fitted bounding box for the point cloud model in various postures, simplifying the model, and providing accurate information such as mass center and size.
In a word, the method adopted by the invention has the advantages of simple principle, clear thought, good point cloud hole repairing and regular body fitting effects, and can achieve the purposes of repairing the hole of the three-dimensional point cloud model of the object grabbed by the mechanical arm and fitting the regular body.
Drawings
FIG. 1 is a flow chart of a method for repairing holes of a three-dimensional point cloud model and fitting a regular body by using a mechanical arm to grab an object according to the invention;
FIG. 2 shows an example of a point cloud model before and after hole repair and a rule body fitting result obtained by the method of the present invention.
Detailed Description
The method schemes in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
As shown in fig. 1, the method for repairing holes of a three-dimensional point cloud model and fitting a regular body by using a mechanical arm to grab an object of the invention comprises the following specific implementation steps:
step 1, detecting holes in a three-dimensional point cloud model of an object grabbed by a mechanical arm to obtain a boundary line.
Firstly, filtering processing for removing outliers is carried out on the point cloud model, and the influence of the outliers on edge detection is reduced. Then, carrying out edge detection on the point cloud model, wherein the principle is as follows: and for each point in the point cloud, observing k adjacent points, solving the normal direction of the point by using a covariance analysis method, projecting the k adjacent points to a tangent plane, and judging whether the point is a hole boundary point according to a maximum angle measurement criterion. And then, connecting the boundary points by using a neighbor method to form a closed curve, and simply clustering the boundaries of different holes according to the distance and the vector deflection angle.
In the edge detection, for a certain point p, the most intuitive method for searching the k neighbor point set N (p) is to calculate the Euclidean distance between the point and all other points and sort the points, but the method has high space and time complexity and low efficiency when facing point clouds with large data volume, so that the point cloud data is divided by using a kd-Tree, wherein the kd-Tree is a data structure convenient for space point search, and the search process of the k neighbor points can be accelerated by limiting the search range. After obtaining N (p), a covariance matrix is established according to the following equation:
and the eigenvector corresponding to the minimum eigenvalue of the matrix E is the normal direction of p. And then projecting all the points of N (p) to a tangent plane determined by the normal direction to obtain a point set N '(p), wherein if p is an inner point, the points of N' (p) are uniformly distributed around p, and if p is a boundary point, no projection point exists in a certain angle range around p. Each projection point qiConnecting element N' (p) with p to obtain vector
And (4) solving the maximum included angle among the vectors as a criterion whether p is a boundary point, namely the maximum angle measurement criterion. As the hole boundary or the missing surface boundary is sharper than the point cloud outer contour, the point cloud contour point and the hole boundary point have difference in the maximum angle, and all the hole boundary points can be accurately screened out by adjusting the threshold of the angle.
And after the hole boundary points are obtained, connecting the boundary points to form a closed curve, and distinguishing boundary lines of different holes. Since boundary points belonging to the same hole are adjacent to each other in space, a k-nearest neighbor method can be used to connect the adjacent boundary points. If the distance between two holes is too close, the boundary points of two holes are mutually 'stuck' only depending on the distance as a criterion, so that the connecting trend is introduced as another criterion, namely for a certain point determined to belong to the boundary B1Is detected by the boundary point P of (a),the last one of them has been determined to belong to the boundary B1The nearest neighbor point of the boundary point P' is Q, and the vector deflection angle is calculated as the criterion of the connection trend:
if θ is greater than a threshold, indicating that the boundary line direction changes greatly when the point P connects to the point Q, and there is a high possibility that the point P connects to another hole boundary, the point Q is discarded and another point P adjacent thereto is examined.
And 2, segmenting and repairing the boundary line obtained in the step 1 based on the prior information of the engineering reality to obtain a simple and closed hole boundary.
In the step 1, the boundary line of the connecting holes is simply clustered on the boundary lines of different holes, but if the holes in the point cloud model are originally nested with each other, the holes cannot be distinguished, and the closed boundary of each hole is obtained. In the three-dimensional point cloud of the object grabbed by the mechanical arm, the relative positions of three claws are fixed during grabbing by the mechanical claw, so that the situation mainly occurs in the mutual nesting of a bottom missing surface and a side claw hole. Since the subsequent hole repairing work depends on the complete and closed hole boundary line, and the bottom missing surface and the side surface claw hole need to be processed separately due to the larger difference, the work of dividing the boundary line is indispensable.
Because the angle is fixed during camera modeling, the missing surface generated due to visual angle limitation always appears at the bottom under a camera coordinate system, and therefore the bottom missing surface and the side claw hole are distinguished by means of the prior information. Firstly, pose correction is carried out on a point cloud model by utilizing attitude angle information provided by a camera, the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane, whether the point belongs to a missing surface boundary or a claw hole boundary can be judged according to the size of a z-axis coordinate of a boundary point, and a proper coordinate threshold value is determined by combining shape information of the model, so that the claw hole boundary and the missing surface boundary are accurately segmented.
Through the segmentation process, a plurality of non-closed boundary lines are generated, wherein the non-closed boundary lines comprise a missing face boundary and a plurality of claw hole boundaries, the missing face boundary and the claw hole boundaries have different degrees of missing at the segmentation position, the number of points contained in the missing face boundary is the largest, the shape is well reserved, the line is fitted by utilizing a cubic B spline curve, uniform interpolation is carried out on a missing area to repair the boundary line, and other claw hole boundaries are also repaired in the same way at the missing position. The closed pore boundary corresponding to each pore is obtained through the steps.
And 3, aiming at the hole boundary obtained in the step 2, filling the hole by combining body shape information by using a hole repairing and point cloud filling method, wherein the method comprises the following steps:
for holes on the side surface, which are generated due to the shielding of the mechanical claws, the hole shape estimation method has the characteristics of regular shape, medium size, fixed position and the like, the geometric shape of the non-hole position on the side surface is complete, the shape of the whole side surface is fitted by using a Poisson surface reconstruction method, the core idea is that point cloud represents the position of the surface of an object, a normal vector represents the inner direction and the outer direction, and the estimation of the smooth surface of the object can be given by implicitly fitting an indication function determined by the object. The basic idea is as follows: obtaining integral relation of point cloud sampling points and an indication function through the gradient relation, obtaining a vector field of a point set by a partitioning method according to the integral relation, calculating approximation of the gradient field of the indication function to form a Poisson equation, then using matrix iteration to obtain an approximate solution according to the Poisson equation, and adopting a moving cube algorithm to extract an isosurface, thereby reconstructing an estimated continuous curved surface from the point cloud. And (3) after Poisson reconstruction, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, reserving a non-hole area of the original point cloud, and finally performing interpolation processing on a continuous curved surface in the filling area to obtain a filling point to complete hole filling.
For a missing surface which is positioned at the bottom and is generated due to the limitation of a reconstruction visual angle, the method has the characteristics of uncertain shape, large size, incapability of predicting the shape of a missing area by a neighborhood point and the like, and is not suitable for filling by the hole repairing method, so the method adopts the idea of point cloud completion for processing. For a regular body with symmetry captured in engineering, the missing surface is always parallel to a normal plane of the mechanical claw in the main direction, so that the missing area at the bottom can be repaired by a top area which is symmetrical with the missing area and is well modeled, and the method comprises the following specific steps of: firstly, a point cloud model is sectioned in a middle height area of the model along a normal plane of a mechanical claw main direction, the intercepted height is adjusted up and down, the height which can enable the area of the section area to be maximum is found, and the section under the height is used as a symmetrical plane of the whole point cloud model; and then combining the missing surface boundary in the hole boundary generated in the step 2, drawing a bounding box to determine a filling area, filling all points on the symmetrical positions of the bounding box into the filling area, and completing the completion of the missing surface.
Step 4, aiming at the complete point cloud model obtained in the step 3, combining the point cloud shape to correct the pose, drawing a minimum bounding box, and completing regular body fitting, wherein the method comprises the following steps:
firstly, further pose correction is carried out on the point cloud model. In step 2, pose correction is performed on the point cloud model once by using pose angle information provided by the camera, and the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane. After the point cloud holes are repaired, each characteristic surface of the point cloud model is more complete than that before the point cloud holes are repaired. And performing Randac plane extraction processing on the complete point cloud model, determining plane equations of the bottom surface and the side surface, and still rotating the point cloud model until the bottom surface is parallel to the horizontal plane, wherein the correction is more accurate than the first pose correction. And projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud around a z axis by 0-180 degrees, and searching a rotating angle which can enable the area of the enclosing frame to be minimum. And aligning the three axes of the point cloud model with the three axes of the world coordinate system through the two rotations.
And at the moment, an AABB bounding box thought is adopted, the maximum and minimum coordinates of the rotated point cloud model are searched along the three-axis direction of the world coordinate system, the bounding box is drawn, the bounding box of the original point cloud can be obtained through inverse rotation, and the regular body fitting is completed.
As shown in fig. 2, the first column of the diagram is an input rectangular point cloud model captured by an original gripper, because the gripper blocks and reconstructs the model, three gripper-shaped holes appear on the side surface, and a large number of defects appear on the bottom surface (the bottom surface is rotated to the upper part for convenient viewing); FIG. 2 is a second column of detected hole boundaries (the original point cloud is labeled with gray, and different holes are labeled with other different grays); the third column in fig. 2 is the repaired point cloud model and the fitting result of the rulers, and it can be seen that the holes and the missing surfaces of the point cloud model are well repaired, and the bounding box can correctly reflect the posture and the size of the model. Therefore, the method can grab the three-dimensional point cloud model of the object by aiming at the mechanical arm, repair holes in the point cloud model, improve the quality of the model and give an accurate regular body fitting result.
Those skilled in the art will appreciate that the invention may be practiced without these specific details. Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but various changes may be apparent to those skilled in the art, and it is intended that all inventive concepts utilizing the inventive concepts set forth herein be protected without departing from the spirit and scope of the present invention as defined and limited by the appended claims.
Claims (5)
1. A method for repairing holes of a three-dimensional point cloud model of an object grabbed by a mechanical arm and fitting a regular body is characterized by comprising the following steps:
(1) detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line;
(2) segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary;
(3) aiming at the hole boundary obtained in the step (2), filling the hole by combining body shape information by using a hole repairing and point cloud filling method to obtain a complete point cloud model;
(4) and (4) aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting.
2. The method for repairing holes of three-dimensional point cloud model and fitting rulers by using mechanical arm to grab objects according to claim 1, wherein the method comprises the following steps: in the step (1), holes existing in a three-dimensional point cloud model of the object grabbed by the mechanical arm are detected to obtain a boundary line, and the method specifically comprises the following steps:
2.1) filtering the point cloud model to remove outliers, so as to reduce the influence of the outliers on edge detection;
2.2) carrying out edge detection on the point cloud model processed in the step 2.1, which specifically comprises the following steps: for each point in the point cloud, k adjacent points of the point cloud are inspected, the normal direction of the point is calculated by using a covariance analysis method, the k adjacent points are projected to a tangent plane, and whether the point is a hole boundary point is judged according to a maximum angle measurement criterion;
and 2.3) connecting the boundary points by using a neighbor method to form a closed curve, and clustering the boundaries of different holes according to the distance and the vector deflection angle.
3. The method for repairing holes of three-dimensional point cloud model and fitting rulers by using mechanical arm to grab objects according to claim 1, wherein the method comprises the following steps: in the step (2), the boundary line obtained in the step (1) is segmented based on the engineering prior information, the boundary line is repaired, and a simple and closed hole boundary is obtained, which specifically comprises the following steps:
3.1) carrying out pose correction on the point cloud model by utilizing angle information provided during camera modeling, and dividing a boundary line containing a plurality of holes by combining the shape and height information of the model so as to distinguish a hole boundary generated on the side surface due to the shielding of a mechanical claw and a missing surface boundary generated on the bottom of the hole due to the limitation of a reconstruction visual angle.
3.2) generating a plurality of non-closed boundary lines through the segmentation of the step 3.1), and repairing the non-closed boundary lines by utilizing cubic B-spline curve fitting and interpolation to obtain a closed hole boundary corresponding to each hole.
4. The method for repairing holes of three-dimensional point cloud model and fitting rulers by using mechanical arm to grab objects according to claim 1, wherein the method comprises the following steps: in the step (3), the hole is filled by combining body shape information by using a hole repairing and point cloud completing method aiming at the hole boundary obtained in the step (2), and the method specifically comprises the following steps:
4.1) for holes on the side surface, which are generated due to the shielding of the mechanical claws, the holes have the characteristics of regular shape, medium size and fixed position, the geometrical shape of the non-missing position of the side surface is complete, and the shape of the whole side surface is fitted by using a Poisson surface reconstruction method to obtain an estimated continuous surface; then, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, and performing interpolation processing on a continuous curved surface in the area to obtain a filling point;
4.2) for the missing surface at the bottom due to the limitation of the reconstruction view angle, the method adopts a point cloud complementing method for processing, namely for a regular body which is captured in the engineering and has symmetry, the missing surface is always parallel to the normal plane of the main direction of the mechanical claw, so that the missing area at the bottom is repaired through the top area which is symmetrical with the missing area at the bottom and has good modeling, the filling area is accurately positioned by combining the hole boundary generated in the step (2), and the influence on the original point of the point cloud is reduced.
5. The method for repairing holes of three-dimensional point cloud model and fitting rulers by using mechanical arm to grab objects according to claim 1, wherein the method comprises the following steps: aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and completing regular body fitting, wherein the method specifically comprises the following steps:
5.1) further pose correction is carried out on the point cloud model, and the specific operation is as follows: performing Randac plane extraction treatment on the complete point cloud model to enable the point cloud model to rotate until the bottom surface is parallel to the horizontal plane; projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud by 0-180 degrees around a z axis, and searching a rotating angle which can enable the area of the enclosing frame to be minimum; aligning the three axes of the point cloud model with the three axes of the world coordinate system through the two rotations;
5.2) searching the maximum and minimum coordinates of the rotated point cloud model along the three-axis direction of the world coordinate system, drawing an enclosure box, and then obtaining the enclosure box of the original point cloud through inverse rotation to complete rule body fitting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110601908.8A CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110601908.8A CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113223189A true CN113223189A (en) | 2021-08-06 |
| CN113223189B CN113223189B (en) | 2022-11-29 |
Family
ID=77081740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110601908.8A Active CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113223189B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114266798A (en) * | 2021-12-14 | 2022-04-01 | 上海应用技术大学 | Point cloud repairing method based on boundary point iteration |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749079A (en) * | 2017-09-25 | 2018-03-02 | 北京航空航天大学 | A kind of quality evaluation of point cloud and unmanned plane method for planning track towards unmanned plane scan rebuilding |
| CN108776993A (en) * | 2018-05-24 | 2018-11-09 | 北京建筑大学 | The modeling method and buried cable work well modeling method of three-dimensional point cloud with hole |
| EP3467783A1 (en) * | 2017-10-06 | 2019-04-10 | Thomson Licensing | Method and device for hole filling of a point cloud |
| CN110415331A (en) * | 2019-06-14 | 2019-11-05 | 中国电建集团昆明勘测设计研究院有限公司 | A kind of profile or hole based on point cloud data quickly detects differentiation and hole repair method |
-
2021
- 2021-05-31 CN CN202110601908.8A patent/CN113223189B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749079A (en) * | 2017-09-25 | 2018-03-02 | 北京航空航天大学 | A kind of quality evaluation of point cloud and unmanned plane method for planning track towards unmanned plane scan rebuilding |
| EP3467783A1 (en) * | 2017-10-06 | 2019-04-10 | Thomson Licensing | Method and device for hole filling of a point cloud |
| CN108776993A (en) * | 2018-05-24 | 2018-11-09 | 北京建筑大学 | The modeling method and buried cable work well modeling method of three-dimensional point cloud with hole |
| CN110415331A (en) * | 2019-06-14 | 2019-11-05 | 中国电建集团昆明勘测设计研究院有限公司 | A kind of profile or hole based on point cloud data quickly detects differentiation and hole repair method |
Non-Patent Citations (2)
| Title |
|---|
| LONG YANG 等: "Shape-controllable geometry completion for point cloud models", 《THE VISUAL COMPUTER》 * |
| 韦争亮 等: "三角网格大面积孔洞光顺修补算法的研究", 《中国机械工程》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114266798A (en) * | 2021-12-14 | 2022-04-01 | 上海应用技术大学 | Point cloud repairing method based on boundary point iteration |
| CN114266798B (en) * | 2021-12-14 | 2023-09-22 | 上海应用技术大学 | Point cloud restoration method based on boundary point iteration |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113223189B (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107833270B (en) | Real-time object three-dimensional reconstruction method based on depth camera | |
| CN112070818B (en) | Robot disordered grabbing method and system based on machine vision and storage medium | |
| US10706622B2 (en) | Point cloud meshing method, apparatus, device and computer storage media | |
| CN109872397B (en) | Three-dimensional reconstruction method of airplane parts based on multi-view stereo vision | |
| EP2430588B1 (en) | Object recognition method, object recognition apparatus, and autonomous mobile robot | |
| JP4865557B2 (en) | Computer vision system for classification and spatial localization of bounded 3D objects | |
| JP4785880B2 (en) | System and method for 3D object recognition | |
| JP2016161569A (en) | Method and system for obtaining 3d pose of object and 3d location of landmark point of object | |
| CN107767456A (en) | A kind of object dimensional method for reconstructing based on RGB D cameras | |
| CN109448045B (en) | SLAM-based planar polygon measurement method and machine-readable storage medium | |
| CN104573614A (en) | Equipment and method for tracking face | |
| JP2005078646A (en) | Method and apparatus for image-based photo-realistic 3d face modelling | |
| JP6483832B2 (en) | Method and system for scanning an object using an RGB-D sensor | |
| CA2135528A1 (en) | 3-d pose refinement | |
| CN113744337A (en) | Synchronous positioning and mapping method integrating vision, IMU and sonar | |
| CN112132876B (en) | Initial pose estimation method in 2D-3D image registration | |
| CN111028345B (en) | Automatic identification and butt joint method for circular pipeline in harbor scene | |
| CN113223189B (en) | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body | |
| CN116977328B (en) | Image quality evaluation method in active vision of vehicle bottom robot | |
| Cociaş et al. | Multiple-superquadrics based object surface estimation for grasping in service robotics | |
| Erol et al. | Visual Hull Construction Using Adaptive Sampling. | |
| Jisen | A study on target recognition algorithm based on 3D point cloud and feature fusion | |
| CN111915632B (en) | Machine learning-based method for constructing truth database of lean texture target object | |
| Zhang et al. | Object detection and grabbing based on machine vision for service robot | |
| Peng et al. | Real time and robust 6D pose estimation of RGBD data for robotic bin picking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |