CN113223189B - Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body - Google Patents
Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body Download PDFInfo
- Publication number
- CN113223189B CN113223189B CN202110601908.8A CN202110601908A CN113223189B CN 113223189 B CN113223189 B CN 113223189B CN 202110601908 A CN202110601908 A CN 202110601908A CN 113223189 B CN113223189 B CN 113223189B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- boundary
- hole
- cloud model
- holes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Landscapes
- Engineering & Computer Science (AREA)
- Architecture (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a hole repairing and regular body fitting method of a three-dimensional point cloud model for grabbing an object by a mechanical arm, which comprises the following steps: (1) Detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line; (2) Segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary; (3) Filling holes by combining body shape information by using a hole repairing and point cloud filling method aiming at the hole boundary obtained in the step (2) to obtain a complete point cloud model; (4) And (4) aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting. The method adopted by the invention has the advantages of simple principle, clear thought and good point cloud hole repairing and regular body fitting effects, and can achieve the purposes of repairing the holes of the three-dimensional point cloud model of the object grabbed by the mechanical arm and fitting the regular body.
Description
Technical Field
The invention relates to a hole repairing and regular body fitting method of a three-dimensional point cloud model for grabbing an object by a mechanical arm, which is suitable for the field of repairing and applying the three-dimensional point cloud model.
Background
A three-dimensional point cloud is composed of a set of points in three-dimensional space, sometimes with attributes such as normals, colors, etc., in addition to containing positional information, and is often used to represent the geometry of three-dimensional objects and scenes. There are many methods for acquiring point clouds, such as scanning with a laser scanner or an optical scanner, or three-dimensional reconstruction of images. Due to the development of three-dimensional scanning technology and modeling technology, three-dimensional point clouds are popular in augmented reality, mobile mapping, gaming, 3D telepresence, historical relic scanning and 3D printing. Due to the limitations of the acquisition technology and the acquisition view angle, various holes often appear in the point cloud model, and the complexity of the structure of the scanned object or the structural defects also cause the holes in the point cloud. The occurrence of holes can reduce the quality of the point cloud model and influence the subsequent three-dimensional reconstruction work. Therefore, repairing the point cloud holes becomes an important issue.
For a three-dimensional point cloud model of a mechanical arm for grabbing an object, the hole is mainly caused by the shielding of a mechanical claw and the visual angle limitation during reconstruction. The shielding of the mechanical claw can cause the mechanical claw and the object to be modeled together in the reconstruction process, and a plurality of holes with regular shapes and medium sizes can be generated on the side surface of the object model after the mechanical claw is removed; the visual angle limitation during reconstruction can cause the bottom of the object to be incapable of imaging, so that the bottom of the object model has large-area loss. The missing surface of the bottom can also be regarded as a hole with a larger area, but the processing method should be different from the hole left by the mechanical claw. The quality of the point cloud model can be reduced by the holes, and subsequent rule body fitting and other model processing work are affected, so that the hole repairing work for grabbing the three-dimensional point cloud model of the object by the mechanical arm is indispensable.
At present, three-dimensional point cloud model hole repairing technologies have made serial progress, and are mainly divided into a method based on a grid model, a method directly based on point cloud, a point cloud completion method based on deep learning, and the like. However, methods for grabbing a three-dimensional point cloud model of an object by a mechanical arm are few, and the current research difficulty mainly appears in the following aspects: 1) The holes are various in types and large in area; 2) The point cloud model is sparse, disordered and poor in quality; 3) The existing method based on the grid model has the advantages that the time complexity of the process of three-angle subdivision and grid drawing is high, the requirement on the quality of a point cloud model is high, the method cannot process large-area loss, and the method is not suitable for a mechanical arm to capture an object three-dimensional point cloud model under the subject; 4) The existing point cloud completion method based on deep learning needs a large amount of data support in the training process, the existing point cloud database is few, the three-dimensional point cloud database for grabbing objects by a mechanical arm is difficult to obtain due to the particularity, and the existing point cloud model is high in noise, disordered and poor in quality, so that the existing point cloud completion method is not suitable for hole repair of the three-dimensional point cloud model for grabbing objects by the mechanical arm.
The regular body fitting work of the three-dimensional point cloud model is beneficial to simplifying the model, reducing the error influence of three-dimensional reconstruction and extracting key information of the model such as mass center, size and the like, thereby improving the accuracy of the motion and attitude control of subsequent objects. The use of a regular body instead of the object itself for the calculation is also more efficient and simpler than the use of the object itself directly. The object shape that snatchs in the combination project is mostly the cuboid, and the cuboid bounding box compare in spheroid, cylinder etc. the dimensional information of reflection object that the shape is simple that can be better simultaneously, so adopt the form of drawing the cuboid bounding box to carry out the regular body fitting of three-dimensional point cloud model.
The AABB bounding box and the OBB bounding box are widely applied at present, wherein the AABB bounding box is generated simply by searching the maximum and minimum values under a space coordinate, but is always parallel to a coordinate axis, does not rotate along with an object, and cannot be accurately attached to the object; the OBB bounding box obtains the main shaft through PCA principal component analysis according to object surface point to can rotate along with the object, but because the arm snatchs the mixed and disorderly, the density of object three-dimensional point cloud model is uneven, can lead to the main shaft skew, thereby leads to the bounding box great error to appear. Therefore, a new bounding box calculation method is required according to the characteristics of the model.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method overcomes the defects of the prior art, provides a hole repairing and regular body fitting algorithm by aiming at a three-dimensional point cloud model of an object grabbed by a mechanical arm and combining the generation reason and uniqueness of holes and actual prior information of engineering, improves the quality of the model and provides an accurate regular body fitting result.
The technical scheme of the invention is as follows: a method for repairing holes of a three-dimensional point cloud model of an object grabbed by a mechanical arm and fitting a regular body comprises the following steps:
(1) Detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line;
(2) Segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary;
(3) Aiming at the hole boundary obtained in the step (2), filling the hole by combining body shape information by using a hole repairing and point cloud filling method to obtain a complete point cloud model;
(4) And (4) aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting.
Further, in the step (1), holes existing in the three-dimensional point cloud model of the object grabbed by the mechanical arm are detected, and the method comprises the following steps:
firstly, filtering processing for removing outliers is carried out on the point cloud model, and the influence of the outliers on edge detection is reduced. Then, carrying out edge detection on the point cloud model, wherein the principle is as follows: and for each point in the point cloud, observing k adjacent points, solving the normal direction of the point by using a covariance analysis method, projecting the k adjacent points to a tangent plane, and judging whether the point is a hole boundary point according to a maximum angle measurement criterion. And then, connecting the boundary points by using a neighbor method to form a closed curve, and simply clustering the boundaries of different holes according to the distance and the vector deflection angle.
In the edge detection, for a certain point p, the most intuitive method for searching the k neighbor point set N (p) is to calculate the Euclidean distance between the point and all other points and sort the points, but the method has high space and time complexity and low efficiency when facing point clouds with large data volume, so that the point cloud data is divided by using kd-Tree, which is a data structure convenient for space point search and can accelerate the search process of the k neighbor points by limiting the search range. After obtaining N (p), a covariance matrix is established according to the following equation:
and the eigenvector corresponding to the minimum eigenvalue of the matrix E is the normal direction of p. And then all the points of N (p) are projected to a tangent plane determined by the normal direction to obtain a point set N '(p), if p is an inner point, the points of N' (p) are uniformly distributed around p, and if p is a boundary point, no projection point exists in a certain angle range around p. Each projection point q i Connecting element N' (p) and p to obtain vector
And solving the maximum included angle between the vectors as a criterion whether p is a boundary point, namely the maximum angle measurement criterion. The point cloud contour points and the hole boundary points are more sharp than the point cloud outer contour due to the hole boundary or the missing surface boundaryThe difference exists in the maximum angle, and all hole boundary points can be accurately screened out by adjusting the threshold value of the angle.
And after the hole boundary points are obtained, connecting the boundary points to form a closed curve, and distinguishing boundary lines of different holes. Since boundary points belonging to the same hole are adjacent to each other in space, a k-nearest neighbor method can be used to connect the adjacent boundary points. If the distance between the two holes is too close, the boundary points of the two holes are mutually stuck only by taking the distance as a criterion, so that a connecting trend is introduced as another criterion, namely for a certain point determined to belong to the boundary B 1 Has been determined to belong to the boundary B 1 The nearest neighbor point of the boundary point P' is Q, and the vector deflection angle is calculated as the criterion of the connection trend:
if θ is greater than a certain threshold, indicating that the boundary line direction has changed greatly when connecting from point P to point Q and is likely to connect to another hole boundary, point Q is discarded and another point close to point P is considered.
Further, in the step (2), the boundary line obtained in the step (1) is segmented and repaired based on the actual prior information of the project, so as to obtain a simple and closed hole boundary, and the method comprises the following steps:
the boundary line of the connecting holes in the step (1) is simply clustered, but if the holes in the point cloud model are originally nested with each other, the holes cannot be distinguished, and the closed boundary of each hole is obtained. In the three-dimensional point cloud of the object grabbed by the mechanical arm, the relative positions of three claws are fixed during grabbing by the mechanical claw, so that the situation mainly occurs in the mutual nesting of a bottom missing surface and a side claw hole. Since the subsequent hole repairing work depends on the complete and closed hole boundary line, and the bottom missing surface and the side surface claw hole need to be processed separately due to the large difference, the work of dividing the boundary line is indispensable.
Because the angle is fixed when the camera is modeled, the missing surface generated by visual angle limitation always appears at the bottom of a camera coordinate system, and the bottom missing surface and the side claw hole are distinguished by the prior information. Firstly, pose correction is carried out on a point cloud model by utilizing attitude angle information provided by a camera, the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane, whether the point belongs to a missing surface boundary or a claw hole boundary can be judged according to the size of a z-axis coordinate of a boundary point, and a proper coordinate threshold value is determined by combining shape information of the model, so that the claw hole boundary and the missing surface boundary are accurately segmented.
Through the segmentation process, a plurality of non-closed boundary lines are generated, wherein the non-closed boundary lines comprise a missing face boundary and a plurality of claw hole boundaries, the missing face boundary and the claw hole boundaries have different degrees of missing at the segmentation position, the number of points contained in the missing face boundary is the largest, the shape is well reserved, the line is fitted by utilizing a cubic B spline curve, uniform interpolation is carried out on a missing area to repair the boundary line, and other claw hole boundaries are also repaired in the same way at the missing position. The closed pore boundary corresponding to each pore is obtained through the steps.
Further, in the step (3), the hole is filled with the body shape information by using a hole repairing and point cloud completing method according to the hole boundary obtained in the step (2), and the method includes:
for holes which are positioned on the side surface and are generated due to blocking of a mechanical claw, the holes have the characteristics of regular shape, medium size, fixed position and the like, the geometric shape of non-hole positions on the side surface is complete, the shape of the whole side surface is fitted by using a Poisson surface reconstruction method, the core idea is that point cloud represents the position of the surface of an object, a normal vector represents the inner direction and the outer direction, and estimation of a smooth object surface can be given by implicitly fitting an indication function determined by the object. The basic idea is as follows: obtaining integral relation of point cloud sampling points and an indication function through the gradient relation, obtaining a vector field of a point set by a partitioning method according to the integral relation, calculating approximation of the gradient field of the indication function to form a Poisson equation, then using matrix iteration to obtain an approximate solution according to the Poisson equation, and adopting a moving cube algorithm to extract an isosurface, thereby reconstructing an estimated continuous curved surface from the point cloud. And (3) after Poisson reconstruction, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, reserving a non-hole area of the original point cloud, and finally performing interpolation processing on a continuous curved surface in the filling area to obtain a filling point to complete hole filling.
For a missing surface which is positioned at the bottom and is generated due to the limitation of the reconstruction visual angle, the method has the characteristics of uncertain shape, large size, incapability of predicting the shape of a missing area by using neighborhood points and the like, and is not suitable for filling the hole by using the hole repairing method, so the method adopts the idea of point cloud completion for processing. For a regular body with symmetry captured in engineering, the missing surface is always parallel to a normal plane of the mechanical claw in the main direction, so that the missing area at the bottom can be repaired by a top area which is symmetrical with the missing area and is well modeled, and the method comprises the following specific steps of: firstly, a point cloud model is sectioned in a middle height area of the model along a normal plane of a mechanical claw main direction, the intercepted height is adjusted up and down, the height which can enable the area of the section area to be maximum is found, and the section under the height is used as a symmetrical plane of the whole point cloud model; and (3) then combining the missing surface boundary in the hole boundary generated in the step (2), drawing a bounding box to determine a filling area, filling all points on the symmetrical positions of the bounding box into the filling area, and completing the completion of the missing surface.
Further, in the step (3), aiming at the complete point cloud model obtained in the step (3), the pose is corrected by combining the point cloud shape, the minimum bounding box is drawn, and the regular body fitting is completed, wherein the method comprises the following steps:
firstly, further pose correction is carried out on the point cloud model. In the step (2), the pose correction is performed on the point cloud model once by using the attitude angle information provided by the camera, and the point cloud is rotated to the position where the bottom surface is parallel to the horizontal plane. After the point cloud holes are repaired, each characteristic surface of the point cloud model is more complete than that before the point cloud holes are repaired. And performing Randac plane extraction processing on the complete point cloud model, determining plane equations of the bottom surface and the side surface, and still rotating the point cloud model until the bottom surface is parallel to the horizontal plane, wherein the correction is more accurate than the first pose correction. And projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud around a z axis by 0-180 degrees, and searching a rotating angle which can enable the area of the enclosing frame to be minimum. And aligning the three axes of the point cloud model with the three axes of the world coordinate system through the two rotations.
And at the moment, an AABB bounding box thought is adopted, the maximum and minimum coordinates of the rotated point cloud model are searched along the three-axis direction of the world coordinate system, the bounding box is drawn, the bounding box of the original point cloud can be obtained through inverse rotation, and the regular body fitting is completed.
Compared with the prior art, the invention has the advantages that:
(1) Aiming at the three-dimensional point cloud model of the object grabbed by the mechanical arm, the generation reasons and characteristics of the holes are fully analyzed, the idea from hole searching, hole processing to hole repairing is adopted, the holes of the point cloud model are repaired on the basis of not damaging the original structure of the point cloud, and a better repairing effect is achieved. The method has strong robustness and low time and space complexity.
(2) The method provided by the invention is used for grabbing the three-dimensional point cloud model of the object based on the mechanical arm, optimizing the method for drawing the bounding box by combining the actual priori knowledge of the engineering, and providing a targeted regular body fitting method, so that the point cloud model in various postures can be drawn into the closely attached bounding box, the model is simplified, and the accurate information such as the mass center and the size is provided.
In a word, the method adopted by the invention has the advantages of simple principle, clear thought and good point cloud hole repairing and regular body fitting effects, and can achieve the purposes of repairing the holes of the three-dimensional point cloud model of the object grabbed by the mechanical arm and fitting the regular body.
Drawings
FIG. 1 is a flow chart of a method for repairing holes of a three-dimensional point cloud model and fitting a regular body by a mechanical arm to grab an object according to the invention;
FIG. 2 shows an example of a point cloud model before and after hole repair and a rule body fitting result obtained by the method of the present invention.
Detailed Description
The method schemes in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
As shown in fig. 1, the method for repairing holes of a three-dimensional point cloud model and fitting a ruled body of an object grabbed by a mechanical arm comprises the following specific implementation steps:
step 1, detecting holes in a three-dimensional point cloud model of an object grabbed by a mechanical arm to obtain a boundary line.
Firstly, filtering processing for removing outliers is carried out on the point cloud model, and the influence of the outliers on edge detection is reduced. Then, carrying out edge detection on the point cloud model, wherein the principle is as follows: and for each point in the point cloud, observing k adjacent points, solving the normal direction of the point by using a covariance analysis method, projecting the k adjacent points to a tangent plane, and judging whether the point is a hole boundary point according to a maximum angle measurement criterion. And then, connecting the boundary points by using a neighbor method to form a closed curve, and simply clustering the boundaries of different holes according to the distance and the vector deflection angle.
In the edge detection, for a certain point p, the most intuitive method for searching the k neighbor point set N (p) is to calculate the Euclidean distance between the point and all other points and sort the points, but the method has high space and time complexity and low efficiency when facing point clouds with large data volume, so that the point cloud data is divided by using kd-Tree, which is a data structure convenient for space point search and can accelerate the search process of the k neighbor points by limiting the search range. After obtaining N (p), a covariance matrix is established according to the following equation:
and the eigenvector corresponding to the minimum eigenvalue of the matrix E is the normal direction of p. And then all the points of N (p) are projected to a tangent plane determined by the normal direction to obtain a point set N '(p), if p is an inner point, the points of N' (p) are uniformly distributed around p, and if p is a boundary point, no projection point exists in a certain angle range around p.Each projection point q i Connecting element N' (p) and p to obtain vector
And (4) solving the maximum included angle among the vectors as a criterion whether p is a boundary point, namely the maximum angle measurement criterion. As the hole boundary or the missing surface boundary is sharper than the point cloud outer contour, the point cloud contour point and the hole boundary point have difference in the maximum angle, and all the hole boundary points can be accurately screened out by adjusting the threshold of the angle.
And after the hole boundary points are obtained, connecting the boundary points to form a closed curve, and distinguishing boundary lines of different holes. Since boundary points belonging to the same hole are adjacent to each other in space, a k-nearest neighbor method can be used to connect the adjacent boundary points. If the distance between the two holes is too close, the boundary points of the two holes are mutually stuck only by taking the distance as a criterion, so that a connecting trend is introduced as another criterion, namely for a certain point determined to belong to the boundary B 1 Has been determined to belong to the boundary B 1 The nearest neighbor point of the boundary point P' is Q, and the vector deflection angle is calculated as the criterion of the connection trend:
if θ is greater than a threshold, indicating that the boundary line direction changes greatly when the point P connects to the point Q, and there is a high possibility that the point P connects to another hole boundary, the point Q is discarded and another point P adjacent thereto is examined.
And 2, segmenting and repairing the boundary line obtained in the step 1 based on the prior information of the engineering reality to obtain a simple and closed hole boundary.
In the step 1, the boundary line of the connecting holes is simply clustered on the boundary lines of different holes, but if the holes in the point cloud model are originally nested with each other, the holes cannot be distinguished, and the closed boundary of each hole is obtained. In the three-dimensional point cloud of the object grabbed by the mechanical arm, the relative positions of three claws are fixed during grabbing by the mechanical claw, so that the situation mainly occurs in the mutual nesting of a bottom missing surface and a side claw hole. Since the subsequent hole repairing work depends on the complete and closed hole boundary line, and the bottom missing surface and the side surface claw hole need to be processed separately due to the larger difference, the work of dividing the boundary line is indispensable.
Because the angle is fixed during camera modeling, the missing surface generated due to visual angle limitation always appears at the bottom under a camera coordinate system, and therefore the bottom missing surface and the side claw hole are distinguished by means of the prior information. Firstly, pose correction is carried out on a point cloud model by utilizing attitude angle information provided by a camera, the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane, whether the point belongs to a missing surface boundary or a claw hole boundary can be judged according to the size of a z-axis coordinate of a boundary point, and a proper coordinate threshold value is determined by combining shape information of the model, so that the claw hole boundary and the missing surface boundary are accurately segmented.
Through the segmentation process, a plurality of non-closed boundary lines are generated, wherein the non-closed boundary lines comprise a missing face boundary and a plurality of claw hole boundaries, the missing face boundary and the claw hole boundaries have different degrees of missing at the segmentation position, the number of points contained in the missing face boundary is the largest, the shape is well reserved, the line is fitted by utilizing a cubic B spline curve, uniform interpolation is carried out on a missing area to repair the boundary line, and other claw hole boundaries are also repaired in the same way at the missing position. The closed pore boundary corresponding to each pore is obtained through the steps.
And 3, aiming at the hole boundary obtained in the step 2, filling the hole by combining body shape information by using a hole repairing and point cloud filling method, wherein the method comprises the following steps:
for holes on the side surface, which are generated due to the shielding of the mechanical claws, the hole shape estimation method has the characteristics of regular shape, medium size, fixed position and the like, the geometric shape of the non-hole position on the side surface is complete, the shape of the whole side surface is fitted by using a Poisson surface reconstruction method, the core idea is that point cloud represents the position of the surface of an object, a normal vector represents the inner direction and the outer direction, and the estimation of the smooth surface of the object can be given by implicitly fitting an indication function determined by the object. The basic idea is as follows: obtaining the integral relation between the point cloud sampling points and the indicating function through the gradient relation, obtaining a vector field of the point set by using a partitioning method according to the integral relation, calculating the approximation of the gradient field of the indicating function to form a Poisson equation, solving an approximate solution through matrix iteration according to the Poisson equation, extracting an isosurface through a moving cube algorithm, and reconstructing an estimated continuous curved surface from the point cloud. And (3) after Poisson reconstruction, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, reserving a non-hole area of the original point cloud, and finally performing interpolation processing on a continuous curved surface in the filling area to obtain filling points to finish hole filling.
For a missing surface which is positioned at the bottom and is generated due to the limitation of a reconstruction visual angle, the method has the characteristics of uncertain shape, large size, incapability of predicting the shape of a missing area by a neighborhood point and the like, and is not suitable for filling by the hole repairing method, so the method adopts the idea of point cloud completion for processing. For a regular body with symmetry captured in engineering, the missing surface is always parallel to a normal plane of the mechanical claw in the main direction, so that the missing area at the bottom can be repaired by a top area which is symmetrical with the missing area and has good modeling, and the method comprises the following specific steps of: firstly, a point cloud model is sectioned in a middle height area of the model along a normal plane of a mechanical claw main direction, the intercepted height is adjusted up and down, the height which can enable the area of the section area to be maximum is found, and the section under the height is used as a symmetrical plane of the whole point cloud model; and then combining the missing surface boundary in the hole boundary generated in the step 2, drawing a bounding box to determine a filling area, filling all points on the symmetrical positions of the bounding box into the area, and completing the completion of the missing surface.
Step 4, aiming at the complete point cloud model obtained in the step 3, combining the point cloud shape to correct the pose, drawing a minimum bounding box, and completing regular body fitting, wherein the method comprises the following steps:
firstly, further pose correction is carried out on the point cloud model. In step 2, the pose of the point cloud model is corrected once by using the pose angle information provided by the camera, and the point cloud is rotated to a position where the bottom surface is parallel to the horizontal plane. After the point cloud holes are repaired, each characteristic surface of the point cloud model is more complete than that before the point cloud holes are repaired. And performing Randac plane extraction processing on the complete point cloud model, determining plane equations of the bottom surface and the side surface, and still rotating the point cloud model until the bottom surface is parallel to the horizontal plane, wherein the correction is more accurate than the first pose correction. And projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud around a z axis by 0-180 degrees, and searching a rotating angle which can enable the area of the enclosing frame to be minimum. And aligning the three axes of the point cloud model with the three axes of the world coordinate system through the two rotations.
And at the moment, an AABB bounding box thought is adopted, the maximum and minimum coordinates of the rotated point cloud model are searched along the three-axis direction of the world coordinate system, the bounding box is drawn, the bounding box of the original point cloud can be obtained through inverse rotation, and the regular body fitting is completed.
As shown in fig. 2, the first column of the diagram is an input rectangular point cloud model captured by an original gripper, because the gripper blocks and reconstructs the model, three gripper-shaped holes appear on the side surface, and a large number of defects appear on the bottom surface (the bottom surface is rotated to the upper part for convenient viewing); FIG. 2 is a second column of detected hole boundaries (the original point cloud is labeled with gray, and different holes are labeled with other different grays); the third column in fig. 2 is the repaired point cloud model and the fitting result of the rule body, and it can be seen that the holes and the missing surfaces of the point cloud model are well repaired, and the bounding box can correctly reflect the posture and the size of the model. Therefore, the method can grab the three-dimensional point cloud model of the object by aiming at the mechanical arm, repair holes in the point cloud model, improve the quality of the model and give an accurate regular body fitting result.
Those skilled in the art will appreciate that the invention may be practiced without these specific details. Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, but various changes may be apparent to those skilled in the art, and it is intended that all inventive concepts utilizing the inventive concepts set forth herein be protected without departing from the spirit and scope of the present invention as defined and limited by the appended claims.
Claims (3)
1. A method for repairing holes of a three-dimensional point cloud model of an object grabbed by a mechanical arm and fitting a regular body is characterized by comprising the following steps:
(1) Detecting holes in a three-dimensional point cloud model of an object grabbed by the mechanical arm to obtain a boundary line;
(2) Segmenting the boundary line obtained in the step (1) based on engineering prior information, repairing the boundary line and obtaining a simple and closed hole boundary;
(3) Aiming at the hole boundary obtained in the step (2), filling the hole by combining body shape information by using a hole repairing and point cloud filling method to obtain a complete point cloud model; in the step (3), the hole is filled by combining body shape information by using a hole repairing and point cloud completing method aiming at the hole boundary obtained in the step (2), and the method specifically comprises the following steps:
4.1 For holes on the side surface, which are generated due to the shielding of a mechanical claw, the holes have the characteristics of regular shape, medium size and fixed position, the geometrical shape of the non-missing position of the side surface is complete, and the shape of the whole side surface is fitted by using a Poisson surface reconstruction method to obtain an estimated continuous surface; then, determining the position of the hole according to the hole boundary generated in the step (2), determining a filling area according to a bounding box of the hole boundary, and performing interpolation processing on a continuous curved surface in the area to obtain a filling point;
4.2 For a missing surface which is positioned at the bottom and is generated due to the limitation of the reconstruction view angle, a point cloud completion method is adopted for processing, namely for a regular body which is captured in the engineering and has symmetry, the missing surface is always parallel to a normal plane of the main direction of the mechanical claw, so that the missing area at the bottom is repaired through a top area which is symmetrical to the missing area and is well modeled, the filling area is accurately positioned by combining the hole boundary generated in the step (2), and the influence on the original points of the point cloud is reduced;
(4) Aiming at the complete point cloud model obtained in the step (3), correcting the pose by combining the point cloud shape, drawing a minimum bounding box, and finishing regular body fitting; the method comprises the following specific steps:
5.1 Carrying out further pose correction on the point cloud model, and specifically operating as follows: performing Randac plane extraction treatment on the complete point cloud model to enable the point cloud model to rotate until the bottom surface is parallel to the horizontal plane; projecting the point cloud to a horizontal plane, drawing an enclosing frame for the projection point cloud along an x axis and a y axis, rotating the projection point cloud around a z axis for 0-180 degrees, and searching a rotating angle which can enable the area of the enclosing frame to be minimum; aligning the three axes of the point cloud model with the three axes of the world coordinate system through the twice rotation;
5.2 The maximum and minimum coordinates of the rotated point cloud model are found along the three-axis direction of the world coordinate system, an enclosure box is drawn, the enclosure box of the original point cloud can be obtained through inverse transformation of rotation, and the fitting of a regular body is completed.
2. The method for repairing holes of three-dimensional point cloud model by grabbing an object by a mechanical arm and fitting a regular body according to claim 1, characterized by comprising the following steps of: in the step (1), holes existing in a three-dimensional point cloud model of the object grabbed by the mechanical arm are detected to obtain a boundary line, and the method specifically comprises the following steps:
2.1 Filtering the point cloud model to remove outliers, so as to reduce the influence of the outliers on edge detection;
2.2 Carrying out edge detection on the point cloud model processed in the step 2.1), specifically: for each point in the point cloud, k adjacent points are inspected, the normal direction of the point is solved by using a covariance analysis method, the k adjacent points are projected to a tangent plane, and whether the point is a hole boundary point is judged according to a maximum angle measurement criterion;
2.3 The boundary points are connected by a nearest neighbor method to form a closed curve, and the boundaries of different holes are clustered according to the distance and the vector deflection angle.
3. The method for repairing holes of three-dimensional point cloud model by grabbing an object by a mechanical arm and fitting a regular body according to claim 1, characterized by comprising the following steps of: in the step (2), the boundary line obtained in the step (1) is segmented based on the engineering prior information, the boundary line is repaired, and a simple and closed hole boundary is obtained, which specifically comprises the following steps:
3.1 Angle information provided during camera modeling is utilized to correct the pose of a point cloud model, and a boundary line containing a plurality of holes is segmented by combining the shape and height information of the model, so that the hole boundary generated by blocking of a mechanical claw on the side surface of the boundary line and the missing surface boundary generated by limitation of a reconstructed view angle on the bottom of the boundary line are distinguished;
3.2 And) generating a plurality of non-closed boundary lines through the segmentation of the step 3.1), and repairing the non-closed boundary lines by utilizing cubic B-spline curve fitting and interpolation to obtain a closed hole boundary corresponding to each hole.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110601908.8A CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110601908.8A CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113223189A CN113223189A (en) | 2021-08-06 |
| CN113223189B true CN113223189B (en) | 2022-11-29 |
Family
ID=77081740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110601908.8A Active CN113223189B (en) | 2021-05-31 | 2021-05-31 | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113223189B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114266798B (en) * | 2021-12-14 | 2023-09-22 | 上海应用技术大学 | Point cloud restoration method based on boundary point iteration |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107749079B (en) * | 2017-09-25 | 2020-03-17 | 北京航空航天大学 | Point cloud quality evaluation and track planning method for unmanned aerial vehicle scanning reconstruction |
| EP3467783A1 (en) * | 2017-10-06 | 2019-04-10 | Thomson Licensing | Method and device for hole filling of a point cloud |
| CN108776993B (en) * | 2018-05-24 | 2022-04-05 | 北京建筑大学 | Modeling method of three-dimensional point cloud with hole and modeling method of underground cable work well |
| CN110415331B (en) * | 2019-06-14 | 2022-08-19 | 中国电建集团昆明勘测设计研究院有限公司 | Method for rapidly detecting, judging and repairing contour or hole based on point cloud data |
-
2021
- 2021-05-31 CN CN202110601908.8A patent/CN113223189B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113223189A (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107833270B (en) | Real-time object three-dimensional reconstruction method based on depth camera | |
| CN112070818B (en) | Robot disordered grabbing method and system based on machine vision and storage medium | |
| US10706622B2 (en) | Point cloud meshing method, apparatus, device and computer storage media | |
| EP2430588B1 (en) | Object recognition method, object recognition apparatus, and autonomous mobile robot | |
| JP4865557B2 (en) | Computer vision system for classification and spatial localization of bounded 3D objects | |
| JP4723834B2 (en) | Photorealistic three-dimensional face modeling method and apparatus based on video | |
| CN113178009B (en) | Indoor three-dimensional reconstruction method utilizing point cloud segmentation and grid repair | |
| JP2016161569A (en) | Method and system for obtaining 3d pose of object and 3d location of landmark point of object | |
| JP6483832B2 (en) | Method and system for scanning an object using an RGB-D sensor | |
| CN107767456A (en) | A kind of object dimensional method for reconstructing based on RGB D cameras | |
| CN108225319B (en) | Monocular vision rapid relative pose estimation system and method based on target characteristics | |
| CN113744337B (en) | Synchronous positioning and mapping method integrating vision, IMU and sonar | |
| CN112132876B (en) | Initial pose estimation method in 2D-3D image registration | |
| CN110910433A (en) | Point cloud matching method based on deep learning | |
| CN113223189B (en) | Method for repairing holes of three-dimensional point cloud model of object grabbed by mechanical arm and fitting ruled body | |
| CN114170284A (en) | Multi-view point cloud registration method based on active landmark point projection assistance | |
| CN112614233B (en) | Fragment splicing method under VR virtual environment | |
| CN116878524A (en) | Dynamic SLAM dense map construction method based on pyramid L-K optical flow and multi-view geometric constraint | |
| Jisen | A study on target recognition algorithm based on 3D point cloud and feature fusion | |
| CN116486015A (en) | Automatic three-dimensional size detection and CAD digital-analog reconstruction method for check cabinet | |
| Erol et al. | Visual Hull Construction Using Adaptive Sampling. | |
| CN111915632A (en) | Poor texture target object truth value database construction method based on machine learning | |
| Zhang et al. | Repair methods of complex holes in a three-dimensional model based on variation level set | |
| Garcia et al. | Efficient generation of discontinuity-preserving adaptive triangulations from range images | |
| Peszor et al. | Estimation of marker placement based on fiducial points for automatic facial animation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |