CN112001261A - 3D feature extraction method, system and medium based on TOLDI descriptor - Google Patents
3D feature extraction method, system and medium based on TOLDI descriptor Download PDFInfo
- Publication number
- CN112001261A CN112001261A CN202010735430.3A CN202010735430A CN112001261A CN 112001261 A CN112001261 A CN 112001261A CN 202010735430 A CN202010735430 A CN 202010735430A CN 112001261 A CN112001261 A CN 112001261A
- Authority
- CN
- China
- Prior art keywords
- descriptor
- toldi
- axis
- coordinate system
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims description 43
- 239000013598 vector Substances 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000001174 ascending effect Effects 0.000 claims description 4
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 9
- 230000000694 effects Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G06T3/067—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Abstract
The invention discloses a 3D feature extraction method, a system and a medium based on a TOLDI descriptor, wherein the method comprises the following steps: drawing three new planes at a position which is parallel to three coordinate planes of a pre-established local coordinate system and has a distance of a support radius r, and respectively projecting all neighbor nodes around a characteristic point p in the three-dimensional object onto the three new planes; dividing a limited projection area into a plurality of square grids on a projection plane; calculating the projection distance from each neighbor node in the grid to the plane on a plurality of square grids, and selecting a point with the minimum projection distance as a representative point of the square grids; and extracting the characteristic of the characteristic point according to the projection distance from the representative point to the plane and the included angle between the representative point and the z axis z (p) in the local coordinate system. The improved local coordinate system has stronger repeatability in a common scene, can achieve better effect under the shielding condition, and prevents the wrong matching condition in the matching process.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a system and a medium for extracting 3D (three-dimensional) features based on TOLDI descriptors.
Background
The features of a three-dimensional object refer to information that can represent the three-dimensional object, such as some geometric information and spatial information, and the features thereof are generally classified into two categories: global features and local features.
In a three-dimensional object, the extraction of local features is divided into two steps: firstly, detecting points with rich information as characteristic points, and then extracting the characteristics of the characteristic points, wherein the characteristic extraction is carried out according to the geometric information and the spatial distribution information around the characteristic points, because the characteristic extraction is carried out on the points, the method has strong robustness to the occlusion, and the extracted characteristics have strong descriptive property. The local features used in this approach are generally classified into two categories: one is a local feature for easily deformable objects and one is a local feature for rigid objects.
The local features applied to the deformable object are extracted mainly according to some features which are not easy to change on the deformable object.
The local descriptor is short for a local feature extraction method, and the local feature applied to a rigid object is generally applied to some objects which are not easy to deform, such as some rigid three-dimensional models. The method mainly utilizes the geometric information and the spatial information of the surrounding points of the feature points to extract the features of the feature points. In general, in three-dimensional objects, local descriptors fall into two broad categories: local Reference Frame (LRF) based descriptors and other descriptors.
Among descriptors with LRF, Tombari et al propose an orientation histogram descriptor (SHOT) that creates a sphere model with LRF near each feature point, divides the sphere into many small blocks along the elevation, azimuth and radial lines, and makes statistics on the angle between the normal of the neighboring point and the normal of the feature point in each block to form a histogram, and finally combines the Histograms to form the feature.
Specifically, a local coordinate system is established by using a covariance matrix, and then features of feature points are extracted according to the coordinate system, however, the speed of establishing the local coordinate system by using the method is slow, and repeatability is low.
Guo et al propose a Rotation Projection Statistics descriptor (ROPS). Firstly, building LRF on each characteristic point, then rotating the neighborhood point by a specific angle around three coordinate axes, respectively projecting the point to xy, xz and yz planes in each rotation, then partitioning the planes, counting the number of points falling into each block, thus forming a two-dimensional matrix, then calculating the central moment and entropy of the matrix, and taking the final result as the characteristic.
Yang et al proposed a Triple Orthogonal Local Depth descriptor (TOLDI) in 2017, which uses the spatial distribution of surrounding neighbor points to establish coordinate axes in addition to a covariance matrix when establishing a Local coordinate system, so that the coordinate system has high stability in a complex scene, but the Local coordinate system establishment mode of the TOLDI descriptor makes the coordinate system have very strong repeatability in the complex scene, because only a small part of the neighbor points is taken to establish the covariance matrix when establishing a z-axis. The purpose of this approach is to reduce the effect of occlusion, but at the same time, the information used in establishing the z-axis is too sparse, and the lack of information may make the established z-axis too far from the true, especially on some objects without occlusion, which is equivalent to wasting a lot of information.
Disclosure of Invention
The invention provides a method, a system and a medium for extracting 3D features based on TOLDI descriptors, aiming at enabling the improved local coordinate system to have stronger repeatability in common scenes and achieve better effect under certain shielding conditions, preventing the wrong matching condition in the matching process, enriching the features and reducing the complexity of calculation.
In order to achieve the above object, the present invention provides a method for extracting 3D features based on a TOLDI descriptor, the method comprising the following steps:
drawing three new planes at a position which is parallel to three coordinate planes of a pre-established local coordinate system and has a distance of a support radius r, and respectively projecting all neighbor nodes around a characteristic point p in the three-dimensional object onto the three new planes;
dividing a limited projection area into a plurality of square grids on a projection plane;
calculating the projection distance from each neighbor node in the grids to the plane on the square grids, and selecting the point with the minimum projection distance as the representative point of the square grids;
and extracting the features of the feature points according to the projection distance from the representative point to a plane and the included angle between the representative point and the z axis z (p) in the local coordinate system.
The further technical scheme of the invention is that the step of drawing three new planes at a distance of a support radius r parallel to three coordinate planes of a pre-established local coordinate system comprises the following steps:
constructing a covariance matrix by using information of all neighbor nodes around the characteristic point p;
and establishing the local coordinate system according to the covariance matrix.
The further technical scheme of the invention is that the step of constructing the covariance matrix by using the information of all neighbor nodes around the characteristic point p point comprises the following steps:
constructing a covariance matrix C on the characteristic point p according to all neighbor nodes in the peripheral radius r:
wherein q isiDenotes the neighbor node numbered i, n denotes the number of points in the neighbor, wiThen the weight, w, that the neighbor node numbered i hasiThe calculation method is as follows:
wi＝r-||p-qi||2。
the further technical scheme of the invention is that the covariance matrix has three eigenvectors, and the step of establishing the local coordinate system according to the covariance matrix comprises the following steps:
arranging the three eigenvectors in an ascending order according to the sizes of the corresponding eigenvalues, and taking the eigenvector corresponding to the minimum value as a z axis;
obtaining a final z-axis z (p) according to the sign of the product result between the z-axis and all the neighbor node vectors:
where k is the number of all neighbor nodes.
A further technical solution of the present invention is that the step of establishing the local coordinate system according to the covariance matrix further includes:
calculate each neighbor node qiVector v projected onto tangent planei：
vi＝pqi-(pqi.z(p)).z(p)，
For all vectors viWeighting to form the x-axis x (p) at the feature point p:
the invention further adopts the technical scheme that the vector viIn the step of weighting to form the x-axis x (p) at the characteristic point p, two weights are calculated as follows:
wi1＝(r-||p-qi||)2，
wi2＝(pqi.z(p))2。
a further technical solution of the present invention is that the step of establishing the local coordinate system according to the covariance matrix further includes: and obtaining a y axis according to the cross product of the z axis z (p) and the x axis x (p).
In order to achieve the above object, the present invention further provides a TOLDI descriptor-based 3D feature extraction system, which includes a memory, a processor, and a TOLDI descriptor-based 3D feature extraction program stored on the processor, where the TOLDI descriptor-based 3D feature extraction program is executed by the processor to perform the steps of the method as described above.
To achieve the above object, the present invention further provides a computer-readable storage medium having a 3D feature extraction program based on a TOLDI descriptor stored thereon, where the 3D feature extraction program based on the TOLDI descriptor executes the steps of the method as described above when being executed by a processor.
The 3D feature extraction method, the system and the medium based on the TOLDI descriptor have the advantages that: according to the technical scheme, three new planes are drawn at a position which is parallel to three coordinate planes of a pre-established local coordinate system and is at a distance of a support radius r, and all neighbor nodes around a characteristic point p in a three-dimensional object are respectively projected onto the three new planes; dividing a limited projection area into a plurality of square grids on a projection plane; calculating the projection distance from each neighbor node in the grids to the plane on the square grids, and selecting the point with the minimum projection distance as the representative point of the square grids; and extracting the features of the feature points according to the projection distance from the representative point to the plane and the included angle between the representative point and the z axis z (p) in the local coordinate system, so that the improved local coordinate system has stronger repeatability in a common scene, can achieve better effect under some shielding conditions, prevents the wrong matching condition in the matching process, enriches the features and reduces the calculation complexity.
Drawings
FIG. 1 is a flowchart illustrating a method for 3D feature extraction based on TOLDI descriptor according to a preferred embodiment of the present invention;
FIG. 2 is a schematic diagram of the formation of a TOLDI descriptor;
FIG. 3 is a local coordinate system at a feature point in the model;
FIG. 4 is a local coordinate system of feature points in a scene corresponding to FIG. 3;
fig. 5 to 8 are schematic diagrams of the matching between two models after the features are extracted when the angle deviation between the LRF and the original LRF is 0 degree, 10 degrees, 20 degrees and 30 degrees;
FIG. 9 is a schematic view of a neighbor point projected in an LRF onto a coordinate plane;
FIG. 10 is a diagram illustrating the case of a mismatch in a projection patch;
FIG. 11 is a schematic diagram illustrating the determination of the orientation of a representative point using information about the angle between the coordinate axes.
The implementation, functional features and advantages of the objects of the present invention will be further described with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Considering the way the local coordinate system of the TOLDI descriptor is established, it has a very strong repeatability in a complex scene, because only a small part of the neighbor points are taken to perform the establishment of the covariance matrix when the z-axis is established. The purpose of this method is to reduce the influence of occlusion, but at the same time, the information used in establishing the z-axis is too sparse, and the lack of information may make the difference between the established z-axis and the true z-axis too large, especially on some objects without occlusion, which is equivalent to wasting much information, and thus, the present invention proposes a solution.
Specifically, referring to fig. 1, the present invention provides a method for extracting 3D features based on a TOLDI descriptor, wherein fig. 1 is a flowchart illustrating a preferred embodiment of the method for extracting 3D features based on a TOLDI descriptor.
As shown in fig. 1, in this embodiment, the method includes the following steps:
and step S10, drawing three new planes at a position parallel to the three coordinate planes of the pre-established local coordinate system and at a distance of a support radius r, and respectively projecting all neighboring nodes around the characteristic point p in the three-dimensional object onto the three new planes.
Step S20, the limited projection area is divided into several square grids on the projection plane.
And step S30, calculating the projection distance between each neighbor node in the grids and the plane on the square grids, and selecting the point with the minimum projection distance as the representative point of the square grids.
And step S40, extracting the feature of the feature point according to the projection distance between the representative point and the plane and the included angle between the representative point and the z axis z (p) in the local coordinate system.
Wherein the step of drawing three new planes parallel to the three coordinate planes of the pre-established local coordinate system and at a distance of the support radius r is preceded by:
constructing a covariance matrix by using information of all neighbor nodes around the characteristic point p;
and establishing the local coordinate system according to the covariance matrix.
The step of constructing the covariance matrix by using the information of all neighbor nodes around the characteristic point p point comprises the following steps:
constructing a covariance matrix C on the characteristic point p according to all neighbor nodes in the peripheral radius r:
wherein q isiDenotes the neighbor node numbered i, n denotes the number of points in the neighbor, wiThen the weight, w, that the neighbor node numbered i hasiThe calculation method is as follows:
wi＝r-||p-qi||2。
the covariance matrix has three eigenvectors, and the step of establishing the local coordinate system according to the covariance matrix comprises:
arranging the three eigenvectors in an ascending order according to the sizes of the corresponding eigenvalues, and taking the eigenvector corresponding to the minimum value as a z axis;
obtaining a final z-axis z (p) according to the sign of the product result between the z-axis and all the neighbor node vectors:
where k is the number of all neighbor nodes.
The step of establishing the local coordinate system according to the covariance matrix further comprises:
calculate each neighbor node qiVector v projected onto tangent planei：
vi＝pqi-(pqi.z(p)).z(p)，
For all vectors viWeighting to form the x-axis x (p) at the feature point p:
wherein, the opposite amount viIn the step of weighting to form the x-axis x (p) at the characteristic point p, the calculation mode of the two weights is as follows:
wi1＝(r-||p-qi||)2，
wi2＝(pqi.z(p))2。
the step of establishing the local coordinate system according to the covariance matrix further comprises: and obtaining a y axis according to the cross product of the z axis z (p) and the x axis x (p).
The method for extracting 3D features based on the TOLDI descriptor according to the present invention is further described in detail with reference to fig. 2 to 11.
The TOLDI descriptor is introduced first.
The TOLDI descriptor is a local descriptor newly proposed in recent years, is proposed by Yang et al in 2017, and mainly aims to efficiently extract features, can directly extract the features on point clouds, saves the step of point cloud meshing, improves the efficiency, directly extracts spatial information, avoids the loss of the spatial information caused by extracting the features after three-dimensional points are projected on a plane, and well retains the original information. In addition, the descriptor only selects a small part of the neighbor nodes in the LRF establishing process instead of selecting all the neighbor nodes to establish the z-axis, so that the descriptor can keep good robustness in a complex scene. A dual guarantee of efficiency and descriptive is achieved.
Besides the covariance matrix, the TOLDI descriptor establishes a local coordinate system, and also establishes a coordinate axis by using the spatial distribution condition of surrounding neighboring points, so that the coordinate system has high stability in a complex scene.
An LRF is established at feature point p with a support radius r. Firstly, calculating a normal vector at a characteristic point as a z-axis of a coordinate system, and selecting a part of points Q in a small neighborhood in order to reduce the influence of shielding on the formation of the coordinate axisz{q1,q2,q3...qkZ-axis, the covariance matrix Cov (Q) can be obtainedz)：
Wherein
After the covariance matrix is obtained, the eigenvalue and the eigenvector can be calculated, the eigenvector n (p) corresponding to the minimum eigenvalue is selected as the candidate of the z-axis, and then the direction of n (p) is determined to obtain the true z-axis:
then, the x-axis is calculated, and each neighbor node q is calculated firstiVector v projected onto tangent planei：
vi＝pqi-(pqi.z(p)).z(p)，
These vectors are then weighted and normalized to form the x-axis x (p) of the LRF at point p:
and selecting projection vectors of all neighbor nodes and weighting, wherein on one hand, points closer to the characteristic points are given larger weight, and the influence of noise and shielding can be reduced. On the other hand, points with relatively large projection distances are given relatively large weights. Because the points with larger projection distance have richer geometric information, the repeatability of the x axis is stronger, and the situation that the x axis is established on a flat neighborhood can be well dealt with. The two weights are calculated as follows:
wi1＝(r-||p-qi||)2，
wi2＝(pqi.z(p))2，
thus, the x-axis and z-axis of the local coordinate system can be obtained, the y-axis can be formed by cross product, and the LRF establishment is completed.
Then, feature extraction is performed. For each feature point, first, 3 planes are constructed, parallel to the three xy, xz, and yz planes and at a distance r (on the positive axis), respectively. Then, all the neighbor points of the feature points are projected to the three planes respectively, for each plane, the distance from the point to the plane is calculated as the feature, the projection area is divided into a plurality of small blocks, a plurality of neighbor points may fall into each small block, and the point projection distance with the minimum projection distance is selected from each small block as the value of the small block. Finally, each patch of each projection has a value, a distance histogram is formed, and the histograms of the 3 projections are integrated to obtain the overall features. The formation of the features is illustrated in fig. 2.
The general idea of the TOLDI descriptor formation is consistent with the ROPS descriptor, but the gridding process is omitted, which saves a lot of time. Meanwhile, the spatial information of the points is directly extracted instead of projecting the three-dimensional points to two dimensions and then extracting the features, the original spatial information is well kept, the information loss is prevented, in addition, a z axis is constructed by utilizing local neighbor nodes, and an x axis is established by utilizing the spatial distribution information of the neighbor nodes, so that a local coordinate system also has good stability in a complex scene, the shielding resistance is strong, and a series of experiments on a database also prove the point.
The invention considers the establishment mode of the local coordinate system of the TOLDI descriptor, so that the TOLDI descriptor has very strong repeatability in a complex scene, and only a small part of the neighbor points are taken to establish the covariance matrix when the z-axis is established. The purpose of this approach is to reduce the influence of occlusion, but at the same time, the information used in establishing the z-axis is too sparse, and the lack of information may make the established z-axis too far from the true, especially on some objects without occlusion, which is equivalent to wasting a lot of information. Therefore, the invention improves Triple Orthogonal Local Depth descriptors (TOLDI), firstly improves the establishing mode of a Local coordinate system (LRF), constructs the coordinate system by using all neighbor nodes, and weights the neighbor nodes according to the importance of the neighbor nodes. The improved local coordinate system has stronger repeatability in common scenes and can achieve better effect under some shielding conditions. And angle information is added, so that on one hand, the wrong matching condition in the matching process can be prevented, on the other hand, the features are enriched, and the calculation is not too complex.
Specifically, the invention utilizes the information of all neighbor points to construct a covariance matrix, but weights are added to each neighbor point in the constructed time, the weights are determined according to the distance between the neighbor point and the characteristic point, points with the closer distance are given larger weights, so that certain robustness to occlusion is achieved, and more information is utilized to construct a z-axis on an object without the occlusion, so that the invention is a relatively stable and efficient scheme. Regarding the feature extraction part, it is known that points have loss of spatial information when projected from three dimensions to a two-dimensional plane, so that the extraction of spatial information of points as features can be considered to extract more abundant features, on one hand, the extraction is more convenient, and on the other hand, the contained information is more abundant. In addition, some other information such as angle information and the like can be added on the basis of the spatial information, on one hand, the information of the descriptor is enriched, and on the other hand, the situation that some spatial information is in mismatching can be avoided. The projection distance of the representative point is extracted from the TOLDI descriptor to be used as a feature, and the included angle between the representative point and the z-axis is added to be used as another feature, so that the descriptor is higher in descriptive performance.
1. Improvement for LRF:
summarizing all LRF-based three-dimensional local descriptors, there are generally two ways to build LRFs: and constructing based on the covariance matrix and the spatial distribution information of the neighborhood points. However, all of them have some disadvantages, the coordinate axis of the LRF constructed by the covariance matrix has a problem of uncertainty of direction, and the coordinate axis of the LRF constructed by the spatial distribution information is easily affected by noise, but is relatively stable in a complex scene. A stable local coordinate system has a significant influence on various performances of a descriptor, and it is assumed that data acquired under different postures are greatly different for a specific feature point, so that spatial information of a three-dimensional object cannot be well utilized, but the distribution of neighbor points around the feature point does not change greatly. Then, important spatial information is extracted based on the LRF, after the LRF is established, neighbor points around the feature point are projected into the established LRF, and new coordinates of the neighbor points are obtained. Fig. 3 and 4 demonstrate the repeatability of the local coordinate system. Where fig. 3 is a local coordinate system at a feature point in the model and fig. 4 is a local coordinate system of a corresponding feature point in the scene. It can be seen that in the model and the scene, the directions of the coordinate axes of the local coordinate system of the same feature point relative to the feature point are substantially the same, which is a local coordinate system with strong repeatability, and the feature extraction performed on the coordinate system leads to higher feature reproducibility, thereby obtaining better experimental results.
Because the local descriptor is a feature extracted from an already established local coordinate system, if there is no coordinate system with strong repeatability, the extracted features on the same feature point may be very different, and the matching result may be greatly influenced. A stable and efficient local coordinate system is the basis for a good local descriptor, and fig. 5 to 8 illustrate the effect of an unstable coordinate system on the feature matching effect.
In fig. 5 to 8, fig. 5, 6, 7 and 8 are the same model, 1000 points are randomly selected, and the feature of the feature point is extracted on a given local coordinate system by using the SHOT descriptor, and the feature matching between two objects is performed. Fig. 5, fig. 6, fig. 7, and fig. 8 show the matching between two models after extracting features when the angle deviation between the LRF and the original LRF is 0 degree, 10 degrees, 20 degrees, and 30 degrees, respectively, there are many cases of mismatching when the deviation angle is 10 degrees, and the proportion of correctly matched features is very small when the deviation angle is 20 degrees or more, and it can be seen that a stable local coordinate system has a crucial influence on the performance of a local descriptor.
The local coordinate system establishing method of TOLDI makes the repeatability of the coordinate system in complex scenes high. However, when the z-axis is established, only a small part of the neighboring points is taken to establish the covariance matrix, and the purpose of doing so is to reduce the influence of occlusion, because on an occluded three-dimensional object, points farther away from the feature point are more likely to be occluded, and points in a close range of the feature point are less likely to be occluded. However, when the point is selected, information used for establishing the z-axis is too sparse, and the z-axis is generally established by using information of all neighbor points around the feature point. The lack of information may cause the difference between the established z-axis and the true z-axis to be too large, especially on some objects without occlusion, which is equivalent to white and wastes much information, so we can try to use the information of all neighboring points to construct a covariance matrix to fully utilize the information of the neighboring nodes according to the previous inspiration, but add weights to each point during construction, where the weights are determined according to the distance of the point from the feature point, and points closer to each other are given greater weights, which can make the local coordinate system have certain robustness to occlusion, and use more information to construct the z-axis on objects without occlusion, which is a more stable and efficient scheme. The improved LRF establishment method is given below.
For a feature point p with a support radius (i.e. the range of the surrounding neighboring nodes) of r, a covariance matrix C can be constructed at point p according to the neighboring nodes within the surrounding radius r:
where n denotes the number of points in the neighborhood, qiThe neighbor node with the number i is shown, and here, the place different from the original method for establishing the local coordinate system of the TOLDI is to utilize all the neighbor nodes to construct the covariance matrix instead of a small part of the neighbor nodes. And the step of calculating the center of gravity of the neighboring points, w, is dispensed withiThe weight of the neighbor node with the number i is represented, and the calculation method is as follows:
wi＝r-||p-qi||2，
then, calculating a z-axis n (p) according to the obtained covariance matrix, wherein the covariance matrix has three eigenvectors, the eigenvectors are arranged in an ascending order according to the sizes of corresponding eigenvalues, the eigenvector corresponding to the minimum eigenvalue is taken as a z-axis, the determination of the z-axis direction is the same as that in the original TOLDI descriptor, and the final z-axis z (p) is obtained by normalizing the calculated sign of the product result between the z-axis and all neighbor point vectors, and k is the number of neighbor nodes:
the rest of the part includes the establishment of the x-axis and the y-axis, consistent with the method of the original TOLDI descriptor. Through the improvement, the improved local coordinate system has stronger repeatability in both an occluded scene and an unoccluded scene, and particularly in the unoccluded scene, the repeatability is stronger than that of the original coordinate system, because the information of more neighborhood points is utilized to construct.
2. Improvements to feature extraction
Extracting features, including geometric features, spatial features, and their statistical features, typically on an already established local coordinate system, makes the extracted features more stable and descriptive. The method has the advantages that the points in the space are known to have loss of space information when being projected to a two-dimensional plane from a three-dimensional space, and the information of a certain dimension is lost, so that the extracted feature richness is insufficient, therefore, the spatial information of the points can be completely extracted to serve as the feature when the richer feature is extracted, on one hand, the extraction is more convenient, and on the other hand, the contained information is richer. In the TOLDI descriptor, the extracted spatial features are the projection distances from the representative points in each cell to the projection plane, the original spatial information of the feature points can be well restored by collecting the projection distances on the three planes, but the mismatching condition may occur during the matching, and for the condition, the included angle between the representative points and the z axis can be increased as another feature on the basis that the original projection distances from the representative points are extracted as the features, so that the information of the descriptor is enriched on one hand, and the misjudgment condition of matching some spatial information can be avoided on the other hand.
As will be described in detail below from the specific process of forming the TOLDI descriptor, after the local coordinate system is established, we draw several planes parallel to the three coordinate planes and at a distance of the support radius, all neighboring nodes around the feature point are projected onto the three new planes respectively, then dividing the limited projection area into N × N square grids on the projection plane, then counting the projection distance from each point in the grids to the plane on the grids, selecting the point with the minimum projection distance as the representative point of the grid, taking the projection distance of the point as the characteristic of the grid, this is considered from the viewpoint of human, because when looking at the surrounding objects from the eyes of human, the points closer to the eyes are more easily perceived by the eyes and have a corresponding more important role, and therefore, the point with the smallest projection distance is selected as the representative point. The feature of the feature point is obtained by integrating the features in the grids on the three projection planes. A graphical representation of the projection is shown in fig. 9. In fig. 9, the left side is a schematic diagram of projection of the neighbor points in the LRF onto a plane, and the right side is a point into which each mesh falls after the mesh is divided on the projection area, where the point with the minimum projection distance is also referred to as a representative point.
Although the projected distance features can well represent the spatial characteristics of the points, the orientation of the points is determined due to the lack of geometric information, and the value of each patch is the minimum distance value. In this case, there are some cases of incorrect matching, as shown in fig. 10, in the matching process, the representative points in the two small blocks are assumed to have the same or similar projection distances, and it is very likely that the two grids are the same when matching is performed, as can be seen from the figure, the projection positions of the two small blocks are far apart and the distribution of the points in the small blocks is also different, and the two small blocks belong to different parts and should not be correct matching, but if the orientation information is not considered and only the spatial information is considered, incorrect matching is caused.
To avoid such a mismatch, a geometric feature may be added to each patch to determine the location of the representative point of each patch. The geometric feature added here is the angle of the representative point in the patch to the z-axis in the local coordinate system. Thus each patch will have two features, the projected distance of the representative point and the angle of the representative point to the z-axis. The reason why the angles between the table points and all the coordinate axes are not replaced is that the dimension of the features needs to be reduced as much as possible, and the mismatching situations are not many, and can be generally detected by only one angle, so that the descriptor is more descriptive, and some mismatching situations are avoided. A schematic of the added angular feature is shown in fig. 11. Where p is the representative point in a certain small block and β is the angular feature we need to add.
In general, in all application scenarios including simple scenarios and some complex scenarios with three-dimensional object overlapping and occlusion, the improved local descriptor has the most stable descriptive property and has an unsophisticated performance under the complex scenario, and is a very stable and descriptive descriptor, i improve the descriptor not only over the TOLDI descriptor and most other local descriptors in the simple scenario, but also over the complex scenario, we use a weighting method to make the influence of some points which are likely to be wrong or missing on constructing the local coordinate system smaller, so that it has a good performance under the complex scenario, and is only slightly worse than the performance of the TOLDI descriptor. But in the case of simple scenarios, the performance of our improved descriptor is much higher than the TOLDI descriptor. In summary, the improved descriptors achieve good results in both descriptive and stability, and are generally superior to other local descriptors mentioned herein.
In summary, according to the technical scheme, three new planes are drawn at a position which is parallel to three coordinate planes of a pre-established local coordinate system and is at a distance of a support radius r, and all neighbor nodes around a feature point p in a three-dimensional object are respectively projected onto the three new planes; dividing a limited projection area into a plurality of square grids on a projection plane; calculating the projection distance from each neighbor node in the grids to the plane on the square grids, and selecting the point with the minimum projection distance as the representative point of the square grids; and extracting the features of the feature points according to the projection distance from the representative point to the plane and the included angle between the representative point and the z axis z (p) in the local coordinate system, so that the improved local coordinate system has stronger repeatability in a common scene, can achieve better effect under some shielding conditions, prevents the wrong matching condition in the matching process, enriches the features and reduces the calculation complexity.
In order to achieve the above object, the present invention further provides a 3D feature extraction system based on a TOLDI descriptor, where the system includes a memory, a processor, and a 3D feature extraction program based on a TOLDI descriptor stored on the processor, and when the 3D feature extraction program based on a TOLDI descriptor is executed by the processor, the steps of the method according to the above embodiments are executed, and are not described herein again.
In order to achieve the above object, the present invention further provides a computer-readable storage medium, where a 3D feature extraction program based on a TOLDI descriptor is stored on the computer-readable storage medium, and when the 3D feature extraction program based on the TOLDI descriptor is executed by a processor, the steps of the method according to the above embodiments are performed, which is not described herein again.
The above description is only for the preferred embodiment of the present invention and is not intended to limit the scope of the present invention, and all equivalent structures or flow transformations made by the present specification and drawings, or used directly or indirectly in other related fields are included in the scope of the present invention.
Claims (9)
1. A method for 3D feature extraction based on TOLDI descriptors, the method comprising the steps of:
drawing three new planes at a position which is parallel to three coordinate planes of a pre-established local coordinate system and has a distance of a support radius r, and respectively projecting all neighbor nodes around a characteristic point p in the three-dimensional object onto the three new planes;
dividing a limited projection area into a plurality of square grids on a projection plane;
calculating the projection distance from each neighbor node in the grids to the plane on the square grids, and selecting the point with the minimum projection distance as the representative point of the square grids;
and extracting the features of the feature points according to the projection distance from the representative point to a plane and the included angle between the representative point and the z axis z (p) in the local coordinate system.
2. The method of claim 1, wherein the step of drawing three new planes parallel to the three coordinate planes of the pre-established local coordinate system and at a distance of a support radius r is preceded by the step of drawing three new planes based on the TOLDI descriptor:
constructing a covariance matrix by using information of all neighbor nodes around the characteristic point p;
and establishing the local coordinate system according to the covariance matrix.
3. The TOLDI descriptor-based 3D feature extraction method according to claim 2, wherein said step of constructing a covariance matrix using information of all neighboring nodes around a feature point p comprises:
constructing a covariance matrix C on the characteristic point p according to all neighbor nodes in the peripheral radius r:
wherein q isiDenotes the neighbor node numbered i, n denotes the number of points in the neighbor, wiThen the weight, w, that the neighbor node numbered i hasiThe calculation method is as follows:
wi＝r-||p-qi||2。
4. the TOLDI descriptor-based 3D feature extraction method according to claim 3, wherein said covariance matrix has three eigenvectors, and the step of building said local coordinate system based on said covariance matrix comprises:
arranging the three eigenvectors in an ascending order according to the sizes of the corresponding eigenvalues, and taking the eigenvector corresponding to the minimum value as a z axis;
obtaining a final z-axis z (p) according to the sign of the product result between the z-axis and all the neighbor node vectors:
where k is the number of all neighbor nodes.
5. The TOLDI descriptor-based 3D feature extraction method according to claim 4, wherein said step of establishing said local coordinate system according to said covariance matrix further comprises:
calculate each neighbor node qiVector v projected onto tangent planei：
vi＝pqi-(pqi.z(p)).z(p)，
For all vectors viWeighting to form the x-axis x (p) at the feature point p:
6. the TOLDI descriptor-based 3D feature extraction method of claim 5, wherein vector viIn the step of weighting to form the x-axis x (p) at the characteristic point p, two weights are calculated as follows:
wi1＝(r-||p-qi||)2，
wi2＝(pqi.z(p))2。
7. the TOLDI descriptor-based 3D feature extraction method according to claim 6, wherein said step of establishing said local coordinate system according to said covariance matrix further comprises: and obtaining a y axis according to the cross product of the z axis z (p) and the x axis x (p).
8. A TOLDI descriptor-based 3D feature extraction system, the system comprising a memory, a processor, and a TOLDI descriptor-based 3D feature extraction program stored on the processor, the TOLDI descriptor-based 3D feature extraction program when executed by the processor performing the steps of the method of any one of claims 1-7.
9. A computer-readable storage medium, having stored thereon a TOLDI descriptor-based 3D feature extraction program, which when executed by a processor performs the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010735430.3A CN112001261A (en) | 2020-07-28 | 2020-07-28 | 3D feature extraction method, system and medium based on TOLDI descriptor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010735430.3A CN112001261A (en) | 2020-07-28 | 2020-07-28 | 3D feature extraction method, system and medium based on TOLDI descriptor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112001261A true CN112001261A (en) | 2020-11-27 |
Family
ID=73467704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010735430.3A Pending CN112001261A (en) | 2020-07-28 | 2020-07-28 | 3D feature extraction method, system and medium based on TOLDI descriptor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112001261A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809456A (en) * | 2015-05-21 | 2015-07-29 | 重庆大学 | Three-dimensional target recognition method based on two-value descriptor |
| CN108256529A (en) * | 2017-11-29 | 2018-07-06 | 深圳慎始科技有限公司 | Global point cloud based on Dian Yun projected outlines signature and distribution matrix describes method |
| CN109215129A (en) * | 2017-07-05 | 2019-01-15 | 中国科学院沈阳自动化研究所 | A kind of method for describing local characteristic based on three-dimensional point cloud |
| CN110781937A (en) * | 2019-10-16 | 2020-02-11 | 广州大学 | Point cloud feature extraction method based on global visual angle |
-
2020
- 2020-07-28 CN CN202010735430.3A patent/CN112001261A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809456A (en) * | 2015-05-21 | 2015-07-29 | 重庆大学 | Three-dimensional target recognition method based on two-value descriptor |
| CN109215129A (en) * | 2017-07-05 | 2019-01-15 | 中国科学院沈阳自动化研究所 | A kind of method for describing local characteristic based on three-dimensional point cloud |
| CN108256529A (en) * | 2017-11-29 | 2018-07-06 | 深圳慎始科技有限公司 | Global point cloud based on Dian Yun projected outlines signature and distribution matrix describes method |
| CN110781937A (en) * | 2019-10-16 | 2020-02-11 | 广州大学 | Point cloud feature extraction method based on global visual angle |
Non-Patent Citations (1)
| Title |
|---|
| 庞震: "三维物体中的局部描述符研究", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 02, pages 138 - 1684 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109753940B (en) | Image processing method and device | |
| US20180315221A1 (en) | Real-time camera position estimation with drift mitigation in incremental structure from motion | |
| US20180315232A1 (en) | Real-time incremental 3d reconstruction of sensor data | |
| CN108986161A (en) | A kind of three dimensional space coordinate estimation method, device, terminal and storage medium | |
| US20230249076A1 (en) | Collision data processing method and apparatus, storage medium, program product, and electronic device | |
| US9613457B2 (en) | Multi-primitive fitting device and operation method thereof | |
| WO2017107865A1 (en) | Image retrieval system, server, database, and related method | |
| CN105654552B (en) | A kind of quick Delaunay network construction methods towards Arbitrary distribution large-scale point cloud data | |
| CN108961385B (en) | SLAM composition method and device | |
| CN110807781A (en) | Point cloud simplification method capable of retaining details and boundary features | |
| CN104966307B (en) | A kind of AR method based on real-time tracking | |
| CN109493384A (en) | Camera position and orientation estimation method, system, equipment and storage medium | |
| Li et al. | 4FP-structure: A robust local region feature descriptor | |
| CN115329111B (en) | Image feature library construction method and system based on point cloud and image matching | |
| CN111415420A (en) | Spatial information determination method and device and electronic equipment | |
| CN115018992A (en) | Method and device for generating hair style model, electronic equipment and storage medium | |
| Liu et al. | A novel rock-mass point cloud registration method based on feature line extraction and feature point matching | |
| WO2017107866A1 (en) | Image retrieval server and system, related retrieval and troubleshooting method | |
| CN114202632A (en) | Grid linear structure recovery method and device, electronic equipment and storage medium | |
| CN111652163B (en) | Transmission line tower line segment matching method and equipment | |
| CN111597367B (en) | Three-dimensional model retrieval method based on view and hash algorithm | |
| Chen et al. | Multi-stage matching approach for mobile platform visual imagery | |
| CN112001261A (en) | 3D feature extraction method, system and medium based on TOLDI descriptor | |
| CN112002007A (en) | Model obtaining method and device based on air-ground image, equipment and storage medium | |
| CN111161289A (en) | Method, device and computer program product for improving contour precision of object in image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |