CN114373002B

CN114373002B - Point cloud registration method and device

Info

Publication number: CN114373002B
Application number: CN202210001281.7A
Authority: CN
Inventors: 陈汉清; 王徽; 余晨; 周迪斌; 戴维焕
Original assignee: Hangzhou Santan Medical Technology Co Ltd
Current assignee: Hangzhou Santan Medical Technology Co Ltd
Priority date: 2022-01-04
Filing date: 2022-01-04
Publication date: 2023-08-18
Anticipated expiration: 2042-01-04
Also published as: CN114373002A

Abstract

The embodiment of the invention provides a point cloud registration method and a point cloud registration device, which relate to the technical field of data processing, and the method comprises the following steps: obtaining target point cloud data; dividing each data point in the target point cloud data into different clusters; selecting a first target combination from among the respective data point combinations; calculating an adjustment parameter; adjusting the position information of the data points belonging to the point cloud data to be adjusted in the target point cloud data; if the target point cloud data after the position information of the data points is adjusted does not meet the preset registration condition, returning to the step of dividing each data point in the target point cloud data into different clusters; and if the target point cloud data after the position information of the data point is adjusted meets the registration condition, taking the target point cloud data after the position information of the data point is adjusted as a point cloud registration result. By applying the point cloud registration scheme provided by the embodiment of the invention, the efficiency of point cloud registration can be improved.

Description

Point cloud registration method and device

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a point cloud registration method and device.

Background

A group of point cloud data of an object such as a person, an animal, etc. generally includes a plurality of data points, and the position information of each data point may reflect a part of the position of the object under the coordinate system where the point cloud data is located, and the position information of the plurality of data points may reflect an overall position of the whole object under the coordinate system where the point cloud data is located. The point cloud registration is to adjust the position information of the data points in the point cloud data in a translation and rotation mode so as to enable the whole positions of the objects reflected by the position information of the data points in different point cloud data to be identical.

In the existing scheme for performing point cloud registration on two sets of point cloud data, for each data point in one set of point cloud data, a corresponding point corresponding to the data point needs to be found in the other set of point cloud data, for example, the corresponding point may be a data point closest to the data point, then parameters such as translation and rotation are calculated based on the distance between each data point in the set of point cloud data and the corresponding point corresponding to each data point, and the position information of the data point in any one set of point cloud data is adjusted according to the calculated parameters to realize point cloud registration. However, since the point cloud data typically contains more data points, it takes longer to determine the corresponding points, resulting in less efficient point cloud registration.

Disclosure of Invention

The embodiment of the invention aims to provide a point cloud registration method and device for improving the efficiency of point cloud registration. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a point cloud registration method, where the method includes:

obtaining target point cloud data of data points in two groups of initial point cloud data of the same object;

dividing each data point in the target point cloud data into different clusters based on the position information of each data point in the target point cloud data and a preset first distance threshold;

In each cluster, for each first data point belonging to one initial point cloud data in the cluster, selecting a first target combination for parameter calculation from various data point combinations containing the first data point, wherein one data point combination contains: a first data point and a second data point in the cluster belonging to another initial point cloud data;

calculating an adjustment parameter according to the distance between the first data point and the second data point in each first target combination;

and adjusting the position information of the data points belonging to the point cloud data to be adjusted in the target point cloud data based on the adjustment parameters, wherein the point cloud data to be adjusted is: any one of the two sets of initial point cloud data;

if the target point cloud data after the position information of the data points is adjusted does not meet the preset registration condition, returning to the step of dividing each data point in the target point cloud data into different clusters;

and if the target point cloud data after the position information of the data point is adjusted meets the registration condition, taking the target point cloud data after the position information of the data point is adjusted as a point cloud registration result.

In one embodiment of the present invention, in each cluster, for each first data point belonging to one initial point cloud data in the cluster, a first target combination for parameter calculation is selected from respective data point combinations including the first data point, including:

In each cluster, for each first data point belonging to one initial point cloud data in the cluster, a data point combination having the shortest distance between two data points included is selected as a first target combination from among respective data point combinations including the first data point.

for each cluster in the target point cloud data, obtaining a weight coefficient of each data point combination in the cluster, wherein the weight coefficient is calculated according to the distance between two data points in the data point combination, the offset and the rotation angle, and the offset is as follows: the relative distance between two data points in the data point combination on a first preset coordinate axis of a preset three-dimensional coordinate system is as follows: an included angle between a straight line where two data points in the data point combination are located and a second preset coordinate axis of the preset three-dimensional coordinate system;

in each cluster, for each first data point belonging to one initial point cloud data in the cluster, a data point combination with the largest weight coefficient is selected as a first target combination from the data point combinations containing the first data point.

In one embodiment of the present invention, the obtaining the target point cloud data of each data point in the two sets of initial point cloud data containing the same object includes:

obtaining two three-dimensional images of the same object;

constructing a three-dimensional model of the object according to each three-dimensional image and generating point cloud data of the object according to the three-dimensional model;

and merging the two groups of generated point cloud data to obtain merged point cloud data serving as target point cloud data.

obtaining two groups of initial point cloud data of the same object;

respectively carrying out downsampling treatment on the two groups of initial point cloud data;

and merging the two groups of initial point cloud data after the downsampling processing to obtain merged target point cloud data.

In one embodiment of the present invention, before dividing each data point in the target point cloud data into different clusters by performing a point cloud segmentation process on the target point cloud data based on the position information of each data point in the target point cloud data and a preset first distance threshold, the method further includes:

Calculating initial adjustment parameters according to the position information of each data point in the target point cloud data;

based on the initial adjustment parameters, adjusting position information of data points belonging to the cloud data to be fixed in the cloud data of the target point to obtain adjusted cloud data of the target point, wherein the cloud data to be fixed is: any one of the initial point cloud data.

In one embodiment of the present invention, the registration condition is:

the distance between two data points contained in each second target combination is smaller than a preset second distance threshold, wherein the second target combination is as follows: and a first target combination redetermined in the target point cloud data after the data points are adjusted.

In one embodiment of the present invention, a distance between the first data point and the second data point included in the data point combination is less than a preset third distance threshold.

In a second aspect, an embodiment of the present invention further provides a point cloud registration apparatus, where the apparatus includes:

the data acquisition module is used for acquiring target point cloud data of data points in two groups of initial point cloud data of the same object;

the data point dividing module is used for dividing each data point in the target point cloud data into different clusters based on the position information of each data point in the target point cloud data and a preset first distance threshold value;

A combination selection module, configured to, in each cluster, select, for each first data point belonging to one initial point cloud data in the cluster, a first target combination for parameter calculation from respective data point combinations including the first data point, where one data point combination includes: a first data point and a second data point in the cluster belonging to another initial point cloud data;

the first calculation module is used for calculating adjustment parameters according to the distance between the first data point and the second data point in each first target combination;

the first adjustment module is configured to adjust, based on the adjustment parameter, position information of a data point belonging to point cloud data to be adjusted in the target point cloud data, where the point cloud data to be adjusted is: any one of the two sets of initial point cloud data;

the condition judging module is used for judging whether the cloud data of the target point after the position information of the data point is adjusted meets the preset registration condition, if so, the result determining module is triggered, and if not, the data point dividing module is triggered;

the result determining module is used for taking the target point cloud data after the position information of the data points is adjusted as a point cloud registration result.

In one embodiment of the present invention, the combination selection module is specifically configured to:

In one embodiment of the present invention, the data obtaining module is specifically configured to:

obtaining two three-dimensional images of the same object;

obtaining two groups of initial point cloud data of the same object;

In one embodiment of the invention, the apparatus further comprises:

the second calculation module is used for calculating initial adjustment parameters according to the position information of each data point in the target point cloud data before dividing each data point in the target point cloud data into different clusters by carrying out point cloud segmentation processing on the target point cloud data based on the position information of each data point in the target point cloud data and a preset first distance threshold;

The second adjustment module is configured to adjust, based on the initial adjustment parameter, position information of data points belonging to cloud data to be fixed in the target point cloud data, to obtain adjusted target point cloud data, where the cloud data to be fixed is: any one of the initial point cloud data.

In one embodiment of the present invention, the registration condition is:

In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

and the processor is used for realizing the point cloud registration method step in the first aspect when executing the program stored in the memory.

In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, where a computer program is stored, where the computer program is executed by a processor to implement the steps of the point cloud registration method described in the first aspect.

The embodiment of the invention has the beneficial effects that:

as can be seen from the above, when the scheme provided by the embodiment of the invention is applied to point cloud registration, each data point in the target point cloud data is divided into different clusters, so that the target data point combination can be determined in a plurality of clusters synchronously, each cluster only contains partial data points in the target point cloud data, and fewer data points need to be processed in the process of determining the target data point combination, thereby improving the efficiency of determining the target data point combination and further improving the efficiency of point cloud registration.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other embodiments may be obtained according to these drawings to those skilled in the art.

Fig. 1 is a schematic flow chart of a first point cloud registration method according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a second point cloud registration method according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of a third point cloud registration method according to an embodiment of the present invention;

fig. 4 is a flowchart of a fourth point cloud registration method according to an embodiment of the present invention;

fig. 5 is a flowchart of a fifth point cloud registration method according to an embodiment of the present invention;

fig. 6 is a flowchart of a sixth point cloud registration method according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a first point cloud registration device according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a second point cloud registration apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by the person skilled in the art based on the present invention are included in the scope of protection of the present invention.

Referring to fig. 1, fig. 1 is a flowchart of a first point cloud registration method according to an embodiment of the present invention, where the method includes the following steps S101 to S107.

Step S101: target point cloud data of data points in two groups of initial point cloud data of the same object are obtained.

Wherein the subject may be a human, an animal, or the like.

The initial point cloud data may be point cloud data converted from a three-dimensional image or a three-dimensional model of the object, and the two sets of initial point cloud data may be converted from different three-dimensional images or three-dimensional models of the object.

The data points in the point cloud data may be represented as W _i ＝{X _i ，Y _i ，Z _i ，K _i (wherein X is _i ，Y _i ，Z _i Respectively represent three coordinates, K, of the data point in a preset three-dimensional coordinate system _i Other information representing the data Point, such as color, gray value, etc., i represents the data Point as the i-th data Point in the Point Cloud data, which may be represented as Point cloud= { W ₁ ，W ₂ ，W ₃ ，……，W _n Where n represents the total number of data points contained in the point cloud data.

Specifically, the following two cases exist for obtaining the target point cloud data.

First case: two sets of initial point cloud data of the same object may be first obtained, and then the two sets of initial point cloud data are combined, so that the combined point cloud data is used as target point cloud data.

Second case: the data processing device with the point cloud data merging function may first obtain the two sets of initial point cloud data, then merge the two sets of initial point cloud data to obtain merged point cloud data, and the execution body of the scheme may obtain the merged point cloud data and use the point cloud data as target point cloud data.

Combining the two sets of initial point cloud data can be understood as overlapping all data points contained in the two sets of point cloud data, and the overlapped point cloud data is the target point cloud data. At this time, the target point cloud data includes all data points in the two sets of initial point cloud data.

For example, if a set of initial point cloud data is { P } ₁ ，P ₂ ，P ₃ ，……，P _u Another set of initial point cloud data is { Q } ₁ ，Q ₂ ，Q ₃ ，……，Q _v Combining the two sets of point cloud data to obtain combined point cloud data as { P } ₁ ，P ₂ ，P ₃ ，……，P _u ，Q ₁ ，Q ₂ ，Q ₃ ，……，Q _v }。

In addition, since there may be a few extreme data points in the initial point cloud data, the extreme data points may be understood as data points that are farther apart from other data points. In this case, other data points except for the extreme data points in the two sets of initial point cloud data may be superimposed, and the superimposed point cloud data may be used as the target point cloud data. At this time, the target point cloud data includes partial data points in the two sets of initial point cloud data.

Step S102: and dividing each data point in the target point cloud data into different clusters based on the position information of each data point in the target point cloud data and a preset first distance threshold.

The data points can be understood as space points in a preset three-dimensional space coordinate system, and the position information of the data points can be understood as coordinates of the space points.

The first distance threshold may be a manually set distance.

Specifically, one data point may be selected as an initial data point from the cloud data of the target point, and then the distances between other data points and the initial data point are calculated according to the initial data point and the position information of other data points, so that the data point with the distance smaller than the first distance threshold value between the other data points and the initial data point is found, and the found data point and the initial data point are used as the data points belonging to the same cluster. After the first cluster is obtained by dividing, a plurality of clusters can be continuously divided from the remaining data points of the target point cloud data according to the mode until all the data points in the target point cloud data are divided into each cluster.

By dividing the data points in the target point cloud data, a plurality of clusters can be obtained, and each cluster comprises a plurality of data points.

In addition, since the target point cloud data includes two data points of the initial point cloud data, when determining a data point having a distance from the initial data point smaller than the first distance threshold, the determined data points may have the same source as the initial data point, i.e., all data points belonging to the same initial point cloud data, or may have different sources from the initial data point, i.e., data points belonging to different initial point cloud data. Thus, in each cluster divided, data points belonging to two initial point cloud data may be included.

In one embodiment of the invention, each data point in the target point cloud data can be divided by the existing KD-tree (k-dimensional tree) algorithm to obtain a plurality of clusters.

In addition, since the target point cloud data is usually three-dimensional point cloud data, each data point in the target point cloud data can be divided by using a three-dimensional KD-tree algorithm.

Step S103: in each cluster, for each first data point belonging to one initial point cloud data in the cluster, a first target combination for parameter calculation is selected from respective data point combinations including the first data point.

Wherein one data point combination comprises: a first data point and a second data point in the cluster belonging to another initial point cloud data.

For example, if the two initial point cloud data are the point cloud data m and the point cloud data n, the first data point may be a data point belonging to the point cloud data m or a data point belonging to the point cloud data n. In the case where the first data point belongs to the point cloud data m, the second data point belongs to the point cloud data n, and similarly, in the case where the first data point belongs to the point cloud data n, the second data point belongs to the point cloud data m.

Specifically, in each cluster, for each first data point belonging to one initial point cloud data in the cluster, the first data point may form a plurality of data point combinations with each second data point belonging to another initial point cloud data in the cluster, the number of the formed data point combinations is the same as the number of the second data points in the cluster, and after the plurality of data point combinations are obtained, a first target combination for parameter calculation may be selected from the plurality of data point combinations. A first target combination may be selected from a plurality of data point combinations corresponding to a first data point, and then the same number of first target combinations as the first data point may be selected from the plurality of data point combinations corresponding to the first data point.

For example, if there are 3 first data points and 4 and second data points in a cluster, 4 data point combinations containing the first data points may be obtained for each first data point, one first target combination may be selected from the 4 data point combinations, and three first target combinations may be selected for the 3 first data points.

Specific implementations of selecting the first target combination from the plurality of data point combinations may be found in the description of step S103A in the embodiment of fig. 2 and the description of steps S103B-S103C in the embodiment of fig. 3 that follow, and are not described in detail herein.

In an embodiment of the present invention, a distance between the first data point and the second data point included in the data point combination is smaller than a preset third distance threshold.

The third distance threshold may be a manually set distance.

Specifically, in each cluster, the distance between the first data point and a different second data point may be different for each first data point in the cluster. In determining the data point combination including the first data point, a second data point having a distance from the first data point less than the third distance threshold may be found according to the distance between the first data point and each second data point, thereby determining the data point combination including the second data point and the first data block.

In the above-mentioned scheme, since there may be second data points in the cluster where the first data point is located, where the distance between the second data point and the first data point is greater than the third distance threshold, if the distance between the first data point and the second data point included in the data point combination is not limited to be less than the third distance threshold, the same number of data point combinations as the second data point in the cluster may be determined for each first data point in each cluster, and if the distance between the first data point and the second data point is limited to be less than the third distance threshold, the determined number of data point combinations including the first data point is less than the number of second data points, so that the selection of the first target combination in fewer data point combinations can improve the selection efficiency of the first target combination.

Step S104: an adjustment parameter is calculated based on the distance between the first data point and the second data point in each of the first target combinations.

Wherein, the adjustment parameters comprise a rotation parameter and a translation parameter.

In one embodiment of the present invention, the adjustment parameter may be calculated according to the distance between two data points in each first target combination by calculating parameters such as translation and rotation in an existing point cloud registration scheme, for example, the existing point cloud registration scheme may be an ICP (Iterative Closest Point ) registration scheme, or other existing point cloud registration schemes that implement registration based on the distance between data points in the point cloud data, which is not limited in the embodiment of the present invention.

In another embodiment of the present invention, the rotation parameter may be further solved according to progressive non-convexity between data points in each first target combination and a distance between two data points in each first target combination based on a non-convexity optimization algorithm and a least-squares method, and the translation parameter may be solved by an adaptive voting mechanism.

The above method for solving the rotation parameters may be understood as performing global rotation estimation on all data points belonging to one initial point cloud data in the target point cloud data, and calculating a plurality of locally optimal rotation parameters, so as to select an optimal rotation parameter from the plurality of rotation parameters.

Step S105: and adjusting the position information of the data points belonging to the point cloud data to be adjusted in the target point cloud data based on the adjustment parameters.

The point cloud data to be adjusted are as follows: any one of the two sets of initial point cloud data.

For example, the point cloud data to be adjusted may be initial point cloud data where the first data point is located, or may be initial point cloud data where the second data point is located.

Specifically, the adjustment direction for adjusting the data points can be determined according to the relative positions between the data points belonging to the point cloud data to be adjusted and the data points belonging to the other point cloud data in the target point cloud data, and then the position information of each data point of the data to be adjusted is adjusted according to the adjustment direction and the parameter value of the adjustment parameter.

The position information of the data points can be represented in the form of three-dimensional coordinates, the adjustment parameters comprise rotation parameters and translation parameters, and for each data point in the point cloud data to be adjusted, the three-dimensional coordinates of the data point can be transformed according to the adjustment parameters, so that the adjustment of the position information is realized.

For example, when the position information of a data point is adjusted based on the rotation parameter, it can be considered that the data point rotates in the three-dimensional space coordinate system with the origin of the three-dimensional space coordinate system as the rotation center according to the rotation parameter.

The translation parameter may include translation amounts on three coordinate axes in the three-dimensional space coordinate system, and when the position information of the data point is adjusted based on the translation parameter, the three-dimensional coordinates of the data point may be adjusted based on the three translation amounts in the directions of the three coordinate axes, respectively.

Step S106: judging whether the target point cloud data after the position information of the data points is adjusted meets the preset registration condition, if so, executing step S107; if not, return to step S102.

The registration condition can be used for verifying whether the target point cloud data after the position information of the data points is adjusted reach the expected registration effect, namely whether the whole positions of the objects reflected by the position information of the data points belonging to different initial point cloud data in the target point cloud data are consistent or not, or whether the consistency is higher.

Specifically, the registration condition can be set first, then whether the target point cloud data after the position information is adjusted meets the registration condition is judged, if not, the target point cloud data is not up to the expected registration effect, at the moment, the adjustment parameter needs to be recalculated, so that the position information of the data points in the target point cloud data is adjusted again until the target point cloud data after the position information is adjusted meets the registration condition; if the target point cloud data after the position information is adjusted meets the registration condition, it is indicated that the target point cloud data has reached the expected registration effect, and at this time, the following step S107 is executed.

Step S107: and taking the target point cloud data after the position information of the data points is adjusted as a point cloud registration result.

The target point cloud data satisfying the registration condition may be considered to have reached the intended registration effect, and therefore, the target point cloud data at this time may be taken as the point cloud registration result.

The above registration conditions are explained below.

In one embodiment of the present invention, the above registration condition is: the distance between two data points contained in each second target combination is smaller than a preset second distance threshold value.

Wherein the second target combination is: and a first target combination redetermined in the target point cloud data after the data points are adjusted.

The second distance threshold may be a manually preset distance, and the second distance threshold is generally smaller than the first distance threshold.

Specifically, for the target point cloud data after the position information of the data points is adjusted, each data point in the target point cloud data can be divided into different clusters again, and a first target combination is selected from each cluster again, and at this time, the selected first target combination is the second target combination.

For each second target combination, a distance between two data points in the second target combination can be calculated according to the position information of the two data points, and whether the distance is smaller than a second distance threshold value is judged.

If the second target combinations with the data point distances larger than the second distance threshold exist in the second target combinations, the cloud data of the target points after the position information of the data points is adjusted does not meet the registration condition; if the data point distance of each second target combination is smaller than the second distance threshold value, the target point cloud data after the position information of the data points is adjusted is indicated to meet the registration condition.

In the above aspect, when the target point cloud data satisfies the above registration condition, the distance between two data points in each first target combination in the target point cloud data is smaller than the above second distance threshold, at this time, the two data points may be considered to be coincident, and the distances between the data points in all the first target combinations are smaller than the second distance threshold, then most data points respectively belonging to two initial point cloud data in the target point cloud data may be considered to be coincident, and the position information of most data points in the initial point cloud data may reflect the overall position of the object, so that, in the target point cloud data satisfying the registration condition, the two overall positions of the same object reflected by the position information of each data point in the target point cloud data may be considered to be coincident. Therefore, the accuracy of point cloud registration is higher by applying the registration condition in the scheme.

In another embodiment of the present invention, since the process of calculating the adjustment parameter according to the position information of the data point in the target point cloud data and adjusting the position information of the data point in the target point cloud data according to the adjustment parameter is usually a continuous iterative optimization process, the error of each iterative optimization process can be calculated according to the position information of the data point in the target point cloud data before and after iterative optimization by using the existing iterative optimization algorithm. Therefore, the registration condition may be that an error calculated by the iterative optimization algorithm is smaller than a preset optimization error.

If the calculated error is larger than the optimized error, the cloud data of the target point after the position information of the data point is adjusted does not meet the registration condition; if the calculated error is smaller than the optimized error, the cloud data of the target point after the position information of the data point is adjusted is indicated to meet the registration condition.

The iterative optimization algorithm may be a cma_es optimization algorithm, or may be another optimization algorithm, which is not limited in this embodiment of the present invention.

A specific implementation manner of selecting the first target combination in step S103 is described below.

In an embodiment of the present invention, referring to fig. 2, a schematic flow chart of a second point cloud registration method is provided, and in this embodiment, compared with the embodiment shown in fig. 1, the step S103 may be implemented by the following step S103A.

Step S103A: in each cluster, for each first data point belonging to one initial point cloud data in the cluster, a data point combination having the shortest distance between two data points included is selected as a first target combination from among respective data point combinations including the first data point.

Specifically, in each cluster, for each first data point in the cluster, a plurality of data point combinations including the first data point may be first determined, distances between data points in each data point combination are calculated according to position information of two data points in each data point combination, and then a data point combination with the shortest data point distance is selected as a first target combination in each data point combination. One first data point corresponds to one first target combination, and a plurality of first data points correspond to a plurality of first target combinations, wherein the number of the first target combinations in each cluster is the same as the number of the first data points in the cluster.

From the above, when the scheme provided by the embodiment of the invention is applied to point cloud registration, for each first data point in each cluster, the data point combination with the shortest data point distance is selected from all data point combinations containing the first data point to be used as the first target combination, so that the first target combination in each cluster can be accurately determined, and the accuracy of point cloud registration is improved.

In selecting the first target combination described above, the selection of the first target combination may be achieved by steps S103B to S103C in the embodiment shown in fig. 3 below, in addition to the manner mentioned in step S103A in the embodiment shown in fig. 2.

In an embodiment of the present invention, referring to fig. 3, a schematic flow chart of a third point cloud registration method is provided, and in this embodiment, compared with the embodiment shown in fig. 1, the above step S103 may be implemented by the following steps S103B-S103C.

Step S103B: and obtaining a weight coefficient of each data point combination in each cluster aiming at each cluster in the target point cloud data.

The weight coefficient is calculated according to the distance between two data points in the data point combination, the offset and the rotation angle.

The offset is as follows: the relative distance between two data points in the data point combination on a first preset coordinate axis of a preset three-dimensional coordinate system.

The rotation angle is as follows: and an included angle between a straight line where two data points in the data point combination are located and a second preset coordinate axis of the preset three-dimensional coordinate system.

In an embodiment of the present invention, the rotation angle may be an angle between a straight line where two data points in the data point combination are located and a preset plane in the three-dimensional coordinate system.

For example, the preset plane may be a plane in which the X axis and the Y axis of the three-dimensional coordinate system are located, a plane in which the X axis and the Z axis are located, or a plane in which the Z axis and the Y axis are located.

The preset three-dimensional coordinate system may be a three-dimensional coordinate system set manually, and each data point in the target point cloud data has a respective three-dimensional coordinate in the three-dimensional coordinate system.

The first preset coordinate axis and the second preset coordinate axis may be manually specified coordinate axes, and the first preset coordinate axis and the second preset coordinate axis may be the same coordinate axis or different coordinate axes.

Specifically, in each cluster, for each data point combination in the cluster, a distance, an offset, and a rotation angle between two data points in the data point combination may be first obtained, and then a weight coefficient of the data point combination may be calculated according to the obtained distance, offset, and rotation angle.

When calculating the weight coefficient according to the distance, the offset and the rotation angle, the parameter weights of the three parameters can be set in advance according to the influence degree of the three parameters on the subsequent selection of the first target combination, and then the parameter values of the three parameters and the respective parameter weights are weighted and summed to obtain the weight coefficient of the data point combination.

For example, when the first target combination is selected, the data point combination with a relatively close distance between two data points is prioritized, so that the influence degree of the data point distance on the selection of the first target combination is relatively high, and a relatively large parameter weight can be set for the data point distance. The offset has a lower impact on the selection of the first target combination, and a smaller parameter weight may be set for the offset.

Step S103C: in each cluster, for each first data point belonging to one initial point cloud data in the cluster, a data point combination with the largest weight coefficient is selected as a first target combination from the data point combinations containing the first data point.

Specifically, in each cluster, for each first data point in the cluster, a plurality of data point combinations including the first data point may be first determined, and then, a data point combination having the largest weight coefficient is selected from the determined respective data point combinations as the first target combination. One first data point corresponds to one first target combination, and a plurality of first data points correspond to a plurality of first target combinations, wherein the number of the first target combinations in each cluster is the same as the number of the first data points in the cluster.

From the above, when the scheme provided by the embodiment of the invention is applied to point cloud registration, the weight coefficient of each data point combination is obtained first, and because the weight coefficient is calculated according to the distance, the offset and the rotation angle between two data points in the data point combination, the weight coefficient of the data point combination integrates the consideration of multiple aspects of information, so that the accuracy of selecting the first target combination can be improved when the first target combination is selected according to the weight coefficient of the data point combination, thereby improving the accuracy of point cloud registration.

The initial point cloud data may be point cloud data obtained by converting a three-dimensional image of the object. In view of this, in one embodiment of the present invention, referring to fig. 4, a flow chart of a fourth point cloud registration method is provided, and in this embodiment, the above step S101 may be implemented by the following steps S101A-S101C, compared with the embodiment shown in fig. 1.

Step S101A: two three-dimensional images of the same object are obtained.

The two three-dimensional images may be images of the object acquired under the same acquisition condition, or images of the object acquired under different acquisition conditions.

The images acquired under the same acquisition condition may be images acquired by the same image acquisition device under the same acquisition angle on objects in the same acquisition environment.

The images acquired under the different acquisition conditions may be images acquired under the condition that one or more factors of the image acquisition device, the acquisition angle, the acquisition environment and the like are different.

Step S101B: for each three-dimensional image, constructing a three-dimensional model of the object according to the three-dimensional image, and generating point cloud data of the object according to the three-dimensional model.

Specifically, the three-dimensional model of the object may be reconstructed from the three-dimensional image by an existing three-dimensional reconstruction technique, and then converted into point cloud data by an existing model conversion technique. The point cloud data generated from the three-dimensional image may be regarded as the above-described initial point cloud data.

For example, the model conversion technique described above may be VTK (Visualization Tool Kit, visualization tool package).

Step S101C: and merging the two groups of generated point cloud data to obtain merged point cloud data serving as target point cloud data.

Similar to the merging of the two initial point cloud data in step S101, it may be understood that all data points in the two sets of generated point cloud data are overlapped, and the overlapped point cloud data is the target point cloud data.

In addition, other data points except for the extreme data points in the two sets of generated point cloud data can be overlapped.

In the above, when the scheme provided by the embodiment of the invention is applied to point cloud registration, the three-dimensional image is converted into point cloud data, and the generated point cloud data are combined, so that the target point cloud data are obtained. Therefore, the point cloud registration scheme provided by the embodiment of the invention not only can be realized based on the point cloud data of the object, but also can be realized based on the three-dimensional image of the object, so that the application range of the point cloud registration scheme is enlarged.

Because the number of data points included in one point cloud data is generally large, before the point cloud data is processed, the point cloud data can be firstly subjected to downsampling processing, so that the number of data points in the point cloud data is reduced, and the calculation amount of the point cloud data processing is reduced.

In view of the above, referring to fig. 5, a flowchart of a fifth point cloud registration method is provided in an embodiment of the present invention, and in comparison with the embodiment shown in fig. 1, in this embodiment, the above step S101 may be implemented by the following steps S101D-S101F.

Step S101D: two sets of initial point cloud data of the same object are obtained.

Step S101E: and respectively carrying out downsampling processing on the two groups of initial point cloud data.

The downsampling process is used to reduce the number of data points in the initial point cloud data.

Specifically, one downsampling process may be performed on one set of initial point cloud data, and two downsampling processes may be performed on two sets of initial point cloud data.

The downsampling process for each set of initial point cloud data may be implemented in two ways.

In a first implementation, the downsampling process may be performed on each initial point cloud data by way of meshing.

Specifically, since the data points of the initial point cloud data can be regarded as space points in the three-dimensional space coordinate system, the three-dimensional space can be divided into a plurality of mesh areas according to a preset size on three coordinate axes of the three-dimensional space coordinate system. For each grid region, all data points located in the grid region are fused. After all the data points in the initial point cloud data are processed in this way, the number of the data points in the initial point cloud data can be reduced, so that downsampling is realized.

When all the data points in the grid area are fused, the average data point of each data point can be calculated according to the position information of each data point, and the coordinates of the average data point on each coordinate axis are the average value of the coordinates of each data point on the same coordinate axis.

In a second implementation, the data points may also be ordered according to their coordinates on any coordinate axis of the three-dimensional space coordinate system, and then deleted at preset intervals according to the order of the data points.

For example, if the predetermined interval is 5, the next data point may be deleted every 5 data points according to the data point sequence.

Step S101F: and merging the two groups of initial point cloud data after the downsampling processing to obtain merged target point cloud data.

This step is similar to the step S101C described above, and will not be described again here.

From the above, when the scheme provided by the embodiment of the invention is applied to point cloud registration, downsampling processing is respectively performed on the two groups of initial point cloud data, so that the number of data points in the two groups of initial point cloud data is reduced, namely the number of data points in the target point cloud data is reduced, and thus, when the target point cloud data is subjected to subsequent processing, the calculated amount of the point cloud data processing can be reduced, the processing efficiency of the target point cloud data is improved, and the efficiency of the point cloud registration is improved.

Since the position information of the data point belonging to one initial point cloud data in the target point cloud data may reflect one overall position of the object, the position information of the data point belonging to another initial point cloud data may also reflect another overall position of the object, and the position difference between the two overall positions may be larger, for example, the position difference may be a distance, an orientation of the object reflected by the overall position, or the like, and thus, after the target point cloud data is obtained, the target point cloud data may be subjected to point cloud rough registration to reduce the difference between the two overall positions of the object reflected by the position information of the data point in the target point cloud data.

In view of the above, referring to fig. 6, a flowchart of a sixth point cloud registration method is provided in an embodiment of the present invention, and in this embodiment, the method further includes the following steps S108 to S109 before the step S102, compared with the embodiment shown in fig. 1.

Step S108: and calculating initial adjustment parameters according to the position information of each data point in the target point cloud data.

The initial adjustment parameters may include rotation parameters and translation parameters.

In one implementation, the initial adjustment parameters may be calculated according to the location information of each data point in the target point cloud data by using an existing point cloud coarse registration technique.

For example, the above-described point cloud coarse registration techniques may be 4PCS (4-Point Congruent Sets), super 4PCS (Super 4-Point Congruent Sets), SK-4PCS (Semantic Keypoint 4-Point Congruent Sets), and the like.

Step S109: and adjusting the position information of the data points belonging to the cloud data to be fixed in the target point cloud data based on the initial adjustment parameters to obtain adjusted target point cloud data.

The cloud data to be fixed are as follows: any one of the initial point cloud data.

This step is similar to step S105 described above, and will not be described again.

As can be seen from the above, when the solution provided by the embodiment of the present invention is applied to point cloud registration, because in the obtained target point cloud data, the position information of each data point may reflect two overall positions of the same object, and the position difference between the two overall positions may be larger. The initial adjustment parameters are calculated according to the position information of each data point in the target point cloud data, so that the position difference between the two integral positions is considered in calculating the initial adjustment parameters, and therefore, when the position information of the data point belonging to the target point cloud data is adjusted based on the initial adjustment parameters, the position difference between the two integral positions can be reduced, the two integral positions reflected by the position information of the data point in the target point cloud data after the position information is adjusted are matched, the position information of the data point in the target point cloud data is adjusted again on the basis that the two integral positions are matched, the frequency of adjusting the position information can be reduced, and the efficiency of point cloud registration is improved.

Corresponding to the point cloud registration method, the embodiment of the invention also provides a point cloud registration device.

In one embodiment of the present invention, referring to fig. 7, there is provided a schematic structural diagram of a first point cloud registration apparatus, the apparatus including:

a data obtaining module 701, configured to obtain target point cloud data of data points in two sets of initial point cloud data including the same object;

the data point dividing module 702 is configured to divide each data point in the target point cloud data into different clusters based on the position information of each data point in the target point cloud data and a preset first distance threshold;

a combination selection module 703, configured to, in each cluster, select, for each first data point belonging to one initial point cloud data in the cluster, a first target combination for performing parameter calculation from respective data point combinations including the first data point, where one data point combination includes: a first data point and a second data point in the cluster belonging to another initial point cloud data;

a first calculation module 704, configured to calculate an adjustment parameter according to a distance between a first data point and a second data point in each first target combination;

The first adjustment module 705 is configured to adjust, based on the adjustment parameter, position information of a data point belonging to point cloud data to be adjusted in the target point cloud data, where the point cloud data to be adjusted is: any one of the two sets of initial point cloud data;

the condition judging module 706 is configured to judge whether the target point cloud data after the position information of the data point is adjusted meets a preset registration condition, if yes, trigger the result determining module 707, and if no, trigger the data point dividing module 702;

the result determining module 707 is configured to take the target point cloud data after the position information of the data point is adjusted as a point cloud registration result.

In one embodiment of the present invention, the combination selection module 703 is specifically configured to:

In one embodiment of the present invention, the data obtaining module 701 is specifically configured to:

obtaining two three-dimensional images of the same object;

obtaining two groups of initial point cloud data of the same object;

In an embodiment of the present invention, referring to fig. 8, a schematic structural diagram of a second point cloud registration apparatus is provided, where in this embodiment, the apparatus further includes:

a second calculation module 708, configured to calculate an initial adjustment parameter according to the position information of each data point in the target point cloud data before dividing each data point in the target point cloud data into different clusters by performing a point cloud segmentation process on the target point cloud data based on the position information of each data point in the target point cloud data and a preset first distance threshold;

the second adjustment module 709 is configured to adjust, based on the initial adjustment parameter, position information of a data point belonging to cloud data to be fixed in the target point cloud data, to obtain adjusted target point cloud data, where the cloud data to be fixed is: any one of the initial point cloud data.

In one embodiment of the present invention, the registration condition is: the distance between two data points contained in each second target combination is smaller than a preset second distance threshold, wherein the second target combination is as follows: and a first target combination redetermined in the target point cloud data after the data points are adjusted.

As can be seen from the above, when the point cloud registration is performed by applying the scheme provided by the embodiment of the present invention, when the target point cloud data satisfies the registration condition, the distance between two data points in each first target combination in the target point cloud data is smaller than the second distance threshold, at this time, the two data points may be considered to be coincident, and when the distances between the data points in all the first target combinations are smaller than the second distance threshold, most data points in the target point cloud data respectively belonging to the two initial point cloud data may be considered to be coincident, and the overall position of the object may be reflected by the position information of most data points in the initial point cloud data, so that in the target point cloud data satisfying the registration condition, the two overall positions of the same object reflected by the position information of each data point may be considered to be coincident. Therefore, the accuracy of point cloud registration is higher by applying the registration condition in the scheme.

In the above-mentioned scheme, since there may be second data points in the cluster where the first data point is located, where the distance between the second data point and the first data point is greater than the third distance threshold, if the distance between the first data point and the second data point included in the data point combination is not limited to be less than the third distance threshold, it is necessary to first determine the same number of data point combinations as the second data point in the cluster for each first data point in each cluster, and if the distance between the first data point and the second data point is limited to be less than the third distance threshold, the determined number of data point combinations including the first data point is less than the number of second data points, so that the selection of the first target combination in fewer data point combinations can improve the selection efficiency of the first target combination.

The embodiment of the present invention also provides an electronic device, as shown in fig. 9, including a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 perform communication with each other through the communication bus 904,

A memory 903 for storing a computer program;

the processor 901 is configured to execute a program stored in the memory 903, and implement the following steps:

In addition, the electronic device may implement other point cloud registration methods as described in the previous method embodiment section, which will not be described in detail herein.

The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer readable storage medium is also provided, in which a computer program is stored, which when executed by a processor, implements the steps of any of the point cloud registration methods described above.

In yet another embodiment of the present invention, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the point cloud registration methods of the above embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, electronic device, computer readable storage medium, and computer program product embodiments, the description is relatively simple, as relevant to the method embodiments being referred to in the section of the description of the method embodiments.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method of point cloud registration, the method comprising:

if the target point cloud data after the position information of the data points is adjusted meets the registration condition, the target point cloud data after the position information of the data points is adjusted is used as a point cloud registration result;

in each cluster, for each first data point belonging to one initial point cloud data in the cluster, selecting a first target combination for parameter calculation from various data point combinations containing the first data point, wherein the first target combination comprises:

2. The method of claim 1, wherein obtaining target point cloud data for each data point in two sets of initial point cloud data comprising the same object comprises:

obtaining two three-dimensional images of the same object;

3. The method of claim 1, wherein obtaining target point cloud data for each data point in two sets of initial point cloud data comprising the same object comprises:

obtaining two groups of initial point cloud data of the same object;

4. The method according to claim 1, wherein before dividing each data point in the target point cloud data into different clusters by performing a point cloud segmentation process on the target point cloud data based on the position information of each data point in the target point cloud data and a preset first distance threshold, the method further comprises:

5. The method of claim 1, wherein the registration condition is:

6. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The distance between the first data point and the second data point contained in the data point combination is smaller than a preset third distance threshold value.

7. A point cloud registration apparatus, the apparatus comprising:

the result determining module is used for taking the target point cloud data after the position information of the data points is adjusted as a point cloud registration result;

the combination selection module is specifically configured to:

8. The apparatus of claim 7, wherein the data acquisition module is specifically configured to:

obtaining two three-dimensional images of the same object;

9. The apparatus of claim 7, wherein the data acquisition module is specifically configured to:

obtaining two groups of initial point cloud data of the same object;

10. The apparatus of claim 7, wherein the apparatus further comprises:

11. The apparatus of claim 7, wherein the registration condition is:

12. The apparatus of claim 7, wherein a distance between a first data point and a second data point included in the combination of data points is less than a preset third distance threshold.

13. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the point cloud registration method of any of claims 1-6 when executing a program stored on a memory.

14. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the point cloud registration method steps of any of claims 1-6.