CN111986239A

CN111986239A - Point cloud registration method and device, computer readable storage medium and electronic equipment

Info

Publication number: CN111986239A
Application number: CN202010883718.5A
Authority: CN
Inventors: 胡洋
Original assignee: Beike Technology Co Ltd
Current assignee: As you can see (Beijing) Technology Co.,Ltd.
Priority date: 2020-08-28
Filing date: 2020-08-28
Publication date: 2020-11-24
Anticipated expiration: 2040-08-28
Also published as: CN111986239B

Abstract

The embodiment of the disclosure discloses a point cloud registration method and a point cloud registration device, wherein the method comprises the following steps: acquiring a first station point cloud and a second station point cloud, wherein the first station point cloud and the second station point cloud are acquired by a point cloud acquisition device at different positions; determining a first target point cloud located in a first target area from the first station point cloud, and determining a second target point cloud located in a second target area from the second station point cloud; respectively determining corresponding point sets from the first target point cloud and the second target point cloud, and determining a coordinate transformation matrix according to the point sets; carrying out registration optimization on the coordinate transformation matrix; and splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix. The embodiment of the disclosure realizes the determination of the coordinate transformation matrix according to part of the point clouds, improves the efficiency of point cloud splicing, and reduces the error rate of point cloud splicing.

Description

Point cloud registration method and device, computer readable storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a point cloud registration method and apparatus, a computer-readable storage medium, and an electronic device.

Background

Three-dimensional reconstruction through scanning of point clouds is the main way of three-dimensional reconstruction of indoor scenes. The method adopts substation type scanning and splicing, which is one of the main technical schemes for acquiring point cloud data. Whether the point clouds can be automatically and rapidly spliced station by station in the acquisition process is a key technology.

The common Point cloud splicing technology mainly comprises two parts, namely rough splicing and optimization, wherein the optimization part commonly comprises an ICP (Iterative Closest Point) algorithm, an NDT (Normal distribution Transform) algorithm and the like, and is mature and effective. The rough splicing part mainly comprises algorithms such as feature-based matching, four-point method-based matching and deep learning-based matching.

Disclosure of Invention

The embodiment of the disclosure provides a point cloud registration method and device, a computer-readable storage medium and electronic equipment.

An embodiment of the present disclosure provides a point cloud registration method, including: acquiring a first station point cloud and a second station point cloud, wherein the first station point cloud and the second station point cloud are acquired by a point cloud acquisition device at different positions; determining a first target point cloud located in a first target area from the first station point cloud, and determining a second target point cloud located in a second target area from the second station point cloud; respectively determining corresponding point sets from the first target point cloud and the second target point cloud, and determining a coordinate transformation matrix according to the point sets; carrying out registration optimization on the coordinate transformation matrix; and splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

In some embodiments, determining a first target point cloud located at a first target area from the first station point cloud and a second target point cloud located at a second target area from the second station point cloud comprises: layering the first station point cloud and the second station point cloud; determining a target layer from layers corresponding to the first station point cloud and the second station point cloud respectively; determining a target layer corresponding to the first station cloud as a first target area, and determining point cloud in the first target area as first target point cloud; and determining a target layer corresponding to the second station cloud as a second target area, and determining the point cloud in the second target area as a second target point cloud.

In some embodiments, determining a corresponding set of points from the first and second target point clouds, respectively, and determining a coordinate transformation matrix from the set of points comprises: executing the following matching steps for a first preset number of times to obtain a first preset number of coordinate transformation matrixes: determining a first point set forming a preset shape from the first target point cloud; determining a second point set forming a shape identical to the preset shape from the second target point cloud; based on the first set of points and the second set of points, a coordinate transformation matrix is determined.

In some embodiments, the predetermined shape is a triangle, the first target point cloud and the second target point cloud are located in a horizontal middle layer; and determining a second point set forming a shape congruent to the preset shape from the second target point cloud, including: determining a first point and a second point from the second target point cloud, wherein the distance between the first point and the second point on the horizontal plane is equal to the distance between the corresponding points in the first target point cloud on the horizontal plane; establishing a third point based on the first point and the second point, so that a triangle formed by the first point, the second point and the third point is congruent with a corresponding triangle in the first target point cloud; and determining a fourth point from the second target point cloud, so that the distance between the fourth point and the third point is smaller than or equal to a first preset distance, and determining the first point, the second point and the fourth point as a second point set.

In some embodiments, the registration optimization of the coordinate transformation matrix comprises: for each coordinate transformation matrix in a first preset number of coordinate transformation matrices, performing coordinate transformation on the first target point cloud based on the coordinate transformation matrices to obtain a first transformed point cloud; determining an error between the first transformed point cloud and the second target point cloud; extracting a second preset number of coordinate transformation matrixes from the determined first preset number of coordinate transformation matrixes according to the sequence of the errors from small to large; and performing registration optimization on each coordinate transformation matrix in the second preset number of coordinate transformation matrices to obtain a second preset number of optimization matrices.

In some embodiments, determining an error between the first transformed point cloud and the second target point cloud comprises: for each point in the point cloud after the first transformation, determining a point, the distance between which and the point is less than or equal to a second preset distance, and the included angle between which and the normal vector of the point is less than or equal to a preset angle, from the second target point cloud as a matching point; based on the number of the obtained matching point pairs, an error between the first transformed point cloud and the second target point cloud is determined.

In some embodiments, stitching the first and second station point clouds based on the optimized coordinate transformation matrix comprises: extracting the corresponding optimization matrix with the minimum error from the second preset number of optimization matrices; based on the extracted optimization matrix, the following verification steps are performed: performing coordinate transformation on the first station cloud by using the extracted optimization matrix to obtain a second converted point cloud; performing occlusion verification based on the second converted point cloud and the second station point cloud; determining whether occlusion verification passes; if the point cloud passes the judgment, determining that the currently obtained second converted point cloud and the second station point cloud are spliced point clouds; if not, re-extracting the optimization matrix with the minimum error from the unextracted optimization matrices in the second preset number of optimization matrices, and continuing to execute the verification step based on the currently extracted optimization matrix.

In some embodiments, performing occlusion verification based on the second transformed point cloud and the second station point cloud comprises: determining the point cloud after the second conversion as the point cloud A, determining the point cloud of the second station as the point cloud B, and executing the following verification steps: for each point in the A point cloud, determining a point which is in a shielding relation with the point and a point which is in a same position relation with the point from the B point cloud, wherein the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is larger than a first distance, the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is smaller than or equal to a third preset distance, the first distance is the distance between the point and the viewpoint of the B point cloud, the distance between the point which is in the same position relation with the viewpoint of the B point cloud is approximately equal to the first distance, and the distance between the point which is in the first distance and the straight line is smaller than or equal to the; if the number of the points with the same position relation is larger than or equal to a first number threshold value and the number of the points with the shielding relation is smaller than or equal to a second number threshold value, determining that a verification condition is met; determining the second station point cloud as the point cloud A, determining the second converted point cloud as the point cloud B, and executing the verification step again; and if the verification conditions are met after the verification steps are executed twice, determining that the occlusion verification is passed.

In some embodiments, after acquiring the first site cloud and the second site cloud, the method further comprises: determining a point with a normal vector approximate to vertical from the first station point cloud and the second station point cloud; performing rotation transformation on the first station point cloud and the second station point cloud to enable the included angle between the point with the normal vector approximate to the vertical direction and the vertical direction to be minimum; determining a point with the minimum vertical coordinate from the first station point cloud and the second station point cloud; and carrying out translation transformation on the first station point cloud and the second station point cloud so that the vertical coordinate of the point with the minimum coordinate in the vertical direction is zero.

In some embodiments, prior to acquiring the first site cloud and the second site cloud, the method further comprises: acquiring an initial first station point cloud and an initial second station point cloud; and respectively carrying out down-sampling on the initial first station point cloud and the initial second station point cloud to obtain the first station point cloud and the second station point cloud.

According to another aspect of the embodiments of the present disclosure, there is provided a point cloud registration apparatus including: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a first station point cloud and a second station point cloud, and the first station point cloud and the second station point cloud are acquired by a point cloud acquisition device at different positions; the first determining module is used for determining a first target point cloud located in a first target area from the first station point cloud and determining a second target point cloud located in a second target area from the second station point cloud; the second determining module is used for respectively determining corresponding point sets from the first target point cloud and the second target point cloud and determining a coordinate transformation matrix according to the point sets; the optimization module is used for carrying out registration optimization on the coordinate transformation matrix; and the splicing module is used for splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

In some embodiments, the first determining module comprises: the layering unit is used for layering the first station point cloud and the second station point cloud; the first determining unit is used for determining a target layer from layers corresponding to the first station point cloud and the second station point cloud respectively; the second determining unit is used for determining a target layer corresponding to the first station cloud as a first target area and determining point cloud in the first target area as first target point cloud; and the third determining unit is used for determining that the target layer corresponding to the second station cloud is a second target area and determining that the point cloud in the second target area is a second target point cloud.

In some embodiments, the second determination module is further to: executing the following matching steps for a first preset number of times to obtain a first preset number of coordinate transformation matrixes: determining a first point set forming a preset shape from the first target point cloud; determining a second point set forming a shape identical to the preset shape from the second target point cloud; based on the first set of points and the second set of points, a coordinate transformation matrix is determined.

In some embodiments, the predetermined shape is a triangle, the first target point cloud and the second target point cloud are located in a horizontal middle layer; and a second determining module determines a second point set forming a shape congruent to the preset shape from the second target point cloud according to the following steps: determining a first point and a second point from the second target point cloud, wherein the distance between the first point and the second point on the horizontal plane is equal to the distance between the corresponding points in the first target point cloud on the horizontal plane; establishing a third point based on the first point and the second point, so that a triangle formed by the first point, the second point and the third point is congruent with a corresponding triangle in the first target point cloud; and determining a fourth point from the second target point cloud, so that the distance between the fourth point and the third point is smaller than or equal to a first preset distance, and determining the first point, the second point and the fourth point as a second point set.

In some embodiments, the optimization module comprises: the determining unit is used for carrying out coordinate transformation on the first target point cloud on the basis of the coordinate transformation matrix for each coordinate transformation matrix in a first preset number of coordinate transformation matrices to obtain a first transformed point cloud; determining an error between the first transformed point cloud and the second target point cloud; the first extraction unit is used for extracting a second preset number of coordinate transformation matrixes from the determined first preset number of coordinate transformation matrixes according to the sequence of the errors from small to large; and the optimization unit is used for carrying out registration optimization on each coordinate transformation matrix in the second preset number of coordinate transformation matrices to obtain a second preset number of optimization matrices.

In some embodiments, the determining unit comprises: the first determining subunit is used for determining, from the second target point cloud, for each point in the first transformed point cloud, a point of which the distance from the point is less than or equal to a second preset distance and the included angle between the point and a normal vector of the point is less than or equal to a preset angle as a matching point; and the second determining subunit is used for determining the error between the first transformed point cloud and the second target point cloud based on the obtained number of the matching point pairs.

In some embodiments, the splicing module comprises: the second extraction unit is used for extracting the corresponding optimization matrix with the minimum error from the second preset number of optimization matrices; a verification unit for performing the following verification steps based on the extracted optimization matrix: performing coordinate transformation on the first station cloud by using the extracted optimization matrix to obtain a second converted point cloud; performing occlusion verification based on the second converted point cloud and the second station point cloud; determining whether occlusion verification passes; if the point cloud passes the judgment, determining that the currently obtained second converted point cloud and the second station point cloud are spliced point clouds; and if the optimization matrixes do not pass the verification unit, re-extracting the optimization matrix with the minimum error from the unextracted optimization matrixes in the second preset number of optimization matrixes, and continuously executing the verification step on the basis of the currently extracted optimization matrix.

In some embodiments, the verification unit comprises: a verification subunit, configured to determine the second converted point cloud as an a point cloud, determine the second station point cloud as a B point cloud, and perform the following occlusion verification steps: for each point in the A point cloud, determining a point which is in a shielding relation with the point and a point which is in a same position relation with the point from the B point cloud, wherein the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is larger than a first distance, the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is smaller than or equal to a third preset distance, the first distance is the distance between the point and the viewpoint of the B point cloud, the distance between the point which is in the same position relation with the viewpoint of the B point cloud is approximately equal to the first distance, and the distance between the point which is in the first distance and the straight line is smaller than or equal to the; a third determining subunit, configured to determine that the verification condition is satisfied if the number of the points in the same-position relationship is greater than or equal to the first number threshold and the number of the points in the occlusion relationship is less than or equal to the second number threshold; a fourth determining subunit, configured to determine the second station point cloud as an a point cloud, determine the second converted point cloud as a B point cloud, and perform the occlusion verification step again; and the fifth determining subunit is used for determining that the occlusion verification passes if the verification conditions are met after the verification steps are executed twice.

In some embodiments, the apparatus further comprises: the third determining module is used for determining a point with a normal vector approximate to vertical from the first station point cloud and the second station point cloud; the first transformation module is used for performing rotation transformation on the first station point cloud and the second station point cloud so that an included angle between a point with a normal vector approximate to vertical and the vertical direction is minimum; the fourth determining module is used for determining a point with the minimum vertical coordinate from the first station point cloud and the second station point cloud; and the second transformation module is used for carrying out translation transformation on the first station point cloud and the second station point cloud so that the vertical direction coordinate of the point with the minimum coordinate in the vertical direction is zero.

In some embodiments, the apparatus further comprises: the second acquisition module is used for acquiring the initial first station point cloud and the initial second station point cloud; and the down-sampling module is used for respectively performing down-sampling on the initial first station point cloud and the initial second station point cloud to obtain the first station point cloud and the second station point cloud.

According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the point cloud registration method described above.

According to another aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; and the processor is used for reading the executable instructions from the memory and executing the instructions to realize the point cloud registration method.

Based on the point cloud registration method, the point cloud registration device, the computer-readable storage medium and the electronic device provided by the embodiments of the disclosure, the target point clouds are extracted from the specific areas of the two station point clouds, the point sets corresponding to each other are determined from the two sets of the target point clouds, the coordinate transformation matrix is determined according to the point sets, the coordinate transformation matrix is subjected to registration optimization, and finally the first station point cloud and the second station point cloud are spliced based on the optimized coordinate transformation matrix, so that the coordinate transformation matrix is determined according to part of the point clouds, the efficiency of point cloud splicing is improved, and the error rate of point cloud splicing is reduced.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.

Fig. 1 is a system diagram to which the present disclosure is applicable.

Fig. 2 is a schematic flow chart of a point cloud registration method according to an exemplary embodiment of the present disclosure.

Fig. 3 is a schematic flow chart of a point cloud registration method according to another exemplary embodiment of the present disclosure.

Fig. 4 is a schematic flow chart of a point cloud registration method according to another exemplary embodiment of the present disclosure.

Fig. 5 is a schematic flow chart of a point cloud registration method according to another exemplary embodiment of the present disclosure.

Fig. 6 is a schematic flow chart of a point cloud registration method according to another exemplary embodiment of the present disclosure.

Fig. 7 is a schematic structural diagram of a point cloud registration apparatus according to an exemplary embodiment of the present disclosure.

Fig. 8 is a schematic structural diagram of a point cloud registration apparatus according to another exemplary embodiment of the present disclosure.

Fig. 9 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.

It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Summary of the application

Because a plurality of planes exist in an indoor scene, the planes are very similar, and errors often occur when the mode of extracting the point cloud characteristic points is adopted for splicing. The four-point method and the related algorithm can reduce the occurrence of errors by increasing the iteration number, but still have the problems of low operation speed and high error rate.

Exemplary System

Fig. 1 illustrates an exemplary system architecture 100 to which a point cloud registration method or apparatus of an embodiment of the present disclosure may be applied.

As shown in fig. 1, system architecture 100 may include terminal device 101, network 102, and server 103. Network 102 is the medium used to provide communication links between terminal devices 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

A user may use terminal device 101 to interact with server 103 over network 102 to receive or send messages and the like. Various communication client applications, such as a photographing application, a three-dimensional model application, an image processing application, and the like, may be installed on the terminal device 101.

The terminal device 101 may be various electronic devices including, but not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle-mounted terminal (e.g., a car navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc.

The server 103 may be a server that provides various services, such as a background point cloud registration server that registers point clouds uploaded by the terminal device 101. The background point cloud registration server can register and splice the received first station point cloud and the second station point cloud.

It should be noted that the point cloud registration method provided by the embodiment of the present disclosure may be executed by the server 103, or may be executed by the terminal device 101, and accordingly, the point cloud registration apparatus may be disposed in the server 103, or may be disposed in the terminal device 101.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. In the case that the stitched point cloud does not need to be acquired from a remote place, the system architecture may not include a network, and only include a server or a terminal device.

Exemplary method

Fig. 2 is a schematic flow chart of a point cloud registration method according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device (such as the terminal device 101 or the server 103 shown in fig. 1), and as shown in fig. 2, the method includes the following steps:

step 201, a first station point cloud and a second station point cloud are obtained.

In this embodiment, the electronic device may obtain the first and second station point clouds either locally or remotely. The first station point cloud and the second station point cloud are acquired by the point cloud acquisition device at different positions. The point cloud collecting device can be a laser radar, a depth camera and the like. Generally, the first station point cloud and the second station point cloud are obtained by scanning a space by a point cloud acquisition device at different positions in the same space (which may be an indoor space or an outdoor space). The first station point cloud and the second station point cloud can be directly acquired by the point cloud acquisition device, and can also be data obtained by preprocessing directly acquired data.

Step 202, a first target point cloud located in a first target area is determined from the first station point cloud, and a second target point cloud located in a second target area is determined from the second station point cloud.

In this embodiment, the electronic device may determine a first target point cloud located at a first target area from the first station point cloud and a second target point cloud located at a second target area from the second station point cloud. The first target area and the second target area correspond to each other, that is, the actual spaces indicated by the first target area and the second target area are the same or similar (that is, the coincidence degree is greater). As an example, the first target area and the second target area may be manually determined areas, or may be layers (for example, the uppermost layer, the lowermost layer, or the target layer) at the same position after layering the first site cloud and the second site cloud.

In some alternative implementations, step 202 may be performed as follows:

first, the first site cloud and the second site cloud are layered.

The layering mode may be horizontal layering (i.e. the layers are arranged up and down), or vertical layering (i.e. the layers are arranged left and right or front and back).

And then, determining a target layer from the layers corresponding to the first station point cloud and the second station point cloud respectively.

Wherein the target layer may be any layer specified. Typically, to select more, more representative point clouds, the target layer may be an intermediate layer. The intermediate layer may be one of the layers located in the middle, or may be a plurality of adjacent layers located in the middle.

And finally, determining that a target layer corresponding to the first site cloud is a first target area, determining that point cloud in the first target area is the first target point cloud, determining that a target layer corresponding to the second site cloud is a second target area, and determining that point cloud in the second target area is the second target point cloud.

According to the implementation mode, point clouds on certain planes can be extracted from the first station point cloud and the second station point cloud through layering to serve as target point clouds, so that the position relation of the point clouds on the planes can be determined by utilizing the target point clouds, and the accuracy and the efficiency of point cloud registration are improved.

Step 203, respectively determining corresponding point sets from the first target point cloud and the second target point cloud, and determining a coordinate transformation matrix according to the point sets.

In this embodiment, the electronic device may determine corresponding point sets from the first target point cloud and the second target point cloud, respectively, and determine the coordinate transformation matrix according to the point sets. The two point sets are determined to be corresponding to each other, that is, the two point sets are respectively formed to have the same or similar shape, and the two points corresponding to each other are located at or close to the same spatial position. By using the coordinates of each point in the two point sets, a rotational translation matrix representing the position relationship between the first station point cloud and the second station point cloud can be calculated and used as a coordinate transformation matrix, and the coordinate transformation matrix can convert the two station point clouds in different coordinate systems into the same coordinate system.

And 204, performing registration optimization on the coordinate transformation matrix.

In this embodiment, the electronic device may perform registration optimization on the coordinate transformation matrix. Specifically, the coordinate transformation matrix may be subjected to registration optimization using an existing registration optimization algorithm (e.g., ICP algorithm, NDT algorithm, etc.). The optimized coordinate transformation matrix can accurately represent the position relation of the two-station point cloud.

And step 205, splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

In this embodiment, the electronic device may stitch the first and second station point clouds based on the optimized coordinate transformation matrix. Specifically, the coordinates of each point in the first station point cloud can be converted into the coordinate system of the second station point cloud by using the optimized coordinate transformation matrix splicing, so that the two station point clouds are spliced.

In some optional implementations, after step 201 and before step 202, the following steps may be further included:

firstly, determining a point with a normal vector approximate to vertical from the first station point cloud and the second station point cloud. Wherein the points where the normal vector is approximately vertical are generally located on a plane approximately parallel to the horizontal plane. Specifically, a point at which an angle between the normal vector and the vertical direction is less than or equal to a preset angle may be determined as a point at which the normal vector is approximately vertical.

And step two, performing rotation transformation on the first station point cloud and the second station point cloud to enable the included angle between the point with the normal vector approximate to the vertical direction and the vertical direction to be minimum. Specifically, the first station cloud as a whole may be rotated by a certain angle, so that the normal vector of the point determined in the first step is approximately parallel to the vertical direction. For example, the normal amounts of the points with the normal vectors approximately vertical may be averaged, and the first station point cloud may be rotated to make the angle between the average normal vector and the vertical direction close to zero. This step may cause the point cloud planes of the ceiling, ground, etc. categories in the indoor scene to be approximately parallel to the horizontal plane.

And step three, determining a point with the minimum vertical coordinate from the first station point cloud and the second station point cloud. As an example, the vertical direction may be a z-axis direction in the coordinate systems of the first and second station point clouds, and the point having the smallest vertical direction coordinate is the point having the smallest z-coordinate.

And fourthly, performing translation transformation on the first station point cloud and the second station point cloud to enable the vertical direction coordinate of the point with the minimum coordinate in the vertical direction to be zero. Continuing with the example of step three, this step is performed even if the z-coordinate of the point with the smallest z-coordinate is zero.

According to the implementation mode, the first station point cloud and the second station point cloud are rotated and translated, the coordinate ranges of the first station point cloud and the second station point cloud are unified, so that the influence of height on registration can be ignored, the point cloud splicing problem is approximate to a two-dimensional rotation and translation problem, the steps of point cloud splicing are simplified, and the splicing efficiency is improved.

In some optional implementations, before step 201, the following steps may also be included:

first, an initial first station point cloud and an initial second station point cloud are obtained. The initial first station point cloud and the initial second station point cloud may be data directly acquired by the point cloud acquisition device.

Then, the initial first station point cloud and the initial second station point cloud are respectively subjected to down-sampling to obtain the first station point cloud and the second station point cloud. The down-sampling method may be any method, for example, using voxel filtering to perform down-sampling, or randomly extracting a certain proportion of point clouds.

According to the implementation mode, the data volume can be reduced through downsampling, and the operation speed is improved.

According to the method provided by the embodiment of the disclosure, the target point clouds are extracted from the specific areas in the two station point clouds, the point sets corresponding to each other are determined from the two sets of target point clouds, the coordinate transformation matrix is determined according to the point sets, the coordinate transformation matrix is subjected to registration optimization, and finally the first station point cloud and the second station point cloud are spliced based on the optimized coordinate transformation matrix, so that the coordinate transformation matrix is determined according to part of the point clouds, the efficiency of point cloud splicing is improved, and the error rate of point cloud splicing is reduced.

With further reference to fig. 3, a schematic flow chart of yet another embodiment of a point cloud registration method is shown. As shown in fig. 3, based on the embodiment shown in fig. 2, step 203 may be performed as follows:

executing the following matching steps (including step 2031 to step 2033) for a first preset number of times to obtain a first preset number of coordinate transformation matrices:

step 2031, a first point set forming a preset shape is determined from the first target point cloud.

In this embodiment, the electronic device may determine a first set of points that make up a preset shape from a first target point cloud. The preset shape may be any shape, such as a triangle, a quadrangle, and the like. The determination method of the first point set may be random extraction, the distance between every two points in the first point set is greater than a preset distance threshold, and the included angle between every two normal vectors of each point is smaller than a preset angle threshold, that is, the first point set is approximately on the same plane.

Step 2032, a second set of points forming a shape congruent to the preset shape is determined from the second target point cloud.

In this embodiment, the electronic device may determine a second set of points that make up a shape congruent to the preset shape from the second target point cloud. As an example, the electronic device may sequentially retrieve corresponding points from the second target point cloud according to the lengths of the edges of the preset shape, so as to obtain a second point set. It should be noted that the total equivalence here may be absolute equivalence, or may be relative equivalence, that is, the length and the included angle of each side are within the preset length and included angle range.

Step 2033, determining a coordinate transformation matrix based on the first set of points and the second set of points.

In this embodiment, the electronic device may determine a coordinate transformation matrix based on the first set of points and the second set of points. In particular, the electronic device may determine a coordinate transformation matrix based on vectors indicated by corresponding edges of two congruent shapes characterized by the first set of points and the second set of points. It should be noted that the method for determining the coordinate transformation matrix is a technology currently known in the art and will not be described herein.

By executing steps 2031 to 2033 for a plurality of times, a first preset number of coordinate transformation matrices can be obtained, and in subsequent step 204, each coordinate transformation matrix can be optimized, and an optimal coordinate transformation matrix is determined from the plurality of optimized coordinate transformation matrices for point cloud stitching. Or one coordinate transformation matrix (for example, the matrix with the minimum error after coordinate transformation) may be selected from the first preset number of coordinate transformation matrices for optimization, and then the optimized coordinate transformation matrix is used for point cloud stitching.

The method provided by the embodiment corresponding to fig. 3 determines the congruent shape from the first target point cloud and the second target point cloud, so as to obtain two point sets matched with each other, and thus, the coordinate transformation matrix can be determined quickly and accurately.

With further reference to fig. 4, a schematic flow chart diagram of yet another embodiment of a point cloud registration method is shown. In this embodiment, the preset shape is a triangle, and the first target point cloud and the second target point cloud are located in a horizontal middle layer (i.e. middle layers in each layer arranged from top to bottom), based on this, step 2032 may include the following steps based on the embodiment shown in fig. 3:

step 20321, determine a first point and a second point from the second target point cloud.

Wherein the distance of the first point and the second point on the horizontal plane is equal to the distance of the corresponding point in the first target point cloud on the horizontal plane. Wherein the corresponding points in the first target point cloud are the respective vertices in the two congruent triangles. For example, the first point set includes three points a, b, and c, the second point set includes three points e, f, and g, if the edge ab is the edge ef, and ═ abc is the edge efg, then a and e are corresponding points to each other, b and c are corresponding points to each other, and c and g are corresponding points to each other. Because the first target point cloud and the second target point cloud are positioned in the horizontal middle layer and the vertical distance of the points included in the first target point cloud and the second target point cloud is smaller, each point can be approximately regarded as the same horizontal plane, and therefore the difficulty in determining the congruent triangle can be reduced.

Specifically, the electronic device may traverse two points in the second target point cloud, and if the distances of the two points on the horizontal plane are equal, mark the two points, e.g., as e, f. It should be noted that the distances between the first point and the second point on the horizontal plane may be completely equal to or approximately equal to each other (for example, the distance difference is smaller than a preset distance difference threshold value) on the horizontal plane of the corresponding points in the first target point cloud.

Step 20322, based on the first point and the second point, a third point is established, such that a triangle formed by the first point, the second point, and the third point is congruent to a corresponding triangle in the first target point cloud.

Specifically, if the first point and the second point are marked as e and f, a point g can be established, so that the edge fg equals to the edge bc, and the edge efg equals to the edge abc, and since the edge ab equals to the edge ef, the triangle abc and the triangle efg can be determined to be congruent.

Step 20323, determine a fourth point from the second target point cloud, so that the distance between the fourth point and the third point is less than or equal to a first preset distance, and determine the first point, the second point, and the fourth point as a second point set.

Specifically, the fourth point is labeled h, and it can be determined that triangle efh is congruent with triangle abc, and the second set of points includes e, f, h.

In the method provided by the embodiment corresponding to fig. 4, the preset shape is set as a triangle, and two congruent triangles are determined from the first target point cloud and the second target point cloud, so that the first point set and the second point set which are matched with each other can be quickly and accurately obtained, and the efficiency of point cloud splicing is improved.

With further reference to fig. 5, a schematic flow chart of yet another embodiment of a point cloud registration method is shown. In the present embodiment, the predetermined shape is a triangle, and based on the embodiment shown in fig. 3 or fig. 4, step 204 may include the following steps:

2041, for each coordinate transformation matrix in a first preset number of coordinate transformation matrices, performing coordinate transformation on the first target point cloud based on the coordinate transformation matrix to obtain a first transformed point cloud; an error between the first transformed point cloud and the second target point cloud is determined.

In this embodiment, for each coordinate transformation matrix of the first preset number of coordinate transformation matrices, the electronic device may perform the steps of:

firstly, based on a coordinate transformation matrix, carrying out coordinate transformation on a first target point cloud to obtain a first transformed point cloud. Specifically, each point in the first target point cloud may be transformed into a coordinate system of the second target point cloud based on the coordinate transformation matrix.

Then, an error between the first transformed point cloud and the second target point cloud is determined. And the error is used for representing the degree of mismatching between the first transformed point cloud and the second target point cloud.

After step 2041 is performed, the error corresponding to each coordinate transformation matrix can be obtained.

In some alternative implementations, the electronic device may determine an error between the first transformed point cloud and the second target point cloud as follows:

firstly, for each point in the point cloud after the first transformation, determining a point which has a distance with the point less than or equal to a second preset distance and an included angle with a normal vector of the point less than or equal to a preset angle from the second target point cloud as a matching point.

Then, based on the number of the obtained matching point pairs, an error between the first transformed point cloud and the second target point cloud is determined. Wherein, the matching point pair is two points which are matched with each other. Specifically, as an example, assuming that the number of pairs of matching points is R, and the number of points in the first transformed point cloud is R (or the number of points in the second target point cloud is R), the error is 1-R/R.

According to the implementation mode, the matching point can be accurately determined by setting the second preset distance and the preset angle, so that the accuracy of determining the error between the point cloud after the first transformation and the second target point cloud is improved.

Step 2042, according to the sequence of the errors from small to large, extracting a second preset number of coordinate transformation matrices from the determined first preset number of coordinate transformation matrices.

Specifically, the coordinate transformation matrices of the first preset number may be sorted in the order of decreasing errors to increasing errors, and then the coordinate transformation matrices of the second preset number may be extracted from the coordinate transformation matrices of the first preset number according to the order.

Step 2043, performing registration optimization on each coordinate transformation matrix in the second preset number of coordinate transformation matrices to obtain a second preset number of optimization matrices.

As an example, an ICP algorithm may be employed to perform registration optimization on each coordinate transformation matrix of the second preset number of coordinate transformation matrices.

The method provided by the embodiment corresponding to fig. 5 determines the error corresponding to each coordinate transformation matrix, and optimizes the coordinate transformation matrices with smaller errors, so that the accuracy of the coordinate transformation matrix obtained after optimization is higher, the coordinate transformation matrix with the highest accuracy can be obtained, and the accuracy of point cloud splicing is improved.

With further reference to fig. 6, a schematic flow chart diagram of yet another embodiment of a point cloud registration method is shown. As shown in fig. 6, based on the embodiment shown in fig. 5, step 205 may include the following steps:

and step 2051, extracting the corresponding optimization matrix with the minimum error from the second preset number of optimization matrices.

And step 2052, performing coordinate transformation on the first site cloud by using the extracted optimization matrix to obtain a second converted point cloud.

Specifically, the electronic device may perform coordinate transformation on each point in the first station-point cloud using the extracted optimization matrix to convert each point in the first station-point cloud into a coordinate system of the second station-point cloud.

And step 2053, performing occlusion verification based on the second converted point cloud and the second station point cloud.

Specifically, since the coordinate system of the second converted point cloud is the coordinate system of the second station point cloud, some points in the second converted point cloud may not have matching points in the second station point cloud, the points may visually obscure the points in the second station point cloud, the number of the obscured points may be counted, and whether occlusion verification is passed or not is determined according to the proportion of the number of obscured points to the total number of the points in the second converted point cloud (for example, the proportion is greater than a preset proportion threshold value, and it is determined not to pass, or else, it is passed).

At step 2054, it is determined whether occlusion verification passes.

If so, go to step 2055, otherwise go to step 2056

Step 2055, determining that the currently obtained second converted point cloud and the second station point cloud are spliced point clouds.

After the step is executed, the loop is exited.

Step 2056, re-extracting the optimization matrix with the minimum error from the unextracted optimization matrices in the second preset number of optimization matrices, and continuing to execute step 2052 based on the currently extracted optimization matrix.

In some alternative implementations, the above step 2053 may be performed as follows:

firstly, determining the second converted point cloud as the point cloud A, determining the second station point cloud as the point cloud B, and executing the following occlusion verification steps:

for each point in the A point cloud, determining a point which is in an occlusion relation with the point and a point which is in a same position relation with the point from the B point cloud, wherein the distance between the point which is in the occlusion relation and the view point of the B point cloud is greater than a first distance, the distance between the point which is in the occlusion relation and the view point of the B point cloud is smaller than or equal to a third preset distance, the first distance is the distance between the point and the view point of the B point cloud (namely the center of the point cloud acquisition device), the distance between the point which is in the same position relation and the view point of the B point cloud is approximately equal to the first distance (for example, the difference between the two distances is smaller than or equal to a preset distance difference threshold), and the distance between the point which is in.

And if the number of the points with the same position relation is larger than or equal to the first number threshold value and the number of the points with the shielding relation is smaller than or equal to the second number threshold value, determining that the verification condition is met.

Then, the second station point cloud is determined to be the point cloud A, the second converted point cloud is determined to be the point cloud B, and the occlusion verification step is executed again.

And if the verification conditions are met after the verification steps are executed twice, determining that the occlusion verification is passed.

According to the implementation mode, whether the current two groups of point clouds pass the shielding verification can be accurately determined by performing the verification steps twice, the accuracy of the finally determined optimization matrix is higher based on the implementation mode, and the accuracy of point cloud splicing is improved.

According to the method provided by the embodiment corresponding to fig. 6, the optimization matrices are sequentially extracted according to the sequence of the errors from small to large, whether the occlusion verification passes or not is judged based on the extracted optimization matrices, and finally the accuracy of the spliced point cloud is the highest, so that the optimal matrix can be effectively determined from the multiple optimization matrices, and the high-accuracy point cloud splicing is performed according to the optimal matrix.

Exemplary devices

Fig. 7 is a schematic structural diagram of a point cloud registration apparatus according to an exemplary embodiment of the present disclosure. The present embodiment can be applied to an electronic device, as shown in fig. 7, the point cloud registration apparatus includes: a first obtaining module 701, configured to obtain a first station point cloud and a second station point cloud, where the first station point cloud and the second station point cloud are collected by a point cloud collecting device at different positions; a first determining module 702, configured to determine a first target point cloud located in a first target area from the first station point cloud, and determine a second target point cloud located in a second target area from the second station point cloud; a second determining module 703, configured to determine corresponding point sets from the first target point cloud and the second target point cloud respectively, and determine a coordinate transformation matrix according to the point sets; an optimization module 704, configured to perform registration optimization on the coordinate transformation matrix; a stitching module 705, configured to stitch the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

In this embodiment, the first obtaining module 701 may obtain the first station point cloud and the second station point cloud locally or remotely. The first station point cloud and the second station point cloud are acquired by the point cloud acquisition device at different positions. The point cloud collecting device can be a laser radar, a depth camera and the like. Generally, the first station point cloud and the second station point cloud are obtained by scanning a space by a point cloud acquisition device at different positions in the same space (which may be an indoor space or an outdoor space). The first station point cloud and the second station point cloud can be directly acquired by the point cloud acquisition device, and can also be data obtained by preprocessing directly acquired data.

In this embodiment, the first determining module 702 may determine a first target point cloud located in a first target area from the first station point cloud and a second target point cloud located in a second target area from the second station point cloud. The first target area and the second target area correspond to each other, that is, the actual spaces indicated by the first target area and the second target area are the same or similar (that is, the coincidence degree is greater). As an example, the first target area and the second target area may be manually determined areas, or may be layers (for example, the uppermost layer, the lowermost layer, or the target layer) at the same position after layering the first site cloud and the second site cloud.

In this embodiment, the second determining module 703 may determine corresponding point sets from the first target point cloud and the second target point cloud, respectively, and determine the coordinate transformation matrix according to the point sets. The two point sets are determined to be corresponding to each other, that is, the two point sets are respectively formed to have the same or similar shape, and the two points corresponding to each other are located at or close to the same spatial position. By using the coordinates of each point in the two point sets, a rotational translation matrix representing the position relationship between the first station point cloud and the second station point cloud can be calculated and used as a coordinate transformation matrix, and the coordinate transformation matrix can convert the two station point clouds in different coordinate systems into the same coordinate system.

In this embodiment, the optimization module 704 may perform registration optimization on the coordinate transformation matrix. Specifically, the coordinate transformation matrix may be subjected to registration optimization using an existing registration optimization algorithm (e.g., ICP algorithm, NDT algorithm, etc.). The optimized coordinate transformation matrix can accurately represent the position relation of the two-station point cloud.

In this embodiment, the stitching module 705 may stitch the first and second station point clouds based on the optimized coordinate transformation matrix. Specifically, the coordinates of each point in the first station point cloud can be converted into the coordinate system of the second station point cloud by using the optimized coordinate transformation matrix splicing, so that the two station point clouds are spliced.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a point cloud registration apparatus according to another exemplary embodiment of the present disclosure.

In some optional implementations, the first determining module 702 may include: a layering unit 7021 configured to layer the first site cloud and the second site cloud; a first determining unit 7022, configured to determine a target layer from each layer corresponding to the first site cloud and the second site cloud respectively; a second determining unit 7023, configured to determine a target layer corresponding to the first station cloud as a first target area, and determine a point cloud in the first target area as a first target point cloud; a third determining unit 7024, configured to determine that the target layer corresponding to the second station cloud is a second target area, and determine that the point cloud in the second target area is a second target point cloud.

In some optional implementations, the second determining module 703 may be further configured to: executing the following matching steps for a first preset number of times to obtain a first preset number of coordinate transformation matrixes: determining a first point set forming a preset shape from the first target point cloud; determining a second point set forming a shape identical to the preset shape from the second target point cloud; based on the first set of points and the second set of points, a coordinate transformation matrix is determined.

In some optional implementations, the preset shape is a triangle, and the first target point cloud and the second target point cloud are located in a horizontal middle layer; and the second determining module 703 determines a second point set constituting a shape congruent to the preset shape from the second target point cloud according to the following steps: determining a first point and a second point from the second target point cloud, wherein the distance between the first point and the second point on the horizontal plane is equal to the distance between the corresponding points in the first target point cloud on the horizontal plane; establishing a third point based on the first point and the second point, so that a triangle formed by the first point, the second point and the third point is congruent with a corresponding triangle in the first target point cloud; and determining a fourth point from the second target point cloud, so that the distance between the fourth point and the third point is smaller than or equal to a first preset distance, and determining the first point, the second point and the fourth point as a second point set.

In some alternative implementations, the optimization module 704 may include: a determining unit 7041, configured to perform coordinate transformation on the first target point cloud based on the coordinate transformation matrix for each coordinate transformation matrix of the first preset number of coordinate transformation matrices, so as to obtain a first transformed point cloud; determining an error between the first transformed point cloud and the second target point cloud; a first extracting unit 7042, configured to extract, according to an order from a small error to a large error, a second preset number of coordinate transformation matrices from the determined first preset number of coordinate transformation matrices; an optimizing unit 7043, configured to perform registration optimization on each coordinate transformation matrix in the second preset number of coordinate transformation matrices to obtain a second preset number of optimization matrices.

In some alternative implementations, determining unit 7041 may include: a first determining subunit 70411, configured to determine, for each point in the first transformed point cloud, a point, whose distance from the point is less than or equal to a second preset distance and whose normal vector included angle with the point is less than or equal to a preset angle, from the second target point cloud as a matching point; a second determining subunit 70412, configured to determine an error between the first transformed point cloud and the second target point cloud based on the number of the obtained pairs of matching points.

In some optional implementations, the splicing module 705 may include: a second extracting unit 7051, configured to extract the corresponding optimization matrix with the smallest error from the second preset number of optimization matrices; a verifying unit 7052, configured to perform the following verifying steps based on the extracted optimization matrix: performing coordinate transformation on the first station cloud by using the extracted optimization matrix to obtain a second converted point cloud; performing occlusion verification based on the second converted point cloud and the second station point cloud; determining whether occlusion verification passes; if the point cloud passes the judgment, determining that the currently obtained second converted point cloud and the second station point cloud are spliced point clouds; a third extracting unit 7053, configured to, if the extracted optimization matrix fails, re-extract the optimization matrix with the smallest error from among the unextracted optimization matrices in the second preset number of optimization matrices, and continue to perform the verifying step based on the currently extracted optimization matrix.

In some optional implementations, the verification unit 7052 includes: a verification subunit 70521, configured to determine that the second converted point cloud is an a point cloud, determine that the second station point cloud is a B point cloud, and perform the following occlusion verification steps: for each point in the A point cloud, determining a point which is in a shielding relation with the point and a point which is in a same position relation with the point from the B point cloud, wherein the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is larger than a first distance, the distance between the point which is in the shielding relation and the viewpoint of the B point cloud is smaller than or equal to a third preset distance, the first distance is the distance between the point and the viewpoint of the B point cloud, the distance between the point which is in the same position relation with the viewpoint of the B point cloud is approximately equal to the first distance, and the distance between the point which is in the first distance and the straight line is smaller than or equal to the; a third determining subunit 70522, configured to determine that the verification condition is satisfied if the number of the points in the same-position relationship is greater than or equal to the first number threshold and the number of the points in the occlusion relationship is less than or equal to the second number threshold; a fourth determining subunit 70523, configured to determine the second station point cloud as the point cloud a, determine the second converted point cloud as the point cloud B, and perform the occlusion verification step again; a fifth determining subunit 70524, configured to determine that occlusion verification passes if the verification condition is satisfied after the verification steps are performed twice.

In some optional implementations, the apparatus may further include: a third determining module 706, configured to determine a point whose normal vector is approximately vertical from the first station point cloud and the second station point cloud; a first transformation module 707, configured to perform rotation transformation on the first station point cloud and the second station point cloud so that an included angle between a point whose normal vector is approximately vertical and the vertical direction is minimum; a fourth determining module 708, configured to determine a point with the smallest vertical coordinate from the first station point cloud and the second station point cloud; the second transformation module 709 is configured to perform translation transformation on the first station point cloud and the second station point cloud, so that a vertical coordinate of a point with a minimum coordinate in the vertical direction is zero.

In some optional implementations, the apparatus further comprises: a second obtaining module 710, configured to obtain an initial first station point cloud and an initial second station point cloud; the down-sampling module 711 is configured to perform down-sampling on the initial first station point cloud and the initial second station point cloud respectively to obtain a first station point cloud and a second station point cloud.

According to the point cloud registration device provided by the embodiment of the disclosure, the target point clouds are extracted from the specific areas in the two station point clouds, the point sets corresponding to each other are determined from the two sets of target point clouds, the coordinate transformation matrix is determined according to the point sets, the coordinate transformation matrix is subjected to registration optimization, and finally the first station point cloud and the second station point cloud are spliced based on the optimized coordinate transformation matrix, so that the coordinate transformation matrix is determined according to part of the point clouds, the efficiency of point cloud splicing is improved, and the error rate of point cloud splicing is reduced.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 9. The electronic device may be either or both of the terminal device 101 and the server 103 as shown in fig. 1, or a stand-alone device separate from them, which may communicate with the terminal device 101 and the server 103 to receive the collected input signals therefrom.

FIG. 9 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

As shown in fig. 9, the electronic device 900 includes one or more processors 901 and memory 902.

The processor 901 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

Memory 902 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and executed by the processor 901 to implement the point cloud registration methods of the various embodiments of the present disclosure above and/or other desired functions. Various content such as the first site cloud, the second site cloud, and so on may also be stored in the computer readable storage medium.

In one example, the electronic device 900 may further include: an input device 903 and an output device 904, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, when the electronic device is the terminal device 101 or the server 103, the input device 903 may be a laser radar, a camera, a mouse, a keyboard, or the like, for inputting a point cloud. When the electronic device is a stand-alone device, the input device 903 may be a communication network connector for receiving the input point cloud from the terminal device 101 and the server 103.

The output device 904 may output various information including the stitched point cloud to the outside. The output devices 904 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device 900 relevant to the present disclosure are shown in fig. 9, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 900 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud registration method according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud registration method according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A point cloud registration method, comprising:

acquiring a first station point cloud and a second station point cloud, wherein the first station point cloud and the second station point cloud are acquired by a point cloud acquisition device at different positions;

determining a first target point cloud located in a first target area from the first station point cloud, and determining a second target point cloud located in a second target area from the second station point cloud;

respectively determining corresponding point sets from the first target point cloud and the second target point cloud, and determining a coordinate transformation matrix according to the point sets;

carrying out registration optimization on the coordinate transformation matrix;

and splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

2. The method of claim 1, wherein the determining a first target point cloud located at a first target area from the first station point cloud and a second target point cloud located at a second target area from the second station point cloud comprises:

layering the first site cloud and the second site cloud;

determining a target layer from layers corresponding to the first station point cloud and the second station point cloud respectively;

determining a target layer corresponding to the first station cloud as a first target area, and determining a point cloud in the first target area as a first target point cloud;

and determining a target layer corresponding to the second station cloud as a second target area, and determining the point cloud in the second target area as a second target point cloud.

3. The method of claim 1, wherein the determining a corresponding set of points from the first and second target point clouds, respectively, and a coordinate transformation matrix from the set of points comprises:

executing the following matching steps for a first preset number of times to obtain a first preset number of coordinate transformation matrixes:

determining a first point set forming a preset shape from the first target point cloud;

determining a second point set forming a shape congruent to the preset shape from the second target point cloud;

a coordinate transformation matrix is determined based on the first set of points and the second set of points.

4. The method of claim 3, wherein the preset shape is a triangle, the first and second target point clouds are located in a horizontal middle layer; and

the determining a second set of points from the second target point cloud that make up a shape congruent to the preset shape includes:

determining a first point and a second point from the second target point cloud, wherein a distance of the first point and the second point on a horizontal plane is equal to a distance of a corresponding point in the first target point cloud on the horizontal plane;

establishing a third point based on the first point and the second point, so that a triangle formed by the first point, the second point and the third point is congruent with a corresponding triangle in the first target point cloud;

determining a fourth point from the second target point cloud, so that the distance between the fourth point and the third point is smaller than or equal to a first preset distance, and determining the first point, the second point and the fourth point as a second point set.

5. A point cloud registration apparatus, comprising:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a first station point cloud and a second station point cloud, and the first station point cloud and the second station point cloud are acquired by a point cloud acquisition device at different positions;

the first determining module is used for determining a first target point cloud located in a first target area from the first station point cloud and determining a second target point cloud located in a second target area from the second station point cloud;

the second determining module is used for respectively determining corresponding point sets from the first target point cloud and the second target point cloud and determining a coordinate transformation matrix according to the point sets;

the optimization module is used for carrying out registration optimization on the coordinate transformation matrix;

and the splicing module is used for splicing the first station point cloud and the second station point cloud based on the optimized coordinate transformation matrix.

6. The apparatus of claim 5, wherein the first determining means comprises:

the layering unit is used for layering the first site cloud and the second site cloud;

the first determining unit is used for determining a target layer from layers corresponding to the first station point cloud and the second station point cloud respectively;

the second determining unit is used for determining that a target layer corresponding to the first station cloud is a first target area and determining that point cloud in the first target area is first target point cloud;

and the third determining unit is used for determining that the target layer corresponding to the second station cloud is a second target area and determining that the point cloud in the second target area is a second target point cloud.

7. The apparatus of claim 5, wherein the second determining means is further for:

8. The apparatus of claim 7, wherein the preset shape is a triangle, the first and second target point clouds being located in a horizontal middle layer; and

the second determining module determines a second point set forming a shape congruent to the preset shape from the second target point cloud according to the following steps:

9. A computer-readable storage medium, the storage medium storing a computer program for performing the method of any of the preceding claims 1-4.

10. An electronic device, the electronic device comprising:

a processor;

a memory for storing the processor-executable instructions;

the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method of any one of the claims 1 to 4.