CN113532311B

CN113532311B - Point cloud splicing method, device, equipment and storage equipment

Info

Publication number: CN113532311B
Application number: CN202010317943.2A
Authority: CN
Inventors: 欧清扬; 赵键
Original assignee: Guangdong Bozhilin Robot Co Ltd
Current assignee: Guangdong Bozhilin Robot Co Ltd
Priority date: 2020-04-21
Filing date: 2020-04-21
Publication date: 2023-06-09
Anticipated expiration: 2040-04-21
Also published as: CN113532311A; WO2021212844A1

Abstract

The embodiment of the invention discloses a point cloud splicing method, a point cloud splicing device, point cloud splicing equipment and a storage medium. The method comprises the following steps: collecting at least one frame of point cloud data for calibration according to a preset scanning track; determining converted optical center coordinates of each of at least one frame of point cloud data for calibration, wherein the converted optical center coordinates of each of the at least one frame of point cloud data for calibration are in a reference coordinate system of reference point cloud data, and the reference point cloud data are any one frame of the at least one frame of point cloud data for calibration; calibrating mechanism parameters of acquisition equipment according to the optical center fitting track of the converted optical center coordinates, wherein the optical center fitting track is a track which is fit according to the converted optical center coordinates; determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters; and splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data. So as to realize the effect of high-precision point cloud splicing.

Description

Point cloud splicing method, device, equipment and storage equipment

Technical Field

The embodiment of the invention relates to a track calibration technology, in particular to a point cloud splicing method, a device, equipment and a storage medium.

Background

The building actual measurement real-quantity robot is based on a high-precision visual sensing system, performs three-dimensional reconstruction on indoor data in a building construction stage, and processes three-dimensional point cloud data through a measurement algorithm, so that each index to be measured is obtained. Because the building data collected by the measuring robot is almost fully sampled, the measuring result can be quantitatively evaluated by an evaluation algorithm of the building point cloud data.

Because the span range of building measurement is usually larger, the point cloud viewing area obtained by single-frame shooting of a three-dimensional camera is limited, and a rigid body transformation matrix among multiple frames of point clouds needs to be calculated, and the rigid body transformation matrix is spliced to obtain complete indoor point cloud data. The transformation matrix is currently calculated mainly by the following way: firstly, the pose transformation of the three-dimensional acquisition equipment is fed back through an inertial measurement unit (Inertial measurement unit, IMU) and instant positioning and map construction (simultaneous localization and mapping, SLAM) information to obtain an initial point cloud registration matrix, and then matching point pairs are extracted through two-dimensional or three-dimensional image features to obtain an optimal matrix for fine registration.

The following problems exist in building measurement by using the point cloud splicing method: firstly, because the house point clouds are large planes which are perpendicular to each other, the image features are less, and the extraction of matching point pairs is not facilitated, so that the initial pose information needs to have higher precision; secondly, the accuracy of positioning information provided by the IMU and the SLAM is low, and the requirement of initial point cloud registration under a building measurement scene is difficult to meet; the conversion relation between the base coordinate system and the camera coordinate system is basically determined through a hand-eye calibration test at present so as to obtain an initial point Yun Peizhun matrix, but the calibration object of the test is usually a calibration plate, which is very small compared with the vision field of building measurement, and the calibrated scanning amplitude is very small in order to be able to shoot the same calibration plate, so that the calibration effect of the calibration test with difference is greatly reduced along with the increase of the vision field.

Disclosure of Invention

The embodiment of the invention provides a point cloud splicing method, a device, equipment and a storage medium, which are used for realizing the effect of high-precision point cloud splicing.

In a first aspect, an embodiment of the present invention provides a point cloud stitching method, where the method includes:

collecting at least one frame of point cloud data for calibration according to a preset scanning track;

determining converted optical center coordinates of the respective pre-conversion optical center coordinates of the at least one frame of point cloud data for calibration in a reference coordinate system of reference point cloud data, wherein the reference point cloud data is any frame of the at least one frame of point cloud data for calibration;

calibrating mechanism parameters of acquisition equipment according to the optical center fitting track of the converted optical center coordinates, wherein the optical center fitting track is a track which is fit according to the converted optical center coordinates;

determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters;

and splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data.

Optionally, the collecting at least one frame of calibration point cloud data with a preset scanning track includes:

Based on a preset scanning track, carrying out point cloud scanning on a region to be rebuilt by acquisition equipment, and acquiring at least one frame of point cloud data for calibration, wherein different characteristic targets are arranged on the region to be rebuilt.

Therefore, a preset scanning track is planned in advance, point cloud scanning is conducted on the area to be rebuilt based on the preset scanning track, time is saved, and scanning efficiency is improved.

Optionally, each of the feature targets is provided with a unique code identifier, so that the obtained point cloud data for calibration can be corresponding to each of the feature targets through the unique code identifier.

Optionally, the feature target is disposed in a scan overlapping area of the to-be-reconstructed area, where the scan overlapping area is determined according to a preset scan trajectory and a view angle of the acquisition device.

Optionally, the determining the post-conversion optical center coordinates of the pre-conversion optical center coordinates of each of the at least one frame of calibration point cloud data in the reference coordinate system of the reference point cloud data includes:

determining respective transformation matrices when other point cloud data in the at least one frame of point cloud data for calibration is converted into a reference coordinate system of the reference point cloud data, wherein the other point cloud data is the point cloud data for calibration of other frames except the reference point cloud data in the at least one frame of point cloud data for calibration; determining the transformed optical center coordinates of the other point cloud data under the reference coordinate system according to the transformed optical center coordinates of the other point cloud data and the corresponding transformation matrixes; the optical center coordinates of the datum point cloud data and the converted optical center coordinates of the other point cloud data form converted optical center coordinates of the at least one frame of calibration point cloud data.

Therefore, the converted optical center coordinates of the optical center of any other point cloud data under the reference coordinate system of the reference point cloud data can be accurately obtained, so that the subsequent converted optical center coordinates of the optical center of the other accurate point cloud data under the reference coordinate system can be conveniently obtained, and the mechanism parameters of the acquisition equipment can be obtained.

Optionally, when determining that other point cloud data in the at least one frame of calibration point cloud data is converted to the reference coordinate system of the reference point cloud data, each transformation matrix includes:

extracting features of the datum point cloud data and any one of the other point cloud data, and determining matching feature pairs for splicing; determining respective conversion parameters when any one of the other point cloud data is converted to the reference coordinate system according to the matching feature pairs; and determining respective transformation matrixes when any one of the other point cloud data is transformed to the reference coordinate system according to the respective transformation parameters.

Thus, a transformation matrix for converting other point cloud data into a reference coordinate system corresponding to the reference point cloud data can be accurately obtained.

Optionally, the conversion parameters include a rotation angle and a translation amount.

Optionally, the calibrating the mechanism parameters of the collecting device according to the optical center fitting track of the converted optical center coordinates includes:

Performing radius-constrained plane circle fitting on at least one converted optical center coordinate, and determining that a plane circle obtained by fitting is an optical center fitting track; and calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track.

Therefore, the mechanism parameters of the acquisition equipment can be calibrated based on the optical center fitting track, and then the target transformation matrix for converting the point cloud data under any high-precision scanning pose into the reference coordinate system can be obtained according to the obtained mechanism parameters.

Optionally, the fitting of the plane circle with radius constraint on at least one transformed optical center coordinate, determining the plane circle obtained by fitting as an optical center fitting track, includes:

determining a projection coordinate point of each converted optical center coordinate on a fitting surface according to at least one converted optical center coordinate and a normal vector of the fitting surface of at least one converted optical center coordinate, wherein the fitting surface is a plane which is synthesized according to the converted optical center coordinate, and the projection coordinate point is a nearest neighbor coordinate point of the converted optical center coordinate projected onto the fitting surface; and carrying out radius constraint plane circle fitting on the plurality of projection coordinate points, and determining the plane circle obtained by fitting as an optical center fitting track.

The method comprises the steps of obtaining projection point coordinates of converted optical center coordinates on a fitting surface, so that accurate mechanism parameters are obtained based on the accurately determined projection coordinate points, fitting a plane circle with radius constraint on a plurality of projection coordinate points, and determining the fitted plane circle as an optical center fitting track, so that high-precision mechanism parameters are obtained based on the optical center fitting track, and further, a target transformation matrix for converting point cloud data under any scanning pose into a reference coordinate system is obtained based on the mechanism parameters.

Optionally, the mechanism parameters include: the axial length of each level of rotating shafts of the acquisition equipment, the axial center coordinates of each level of rotating shafts and the included angles of each level of rotating shafts;

calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track, wherein the calibration comprises the following steps:

determining that the radius of a fitted plane circle corresponding to the current optical center fitting track is the axial length of a current-stage rotating shaft of the acquisition equipment, and the center of the fitted plane circle is the axial coordinate of the current-stage rotating shaft; performing radius-constrained plane circle fitting on the axis coordinates of at least one current-stage rotating shaft, and determining that the radius of a plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axial length of a next-stage rotating shaft, wherein the circle center of the plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axis coordinates of the next-stage rotating shaft; and determining an included angle formed between each level of rotating shafts according to the normal vector of the fitted plane circle of each level of rotating shafts, wherein in the acquisition equipment, the next level of rotating shaft is positioned at the inner side of the current level of rotating shaft.

The axial length and the axial center coordinates of the current rotating shaft of the acquisition equipment are fitted according to the current optical center fitting track, then the optical center fitting track is carried out based on the axial center coordinates of the current rotating shaft, the axial length and the axial center coordinates of the next-stage rotating shaft can be fitted, the included angle formed between the rotating shafts at all stages can be determined based on the normal vector of the fitted plane circles of the rotating shafts at all stages, and thus the mechanism parameters of the acquisition equipment can be calibrated based on the optical center fitting track, so that the target transformation matrix for converting the point cloud data under any scanning pose into the reference coordinate system can be obtained based on the mechanism parameters. Meanwhile, according to the obtained shaft length and shaft center coordinates of each rotating shaft, the motor motion precision of the acquisition equipment can be checked, in the prior art, the actual rotation angle information of the motor is fed back through an inclination angle sensor of the acquisition equipment, the actual rotation angle information is based on gravity sensing, and the measured angle is in a world coordinate system instead of a coordinate system of a camera. Without complicated calibration conversion, it is difficult to feedback the actual rotation angle of the motor with high accuracy. The captured camera optical center represents a motion track under a camera coordinate system, namely reflects the actual motor scanning pose.

Optionally, the determining, according to the mechanism parameter, a target transformation matrix for transforming the original point cloud data acquired by the acquisition device under any scanning pose to the reference coordinate system includes:

acquiring the rotation angle and displacement of an end scanner of the acquisition equipment under any scanning pose; and determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters, the rotation angle and the displacement.

Therefore, the initial point cloud data under any scanning pose is spliced based on the target transformation matrix so as to reconstruct the three-dimensional region to be reconstructed.

Optionally, according to the target transformation matrix, the original point cloud data is spliced to obtain spliced target point cloud data, which includes:

and according to the target transformation matrix, converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system to obtain spliced target point cloud data.

Therefore, the initial point cloud data under any scanning pose is unified to the reference coordinate system of the reference optical center coordinates of the reference point cloud data, the effect of high-precision splicing of the initial point cloud data under any scanning pose can be achieved, and further high-precision three-dimensional reconstruction of the area to be reconstructed is achieved.

In a second aspect, an embodiment of the present invention further provides a point cloud stitching device, where the device includes:

the calibration point cloud data acquisition module is used for acquiring at least one frame of calibration point cloud data according to a preset scanning track;

the optical center coordinate conversion module is used for determining converted optical center coordinates of the optical center coordinates before conversion of the at least one frame of point cloud data under a reference coordinate system of the reference point cloud data based on the at least one frame of point cloud data for calibration, wherein the reference point cloud data is any frame of the at least one frame of point cloud data for calibration;

the mechanism parameter calibration module is used for calibrating mechanism parameters of the acquisition equipment according to the optical center fitting track of the converted optical center coordinates, wherein the optical center fitting track is a track which is synthesized according to the converted optical center coordinates;

the target transformation matrix determining module is used for determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters;

and the point cloud data splicing module is used for splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud.

In a third aspect, an embodiment of the present invention further provides an apparatus, including:

one or more processors;

a storage means for storing one or more programs;

and when the one or more programs are executed by the one or more processors, the one or more processors implement the point cloud stitching method according to any one of the embodiments of the present invention.

In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the point cloud stitching method according to any of the embodiments of the present invention.

According to the technical scheme, at least one frame of point cloud data for calibration is acquired through a preset scanning track, the optical center coordinates of the at least one frame of point cloud data for calibration are unified to the same coordinate system, and the mechanism parameters of the acquisition equipment are calibrated through the optical center fitting track of the optical center coordinates under the same coordinate system, so that the calibration of the acquisition equipment is realized. Based on the mechanism parameters of the acquisition equipment, a target transformation matrix for converting the initial point cloud data under any scanning pose into a reference coordinate system can be obtained, so that high-precision point cloud splicing of the initial point cloud data under any scanning pose in the area to be reconstructed is realized.

Drawings

Fig. 1 is a flowchart of a point cloud stitching method in a first embodiment of the present invention;

FIG. 2 is a schematic diagram of a feature target arrangement in accordance with a first embodiment of the invention;

FIG. 3 is a schematic diagram of a preset scan trajectory in accordance with a first embodiment of the present invention;

fig. 4 is a flowchart of a point cloud stitching method in the second embodiment of the present invention;

fig. 5 is a schematic diagram of unifying optical centers of other point cloud data to reference coordinates in the second embodiment of the present invention;

fig. 6 is a flowchart of a point cloud stitching method in the third embodiment of the present invention;

FIG. 7 is a schematic diagram of a dual axis mechanism of the acquisition device in a third embodiment of the present invention;

FIG. 8 is a schematic diagram of a trace of a calibration point Yun Nige in a third embodiment of the invention;

fig. 9 is a flowchart of a point cloud stitching method in a fourth embodiment of the present invention;

fig. 10 is a schematic structural diagram of a point cloud splicing device in a fifth embodiment of the present invention;

fig. 11 is a schematic structural view of an apparatus in a sixth embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.

Example 1

Fig. 1 is a flowchart of a point cloud splicing method according to an embodiment of the present invention, where the embodiment is applicable to a situation where point clouds measured at any angle are spliced, the method may be performed by a point cloud splicing device, the point cloud splicing device may be implemented by software and/or hardware, and the point cloud splicing device may be configured on a computing device, and specifically includes the following steps:

s110, acquiring at least one frame of point cloud data for calibration according to a preset scanning track.

The preset scan trajectory may be, for example, a scan trajectory that is planned in advance based on a scene of the region to be reconstructed before scanning the region to be reconstructed. The calibration point cloud data can be any point cloud data of the area to be reconstructed. The calibration point cloud data can be point cloud data of any area to be reconstructed, which is shot at equal angles through acquisition equipment, and the target conversion matrix for splicing initial point cloud data of any scanning pose in the area to be reconstructed is obtained based on the acquired calibration point cloud data, so that high-precision point cloud splicing of the initial point cloud data of any scanning pose in the area to be reconstructed is realized.

Optionally, the collecting at least one frame of calibration point cloud data with a preset scanning track may specifically be: based on a preset scanning track, carrying out point cloud scanning on a region to be rebuilt by acquisition equipment, and acquiring at least one frame of point cloud data for calibration, wherein different characteristic targets are arranged on the region to be rebuilt.

The acquisition device may be, for example, a device that acquires at least one frame of calibration point cloud data of the region to be reconstructed, for example, a camera or the like. The area to be reconstructed may be an area reconstructed according to the acquired calibration point cloud data of any area, for example, the area to be reconstructed may be any building, any room, or the like. Taking a room in a large building as an example, because the point cloud in the room is generally a large plane perpendicular to each other, the image features are less, which is not beneficial to the extraction of the matched point pairs, and thus the initial pose information of the acquisition device cannot be accurately confirmed, therefore, referring to the feature target arrangement schematic diagram shown in fig. 2, different feature targets can be arranged on the area to be reconstructed for adding image features to obtain the accurate initial pose information of the acquisition device, the feature targets can be marks for marking, for example, marking paper, and unique marks can be arranged on the marking paper, so that the obtained point cloud data for marking can be corresponding to the feature targets through unique code marks, and the point cloud data for marking in the area to be reconstructed can be spliced based on the corresponding relation between the unique marks and the feature targets. Specifically, the feature target may be disposed in a scan overlap region of the region to be reconstructed, where the scan overlap region is determined according to a preset scan trajectory and field angle of the acquisition device. Therefore, a preset scanning track is planned in advance, point cloud scanning is conducted on the area to be rebuilt based on the preset scanning track, time is saved, and scanning efficiency is improved.

Exemplary, common scanning modes of the acquisition device include single/multi-track linear scanning, single/multi-axis rotational scanning, combined linear and rotational scanning, and the like. The tracks of the scanning modes are all a combination of straight lines and circles, for example, taking a shooting mode of biaxial pitching rotation as an example, as shown in a preset scanning track schematic diagram in fig. 3, and the three-dimensional camera scans the area to be reconstructed, on which the feature target is arranged, in a mode of pitch rotation for 4 times and azimuth rotation for 12 times.

S120, determining converted optical center coordinates of the optical center coordinates before conversion of the at least one frame of point cloud data for calibration under a reference coordinate system of reference point cloud data, wherein the reference point cloud data is any frame of the at least one frame of point cloud data for calibration.

The reference point cloud data may be, for example, point cloud data of other points that move or otherwise convert based on the reference point cloud data, where the reference point cloud data may be any frame of point cloud data selected from calibration point cloud data as the reference point cloud data, and optionally, first frame of calibration point cloud data may be used as the reference point cloud data. The pre-conversion optical center coordinates may be initial optical center coordinates of at least one frame of calibration point cloud data. The converted optical center coordinates may be optical center coordinates obtained by converting at least one frame of calibration point cloud data into a reference coordinate system of the reference point cloud data. Because the acquisition equipment is moving all the time when acquiring the calibration point cloud data of the area to be reconstructed, the optical center coordinates of the acquired calibration point cloud data are not in the same coordinate system, and if the area to be reconstructed is to be reconstructed by splicing the calibration point cloud data, the optical center coordinates of the calibration point cloud data are required to be unified to the same coordinate system. According to the datum point cloud data, the pre-conversion optical center coordinates of other calibration point cloud data except the datum point cloud data can be converted into the datum coordinate system corresponding to the datum point cloud data through movement of a certain distance and/or rotation of a certain angle, and converted optical center coordinates of optical centers of other calibration point cloud data except the datum point cloud data in the datum coordinate system corresponding to the datum point cloud data are obtained, so that a target conversion matrix for splicing initial point cloud data in any scanning pose in a region to be reconstructed is conveniently obtained based on the calibration point cloud data in the same coordinate system, and high-precision point cloud splicing of the initial point cloud data in any scanning pose in the region to be reconstructed is achieved.

S130, calibrating mechanism parameters of the acquisition equipment according to the optical center fitting track of the converted optical center coordinates, wherein the optical center fitting track is a track which is fit according to the converted optical center coordinates.

The mechanism parameter may be parameter information of the acquisition device, for example, an axial length of each rotation axis of the acquisition device, an axial center coordinate of each rotation axis, an included angle formed by each rotation axis, and the like. And carrying out plane circle fitting on the optical center coordinates of the point cloud data for calibration unified to the reference coordinate system, obtaining a normal vector of the fitted circle, a circle center coordinate and a radius of the fitted circle according to the fitted track, obtaining the mechanism parameters of the acquisition equipment in the reference coordinate system according to the corresponding relation between the normal vector of the fitted circle, the circle center coordinate and the radius of the fitted circle and the mechanism parameters of the acquisition equipment based on the obtained normal vector of the fitted circle, the circle center coordinate and the radius of the fitted circle, and obtaining a target transformation matrix for converting the initial point cloud data under any scanning pose into the reference coordinate system based on the obtained mechanism parameters of the acquisition equipment so as to realize high-precision point cloud splicing of the initial point cloud data under any scanning pose in the area to be reconstructed. Meanwhile, track fitting is carried out on the basis of optical center coordinates of the point cloud data for calibration under a reference coordinate system, and the mechanism parameters of the acquisition equipment under the reference coordinate system are obtained according to the fitted tracks, so that the method can also be used as a mode for checking the mechanism parameter design values of the acquisition equipment, manufacturing errors and fit gaps exist between the actual mechanism parameters and the design values of the acquisition equipment, even if three-dimensional coordinate scanning measurement is carried out on the outer surface of the acquisition equipment, the actual distances among joints of the structures of the acquisition equipment are difficult to obtain, and the track fitting of the optical center coordinates of the point cloud data for calibration under the reference coordinate system directly reflects the actual motion pose of the acquisition equipment, so that the mechanism parameters of the acquisition equipment are fitted.

And S140, determining a target transformation matrix for converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters.

For example, the initial point cloud data may be point cloud data of an area to be reconstructed acquired by the acquisition device under any scanning pose. The target transformation matrix may be a matrix of transformation of point cloud data under an arbitrary scanning pose to a reference coordinate system. After the mechanism parameters of the acquisition equipment under the reference coordinate system are obtained, a target conversion matrix for converting the initial point cloud data under any scanning pose into the reference coordinate system can be determined according to the sum of the parameter information of the acquisition equipment under any scanning pose and forward kinematics, so that high-precision point cloud splicing of the initial point cloud data under any scanning pose in the area to be reconstructed can be realized based on the obtained target conversion matrix.

And S150, splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data.

The target point cloud data may be, for example, point cloud data obtained by subjecting initial point cloud data to a target transformation matrix, and three-dimensional reconstruction of the region to be reconstructed may be achieved by using the target point cloud data. The parameter information of the initial point cloud data under any scanning pose can be added into the target conversion matrix, so that the initial point cloud data under any scanning pose can be unified under a reference coordinate system of the reference point cloud data, the registration of the point cloud data of the initial point cloud data under any scanning pose is completed, the complete point cloud data of the area to be reconstructed is obtained, and the high-precision three-dimensional reconstruction is carried out on the area to be reconstructed based on the complete point cloud data of the area to be reconstructed.

Example two

Fig. 4 is a flowchart of a point cloud stitching method provided in a second embodiment of the present invention, where the embodiment of the present invention is a further optimization of the foregoing embodiment based on the foregoing embodiment, and specifically includes the following steps:

s210, acquiring at least one frame of point cloud data for calibration according to a preset scanning track.

S220, determining respective transformation matrixes when other point cloud data in the at least one frame of point cloud data for calibration are converted into a reference coordinate system of the reference point cloud data, wherein the other point cloud data are the point cloud data for calibration of other frames except the reference point cloud data in the at least one frame of point cloud data for calibration.

The reference coordinate system may be, for example, a coordinate system in which the reference point cloud data is located. The transformation matrix may be a matrix that converts other point cloud data into a reference coordinate system corresponding to the reference point cloud data. Because the collected point cloud data for calibration are not in the same coordinate system, in order to unify other point cloud data to the coordinate system of the reference point cloud data, for example, a Scale-invariant feature transform (SIFT) algorithm and a latest iterative algorithm (Iterative Closest Point, ICP) may be used to determine respective corresponding transformation matrices when other point cloud data are transformed to the reference coordinate system corresponding to the reference point cloud data, so that the mechanism parameters of the collecting device in the reference coordinate system can be obtained by respective corresponding transformation matrices when the other point cloud data are subsequently transformed to the reference coordinate system corresponding to the reference point cloud data based on the other point cloud data.

Optionally, when determining that other point cloud data in the at least one frame of point cloud data for calibration converts to the reference coordinate system of the reference point cloud data, the respective transformation matrix may specifically be: extracting features of the datum point cloud data and any one of the other point cloud data, and determining matching feature pairs for splicing; determining respective conversion parameters when any one of the other point cloud data is converted to the reference coordinate system based on the matching feature pairs; and determining respective transformation matrixes when any one of the other point cloud data is transformed to the reference coordinate system based on the respective transformation parameters.

Illustratively, the matching feature pair may be a feature that can match the reference point cloud data and any other point cloud data, for example, may be a feature target in the first embodiment described above, or the like. Specifically, the position of the feature target is determined based on RGB values of other point cloud data and reference point cloud data, and the matching feature pair is determined based on the determined center position of the feature target to obtain the angle and distance between the feature targets matched with the feature targets of any other point cloud data and reference point cloud data. In the matching of the feature targets, specifically, the feature targets of the next frame and the previous frame may be matched, for example, taking the case that the reference point cloud data is the first frame of point cloud data, firstly, the feature targets of the 2 nd frame of point cloud data and the feature targets of the 1 st frame of point cloud data are matched, after the matching is completed, the point cloud data of the 2 nd frame is just under the base coordinate system of the 1 st frame of point cloud data, then the 3 rd frame of point cloud data and the feature targets of the 2 nd frame of point cloud data are matched, and so on until all other point cloud data are matched, so that the situation that the other point cloud data cannot be matched with the reference point cloud data when the matching features matched with the reference point cloud data may not exist in the other point cloud data of the frame far from the reference point cloud data can be avoided. The conversion parameter may be a parameter for converting any other point cloud data into the reference coordinate system, alternatively, the conversion parameter may be a rotation angle and a translation amount, where the rotation angle may be a rotation angle through which any other point cloud data may be converted into the reference coordinate system, and the translation amount may be a translation distance through which any other point cloud data may be converted into the reference coordinate system.

Exemplary, based on the matching feature pairs, conversion parameters for converting any other point cloud data into the reference coordinate system can be obtained based on a conversion rule, for example, specifically, matching feature pairs determined based on RGB, extracting center coordinates of feature targets of any matching feature pair, obtaining center distances of the center coordinates of the two feature targets, constructing a root mean square error function, and fitting errors by a nearest iteration algorithm (Iterative Closest Point, ICP)And stopping iteration until the error function is smaller than the threshold value, wherein the parameter of the error function obtained after stopping iteration is the conversion parameter. After the conversion parameters are determined, the conversion parameters are combined to form a conversion matrix corresponding to any other point cloud data, so that the conversion matrix corresponding to each other can be accurately obtained when any other point cloud data is converted to the reference coordinate system corresponding to the reference point cloud data. For example, the transformation matrix here may be:

wherein (1)>

For a rotation matrix of rotation angles +.>

A translation matrix that is the amount of translation. Thus, when other point cloud data are accurately converted into a reference coordinate system corresponding to the reference point cloud data, respective transformation matrixes can be obtained.

S230, determining the transformed optical center coordinates of the other point cloud data under the reference coordinate system according to the transformed optical center coordinates of the other point cloud data and the corresponding transformation matrixes.

For example, the pre-conversion optical center coordinates of the other point cloud data are (0, 0), and the post-conversion optical center coordinates of the pre-conversion optical center coordinates of the other point cloud data in the reference coordinate system can be determined by using the obtained respective transformation matrix of the other point cloud data and the pre-conversion optical center coordinates of the other point cloud data according to the following calculation formula:

wherein, the liquid crystal display device comprises a liquid crystal display device,

for the respective change matrix of other point cloud data, < +.>

Pre-conversion optical center coordinates for other point cloud data, < ->

And converting the pre-conversion optical center coordinates of the other point cloud data into converted optical center coordinates under a reference coordinate system. The optical center coordinates of the datum point cloud data and the converted optical center coordinates of other point cloud data form converted optical center coordinates of at least one frame of point cloud data for calibration. Therefore, the optical center coordinates of the optical center of any other point cloud data under the reference coordinate system of the reference point cloud data can be accurately obtained, so that the subsequent optical center coordinates of the optical center of the accurate other point cloud data under the reference coordinate system can be conveniently obtained, and the mechanism parameters of the acquisition equipment can be obtained.

For example, according to the pre-conversion optical center coordinates of other point cloud data and the corresponding transformation matrix, the post-conversion optical center coordinates of the pre-conversion optical center coordinates of other point cloud data under the reference coordinate system are determined, so that the calibration point cloud data are unified under the coordinate system of a certain calibration point cloud data, it can be understood that the shooting is performed in a shooting mode of biaxial pitching rotation, and because the biaxial axes of the acquisition equipment are mutually perpendicular, the optical centers of the acquisition equipment are actually distributed on a spherical surface with the oblique triangle side formed by the biaxial axial lengths as the spherical radius, and therefore, the track of the optical center of the acquisition equipment presents a spherical surface. Referring to the schematic diagram of the other point cloud data shown in fig. 5 in which the optical centers of the other point cloud data are unified to the reference coordinate, the a diagram in fig. 5 is all the point cloud data for calibration, wherein a is the reference point cloud data, the other point cloud data are other point cloud data, and as can be seen from the a diagram in fig. 5, all the point cloud data for calibration are dispersed and not associated with each other, and after the other point cloud data are transformed by the respective change matrix, the optical centers of the other point cloud data are unified to the reference coordinate system of the reference point cloud data, so that the point cloud data for calibration are related to each other, as shown in the b diagram in fig. 5, for example, the point cloud data for calibration are unified to the surface of a sphere, so that the positional relationship between any one of the other point cloud data and the other point cloud data for calibration can be clearly known, and the mechanism parameters of the acquisition equipment can be determined later.

S240, calibrating mechanism parameters of the acquisition equipment according to the optical center fitting track of the converted optical center coordinates, wherein the optical center fitting track is a track which is fit according to the converted optical center coordinates.

S250, determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters.

And S260, splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data.

According to the technical scheme, when other point cloud data in the at least one frame of point cloud data for calibration are converted into the reference coordinate system of the datum point cloud data, the transformation matrixes are respectively formed, wherein the other point cloud data are the point cloud data for calibration of other frames except the datum point cloud data in the at least one frame of point cloud data for calibration, so that the mechanism parameters of the acquisition equipment under the reference coordinate system can be obtained by respectively forming the transformation matrixes when the other point cloud data are converted into the reference coordinate system corresponding to the datum point cloud data. And determining the converted optical center coordinates of the other point cloud data under the reference coordinate system according to the converted optical center coordinates of the other point cloud data and the corresponding respective transformation matrixes, so that the optical center coordinates of the optical center of any other point cloud data under the reference coordinate system of the reference point cloud data can be accurately obtained, and the mechanism parameters of the acquisition equipment can be obtained based on the optical center coordinates of the optical center of the accurate other point cloud data under the reference coordinate system.

Example III

Fig. 6 is a flowchart of a point cloud stitching method provided in a third embodiment of the present invention, where the embodiment of the present invention is a further optimization of the foregoing embodiment based on the foregoing embodiment, and specifically includes the following steps:

s310, acquiring at least one frame of point cloud data for calibration according to a preset scanning track.

S320, determining respective transformation matrixes when other point cloud data in the at least one frame of point cloud data for calibration is converted into a reference coordinate system of the reference point cloud data, wherein the other point cloud data is the point cloud data for calibration of other frames except the reference point cloud data in the at least one frame of point cloud data for calibration.

S330, determining the transformed optical center coordinates of the other point cloud data under the reference coordinate system according to the transformed optical center coordinates of the other point cloud data and the corresponding transformation matrixes.

S340, performing radius-constrained plane circle fitting on at least one converted optical center coordinate, and determining the plane circle obtained by fitting as an optical center fitting track.

For example, the radius-constrained planar circle fit may be a planar circle fit with a radius to at least one transformed optical center coordinate. And performing radius-constrained planar circle fitting on at least one converted optical center coordinate to obtain a planar circle, namely an optical center fitting track. Therefore, according to the fitted optical center fitting track, the mechanism parameters of the acquisition equipment can be calibrated based on the corresponding relation between the parameters in the optical center fitting track and the mechanism parameters of the acquisition equipment.

Optionally, performing radius constraint planar circle fitting on at least one converted optical center coordinate, and determining that a planar circle obtained by fitting is an optical center fitting track, which may specifically be: determining a projection coordinate point of each converted optical center coordinate on a fitting surface according to at least one converted optical center coordinate and a normal vector of the fitting surface of at least one converted optical center coordinate, wherein the fitting surface is a plane which is synthesized according to the converted optical center coordinate, and the projection coordinate point is a nearest neighbor coordinate point of the converted optical center coordinate projected onto the fitting surface; and carrying out radius constraint plane circle fitting on the plurality of projection coordinate points, and determining the plane circle obtained by fitting as an optical center fitting track.

Taking azimuth rotation of a biaxial rotation acquisition device as an example, assuming that the azimuth is rotated by 12 points in units of 30 degrees, the optical center coordinates of the optical center of other point cloud data obtained by the transformation matrix under the basic coordinate system are as follows: p1 (x 1, y1, z 1), …, P12 (x 12, y12, z 12). And when the three-dimensional circle is fitted, a three-dimensional rotation plane (normal vector) is fitted, and then the three-dimensional circle center is fitted, wherein the fitting of the rotation plane can be calculated by a least square method, and the residual error of all the point cloud data for calibration to the fitting surface is minimized.

Let the equation of the space plane be ax+by+cz=1, then a plane matrix equation of 12 calibration point cloud data can be listed:

recording device

At this time, the weight matrix is an identity matrix, and the least square method vtpv=min indicates that the direction coefficient of the normal vector of the space is:

X＝(N ^T N) ^-1 N ^T I

the obtained normal vector coefficients are unitized, a fitting space plane equation is set as a ' x+b ' y+c ' z+d=0, projection points of the 12 calibration point cloud data of P1-P12 on the fitting plane are set as P1' (x 1', y1', z1 '), …, P12' (x 12', y12', z12 '), and projection point coordinates are as follows:

wherein k= -a' x _i -b′y _i -c′z _i -d。

The plane circle fitting of radius constraint is performed on a plurality of projection coordinate points, specifically may be according to the formula:

thus, all the point cloud data for calibration and the fitted plane circle are obtainedThe distances of the circle center coordinates are consistent, and then the optical center fitting track is obtained.

And determining the projection coordinate point of each converted optical center coordinate on the fitting surface according to at least one converted optical center coordinate and the normal vector of the fitting surface of at least one converted optical center coordinate, so that the subsequent fitting of a plane circle with radius constraint based on the precisely determined projection coordinate point can be performed, and the accurate mechanism parameters of the acquisition equipment can be obtained based on the fitted plane circle.

S350, calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track.

The mechanical parameters here may be, for example, the axial length of the rotation axes of the stages in the acquisition device, the axial coordinates of the rotation axes of the stages and the angle of the rotation axes of the stages. And carrying out radius-constrained plane circle fitting on the optical center coordinates of the point cloud data for calibration under a reference coordinate system based on the optical center fitting track, so as to obtain the radius and the circle center of the fitted plane circle, and obtaining the axial length of each level of rotating shafts and the axial center coordinates of each level of rotating shafts based on the relation between the radius and the circle center of the plane circle and the axial length and the axial center coordinates of each level of rotating shafts. Thus, high-precision mechanism parameters can be obtained, so that a target conversion matrix for converting initial point cloud data under any scanning pose into a reference coordinate system can be obtained based on the mechanism parameters.

Optionally, calibrating the mechanism parameters of the acquisition device according to the optical center fitting track may specifically be: determining that the radius of a fitted plane circle corresponding to the current optical center fitting track is the axial length of a current-stage rotating shaft of the acquisition equipment, and the center of the fitted plane circle is the axial coordinate of the current-stage rotating shaft; performing radius-constrained plane circle fitting on the axis coordinates of at least one current-stage rotating shaft, and determining that the radius of a plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axial length of a next-stage rotating shaft, wherein the circle center of the plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axis coordinates of the next-stage rotating shaft; and determining an included angle formed between each level of rotating shafts according to the normal vector of the fitted plane circle of each level of rotating shafts, wherein in the acquisition equipment, the next level of rotating shaft is positioned at the inner side of the current level of rotating shaft.

By way of example, the acquisition device shown with reference to fig. 7 is a schematic illustration of a biaxial mechanism, where the current axis of rotation may be the axis of rotation of the outermost end of the acquisition device, e.g. axis of rotation a as in fig. 7. The rotation axis of the next stage may be the rotation axis of the current rotation axis toward the inner layer in the collection device, for example, as rotation axis B in fig. 7. The current rotation axis is here located at the outer end of the acquisition device compared to the rotation axis of the next stage. The radius of the fitted plane circle corresponding to the current optical center fitting track is the axial length of the current-stage rotating shaft of the acquisition equipment, the circle center of the fitted plane circle is the axial center coordinate of the current-stage rotating shaft, after the axial center coordinate of the current-stage rotating shaft is determined, the axial center coordinate of at least one current-stage rotating shaft is subjected to radius-constrained plane circle fitting, the radius of the plane circle fitted by the axial center coordinate of at least one current-stage rotating shaft is taken as the axial length of the next-stage rotating shaft, the circle center of the plane circle fitted by the axial center coordinate of at least one current-stage rotating shaft is taken as the axial center coordinate of the next-stage rotating shaft, and the included angle formed between the rotating shafts of all stages can be determined according to the normal vector of the fitted plane circle of each stage rotating shaft, wherein the next-stage rotating shaft is positioned at the inner side of the current-stage rotating shaft in the acquisition equipment. Therefore, the included angle formed by the axial length, the axial center coordinate and the rotating shafts of each level can be obtained, so that the target conversion matrix for converting the initial point cloud data under any scanning pose into the reference coordinate system can be obtained based on the axial length, the axial center coordinate and the included angle formed by the rotating shafts of each level. Meanwhile, according to the obtained shaft length of each level of rotation shaft, the shaft center coordinates and the included angle formed between each level of rotation shaft, the motor motion precision of the acquisition equipment can be checked, in the prior art, the actual rotation angle information of the motor is fed back through the inclination angle sensor of the acquisition equipment, the actual rotation angle information is based on gravity sensing, and the measured angle is in a world coordinate system instead of a camera coordinate system. Without complicated calibration conversion, it is difficult to feedback the actual rotation angle of the motor with high accuracy. The captured camera optical center represents a motion track under a reference coordinate system, namely reflects the actual motor scanning pose.

It should be noted that, referring to the schematic trace of the calibration point Yun Nige shown in fig. 8, as shown in fig. 5, although other point cloud data is unified to the reference coordinate system of the reference point cloud data, the positional relationship between other point cloud data and the reference point cloud data can be known, but the positional relationship between other point cloud data is not clear, therefore, it is necessary to unify the other point cloud data and the reference point cloud data to another coordinate system, as shown in a diagram a in fig. 8, firstly, the plane circle fitted by the optical center coordinates of the current rotation axis is determined by the above method based on the optical center coordinates of the current rotation axis, when the plane circle fitted by the optical center coordinates of the current rotation axis is completed, the circle center of the plane circle fitted by the optical center coordinates of the current rotation axis is determined, then the circle center of the plane circle fitted by the optical center coordinates of the current rotation axis is determined, and then the fitting of the optical center coordinates of the next rotation axis is performed by the above fitting method, as shown in a diagram b in fig. 8, and so on, the other point cloud data and the reference point cloud data can be unified to the same coordinate system, namely, the plane circle fitted by the above method, and the position relationship between the other point cloud data and the reference point cloud data can be determined, and the plane coordinate is the plane circle fitted from the inside to the most point coordinate fitting device.

S360, determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters.

And S370, splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data.

According to the technical scheme provided by the embodiment of the invention, the plane circle obtained by fitting is determined to be the optical center fitting track by performing radius-constrained plane circle fitting on at least one converted optical center coordinate, so that the mechanical parameters of the acquisition equipment can be calibrated based on the corresponding relation between the parameters in the optical center fitting track and the mechanical parameters of the acquisition equipment according to the fitted optical center fitting track. And calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track, so that the high-precision mechanism parameters can be obtained, and a target conversion matrix for converting initial point cloud data under any scanning pose into a reference coordinate system can be obtained based on the mechanism parameters.

Example IV

Fig. 9 is a flowchart of a point cloud stitching method provided in a fourth embodiment of the present invention, where the embodiment of the present invention is a further optimization of the foregoing embodiment based on the foregoing embodiment, and specifically includes the following steps:

S410, acquiring at least one frame of point cloud data for calibration according to a preset scanning track.

S420, determining respective transformation matrixes when other point cloud data in the at least one frame of point cloud data for calibration are converted into a reference coordinate system of the reference point cloud data, wherein the other point cloud data are the point cloud data for calibration of other frames except the reference point cloud data in the at least one frame of point cloud data for calibration.

S430, determining the transformed optical center coordinates of the other point cloud data under the reference coordinate system according to the transformed optical center coordinates of the other point cloud data and the corresponding transformation matrixes.

S440, performing radius-constrained plane circle fitting on at least one converted optical center coordinate, and determining the plane circle obtained by fitting as an optical center fitting track.

S450, calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track.

S460, acquiring the rotation angle and the displacement of the tail end scanner of the acquisition equipment under any scanning pose; and determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters, the rotation angle and the displacement.

For example, the rotation angle and displacement of the end scanner of the acquisition device in any scanning pose may be obtained by a sensor of the end scanner of the acquisition device. The rotation angle may be the rotation angle at which the end scanner of the acquisition device moves from one scanning pose to the next. The displacement may be the distance that the end scanner of the acquisition device moves from one scanning pose to the next. The original point cloud data may be point cloud data acquired by the acquisition device under any scanning pose. According to the rotation angle and displacement of the end scanner of the acquisition equipment under any scanning pose and the mechanism parameters, a mosaic transformation matrix of the original point cloud data under any scanning pose relative to a reference coordinate system can be obtained, wherein the mosaic transformation matrix can be specifically: a transformation matrix from the axis coordinates of the azimuth rotation shaft to the origin (the center of sphere in fig. 8), a rotation matrix from the azimuth rotation shaft, a transformation matrix from the axis coordinates of the pitch rotation shaft to the axis coordinates of the azimuth rotation shaft, a transformation matrix from the rotation matrix of the pitch rotation shaft and the center coordinates of any initial point cloud data to the center coordinates of the pitch rotation shaft of the acquisition device. The method comprises the steps that a splicing transformation matrix of original point cloud data under any scanning pose relative to a reference coordinate system is obtained, and a target transformation matrix of conversion of the original point cloud data acquired by acquisition equipment under any scanning pose to the reference coordinate system is obtained based on the following formula:

Wherein RT _PanCent2L As a transformation matrix from the axis coordinates of the azimuth rotation axis to the origin (the center of sphere in fig. 8),

as a rotation matrix of azimuth rotation axis, RT _{TilCent2PanCent} R is a transformation matrix from the axis coordinate of the pitching rotation shaft to the axis coordinate of the azimuth rotation shaft _θ For a rotation matrix of pitching rotation axes, RT _L2TilCent The transformation matrix from the optical center coordinate of any initial point cloud data to the axis coordinate of the pitching rotating shaft.

As shown in fig. 7, the two rotation axes of the acquisition device are divided into a pair of rotation axes, the acquisition device in any embodiment of the present invention is an example of a dual-axis mechanism, the motion track of the mechanism is a superposition of two circumferences, at least three points can determine the direction of a circle and a circle mandrel, a point cloud can be shot every 120 ° of rotation, the 360 ° omnidirectional point cloud is spliced through visual features, that is, the point cloud is unified to the base coordinate system of the reference image, the zero coordinates of three frames are equal to the optical center coordinates in the base coordinate system, and the track curve of azimuth rotation can be obtained by the 3 point coordinates. Meanwhile, the biaxial mechanism can obtain the circumferential track of the two shafts fitted by at least 3 optical centers of each shaft and 6 space coordinates, and can obtain the pose transformation matrix of pitching and rotating at any angle, so that the calibration of the three-dimensional camera acquisition equipment mechanism is simplified to a great extent, and the three-dimensional reconstruction of the room can be completed as long as the biaxial motor has higher repeated positioning precision.

According to the mechanism parameters and the rotation angles and the displacement of the tail end scanner of the acquisition equipment under any scanning pose, a target transformation matrix for converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system is determined, so that the original point cloud data under any scanning pose is spliced based on the target transformation matrix for three-dimensional reconstruction of the area to be reconstructed.

And S470, converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the target transformation matrix to obtain spliced target point cloud data.

For example, the target point cloud data may be formed by converting original point cloud data acquired by the acquisition device under any scanning pose into point cloud data under a reference coordinate system based on a target transformation matrix. After the target transformation matrix is determined, the initial point cloud data under any scanning pose can be converted into the reference coordinate system of the reference point cloud data, so that the initial point cloud data under any scanning pose is unified into the reference coordinate system of the reference point cloud data, the effect of high-precision splicing of the initial point cloud data under any scanning pose can be achieved, and further high-precision three-dimensional reconstruction of the region to be reconstructed is achieved.

It should be noted that after unifying the initial point cloud data under any scanning pose to the reference coordinate system of the reference point cloud data, coarse matching of the initial point cloud data under any scanning pose is achieved, but there may be a case that the initial point cloud data of a pole is not matched with other point cloud data under the reference coordinate, so that the ICP algorithm in the second embodiment of the present invention may be adopted to perform fine matching to obtain more accurate and complete point cloud data of the area to be reconstructed.

According to the technical scheme, the rotation angle and the displacement of the tail end scanner of the acquisition equipment under any scanning pose are obtained; and determining a target transformation matrix for converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters, the rotation angle and the displacement, so that the original point cloud data under any scanning pose is spliced based on the target transformation matrix to reconstruct the three-dimensional region to be reconstructed. According to the target transformation matrix, original point cloud data acquired by the acquisition equipment under any scanning pose are converted into the reference coordinate system to obtain spliced target point cloud data, so that the original point cloud data under any scanning pose are unified to the reference coordinate system of the reference point cloud data, the effect of high-precision splicing of the original point cloud data under any scanning pose can be achieved, and further high-precision three-dimensional reconstruction of an area to be reconstructed is achieved.

Example five

Fig. 10 is a schematic structural diagram of a point cloud splicing device according to a fifth embodiment of the present invention, as shown in fig. 10, where the device includes: the system comprises a point cloud data acquisition module 31 for calibration, an optical center coordinate determination module 32, a mechanism parameter calibration module 33, a target transformation matrix determination module 34 and a point cloud data splicing module 35.

The calibration point cloud data acquisition module 31 is configured to acquire at least one frame of calibration point cloud data according to a preset scanning track;

a photo-center coordinate determining module 32, configured to determine, based on the at least one frame of point cloud data for calibration, post-conversion photo-center coordinates of pre-conversion photo-center coordinates of each of the at least one frame of point cloud data for calibration in a reference coordinate system of the reference point cloud data, where the reference point cloud data is any one frame of the at least one frame of point cloud data for calibration;

the mechanism parameter determining module 33 is configured to calibrate a mechanism parameter of the collecting device according to a light center fitting track of the converted light center coordinates, where the light center fitting track is a track that is synthesized according to the converted light center coordinates;

the target transformation matrix determining module 34 is configured to determine, according to the mechanism parameter, a target transformation matrix for converting original point cloud data acquired by the acquisition device under any scanning pose into the reference coordinate system;

And the point cloud data stitching module 35 is configured to stitch the original point cloud data according to the target transformation matrix to obtain a stitched target point cloud.

On the basis of the technical scheme of the embodiment of the invention, the calibration point cloud data acquisition module 31 is specifically used for:

Optionally, the feature target is disposed in a scan overlapping area of the area to be reconstructed, where the scan overlapping area is determined according to a preset scan trajectory and a view angle of the acquisition device.

On the basis of the technical solution of the embodiment of the present invention, the optical center coordinate determining module 32 includes:

a transformation matrix determining unit configured to determine respective transformation matrices when other point cloud data in the at least one frame of calibration point cloud data is converted into a reference coordinate system of the reference point cloud data, where the other point cloud data is calibration point cloud data of other frames than the reference point cloud data in the at least one frame of calibration point cloud data;

And the optical center coordinate determining unit is used for determining the converted optical center coordinates of the other point cloud data under the reference coordinate system according to the converted optical center coordinates of the other point cloud data and the corresponding respective transformation matrixes.

Optionally, the optical center coordinates of the reference point cloud data and the converted optical center coordinates of the other point cloud data form converted optical center coordinates of the at least one frame of calibration point cloud data.

On the basis of the technical scheme of the embodiment of the invention, the transformation matrix determining unit comprises:

the matching feature pair determining subunit is used for extracting features of the datum point cloud data and any one of the other point cloud data and determining matching feature pairs for splicing;

a conversion parameter determining subunit, configured to determine, according to the matching feature pair, respective conversion parameters when any one of the other point cloud data is converted to the reference coordinate system;

and the transformation matrix determining subunit is used for determining respective transformation matrixes when any one of the other point cloud data is transformed to the reference coordinate system according to the respective transformation parameters.

On the basis of the technical scheme of the embodiment of the invention, the mechanism parameter calibration module 33 comprises:

the optical center fitting track fitting unit is used for performing radius-constrained plane circle fitting on at least one of the converted optical center coordinates, and determining that a plane circle obtained by fitting is an optical center fitting track;

and the mechanism parameter calibration unit is used for calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track.

On the basis of the technical scheme of the embodiment of the invention, the optical center fitting track fitting unit comprises:

a projection coordinate point determining subunit, configured to determine a projection coordinate point of each converted optical center coordinate on a fitting surface according to at least one converted optical center coordinate and a normal vector of the fitting surface of at least one converted optical center coordinate, where the fitting surface is a plane that is synthesized according to the converted optical center coordinate, and the projection coordinate point is a nearest neighbor coordinate point of the converted optical center coordinate projected onto the fitting surface;

and the optical center fitting track fitting subunit is used for carrying out radius-constrained plane circle fitting on the plurality of projection coordinate points, and determining the plane circle obtained by fitting as the optical center fitting track.

Optionally, the mechanism parameters include: the axial length of each level of rotating shafts, the axial center coordinates of each level of rotating shafts and the included angles of each level of rotating shafts in the acquisition equipment.

On the basis of the technical scheme of the embodiment of the invention, the mechanism parameter calibration unit comprises:

a current rotation axis mechanism parameter determining subunit, configured to determine that a radius of a fitted plane circle corresponding to the current optical center fitting track is an axial length of a current-stage rotation axis of the collection device, and a center of the fitted plane circle is an axial coordinate of the current-stage rotation axis;

and the next-stage rotating shaft mechanism parameter determining subunit is used for performing radius-constrained planar circle fitting on the axis coordinates of at least one current-stage rotating shaft, determining that the radius of the planar circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axial length of the next-stage rotating shaft, and determining that the circle center of the planar circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axis coordinates of the next-stage rotating shaft.

And the included angle determining subunit of each level of rotation shafts is used for determining included angles formed between each level of rotation shafts according to the normal vector of the fitted plane circle of each level of rotation shafts.

Optionally, in the collecting device, the next-stage rotation axis is located inside the current rotation axis.

On the basis of the technical solution of the embodiment of the present invention, the target transformation matrix determining module 34 includes:

a parameter acquisition unit for acquiring rotation angle and displacement of an end scanner of the acquisition equipment under any scanning pose;

and the target transformation matrix determining unit is used for determining a target transformation matrix for converting the original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters, the rotation angle and the displacement.

On the basis of the technical scheme of the embodiment of the invention, the point cloud data splicing module 35 is specifically configured to:

The point cloud splicing device provided by the embodiment of the invention can execute the point cloud splicing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

Example six

Fig. 11 is a schematic structural diagram of an apparatus according to a sixth embodiment of the present invention, and as shown in fig. 11, the apparatus includes a processor 40, a memory 41, an input device 42 and an output device 43; the number of processors 40 in the device may be one or more, one processor 40 being taken as an example in fig. 11; the processor 40, the memory 41, the input means 42 and the output means 43 in the device may be connected by a bus or other means, in fig. 11 by way of example.

The memory 41 is used as a computer readable storage medium, and may be used to store software programs, computer executable programs, and modules, such as program instructions/modules (e.g., the calibration point cloud data acquisition module 31, the optical center coordinate determination module 32, the mechanism parameter calibration module 33, the target transformation matrix determination module 34, and the point cloud data stitching module 35) corresponding to the point cloud stitching method in the embodiment of the present invention. The processor 40 executes various functional applications of the device and data processing, i.e. implements the above-described point cloud stitching method, by running software programs, instructions and modules stored in the memory 41.

The memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for functions; the storage data area may store data created according to the use of the terminal, etc. In addition, memory 41 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 41 may further include memory located remotely from processor 40, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input means 42 may be used to receive entered numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output means 43 may comprise a display device such as a display screen.

Example seven

The seventh embodiment of the present invention also provides a storage medium containing computer-executable instructions for performing a point cloud stitching method when executed by a computer processor.

Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above-described method operations, and may also perform the related operations in the point cloud stitching method provided in any embodiment of the present invention.

From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.

It should be noted that, in the embodiment of the point cloud splicing apparatus, each unit and module included are only divided according to the functional logic, but not limited to the above division, so long as the corresponding function can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.

Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims

1. The point cloud splicing method is characterized by comprising the following steps of:

splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud data;

the step of collecting at least one frame of point cloud data for calibration by a preset scanning track comprises the following steps:

based on a preset scanning track, carrying out point cloud scanning on a region to be rebuilt by acquisition equipment, and acquiring at least one frame of point cloud data for calibration, wherein different characteristic targets are arranged on the region to be rebuilt;

the determining the transformed optical center coordinates of the at least one frame of calibration point cloud data under the reference coordinate system of the reference point cloud data includes:

determining respective transformation matrices when other point cloud data in the at least one frame of point cloud data for calibration is converted into a reference coordinate system of the reference point cloud data, wherein the other point cloud data is the point cloud data for calibration of other frames except the reference point cloud data in the at least one frame of point cloud data for calibration;

Determining the transformed optical center coordinates of the other point cloud data under the reference coordinate system according to the transformed optical center coordinates of the other point cloud data and the corresponding transformation matrixes;

the optical center coordinates of the datum point cloud data and the converted optical center coordinates of the other point cloud data form converted optical center coordinates of the at least one frame of calibration point cloud data.

2. The method of claim 1, wherein each of the signature targets is provided with a unique coded identifier, such that the obtained calibration point cloud data can be associated with each of the signature targets by the unique coded identifier.

3. The method of claim 1, wherein the signature target is disposed in a scan overlap region of the region to be reconstructed, wherein the scan overlap region is determined from a preset scan trajectory and field angle of the acquisition device.

4. The method of claim 1, wherein determining the respective transformation matrices when converting other point cloud data in the at least one frame of calibration point cloud data to the reference coordinate system of the reference point cloud data comprises:

Extracting features of the datum point cloud data and any one of the other point cloud data, and determining matching feature pairs for splicing;

determining respective conversion parameters when any one of the other point cloud data is converted to the reference coordinate system according to the matching feature pairs;

and determining respective transformation matrixes when any one of the other point cloud data is transformed to the reference coordinate system according to the respective transformation parameters.

5. The method of claim 4, wherein the conversion parameters include an angle of rotation and an amount of translation.

6. The method of claim 1, wherein calibrating the mechanical parameters of the acquisition device based on the centroid-fitted trajectory of the converted centroid coordinates comprises:

performing radius-constrained plane circle fitting on at least one converted optical center coordinate, and determining that a plane circle obtained by fitting is an optical center fitting track;

and calibrating the mechanism parameters of the acquisition equipment according to the optical center fitting track.

7. The method of claim 6, wherein said fitting a plane circle with radius constraints to at least one of said transformed optical center coordinates, determining the fitted plane circle as an optical center fit trajectory, comprises:

Determining a projection coordinate point of each converted optical center coordinate on a fitting surface according to at least one converted optical center coordinate and a normal vector of the fitting surface of at least one converted optical center coordinate, wherein the fitting surface is a plane which is synthesized according to the converted optical center coordinate, and the projection coordinate point is a nearest neighbor coordinate point of the converted optical center coordinate projected onto the fitting surface;

and carrying out radius constraint plane circle fitting on the plurality of projection coordinate points, and determining the plane circle obtained by fitting as an optical center fitting track.

8. The method of claim 7, wherein the mechanism parameters comprise: the axial length of each level of rotating shafts of the acquisition equipment, the axial center coordinates of each level of rotating shafts and the included angles of each level of rotating shafts;

determining that the radius of a fitted plane circle corresponding to the current optical center fitting track is the axial length of a current-stage rotating shaft of the acquisition equipment, and the center of the fitted plane circle is the axial coordinate of the current-stage rotating shaft;

performing radius-constrained plane circle fitting on the axis coordinates of at least one current-stage rotating shaft, and determining that the radius of a plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axial length of a next-stage rotating shaft, wherein the circle center of the plane circle fitted by the axis coordinates of at least one current-stage rotating shaft is the axis coordinates of the next-stage rotating shaft;

Determining an included angle formed between the rotating shafts of all levels according to the normal vector of the fitted plane circle of the rotating shafts of all levels;

wherein, in the acquisition device, the next-stage rotation axis is located inside the current-stage rotation axis.

9. The method according to claim 1, wherein determining, according to the mechanism parameter, a target transformation matrix for transforming the original point cloud data acquired by the acquisition device in any scanning pose into the reference coordinate system includes:

acquiring the rotation angle and displacement of an end scanner of the acquisition equipment under any scanning pose;

and determining a target transformation matrix for converting original point cloud data acquired by the acquisition equipment under any scanning pose into the reference coordinate system according to the mechanism parameters, the rotation angle and the displacement.

10. The method of claim 1, wherein stitching the raw point cloud data according to the target transformation matrix to obtain stitched target point cloud data comprises:

11. A point cloud stitching device, comprising:

the optical center coordinate conversion module is used for determining converted optical center coordinates of the optical center coordinates before conversion of the at least one frame of point cloud data under a reference coordinate system of reference point cloud data based on the at least one frame of point cloud data for calibration, wherein the reference point cloud data is any frame of the at least one frame of point cloud data for calibration;

the point cloud data splicing module is used for splicing the original point cloud data according to the target transformation matrix to obtain spliced target point cloud;

The point cloud data acquisition module for calibration is specifically used for carrying out point cloud scanning on a region to be rebuilt by acquisition equipment based on a preset scanning track, and acquiring at least one frame of point cloud data for calibration, wherein different characteristic targets are arranged on the region to be rebuilt;

the optical center coordinate determining module includes:

the optical center coordinate determining unit is used for determining the converted optical center coordinates of the other point cloud data under the reference coordinate system according to the converted optical center coordinates of the other point cloud data and the corresponding respective transformation matrixes;

12. An apparatus, the apparatus comprising:

one or more processors;

a storage means for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the point cloud stitching method of any of claims 1-10.

13. A storage medium containing computer executable instructions which, when executed by a computer processor, are for performing the point cloud stitching method of any of claims 1-10.