CN118429550A - Three-dimensional reconstruction method, system, electronic equipment and storage medium - Google Patents

Abstract

The application provides a three-dimensional reconstruction method, a system, electronic equipment and a storage medium, wherein the three-dimensional reconstruction method comprises the following steps: acquiring first point cloud data obtained by scanning an object to be scanned by a scanning module with the farthest working distance; wherein the first point cloud data comprises a plurality of first three-dimensional points; determining characteristic parameters of the first point cloud data; determining a target area on an object to be scanned according to the characteristic parameters; determining a target scanning module from a plurality of scanning modules according to the characteristic parameters in the target area; acquiring second point cloud data obtained by scanning a target area by a target scanning module; wherein the second point cloud data comprises a plurality of second three-dimensional points; determining the weight of each first three-dimensional point and each second three-dimensional point; and fusing the plurality of first three-dimensional points and the plurality of second three-dimensional points corresponding to the at least one target area according to the weights to obtain three-dimensional reconstruction data of the object to be scanned. The application relates to the technical field of three-dimensional reconstruction, which can improve the accuracy of three-dimensional reconstruction of an object to be scanned.

Description

Three-dimensional reconstruction method, system, electronic device and storage medium

Technical Field

The present application relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional reconstruction method, system, electronic device and storage medium.

Background technique

At present, when implementing cross-scale scanning of objects in large-scale scenes, multiple scanning devices are usually relied on to scan the scanned objects to ensure the integrity of the point cloud data. For example, a handheld structured light scanning module is usually used to ensure the integrity of the global data, and at the same time, high-precision fixed equipment is relied on to obtain high-precision detail data. Then, in the point cloud reconstruction process, different point cloud data from multiple devices are used for splicing processing, and the point cloud weights are relied on to merge the point clouds in the same area. However, this method is prone to obvious excessive traces and mesh connection breakage risks after point cloud data reconstruction, resulting in low accuracy of 3D reconstruction data. In addition, it relies on the back-end processing of mesh data, and fixed equipment is difficult to scan large targets, which is time-inefficient.

Summary of the invention

In view of the above, it is necessary to propose a three-dimensional reconstruction method, system, electronic device and storage medium to solve the technical problems of low accuracy and scanning efficiency of three-dimensional reconstruction data.

The present application provides a three-dimensional reconstruction method, which is applied to an electronic device, wherein the electronic device is communicatively connected to a plurality of scanning modules, the plurality of scanning modules are used to scan an object to be scanned, and each scanning module has a different working distance range. The method comprises: obtaining first point cloud data obtained by scanning the object to be scanned by a scanning module with the longest working distance; wherein the first point cloud data comprises a plurality of first three-dimensional points; determining characteristic parameters of the first point cloud data; determining a target area on the object to be scanned according to the characteristic parameters; determining a target scanning module from the plurality of scanning modules according to the characteristic parameters in the target area; obtaining second point cloud data obtained by the target scanning module scanning the target area; wherein the second point cloud data comprises a plurality of second three-dimensional points; determining the weights of each first three-dimensional point and each second three-dimensional point; and fusing the plurality of first three-dimensional points and the plurality of second three-dimensional points corresponding to at least one of the target areas according to the weights to obtain three-dimensional reconstruction data of the object to be scanned.

In some embodiments, the feature parameters include: the category of the object to be scanned; and curvature data of the first point cloud data.

In some embodiments, determining a target scanning module from the multiple scanning modules based on the characteristic parameters in the target area includes: traversing the category of each scanning module; when the category of the traversed scanning module is the same as the category of the object to be scanned, determining the scanning module corresponding to the category as the target scanning module.

In some embodiments, determining the target scanning module from the multiple scanning modules based on the characteristic parameters in the target area includes: determining the physical resolution required for scanning the target area based on the curvature data; traversing the point cloud physical resolution corresponding to each scanning module to determine the difference between the point cloud physical resolution and the physical resolution required for the target area; and determining, from the scanning modules whose point cloud physical resolution is greater than the physical resolution required for the target area, the scanning module corresponding to the smallest difference as the target scanning module.

In some embodiments, the fusion resolution in the electronic device is set according to the point cloud physical resolution of the target scanning module, and the fusion resolution is used to characterize the resolution of the three-dimensional reconstruction data of the target area.

In some embodiments, the number of light rays emitted by the target scanning module when scanning the object to be scanned is determined based on the distance between the target scanning module and the object to be scanned.

In some embodiments, the weight of each first three-dimensional point is determined according to the position of the light to which each first three-dimensional point belongs in the corresponding scanning module; the weight of each second three-dimensional point is determined according to the position of the light to which each second three-dimensional point belongs in the target scanning module; and/or the weight of each first three-dimensional point is determined according to the distance between the scanning module corresponding to each first three-dimensional point and the object to be scanned; the weight of each second three-dimensional point is determined according to the distance between the target scanning module and the object to be scanned; and/or the weight of each first three-dimensional point is determined according to the incident angle of the light to which each first three-dimensional point belongs irradiating the object to be scanned; the weight of each second three-dimensional point is determined according to the incident angle of the light to which each second three-dimensional point belongs irradiating the object to be scanned.

An embodiment of the present application also provides a three-dimensional reconstruction system, which includes: multiple scanning modules and an electronic device; the multiple scanning modules are used to scan an object to be scanned, and each scanning module has a different distance from the object to be scanned during scanning; the electronic device is used to obtain first point cloud data collected by the scanning module corresponding to the largest distance; determine characteristic parameters of the first point cloud data; determine a target scanning module from the multiple scanning modules based on the characteristic parameters; obtain second point cloud data obtained by the target scanning module scanning the object to be scanned; and fuse the first point cloud data and the second point cloud data to obtain three-dimensional reconstruction data of the object to be scanned.

An embodiment of the present application further provides an electronic device, comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the three-dimensional reconstruction method.

An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the three-dimensional reconstruction method.

It can be seen from the above technical solutions that the embodiment of the present application deploys multiple scanning modules with different distances in the scanning system, and preferentially uses the scanning module with the longest distance to obtain the first point cloud data during the real-time scanning process, and then determines the characteristic parameters of the object to be scanned according to the first point cloud data, and determines the physical resolution required for the object to be scanned according to the characteristic parameters, and finally adjusts the adaptive scanning module to perform three-dimensional reconstruction of the object to be scanned. In this way, the accuracy of analyzing the features of different structures and shapes can be improved on the basis of ensuring the integrity of the point cloud data, thereby improving the accuracy of the three-dimensional reconstruction data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a three-dimensional reconstruction system provided in an embodiment of the present application.

FIG. 2 is a flow chart of a three-dimensional reconstruction method provided in an embodiment of the present application.

FIG. 3 is a flow chart of a method for determining a target scanning module provided in an embodiment of the present application.

FIG. 4 is a flow chart of a method for determining a target scanning module provided in another embodiment of the present application.

FIG. 5 is a flowchart of a method for determining weights of a first three-dimensional point and a second three-dimensional point provided by an embodiment of the present application.

FIG. 6 is a flowchart of a method for determining weights of a first three-dimensional point and a second three-dimensional point provided by another embodiment of the present application.

FIG. 7 is a flowchart of a method for determining weights of a first three-dimensional point and a second three-dimensional point provided by yet another embodiment of the present application.

FIG8 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

Detailed ways

In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and/or" used herein includes any and all combinations of one or more related listed items.

The embodiment of the present application provides a three-dimensional reconstruction method, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA), a digital processor (Digital Signal Processor, DSP), an embedded device, etc.

An electronic device can be any electronic product that can interact with customers, such as personal computers, tablet computers, smart phones, personal digital assistants (PDAs), game consoles, Internet Protocol Television (IPTV), smart wearable devices, etc.

The electronic device may also include a network device and/or a client device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

The networks where electronic devices are located include but are not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

As shown in FIG1 , a three-dimensional reconstruction system provided by the present application includes an electronic device 100 and a plurality of scanning modules, and the electronic device 100 is communicatively connected to a database 200. The electronic device is also communicatively connected to a plurality of scanning modules, such as the first scanning module 300, the second scanning module 400 and the third scanning module 500 shown in FIG1 . Among them, the plurality of scanning modules are used to scan an object to be scanned 600 so as to perform three-dimensional reconstruction of the object to be scanned 600. Among them, the distance between each scanning module and the object to be scanned 600 is different during scanning. The database 200 can be a data storage device built into the electronic device 100, or it can be an external data storage device communicatively connected to the electronic device 100, and the present application does not limit this. The database 200 is used to store point cloud data obtained by scanning the object to be scanned 600 by a plurality of scanning modules.

In one embodiment of the present application, the electronic device 100 obtains the first point cloud data obtained by scanning the object to be scanned by the scanning module corresponding to the largest distance from the database 200, wherein the first point cloud data includes a plurality of first three-dimensional points. The scanning module corresponding to the largest distance may be the scanning module 300 shown in FIG. 1. The electronic device 100 is also used to determine the characteristic parameters of the first point cloud data, and determine the target scanning module from the plurality of scanning modules according to the characteristic parameters, wherein the target scanning module may be the second scanning module 400 or the third scanning module 500 shown in FIG. 1. The electronic device 100 is also used to obtain the second point cloud data obtained by scanning the object to be scanned 600 by the target scanning module (for example, the second scanning module 400 or the third scanning module 500 shown in FIG. 1), wherein the second point cloud data includes a plurality of second three-dimensional points. The electronic device 100 is also used to determine the weight of each first three-dimensional point and each second three-dimensional point, and fuse the plurality of first three-dimensional points and the plurality of second three-dimensional points according to the weight to obtain the three-dimensional reconstruction data of the object to be scanned 600, wherein the three-dimensional reconstruction data is used to characterize the structure and shape of the object to be scanned 600.

As shown in Figure 2, it is a flow chart of a 3D reconstruction method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. A 3D reconstruction method provided by an embodiment of the present application includes the following steps.

S20, obtaining first point cloud data obtained by scanning the object to be scanned by a scanning module with the longest working distance; wherein the first point cloud data includes a plurality of first three-dimensional points.

In an embodiment of the present application, the object to be scanned may be any object whose geometric shape needs to be measured, such as a car, a medical device, or a product part, and the present application does not limit the properties and shape of the object to be scanned. Before measuring the object to be scanned, a small number of object markers may be attached to the object to be scanned, and the object markers may be reflective markers or reflective textures.

In one embodiment of the present application, a camera and a light source transmitter are provided in the scanning module. The light source transmitter is used to project structured light onto the surface of the object to be scanned; the camera is used to capture the structured light reflected by the surface of the object to be scanned, and determine the point cloud data corresponding to the object to be scanned based on the reflected light. In order to ensure the integrity of the point cloud data, the object to be scanned can be scanned based on multiple scanning modules, wherein each scanning module has a different working distance range and requires a different distance from the object to be scanned when used. In this way, point cloud data of different depths and angles of the object to be scanned can be obtained. The present application does not limit the specific scanning posture of the scanning module and the position of each scanning module;

The light source may be an optical device, a DLP projector, an LED, etc., and the projected structured light may be a single light ray, multiple parallel light rays, multiple groups of intersecting parallel lines, speckle, a grating, etc.

In one embodiment of the present application, the first point cloud data may be a collection of point cloud data obtained when the scanning module with the longest working distance scans the object to be scanned. The first point cloud data includes a plurality of first three-dimensional points, each of which corresponds to a three-dimensional coordinate. Exemplarily, the scanning module with the longest distance from the object to be scanned may be the scanning module 300 shown in FIG. 1 .

In one embodiment of the present application, the three-dimensional coordinates of the first three-dimensional point are used to represent the position of the first three-dimensional point in a preset coordinate system. The coordinate origin of the preset coordinate system may be the position of the scanning module in the target space where the object to be scanned is located, or may be a pre-set point in the target space, which is not limited in the present application.

S21: Determine characteristic parameters of the first point cloud data.

In one embodiment of the present application, the characteristic parameters of the first point cloud data are used to describe the local attributes of each first three-dimensional point. The characteristic parameters include: the category of the object to be scanned; and the curvature data of the first point cloud data. The category of the object to be scanned is used to characterize the structure and shape of the object to be scanned. Exemplarily, when the category of the object to be scanned is a sphere, it indicates that the surface structure of the object to be scanned is a spherical structure; when the category of the object to be scanned is a cube, it indicates that the surface structure of the object to be scanned is a cube structure. The present application does not limit the specific category of the object to be scanned. The curvature data of the first point cloud data is used to characterize the roughness of the surface of the object to be scanned or the complexity of the surface shape. When the curvature data is larger, it indicates that the surface roughness or the complexity of the surface shape of the object to be scanned is higher; when the curvature data is smaller, it indicates that the surface roughness or the complexity of the surface shape of the object to be scanned is lower.

In some embodiments, the characteristic parameters of the first point cloud data also include: a normal vector, which is used to characterize the normal direction of the surface where each first three-dimensional point is located; surface roughness, which is used to characterize the smoothness or roughness of the point cloud surface and can be used to analyze the texture of the object surface; color value, intensity value, incident direction of light, etc. of the first point cloud data.

S22, determining a target area on the object to be scanned according to the characteristic parameters, and determining a target scanning module from the multiple scanning modules according to the characteristic parameters in the target area.

In one embodiment of the present application, in order to perform three-dimensional reconstruction of the scanned object based on point cloud data obtained by scanning the scanned object by multiple scanning modules, thereby improving the accuracy of three-dimensional reconstruction, a target scanning module can be determined from multiple scanning modules based on characteristic parameters of the first point cloud data.

In one embodiment of the present application, during scanning, in order to improve the accuracy of three-dimensional reconstruction of a region with a relatively complex structure and shape on the object to be scanned, a target region may be first determined on the object to be scanned based on a characteristic parameter. Exemplarily, the characteristic parameter may be curvature data of the first point cloud data. When the curvature data of the point cloud data corresponding to any region on the object to be scanned is large, it indicates that the structure and shape of the region on the object to be scanned are relatively complex, and the region may be determined to be a target region.

In one embodiment of the present application, each scanning module is suitable for scanning objects to be scanned at different distances, structures and shapes. For example, the light emitted by some scanning modules can be used to scan objects to be scanned with a spherical structure; the light emitted by other scanning modules can be used to scan objects to be scanned with a cubic structure. The present application does not limit the types of objects to be scanned to which the scanning modules are applicable. Specifically, when the characteristic parameters include the category of the object to be scanned, the method for determining the target scanning module can refer to the detailed description corresponding to FIG. 3.

In one embodiment of the present application, the curvature data of the first point cloud data is used to characterize the local geometric shape of the surface of the first point cloud data, and can be used to describe the curvature or curvature change of the first three-dimensional point. When the curvature data is larger, it indicates that the surface roughness or the complexity of the surface shape of the object to be scanned is higher, and the object to be scanned can be scanned according to a scanning module with a higher physical resolution to improve the accuracy of the three-dimensional reconstruction of the object to be scanned; when the curvature data is smaller, it indicates that the surface roughness or the complexity of the surface shape of the object to be scanned is lower, and the object to be scanned can be scanned according to a scanning module with a lower physical resolution, thereby improving the efficiency of the three-dimensional reconstruction of the object to be scanned on the basis of ensuring the accuracy of the three-dimensional reconstruction. Specifically, when the characteristic parameters include the curvature data of the first point cloud data, the method for determining the target scanning module can refer to the detailed description corresponding to Figure 4.

In one embodiment of the present application, in order to ensure the accuracy of three-dimensional reconstruction of the scanned object and improve the efficiency of three-dimensional reconstruction, the method further includes: setting the fusion resolution in the electronic device according to the point cloud physical resolution of the target scanning module, and the fusion resolution is used to characterize the resolution of the three-dimensional reconstruction data of the target area. Among them, the point cloud physical resolution of the scanning module may be the optical resolution of the scanning module, which is used to measure the precision of the photosensitive device in the scanning module. Specifically, the point cloud physical resolution may be the actual number of light points that can be captured per square inch of area by the optical components of the scanning module, usually expressed in dpi (dots per inch), and the point cloud physical resolution can reflect the ability of the scanning module to capture the details of the point cloud data. The higher the point cloud physical resolution, the richer the details of the point cloud data captured by the scanning module.

In one embodiment of the present application, in order to improve the accuracy of three-dimensional reconstruction of the object to be scanned, the target space where the object to be scanned is located can be divided into multiple voxels to provide data support for the voxelization of point cloud data. The point cloud data can be subsequently divided based on multiple voxels to obtain the point cloud data corresponding to each voxel, and the point cloud data corresponding to each voxel can be fused to improve the accuracy of three-dimensional reconstruction. Among them, the number of voxels in the target space. Exemplarily, the size of the voxel can be 1mm*1mm*1mm, or it can be 0.3mm*0.3mm*0.3mm, which is not limited in the present application.

In one embodiment of the present application, during the three-dimensional reconstruction of the object to be scanned, if the voxel size is too high, the target space will be segmented in insufficient granularity, resulting in information loss; if the voxel size is too low, the number of voxels in the target space will be too large, which will lead to the problem that the three-dimensional reconstruction takes a long time and the computing resource occupancy rate of the electronic device is high. In order to avoid the above problems, the fusion resolution in the electronic device can be set according to the physical resolution of the point cloud of the target scanning module, and the fusion resolution is used to characterize the resolution of the three-dimensional reconstruction data corresponding to the object to be scanned.

In one embodiment of the present application, taking a laser source as an example; when the camera in the scanning module performs light alignment, since the light projected by the scanning module is dense and has a consistent geometric shape, alignment errors may occur, resulting in a high error in the point cloud data. In order to improve the accuracy of the acquired point cloud data, the number of light rays emitted by the scanning module can be reduced. Specifically, the scanning module emits multiple light rays to the object to be scanned during the three-dimensional reconstruction process, wherein the multiple light rays include a central light ray and the remaining light rays that are distributed in parallel and dispersed around the central light ray. Since the remaining projected light planes are severely curved, the point cloud data generated by the remaining light rays will show a large deformation, and therefore the accuracy of the point cloud data generated by the remaining light rays will gradually decrease.

In one embodiment of the present application, during the three-dimensional reconstruction of the object to be scanned, the method further includes: determining the number of light rays emitted by the target scanning module when scanning the object to be scanned based on the distance between the target scanning module and the object to be scanned. Specifically, when the object to be scanned is scanned using the target scanning module, at least one pair of light rays in the scanning module can be cancelled, wherein the number of at least one light ray can be determined according to the distance between the scanning module and the object to be scanned, the farther the distance is, the fewer the number of light rays to be cancelled, and the closer the distance is, the more the number of light rays to be cancelled. Exemplarily, when the distance between the scanning module farthest from the object to be scanned (for example, the scanning module 300 shown in FIG. 1) and the object to be scanned (for example, the object to be scanned 600 shown in FIG. 1) is 10 cm, if the target scanning module (for example, the object to be scanned shown in FIG. 1) emits 7 light rays to scan the object to be scanned, at least one pair of light rays located on the outside of the 7 light rays can be cancelled. This can reduce the number of rays, thereby reducing the complexity of point cloud data, and thus improving the efficiency of point cloud data processing; it can also reduce the number of outer rays, thereby reducing the point cloud data corresponding to the edge of the object to be scanned, and thus reducing the deformation of the point cloud data, thereby improving the accuracy of point cloud data processing.

S23, acquiring second point cloud data obtained by the target scanning module scanning the target area; wherein the second point cloud data includes a plurality of second three-dimensional points.

In one embodiment of the present application, the second point cloud data may be a collection of point cloud data obtained when the target scanning module scans the object to be scanned. The second point cloud data includes a plurality of second three-dimensional points, each of which corresponds to a three-dimensional coordinate. Exemplarily, the target scanning module may be the scanning module 400 or the scanning module 500 as shown in FIG. 1 .

In one embodiment of the present application, the three-dimensional coordinates of the second three-dimensional point are used to represent the position of the second three-dimensional point in a preset coordinate system. The coordinate origin of the preset coordinate system may be the position of the scanning module in the target space where the object to be scanned is located, or may be a pre-set point in the target space, which is not limited in the present application.

S24, determining the weight of each first three-dimensional point and each second three-dimensional point.

In one embodiment of the present application, in order to improve the accuracy of three-dimensional reconstruction of the object to be scanned, the weights of each first three-dimensional point and each second three-dimensional point can be determined in combination with the distances between multiple scanning modules and the object to be scanned, the positions of the light rays that irradiate to form the first three-dimensional points and the second three-dimensional points in the corresponding scanning modules, and the incident angles of the light rays when irradiating the modules to be scanned.

In one embodiment of the present application, the incident angle of the light corresponding to the first three-dimensional point when irradiating the module to be scanned can be determined according to the three-dimensional coordinates of the first three-dimensional point in the preset coordinate system and the three-dimensional coordinates of the scanning module in the preset coordinate system; and the incident angle of the light corresponding to the second three-dimensional point when irradiating the module to be scanned can be determined according to the three-dimensional coordinates of the second three-dimensional point in the preset coordinate system and the three-dimensional coordinates of the scanning module in the preset coordinate system. Specifically, the coordinate line between the first three-dimensional point and the scanning module can be determined, and the normal vector of the first point cloud data can be determined, and the angle between the coordinate line and the normal vector of the first point cloud data can be calculated to obtain the incident angle of the light corresponding to the first three-dimensional point when irradiating the module to be scanned; the coordinate line between the second three-dimensional point and the scanning module can be determined, and the normal vector of the second point cloud data can be determined, and the angle between the coordinate line and the normal vector of the second point cloud data can be calculated to obtain the incident angle of the light corresponding to the second three-dimensional point when irradiating the module to be scanned.

In one embodiment of the present application, for a method of determining the weights of the first three-dimensional point and the second three-dimensional point, please refer to the detailed description corresponding to FIG. 5 , and/or the detailed description corresponding to FIG. 6 , and/or the detailed description corresponding to FIG. 7 .

S25, fusing the plurality of first three-dimensional points and the plurality of second three-dimensional points corresponding to at least one of the target regions according to the weights to obtain three-dimensional reconstruction data of the object to be scanned.

In one embodiment of the present application, in order to fit the approximate plane of the surface of the object to be scanned in each voxel according to the point cloud data in each voxel, the first three-dimensional point in each voxel and the second three-dimensional point corresponding to at least one target area can be fitted based on the least squares method to obtain the target point and the target normal vector corresponding to each voxel, and then the approximate plane of the surface of the object to be scanned in each voxel can be determined based on the target point and the target normal vector. Wherein, when fitting the first three-dimensional point in each voxel and the second three-dimensional point corresponding to at least one target area based on the least squares method, the confidence of the first three-dimensional point and the second three-dimensional point can be adjusted during the fitting process according to the weights of the first three-dimensional point and the second three-dimensional point. In this way, the accuracy of the three-dimensional reconstruction data of the object to be scanned can be improved.

In one embodiment of the present application, after obtaining the target point and target normal vector corresponding to each voxel, the approximate plane of the object to be scanned in the voxel can be determined based on the target point and the corresponding target normal vector, and then the approximate plane of each voxel can be fused to obtain the three-dimensional reconstruction data of the object to be scanned.

Exemplarily, when there are curves C1 and C2 on the surface of the object to be scanned, and curves C1 and C2 intersect at a three-dimensional point P, if the tangent vector of curve C1 at the three-dimensional point P is T1, and the tangent vector of curve C2 at the three-dimensional point P is T2, then both tangent vector T1 and tangent vector T2 are on the tangent plane of the surface at the three-dimensional point P. The normal vector of the three-dimensional point P can be determined based on vectors T1 and T2, and the direction of the tangent plane can be determined by the three-dimensional coordinates of the three-dimensional point P and the normal vector, and the surface of the object to be scanned at the three-dimensional point P can be characterized based on the tangent plane. In order to obtain the approximate plane corresponding to the plane of the object to be scanned, the local approximate plane of each voxel can be determined according to the above method, and the plane of the object to be scanned can be constructed based on the local approximate plane of each voxel.

In one embodiment of the present application, when the three-dimensional coordinates of the three-dimensional point P in the preset coordinate system are (x ₀ , y ₀ , z ₀ ) and the target normal vector n is (A, B, C), the plane equation of the three-dimensional point P can be determined according to the following relationship:

; Among them, (A, B, C) represents the normal vector of the plane where the three-dimensional point P is located, and (x ₀ , y ₀ , z ₀ ) represents the three-dimensional coordinates of the three-dimensional point P.

It can be seen from the above technical solutions that the embodiment of the present application deploys multiple scanning modules with different distances in the scanning system, and preferentially uses the scanning module with the longest distance to obtain the first point cloud data during the real-time scanning process, and then determines the characteristic parameters of the object to be scanned according to the first point cloud data, and determines the physical resolution required for the object to be scanned according to the characteristic parameters, and finally adjusts the adaptive scanning module to perform three-dimensional reconstruction of the object to be scanned. In this way, the accuracy of analyzing the features of different structures and shapes can be improved on the basis of ensuring the integrity of the point cloud data, thereby improving the accuracy of the three-dimensional reconstruction data.

As shown in Figure 3, it is a flow chart of a method for determining a target scanning module provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a target scanning module provided in an embodiment of the present application includes the following steps.

S30, traversing the categories of each scanning module.

In one embodiment of the present application, each scanning module is suitable for scanning objects to be scanned of different structures and shapes. For example, the light emitted by some scanning modules can be used to scan objects to be scanned with a spherical structure; the light emitted by other scanning modules can be used to scan objects to be scanned with a cubic structure. In order to determine the scanning module suitable for scanning the object to be scanned, the category of each scanning module in the multiple scanning modules can be traversed. The present application does not limit the order of traversing the multiple scanning modules.

S31, when the category of the traversed scanning module is the same as the category of the object to be scanned, determining the scanning module corresponding to the category as the target scanning module.

In one embodiment of the present application, when the category of the traversed scanning module is the same as the category of the object to be scanned, it indicates that the traversed scanning module is suitable for scanning the object to be scanned for three-dimensional reconstruction. Exemplarily, when the traversed scanning module is a sphere, and the structure of the object to be scanned is also a sphere, it can be confirmed that the traversed scanning module is the target scanning module. Then, the second point cloud data obtained by the target scanning module scanning the object to be scanned can be obtained. It can provide data support for the subsequent fusion of multiple point clouds to obtain three-dimensional reconstruction data.

As shown in Figure 4, it is a flow chart of a method for determining a target scanning module provided by another embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a target scanning module provided by an embodiment of the present application includes the following steps.

S40: Determine a physical resolution required for scanning the target area according to the curvature data.

In one embodiment of the present application, the curvature data of the first three-dimensional point can be determined according to the three-dimensional coordinates corresponding to the first three-dimensional point, and the physical resolution required for scanning the object to be scanned can be determined according to the curvature data. When the curvature data is larger, the physical resolution is larger, and when the curvature data is smaller, the physical resolution is smaller. Specifically, when the curvature data is larger, it indicates that the structure of the surface of the object to be scanned is more complex. It can be determined that the size of the voxel required for scanning the object to be scanned is smaller, and the physical resolution is larger, so that the fine-grainedness of voxel processing of point cloud data can be improved, thereby improving the accuracy of three-dimensional reconstruction; when the curvature data is smaller, it indicates that the surface of the object to be scanned is smoother. It can be determined that the size of the voxel required for scanning the object to be scanned is larger, and the physical resolution is smaller, thereby improving the efficiency of point cloud data processing.

S41, traversing the point cloud physical resolution corresponding to each scanning module, and determining the difference between the point cloud physical resolution and the physical resolution required by the target area.

In one embodiment of the present application, the point cloud physical resolution of the scanning module may be the optical resolution of the scanning module, which is used to measure the precision of the photosensitive device in the scanning module. Specifically, the point cloud physical resolution may be the actual number of light points that can be captured per square inch of the optical components of the scanning module, usually expressed in dpi (dots per inch). The point cloud physical resolution can reflect the ability of the scanning module to capture the details of the point cloud data. The higher the point cloud physical resolution, the richer the details of the point cloud data captured by the scanning module. In order to determine the scanning module suitable for scanning the object to be scanned, the point cloud physical resolution corresponding to each scanning module in the multiple scanning modules may be traversed. The present application does not limit the order of traversing multiple scanning modules.

In one embodiment of the present application, in order to determine a point cloud physical resolution close to the physical resolution required by the target area, the difference between the point cloud physical resolution and the physical resolution required by the target area is also determined. Subsequently, the point cloud physical resolution close to the required physical resolution and the corresponding target scanning module can be determined based on the difference.

S42, from the scanning modules whose point cloud physical resolution is greater than the physical resolution required by the target area, determine the scanning module corresponding to the smallest difference as the target scanning module.

In one embodiment of the present application, when the point cloud physical resolution of the traversed scanning module is greater than the physical resolution required for the target area, and the difference between the point cloud physical resolution and the physical resolution required for the target area is the smallest, it indicates that the scanning module is suitable for scanning the object to be scanned, and the accuracy of the second point cloud data scanned by the scanning module meets the corresponding physical resolution, so it can be determined that the scanning module is the target scanning module.

As shown in FIG5 , it is a flow chart of a method for determining the weights of a first three-dimensional point and a second three-dimensional point provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining the weights of a first three-dimensional point and a second three-dimensional point provided in an embodiment of the present application includes the following steps.

S50, determining the weight of each first three-dimensional point according to the position of the light to which each first three-dimensional point belongs in the corresponding scanning module.

In one embodiment of the present application, a scanning module emits a plurality of light rays to illuminate an object to be scanned, and receives light rays reflected by the object to be scanned to obtain point cloud data of the object to be scanned. The farther the position of the light rays irradiating the object to be scanned in the corresponding scanning module is from the center of the scanning module, the closer the part of the object to be scanned illuminated by the light rays is to the outside, and thus the weight of the first three-dimensional point obtained by scanning the object to be scanned by the light rays is lower. The closer the position of the light rays irradiating the object to be scanned in the corresponding scanning module is from the center of the scanning module, the closer the part of the object to be scanned illuminated by the light rays is to the center of the object to be scanned, and thus the weight of the first three-dimensional point obtained by scanning the object to be scanned by the light rays is higher.

S51, determining the weight of each second three-dimensional point according to the position of the light to which each second three-dimensional point belongs in the target scanning module.

In one embodiment of the present application, when the position of the light irradiating the object to be scanned in the target scanning module is farther from the center of the target scanning module, the part of the object to be scanned irradiated by the light is closer to the outside, so the weight of the second three-dimensional point obtained by scanning the object to be scanned by the light is lower. When the position of the light irradiating the object to be scanned in the target scanning module is closer to the center of the target scanning module, the part of the object to be scanned by the light is closer to the center of the object to be scanned, so the weight of the second three-dimensional point obtained by scanning the object to be scanned by the light is higher.

As shown in Figure 6, it is a flow chart of a method for determining the weights of a first three-dimensional point and a second three-dimensional point provided by another embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining the weights of a first three-dimensional point and a second three-dimensional point provided by an embodiment of the present application includes the following steps.

S60: Determine the weight of each first three-dimensional point according to the distance between the scanning module corresponding to each first three-dimensional point and the object to be scanned.

In one embodiment of the present application, the greater the distance between the scanning module and the object to be scanned, the coarser the features of the surface of the object to be scanned represented by the point cloud data obtained by the scanning module, and the lower the weight of the point cloud data obtained by scanning the object to be scanned by the scanning module; the closer the distance between the scanning module and the object to be scanned, the higher the accuracy of the features of the surface of the object to be scanned represented by the point cloud data obtained by the scanning module, and the higher the weight of the point cloud data obtained by scanning the object to be scanned by the scanning module.

S61, determining the weight of each second three-dimensional point according to the distance between the target scanning module and the object to be scanned.

In one embodiment of the present application, the greater the distance between the target scanning module and the object to be scanned, the rougher the features of the surface of the object to be scanned represented by the second point cloud data obtained by the target scanning module, and the lower the weight of the second three-dimensional point; the closer the distance between the scanning module and the object to be scanned, the higher the accuracy of the features of the surface of the object to be scanned represented by the point cloud data obtained by the scanning module, and the higher the weight of the second three-dimensional point.

As shown in FIG. 7 , it is a flowchart of a method for determining the weights of a first three-dimensional point and a second three-dimensional point provided by another embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The method for determining the weights of a first three-dimensional point and a second three-dimensional point provided by an embodiment of the present application includes the following steps.

S70, determining a weight of each first three-dimensional point according to an incident angle of the light to which each first three-dimensional point belongs irradiating the object to be scanned.

In one embodiment of the present application, when determining the incident angle corresponding to the light to which the first three-dimensional point belongs, the coordinate line between the first three-dimensional point and the scanning module can be determined, and the normal vector of the first point cloud data can be determined, and the angle between the coordinate line and the normal vector of the first point cloud data can be calculated to obtain the incident angle of the light corresponding to the first three-dimensional point when irradiating the module to be scanned. When the incident angle is larger, it indicates that the light is closer to the direction perpendicular to the surface of the object to be scanned, so the weight of the first point cloud data is higher; when the incident angle is smaller, it indicates that the light is closer to the direction tangent to the surface of the object to be scanned, so the weight of the first point cloud data is lower.

S71, determining a weight of each second three-dimensional point according to an incident angle of the light to which each second three-dimensional point belongs irradiating the object to be scanned.

In one embodiment of the present application, the coordinate line between the second three-dimensional point and the scanning module can be determined, and the normal vector of the second point cloud data can be determined, and the angle between the coordinate line and the normal vector of the second point cloud data can be calculated to obtain the incident angle of the light corresponding to the second three-dimensional point when irradiating the module to be scanned. When the incident angle is larger, it indicates that the light is closer to the direction perpendicular to the surface of the object to be scanned, so the weight of the second point cloud data is higher; when the incident angle is smaller, it indicates that the light is closer to the direction tangent to the surface of the object to be scanned, so the weight of the second point cloud data is lower.

Please refer to Figure 8, which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement a three-dimensional reconstruction method described in any of the above embodiments.

In an embodiment of the present application, the electronic device 100 further includes a bus, and a computer program stored in the memory 12 and executable on the processor 13, such as a three-dimensional reconstruction program.

FIG8 only shows an electronic device 100 having a memory 12 and a processor 13. Those skilled in the art will appreciate that the structure shown in FIG4 does not limit the electronic device 100 and may include fewer or more components than shown, or a combination of certain components, or a different arrangement of components.

In conjunction with Figure 2, the memory 12 in the electronic device 100 stores multiple computer-readable instructions to implement a three-dimensional reconstruction method, and the processor 13 can execute the multiple instructions to achieve: obtaining the first point cloud data obtained by scanning the object to be scanned by the scanning module with the longest working distance; wherein the first point cloud data includes multiple first three-dimensional points; determining the characteristic parameters of the first point cloud data; determining the target area on the object to be scanned according to the characteristic parameters; determining the target scanning module from the multiple scanning modules according to the characteristic parameters in the target area; obtaining the second point cloud data obtained by the target scanning module scanning the target area; wherein the second point cloud data includes multiple second three-dimensional points; determining the weights of each first three-dimensional point and each second three-dimensional point; and fusing the multiple first three-dimensional points and the multiple second three-dimensional points corresponding to at least one of the target areas according to the weights to obtain the three-dimensional reconstruction data of the object to be scanned.

Specifically, the specific implementation method of the processor 13 for the above instructions can refer to the description of the relevant steps in the embodiment corresponding to Figure 2, which will not be repeated here.

Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may be a bus-type structure or a star-type structure. The electronic device 100 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 100 may also include input and output devices, network access devices, etc.

It should be noted that the electronic device 100 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 100, such as a mobile hard disk of the electronic device 100. In other embodiments, the memory 12 can also be an external storage device of the electronic device 100, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 100. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 100, such as a code of a three-dimensional reconstruction program, etc., but also can be used to temporarily store data that has been output or is to be output.

In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 100, and uses various interfaces and lines to connect various components of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing programs or modules stored in the memory 12 (for example, executing a three-dimensional reconstruction program, etc.), and calling data stored in the memory 12.

The processor 13 executes the operating system and various installed applications of the electronic device 100. The processor 13 executes the applications to implement the steps in each of the above-mentioned three-dimensional reconstruction method embodiments, such as the steps shown in FIG. 2 .

Exemplarily, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules/units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 100.

The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of a three-dimensional reconstruction method described in each embodiment of the present application.

If the module/unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.

Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one arrow is used in FIG4 , but it does not mean that there is only one bus or one type of bus. The bus is configured to realize connection and communication between the memory 12 and at least one processor 13, etc.

An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement a three-dimensional reconstruction method described in any of the above embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims (10)

1. A three-dimensional reconstruction method applied to an electronic device, wherein the electronic device is communicatively connected to a plurality of scanning modules, the plurality of scanning modules are used for scanning an object to be scanned, and each scanning module has a different working distance range, the method comprising:

Acquiring first point cloud data obtained by scanning the object to be scanned by a scanning module with the farthest working distance; wherein the first point cloud data comprises a plurality of first three-dimensional points;

Determining characteristic parameters of the first point cloud data;

Determining a target area on the object to be scanned according to the characteristic parameters;

determining a target scanning module from the plurality of scanning modules according to the characteristic parameters in the target area;

acquiring second point cloud data obtained by scanning the target area by the target scanning module; wherein the second point cloud data includes a plurality of second three-dimensional points;

Determining the weight of each first three-dimensional point and each second three-dimensional point;

And fusing the plurality of first three-dimensional points and the plurality of second three-dimensional points corresponding to at least one target area according to the weights to obtain three-dimensional reconstruction data of the object to be scanned.

2. The three-dimensional reconstruction method according to claim 1, wherein the characteristic parameters include:

The category of the object to be scanned; and curvature data of the first point cloud data.

3. The three-dimensional reconstruction method according to claim 2, wherein the determining a target scan module from the plurality of scan modules according to the characteristic parameter in the target region comprises:

Traversing the category of each scanning module;

And when the category of the traversed scanning module is the same as the category of the object to be scanned, determining the scanning module corresponding to the category as a target scanning module.

4. The three-dimensional reconstruction method according to claim 2, wherein the determining a target scan module from the plurality of scan modules according to the characteristic parameters in the target region comprises:

Determining a physical resolution required for scanning the target area according to the curvature data;

Traversing the point cloud physical resolution corresponding to each scanning module, and determining the difference between the point cloud physical resolution and the physical resolution required by the target area;

and determining the scanning module corresponding to the smallest difference value as a target scanning module from the scanning modules with the point cloud physical resolution larger than the physical resolution required by the target area.

5. The three-dimensional reconstruction method according to claim 4, further comprising:

And setting fusion resolution in the electronic equipment according to the point cloud physical resolution of the target scanning module, wherein the fusion resolution is used for representing the resolution of the three-dimensional reconstruction data of the target area.

6. The three-dimensional reconstruction method according to claim 1, further comprising:

and determining the quantity of light rays emitted by the target scanning module when the target scanning module scans the object to be scanned based on the distance between the target scanning module and the object to be scanned.

7. The three-dimensional reconstruction method of claim 1, wherein the determining the weights for each first three-dimensional point and each second three-dimensional point comprises:

determining the weight of each first three-dimensional point according to the position of the ray of each first three-dimensional point in the corresponding scanning module; determining the weight of each second three-dimensional point according to the position of the ray of each second three-dimensional point in the target scanning module; and/or

Determining the weight of each first three-dimensional point according to the distance between the scanning module corresponding to each first three-dimensional point and the object to be scanned; determining the weight of each second three-dimensional point according to the distance between the target scanning module and the object to be scanned; and/or

Determining the weight of each first three-dimensional point according to the incidence angle of the light rays of each first three-dimensional point to irradiate the object to be scanned; and determining the weight of each second three-dimensional point according to the incidence angle of the light rays of each second three-dimensional point to irradiate the object to be scanned.

8. A three-dimensional reconstruction system, the system comprising: a plurality of scanning modules and electronic equipment;

the plurality of scanning modules are used for scanning the object to be scanned, and the distance between each scanning module and the object to be scanned is different;

The electronic equipment is used for acquiring first point cloud data acquired by the scanning module corresponding to the largest distance; determining characteristic parameters of the first point cloud data; determining a target scanning module from the plurality of scanning modules according to the characteristic parameters; acquiring second point cloud data obtained by scanning the object to be scanned by the target scanning module; and fusing the first point cloud data and the second point cloud data to obtain three-dimensional reconstruction data of the object to be scanned.

9. An electronic device comprising a processor and a memory, wherein the processor is configured to implement the three-dimensional reconstruction method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional reconstruction method according to any one of claims 1 to 7.