CN116416366A

CN116416366A - 3D model construction method and device and electronic equipment

Info

Publication number: CN116416366A
Application number: CN202111663088.1A
Authority: CN
Inventors: 张津维; 刘海强; 李凌云
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-12-30
Filing date: 2021-12-30
Publication date: 2023-07-11
Also published as: WO2023124676A1

Abstract

The embodiment of the application provides a 3D model construction method, a device and electronic equipment, wherein the method combines plane shape extraction, space intersection calculation, space plane projection, polyhedral topology construction, surface optimization calculation generation model and the like to carry out model simplified reconstruction on an object, the precision of the constructed 3D model can be improved, a model of LOD3 level is obtained, and the construction efficiency of the 3D model can be improved by an automatic construction mode.

Description

3D model construction method and device and electronic equipment

Technical Field

The embodiment of the application relates to a 3D model construction technology, in particular to a 3D model construction method, a device and electronic equipment.

Background

The 3D map contains more abundant information compared with the 2D map, so that a user can see a three-dimensional object when viewing the 3D map, and the user experience is high. In the process of constructing the 3D map, an accurate and fine 3D model needs to be constructed, such as a 3D model of a building in a city.

At present, the 3D model construction in the prior art has the problems of low precision and low construction efficiency.

Disclosure of Invention

The embodiment of the application provides a 3D model construction method, a device and electronic equipment, which can improve the accuracy and the construction efficiency of constructing a 3D model.

In a first aspect, an embodiment of the present application provides a 3D model building method, and an execution body for executing the method may be an electronic device or a chip in the electronic device, and the electronic device is described below as an example. In the method, the electronic device may extract a plurality of planes of the object from the data to be processed. The object is an object to be constructed into a 3D model, and the data to be processed is RGB point cloud or Mesh grid. In one embodiment, the data to be processed is an RGB point cloud or Mesh grid containing the object, or the data to be processed is an RGB point cloud or Mesh grid of the object.

After the electronic equipment extracts a plurality of planes of the object, every two planes can be subjected to space intersection calculation to obtain a plurality of candidate surfaces of the object and at least two polyhedrons formed by the plurality of candidate surfaces. In this embodiment of the present application, the electronic device may project the data to be processed to each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data.

When the data to be processed is the RGB point cloud, the first data is a first RGB point cloud, the first RGB point cloud is included in the RGB point cloud, and the projection of the data to be processed to each candidate surface is the projection of the RGB point cloud to each candidate surface. When the data to be processed is the Mesh grid, the first data is a first Mesh grid, and the first Mesh grid is contained in the Mesh grid, wherein the projection of the data to be processed to each candidate surface is the projection of the vertex of the Mesh grid to each candidate surface. It should be appreciated that the first candidate surface is included in the plurality of candidate surfaces, and the first candidate surface is a surface of the first polyhedron.

The electronic device may obtain the internal and external labels of the first polyhedron according to the normal vector of the first data and the vector from the center point of the first polyhedron to the first data. Wherein the internal and external labels characterize the first polyhedron as either an external polyhedron or an internal polyhedron of the object. The electronic device may construct a 3D model of the object according to the internal and external labels of the first polyhedron. In one embodiment, the electronic device, after obtaining the 3D model of the object, may output the 3D model of the object so that the user may view the constructed 3D model in real time, such that the 3D model is visualized.

In the embodiment of the application, the electronic equipment is used for carrying out model simplified reconstruction on the object by combining plane shape extraction, space intersection calculation, space plane projection, polyhedral topology construction, surface optimization calculation generation model and the like, so that the precision of the 3D model can be improved, a model of LOD3 level can be obtained, and the construction efficiency of the 3D model can be improved in an automatic construction mode.

The following describes a process of extracting a planar shape of an electronic device:

the electronic device may use the plane in which the coplanar data to be processed is located as a plane, so as to obtain the plurality of planes. When the data to be processed is the RGB point cloud, the electronic device may use a plane where the co-planar RGB point cloud is located as a plane according to a spatial position relationship of the RGB point cloud. For example, the electronic device may employ a region growing algorithm or a random sampling consensus algorithm, where a plane where the co-planar RGB point clouds lie is taken as a plane. When the data to be processed is the Mesh grid, the electronic device can take the plane where the coplanar Mesh grid is located as a plane according to the adjacent topological relation of the Mesh grid. For example, the electronic device may use a region growing algorithm or a random sampling coincidence algorithm, where a plane where the coplanar Mesh grid is located is regarded as a plane.

The following description is made of a process of performing spatial intersection calculation on an electronic device:

after obtaining a plurality of planes of the object, the electronic device can perform two-by-two space intersection calculation on the planes to obtain a plurality of intersection lines, further perform constraint triangulation calculation on each space plane formed by the intersection lines by taking the intersection lines as boundaries to obtain a plurality of candidate surfaces, so that the electronic device can obtain the at least two polyhedrons according to the adjacent relation of each candidate surface.

The following describes a process of performing spatial plane projection on an electronic device:

wherein the electronic device may project the data to be processed to each candidate surface. When the data to be processed is the RGB point cloud, the electronic device may project the RGB point cloud to each candidate surface, that is, project each RGB point cloud to each candidate surface. When the data to be processed is the Mesh grid, the electronic device may project vertices of the Mesh grid to each candidate surface. After the projection, the electronic device may determine a first candidate surface corresponding to the first data according to a distance between the first data and each candidate surface and a positional relationship between a projection point of the first data and each candidate surface. The polyhedron to which the first candidate surface belongs is the first polyhedron.

In one possible implementation manner, the electronic device may construct a topological relation among "point-to-point, point-to-surface, surface-to-surface, and surface-to-body" according to a manner of adopting the topological combination diagram, so as to obtain the first polyhedron to which the first candidate surface belongs.

In one possible implementation, the electronic device may use, as the first candidate surface corresponding to the first data, a candidate surface closest to the first data and into which the projection point of the first data falls.

The following description is made on the process of generating a model by performing surface optimization calculation on electronic equipment:

in the above description, the electronic device may obtain a first candidate surface and a first polyhedron corresponding to the first data, for example, a first candidate surface and a first polyhedron corresponding to the first RGB, or may obtain a first candidate surface and a first polyhedron corresponding to the first Mesh grid.

In one embodiment, the electronic device may obtain weights of the internal and external labels of the first polyhedron according to a normal vector of the first data and a correlation between a center point of the first polyhedron and a vector of the first data, and when the first data is the first Mesh, the vector from the center point of the first polyhedron to the first data is a vector from the center point of the first polyhedron to the center point of the first Mesh. The electronic device may obtain a normal vector of each first data, and a correlation between a center point of the first polyhedron and a vector of each first data corresponding to the first polyhedron, so as to obtain weights of internal and external labels of the first polyhedron according to the number of positive correlations and the number of negative correlations. Illustratively, the greater the number of positive correlations, the greater the weight of the inner labels of the first polyhedron, and the greater the number of negative correlations, the greater the weight of the outer labels of the first polyhedron.

Further, the electronic device may acquire the internal and external labels of the first polyhedron by using weights of the internal and external labels of all the first polyhedrons included in the object as nodes of the graph structure and using an adjacent area between the first polyhedron and an adjacent polyhedron as sides of the graph structure. In one embodiment, the electronic device may use weights of the inner and outer labels of all the first polyhedrons in the object as nodes of the graph structure, and use an adjacent area between the first polyhedron and an adjacent polyhedron as an edge of the graph structure, and acquire the inner and outer labels of the first polyhedron by adopting a maximum flow algorithm or a minimum cut algorithm.

The electronic device may take, as the surface of the object, the adjacent surfaces of the inner polyhedron and the outer polyhedron in the object after obtaining the inner and outer labels of each first polyhedron of the object (i.e. characterizing whether the first polyhedron belongs to the outer polyhedron or the inner polyhedron of the object), and then splice the surfaces to construct the 3D model of the object.

In one embodiment, when the data to be processed is an RGB point cloud or a Mesh grid of the object, the electronic device further needs to obtain the data to be processed from initial data before constructing the 3D model, where the initial data includes at least one initial RGB point cloud or an initial Mesh grid of the object.

The electronic device may process the initial data to obtain an RGB point cloud or Mesh grid of the object. Specifically, the electronic device may sample the initial data in a multi-scale space, so as to obtain a sampling result of at least two scales. When the initial data is an initial RGB point cloud, the electronic device may execute the following steps a and B:

a, dividing the initial RGB point cloud by adopting an ith scale grid, and obtaining an ith center point cloud in each grid to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith center point cloud, the ith center point cloud is a center point of point clouds contained in the grids, i is an integer which is greater than or equal to 1 and is smaller than N, and N is an integer which is greater than 1.

B, judging whether the i is smaller than N; if yes, the step A is continuously executed with the i+1, and if not, the sampling results of at least two scales are obtained.

When the initial data is an initial Mesh grid, the electronic device may execute the following steps a and B:

and A, taking the vertexes of the initial Mesh grid as sphere centers, dividing the initial Mesh grid by spheres with the radius of the ith scale, and obtaining an ith vertex in each sphere to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith vertex, i is an integer which is greater than or equal to 1 and is smaller than N, and N is an integer which is greater than 1.

As described above, considering that since the urban building is large in size and large in local structural difference, the high-resolution sampling can retain a fine structure but lose global information of the object, while the low-resolution sampling can have a larger receptive field but lose geometric details (i.e., local information) of a small object, in order to effectively extract structural information of the building, in the embodiment of the present application, multi-scale spatial sampling can be adopted, so that global information and local information can be obtained.

After the electronic equipment acquires the sampling results of at least two scales, the electronic equipment can combine the topographic information of the area to which the initial data belongs to perform regularization processing on the sampling results of each scale to obtain the processed sampling results of at least two scales. Wherein the topographic information comprises: a range of positions of a plurality of terrain grids of the area to which the initial data belongs, and an altitude value of each terrain grid. The electronic device subtracts the altitude value of the first terrain grid from the Z value of the initial data falling in the position range of the first terrain grid to complete regularization processing of the initial data falling in the position range of the first terrain grid, wherein the first terrain grid is contained in the plurality of terrain grids.

In the embodiment of the application, the sampling result of each scale is regularized, so that the perpendicularity characteristics of objects of different urban scenes (or different areas) are consistent, the problem of large fitting error of the neural network model caused by large difference of terrain information is avoided, and the accuracy is improved.

After obtaining the processed sampling results of at least two scales, the electronic device can input the processed sampling results of at least two scales into a pre-trained neural network model to obtain data carrying the label of the object. Exemplary, if the object is a building, the data carrying the tag of the object is, for example, RGB point cloud or Mesh grid carrying the tag of the building. In this embodiment, the electronic device may segment the data carrying the tag of the object to obtain the data to be processed, for example, RGB point clouds or Mesh grids corresponding to each object. Thus, RGB point clouds or Mesh grids corresponding to each object are obtained, and the subsequent steps such as plane extraction are performed, so that the planes of different objects can be prevented from being modeled together, and the accuracy of 3D modeling can be improved.

In a second aspect, an embodiment of the present application provides a 3D model building apparatus, which may be an electronic device or a chip in an electronic device as in the first aspect, where the apparatus may include:

the plane feature extraction module is used for extracting a plurality of planes of an object from data to be processed, the object is an object of a 3D model to be constructed, and the data to be processed is an RGB point cloud or Mesh grid.

And the space intersection calculation module is used for carrying out space intersection calculation on each plane in pairs to obtain a plurality of candidate surfaces of the object and at least two polyhedrons formed by the plurality of candidate surfaces.

The polyhedron topology construction module is used for projecting the data to be processed to each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data, wherein the first data is contained in the data to be processed, when the data to be processed is the RGB point cloud, the first data is the first RGB point cloud, when the data to be processed is the Mesh grid, the first data is the first Mesh grid, the first candidate surfaces are contained in the plurality of candidate surfaces, and the first candidate surfaces are the surfaces of the first polyhedron.

The surface optimization generation module is used for acquiring the internal and external labels of the first polyhedron according to the normal vector of the first data and the vector from the central point of the first polyhedron to the first data, and constructing the 3D model of the object according to the internal and external labels of the first polyhedron. Wherein the internal and external labels characterize the first polyhedron as either an external polyhedron or an internal polyhedron of the object.

In one possible implementation manner, the plane feature extraction module is specifically configured to use a plane where coplanar data to be processed is located as a plane, so as to obtain the multiple planes.

When the data to be processed is the RGB point cloud, in a possible implementation manner, the plane feature extraction module is specifically configured to use a plane where the co-planar RGB point cloud is located as a plane according to a spatial position relationship of the RGB point cloud.

When the data to be processed is the Mesh grid, in a possible implementation manner, the plane feature extraction module is specifically configured to take a plane where the coplanar Mesh grid is located as a plane according to an adjacent topological relation of the Mesh grid.

In one possible implementation manner, the space intersection calculation module is specifically configured to perform space intersection calculation on each plane two by two to obtain a plurality of intersection lines; taking the intersecting lines as boundaries, and performing constraint triangulation calculation on each space plane formed by the intersecting lines to obtain a plurality of candidate surfaces; and obtaining the at least two polyhedrons according to the adjacent relation of each candidate surface.

In one possible implementation manner, the polyhedral topology construction module is specifically configured to project the data to be processed onto each candidate surface, and when the data to be processed is the Mesh grid, the data to be processed is projected to be the vertex of the Mesh grid; and determining a first candidate surface corresponding to the first data according to the distance between the first data and each candidate surface and the position relation between the projection point of the first data and each candidate surface.

In one possible implementation manner, the polyhedral topology construction module is specifically configured to take, as a first candidate surface corresponding to the first data, a candidate surface closest to the first data and into which a projection point of the first data falls.

In a possible implementation manner, a surface optimization generating module is specifically configured to obtain weights of internal and external labels of the first polyhedron according to a normal vector of the first data and a correlation between a center point of the first polyhedron and a vector of the first data, where when the first data is the first Mesh, the vector from the center point of the first polyhedron to the first data is a vector from the center point of the first polyhedron to the center point of the first Mesh; and taking the weights of the inner and outer labels of all the first polyhedrons contained in the object as nodes of a graph structure, and taking the adjacent area between the first polyhedron and the adjacent polyhedron as an edge of the graph structure to acquire the inner and outer labels of the first polyhedron.

In one possible implementation, the first polyhedron corresponds to a plurality of first data, and the correlation includes a positive correlation and a negative correlation. The surface optimization generation module is specifically used for acquiring a normal vector of each first data and the correlation between the central point of the first polyhedron and the vector of each first data corresponding to the first polyhedron; and acquiring the weight of the internal and external labels of the first polyhedron according to the number of positive correlations and the number of negative correlations.

In one possible implementation manner, the surface optimization generating module is specifically configured to use weights of inner and outer labels of all first polyhedrons in the object as nodes of a graph structure, use an adjacent area between the first polyhedron and an adjacent polyhedron as an edge of the graph structure, and obtain the inner and outer labels of the first polyhedron by adopting a maximum flow algorithm or a minimum cut algorithm.

In one possible implementation manner, the surface optimization generating module is specifically configured to take an adjacent surface of an internal polyhedron and an external polyhedron in the object as a surface of the object; and constructing a 3D model of the object according to the surface of the object.

In one possible implementation, the data to be processed includes or is an RGB point cloud or Mesh grid of the object.

When the data to be processed is an RGB point cloud or a Mesh grid of the object, in one possible implementation manner, the 3D semantic segmentation module and the instance segmentation module are configured to obtain the data to be processed from initial data, where the initial data includes at least one initial RGB point cloud or an initial Mesh grid of the object.

In one possible implementation manner, the 3D semantic segmentation module is specifically configured to perform multi-scale spatial sampling on the initial data to obtain sampling results of at least two scales, perform regularization processing on the sampling results of each scale in combination with terrain information of an area to which the initial data belongs to obtain sampling results of at least two scales after processing, and input the sampling results of at least two scales after processing into a neural network model trained in advance to obtain data carrying a tag of the object.

The instance segmentation module is specifically configured to segment the data carrying the tag of the object to obtain the data to be processed.

In one possible implementation manner, when the initial data is an initial RGB point cloud, the 3D semantic segmentation module is specifically configured to perform step a and step B:

In one possible implementation manner, when the initial data is an initial Mesh grid, the 3D semantic segmentation module is specifically configured to execute step a and step B:

In one possible implementation, the terrain information includes: a range of positions of a plurality of terrain grids of the area to which the initial data belongs, and an altitude value of each terrain grid.

The 3D semantic segmentation module is specifically configured to subtract the altitude value of the first terrain grid from the Z value of the initial data falling into the position range of the first terrain grid, so as to complete regularization processing of the initial data falling into the position range of the first terrain grid, where the first terrain grid is included in the plurality of terrain grids.

In a possible implementation, the surface optimization generation module is specifically configured to output a 3D model of the object.

In a third aspect, embodiments of the present application provide an electronic device, which may include: a processor and a memory. The memory is for storing computer executable program code, the program code comprising instructions; the instructions, when executed by a processor, cause the electronic device to perform the method as in the first aspect.

In a fourth aspect, an embodiment of the present application provides an electronic device, where the electronic device may be the 3D model building apparatus of the second aspect or the electronic device of the first aspect. The electronic device may comprise means, modules or circuits for performing the method provided in the first aspect above.

In a fifth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the first aspect described above.

In a sixth aspect, embodiments of the present application provide a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the method of the first aspect described above.

The advantages of each of the possible implementation manners of the second aspect to the sixth aspect may be referred to as the advantages brought by the first aspect, and are not described herein.

Drawings

Fig. 1 is a schematic flow chart of an embodiment of a 3D model building method provided in the embodiments of the present application;

fig. 2 is a schematic flow chart of another embodiment of a 3D model building method according to the embodiments of the present application;

FIG. 3 is a schematic diagram of multi-scale spatial sampling provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a grid in a DTM according to an embodiment of the disclosure;

FIG. 5 is a schematic diagram of a U-Net network architecture used in embodiments of the present application;

FIG. 6 is a schematic diagram of partitioning an object using an exemplary algorithm according to an embodiment of the present application;

FIG. 7A is a schematic diagram of a region growing algorithm for processing a point cloud according to an embodiment of the present application;

FIG. 7B is a schematic diagram of a flat proxy provided in an embodiment of the present application;

fig. 7C is a schematic diagram of determining a Mesh grid coplanarity by using a topology relationship of the Mesh grid according to an embodiment of the present application;

FIG. 8A is a schematic diagram of a planar intersection calculation performed by two-by-two in the embodiment of the present application;

FIG. 8B is a schematic view of a point cloud projected onto a candidate surface according to an embodiment of the present disclosure;

fig. 8C is a schematic diagram of a topological relation of the polyhedron 1 according to an embodiment of the present application;

FIG. 8D is a diagram illustrating correlation between acquired vectors according to an embodiment of the present disclosure;

FIG. 9 is a schematic view of an inner polyhedron and an outer polyhedron provided in an embodiment of the present application;

FIG. 10 is a schematic view of surfaces of inner and outer polyhedrons adjacent to each other provided in an embodiment of the present application;

FIG. 11 is a flowchart illustrating another embodiment of a 3D model building method according to an embodiment of the present application;

FIG. 12 is a schematic flow chart of training a neural network model according to an embodiment of the present disclosure;

FIG. 13 is a schematic flow chart of another embodiment of a 3D model building method according to the embodiments of the present application;

fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

Fig. 15 is a schematic structural diagram of another electronic device according to an embodiment of the present application.

Detailed Description

The terms first, second, third and the like in the description and in the claims and drawings are used for distinguishing between different objects and not for limiting the specified sequence.

In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

The terminology used in the description of the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application, as will be described in detail with reference to the accompanying drawings.

Building a 3D map requires building accurate, fine-grained 3D models, such as building 3D models of buildings in cities, to accurately describe environmental information. A level of detail (LOD) model is one way to express the 3D shape of an object, and depending on the level of detail of the model, the LOD model may include: the higher the level of LOD0, LOD1, LOD2, LOD3, etc., the higher the level of detail of the LOD model. Taking an LOD model of a building as an example, the LOD 0-level model comprises a 2D plane outline of the building, the LOD 1-level model comprises an outline and a height of the building, the LOD 2-level model is added with a roof structure on the basis of the LOD1 level, and the LOD 3-level model is added with a building facade structure on the basis of the LOD2 level.

The LOD model is mainly applied to the following scenarios:

1. in the visual positioning technology, quick positioning can be realized according to an LOD model.

2. In the augmented reality (augmented reality, AR) technology, the virtual-real effect and the occlusion effect of an object can be obtained according to the spatial visual relationship in the LOD model.

3. The shadow rendering effect in AR technology requires ray tracing calculations with the surrounding LOD model.

Currently, the method of constructing LOD models in the prior art can be shown as follows in 1-2, 1-2 being an example illustration of the drawbacks characterizing the prior art:

1. LOD models of buildings are constructed based on building horizontal contours (foltprints) and digital surface models (digital surface model, DSMs).

The LOD model constructed in the technology has poor accuracy and fineness, such as a 3D model with LOD1 level and LOD2 level can be constructed, but a model with LOD1 level cannot be constructed. For example, when using this technique to construct a 3D model of a building, irregular roof and facade structures of the building cannot be accurately expressed.

2. And constructing the building structure outline manually point by point and face by face according to the three-dimensional live-action model or the aerial image, and generating a lightweight LOD3 model with high detail.

In the technology, the modeling is completely based on manual modeling, the modeling time is long, the efficiency is low, and the manual modeling depends on professional computer aided design (computer aided design, CAD) modeling software, so that the large-scale quick popularization and coverage are difficult.

Based on the problem of 3D model construction in the prior art, the embodiment of the application provides a 3D model construction method, which uses three-dimensional RGB point clouds, and can automatically construct a LOD3 model with high accuracy and high precision of a building from automatic point cloud 3D semantic segmentation and building instance segmentation, and can restore a real building structure and a geographic position in a 1:1 mode, so that modeling time is short and efficiency is high.

In an embodiment, the three-dimensional RGB point cloud used in the embodiment of the present application may be replaced by a three-dimensional Mesh model (abbreviated as Mesh) or a Mesh with textures, and in the following embodiment, the 3D model construction method provided in the embodiment of the present application is described by taking the RGB point cloud as an example, and the 3D model construction method provided in the embodiment of the present application is described by taking the Mesh as an example. In one embodiment, the 3D model building method provided by the embodiment of the present application may be applied not only to building 3D models, but also to building 3D models of other objects. Other objects include, but are not limited to: objects, human bodies, numbers, vehicles, etc. In the following examples of the embodiment, a 3D model building object is described as an example.

The terms used in the embodiments of the present application are defined as:

RGB point cloud: a point cloud containing color information (RGB). It should be understood that the RGB point cloud in the embodiments of the present application may include three-dimensional coordinates (X, Y, Z) of the point cloud and color information.

In one embodiment, the RGB point cloud used in the embodiments of the present application is a dense RGB point cloud.

In an embodiment, taking an example of building a 3D map of a city, the RGB point cloud used in the embodiment of the present application may be: and acquiring urban overhead multi-angle RGB images by adopting an aerial photography acquisition mode such as unmanned aerial vehicle, airplane and the like, and reconstructing the generated RGB point cloud by a multi-view stereo (MVS) technology. In one embodiment, the RGB point cloud used in the embodiments of the present application may be: and adopting the RGB point cloud generated by fusion and reconstruction of aerial RGB images and ground street view RGB images. In such an embodiment, the RGB point clouds may include RGB point clouds of multiple types of objects in a city, buildings, plants, and the like.

In one embodiment, when constructing an indoor 3D map, an indoor RGB point cloud may be acquired using a method not limited to the following: a hand-held/backpack laser scanner, a fixed-mount laser scanner is used to scan the room, or a depth camera, panoramic camera, or a common RGB camera is used to capture the room. The method adopts a panoramic camera or a common RGB camera to shoot, three-dimensional RGB point cloud is generated by shooting multi-position and multi-view images and reconstructing through MVS technology, and the rest modes can directly obtain the RGB point cloud through scanning or shooting. Depth cameras may include, but are not limited to: structured-light (stereoscopic-light) depth cameras, binocular vision (stereo) depth cameras, and time of flight (TOF) depth cameras.

Mesh grid: the Mesh grid can be realized by a triangular grid, a quadrilateral grid or a polygonal grid. Illustratively, the Mesh grid may include: three-dimensional coordinates of each vertex of the mesh, color information, and normals and tangents. The vertex information of the mesh may be obtained by converting the texture of the mesh, which is not described in detail in the embodiment of the present application.

In one embodiment, the Mesh grid model used in the embodiments of the present application is a dense Mesh grid model.

In one embodiment, taking an example of building a 3D map of a city, the Mesh grid model used in the embodiment of the present application is: and acquiring urban overhead multi-angle RGB images by adopting aerial photography acquisition modes such as unmanned aerial vehicle, airplane and the like, and reconstructing the generated Mesh grid model by a multi-view stereo (MVS) technology. In one embodiment, the Mesh grid used in the embodiment of the present application is: and adopting the Mesh grid model generated by fusion and reconstruction of the aerial RGB image and the ground street view RGB image.

In one embodiment, when constructing an indoor 3D map, the Mesh grid in the room may be acquired using the method of acquiring RGB point clouds as described above.

Building example: i.e. a building.

Example segmentation: if the example is a building, the example is divided into the buildings to obtain a single building. By way of example, if the example includes 3 buildings, then the example partitions are used to differentiate each building, resulting in a single building. It is conceivable that the point cloud instance division is to distinguish the point clouds of each building in the point clouds, so as to obtain the point clouds corresponding to each single building.

Terrain information: the terrain information includes two-dimensional coordinates (X, Y) of each location, and altitude values at each location. In one embodiment, the terrain information used in embodiments of the present application may be digital terrain models (digital terrain model), DTM terrain information computationally generated using terrain extraction algorithms, or the like.

The object is: objects to be built into the 3D model, such as buildings, human bodies, numbers, vehicles, etc.

In an embodiment, an execution body for executing the 3D model building method provided in the embodiment of the present application may be an electronic device, where the electronic device may include, but is not limited to,: the form of the electronic device in the embodiment of the present application is not specifically limited, and the electronic device will be described as an example in the following embodiment.

The following describes a 3D model building method provided in the embodiment of the present application with reference to specific embodiments. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes. Fig. 1 is a schematic flow chart of an embodiment of a 3D model building method according to an embodiment of the present application. Referring to fig. 1, the 3D model construction method provided in the embodiment of the present application may include:

s101, responding to a construction instruction of a 3D model, and performing multi-scale spatial sampling on RGB point clouds to be constructed of the 3D model to obtain sampling results of at least two scales.

In one embodiment, a 3D model building platform is provided in an embodiment of the present application, where the platform is deployed in an electronic device, and a user may import an RGB point cloud of a 3D model to be built into the 3D model building platform. Alternatively, the 3D model building platform may use a locally stored RGB point cloud of the 3D model to be built.

In one embodiment, assuming that the building of the 3D model to be constructed is a target area in a city, the RGB point cloud used in the embodiment of the present application is a point cloud of the target area. The target region may be understood as a region where the 3D model is to be constructed, and may be exemplified by: in the range of 1 km around the building a, or the area where the road B is located, etc., the embodiment of the present application does not limit how the target area in the city is divided.

In one embodiment, the 3D model build platform may provide an interface for a user to input build instructions or for the user to do other operations. For example, if the interface of the 3D model building platform includes a control for building a 3D model, a user clicks the control for building a 3D model, and may input a building instruction to the electronic device. Accordingly, the electronic device may begin building the 3D model in response to receiving a build instruction for the 3D model.

The electronic device may perform multi-scale spatial sampling on RGB point clouds (RGB point clouds or point clouds for short) to be built into the 3D model in response to receiving a building instruction of the 3D model, so as to obtain a sampling result of at least two scales. Considering that the fine structure can be reserved but the global information of the object is lost by the high-resolution sampling due to the large size of the urban building and the large difference of the local structures, and the geometric details (namely the local information) of the small object is lost by the large receptive field can be possessed by the low-resolution sampling, in order to effectively extract the structural information of the building, the multi-scale space sampling RGB point cloud can be adopted in the embodiment of the application.

In one embodiment, the RGB point cloud to be modeled in 3D may be referred to as an initial RGB point cloud. The electronic equipment can divide the RGB point cloud by adopting grids with different sizes, so as to obtain the division results of the grids with different sizes. In one embodiment, the grid may be understood as a cube. The following describes a process of multi-scale spatial sampling of an RGB point cloud to be built into a 3D model by an electronic device:

1. And segmenting the RGB point cloud by adopting a grid of a first scale. Referring to fig. 3, the RGB point clouds are arranged in a three-dimensional space, the grids of the first scale are 2cm×2cm, the RGB point clouds can be segmented by using the grids of 2cm×2cm, and the RGB point clouds fall into different grids. In one embodiment, one point cloud may be sampled in each grid, the three-dimensional coordinates of the point cloud may be the average of the three-dimensional coordinates of all the point clouds falling in the grid, and the color information (i.e., RGB values) of the point cloud may be the average of the RGB values of all the point clouds falling in the grid. In one embodiment, the point cloud sampled in each grid may be referred to as a first center point cloud, as shown by the black dots in each grid.

Thus, the electronic device can obtain the sampling result of the first scale. The sampling result of the first scale may include: the first center point cloud (including three-dimensional coordinates and color information of the first center point cloud) in each grid.

2. And segmenting the RGB point cloud by adopting a grid of a second scale. The second scale is larger than the first scale, and illustratively, the grids of the second scale are 4cm×4cm, the electronic device may divide the RGB point cloud by using the grids of 4cm×4cm, and the RGB point cloud falls into different grids. In one embodiment, one point cloud may be sampled in each grid, the three-dimensional coordinates of the point cloud may be the average of the three-dimensional coordinates of all the point clouds falling in the grid, and the color information (i.e., RGB values) of the point cloud may be the average of the RGB values of all the point clouds falling in the grid. In one embodiment, the point cloud sampled in each grid may be referred to as a second center point cloud.

Thus, the electronic device can obtain the sampling result of the second scale. The sampling result of the second scale may include: the second center point cloud (including three-dimensional coordinates and color information of the second center point cloud) in each grid.

3. And so on, until the RGB point cloud is segmented by adopting the grid of the N-th scale, the related description in the description of the process of segmenting the RGB point cloud by adopting the grid of the first scale and the description of the process of segmenting the RGB point cloud by adopting the grid of the second scale can be referred to. Thus, the electronic device can obtain the sampling result of the Nth scale. The sampling result of the nth scale may include: the nth center point cloud (including three-dimensional coordinates and color information of the nth center point cloud) in each grid. N is an integer greater than 1, and in one embodiment, the value of N depends on the object, and may be, for example, 3-5.

Thus, the electronic device can obtain sampling results of N scales.

In one embodiment, steps A-B may be summarized as steps 1-3 above:

a, segmenting an initial RGB point cloud by adopting an ith scale grid, and acquiring an ith center point cloud in each grid to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith center point cloud, the ith center point cloud is a center point of the point clouds contained in the grids, i is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1. Illustratively, when i is 1, the i-th scale is the first scale and the grid is 2cm.

B, judging whether i is smaller than N; if yes, the step A is continuously executed with the i+1, and if not, sampling results of at least two scales are obtained.

S102, regularizing the sampling result of each scale by combining the topographic information of the region to which the RGB point cloud of the 3D model to be constructed belongs to, and obtaining the sampling result of at least two scales after the processing.

Topography relief significantly affects the spatial perpendicularity characteristics of the RGB point cloud, which is one of the important characteristics to distinguish between buildings and non-buildings, and differences in the spatial perpendicularity characteristics of the point cloud caused by topography relief can result in an inability to correctly distinguish between buildings and non-buildings. In this way, in the embodiment of the present application, regularization (regularization) is performed on the sampling result of each scale in combination with the topographic information of the area to which the RGB point cloud of the 3D model to be constructed belongs, so that the verticality characteristics of the RGB point clouds of the buildings in different urban scenes (or different areas) have consistency, and the problem that the fitting error of the neural network model is large due to large difference of the topographic information is avoided (the neural network model can refer to the related description in fig. 12 below).

For example, if the average altitude value of area 1 is Akm and the average altitude value of area 2 is Bkm, the Z values of the point clouds of buildings having the same height will be different because the altitude values are different. In order to avoid the problem of inconsistent verticality characteristics of the RGB point clouds caused by different terrains, the electronic device can combine the terrains of the areas where the RGB point clouds belong to perform regularization processing on the sampling result of each scale, so that the baselines of the RGB point clouds in different areas are identical, and errors caused by the terrains are eliminated.

In one embodiment, taking area 1 as an example, the topographic information of area 1 may include: two-position coordinates (X, Y) at each location of the region 1, and altitude values at the corresponding locations. In one embodiment, the terrain information for region 1 may be stored in a rasterized form. Illustratively, referring to fig. 4, the area 1 is formed with a plurality of grids (quadrilaterals), each of which corresponds to a range of two-dimensional coordinates and an altitude value. For example, the two-dimensional coordinates corresponding to one grid range is: in the embodiment of the present application, the electronic device may subtract the altitude value corresponding to the grid from the Z value of the RGB point cloud having the two-dimensional coordinate falling within the range of the two-dimensional coordinate corresponding to the grid, so as to complete the regularization processing of the RGB point cloud having the two-dimensional coordinate falling within the range of the two-dimensional coordinate corresponding to the grid. In one embodiment, to facilitate distinguishing grids in topographical information from grids in multi-scale sampling as described above, the grids in topographical information may be referred to as "topographical grids.

Illustratively, the range of two-dimensional coordinates corresponding to the grid is: (X1, Y1), (X2, Y2) to (X3, Y3), (X4, Y4), the altitude value corresponding to the grid being Z _H1 In the sampling result of the first scale, the RGB point cloud falling into the "two-dimensional coordinate range" includes a point cloud 1 and a point cloud 2, the Z value of the point cloud 1 is Z1, and the Z value of the point cloud 2 is Z2. The electronic device may subtract Z from Z1 _H1 As the Z value of RGB point cloud after the regular treatment of point cloud 1, subtracting Z from Z2 _H1 And (3) the Z value of the RGB point cloud after being processed regularly as the point cloud 2.

While the regularization processing manner of the point cloud of the sampling result of the first scale is described above, it should be understood that the regularization processing manner of the sampling result of the different scales may refer to a related description of the regularization processing manner of the point cloud of the sampling result of the first scale. In this way, the electronic device can obtain the processed sampling result of at least two scales.

S103, inputting the processed sampling results of at least two scales into a pre-trained neural network model to obtain RGB point clouds carrying object labels.

The neural network model is used for classifying the RGB point cloud to obtain the RGB point cloud belonging to the object. For example, the object is a building, the neural network model is used for distinguishing RGB point clouds of the building from non-building, and further the RGB point clouds belonging to the building are labeled with a "building", that is, the neural network model can output the RGB point clouds carrying the "building" label. The training process of the neural network model can be described with reference to fig. 12. In one embodiment, the "RGB point cloud with object tags" output by the neural network model may also be referred to as a full-scale RGB point cloud with semantic tags. Semantic tags may be understood as tags identified as "objects".

In one embodiment, the neural network model used in embodiments of the present application is derived based on a U-Net network architecture. Fig. 5 is a schematic diagram of a U-Net network. Referring to fig. 5, the u-Net network architecture includes: an encoder and a decoder. The encoder is used for extracting the characteristics of the RGB point cloud, and the decoder is used for carrying out semantic segmentation based on the characteristics of the RGB point cloud. In the embodiment of the present application, the U-Net network architecture is not described in detail, and reference may be made to the description of the existing U-Net network architecture.

In one embodiment, the random downsampling of the U-Net network architecture package samples RGB point clouds layer by layer, which can improve the generalization ability of the neural network model and the speed of RGB point cloud semantic segmentation (i.e., distinguishing RGB point clouds of buildings and non-buildings). It should be appreciated that N in fig. 5 characterizes the number of point clouds processed at a time, and is 10240 by way of example. The number of downsamples for the N/4 token point cloud is 2560.TF (Transformer) characterizes a point cloud class attention mechanism, and expresses the weight of the adjacent point by calculating the similarity of semantic features of the target point cloud and the adjacent point cloud. US (up sampling) characterizes the upsampling. DS (down sampling) characterizes the downsampling. MLP (multi-layer preference) can be understood as multi-scale feature fusion of a multi-layer perceptron or point cloud.

In this embodiment of the present application, attention is paid to a manner in which an encoder extracts features of an RGB point cloud, and the following is a comparison description of a process in which the encoder extracts features of the RGB point cloud in the embodiment of the present application and a process in which the encoder extracts features of the RGB point cloud in the prior art:

in the prior art, taking an example of an encoder extracting features of the RGB point cloud 1 as an example, the encoder may obtain a plurality of point clouds within a preset spatial range of the point cloud 1. The preset spatial range of the distance point cloud 1 can be understood as follows: the point cloud 1 is taken as a sphere center, and the preset distance is taken as the sphere range of the radius. The encoder can extract the characteristics of the point cloud 1 and each point cloud in the preset space range of the point cloud 1, so that the characteristics of the point cloud 1 and the characteristics of each point cloud in the preset space range of the point cloud 1 are fused, and the characteristics of the point cloud 1 are obtained.

Characteristics of the point cloud may include, but are not limited to: spatial geometry and spectral characteristics. Spatial geometric features may include, but are not limited to: flatness, sphericity, straightness, verticality, and the like. Spectral features may include, but are not limited to: chromaticity, luminance, saturation. If the feature vector 1 (e.g. A1) is used to characterize the feature of the point cloud 1, and the feature vector 2 (e.g. A2) is used to characterize the feature of each point cloud within the preset spatial range of the point cloud 1, then the fusion of the feature of the point cloud 1 and the feature of each point cloud can be understood as follows: feature vectors of feature vector 1 and feature vector 2 are fused, for example, the fused feature vector is (A1, A2).

Different from the prior art, taking the characteristic of the encoder for extracting the RGB point cloud 1 as an example, the encoder can obtain a plurality of point clouds within a preset spatial range of the point cloud 1 in sampling results of different scales. For example, the encoder may acquire a plurality of point clouds within a preset spatial range of the point cloud 1 in a sampling result of a first scale, acquire a plurality of point clouds within a preset spatial range of the point cloud 1 in a sampling result of a second scale, …, and acquire a plurality of point clouds within a preset spatial range of the point cloud 1 in a sampling result of an nth scale. The encoder can extract the characteristics of each point cloud in the preset space range of the point cloud 1 in the sampling results of different scales, so that the characteristics of the point cloud 1 and the characteristics of each point cloud in the sampling results of different scales are fused, and the characteristics of the point cloud 1 are obtained.

In the embodiment of the application, because the RGB point cloud can be subjected to multi-scale spatial sampling in the process of preprocessing the RGB point cloud, when the encoder extracts the characteristics of the point cloud, the point cloud can be fused with the characteristics of the point cloud in sampling results of different scales to obtain the characteristics of the point cloud. Because the embodiment of the application combines the characteristics of the point clouds with different scales, the characteristics of the point clouds are richer than those of the point clouds in the prior art, the fusion of local characteristics and global characteristics constructed according to the spatial relationship is realized, the accuracy of classification of the neural network model can be improved, and the accuracy of distinguishing buildings from non-buildings can be improved.

S104, dividing the object according to the RGB point cloud carrying the object tag to obtain the RGB point cloud corresponding to each object.

In one embodiment, the RGB point cloud for each object individually may be referred to as the data to be processed.

Taking an object as a building as an example, the point cloud segmentation can be simply understood as: distinguishing the point clouds of each building in the point clouds to obtain the point clouds corresponding to the single building. The RGB point cloud carrying the object tag comprises RGB point clouds of a plurality of individual objects.

Illustratively, because the point clouds belonging to the same building are far less spatially apart from one another than the point clouds belonging to different buildings. According to the principle, in one embodiment, the electronic device may perform cluster analysis on the RGB point clouds carrying the object tag by using an unsupervised method with the spatial distance as a threshold value, so as to cluster the point clouds belonging to the same building together, and obtain the RGB point clouds corresponding to each individual object. In this embodiment, the principle and the process of clustering are not described in detail, and reference may be made to description of the principle and the process of clustering in the prior art.

In one embodiment, because the space distance between the point clouds belonging to different buildings is large, the density of the point clouds belonging to different buildings is smaller than that of the point clouds belonging to the same building, so in one embodiment, the electronic device may perform cluster analysis on the RGB point clouds carrying the object tag by using the unsupervised method with the space distance as the threshold and the density as the threshold, so as to cluster the point clouds belonging to the same building together, and obtain the RGB point clouds corresponding to each object individually.

Referring to fig. 6, an unsupervised cluster analysis with a spatial distance as a threshold (or an addition density as a threshold) may cluster point clouds belonging to the same building together, such as the point clouds in each shadow circle belonging to the same building in fig. 6. In this way, the electronic device may obtain an RGB point cloud corresponding to each object (e.g., building) individually.

After obtaining the RGB point cloud corresponding to each object (such as a building), the electronic device may perform simplified building model reconstruction on each object according to the RGB point cloud corresponding to each object to obtain a 3D model of each object. It should be understood that, in the following embodiments, each individual object may be referred to as a first object, and S105 to S115 are the processing procedures of the RGB point clouds corresponding to the first object, that is, the RGB point clouds corresponding to each object (such as each building) may obtain the 3D model of the object according to the processing manner of S105 to S115.

By way of example, taking the first object as a building, the geometry of any building can be approximately expressed by using a flattened shape, so in the embodiment of the application, a 3D model of the building is reconstructed by extracting the plane of the building and performing geometric stitching. In the process of 3D model construction, planar shape extraction is the first link, i.e., the electronic device executes S105.

S105, acquiring plane information of the first object according to the RGB point cloud of the first object.

In one embodiment, each object is characterized by a first object, the RGB point cloud of the first object, i.e., the point cloud of each object alone. The plane information of the first object can be understood as: information of a plane contained in the first object.

In one embodiment, for an RGB point cloud of a first object, the electronic device may utilize the spatial distance between the RGB point clouds as a proximity determination to determine which RGB point clouds are coplanar. For example, the electronic device may employ a region growing algorithm or an efficient random sampling consensus (efficient random sample consensus, efficientrac) algorithm to utilize the spatial distance between the RGB point clouds as a proximity determination to determine which RGB point clouds are coplanar. That is, the electronic device may acquire plane information of the first object based on the RGB point cloud of the first object by using a region growing algorithm or an efficiency RANSAC algorithm. The following description will be given by taking an area growth algorithm as an example for an electronic device:

the electronic device may select a seed point (i.e., a seed point cloud) from the RGB point clouds of the first object, so as to calculate a spatial positional relationship between the seed point and surrounding neighboring points (i.e., the point clouds surrounding the seed point cloud), and determine whether the seed point is coplanar with the surrounding neighboring points in a manner similar to a fitting plane, so as to form a plane for the coplanar point clouds.

For example, referring to fig. 7A, the electronic device selects a seed point 1 from RGB point clouds of a first object, calculates whether the point clouds within a first preset spatial range from the seed point 1 are coplanar with the seed point 1, and if so, uses the seed point 1 and the point clouds within the first preset spatial range from the seed point 1 as a plane 1. The electronic device may also calculate whether the point cloud within the second preset spatial range of the seed point 1 is coplanar with the seed point 1, and if so, the range of the electronic device that the electronic device can be expanded to be away from the second preset spatial range of the seed point cloud. And analogically, until the seed point cloud is not coplanar with the point cloud within the M-th preset space range from the seed point 1, and obtaining the plane 1.

In this embodiment, the electronic device processes the RGB point cloud of the first object by using a region growing algorithm, and may obtain plane information of the first object. For example, the plane information of the first object may include: a plane parameter of at least one plane proxy (planar proxy), an area scope of each plane proxy, and a first index of each plane proxy. A flat proxy may be understood as a flat, the flat parameters of which include: the three-dimensional coordinates of the center point of the planar proxy and the normal vector of the planar proxy. As shown in fig. 7B, the center point of plane 1 is p, and the normal vector of plane 1 is n. The area scope of a flat proxy can be understood as: area of the planar proxy (i.e., the extent of the point cloud that constitutes the planar proxy).

The first index of the flat proxy includes: an index of a point cloud contained in the plane proxy (may be simply referred to as a first point index), and an index of a plane (may be simply referred to as a first plane index). In one embodiment, after obtaining multiple planes (i.e., plane agents) of the first object, the electronic device may index the plane agents to obtain the indexes of the planes, and in addition, the electronic device may index the point clouds included in the plane agents to obtain the indexes of the point clouds included in the plane agents.

In one embodiment, after obtaining the plane information of the first object, the electronic device further needs to perform plane optimization projection calculation, that is, all point clouds belonging to the same plane are projected onto the plane, and according to the spatial distance between the planes of the first object, all planes with coplanar spaces are combined. That is, the electronic device may merge all coplanar planar agents into one plane.

In one embodiment, in an example of 3D model construction using a Mesh grid model, for a Mesh grid, an electronic device may make a determination using the topology of the Mesh grid to determine which Mesh grids are coplanar. In this embodiment, for example, referring to fig. 7C, the electronic device selects a Mesh grid 1 from the Mesh grids of the first object, calculates whether the Mesh grids around the Mesh grid 1 are coplanar with the Mesh grid 1 by using the topological relation of the Mesh grid 1, and if so, uses the Mesh grids around the Mesh grid 1 and the Mesh grid 1 as a plane 1. Taking Mesh grid 1 and Mesh grids around the Mesh grid as an example of Mesh grid 2, the electronic device further continues to utilize the topological relation of Mesh grid 2 to calculate whether the Mesh grids around Mesh grid 2 are coplanar with Mesh grid 2, if so, mesh grid 2 and Mesh grids around the Mesh grid are used as a plane 2. And analogizing until the Mesh grid is not coplanar with surrounding Mesh grids, and obtaining a plane agent.

After extracting the plane of the first object, in order to achieve the planar geometry stitching, all planar agents of the first object need to be subjected to spatial intersection calculation to obtain candidate surfaces of the object, i.e. the electronic device performs S106-S107.

S106, carrying out space intersection calculation on each plane agent of the first object to obtain a plurality of intersection lines.

The plane agent may be understood as a plane, and in S105, each plane of the first object may be obtained, and in this embodiment of the present application, the electronic device may perform two-by-two space intersection calculation on each plane agent of the first object, and one intersection line may be obtained after two planes intersect, so that after two planes intersect two by two in the first object, a plurality of intersection lines may be obtained. Illustratively, referring to FIG. 8A, six planes are computed for two-by-two spatial intersections, resulting in 12 intersecting lines, i.e., 12 sides of a cube. As shown in fig. 8A, the six planes include three sets of planes parallel to each other, such as plane 1 and plane 2 parallel to each other, plane 3 and plane 4 parallel to each other, and plane 5 and plane 6 parallel to each other.

In one embodiment, the two-by-two spatial intersection calculation method adopted in the embodiment of the application includes, but is not limited to: full-scale two-by-two intersection algorithm, minimum bipartite space method (binary space partitioning, BSP), dynamic segmentation method (kinetic partition).

In one embodiment, when the electronic device performs the two-by-two space intersection calculation on each plane agent of the first object by adopting the BSP algorithm, the electronic device may obtain the priority of each plane agent of the first object based on the first index of the plane agent of the first object obtained in S105, and then determine the order of performing the two-by-two space intersection calculation based on the priority. Illustratively, the first index of the flat proxy includes: in the first point index, in the embodiment of the present application, the priority of the plane agent may be represented by the number of indexes including the point cloud in the plane agent (i.e., the number of point clouds falling into the plane), where, if the number of indexes including the point cloud in the plane agent (i.e., the number of point clouds falling into the plane) is greater, the priority of the plane agent is higher.

For example, the electronic device may perform the intersection calculation of the planar agents with the priorities located at the first position and the second position in two space according to the priorities of the planar agents, to obtain an intersection line, and then perform the intersection calculation of the planar agents with the priorities located at the first position and the third position in two space to obtain an intersection line. And by analogy, the electronic equipment can obtain a plurality of intersecting lines which are calculated by each plane agent in a two-by-two space intersecting manner.

And S107, carrying out constraint triangulation calculation on each space plane formed by the intersecting lines by taking the intersecting lines as boundaries to obtain candidate surfaces.

Three intersecting lines can form a triangular patch, and in this embodiment of the present application, the electronic device may use the intersecting lines as boundaries, and perform constraint triangulation (constrained triangulation) calculation on each spatial plane formed by the intersecting lines, to obtain candidate surfaces.

In one embodiment, the candidate surface is triangular in shape, and the embodiment of the present application does not limit the shape of the candidate surface. In one embodiment, taking the shape of the candidate surface as a triangle, the electronic device may then characterize the candidate surface with three vertices (i.e., the intersection points of the intersecting lines) that form the candidate surface. That is, the electronic device may store coordinates of three vertices of the candidate surface when storing the candidate surface. For example, the electronic device may index number the three vertices, i.e., characterize the candidate surface with three vertex indices, which may be referred to as second point indices in this embodiment.

In one embodiment, the candidate surface may also be characterized by three edges of three vertices. That is, the electronic device may store three edges of the candidate surface when storing the candidate surface. For example, the electronic device may index the three edges, i.e., characterize the candidate surface with three edge indices, which may be referred to as second edge indices in this embodiment.

In one embodiment, the electronic device may also index the candidate surfaces by a face index, referred to as a second face index. Because each candidate surface may be composed of at least three edges, one second face index corresponds to at least three second edge indices, and the intersection of the at least three edges is the vertex of the candidate surface, and thus one second face index corresponds to at least three second point indices. That is, after obtaining the candidate surfaces, the electronic device may store the second face index of each candidate surface, and at least three second side indexes and at least three second point indexes corresponding to the second face index.

After obtaining the candidate surface of the first object, in order to determine the spatial relationship between the RGB point cloud of the first object and the calculated candidate surface of the first object, in this embodiment of the present application, a spatial plane projection manner may be adopted to project the RGB point cloud of the first object onto the candidate surface of the first object, that is, the electronic device executes S108.

S108, projecting the RGB point cloud of the first object to the candidate surface of the first object to obtain the candidate surface corresponding to the RGB point cloud of the first object.

In one embodiment, the electronic device may project an RGB point cloud of the first object onto each candidate surface of the first object, and further determine a candidate surface corresponding to the RGB point cloud projection based on two conditions of a distance between the RGB point cloud and each candidate surface and whether the RGB point cloud falls into each candidate surface. The electronic device may have a minimum distance from the point cloud, and the point cloud may project a candidate surface corresponding to the point cloud of the candidate surface that falls into.

For example, referring to fig. 8B, the candidate surfaces are surface 1 and surface 2, the electronic device projects the point cloud 1 onto the surface 1 and the surface 2, respectively, and the electronic device may use the surface 1 on which the projection falls as the candidate surface corresponding to the point cloud 1. In one embodiment, if the projections of the point cloud 1 fall within the range of the surfaces 1 and 2, the electronic device may use the surface with the smallest distance from the point cloud 1 as the candidate surface corresponding to the point cloud 1.

Thus, based on the projection method in S108, the electronic device may acquire a candidate surface corresponding to each RGB point cloud of the first object. In one embodiment, the electronic device may store a correspondence of each RGB point cloud of the first object with the second face index of the candidate surface.

In one embodiment, if all candidate surfaces are directly used as optimization variables, the solution space becomes very large by solving the following equation one, the complexity of model solution is very high, and convergence is very difficult in many cases. Therefore, in the embodiment of the application, the dimension optimization variable can be raised first, so that the solution space is reduced, and the efficiency of optimizing and solving is improved, namely, the electronic equipment executes S109-S111.

S109, constructing a topological relation, wherein the topological relation comprises the following steps: point-to-point, point-to-plane, plane-to-volume, topological relationships.

It should be understood that S108 and S109 are not sequentially distinguished, and may be performed simultaneously.

In one embodiment, based on the description in S107 and the principle that the points form sides, sides form faces, and face forms, the electronic device may further obtain an abutment face of each candidate surface, and further obtain a polyhedron formed by the candidate surface and the abutment face based on the abutment face of the candidate surface. In this embodiment, the electronic device may index the polyhedron to obtain the second body index. It should be appreciated that the second volume index may correspond to a second face index, a second side index, and a second point index of the plurality of candidate faces comprising the polyhedron. In one embodiment, the electronic device may obtain an adjacency relationship for each polyhedron based on coplanar polyhedrons.

In this embodiment, the electronic device may index all constructed faces based on each polyhedron, index all constructed edges, and index all constructed points to construct a topology containing "point-to-point, point-to-face, face-to-body", and second point index according to the second face index, the second edge index, and the second point index corresponding to the second body index. For example, referring to fig. 8C, for polyhedron 1, the electronic device may index the second volume of polyhedron 1, e.g., the second volume of polyhedron 1 indexes all constructed faces (face 1, face 2, face 3, etc.), each indexing all constructed edges (e.g., edges 1, 2, 3, and 4 of the face 1 index), each indexing all constructed points (e.g., points 1 and 2 of edge 1 multi-index). By way of example, the topological relationships are those of the "polyhedron 1", "face 1, face 2, face 3, etc", "side 1, side 2, side 3, side 4, etc", and the "point 1, point 2, etc" members.

As shown in fig. 8C, the polyhedron adjacent to the polyhedron 1 includes a polyhedron 2.

In one embodiment, the electronic room may construct a topological relationship between "point-point, point-plane, plane-body" in a topological combination diagram (combinatorial maps) based on the planar polygons cut after spatial intersection.

S110, acquiring all the minimum polyhedrons contained in the first object according to the topological relation.

The topological relation includes a topological relation among points, surfaces and volumes, and in this embodiment, it may be determined according to the topological relation, which surfaces are configured as spatial minimum polyhedrons, for example, the electronic device may obtain all minimum polyhedrons (linear cell complex) included in the first object through global calculation, that is, the first polyhedron described below, and the first polyhedron in the object may be multiple.

S111, obtaining a polyhedron corresponding to the RGB point cloud of the first object according to the candidate surface corresponding to the RGB point cloud of the first object and the candidate surface in each polyhedron in the first object.

According to the description in S108, the electronic device may obtain the candidate surface corresponding to the RGB point cloud, and according to the description in S109-S110, the electronic device may obtain the topological relation between the polyhedron and the candidate surface, so in S108-S111, the electronic device may obtain the polyhedron corresponding to the RGB point cloud, that is, the electronic device may construct the mapping relation between the RGB point cloud and the polyhedron.

In one embodiment, taking an example that a point cloud in the RGB point clouds includes a first RGB point cloud, a candidate surface corresponding to the first RGB point cloud may be referred to as a first candidate surface, and a polyhedron corresponding to the first RGB point cloud may be referred to as a first polyhedron. In one embodiment, the first RGB point cloud may be referred to as first data, where the first data is included in the data to be processed, i.e. the first RGB point cloud is included in the RGB point cloud. For example, the candidate surface corresponding to the RGB point cloud is the surface 1, and the surface 1 is the surface in the polyhedron 1, so the polyhedron corresponding to the RGB point cloud is the polyhedron 1.

In summary, the electronic device may obtain a polyhedron corresponding to the RGB point cloud of the first object, but in the embodiment of the present application, a building is divided into a plurality of polyhedrons, where the polyhedron may be a polyhedron inside the building or an polyhedron outside the building, and in 3D modeling, a set of polyhedrons outside the building is seen by a 3D model user of the built building. Therefore, the electronic device in the embodiment of the application needs to distinguish the polyhedron into an internal polyhedron or an external polyhedron of the building so as to construct a 3D model of the building. That is, the electronic device needs to perform S112-S115.

It should be understood that the first object is constituted by a plurality of polyhedrons, and the following embodiments will be described by taking, as an example, the processing steps in which the first polyhedron represents each polyhedron:

s112, obtaining a normal vector of the RGB point cloud corresponding to the first polyhedron and the correlation between the central point of the first polyhedron and the vector of the RGB point cloud.

If the polyhedron corresponding to the RGB point cloud of the first object is obtained in S111, the object may include a plurality of first polyhedrons, and accordingly, one first polyhedron may correspond to a plurality of RGB point clouds. The following description will be given by taking a first polyhedron as an example, where the first polyhedron obtains a normal vector of an RGB point cloud in the first polyhedron, and a correlation between a center point of the first polyhedron and a vector of the RGB point cloud:

referring to fig. 8D, if the RGB point clouds corresponding to the first polyhedron are point clouds P1 and P2, for example, the electronic device may obtain a normal vector n1 of the point cloud P1 and a vector m1 from a center point C of the first polyhedron (i.e. a position where a center of the first polyhedron is located) to the point cloud P1, so as to obtain a correlation between the normal vector n1 and the vector m 1. For the point cloud P2, for example, the electronic device may obtain a normal vector n2 of the point cloud P2 and a vector m2 from a center point of the first polyhedron (i.e. a position where a center of the first polyhedron is located) to the point cloud P2, so as to obtain a correlation between the normal vector n2 and the vector m 2.

The correlation may include a positive correlation and a negative correlation. Illustratively, as normal vector n1 is positively correlated with vector m1, normal vector n2 is negatively correlated with vector m 2.

S113, according to the normal vector of the RGB point cloud corresponding to the first polyhedron and the correlation between the central point of the first polyhedron and the vector of the RGB point cloud, the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron is obtained.

The step of S112 is described above with reference to the first polyhedral corresponding point clouds P1 and P2 in fig. 8D. Because the first polyhedron may correspond to multiple point clouds, the electronic device may obtain multiple correlations of the first polyhedron (i.e., the correlation of the normal vector of each RGB point cloud to which the first polyhedron corresponds and the vector of the center point of the first polyhedron to each RGB point cloud). In this embodiment of the present application, the electronic device may obtain the weight of the first polyhedron belonging to the inner polyhedron (i.e., the polyhedron inside the object) or the outer polyhedron (i.e., the polyhedron outside the object) according to the normal vector of the RGB point cloud corresponding to the first polyhedron and the correlation of the vector from the first polyhedron to the RGB point cloud (i.e., the correlation of the normal vector of each RGB point cloud corresponding to the first polyhedron and the vector from the center point of the first polyhedron to each RGB point cloud).

The following description is given by taking a normal vector of each RGB point cloud corresponding to the first polyhedron as a first normal, and taking a vector from a center point of the first polyhedron to each RGB point cloud as a second normal:

it should be understood that the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron is related to the positive and negative numbers (numbers) of correlations of the first normal and the second normal. Illustratively, the positive correlation characterizes the first polyhedron as belonging to the inner polyhedron and the negative correlation characterizes the first polyhedron as belonging to the outer polyhedron. Wherein the weight of the first polyhedron belonging to the inner polyhedron is higher if the number of positive correlations of the first normal direction and the second normal direction is larger, and the weight of the first polyhedron belonging to the outer polyhedron is higher if the number of negative correlations of the first normal direction and the second normal direction is larger.

For example, if the first polyhedron corresponds to M point clouds, the electronic device may obtain N correlations of the first polyhedron. If the number of positive correlations is a and the number of negative correlations is (M-a), in one embodiment, the electronic device may determine that the first polyhedron belongs to the outer polyhedron with a weight of (M-a)/M, and the first candidate surface belongs to the inner polyhedron with a weight of a/M.

In one embodiment, the weight of the first polyhedron belonging to the inner or outer polyhedron may be referred to as the weight of the inner and outer labels of the first polyhedron. If the weight of the first polyhedron belonging to the inner polyhedron can represent the weight of the inner label of the first polyhedron, the weight of the first polyhedron belonging to the outer polyhedron can represent the weight of the outer label of the first polyhedron. Referring to fig. 9, in fig. 9, an inner polyhedron is represented by a white area, an outer polyhedron is represented by a shadow area, a point cloud corresponding to a first polyhedron is represented by dots, and a normal vector of the point cloud corresponding to the first polyhedron is represented by arrows.

S114, determining an inner label and an outer label of the first polyhedron according to the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron and the adjacent area between the first polyhedron and the adjacent polyhedron of the first polyhedron, wherein the inner label and the outer label represent the first polyhedron as the outer polyhedron or the inner polyhedron.

In this embodiment of the present application, the electronic device may use a weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron as a node of the graph structure, and use an adjacent area between the first polyhedron and the polyhedron adjacent to the first polyhedron as an edge of the graph structure, to perform the graph structure solving, and obtain the inner and outer labels of the first polyhedron. It should be appreciated that the graph structure needs to keep the energy expended to a minimum. In one embodiment, the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron may be used as a node of the graph structure, the adjacent area between the first polyhedron and the polyhedron adjacent to the first polyhedron is used as an edge of the graph structure, and a graph-cut (graph-cut) method is used to solve the graph structure so as to obtain the inner label and the outer label of the first polyhedron.

In an embodiment, in the embodiment of the present application, the electronic device may use weights of internal and external labels of all first polyhedrons in the object as nodes of a graph structure, use an adjacent area between the first polyhedron and an adjacent polyhedron as an edge of the graph structure, and use a maximum flow algorithm or a minimum cut algorithm, and use a graph-cut method to solve the graph structure, so as to obtain the internal and external labels of the first polyhedron.

In one embodiment, the electronic device may perform the graph structure solution using equation one as follows:

e= ΣD (x) +λ ΣS (x, y) equation one

Wherein Σd (x) is Data term, a cost function characterizing the first polyhedron being divided into an inner or an outer polyhedron, λ Σs (x, y) is Smoothness term, a surface area cost function characterizing the sum of energy between each two adjacent first polyhedrons, E. The final optimization objective of the graph structure solution is to solve for the polyhedral label that minimizes E (i.e., labeled as an inside-outside label scheme). According to the graph structure optimization theory, the method of maximum flow/minimum cut is adopted to solve the formula I.

Wherein the inside-outside label characterizes the first polyhedron as an external polyhedron (i.e., an external polyhedron) or an internal polyhedron (i.e., an internal polyhedron) of the object. For example, if the inner and outer labels of the first polyhedron are inner labels, the first polyhedron is characterized as a polyhedron inside the first object, and if the inner and outer labels of the first polyhedron are outer labels, the first polyhedron is characterized as a polyhedron outside the first object.

S115, according to the inner and outer labels of all the first polyhedrons in the first object, the adjacent surfaces of the inner polyhedron and the outer polyhedron are used as the 3D model surfaces of the first object, and a 3D model of the first object is constructed.

In one embodiment, after acquiring the internal and external labels of all the first polyhedrons in the first object, the electronic device may use the surfaces of the internal polyhedron and the external polyhedron adjacent to each other as the 3D model surface of the first object, and further construct the 3D model of the first object based on the surfaces of the internal polyhedron and the external polyhedron adjacent to each other. Referring to fig. 10, an inner polyhedron included in the first object is characterized by a white region, an outer polyhedron is characterized by a shadow region, and the electronic device may stitch the surfaces of the 3D model of the first object with the surfaces of the inner polyhedron and the outer polyhedron abutted as the 3D model surfaces of the first object (as thick solid lines in fig. 10) to construct a 3D model.

In one embodiment, after the electronic device obtains the 3D model of the first object, S116 may be further performed:

s116, the electronic device outputs the 3D model of the first object.

For example, after obtaining the 3D models of all the first objects in the target area, the electronic device may output the 3D models of the first objects on an interface of the electronic device, so that the user may see the constructed 3D models of the first objects of the target area on the electronic device. Alternatively, in one embodiment, the electronic device may output a 3D closed surface geometry model data file of the first object, as well as a 3D model contour structure line file. In one embodiment, the 3D closed surface geometric model data file includes: all surface information in the 3D model, and information of vertices of the surface. The 3D model contour structure line file comprises the following steps: information of points and lines of the contour structure of the 3D model.

In the embodiment of the application, before the RGB point cloud is classified, on one hand, multi-scale space sampling can be carried out on the RGB point cloud, global information and local information of the RGB point cloud can be ensured to be obtained through sampling, and further accuracy of semantic segmentation of the RGB point cloud is improved, on the other hand, the sampling result of each scale can be regularized by combining with the terrain information of the area where the RGB point cloud belongs, the problem that fitting errors of a neural network model are large due to large terrain information difference is avoided, and accuracy of semantic segmentation of the neural network model is improved.

In addition, in the embodiment of the application, the model simplification reconstruction is performed on the first object by combining plane shape extraction, space intersection calculation, space plane projection, polyhedral topology construction, surface optimization calculation generation model and the like, so that the accuracy of the 3D model can be improved, and a model of LOD3 level can be obtained.

The above-mentioned method for constructing a 3D model provided in the embodiment of the present application is illustrated by using RGB point clouds in fig. 1, and the following method for constructing a 3D model provided in the embodiment of the present application is illustrated by using Mesh grids as an example. Referring to fig. 2, the 3D model construction method provided in the embodiment of the present application may include:

s201, responding to a construction instruction of the 3D model, and performing multi-scale space sampling on a Mesh grid to be constructed of the 3D model to obtain sampling results of at least two scales.

In one embodiment, the Mesh grid to be modeled in 3D may be referred to as an initial Mesh grid.

In one embodiment, the electronic device may divide the Mesh grid by using the vertices in the Mesh grid as the sphere centers and drawing spheres with different radii, so as to obtain the result of dividing the spheres with different sizes. The following describes a process of multi-scale spatial sampling of Mesh grids to be constructed into a 3D model by electronic equipment:

1. and cutting the Mesh grid by adopting the spheres with the first scale. The vertexes of the Mesh grid are arranged in a three-dimensional space, the Mesh grid is segmented by adopting spheres of a first scale, and each sphere comprises a plurality of vertexes. In one embodiment, one vertex (may be referred to as a center vertex) may be sampled in each sphere, the center vertex may be a center vertex of all vertices contained in the sphere, for example, three-dimensional coordinates of the center vertex may be a mean of three-dimensional coordinates of all vertices in the sphere, and color information (i.e., RGB values) of the center vertex may be a mean of RGB values of all vertices in the sphere. In one embodiment, the point cloud sampled in each sphere may be referred to as a first center vertex.

Thus, the electronic device can obtain the sampling result of the first scale. The sampling result of the first scale may include: the first center vertex in each grid (including the three-dimensional coordinates and color information of the first center point cloud).

2. And cutting the Mesh grid by adopting spheres with a second scale. The second scale is larger than the first scale, in this embodiment, a vertex may be sampled in each sphere, the three-dimensional coordinates of the vertex may be the average of the three-dimensional coordinates of all vertices in the sphere, and the color information (i.e., RGB values) of the vertex may be the average of the RGB values of all vertices in the sphere. In one embodiment, the sampled vertices in each sphere may be referred to as the second center vertices.

Thus, the electronic device can obtain the sampling result of the second scale. The sampling result of the second scale may include: a second center vertex in each sphere (including three-dimensional coordinates and color information of the second center vertex).

3. And so on, until the Mesh grid is segmented by adopting the sphere of the nth scale, the related description in the above description of the Mesh grid segmented by adopting the sphere of the first scale and the Mesh grid segmented by adopting the sphere of the second scale can be referred to. Thus, the electronic device can obtain the sampling result of the Nth scale. The sampling result of the nth scale may include: an nth center vertex in each sphere (including three-dimensional coordinates and color information of the nth center vertex).

Thus, the electronic device can obtain sampling results of N scales.

In one embodiment, steps A-B may be summarized as steps 1-3 above:

and A, taking the vertexes of the initial Mesh as sphere centers, dividing the initial Mesh by spheres with the radius of the ith scale, and obtaining an ith vertex in each sphere to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith vertex, i is an integer which is greater than or equal to 1 and is less than N, and N is an integer which is greater than 1.

S202, regularizing the sampling result of each scale by combining the topographic information of the area of the Mesh grid to be constructed into a 3D model to obtain the sampling result of at least two scales after the processing.

In one embodiment, taking the Mesh grid area as the area 1, for example, referring to fig. 4, the area 1 is formed with a volt, and the area 1 may be divided into a plurality of grids, where each grid corresponds to a two-dimensional coordinate range and an altitude value. The range of two-dimensional coordinates corresponding to one grid is shown as: in the embodiment of the present application, the electronic device may subtract the altitude value corresponding to the grid from the Z value of the vertex of the Mesh grid whose two-dimensional coordinates fall within the range of the two-dimensional coordinates corresponding to the grid, so as to complete the regularization processing of the Mesh grid falling into the grid.

Illustratively, the range of two-dimensional coordinates corresponding to the grid is: (X1, Y1), (X2, Y2) to (X3, Y3), (X4, Y4), the altitude value corresponding to the grid being Z _H1 In the sampling result of the first scale, vertices falling within the "range of two-dimensional coordinates" include vertices of Mesh grid 1 and vertices of Mesh grid 2. The electronic device may subtract Z from both the vertices of Mesh grid 1 and Mesh grid 2 _H1 As the Z values of the vertices of Mesh grid 1 and the vertices of Mesh grid 2 after regularized processing. In one embodiment, if the vertices of one Mesh grid fall into two grids, the altitude value of the corresponding grid may be subtracted from the Z value of the vertex of the Mesh grid.

S203, inputting the processed sampling results of at least two scales into a pre-trained neural network model to obtain a Mesh grid carrying object labels.

It should be understood that the neural network model used in S203 is different from the neural network model used in S103, and the neural network model used in S203 can distinguish Mesh grids of different kinds of objects. The training process of the neural network model used in S203 may refer to the process of the neural network model used in S103, which is not described in detail in the embodiment of the present application. The neural network model in S203 is used to classify Mesh grids, and obtain Mesh grids belonging to the object. By way of example, the object is a building, the neural network model is used to distinguish Mesh grids of the building from non-building, and Mesh grids belonging to the building are labeled "building",

S204, dividing the objects according to the Mesh grids carrying the object labels to obtain Mesh grids corresponding to each object.

In one embodiment, the Mesh grid corresponding to the object obtained in S204 may be referred to as data to be processed.

Unlike S104, the electronic device in S204 may perform cluster analysis on vertices (corresponding to Mesh grids) of the Mesh grid carrying the object tag by using an unsupervised method using the spatial distance as a threshold, so as to cluster the point clouds belonging to the same building together, and obtain the Mesh grid corresponding to each individual object.

Or in one embodiment, therefore, in one embodiment, the electronic device may perform cluster analysis on vertices (corresponding to Mesh grids) of the Mesh grid carrying the object tag by using an unsupervised method with a spatial distance as a threshold and a density as a threshold, so as to cluster point clouds belonging to the same building together, so as to obtain a Mesh grid corresponding to each object individually.

S205, obtaining plane information of the first object according to the Mesh grid of the first object.

In this embodiment of the present application, the electronic device may use a plane where the coplanar Mesh grid is located as a plane according to an adjacency topology relationship of the Mesh grid, so as to obtain plane information of the first object.

S205 may refer to the related description in fig. 7C in S105.

S206, performing space intersection calculation on each plane agent of the first object to obtain a plurality of intersection lines.

S207, constraint triangulation calculation is carried out on each space plane formed by the intersecting lines by taking the intersecting lines as boundaries, and candidate surfaces are obtained.

S206-S207 may refer to the relevant description of S106-S107.

And S208, projecting the vertexes of the Mesh grid of the first object to the candidate surfaces of the first object to obtain the candidate surfaces corresponding to the Mesh grid of the first object.

The specific embodiment in S208 may refer to the related description in S108, and the vertex of the Mesh grid of the first object may be regarded as the RGB point cloud of the first object.

S209, constructing a topological relation, wherein the topological relation comprises: point-to-point, point-to-plane, plane-to-volume, topological relationships.

S210, acquiring all the minimum polyhedrons contained in the first object according to the topological relation.

S211, obtaining a polyhedron corresponding to the Mesh grid of the first object according to the candidate surface corresponding to the Mesh grid of the first object and the candidate surface in each polyhedron in the first object.

S209-S211 may refer to the related description in S109-S111, such as the vertices of the Mesh grid of the first object may be regarded as the RGB point cloud of the first object.

In one embodiment, taking a Mesh grid of the first object as an example, a candidate surface corresponding to the first Mesh grid may be referred to as a first candidate surface, and a polyhedron corresponding to the first Mesh grid may be referred to as a first polyhedron. In one embodiment, the first Mesh grid may be referred to as first data, which is included in the data to be processed, i.e. the first Mesh grid is included in the Mesh grid of the first object.

S212, obtaining a normal vector of the Mesh grid corresponding to the first polyhedron and correlation between a central point of the first polyhedron and the vector of the Mesh grid.

In one embodiment, because the Mesh grid is a polygon, the normal vector of the Mesh grid may be the normal vector of the polygon. The vector from the center point of the first polyhedron to the Mesh grid may be: a vector from the center point of the first polyhedron to the center point of the Mesh grid may refer to the description of "the center point of the first polyhedron to the RGB point cloud". Alternatively, in one embodiment, the vector from the center point of the first polyhedron to the Mesh grid may be: a vector from the center point of the first polyhedron to any vertex of the Mesh grid.

S213, according to the normal vector of the Mesh grid corresponding to the first polyhedron and the correlation between the central point of the first polyhedron and the vector of the Mesh grid, the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron is obtained.

S214, determining an inner label and an outer label of the first polyhedron according to the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron and the adjacent area between the first polyhedron and the adjacent polyhedron of the first polyhedron, wherein the inner label and the outer label represent the first polyhedron as the outer polyhedron or the inner polyhedron.

S215, according to the inner and outer labels of all the first polyhedrons in the first object, the adjacent surfaces of the inner polyhedron and the outer polyhedron are used as the 3D model surfaces of the first object, and a 3D model of the first object is constructed.

S213 to S215 may refer to the related description in S113 to S115. After the electronic apparatus performs S215, S116 described above may be performed.

In this way, in the embodiment of the present application, a 3D model may be constructed based on the initial Mesh grid, and the technical effects described in fig. 1 may also be achieved, which may specifically refer to the related description of fig. 1, and will not be described herein.

From the above description of the embodiments of fig. 1 and 2, the flow shown in fig. 1 and 2 may be simplified to that shown in fig. 11, with the understanding that fig. 11 is an illustration of a carding process of an RGB point cloud. Referring to fig. 11, a method for constructing a 3D model provided in an embodiment of the present application may include: semantic segmentation, instance segmentation, plane shape extraction, space intersection calculation, space plane projection, polyhedral topology construction, surface optimization calculation, model generation and the like. In the above description, S101-S103, S201-S203 are "semantic segmentation", S104, S204 are "instance segmentation", S105, S205 are "planar shape extraction", S106-S107, S206-S207 are "space intersection calculation", S108, S208 are "space planar projection", S109-S111, S209-S211 are "polyhedral topology construction", and S112-S115, S212-S215 are "surface optimization calculation generation model".

In one embodiment, S105-S115, S205-S215 may be referred to as a "model reduced reconstruction" process.

Fig. 12 is a schematic diagram of a training neural network model according to an embodiment of the present application. In one embodiment, the device for training the neural network model may be referred to as a training device, which may be a computer, a dedicated neural network model training machine, or the like. After the training device trains to obtain the neural network model, the neural network model can be preset in the electronic device.

In the following, a training device is described briefly by taking a neural network model obtained by training a U-Net network architecture, which includes an input layer, at least one hidden layer, and an output layer. The input layer is used for receiving training data and distributing the training data to neurons of the hidden layer. Neurons of the hidden layer are used for calculating according to the data, and the calculation result is output to the output layer. The output layer is used for outputting the operation result.

Referring to fig. 12, the process of training the neural network model by the training device may include:

and S1201, initializing the weight value of each hidden layer neuron in the U-NET network architecture.

For example, the training device may initialize the weight values of neurons of each hidden layer to a random weight value that is a gaussian distribution.

S1202, dividing training data into N batches.

The training data is an RGB point cloud containing labels of objects.

The training device may divide the training data into N batches to iteratively train the U-NET network architecture using the N batches of data. For example, the training device may divide the training data into N batches. Wherein N is an integer greater than 1.

S1203, inputting the data of the ith batch into a U-NET network architecture to obtain the cross entropy loss of the data of the ith batch.

Illustratively, at the beginning of training, i.e., when i is 1, the training device inputs the 1 st lot of data into the U-NET network architecture, which may output a cross entropy loss of the 1 st lot of data (cross entropy loss). It should be appreciated that cross entropy loss is used to characterize the similarity of the object to which the RGB point cloud belongs and the object to which the RGB point cloud belongs that the training device predicts using the U-NET network architecture. The smaller the cross entropy loss, the more accurate the weight values characterizing neurons of each hidden layer in the U-NET network architecture. Wherein i is an integer greater than or equal to 1 and less than or equal to N.

And S1204, updating the weight value of each hidden layer neuron in the U-NET network architecture according to the cross entropy loss of the data of the ith batch.

Illustratively, the training device may update the initial weight values of neurons of each hidden layer based on cross entropy loss of lot 1 data. The training device can determine the error between the similarity of the object to which the RGB point cloud belongs and the object to which the RGB point cloud belongs, which is predicted by adopting the U-NET network architecture, and 100% according to the cross entropy loss of the data of the 1 st batch, and further update the weight value of the neuron of each hidden layer according to the error. For example, the training device may update the weights of neurons of each hidden layer using a gradient descent method (gradient descent) or a random gradient descent method (stochastic gradient decent).

S1205, judging whether i is smaller than N. If yes, add 1 to i, and execute S1203. If not, execution proceeds to S1206.

The training device may determine whether i is less than N to determine whether training of N batches of data of training data is complete. If i is less than N, the training device may add 1 to i, and then continue to S1203. If i is 1 and N is 10, the training device determines that i is smaller than N, and then inputs the data of the 2 nd batch into the U-NET network architecture with updated weight value, so as to obtain the cross entropy loss of the data of the 2 nd batch. Similarly, the training device may update the weight value of the neurons of each hidden layer by using a gradient descent method or a random gradient descent method according to the cross entropy loss of the data of the 2 nd batch. And iterating continuously until i is equal to N, inputting the data of the nth batch into the U-NET network architecture by the training equipment, and updating the weight value of the neuron of each hidden layer by the training equipment according to the cross entropy loss of the data of the nth batch. Wherein, when i is equal to N, the training device may perform S1206 described below.

S1206, determining whether training is converged according to the target cross entropy loss and the cross entropy loss of the data of the nth batch. If the training is converged, S1207 is executed, and if the training is not converged, S1202 is returned to.

The user can preset the target cross entropy loss as the convergence basis of training. When the training equipment obtains the cross entropy loss of the data of the nth batch, whether the training is converged or not can be determined according to the cross entropy loss of the data of the nth batch and the target cross entropy loss. If the cross entropy loss of the data of the nth batch is less than or equal to the target cross entropy loss, the object which is predicted according to the adoption of the U-NET network architecture and belongs to the RGB point cloud is characterized to be close to the object which is predicted to be belongs to the RGB point cloud, and the training equipment can determine training convergence. If the cross entropy loss of the data of the nth batch output by the U-NET network architecture is larger than the target cross entropy loss, determining that training is not converged. And when the training is not converged, the training equipment returns to execute S102, namely the preprocessed training data is divided into N batches, and the U-NET network architecture is continuously trained by adopting the N batches of data until the training is converged.

S1207, training ends.

And if the training equipment determines that the training converges, ending the training to obtain the neural network model. The neural network model may be: and the training equipment updates the U-NET network architecture after the weight values of the neurons of each hidden layer according to the cross entropy loss of the data of the nth batch.

In one embodiment, after the training device may preprocess the training data, the training data may be input to the U-NET network architecture, and the preprocessing process of the training data by the training device may refer to the descriptions in S101-S102, and it should be noted that, in the process of training the training device to obtain the neural network model, the training device may extract features of the training data, and extracting features of the training data may refer to the related descriptions in S103, which will not be described herein.

In an embodiment, the 3D model construction method provided in the embodiment of the present application may be applied to a 3D model construction scenario of a campus (e.g., a factory park), where, because an area of an application scenario of the park level is smaller than that of a scenario of the city level, and a corresponding topographic relief change is not large, when the electronic device performs the steps S101-S115 (or S201-S215) as described above, the electronic device may not perform S102 (or S202), that is, need not combine topographic information of an area to which an RGB point cloud of the 3D model to be constructed belongs, to perform regularization processing on a sampling result of each scale.

In one embodiment, the 3D model construction method provided in the embodiment of the present application may be applied to a 3D model construction scene of an indoor scene. Since the indoor application scene is more complex than the object class of the city-level scene and the indoor geometric feature is relatively simpler than the city-level outdoor scene, in the embodiment of the present application, the electronic device may perform steps such as S101-S115 (or S201-S215) directly S105-S115 (or S205-S215) instead of S101-S104 (S201-S204) to construct the indoor structured 3D modeling.

It should be understood that the 3D model building method provided in the embodiment of the present application may also be applied to other scenarios, where the electronic device may perform part or all of the steps in S101-S115 (or S201-S215) in an applicable manner, which all fall within the protection scope of the embodiment of the present application.

In summary, in one embodiment, referring to fig. 13, the method for constructing a 3D model provided in the embodiment of the present application may include:

s1301, extracting a plurality of planes of an object from data to be processed, wherein the object is an object of a 3D model to be constructed, and the data to be processed is RGB point cloud or Mesh grid.

The data to be processed is an RGB point cloud or Mesh grid, and the RGB point cloud or Mesh grid of the object obtained in S104 and S204 is the data to be processed. The process of extracting the plurality of planes of the object from the data to be processed may refer to the related descriptions in S105 and S205.

S1302, carrying out space intersection calculation on each plane in pairs to obtain a plurality of candidate surfaces of the object and at least two polyhedrons formed by the plurality of candidate surfaces.

And S1303, projecting the data to be processed to each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data, wherein the first data is contained in the data to be processed, when the data to be processed is RGB point cloud, the first data is the first RGB point cloud, when the data to be processed is Mesh grid, the first data is the first Mesh grid, the first candidate surface is contained in a plurality of candidate surfaces, and the first candidate surface is the surface of the first polyhedron.

S1304, obtaining an inner label and an outer label of the first polyhedron according to the normal vector of the first data and the vector from the central point of the first polyhedron to the first data, wherein the inner label and the outer label represent the first polyhedron as the polyhedron outside or the polyhedron inside the object.

S1302-S1302 may refer to the relevant descriptions in S106-S114 and S206-S214.

S1305, constructing a 3D model of the object according to the inner and outer labels of the first polyhedron.

In one embodiment, the electronic device may output the 3D model of the object after constructing the 3D model of the object.

S1302 to S1302 may refer to the related descriptions in S115 and S215.

In one embodiment, the device to be processed is obtained after the electronic device processes the initial data. The initial data includes an initial RGB point cloud or an initial Mesh grid of at least one object, and the initial data may include: and carrying out multi-scale space sampling on the initial data to obtain sampling results of at least two scales, and carrying out regularization processing on the sampling results of each scale by combining the topographic information of the area to which the initial data belongs to obtain the processed sampling results of at least two scales. The electronic device can input the processed sampling results with at least two scales into a pre-trained neural network model to obtain data carrying the object tag, and perform object segmentation on the data carrying the object tag to obtain data to be processed.

In one embodiment, the process of processing the initial data by the electronic device may refer to the relevant descriptions in S101-S104, and S201-S204.

In this way, in the embodiment of the present application, a 3D model may be constructed based on an initial Mesh grid or an initial RGB point cloud, and the technical effects described in the foregoing fig. 1 may also be achieved, which may specifically refer to the related description in fig. 1, and will not be described herein in detail.

Fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 14, an electronic device 1400 may include: a 3D semantic segmentation module 1401, an instance segmentation module 1402, a planar feature extraction module 1403, a spatial intersection computation module 1404, a polyhedral topology construction module 1405, and a surface optimization generation module 1406.

The 3D semantic segmentation module 1401 is configured to identify and extract point clouds of different categories, such as point clouds of buildings, roads, vegetation, and the like. Specifically, the 3D semantic segmentation module is configured to execute S101-S103 (or S201-S203) in the above embodiment.

An instance segmentation module 1402 is configured to extract a point cloud or Mesh grid of a single object from an existing single class point cloud. Specifically, the instance segmentation module is configured to execute S104 or S204 in the above embodiment.

The plane feature extraction module 1403 is configured to extract a plane shape from the building point cloud or Mesh grid data. Specifically, the planar feature extraction module may perform S105 or S205 in the above embodiment.

A spatial intersection calculation module 1404, configured to perform 3D spatial intersection calculation using the extracted planar shape, and generate a planar polygon. Specifically, the spatial intersection calculation module may perform S106-S108 (or S206-S208) in the above embodiments.

A polyhedral topology construction module 1405 for generating a space-minimum polyhedron using planar polygons. Specifically, the polyhedral topology construction module may perform S109 to S111 (or S209 to S211) in the above embodiment.

A surface optimization generation module 1406 extracts simplified model surfaces from the polyhedron to construct a 3D model. Specifically, the surface optimization generation module may perform S112-S115 (or S212-S215) as in the above embodiments.

In one embodiment, the planar feature extraction module, the spatial intersection computation module, the polyhedral topology construction module, and the surface optimization generation module may be integrated as a single body, such as a building model reduction reconstruction module for performing S105-S115 (or S205-S215) in the above embodiments.

Specifically, in one embodiment, the plane feature extraction module 1403 is configured to extract a plurality of planes of an object from data to be processed, where the object is an object to be constructed into a 3D model, and the data to be processed is an RGB point cloud or Mesh grid.

The space intersection calculation module 1404 is configured to perform a two-by-two space intersection calculation on each plane to obtain a plurality of candidate surfaces of the object, and at least two polyhedrons formed by the plurality of candidate surfaces.

The polyhedral topology construction module 1405 is configured to project data to be processed onto each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data, where the first data is included in the data to be processed, the first data is a first RGB point cloud when the data to be processed is an RGB point cloud, the first data is a first Mesh grid when the data to be processed is a Mesh grid, the first candidate surface is included in the plurality of candidate surfaces, and the first candidate surface is a surface of the first polyhedron.

The surface optimization generating module 1406 is configured to obtain an internal and external label of the first polyhedron according to the normal vector of the first data and the vector from the center point of the first polyhedron to the first data, and construct a 3D model of the object according to the internal and external label of the first polyhedron. Wherein the internal and external labels characterize the first polyhedron as an external polyhedron or an internal polyhedron of the object.

In one possible implementation, the plane feature extraction module 1403 is specifically configured to take a plane in which coplanar data to be processed is located as a plane, so as to obtain multiple planes.

When the data to be processed is an RGB point cloud, in one possible implementation manner, the plane feature extraction module 1403 is specifically configured to take a plane where the co-planar RGB point cloud is located as a plane according to a spatial position relationship of the RGB point cloud.

When the data to be processed is a Mesh grid, in a possible implementation manner, the plane feature extraction module 1403 is specifically configured to take, as a plane, a plane where the coplanar Mesh grid is located according to an adjacency topology relationship of the Mesh grid.

In one possible implementation manner, the spatial intersection calculation module 1404 is specifically configured to perform two-to-two spatial intersection calculation on each plane to obtain a plurality of intersection lines; taking the intersecting lines as boundaries, and performing constraint triangulation calculation on each space plane formed by the intersecting lines to obtain a plurality of candidate surfaces; at least two polyhedrons are obtained according to the adjacent relation of each candidate surface.

In one possible implementation, the polyhedral topology construction module 1405 is specifically configured to project data to be processed onto each candidate surface, where when the data to be processed is a Mesh, the projecting is performed as vertices of the Mesh; and determining the first candidate surface corresponding to the first data according to the distance between the first data and each candidate surface and the position relation between the projection point of the first data and each candidate surface.

In one possible implementation, the polyhedral topology construction module 1405 is specifically configured to take, as the first candidate surface corresponding to the first data, a candidate surface closest to the first data and into which the projection point of the first data falls.

In one possible implementation manner, the surface optimization generating module 1406 is specifically configured to obtain weights of internal and external labels of the first polyhedron according to a normal vector of the first data and a correlation between a center point of the first polyhedron and a vector of the first data, where when the first data is the first Mesh, the vector from the center point of the first polyhedron to the first data is a vector from the center point of the first polyhedron to the center point of the first Mesh; and taking the weights of the inner and outer labels of all the first polyhedrons contained in the object as nodes of the graph structure, and taking the adjacent area between the first polyhedron and the adjacent polyhedron as an edge of the graph structure to acquire the inner and outer labels of the first polyhedron.

In one possible implementation, the first polyhedron corresponds to a plurality of first data, and the correlation includes a positive correlation and a negative correlation. The surface optimization generating module 1406 is specifically configured to obtain a normal vector of each first data and a correlation between a center point of the first polyhedron and a vector of each first data corresponding to the first polyhedron; and acquiring the weight of the inner and outer labels of the first polyhedron according to the number of positive correlations and the number of negative correlations.

In one possible implementation manner, the surface optimization generating module 1406 is specifically configured to obtain the inner and outer labels of the first polyhedron by using weights of the inner and outer labels of all the first polyhedrons in the object as nodes of the graph structure, using an adjacent area between the first polyhedron and an adjacent polyhedron as an edge of the graph structure, and adopting a maximum flow algorithm or a minimum cut algorithm.

In one possible implementation, the surface optimization generation module 1406 is specifically configured to take an adjacent surface of an internal polyhedron and an external polyhedron in the object as a surface of the object; from the surface of the object, a 3D model of the object is constructed.

In one possible implementation, the data to be processed includes an RGB point cloud or Mesh grid of the object, or the data to be processed is an RGB point cloud or Mesh grid of the object.

When the data to be processed is an RGB point cloud or a Mesh grid of an object, in one possible implementation manner, the 3D semantic segmentation module 1401 and the instance segmentation module 1402 are configured to obtain the data to be processed from initial data, where the initial data includes at least one initial RGB point cloud or an initial Mesh grid of the object.

In one possible implementation manner, the 3D semantic segmentation module 1401 is specifically configured to perform multi-scale spatial sampling on initial data to obtain sampling results of at least two scales, perform regularization processing on the sampling results of each scale in combination with topographic information of an area to which the initial data belongs to obtain sampling results of at least two scales after processing, and input the sampling results of at least two scales after processing into a neural network model trained in advance to obtain data carrying a label of an object.

The instance segmentation module 1402 is specifically configured to perform object segmentation on data carrying a tag of an object to obtain data to be processed.

In one possible implementation, when the initial data is an initial RGB point cloud, the 3D semantic segmentation module 1401 is specifically configured to perform step a and step B:

a, segmenting an initial RGB point cloud by adopting an ith scale grid, and acquiring an ith center point cloud in each grid to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith center point cloud, the ith center point cloud is a center point of the point clouds contained in the grids, i is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1.

In one possible implementation, when the initial data is an initial Mesh grid, the 3D semantic segmentation module 1401 is specifically configured to perform step a and step B:

a, the vertexes of the initial Mesh grid are taken as sphere centers, the initial Mesh grid is segmented by spheres with the radius of the ith scale, and an ith vertex in each sphere is obtained to obtain a sampling result of the ith scale, wherein the sampling result of the ith scale comprises three-dimensional coordinates and color information of the ith vertex, i is an integer which is greater than or equal to 1 and is smaller than N, and N is an integer which is greater than 1.

In one possible implementation, the terrain information includes: a range of locations of a plurality of terrain grids of the area to which the initial data belongs, and an altitude value of each terrain grid.

The 3D semantic segmentation module 1401 is specifically configured to subtract the altitude value of the first terrain grid from the Z value of the initial data falling in the position range of the first terrain grid, so as to complete regularization processing on the initial data falling in the position range of the first terrain grid, where the first terrain grid is included in the plurality of terrain grids.

In one possible implementation, the surface optimization generation module 1406 is specifically configured to output a 3D model of the object.

The electronic device provided in the embodiments of the present application may implement the embodiments in fig. 1, fig. 2, and fig. 13 as described above, and reference may be made to the relevant descriptions in the embodiments in fig. 1, fig. 2, and fig. 13 as described above.

In an embodiment, referring to fig. 15, an electronic device is further provided in an embodiment of the present application, where the electronic device may be an electronic device described in the foregoing embodiment, and the electronic device may include: a processor 1501 (e.g., a CPU), a memory 1502. The memory 1502 may include a random-access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one magnetic disk memory, in which various instructions may be stored in the memory 1502 for performing various processing functions and implementing method steps of the present application.

Optionally, the electronic device related to the present application may further include: a power supply 1503, a communication bus 1504, and a communication port 1505. The communication port 1505 is used to implement connection communication between the electronic device and other peripheral devices. In the present embodiment, the memory 1502 is used for storing computer executable program code, which includes instructions; when the processor 1501 executes the instructions, the instructions cause the processor 1501 of the electronic device to perform the actions in the above-described method embodiments, and the implementation principle and technical effects are similar, which are not described herein again.

It should be noted that the modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (application specific integrated circuit, ASIC), or one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (central processing unit, CPU) or other processor that may invoke the program code, such as a controller. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.) means from one website, computer, server, or data center. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc., that contain an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

The term "plurality" herein refers to two or more. The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship; in the formula, the character "/" indicates that the front and rear associated objects are a "division" relationship. In addition, it should be understood that in the description of this application, the words "first," "second," and the like are used merely for distinguishing between the descriptions and not for indicating or implying any relative importance or order.

It will be appreciated that the various numerical numbers referred to in the embodiments of the present application are merely for ease of description and are not intended to limit the scope of the embodiments of the present application.

It should be understood that, in the embodiments of the present application, the sequence number of each process described above does not mean that the execution sequence of each process should be determined by the function and the internal logic of each process, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Claims

1. A 3D model construction method, comprising:

extracting a plurality of planes of an object from data to be processed, wherein the object is an object of a 3D model to be constructed, and the data to be processed is RGB point cloud or Mesh grid;

performing space intersection calculation on each plane to obtain a plurality of candidate surfaces of the object and at least two polyhedrons formed by the plurality of candidate surfaces;

projecting the data to be processed to each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to first data, wherein the first data is contained in the data to be processed, when the data to be processed is the RGB point cloud, the first data is a first RGB point cloud, when the data to be processed is the Mesh grid, the first data is a first Mesh grid, the first candidate surfaces are contained in the plurality of candidate surfaces, and the first candidate surfaces are the surfaces of the first polyhedron;

acquiring an inner and outer label of the first polyhedron according to the normal vector of the first data and the vector from the central point of the first polyhedron to the first data, wherein the inner and outer label represents that the first polyhedron is an external polyhedron or an internal polyhedron of the object;

And constructing a 3D model of the object according to the internal and external labels of the first polyhedron.

2. The method of claim 1, wherein extracting the plurality of planes of the object from the data to be processed comprises:

and taking the plane in which the coplanar data to be processed are located as a plane to obtain the planes.

3. The method according to claim 2, wherein when the data to be processed is the RGB point cloud, the taking the plane in which the coplanar data to be processed is located as a plane includes:

and taking the plane where the coplanar RGB point clouds are located as a plane according to the spatial position relation of the RGB point clouds.

4. The method according to claim 2, wherein when the data to be processed is the Mesh grid, the taking the plane in which the coplanar data to be processed is located as a plane includes:

and taking the plane where the coplanar Mesh grid is positioned as a plane according to the adjacent topological relation of the Mesh grid.

5. The method according to any one of claims 1-4, wherein the performing a two-by-two spatial intersection calculation on each plane to obtain a plurality of candidate surfaces of the object, and at least two polyhedrons formed by the plurality of candidate surfaces, comprises:

Performing space intersection calculation on each plane to obtain a plurality of intersection lines;

taking the intersecting lines as boundaries, and performing constraint triangulation calculation on each space plane formed by the intersecting lines to obtain a plurality of candidate surfaces;

and obtaining the at least two polyhedrons according to the adjacent relation of each candidate surface.

6. The method according to any one of claims 1-5, wherein projecting the data to be processed onto each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data includes:

projecting the data to be processed to each candidate surface, and projecting the data to be processed to be the vertexes of the Mesh grid when the data to be processed is the Mesh grid;

and determining a first candidate surface corresponding to the first data according to the distance between the first data and each candidate surface and the position relation between the projection point of the first data and each candidate surface, wherein the polyhedron to which the first candidate surface belongs is the first polyhedron.

7. The method of claim 6, wherein determining the first candidate surface corresponding to the first data according to the distance between the first data and each candidate surface and the positional relationship between the projection point of the first data and each candidate surface comprises:

And taking the candidate surface which is closest to the first data and in which the projection point of the first data falls as a first candidate surface corresponding to the first data.

8. The method of any of claims 1-7, wherein the obtaining the inner and outer labels of the first polyhedron from the normal vector of the first data and the vector from the center point of the first polyhedron to the first data comprises:

acquiring weights of internal and external labels of the first polyhedron according to a normal vector of the first data and the correlation between the central point of the first polyhedron and the vector of the first data, wherein when the first data is the first Mesh, the vector from the central point of the first polyhedron to the first data is the vector from the central point of the first polyhedron to the central point of the first Mesh;

and taking the weights of the inner and outer labels of all the first polyhedrons contained in the object as nodes of a graph structure, and taking the adjacent area between the first polyhedron and the adjacent polyhedron as an edge of the graph structure to acquire the inner and outer labels of the first polyhedron.

9. The method of claim 8, wherein the first polyhedron corresponds to a plurality of first data, and wherein the correlation comprises a positive correlation and a negative correlation; the obtaining the weight of the inner and outer labels of the first polyhedron according to the normal vector of the first data and the correlation between the central point of the first polyhedron and the vector of the first data comprises the following steps:

The method comprises the steps of obtaining a normal vector of each first data and the correlation between the central point of the first polyhedron and the vector of each first data corresponding to the first polyhedron;

and acquiring the weight of the internal and external labels of the first polyhedron according to the number of positive correlations and the number of negative correlations.

10. The method according to claim 8 or 9, wherein the obtaining the internal and external labels of the first polyhedron with weights of the internal and external labels of all the first polyhedrons contained in the object as nodes of the graph structure and with an adjacent area between the first polyhedron and an adjacent polyhedron as sides of the graph structure includes:

and taking the weights of the inner and outer labels of all the first polyhedrons in the object as nodes of a graph structure, taking the adjacent area between the first polyhedron and the adjacent polyhedron as an edge of the graph structure, and acquiring the inner and outer labels of the first polyhedron by adopting a maximum flow algorithm or a minimum cut algorithm.

11. The method according to any one of claims 1-10, wherein said constructing a 3D model of said object from the internal and external labels of said first polyhedron comprises:

taking the adjacent surfaces of the inner polyhedron and the outer polyhedron in the object as the surfaces of the object;

And constructing a 3D model of the object according to the surface of the object.

12. The method according to any one of claims 1-11, wherein the data to be processed comprises or is an RGB point cloud or Mesh grid of the object.

13. The method according to any one of claims 1-12, further comprising, when the data to be processed is an RGB point cloud or Mesh grid of the object, before extracting the plurality of planes of the object from the data to be processed:

and acquiring the data to be processed from initial data, wherein the initial data comprises an initial RGB point cloud or an initial Mesh grid of at least one object.

14. The method of claim 13, wherein the obtaining the data to be processed from the initial data comprises:

performing multi-scale space sampling on the initial data to obtain sampling results of at least two scales;

regularizing the sampling result of each scale by combining the topographic information of the area to which the initial data belongs to obtain the sampling result of at least two scales after processing;

inputting the processed sampling results of at least two scales into a pre-trained neural network model to obtain data carrying the label of the object;

And carrying out object segmentation on the data carrying the object tag to obtain the data to be processed.

15. The method of claim 14, wherein when the initial data is an initial RGB point cloud, performing multi-scale spatial sampling on the initial RGB point cloud to obtain a sampling result of at least two scales, including:

dividing the initial RGB point cloud by adopting an ith scale grid to obtain an ith center point cloud in each grid to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith center point cloud, the ith center point cloud is a center point of point clouds contained in the grids, i is an integer which is greater than or equal to 1 and is smaller than N, and N is an integer which is greater than 1;

16. The method according to claim 14, wherein when the initial data is an initial Mesh grid, performing multi-scale spatial sampling on the initial Mesh grid to obtain a sampling result of at least two scales, including:

A, taking the vertexes of the initial Mesh grid as sphere centers, dividing the initial Mesh grid by spheres with the radius of an ith scale, and obtaining an ith vertex in each sphere to obtain an ith scale sampling result, wherein the ith scale sampling result comprises three-dimensional coordinates and color information of the ith vertex, i is an integer which is greater than or equal to 1 and is less than N, and N is an integer which is greater than 1;

17. The method of any one of claims 14-16, wherein the topographical information comprises: the position ranges of a plurality of terrain grids of the area to which the initial data belong, and the altitude value of each terrain grid;

the regularization processing is carried out on the sampling result of each scale by combining the topographic information of the area to which the initial data belong, and the regularization processing comprises the following steps:

subtracting the altitude value of the first terrain grid from the Z value of the initial data falling in the position range of the first terrain grid to complete regularization processing of the initial data falling in the position range of the first terrain grid, wherein the first terrain grid is contained in the plurality of terrain grids.

18. The method according to any one of claims 1-17, wherein after constructing the 3D model of the object from the internal and external labels of the first polyhedron, further comprising:

outputting the 3D model of the object.

19. A 3D model construction apparatus, comprising:

the plane feature extraction module is used for extracting a plurality of planes of an object from data to be processed, wherein the object is an object of a 3D model to be constructed, and the data to be processed is RGB point cloud or Mesh grid;

the space intersection calculation module is used for carrying out space intersection calculation on each plane in pairs to obtain a plurality of candidate surfaces of the object and at least two polyhedrons formed by the plurality of candidate surfaces;

the polyhedron topology construction module is used for projecting the data to be processed to each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data, wherein the first data is contained in the data to be processed, when the data to be processed is the RGB point cloud, the first data is the first RGB point cloud, when the data to be processed is the Mesh grid, the first data is the first Mesh grid, the first candidate surfaces are contained in the plurality of candidate surfaces, and the first candidate surfaces are the surfaces of the first polyhedron;

The surface optimization generation module is used for acquiring an internal and external label of the first polyhedron according to a normal vector of the first data and a vector from a central point of the first polyhedron to the first data, and constructing a 3D model of the object according to the internal and external label of the first polyhedron, wherein the internal and external label represents the first polyhedron as an external polyhedron or an internal polyhedron of the object.

20. An electronic device, comprising: a processor and a memory;

the memory stores computer-executable instructions;

the processor executing computer-executable instructions stored in the memory, causing the processor to perform the method of any one of claims 1-18.

21. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program or instructions, which when executed, implement the method of any of claims 1-18.

22. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the method of any of claims 1-18.