WO2023124676A1 - Procédé de construction de modèle 3d, appareil et dispositif électronique - Google Patents

Procédé de construction de modèle 3d, appareil et dispositif électronique Download PDF

Info

Publication number
WO2023124676A1
WO2023124676A1 PCT/CN2022/134185 CN2022134185W WO2023124676A1 WO 2023124676 A1 WO2023124676 A1 WO 2023124676A1 CN 2022134185 W CN2022134185 W CN 2022134185W WO 2023124676 A1 WO2023124676 A1 WO 2023124676A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
polyhedron
point cloud
processed
model
Prior art date
Application number
PCT/CN2022/134185
Other languages
English (en)
Chinese (zh)
Inventor
张津维
刘海强
李凌云
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023124676A1 publication Critical patent/WO2023124676A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models

Definitions

  • the embodiments of the present application relate to 3D model construction technologies, and in particular, to a 3D model construction method, device, and electronic equipment.
  • 3D maps contain richer information.
  • users view 3D maps, they can see three-dimensional objects, and the user experience is high.
  • it is necessary to build an accurate and high-precision 3D model, such as building a 3D model of a building in a city.
  • Embodiments of the present application provide a 3D model construction method, device, and electronic equipment, which can improve the accuracy and construction efficiency of 3D model construction.
  • the embodiment of the present application provides a method for constructing a 3D model, and the executing subject for executing the method may be an electronic device or a chip in the electronic device.
  • the electronic device is taken as an example for illustration below.
  • the electronic device can extract multiple planes of the object from the data to be processed.
  • the object is an object to be constructed with a 3D model
  • the data to be processed is an RGB point cloud or a Mesh grid.
  • the data to be processed is an RGB point cloud or a Mesh grid containing an object, or the data to be processed is an RGB point cloud or a Mesh grid of an object.
  • the electronic device After the electronic device extracts multiple planes of the object, it may perform pairwise space intersection calculation on each plane to obtain multiple candidate surfaces of the object and at least two polyhedrons formed by the multiple candidate surfaces.
  • the electronic device may project the data to be processed onto each candidate surface to obtain the first candidate surface and the first polyhedron corresponding to the first data.
  • the first data is included in the data to be processed, and when the data to be processed is the RGB point cloud, the first data is the first RGB point cloud, and the first RGB point cloud is included in the RGB point cloud, where the data to be processed is projected to each candidate surface, that is, the RGB point cloud is projected to each candidate surface.
  • the data to be processed is the Mesh grid
  • the first data is a first Mesh grid
  • the first Mesh grid is included in the Mesh grid
  • the data to be processed is projected onto each candidate surface , which is to project the vertices of the Mesh grid to each candidate surface.
  • the first candidate surface is included in the plurality of candidate surfaces, and the first candidate surface is a surface of the first polyhedron.
  • the electronic device may acquire the inner and outer labels of the first polyhedron according to the normal vector of the first data and the vector from the center point of the first polyhedron to the first data.
  • the inner and outer labels indicate that the first polyhedron is an external polyhedron or an internal polyhedron of the object.
  • the electronic device may construct the 3D model of the object according to the inner and outer labels of the first polyhedron.
  • the electronic device may output the 3D model of the object, so that the user can watch the constructed 3D model in real time, so that the 3D model can be visualized.
  • the electronic device combines plane shape extraction, space intersection calculation, space plane projection, polyhedron topology construction, surface optimization calculation generation model, etc. to simplify and reconstruct the model of the object, which can improve the accuracy of the 3D model to obtain LOD3 level model, and the way of automatic construction can improve the construction efficiency of 3D model.
  • the electronic device may use the plane where the coplanar data to be processed is located as a plane to obtain the multiple planes.
  • the electronic device may use the plane where the coplanar RGB point cloud is located as a plane according to the spatial position relationship of the RGB point cloud.
  • the electronic device may adopt a region growing algorithm or a random sampling consensus algorithm, and use the plane where the coplanar RGB point cloud is located as a plane.
  • the electronic device may use the plane where the coplanar Mesh grids are located as a plane according to the adjacency topology relationship of the Mesh grid.
  • the electronic device may adopt a region growing algorithm or a random sampling consistent algorithm, and use the plane where the coplanar Mesh grids are located as a plane.
  • the electronic device can perform pairwise spatial intersection calculations on the planes to obtain multiple intersecting lines, and then use the intersecting lines as boundaries to perform constrained triangulation on each spatial plane formed by the intersecting lines and calculating to obtain the plurality of candidate surfaces, so that the electronic device can obtain the at least two polyhedrons according to the adjacency relationship of each candidate surface.
  • the electronic device may project the data to be processed to each candidate surface.
  • the data to be processed is the RGB point cloud
  • the electronic device may project the RGB point cloud onto each candidate surface, that is, project each RGB point cloud onto each candidate surface.
  • the data to be processed is the Mesh grid
  • the electronic device may project the vertices of the Mesh grid to each candidate surface. After projection, the electronic device may determine that the first data corresponds to The first candidate surface for .
  • the polyhedron to which the first candidate surface belongs is the first polyhedron.
  • the electronic device can construct the topological relationship between "point-point, point-surface, surface-surface, surface-body" according to the way of topological combination graph, and then obtain the first candidate surface belongs to The first polyhedron of .
  • the electronic device may use a candidate surface that is closest to the first data and on which a projection point of the first data falls, as the first candidate surface corresponding to the first data.
  • the electronic device can obtain the first candidate surface and the first polyhedron corresponding to the first data, such as the first candidate surface and the first polyhedron corresponding to the first RGB, or can obtain the first Mesh grid Corresponding to the first candidate surface, the first polyhedron.
  • the electronic device may acquire the first polyhedron according to the normal vector of the first data and the correlation between the center point of the first polyhedron and the vector of the first data.
  • the electronic device may acquire a normal vector of each first data, and a correlation between a center point of the first polyhedron and a vector of each first data corresponding to the first polyhedron, Further, weights of inner and outer labels of the first polyhedron are obtained according to the number of positive correlations and the number of negative correlations. Exemplarily, if the number of positive correlations is greater, the weight of the inner label of the first polyhedron is greater, and if the number of negative correlations is greater, the weight of the outer label of the first polyhedron is greater.
  • the electronic device may use the weights of the inner and outer labels of all the first polyhedrons included in the object as the nodes of the graph structure, and use the adjacent area between the first polyhedron and adjacent polyhedrons as the graph structure , get the inner and outer labels of the first polyhedron.
  • the electronic device can use the weights of the inner and outer labels of all the first polyhedrons in the object as the nodes of the graph structure, and use the adjacency between the first polyhedron and adjacent polyhedrons
  • the area is used as the edge of the graph structure, and the maximum flow algorithm or the minimum cut algorithm is used to obtain the inner and outer labels of the first polyhedron.
  • the electronic device obtains the inner and outer labels of each first polyhedron of the object (that is, characterizes whether the first polyhedron belongs to the outer polyhedron or the inner polyhedron of the object)
  • the adjacency of the inner polyhedron and the outer polyhedron in the object can be
  • the surface is used as the surface of the object, and then the surfaces are stitched together to construct a 3D model of the object.
  • the electronic device when the data to be processed is the RGB point cloud or Mesh grid of the object, the electronic device also needs to obtain the data to be processed from the initial data before building the 3D model, the The initial data contains an initial RGB point cloud or an initial Mesh grid of at least one object.
  • the electronic device can process the initial data to obtain the RGB point cloud or Mesh grid of the object. Specifically, the electronic device may perform multi-scale spatial sampling on the initial data to obtain sampling results of at least two scales. Wherein, when the initial data is an initial RGB point cloud, the electronic device may perform the following steps A and B:
  • the i-th scale sampling result includes The three-dimensional coordinates and color information of the i-th center point cloud, the i-th center point cloud is the center point of the point cloud contained in the grid, the i is an integer greater than or equal to 1 and less than N, and the N is an integer greater than 1.
  • step B judging whether the i is smaller than N; if so, adding i+1, and continuing to perform step A, if not, obtaining the sampling results of the at least two scales.
  • the electronic device may perform the following steps A and B:
  • the sampling result of the i-th scale includes the three-dimensional coordinates and color information of the i-th vertex, the i is an integer greater than or equal to 1 and less than N, and the N is an integer greater than 1.
  • step B judging whether the i is smaller than N; if so, adding i+1, and continuing to perform step A, if not, obtaining the sampling results of the at least two scales.
  • multi-scale spatial sampling can be used to obtain global information and local information.
  • the electronic device may perform regularization processing on the sampling results of each scale in combination with terrain information of the area to which the initial data belongs, to obtain processed sampling results of at least two scales.
  • the terrain information includes: the location range of multiple terrain grids in the area to which the initial data belongs, and the altitude value of each terrain grid.
  • the electronic device subtracts the Z value of the initial data falling into the position range of the first terrain grid from the altitude value of the first terrain grid to complete the calculation of the location falling into the first terrain grid.
  • Regularization processing of initial data in a location range the first terrain grid is included in the plurality of terrain grids.
  • regularization processing is performed on the sampling results of each scale, so that the vertical characteristics of objects in different urban scenes (or different regions) are consistent, and the artificial neural network model caused by large differences in terrain information is avoided. To solve the problem of large error, improve the accuracy.
  • the electronic device may input the processed sampling results of at least two scales into a pre-trained neural network model to obtain data carrying the label of the object.
  • the data carrying the label of the object such as RGB point cloud or Mesh grid carrying the building label.
  • the RGB point cloud or Mesh grid carrying the building label may contain multiple objects (such as buildings).
  • the electronic device may perform object segmentation on the data carrying the label of the object , to obtain the data to be processed, such as the RGB point cloud or Mesh grid corresponding to each object. In this way, the RGB point cloud or Mesh grid corresponding to each object is obtained, and subsequent steps such as plane extraction can be performed to avoid modeling the planes of different objects together and improve the accuracy of 3D modeling.
  • the embodiment of the present application provides a 3D model construction device, which may be the electronic device or the chip in the electronic device in the first aspect above, and the device may include:
  • the plane feature extraction module is used to extract multiple planes of the object from the data to be processed, the object is an object to be constructed with a 3D model, and the data to be processed is an RGB point cloud or a Mesh grid.
  • the spatial intersection calculation module is configured to perform pairwise spatial intersection calculations on each plane to obtain multiple candidate surfaces of the object and at least two polyhedrons formed by the multiple candidate surfaces.
  • a polyhedron topology construction module configured to project the data to be processed onto each candidate surface to obtain a first candidate surface and a first polyhedron corresponding to the first data, the first data being included in the data to be processed,
  • the data to be processed is the RGB point cloud
  • the first data is the first RGB point cloud
  • the data to be processed is the Mesh grid
  • the first data is the first Mesh network grid
  • the first candidate surface is included in the plurality of candidate surfaces
  • the first candidate surface is a surface of the first polyhedron.
  • a surface optimization generation module configured to obtain the inner and outer labels of the first polyhedron according to the normal vector of the first data and the vector from the center point of the first polyhedron to the first data, and A 3D model of the object is constructed according to the inner and outer labels of the first polyhedron.
  • the inner and outer labels indicate that the first polyhedron is an external polyhedron or an internal polyhedron of the object.
  • the plane feature extraction module is specifically configured to use the plane where the coplanar data to be processed is located as a plane, so as to obtain the multiple planes.
  • the plane feature extraction module is specifically configured to, according to the spatial position relationship of the RGB point cloud, place the coplanar RGB point cloud at plane as a plane.
  • the plane feature extraction module is specifically configured to, according to the adjacency topology of the Mesh grid, locate the coplanar Mesh grid plane as a plane.
  • the space intersection calculation module is specifically used to perform pairwise space intersection calculations on the planes to obtain multiple intersection lines; take the intersection lines as boundaries, and calculate each space plane formed by the intersection lines Perform constraint triangulation calculation to obtain the plurality of candidate surfaces; obtain the at least two polyhedrons according to the adjacency relationship of each candidate surface.
  • the polyhedron topology construction module is specifically configured to project the data to be processed to each candidate surface, and when the data to be processed is the Mesh grid, the projection is the The vertices of the Mesh grid; according to the distance between the first data and each of the candidate surfaces, and the positional relationship between the projection point of the first data and each of the candidate surfaces, determine the corresponding position of the first data first candidate surface.
  • the polyhedron topology construction module is specifically configured to use the candidate surface that is closest to the first data and that the projection point of the first data falls into as the corresponding surface of the first data. first candidate surface.
  • the surface optimization generation module is specifically configured to: according to the normal vector of the first data, and the correlation between the center point of the first polyhedron and the vector of the first data , to obtain the weights of the inner and outer labels of the first polyhedron, when the first data is the first Mesh grid, the vector from the center point of the first polyhedron to the first data is The vector from the center point of the first polyhedron to the center point of the first Mesh grid; the weights of the internal and external labels of all the first polyhedrons contained in the object are used as the nodes of the graph structure, with The adjacent area between the first polyhedron and the adjacent polyhedron is used as the edge of the graph structure, and the inner and outer labels of the first polyhedron are obtained.
  • the first polyhedron corresponds to a plurality of first data
  • the correlation includes a positive correlation and a negative correlation.
  • the surface optimization generation module is specifically used to obtain the normal vector of each first data, and the correlation between the center point of the first polyhedron and the vector of each first data corresponding to the first polyhedron ; Obtain the weights of the inner and outer labels of the first polyhedron according to the number of positive correlations and the number of negative correlations.
  • the surface optimization generation module is specifically configured to use the weights of the inner and outer labels of all the first polyhedrons in the object as the nodes of the graph structure, and use the weights of the first polyhedron and adjacent
  • the adjacency area between the polyhedra is used as the edge of the graph structure, and the maximum flow algorithm or the minimum cut algorithm is used to obtain the inner and outer labels of the first polyhedron.
  • the surface optimization generation module is specifically configured to use the adjoining surfaces of the internal polyhedron and the external polyhedron in the object as the surface of the object; according to the surface of the object, construct the 3D model of the object.
  • the data to be processed includes an RGB point cloud or a Mesh grid of the object, or the data to be processed is an RGB point cloud or a Mesh grid of the object.
  • the 3D semantic segmentation module and the instance segmentation module are used to obtain the data to be processed from the initial data
  • the initial data includes an initial RGB point cloud or an initial Mesh grid of at least one object.
  • the 3D semantic segmentation module is specifically configured to perform multi-scale spatial sampling on the initial data to obtain at least two-scale sampling results, and combined with the terrain information of the area to which the initial data belongs, each Regularize the sampling results of two scales to obtain the processed sampling results of at least two scales, and input the processed sampling results of at least two scales into the pre-trained neural network model to obtain the The object's label data.
  • the instance segmentation module is specifically configured to perform object segmentation on the data carrying the tag of the object to obtain the data to be processed.
  • the 3D semantic segmentation module is specifically configured to perform step A and step B:
  • the i-th scale sampling result includes The three-dimensional coordinates and color information of the i-th center point cloud, the i-th center point cloud is the center point of the point cloud contained in the grid, the i is an integer greater than or equal to 1 and less than N, and the N is an integer greater than 1.
  • step B judging whether the i is smaller than N; if so, adding i+1, and continuing to perform step A, if not, obtaining the sampling results of the at least two scales.
  • the 3D semantic segmentation module is specifically configured to perform step A and step B:
  • the sampling result of the i-th scale includes the three-dimensional coordinates and color information of the i-th vertex, the i is an integer greater than or equal to 1 and less than N, and the N is an integer greater than 1.
  • step B judging whether the i is smaller than N; if so, adding i+1, and continuing to perform step A, if not, obtaining the sampling results of the at least two scales.
  • the terrain information includes: location ranges of multiple terrain grids in the area to which the initial data belongs, and an altitude value of each terrain grid.
  • the 3D semantic segmentation module is specifically used to subtract the altitude value of the first terrain grid from the Z value of the initial data falling into the position range of the first terrain grid, so as to complete the analysis of the Regularization processing of initial data in a location range of a first terrain grid included in the plurality of terrain grids.
  • the surface optimization generation module is specifically configured to output the 3D model of the object.
  • an embodiment of the present application provides an electronic device, and the electronic device may include: a processor and a memory.
  • the memory is used to store computer-executable program codes, and the program codes include instructions; when the processor executes the instructions, the instructions cause the electronic device to execute the method in the first aspect.
  • an embodiment of the present application provides an electronic device, and the electronic device may be the 3D model construction apparatus of the second aspect or the electronic device described in the first aspect.
  • the electronic device may include a unit, module or circuit for performing the method provided in the above first aspect.
  • the embodiments of the present application provide a computer program product including instructions, which, when run on a computer, cause the computer to execute the method in the first aspect above.
  • an embodiment of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, it causes the computer to execute the method in the above-mentioned first aspect.
  • FIG. 1 is a schematic flow diagram of an embodiment of a 3D model construction method provided in an embodiment of the present application
  • FIG. 2 is a schematic flowchart of another embodiment of the 3D model construction method provided by the embodiment of the present application.
  • FIG. 3 is a schematic diagram of multi-scale spatial sampling provided by an embodiment of the present application.
  • Fig. 4 is the schematic diagram of grid in the DTM that the embodiment of the present application provides;
  • Fig. 5 is the schematic diagram of the U-Net network framework that the embodiment of the present application uses;
  • FIG. 6 is a schematic diagram of an object segmented using an algorithm according to an embodiment of the present application.
  • FIG. 7A is a schematic diagram of point cloud processing by the region growing algorithm used in the embodiment of the present application.
  • FIG. 7B is a schematic diagram of a plane proxy provided by an embodiment of the present application.
  • FIG. 7C is a schematic diagram of judging the coplanarity of Mesh grids using the topological relationship of Mesh grids provided by the embodiment of the present application;
  • FIG. 8A is a schematic diagram of two-to-two space intersection calculations performed on planes provided by the embodiment of the present application.
  • FIG. 8B is a schematic diagram of projecting a point cloud onto a candidate surface according to an embodiment of the present application.
  • FIG. 8C is a schematic diagram of the topological relationship of the polyhedron 1 provided in the embodiment of the present application.
  • FIG. 8D is a schematic diagram of the correlation between acquisition vectors provided by the embodiment of the present application.
  • Fig. 9 is a schematic diagram of an inner polyhedron and an outer polyhedron provided by an embodiment of the present application.
  • Figure 10 is a schematic diagram of the adjacent surface of the inner polyhedron and the outer polyhedron provided by the embodiment of the present application;
  • FIG. 11 is a schematic flowchart of another embodiment of the 3D model construction method provided by the embodiment of the present application.
  • FIG. 12 is a schematic flow diagram of training a neural network model provided by an embodiment of the present application.
  • FIG. 13 is a schematic flowchart of another embodiment of the 3D model construction method provided by the embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 15 is another schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • words such as “exemplary” or “for example” are used for example, illustration or explanation. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • the level of detail (LOD) model is a way to express the 3D shape of the object.
  • the LOD model can include: LOD0 level, LOD1 level, LOD2 level, and LOD3 level according to the detail of the model. The higher the level, the LOD model The higher the degree of detail.
  • the model of the LOD0 level includes the 2D plane outline of the building
  • the model of the LOD1 level includes the outline and height of the building
  • the model of the LOD2 level adds the structure of the roof on the basis of the LOD1 level
  • the LOD3 level model adds the structure of the building facade on the basis of the LOD2 level.
  • the LOD model is mainly used in the following scenarios:
  • fast positioning can be realized according to the LOD model.
  • augmented reality augmented reality, AR
  • the virtual and real effects and occlusion effects of objects can be obtained according to the spatial visual relationship in the LOD model.
  • the light and shadow rendering effect in AR technology needs to perform ray tracing calculation with the LOD model of the surrounding environment.
  • the embodiment of the present application provides a 3D model construction method, using 3D RGB point cloud, from automatic point cloud 3D semantic segmentation, building instance segmentation, to automatic construction of buildings with high accuracy
  • the high-precision LOD3 model can restore the real building structure and geographical location at 1:1, with short modeling time and high efficiency.
  • the 3D RGB point cloud used in the embodiment of the present application can also be replaced with a 3D Mesh grid model (referred to as Mesh grid) or a Mesh grid with texture.
  • Mesh grid 3D Mesh grid model
  • the 3D model construction method provided in the embodiment of the present application may not only be applied to the construction of a 3D model of a building, but may also be applied to the construction of a 3D model of other objects.
  • Other objects include, but are not limited to: objects, human bodies, numbers, vehicles, etc.
  • the object of the 3D construction model is a building as an example for illustration.
  • RGB point cloud it can be a point cloud containing color information (RGB). It should be understood that the RGB point cloud in the embodiment of the present application may include three-dimensional coordinates (X, Y, Z) and color information of the point cloud.
  • the RGB point cloud used in the embodiment of the present application may be a dense RGB point cloud, and the embodiment of the present application does not limit the type of the point cloud.
  • the RGB point cloud used in the embodiment of the present application can be: adopt aerial photography acquisition methods such as drones and airplanes to obtain multi-angle RGB images over the city, and then use multiple The generated RGB point cloud is reconstructed by multi-view stereo (MVS) technology.
  • the RGB point cloud used in the embodiment of the present application may be: an RGB point cloud generated by fusion and reconstruction of an aerial RGB image and a ground street view RGB image.
  • the RGB point cloud may include RGB point clouds of various types of objects such as buildings and plants in the city.
  • the indoor RGB point cloud when constructing an indoor 3D map, can be acquired in a way that is not limited to: using a handheld/backpack laser scanner, a fixed bracket laser scanner to scan the room, or using a depth camera , panoramic camera or ordinary RGB camera to shoot indoors.
  • the way of shooting with panoramic camera or ordinary RGB camera needs to take multi-position and multi-view images and reconstruct them through MVS technology to generate 3D RGB point clouds.
  • Other methods can directly obtain RGB point clouds through scanning or shooting.
  • Depth cameras may include, but are not limited to: structured-light depth cameras, stereo depth cameras, and optical time-of-flight (TOF) depth cameras.
  • Mesh grid It is a way to express the geometric shape of an object through a combination of points, lines, and surfaces.
  • Mesh grids can be realized by triangular meshes, quadrilateral meshes, or polygonal meshes.
  • the Mesh grid may include: the three-dimensional coordinates, color information, and normal and tangent of each vertex of the grid.
  • the vertex information of the grid may be obtained by converting the texture of the grid, which is not described in this embodiment of the present application.
  • the Mesh model used in the embodiment of the present application is a dense Mesh model.
  • the Mesh grid model used in the embodiment of the present application is: using aerial photography acquisition methods such as drones and airplanes to obtain multi-angle RGB images over the city, and then through multiple Multi-view stereo (MVS) technology reconstructs the generated Mesh grid model.
  • the Mesh grid used in the embodiment of the present application is: a Mesh grid model generated by fusion and reconstruction of aerial RGB images and ground street scene RGB images.
  • the indoor Mesh grid when constructing an indoor 3D map, can be obtained by using the above method for obtaining an RGB point cloud.
  • Examples of buildings ie buildings.
  • Instance segmentation if the instance is a building, the instance segmentation is to divide the building to obtain a single building. Exemplarily, if the instance contains 3 buildings, the instance segmentation is used to distinguish each building to obtain a single building. It is conceivable that the point cloud instance segmentation is to distinguish the point cloud of each building in the point cloud to obtain the point cloud corresponding to each single building.
  • the terrain information includes the two-dimensional coordinates (X, Y) of each location, and the altitude value at each location.
  • the terrain information used in the embodiment of the present application may be a digital terrain model (digital terrain model) or DTM terrain information calculated and generated by using a terrain extraction algorithm.
  • Object the object to be constructed into a 3D model, such as buildings, human bodies, figures, vehicles, etc.
  • the subject of execution of the 3D model construction method provided by the embodiment of the present application may be an electronic device, which may include but not limited to: 3D model construction computers, servers, server clusters, etc., the implementation of the present application
  • the form of the electronic device is not specifically limited in the examples, and the following embodiments take the electronic device as an example for illustration.
  • Fig. 1 is a schematic flow chart of an embodiment of the 3D model construction method provided by the embodiment of the present application. Referring to Fig. 1, the 3D model construction method provided by the embodiment of the present application may include:
  • the embodiment of the present application provides a 3D model construction platform, which is deployed in an electronic device, and a user can import an RGB point cloud of a 3D model to be constructed to the 3D model construction platform.
  • the 3D model building platform can use locally stored RGB point clouds of the 3D model to be built.
  • the RGB point cloud used in the embodiment of the present application is the point cloud of the target area.
  • the target area can be understood as the area where the 3D model is to be constructed.
  • the target area can be exemplarily: within 1 kilometer around the A building, or the area where the B road is located. Do limit.
  • the 3D model construction platform may provide an interface for the user to input construction instructions or perform other operations.
  • the interface of the 3D model building platform includes a "build 3D model” control
  • the user can input a build instruction to the electronic device by clicking on the "build 3D model” control.
  • the electronic device may start to build the 3D model in response to receiving the 3D model building instruction.
  • the electronic device may perform multi-scale spatial sampling on the RGB point cloud (abbreviated as RGB point cloud or point cloud) of the 3D model to be constructed to obtain sampling results of at least two scales.
  • RGB point cloud abbreviated as RGB point cloud or point cloud
  • high-resolution sampling can retain fine structures but lose the global information of objects
  • low-resolution sampling can have a larger receptive field but lose geometric details of small objects ( That is, local information)
  • multi-scale spatial sampling of RGB point clouds can be used.
  • the RGB point cloud of the 3D model to be constructed may be called an initial RGB point cloud.
  • the electronic device can use grids of different sizes to segment the RGB point cloud respectively, and then obtain the results of grid segmentation of different sizes.
  • the grid can be understood as a cube. The process of multi-scale spatial sampling of the RGB point cloud of the electronic device to be constructed as a 3D model is described as follows:
  • the RGB point cloud is arranged in a three-dimensional space.
  • the grid of the first scale is 2cm ⁇ 2cm ⁇ 2cm.
  • the RGB point cloud can be segmented with a grid of 2cm ⁇ 2cm ⁇ 2cm.
  • Point clouds fall into different rasters.
  • a point cloud can be sampled in each grid, and the three-dimensional coordinates of the point cloud can be the mean value of the three-dimensional coordinates falling into all point clouds in the grid, and the color information of the point cloud (ie, RGB value) can be the mean value of the RGB values falling into all point clouds in the grid.
  • the point cloud sampled in each grid can be called the first central point cloud, as shown by the black dots in each grid in the figure.
  • the electronic device can obtain the sampling result of the first scale.
  • the sampling result of the first scale may include: the first central point cloud (including the three-dimensional coordinates and color information of the first central point cloud) in each grid.
  • the second-scale grid to segment the RGB point cloud.
  • the second scale is larger than the first scale.
  • the grid of the second scale is 4cm ⁇ 4cm ⁇ 4cm
  • the electronic device can use the grid of 4cm ⁇ 4cm ⁇ 4cm to segment the RGB point cloud, and the RGB point cloud falls into different grids.
  • a point cloud can be sampled in each grid, and the three-dimensional coordinates of the point cloud can be the mean value of the three-dimensional coordinates falling into all point clouds in the grid, and the color information of the point cloud (ie, RGB value) can be the mean value of the RGB values falling into all point clouds in the grid.
  • the point cloud sampled in each grid can be called the second central point cloud.
  • the electronic device can obtain the sampling result of the second scale.
  • the sampling result of the second scale may include: the second central point cloud (including the three-dimensional coordinates and color information of the second central point cloud) in each grid.
  • the sampling result of the Nth scale may include: the Nth central point cloud (including the three-dimensional coordinates and color information of the Nth central point cloud) in each grid.
  • N is an integer greater than 1. In an embodiment, the value of N depends on the object. Exemplarily, N may be 3-5.
  • the electronic device can obtain sampling results of N scales.
  • step A-step B the above 1-3 can be summarized as step A-step B:
  • the i-th center point cloud is the center point of the point cloud contained in the grid
  • i is an integer greater than or equal to 1 and less than N
  • N is an integer greater than 1.
  • i is 1, the i-th scale is the first scale, and the grid is 2cm ⁇ 2cm ⁇ 2cm.
  • step B Determine whether i is smaller than N; if so, add i+1 and continue to perform step A, if not, obtain sampling results of at least two scales.
  • Topographical undulations significantly affect the spatial verticality of RGB point clouds, and spatial verticality is one of the important features to distinguish buildings from non-buildings, and the difference in spatial verticality of point clouds caused by terrain undulations will lead to inability to Correctly distinguish between buildings and non-buildings.
  • the sampling results of each scale are regularized (regularization), so that the buildings of different urban scenes (or different areas)
  • the vertical characteristics of the RGB point cloud are consistent, which avoids the problem of large fitting errors of the neural network model caused by large differences in terrain information (the neural network model can refer to the relevant description in Figure 12 below).
  • the electronic device can combine the terrain information of the area where the RGB point cloud belongs to, and perform regularization processing on the sampling results of each scale, so that The baselines of RGB point clouds in different regions are the same to eliminate errors caused by terrain information.
  • the terrain information of area 1 may include: the two-bit coordinates (X, Y) of each position in area 1, and the altitude value at the corresponding position.
  • the terrain information of area 1 may be stored in a rasterized form. Exemplarily, as shown in FIG. 4 , the terrain of area 1 has ups and downs, and area 1 can be divided into multiple grids (quadrilaterals), and each grid corresponds to a range of two-dimensional coordinates and an altitude value.
  • the electronic device can Subtract the Z value of the RGB point cloud whose two-dimensional coordinates fall into the range of the two-dimensional coordinates corresponding to the grid, and subtract the altitude value corresponding to the grid, that is, to complete the calculation that falls within the range of the two-dimensional coordinates corresponding to the grid Regularization of the RGB point cloud.
  • the grids in the terrain information may be called "terrain grids".
  • the range of the two-dimensional coordinates corresponding to the grid is: (X1, Y1), (X2, Y2) to (X3, Y3), (X4, Y4), and the altitude value corresponding to the grid is Z H1
  • the RGB point cloud falling into the "range of two-dimensional coordinates" includes point cloud 1 and point cloud 2
  • the Z value of point cloud 1 is Z1
  • the Z value of point cloud 2 is Z2.
  • the electronic device may subtract Z H1 from Z1 as the Z value of the RGB point cloud after point cloud 1 regularization, and subtract Z H1 from Z2 as the Z value of the RGB point cloud after point cloud 2 regularization processing.
  • the regularization processing method of the point cloud of the sampling result of the first scale is introduced above. It should be understood that the regularization processing method of the sampling result of different scales can refer to the regularization processing method of the point cloud of the sampling result of the first scale. related description. In this way, the electronic device can obtain processed sampling results of at least two scales.
  • the neural network model is used to classify the RGB point cloud to obtain the RGB point cloud belonging to the object.
  • the object is a building
  • the neural network model is used to distinguish RGB point clouds of buildings and non-buildings, and then label the RGB point clouds belonging to buildings as "buildings", that is, the neural network model can output carrying RGB point cloud with "building” tag.
  • the "RGB point cloud with object labels" output by the neural network model may also be referred to as a full RGB point cloud with semantic labels.
  • Semantic tags can be understood as tags identified as "objects”.
  • the neural network model used in the embodiment of the present application is obtained based on the U-Net network architecture.
  • FIG. 5 is a schematic structural diagram of a U-Net network.
  • the U-Net network architecture includes: an encoder and a decoder. The encoder is used to extract the features of the RGB point cloud, and the decoder is used for semantic segmentation based on the features of the RGB point cloud.
  • the U-Net network architecture will not be described in detail, and the description of the existing U-Net network architecture can be referred to.
  • the U-Net network framework includes random downsampling of samples, and downsamples the RGB point cloud layer by layer, which can improve the generalization ability of the neural network model and the semantic segmentation of the RGB point cloud (that is, to distinguish between buildings and non- RGB point cloud of the building) speed.
  • N in FIG. 5 represents the number of point clouds to be processed at one time, for example, N is 10240.
  • N/4 represents that the number of point cloud downsampling is 2560.
  • TF Transformer
  • TF represents a point cloud category attention mechanism, which expresses the weight of neighboring points by calculating the similarity between the target point cloud and the semantic features of neighboring point clouds.
  • US up sampling
  • DS down sampling
  • MLP multi-layer perception
  • MLP multi-layer perception
  • the encoder extracts the features of the RGB point cloud 1 as an example for illustration, and the encoder can acquire multiple point clouds within the preset spatial range of the point cloud 1 .
  • the preset space range from the point cloud 1 can be understood as: within the range of a sphere with the point cloud 1 as the center and the preset distance as the radius.
  • the encoder can extract point cloud 1 and the features of each point cloud within the preset space range of point cloud 1, and then fuse the features of point cloud 1 with the features of each point cloud within the preset space range of point cloud 1 , get the features of point cloud 1.
  • the features of the point cloud may include but not limited to: spatial geometric features, spectral features.
  • Spatial geometric features may include, but are not limited to: planarity, sphericality, linearity, verticality, etc.
  • Spectral features may include, but are not limited to: hue, brightness, saturation.
  • feature vector 1 (such as A1) is used to characterize the feature of point cloud 1
  • feature vector 2 (such as A2) is used to characterize the feature of each point cloud within the preset space range of point cloud 1, then the feature of point cloud 1 and the feature of each point cloud
  • the fusion of the features of each point cloud can be understood as: fusing the feature vectors of feature vector 1 and feature vector 2, for example, the fused feature vector is (A1, A2).
  • the encoder extracts the features of the RGB point cloud 1 as an example for illustration.
  • the encoder can obtain multiple point clouds within the preset spatial range of the point cloud 1 from the sampling results of different scales.
  • the encoder can obtain multiple point clouds within the preset spatial range of point cloud 1 in the sampling result of the first scale, and obtain the preset spatial range of point cloud 1 in the sampling result of the second scale
  • Multiple point clouds within , ..., and in the sampling results of the Nth scale multiple point clouds within the preset spatial range of point cloud 1 are obtained.
  • the encoder can extract point cloud 1 and the features of each point cloud in the preset space range of point cloud 1 in the sampling results of different scales, and then combine the features of point cloud 1 and the sampling results of different scales for each point cloud The features are fused to obtain the features of point cloud 1.
  • the RGB point cloud can be sampled in multi-scale space during the preprocessing process of the RGB point cloud, so when the encoder extracts the features of the point cloud, the point cloud can be compared with the point cloud at different scales
  • the features of the point cloud in the sampling results are fused to obtain the features of the point cloud.
  • the features of point clouds of different scales are combined in the embodiment of the present application, the features of point clouds are more abundant than those of point clouds in the prior art, and the fusion of local features and global features based on spatial relationships is realized. , can improve the accuracy of neural network model classification, that is, can improve the accuracy of distinguishing between buildings and non-buildings.
  • the RGB point cloud corresponding to each object may be referred to as data to be processed.
  • the point cloud segmentation can be simply understood as: distinguishing the point cloud of each building in the point cloud to obtain the point cloud corresponding to a single building.
  • the RGB point cloud carrying object labels includes RGB point clouds of multiple individual objects.
  • the spatial distance between point clouds belonging to the same building is much smaller than the spatial distance between point clouds belonging to different buildings.
  • the electronic device can use an unsupervised method with the spatial distance as the threshold to perform cluster analysis on the RGB point cloud carrying the object label, so as to cluster the point clouds belonging to the same building. Class together to get the RGB point cloud corresponding to each individual object.
  • the principle and process of clustering are not described in detail, and the description of the principle and process of clustering in the prior art may be referred to.
  • point clouds belonging to different buildings have a large spatial distance, so the density of point clouds belonging to different buildings is smaller than that of point clouds belonging to the same building.
  • the electronic device can use an unsupervised method with the spatial distance as the threshold and the density as the threshold to perform cluster analysis on the RGB point cloud carrying the object label, so as to cluster the point clouds belonging to the same building Class together, you get the RGB point cloud corresponding to each individual object.
  • the unsupervised cluster analysis with the spatial distance as the threshold can cluster the point clouds belonging to the same building together, as shown in each shaded circle in Figure 6
  • the point cloud belongs to the point cloud of the same building.
  • the electronic device can obtain an RGB point cloud corresponding to each individual object (such as a building).
  • each individual object may be referred to as the first object
  • the following S105-S115 is the processing process of the RGB point cloud corresponding to the first object, that is, each object (such as each building)
  • the corresponding RGB point cloud can be processed according to S105-S115 to obtain the 3D model of the object.
  • the geometric structure of any building can be approximated by a planar shape. Therefore, in the embodiment of this application, the plane of the building is extracted and geometrically spliced to Reconstruct the 3D model of the building. In the process of building a 3D model, plane shape extraction is the first link, that is, the electronic device executes S105.
  • each individual object is represented by the first object, and the RGB point cloud of the first object is the point cloud of each individual object.
  • the plane information of the first object may be understood as: information of the plane contained in the first object.
  • the electronic device may use the spatial distance between the RGB point clouds as a proximity judgment to determine which RGB point clouds are coplanar.
  • the electronic device may adopt a region growing (region growing) algorithm or an efficient random sampling consensus (efficient random sample consensus, Efficient RANSAC) algorithm, using the spatial distance between RGB point clouds as a proximity judgment to determine which RGB point clouds Coplanar. That is, the electronic device may use a region growing algorithm or an Efficient RANSAC algorithm to obtain plane information of the first object based on the RGB point cloud of the first object.
  • the following uses the region growing algorithm for electronic equipment as an example for illustration:
  • the electronic device can select the seed point (ie, the seed point cloud) in the RGB point cloud of the first object, to calculate the spatial position relationship between the seed point and the surrounding adjacent points (ie, the point cloud around the seed point cloud), to approximately fit the plane
  • the way to judge whether the seed point is coplanar with the surrounding neighboring points is to cluster the coplanar point cloud to form a plane.
  • the electronic device selects a seed point 1 in the RGB point cloud of the first object, and calculates whether the point cloud within the first preset space range from the seed point 1 is coplanar with the seed point 1, and if plane, the seed point 1 and the point cloud within the first preset space range from the seed point 1 are regarded as a plane 1.
  • the electronic device can also calculate whether the point cloud within the second preset space range of the seed point 1 is coplanar with the seed point 1. If it is coplanar, the electronic device can expand the range of the plane 1 to the second preset space range of the seed point cloud. . By analogy, until the seed point cloud is not coplanar with the point cloud within the Mth preset space range from the seed point 1, plane 1 is obtained.
  • the electronic device processes the RGB point cloud of the first object by using a region growing algorithm to obtain plane information of the first object.
  • the planar information of the first object may include: planar parameters of at least one planar proxy, the area range of each planar proxy, and the first index of each planar proxy.
  • the plane proxy can be understood as a plane, and the plane parameters of the plane proxy include: the three-dimensional coordinates of the center point of the plane proxy, and the normal vector of the plane proxy. As shown in FIG. 7B , the center point of plane 1 is p, and the normal vector of plane 1 is n.
  • the area range of the plane proxy can be understood as: the area of the plane proxy (that is, the range of the point cloud constituting the plane proxy).
  • the first index of the plane proxy includes: the index of the point cloud contained in the plane proxy (which may be referred to as the first point index for short), and the index of the plane (which may be referred to as the first surface index for short).
  • the electronic device after the electronic device obtains multiple planes (that is, plane agents) of the first object, it can perform index numbering for the plane agents to obtain the index of the planes.
  • the point cloud is indexed and numbered to obtain the index of the point cloud contained in the plane proxy.
  • the electronic device after the electronic device obtains the plane information of the first object, it needs to perform plane optimization projection calculation, that is, project all point clouds belonging to the same plane onto the plane, and calculate Spatial distance, merging planes where all spaces are coplanar. That is to say, the electronic device can combine all coplanar plane agents into one plane.
  • the electronic device may use the topological relationship of the Mesh grids to determine which Mesh grids are coplanar.
  • the electronic device selects a Mesh grid 1 in the Mesh grid of the first object, and uses the topological relationship of the Mesh grid 1 to calculate the Mesh grid around the Mesh grid 1. Whether the grid is coplanar with the Mesh grid 1, and if so, the Mesh grid 1 and the Mesh grids around the Mesh grid are regarded as a plane 1.
  • the electronic device continues to use the topological relationship of the Mesh grid 2 to calculate whether the Mesh grids around the Mesh grid 2 are consistent with The Mesh grid 2 is coplanar, and if it is coplanar, the Mesh grid 2 and the Mesh grid surrounding the Mesh grid are regarded as a plane 2.
  • a plane proxy is obtained.
  • the electronic device After extracting the plane of the first object, in order to achieve plane geometric splicing, it is necessary to perform space intersection calculation on all plane proxies of the first object to obtain candidate surfaces of the object, that is, the electronic device executes S106-S107.
  • a plane agent can be understood as a plane.
  • each plane of the first object can be obtained.
  • the electronic device can perform pairwise space intersection calculations on each plane agent of the first object. After the intersection of the two planes, it can be obtained One intersecting line, so after the two planes in the first object intersect each other, multiple intersecting lines can be obtained.
  • FIG. 8A two-by-two spatial intersection calculations are performed on six planes to obtain 12 intersection lines, that is, 12 sides of a cube.
  • the six planes include three groups of two parallel planes, such as plane 1 and plane 2 parallel to each other, plane 3 and plane 4 parallel to each other, and plane 5 and plane 6 parallel to each other.
  • the pairwise space intersection calculation method used in the embodiment of the present application includes but is not limited to: full pairwise intersection algorithm, minimum bisection space method (binary space partitioning, BSP), dynamic partition method (kinetic partition).
  • the electronic device may obtain the first index of the plane proxy of the first object based on the above S105
  • the priorities of the plane proxies of the first object are determined based on the priority to determine the order of pairwise spatial intersection calculations.
  • the first index of the plane proxy includes: a first point index.
  • the plane can be characterized by the number of indexes including point clouds in the plane proxy (that is, the number of point clouds falling into the plane)
  • the priority of the agent for example, the more the number of indexes including point clouds in the plane agent (that is, the number of point clouds falling into the plane), the higher the priority of the plane agent.
  • the electronic device may perform pairwise spatial intersection calculations on the planar agents with the first and second priorities to obtain an intersecting line, and then calculate the first and second planar agents with the first and second priorities.
  • the plane agent at the third position firstly calculates the intersection of two spaces, and obtains an intersection line.
  • the electronic device can obtain a plurality of intersecting lines for each plane agent to perform pairwise space intersection calculations.
  • intersection line As a boundary, perform constrained triangulation calculation on each space plane formed by the intersection line to obtain a candidate surface.
  • Three intersecting intersecting lines can form a triangular patch.
  • the electronic device can use the intersecting lines as a boundary, and perform constrained triangulation calculation on each space plane formed by the intersecting lines to obtain a candidate surface .
  • the shape of the candidate surface is a triangle, and this embodiment of the present application does not limit the shape of the candidate surface.
  • the candidate surface may be represented by three vertices (that is, intersection points of intersection lines) constituting the candidate surface. That is to say, when storing the candidate surface, the electronic device may store the coordinates of the three vertices of the candidate surface.
  • the electronic device may perform index numbering on the three vertices, that is, represent the candidate surface with three vertex indices. In this embodiment, the indices of the three vertices may be referred to as second point indices.
  • the candidate surface may also be represented by three edges formed by three vertices. That is to say, when storing the candidate surface, the electronic device may store three sides of the candidate surface. Exemplarily, the electronic device may index the three sides, that is, represent the candidate surface with three side indices. In this embodiment, the indices of the three sides may be referred to as second side indices.
  • the electronic device may also perform surface index numbering on the candidate surfaces, which is called a second surface index. Because each candidate surface can be composed of at least three edges, one second surface index corresponds to at least three second edge indexes, and the intersection of at least three edges is the vertex of the candidate surface, so one second surface index corresponds to at least three second edge indexes. a second point index. That is to say, after obtaining the candidate surfaces, the electronic device may store the second surface index of each candidate surface, and at least three second edge indexes and at least three second point indexes corresponding to the second surface index.
  • the method of spatial plane projection can be used to project the first object
  • the RGB point cloud of is projected to the candidate surface of the first object, that is, the electronic device executes S108.
  • the electronic device can project the RGB point cloud of the first object to each candidate surface of the first object, and then based on the distance from the RGB point cloud to each candidate surface and whether it falls into each candidate surface condition to determine the candidate surface corresponding to the RGB point cloud projection. Wherein, the electronic device can select the candidate surface corresponding to the point cloud that has the smallest distance from the point cloud and on which the point cloud can be projected.
  • the candidate surfaces are Surface 1 and Surface 2
  • the electronic device projects the point cloud 1 onto Surface 1 and Surface 2 respectively, and the electronic device can use the surface 1 on which the projection falls as the corresponding point cloud 1. candidate surface.
  • the electronic device may use the surface with the smallest distance to the point cloud 1 as the candidate surface corresponding to the point cloud 1.
  • the electronic device may acquire a candidate surface corresponding to each RGB point cloud of the first object.
  • the electronic device may store a correspondence between each RGB point cloud of the first object and the second surface index of the candidate surface.
  • the dimension optimization variable can be increased first, so that the solution space is reduced, and the efficiency of optimization solution is improved, that is, the electronic device executes S109-S111.
  • the topological relationship includes: topological relationships between points and points, points and surfaces, surfaces and surfaces, and surfaces and volumes.
  • the electronic device can also obtain the adjoining surfaces of each candidate surface, and then based on the adjoining surfaces of the candidate surfaces, obtain A polyhedron composed of candidate surfaces and adjacent faces.
  • the electronic device can index and number the polyhedron to obtain the second body index.
  • the second volume index may correspond to the second face index, the second edge index, and the second point index of the multiple candidate faces constituting the polyhedron.
  • the electronic device may acquire the adjacency relationship of each polyhedron based on the coplanar polyhedrons.
  • the electronic device can index all constructed faces based on each polyhedron according to the second face index, the second edge index, and the second point index corresponding to the second volume index, and each face indexes all constructed edges , each edge indexes all constructed points to construct topological relationships including "points to points, points to surfaces, surfaces to surfaces, surfaces to volumes". Exemplarily, referring to FIG.
  • the electronic device can index the second volume of the polyhedron 1, such as the second volume index of the polyhedron 1 for all constructed faces (face 1, face 2, face 3, etc.), each Each face indexes all constructed edges (such as edge 1, edge 2, edge 3, and edge 4 indexed by face 1), and each edge indexes all constructed points (such as point 1 and point 2 cited in edge 1).
  • the topological relationship is "polyhedron 1", “face 1, face 2, face 3, etc.”, "edge 1, edge 2, edge 3, edge 4, etc.”, and "point 1, point 2, etc.” components topological relationship.
  • polyhedrons adjacent to polyhedron 1 include polyhedron 2 .
  • the electronic room can construct the "point-point, point-surface, surface-surface, surface-body" in the way of topological combination maps (combinatorial maps). topological relationship.
  • the topological relationship includes the topological relationship between "point and point, point and surface, surface and surface, and surface and body".
  • it can be judged which surfaces constitute the minimum polyhedron according to the topological relationship.
  • the electronic device can obtain all minimum polyhedrons (linear cell complexes) contained in the first object through global calculation, that is, the first polyhedron described below, and there may be multiple first polyhedrons in the object.
  • the electronic device can obtain the candidate surface corresponding to the RGB point cloud, and according to the description in S109-S110, the electronic device can obtain the topological relationship between the polyhedron and the candidate surface, so S108-S111, the electronic device can obtain RGB The polyhedron corresponding to the point cloud, that is, the electronic device can construct the mapping relationship between the RGB point cloud and the polyhedron.
  • the candidate surface corresponding to the first RGB point cloud can be called the first candidate surface, and the polyhedron corresponding to the first RGB point cloud can be called the first polyhedron.
  • the first RGB point cloud may be referred to as first data, and the first data is included in the data to be processed, that is, the first RGB point cloud is included in the RGB point cloud.
  • the candidate surface corresponding to the RGB point cloud is surface 1, and surface 1 is a surface in polyhedron 1, so the polyhedron corresponding to the RGB point cloud is polyhedron 1.
  • the electronic device can obtain the polyhedron corresponding to the RGB point cloud of the first object, but as far as a building is concerned, in the embodiment of the present application, a building is divided into multiple polyhedrons, and the polyhedron can be a polyhedron inside the building or External polyhedrons, in 3D modeling, the constructed 3D model of the building is seen by the user as a collection of external polyhedrons of the building. Therefore, in the embodiment of the present application, the electronic device also needs to distinguish the polyhedron as the polyhedron inside the building or the polyhedron outside, so as to construct the 3D model of the building. That is to say, the electronic device needs to execute S112-S115.
  • the first object is composed of multiple polyhedrons.
  • the processing steps of each polyhedron represented by the first polyhedron are taken as an example for illustration:
  • the object may include multiple first polyhedrons.
  • one first polyhedron may correspond to multiple RGB point clouds.
  • the following uses a first polyhedron, and the first polyhedron obtains the normal vector of an RGB point cloud in the first polyhedron, and the correlation between the center point of the first polyhedron and the vector of the RGB point cloud Sex as an example:
  • the RGB point cloud corresponding to the first polyhedron is point cloud P1 and P2
  • such as the electronic device can obtain the normal vector n1 of the point cloud P1, and the center point C of the first polyhedron (that is, the first The position of the center of the polyhedron) to the vector m1 of the point cloud P1, and then obtain the correlation between the normal vector n1 and the vector m1.
  • the electronic device can obtain the normal vector n2 of the point cloud P2 and the vector m2 from the center point of the first polyhedron (that is, the position of the center of the first polyhedron) to the point cloud P2, Further, the correlation between the normal vector n2 and the vector m2 is obtained.
  • Correlations can include positive and negative correlations. Exemplarily, if the normal vector n1 is positively correlated with the vector m1, the normal vector n2 is negatively correlated with the vector m2.
  • the step of S112 is described by taking the point clouds P1 and P2 corresponding to the first polyhedron as an example. Because the first polyhedron can correspond to multiple point clouds, the electronic device can obtain multiple correlations of the first polyhedron (that is, the normal vector of each RGB point cloud corresponding to the first polyhedron and the first polyhedron The correlation between the center point of the facet and the vector of each RGB point cloud).
  • the electronic device can be based on the correlation between the normal vector of the RGB point cloud corresponding to the first polyhedron and the vector from the first polyhedron to the RGB point cloud (that is, each The correlation between the normal vector of the RGB point cloud and the center point of the first polyhedron to the vector of each RGB point cloud), and obtain the first polyhedron belonging to the inner polyhedron (ie, the inner polyhedron of the object) or the outer polyhedron ( That is, the weight of the polyhedron outside the object).
  • the following takes the normal vector of each RGB point cloud corresponding to the first polyhedron as the first normal direction, and takes the vector from the center point of the first polyhedron to each RGB point cloud as the second normal direction as an example. illustrate:
  • the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron is related to the positive and negative number (quantity) of the correlation between the first normal direction and the second normal direction.
  • a positive correlation indicates that the first polyhedron belongs to the inner polyhedron
  • a negative correlation indicates that the first polyhedron belongs to the outer polyhedron.
  • the weight of the first polyhedron belonging to the inner polyhedron is higher, and if the number of negative correlations between the first normal direction and the second normal direction The larger the number, the higher the weight that the first polyhedron belongs to the outer polyhedron.
  • the electronic device may acquire N correlations of the first polyhedron. For example, the number of positive correlations is a, and the number of negative correlations is (M-a). In one embodiment, the electronic device can determine that the weight of the first polyhedron belonging to the outer polyhedron is (M-a)/M. A candidate surface belongs to the inner polyhedron with weight a/M.
  • the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron may be referred to as the weight of the inner and outer labels of the first polyhedron.
  • the weight of the first polyhedron belonging to the inner polyhedron may represent the weight of the inner label of the first polyhedron
  • the weight of the first polyhedron belonging to the outer polyhedron may represent the weight of the outer label of the first polyhedron.
  • the inner polyhedron is represented by a white area
  • the outer polyhedron is represented by a shaded area
  • the point cloud corresponding to the first polyhedron is represented by a dot
  • the point cloud corresponding to the first polyhedron is represented by an arrow. normal vector.
  • the electronic device may use the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron as the node of the graph structure, and use the adjacency area between the first polyhedron and the polyhedron adjacent to the first polyhedron as Solve the graph structure for the edges of the graph structure to obtain the inner and outer labels of the first polyhedron. It should be understood that the graph structure needs to keep dissipating energy to a minimum.
  • the weight of the first polyhedron belonging to the inner polyhedron or the outer polyhedron can be used as the node of the graph structure, and the adjacency area between the first polyhedron and the adjacent polyhedrons of the first polyhedron can be used as the For the edges of the graph structure, a graph-cut method is used to solve the graph structure, so as to obtain the inner and outer labels of the first polyhedron.
  • the electronic device may use the weights of the inner and outer labels of all the first polyhedrons in the object as the nodes of the graph structure, and use the weights of the first polyhedron and adjacent polyhedrons
  • the adjacency area between is used as the edge of the graph structure, using the maximum flow algorithm or the minimum cut algorithm, using the graph-cut (graph-cut) method, to solve the graph structure, to obtain the inner and outer labels of the first polyhedron.
  • the electronic device may use the following formula 1 to solve the graph structure:
  • ⁇ D(x) is the Data term, which represents the cost function of the first polyhedron being divided into inner polyhedron or outer polyhedron
  • ⁇ S(x, y) is the Smoothness term, which represents every two adjacent first polyhedrons Surface area cost function between polyhedrons
  • E represents the sum of energies.
  • the ultimate optimization goal of graph structure solving is to solve the polyhedron labeling that minimizes E (ie labeling as an inner and outer label scheme). According to the graph structure optimization theory, this patent adopts the method of maximum flow/minimum cut to solve the above formula one.
  • the inner and outer labels indicate that the first polyhedron is an outer polyhedron (ie, an outer polyhedron) or an inner polyhedron (ie, an inner polyhedron) of the object.
  • the inner and outer labels of the first polyhedron are inner labels, it indicates that the first polyhedron is a polyhedron inside the first object, and if the inner and outer labels of the first polyhedron are outer labels, it indicates that the first polyhedron is the polyhedron outside the first object.
  • a surface adjoining the inner polyhedron and the outer polyhedron is used as a 3D model surface of the first object to construct a 3D model of the first object.
  • the electronic device after the electronic device obtains the inner and outer labels of all the first polyhedrons in the first object, it can use the adjacent surface of the inner polyhedron and the outer polyhedron as the surface of the 3D model of the first object, and then based on the inner polyhedron Surfaces adjoining the outer polyhedron construct a 3D model of the first object.
  • the inner polyhedron included in the first object is characterized by a white area
  • the outer polyhedron is characterized by a shaded area.
  • the electronic device can use the surface adjacent to the inner polyhedron and the outer polyhedron as the 3D model surface of the first object (as shown in FIG. 10 . Thick solid line), splicing the surfaces of the 3D model of the first object to construct the 3D model.
  • the electronic device may also execute S116:
  • the electronic device outputs the 3D model of the first object.
  • the electronic device can output the 3D models of the first objects on the interface of the electronic device, so that the user can see the constructed target area on the electronic device 3D model of the first object.
  • the electronic device may output a 3D closed surface geometric model data file of the first object and a 3D model contour structure line file.
  • the 3D closed surface geometric model data file includes: all surface information in the 3D model, and surface vertex information.
  • the 3D model outline structure line file includes: the information of points and lines of the outline structure of the 3D model.
  • the RGB point cloud before the RGB point cloud is classified, on the one hand, the RGB point cloud can be sampled in a multi-scale space, which can ensure that the sampling can obtain the global information and local information of the RGB point cloud, thereby improving the semantics of the RGB point cloud
  • the accuracy of segmentation on the other hand, can also combine the terrain information of the area where the RGB point cloud belongs to, and regularize the sampling results of each scale to avoid the large error of neural network model fitting caused by the large difference in terrain information. problem, improving the accuracy of semantic segmentation with neural network models.
  • the combination of plane shape extraction, space intersection calculation, space plane projection, polyhedron topology construction, surface optimization calculation and generation model is used to simplify and reconstruct the model of the first object, which can improve the accuracy of the 3D model, and Get the LOD3 level model.
  • the 3D model construction method provided by the embodiment of the present application is described in the above FIG. 1 by taking the RGB point cloud as an example, and the 3D model construction method provided by the embodiment of the present application is described below by taking the Mesh grid as an example.
  • the 3D model construction method provided by the embodiment of the present application may include:
  • S201 in response to a 3D model construction instruction, perform multi-scale space sampling on the Mesh grid to be constructed 3D model, to obtain sampling results of at least two scales.
  • the Mesh of the 3D model to be constructed may be called an initial Mesh.
  • the electronic device may take the vertex in the Mesh grid as the center of the sphere, draw spheres with different radii to segment the Mesh grid, and then obtain the results of sphere segmentation with different sizes.
  • the following describes the process of multi-scale spatial sampling of the Mesh grid to be constructed by the electronic device for the 3D model:
  • the vertices of the Mesh grid are arranged in a three-dimensional space, and the first-scale sphere is used to segment the Mesh grid, and each sphere contains multiple vertices.
  • a vertex (may be referred to as a center vertex) can be sampled in each sphere, and the center vertex can be the center vertex of all vertices contained in the sphere, such as the three-dimensional coordinates of the center vertex can be the The mean value of the three-dimensional coordinates of all vertices in the sphere, the color information (ie RGB value) of the central vertex may be the mean value of the RGB values of all vertices in the sphere.
  • the point cloud sampled in each sphere can be called the first central vertex.
  • the electronic device can obtain the sampling result of the first scale.
  • the sampling result of the first scale may include: the first center vertex (including the three-dimensional coordinates and color information of the first center point cloud) in each grid.
  • the second-scale sphere to segment the Mesh grid.
  • the second scale is greater than the first scale.
  • a vertex can be sampled in each sphere, and the three-dimensional coordinates of the vertex can be the mean value of the three-dimensional coordinates of all vertices in the sphere, and the color information (i.e. RGB value) of the vertex ) can be the mean of the RGB values of all vertices in the sphere.
  • the sampled vertex in each sphere may be called the second central vertex.
  • the sampling result of the second scale may include: the second central vertex (including the three-dimensional coordinates and color information of the second central vertex) in each sphere.
  • the sampling result of the Nth scale may include: the Nth central vertex (including the three-dimensional coordinates and color information of the Nth central vertex) in each sphere.
  • the electronic device can obtain sampling results of N scales.
  • step A-step B the above 1-3 can be summarized as step A-step B:
  • the sampling result of the i-th scale includes the three-dimensional coordinates and color information of the i-th vertex, the i is an integer greater than or equal to 1 and less than N, and the N is an integer greater than 1.
  • step B judging whether the i is smaller than N; if so, adding i+1, and continuing to perform step A, if not, obtaining the sampling results of the at least two scales.
  • the electronic device can convert the two-dimensional coordinates
  • the Z value of the vertex of the Mesh grid that falls within the range of the two-dimensional coordinates corresponding to the grid is subtracted from the altitude value corresponding to the grid, that is, the regularization of the Mesh grid that falls into the grid is completed.
  • the range of the two-dimensional coordinates corresponding to the grid is: (X1, Y1), (X2, Y2) to (X3, Y3), (X4, Y4), and the altitude value corresponding to the grid is Z H1
  • the vertices falling into the "range of two-dimensional coordinates" include the vertices of Mesh 1 and Mesh 2 .
  • the electronic device may subtract Z H1 from both the vertices of Mesh 1 and Mesh 2 as Z values of the vertices of Mesh 1 and Mesh 2 after regularization.
  • the Z value of the vertex of the Mesh grid can be subtracted from the altitude value of the corresponding grid.
  • the neural network model used in S203 is different from the neural network model used in S103, and the neural network model used in S203 can distinguish Mesh grids of different types of objects.
  • the neural network model in S203 is used to classify the Mesh grids to obtain the Mesh grids belonging to the object.
  • the object is a building
  • the neural network model is used to distinguish the Mesh grids of buildings and non-buildings, and then label the Mesh grids belonging to buildings as "buildings"
  • the Mesh corresponding to the object obtained in S204 may be referred to as data to be processed.
  • the electronic device can use the unsupervised method with the spatial distance as the threshold to perform cluster analysis on the vertices (corresponding to the Mesh grid) of the Mesh grid carrying the object label, so as to group the vertices belonging to the same building
  • the point clouds of objects are clustered together to obtain the Mesh grid corresponding to each individual object.
  • the electronic device can use an unsupervised method with the spatial distance as the threshold and the density as the threshold, and the vertices of the Mesh grid carrying the object label (corresponding to the Mesh Grid) for cluster analysis to cluster the point clouds belonging to the same building together to obtain the Mesh grid corresponding to each individual object.
  • the electronic device may use the plane where the coplanar Mesh grids are located as a plane according to the adjacency topology relationship of the Mesh grids, so as to acquire the plane information of the first object.
  • intersection line As a boundary, perform constrained triangulation calculation on each space plane formed by the intersection line to obtain a candidate surface.
  • topological relationship includes: topological relationships between points and points, points and surfaces, surfaces and surfaces, and surfaces and volumes.
  • the vertices of the Mesh grid of the first object may be regarded as the RGB point cloud of the first object.
  • the candidate surface corresponding to the first Mesh grid can be called the first candidate surface, and the polyhedron corresponding to the first Mesh grid Can be called the first polyhedron.
  • the first Mesh may be referred to as first data, and the first data is included in the data to be processed, that is, the first Mesh is included in the Mesh of the first object.
  • the normal vector of the Mesh may be the normal vector of the polygon.
  • the vector from the center point of the first polyhedron to the Mesh grid can be: the vector from the center point of the first polyhedron to the center point of the Mesh grid, and this vector can refer to "the center point of the first polyhedron to RGB point cloud" description.
  • the vector from the center point of the first polyhedron to the Mesh grid may be: the vector from the center point of the first polyhedron to any vertex of the Mesh grid.
  • a surface adjoining the inner polyhedron and the outer polyhedron is used as a 3D model surface of the first object to construct a 3D model of the first object.
  • S213-S215 refer to related descriptions in S113-S115.
  • the electronic device may execute the above S116.
  • a 3D model can be constructed based on the initial Mesh grid, and the technical effect described in FIG. 1 can also be achieved.
  • the technical effect refer to the relevant description in FIG. 1 , and details are not repeated here.
  • FIG. 11 is an example of an RGB point cloud sorting process for illustration.
  • the method for constructing a 3D model provided by the embodiment of the present application may include steps such as semantic segmentation, instance segmentation, plane shape extraction, space intersection calculation, space plane projection, polyhedron topology construction, and surface optimization calculation to generate a model.
  • S101-S103, S201-S203 are the process of "semantic segmentation”
  • S104, S204 are the process of "instance segmentation”
  • S105, S205 are the process of "plane shape extraction”
  • S106-S107, S206-S207 are " S108
  • S208 is the process of "space plane projection”
  • S109-S111, S209-S211 is the process of "polyhedron topology construction”
  • S112-S115, S212-S215 is the "surface optimization calculation generation model” the process of.
  • S105-S115 and S205-S215 may be referred to as a process of "model simplification and reconstruction".
  • FIG. 12 is a schematic diagram of a training neural network model provided by an embodiment of the present application.
  • a device for training a neural network model may be called a training device, and the training device may be a computer, a dedicated neural network model training machine, etc.
  • the embodiment of the present application does not limit the form of the training device.
  • the neural network model can be preset in the electronic device.
  • the U-Net network architecture includes an input layer, at least one hidden layer and an output layer.
  • the input layer is used to receive training data and distribute the training data to neurons in the hidden layer.
  • the neurons in the hidden layer are used to perform calculations based on the data, and output the calculation results to the output layer.
  • the output layer is used to output the operation result.
  • the process of training the neural network model of the training device may include:
  • the training device may initialize the weight values of the neurons of each hidden layer to random weight values subject to Gaussian distribution.
  • the training data is an RGB point cloud containing object labels.
  • the training device can divide the training data into N batches, so as to use the N batches of data to iteratively train the U-NET network architecture. For example, the training device may divide the training data into N batches on average. Wherein, N is an integer greater than 1.
  • the training device inputs the first batch of data into the U-NET network architecture, and the U-NET network architecture can output the intersection of the first batch of data Entropy loss (cross entropy loss).
  • cross entropy loss is used to characterize the similarity between the object to which the RGB point cloud is predicted by the training device using the U-NET network architecture and the object to which the RGB point cloud belongs. The smaller the cross-entropy loss, the more accurate the weight value of neurons representing each hidden layer in the U-NET network architecture.
  • i is an integer greater than or equal to 1 and less than or equal to N.
  • the training device may update the initial weight values of neurons in each hidden layer according to the cross-entropy loss of the first batch of data.
  • the training device can determine the error between the similarity between the object of the RGB point cloud predicted by the U-NET network architecture and the object to which the RGB point cloud belongs and 100%, and then The weight values of neurons in each hidden layer are updated according to the error.
  • the training device may use a gradient descent method (gradient decent) or a stochastic gradient descent method (stochastic gradient decent) to update the weight values of neurons in each hidden layer.
  • the training device can judge whether i is smaller than N, so as to determine whether the training of the N batches of data of the training data is completed. Wherein, if i is smaller than N, the training device may add 1 to i, and continue to execute S1203. For example, when i is 1 and N is 10, the training device determines that i is less than N, and then inputs the second batch of data into the U-NET network architecture for updating the weight value to obtain the second batch of data. Cross entropy loss. Similarly, the training device can use the gradient descent method or the stochastic gradient descent method to update the weight values of the neurons in each hidden layer according to the cross-entropy loss of the second batch of data.
  • the training device inputs the Nth batch of data into the U-NET network architecture, and the training device updates each hidden layer according to the cross-entropy loss of the Nth batch of data The weight value of the neuron.
  • the training device may perform the following S1206.
  • S1206 Determine whether the training is converged according to the target cross-entropy loss and the cross-entropy loss of the Nth batch of data. If the training converges, execute S1207; if the training does not converge, execute S1202.
  • the training device obtains the cross-entropy loss of the Nth batch of data, it can determine whether the training is converged according to the cross-entropy loss of the Nth batch of data and the target cross-entropy loss. Among them, if the cross-entropy loss of the Nth batch of data is less than or equal to the target cross-entropy loss, then the object belonging to the RGB point cloud predicted by the U-NET network architecture is close to the object belonging to the RGB point cloud, and the training device The training convergence can be determined.
  • the training device returns to execute S102, that is, continues to divide the preprocessed training data into N batches, and uses the N batches of data to continue training the U-NET network architecture until the training converges.
  • the neural network model may be a U-NET network architecture after the training device updates the weight values of neurons in each hidden layer according to the cross-entropy loss of the Nth batch of data.
  • the training device can preprocess the training data and input it into the U-NET network architecture.
  • the preprocessing process of the training data by the training device can refer to the description in S101-S102. It should be noted that, During the process of training the neural network model, the training device can extract the features of the training data. The features of the training data can be extracted with reference to the relevant description in S103, which will not be repeated here.
  • the 3D model construction method provided by the embodiment of the present application can be applied to the 3D model construction scene of a park (such as a factory park).
  • a park such as a factory park.
  • the scene is small, and the corresponding terrain fluctuations do not change much. Therefore, when the electronic device performs the steps of S101-S115 (or S201-S215) above, it is not necessary to perform S102 (or S202), that is, it is not necessary to combine the RGB of the 3D model to be constructed.
  • the topographic information of the area to which the point cloud belongs regularizes the sampling results of each scale.
  • the 3D model construction method provided in the embodiment of the present application may be applied to a 3D model construction scene of an indoor scene. Since the indoor application scene has more complex object categories than the city-level scene, and the indoor geometric structure features are relatively simpler than the city-level outdoor scene, so in the embodiment of the present application, the electronic device performs the above S101-S115 (or In the steps of S201-S215), S101-S104 (S201-S204) may not be executed, but S105-S115 (or S205-S215) may be directly executed to construct the indoor structured 3D modeling.
  • the 3D model construction method provided in the embodiment of the present application can also be applied to other scenarios, and in other scenarios, the electronic device can perform some or all of the steps in S101-S115 (or S201-S215) as appropriate , all belong to the scope of protection of the embodiments of the present application.
  • the 3D model construction method provided by the embodiment of the present application may include:
  • the data to be processed is an RGB point cloud or a Mesh grid, such as the RGB point cloud or Mesh grid of the object obtained in S104 and S204 is the data to be processed.
  • RGB point cloud or Mesh grid such as the RGB point cloud or Mesh grid of the object obtained in S104 and S204 is the data to be processed.
  • S105 and S205 For the process of extracting multiple planes of the object from the data to be processed, reference may be made to relevant descriptions in S105 and S205.
  • S1302. Perform pairwise space intersection calculation on each plane to obtain multiple candidate surfaces of the object and at least two polyhedrons formed by the multiple candidate surfaces.
  • the first data is included in the data to be processed.
  • the data to be processed is an RGB point cloud
  • the first One data is a first RGB point cloud
  • the data to be processed is a Mesh grid
  • the first data is a first Mesh grid
  • the first candidate surface is included in a plurality of candidate surfaces
  • the first candidate surface is a first multi-face body surface.
  • the electronic device may output the 3D model of the object.
  • the device to be processed is acquired after the electronic device processes the initial data.
  • the initial data includes an initial RGB point cloud or an initial Mesh grid of at least one object
  • the initial data may include: sampling the initial data in a multi-scale space, obtaining sampling results of at least two scales, and combining the initial data belonging to For the topographic information of the region, the sampling results of each scale are regularized to obtain the processed sampling results of at least two scales.
  • the electronic device can input the processed sampling results of at least two scales into a pre-trained neural network model to obtain data with object labels, and perform object segmentation on the data with object labels to obtain data to be processed.
  • a 3D model can be constructed based on the initial Mesh grid or the initial RGB point cloud, and the technical effect described in Figure 1 above can also be achieved.
  • the technical effect please refer to the relevant description in Figure 1, which will not be repeated here. .
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • electronic device 1400 may include: 3D semantic segmentation module 1401, instance segmentation module 1402, plane feature extraction module 1403, space intersection calculation module 1404, polyhedron topology construction module 1405, and surface optimization generation module 1406.
  • the 3D semantic segmentation module 1401 is used to identify and extract point clouds of different categories, for example, point clouds of buildings, roads, vegetation, etc. can be identified and extracted.
  • the 3D semantic segmentation module is used to execute S101-S103 (or S201-S203) in the above embodiment.
  • the instance segmentation module 1402 is used to extract the point cloud or Mesh grid of a single object from the existing single-category point cloud. Specifically, the instance segmentation module is used to execute S104 or S204 in the above embodiment.
  • the plane feature extraction module 1403 is used to extract the plane shape from the building point cloud or Mesh grid data. Specifically, the plane feature extraction module may execute S105 or S205 in the above embodiment.
  • the space intersection calculation module 1404 is configured to use the extracted plane shape to perform 3D space intersection calculation to generate a plane polygon. Specifically, the spatial intersection calculation module may execute S106-S108 (or S206-S208) in the above embodiment.
  • the polyhedron topology construction module 1405 is used to generate a polyhedron with the smallest space by using planar polygons. Specifically, the polyhedron topology construction module can execute S109-S111 (or S209-S211) in the above embodiment.
  • a surface optimization generation module 1406, configured to extract the simplified model surface from the polyhedron to construct a 3D model.
  • the surface optimization generating module may execute S112-S115 (or S212-S215) in the above embodiment.
  • the plane feature extraction module, the space intersection calculation module, the polyhedron topology construction module, and the surface optimization generation module can be integrated into one, such as being integrated into a simplified reconstruction module of a building model, and a simplified reconstruction module of a building model It is used to execute S105-S115 (or S205-S215) in the above embodiment.
  • the plane feature extraction module 1403 is used to extract multiple planes of the object from the data to be processed, the object is an object to be constructed with a 3D model, and the data to be processed is an RGB point cloud or a Mesh grid .
  • the spatial intersection calculation module 1404 is configured to perform pairwise spatial intersection calculations on each plane to obtain multiple candidate surfaces of the object and at least two polyhedrons formed by the multiple candidate surfaces.
  • the polyhedron topology construction module 1405 is used to project the data to be processed onto each candidate surface to obtain the first candidate surface and the first polyhedron corresponding to the first data.
  • the first data is included in the data to be processed.
  • the data to be processed is For RGB point clouds
  • the first data is the first RGB point cloud
  • the data to be processed is a Mesh grid
  • the first data is the first Mesh grid
  • the first candidate surface is included in multiple candidate surfaces
  • the first candidate The surface is a surface of the first polyhedron.
  • the surface optimization generation module 1406 is used to obtain the inner and outer labels of the first polyhedron according to the normal vector of the first data and the vector from the center point of the first polyhedron to the first data, and obtain the inner and outer labels of the first polyhedron according to the first polyhedron's Inside and outside the label, build the 3D model of the object.
  • the inner and outer labels indicate that the first polyhedron is an external polyhedron or an internal polyhedron.
  • the plane feature extraction module 1403 is specifically configured to use the plane where the coplanar data to be processed is located as a plane to obtain multiple planes.
  • the plane feature extraction module 1403 is specifically configured to use the plane where the coplanar RGB point cloud is located as a plane according to the spatial position relationship of the RGB point cloud .
  • the plane feature extraction module 1403 is specifically configured to use the plane where the coplanar Mesh grid is located as a plane according to the adjacency topology of the Mesh grid .
  • the spatial intersection calculation module 1404 is specifically used to perform pairwise spatial intersection calculations on each plane to obtain multiple intersection lines; take the intersection lines as boundaries, and perform Constrained triangulation calculations are performed to obtain multiple candidate surfaces; according to the adjacency relationship of each candidate surface, at least two polyhedrons are obtained.
  • the polyhedron topology construction module 1405 is specifically used to project the data to be processed to each candidate surface.
  • the projection is the vertex of the Mesh grid; according to The distance between the first data and each candidate surface, and the positional relationship between the projection point of the first data and each candidate surface determine the first candidate surface corresponding to the first data.
  • the polyhedron topology construction module 1405 is specifically configured to use the candidate surface that is closest to the first data and that the projection point of the first data falls into as the first candidate surface corresponding to the first data.
  • the surface optimization generation module 1406 is specifically configured to obtain the first multihedron according to the normal vector of the first data and the correlation between the center point of the first polyhedron and the vector of the first data. The weight of the inner and outer labels of the face.
  • the first data is the first Mesh grid
  • the vector from the center point of the first polyhedron to the first data is the vector from the center point of the first polyhedron to the first Mesh grid.
  • the vector of the center point; the weights of the inner and outer labels of all the first polyhedrons contained in the object are used as the nodes of the graph structure, and the adjacency areas between the first polyhedron and adjacent polyhedrons are used as the edges of the graph structure to obtain The inner and outer labels of the first polyhedron.
  • the first polyhedron corresponds to a plurality of first data
  • the correlation includes positive correlation and negative correlation.
  • the surface optimization generation module 1406 is specifically used to obtain the normal vector of each first data, and the correlation between the center point of the first polyhedron and the vector of each first data corresponding to the first polyhedron; The number of correlations and the number of negative correlations get the weights of the inner and outer labels of the first polyhedron.
  • the surface optimization generation module 1406 is specifically configured to use the weights of the inner and outer labels of all the first polyhedrons in the object as the nodes of the graph structure, and use the weights of the first polyhedron and adjacent polyhedrons The adjacency area between is used as the edge of the graph structure, and the maximum flow algorithm or the minimum cut algorithm is used to obtain the inner and outer labels of the first polyhedron.
  • the surface optimization generation module 1406 is specifically configured to use the adjoining surfaces of the inner polyhedron and the outer polyhedron in the object as the surface of the object; and construct a 3D model of the object according to the surface of the object.
  • the data to be processed includes an RGB point cloud or a Mesh grid of an object, or the data to be processed is an RGB point cloud or a Mesh grid of an object.
  • the 3D semantic segmentation module 1401 and the instance segmentation module 1402 are used to obtain the data to be processed from the initial data, the initial data includes Initial RGB point cloud or initial Mesh of at least one type of object.
  • the 3D semantic segmentation module 1401 is specifically configured to perform multi-scale spatial sampling on the initial data to obtain sampling results of at least two scales, combined with the topographic information of the area to which the initial data belongs, to perform a multi-scale spatial sampling for each scale
  • the sampling results are subjected to regularization processing to obtain processed sampling results of at least two scales, and input the processed sampling results of at least two scales into a pre-trained neural network model to obtain data carrying object labels.
  • the instance segmentation module 1402 is specifically configured to perform object segmentation on data carrying object tags to obtain data to be processed.
  • the 3D semantic segmentation module 1401 is specifically configured to perform step A and step B:
  • the i-th center point cloud is the center point of the point cloud contained in the grid, i is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1.
  • step B Determine whether i is smaller than N; if so, add i+1 and continue to perform step A, if not, obtain sampling results of at least two scales.
  • the 3D semantic segmentation module 1401 is specifically configured to perform step A and step B:
  • A take the vertex of the initial Mesh grid as the center of the sphere, split the initial Mesh grid with a sphere of the radius of the i-th scale, and obtain an i-th vertex in each sphere to obtain the sampling result of the i-th scale, the i-th
  • the sampling result of the scale includes the three-dimensional coordinates and color information of the i-th vertex, where i is an integer greater than or equal to 1 and less than N, and N is an integer greater than 1.
  • step B Determine whether i is smaller than N; if so, add i+1 and continue to perform step A, if not, obtain sampling results of at least two scales.
  • the terrain information includes: location ranges of multiple terrain grids in the area to which the initial data belongs, and an altitude value of each terrain grid.
  • the 3D semantic segmentation module 1401 is specifically used to subtract the altitude value of the first terrain grid from the Z value of the initial data falling into the position range of the first terrain grid, so as to complete the classification of the data falling into the first terrain grid
  • Regularization processing of the initial data in the location range of , the first terrain grid is included in multiple terrain grids.
  • the surface optimization generation module 1406 is specifically configured to output a 3D model of the object.
  • the electronic device provided in the embodiment of the present application can implement the embodiments in FIG. 1 , FIG. 2 and FIG. 13 above, and reference can be made to relevant descriptions in the embodiments in FIG. 1 , FIG. 2 and FIG.
  • an embodiment of the present application further provides an electronic device, which may be the electronic device described in the above embodiments, and the electronic device may include: a processor 1501 (such as a CPU ), memory 1502.
  • the memory 1502 may include a high-speed random-access memory (random-access memory, RAM), and may also include a non-volatile memory (non-volatile memory, NVM), such as at least one disk memory, and various instructions may be stored in the memory 1502 , so as to complete various processing functions and realize the method steps of the present application.
  • the electronic device involved in this application may further include: a power supply 1503 , a communication bus 1504 and a communication port 1505 .
  • the above-mentioned communication port 1505 is used to realize connection and communication between the electronic device and other peripheral devices.
  • the memory 1502 is used to store computer-executable program codes, and the program codes include instructions; when the processor 1501 executes the instructions, the instructions cause the processor 1501 of the electronic device to perform the actions in the above-mentioned method embodiments. The principles and technical effects are similar and will not be repeated here.
  • modules or components described in the above embodiments may be one or more integrated circuits configured to implement the above method, for example: one or more application specific integrated circuits (ASIC), or , one or more microprocessors (digital signal processor, DSP), or, one or more field programmable gate arrays (field programmable gate array, FPGA), etc.
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • FPGA field programmable gate array
  • the processing element can be a general-purpose processor, such as a central processing unit (central processing unit, CPU) or other processors that can call program codes such as control device.
  • these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).
  • SOC system-on-a-chip
  • a computer program product includes one or more computer instructions.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center.
  • DSL digital subscriber line
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media.
  • Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)).
  • sequence numbers of the above-mentioned processes do not mean the order of execution, and the order of execution of the processes should be determined by their functions and internal logic, and should not be used in the implementation of this application.
  • the implementation of the examples constitutes no limitation.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Remote Sensing (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

Des modes de réalisation de la présente demande concernent un procédé de construction de modèle 3D, un appareil et un dispositif électronique. Selon le procédé, grâce à la combinaison d'une extraction de forme plane, d'un calcul d'intersection spatiale, d'une projection de plan spatial, d'une construction de topologie polyédrique, d'un calcul d'optimisation de surface permettant la génération de modèle, et de procédés analogues pour une simplification et une reconstruction de modèle d'un objet, la précision du modèle 3D construit peut être améliorée, ce qui permet d'obtenir un modèle à un niveau LOD3. De plus, l'efficacité de construction du modèle 3D peut être améliorée dans un mode de construction automatisé.
PCT/CN2022/134185 2021-12-30 2022-11-24 Procédé de construction de modèle 3d, appareil et dispositif électronique WO2023124676A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111663088.1A CN116416366A (zh) 2021-12-30 2021-12-30 3d模型构建方法、装置和电子设备
CN202111663088.1 2021-12-30

Publications (1)

Publication Number Publication Date
WO2023124676A1 true WO2023124676A1 (fr) 2023-07-06

Family

ID=86997516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/134185 WO2023124676A1 (fr) 2021-12-30 2022-11-24 Procédé de construction de modèle 3d, appareil et dispositif électronique

Country Status (2)

Country Link
CN (1) CN116416366A (fr)
WO (1) WO2023124676A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116580160A (zh) * 2023-07-12 2023-08-11 深圳大学 一种建筑物单体分割和结构重建耦合处理方法及相关装置
CN116777913A (zh) * 2023-08-22 2023-09-19 青岛市勘察测绘研究院 一种三维模型的套合精度检测方法、装置、设备及介质
CN117953167A (zh) * 2024-03-27 2024-04-30 贵州道坦坦科技股份有限公司 基于点云数据的高速公路附属设施建模方法及系统
CN118134980A (zh) * 2024-05-07 2024-06-04 南京信息工程大学 一种基于几何与纹理一致性的倾斜摄影模型轻量化方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118097066B (zh) * 2024-04-26 2024-06-21 厦门两万里文化传媒有限公司 一种自动构建3d模型的生成方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140354626A1 (en) * 2010-05-12 2014-12-04 Google Inc. Block Based Level of Detail Representation
US20160232259A1 (en) * 2015-02-06 2016-08-11 Clearedge3D, Inc. Apparatus and method for interactively extracting shapes from a point cloud
WO2018133851A1 (fr) * 2017-01-22 2018-07-26 腾讯科技（深圳）有限公司 Procédé et appareil de traitement des données de nuage de points, et support de stockage informatique
CN109325998A (zh) * 2018-10-08 2019-02-12 香港理工大学 一种基于点云数据的室内3d建模方法、系统及相关装置
CN113743155A (zh) * 2020-05-28 2021-12-03 华为技术有限公司 一种构建物体的多细节层次模型的方法及计算机设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140354626A1 (en) * 2010-05-12 2014-12-04 Google Inc. Block Based Level of Detail Representation
US20160232259A1 (en) * 2015-02-06 2016-08-11 Clearedge3D, Inc. Apparatus and method for interactively extracting shapes from a point cloud
WO2018133851A1 (fr) * 2017-01-22 2018-07-26 腾讯科技（深圳）有限公司 Procédé et appareil de traitement des données de nuage de points, et support de stockage informatique
CN109325998A (zh) * 2018-10-08 2019-02-12 香港理工大学 一种基于点云数据的室内3d建模方法、系统及相关装置
CN113743155A (zh) * 2020-05-28 2021-12-03 华为技术有限公司 一种构建物体的多细节层次模型的方法及计算机设备

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116580160A (zh) * 2023-07-12 2023-08-11 深圳大学 一种建筑物单体分割和结构重建耦合处理方法及相关装置
CN116580160B (zh) * 2023-07-12 2023-12-15 深圳大学 一种建筑物单体分割和结构重建耦合处理方法及相关装置
CN116777913A (zh) * 2023-08-22 2023-09-19 青岛市勘察测绘研究院 一种三维模型的套合精度检测方法、装置、设备及介质
CN116777913B (zh) * 2023-08-22 2023-11-03 青岛市勘察测绘研究院 一种三维模型的套合精度检测方法、装置、设备及介质
CN117953167A (zh) * 2024-03-27 2024-04-30 贵州道坦坦科技股份有限公司 基于点云数据的高速公路附属设施建模方法及系统
CN117953167B (zh) * 2024-03-27 2024-05-28 贵州道坦坦科技股份有限公司 基于点云数据的高速公路附属设施建模方法及系统
CN118134980A (zh) * 2024-05-07 2024-06-04 南京信息工程大学 一种基于几何与纹理一致性的倾斜摄影模型轻量化方法

Also Published As

Publication number Publication date
CN116416366A (zh) 2023-07-11

Similar Documents

Publication Publication Date Title
WO2023124676A1 (fr) Procédé de construction de modèle 3d, appareil et dispositif électronique
Wang et al. Lidar point clouds to 3-D urban models $: $ A review
CN110120097B (zh) 大场景机载点云语义建模方法
Li et al. Feature-preserving 3D mesh simplification for urban buildings
US7940279B2 (en) System and method for rendering of texel imagery
Verdie et al. LOD generation for urban scenes
US7961934B2 (en) Probable reconstruction of surfaces in occluded regions by computed symmetry
US8711143B2 (en) System and method for interactive image-based modeling of curved surfaces using single-view and multi-view feature curves
CN113781667B (zh) 三维结构简化重建方法、装置、计算机设备和存储介质
CN113066162B (zh) 一种用于电磁计算的城市环境快速建模方法
CN113706713A (zh) 一种实景三维模型裁剪方法、装置及计算机设备
CN113689535B (zh) 基于无人机影像的建筑物模型的生成方法和装置
WO2021238359A1 (fr) Procédé de construction d'un modèle à multiples niveaux de détail d'un objet, et dispositif informatique
CN107545602B (zh) 基于LiDAR点云的空间拓扑关系约束下的建筑物建模方法
WO2023164933A1 (fr) Procédé de modélisation de bâtiment et appareil associé
Han et al. Urban scene LOD vectorized modeling from photogrammetry meshes
Peters et al. Visibility Analysis in a Point Cloud Based on the Medial Axis Transform.
CN116958379A (zh) 图像渲染方法、装置、电子设备、存储介质及程序产品
Zhao et al. Completing point clouds using structural constraints for large-scale points absence in 3D building reconstruction
Ma et al. Rapid reconstruction of a three-dimensional mesh model based on oblique images in the Internet of Things
Frommholz et al. Reconstructing buildings with discontinuities and roof overhangs from oblique aerial imagery
CN117152237A (zh) 一种距离场生成方法、装置、电子设备及存储介质
CN115311418B (zh) 一种多细节层次树木模型单体化重建方法及装置
Sahebdivani et al. Deep learning based classification of color point cloud for 3D reconstruction of interior elements of buildings
Cui et al. A Review of Indoor Automation Modeling Based on Light Detection and Ranging Point Clouds.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22913934

Country of ref document: EP

Kind code of ref document: A1