WO2023164933A1 - 一种建筑物建模方法以及相关装置 - Google Patents

一种建筑物建模方法以及相关装置 Download PDF

Info

Publication number
WO2023164933A1
WO2023164933A1 PCT/CN2022/079311 CN2022079311W WO2023164933A1 WO 2023164933 A1 WO2023164933 A1 WO 2023164933A1 CN 2022079311 W CN2022079311 W CN 2022079311W WO 2023164933 A1 WO2023164933 A1 WO 2023164933A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
building
point cloud
cloud data
information
Prior art date
Application number
PCT/CN2022/079311
Other languages
English (en)
French (fr)
Inventor
黄经纬
张珊珊
段博
张彦峰
孙明伟
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2022/079311 priority Critical patent/WO2023164933A1/zh
Publication of WO2023164933A1 publication Critical patent/WO2023164933A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation

Definitions

  • the present application relates to the field of computers, in particular to a building modeling method and related devices.
  • Scene vectorization is to automatically convert sensor information into a CAD model that contains semantics, a single instance, and a combination of 3D primitives with very few parameters, which meets and exceeds the quality standard of manual modeling.
  • Scene vectorization is widely used in many industries such as game animation, architectural design, smart factory and digital city. At present, scene vectorization relies heavily on manual processing. Due to high labor costs, low fit with the real environment, and ambiguous semantics, it can only achieve human-computer interaction in limited scenarios, and cannot even achieve further intelligent digital analysis.
  • the building information model (building information model, BIM) is obtained by extracting the surface of the data collected by the sensor, which strongly depends on the accuracy of the collected data. If part of the structural plane of the building is not collected by the sensor , the accuracy of the constructed building model is low.
  • existing building BIMs can be built based on point cloud data collected from buildings and rely on closed-loop assumptions. In some scenarios, due to omissions in the collected point cloud data, it is possible to Some surfaces cannot collect complete point cloud data, and thus, cannot form a complete closed loop. In this case, part of the wall or ground area of the building will be missed in BIM.
  • the building modeling method provided in the embodiment of the present application can still obtain a relatively high-precision building model when the data collected for the building is not comprehensive.
  • the present application provides a building modeling method, the method comprising: acquiring point cloud data collected for a target building, the point cloud data including multiple walls of the target building First point cloud data; according to the first point cloud data, topological information is obtained, and the topological information includes the position information of the projected line segments of each of the walls, and at least one of the projected line segments surrounded by a plurality of the projected line segments The position information of the ground area; wherein, a plurality of the projected line segments are formed by the intersection of the straight lines where the projected lines of the plurality of walls on the horizontal plane are located; according to the topological information, through the target neural network, determine each Whether the projected line segment or the ground area is reserved, wherein the reserved projected line segment or the ground area is used to construct the building information model (BIM) of the target building.
  • BIM building information model
  • the BIM of the building is constructed based on the point cloud data collected from the building and relying on the closed-loop assumption.
  • the projection line on the wall it can be extended, and the line segment between the intersection points of the extended projection line (that is, the straight line where the projection line is located) can be used as a candidate projection line segment.
  • the line segment may be real (for example, the point cloud data itself is complete, or the point cloud data itself is missing), or it may not exist (because it is obtained by the intersection of extension lines, it may be true that it does not exist)
  • the neural network can be used to determine whether to keep the projected line segment obtained by the intersection of the extension line, and then the wall with missing part of the point cloud data can also be restored in BIM, thereby increasing the accuracy of BIM construction.
  • the plurality of walls include a first wall, and the first projection line segment of the first wall includes a projection line of the first wall on a horizontal plane and a Extension cord.
  • the topology information further includes position information of a point of intersection where projection lines of the plurality of wall surfaces on the horizontal plane intersect.
  • the structure may include shape and position
  • the projected line segment of the wall on the horizontal plane The structure of the wall surface of the target building is described by the position, and the structure of the ground area of the target building is described by the position of the ground area.
  • the position of the wall surface can be described by the position of the projected line segment of the wall surface on the horizontal plane, and the position of the projected line segment can be determined by the positions of the endpoints at both ends, or the center point of the line segment (or other feature points) combined with directions to describe.
  • the location information of the at least one ground area may also be represented by a location of a surrounding projection line segment.
  • the topology information may include position information of an intersection point where projection lines of the plurality of walls on the horizontal plane intersect.
  • the position information of the intersection point can be described as the coordinate value where the intersection point is located.
  • the position information of each projection line segment is represented by the positions of endpoints at both ends of the target projection line, where the endpoints are intersection points between the target projection lines of the plurality of walls.
  • the location information of the at least one ground area is represented by a location of a surrounding projection line segment.
  • the topology information includes the intersection of the lines where the projection lines of the plurality of walls on the horizontal plane intersect; and the arrangement of the intersection, the projection line segment, and the ground area
  • the data structure used by layout relations includes arrangement.
  • the topology information may be expressed as an arrangement structure (Arrangement).
  • the arrangement layout structure may include the cut atomic line segment (ie, the projection line segment in the embodiment of the present application) and the patch surrounded by the atomic line segment (ie, the ground area in the embodiment of the present application).
  • Arrangement allows isolated points and edges, which is a more general way of expressing planar graphs.
  • the plan prediction based on ArrangementNet not only utilizes explicit geometric data features, but also can learn the potential topological relationship between line segments to generate higher quality plan.
  • the topological information includes the projected line segment obtained by extending the projected line and the ground area, it is necessary to identify whether this part of the projected line segment and the ground area actually exist, and then determine whether it is necessary to This part of the projected line segment and the ground area are retained in BIM.
  • the topological information can be converted into a graph structure, and based on the trained graph neural network (also including the downstream task network, such as the fully connected layer and the output layer) to identify whether this part needs to be preserved in the BIM Project line segments as well as ground areas.
  • the trained graph neural network also including the downstream task network, such as the fully connected layer and the output layer
  • the target graph structure may be obtained according to the topology information, and the target graph structure includes a plurality of nodes and connection relationships between the nodes, where each node corresponds to a An object, where the object is the wall or the ground area.
  • some nodes in the target graph structure correspond to walls.
  • this part of nodes may have a one-to-one correspondence with walls, that is, one node corresponds to one wall.
  • some nodes in the target graph structure correspond to the ground area.
  • this part of the nodes can have a one-to-one correspondence with the ground area, that is, one node corresponds to one ground area.
  • each node can be represented as a feature vector, in a possible implementation, each node in the plurality of nodes includes an object vector for representing the corresponding wall, the The target vector may be related to at least one of the following: the position information of the corresponding wall in the topology information (for example, but not limited to, the position of the center point of the projected line segment corresponding to the wall); or, the corresponding The degree of coincidence (or consistency) between the position information of the wall in the topology information and the position information of the corresponding wall in the point cloud data.
  • the position information of the corresponding wall in the topology information for example, but not limited to, the position of the center point of the projected line segment corresponding to the wall
  • the corresponding The degree of coincidence or consistency
  • each node may be represented as a feature vector, and in a possible implementation, each node in the plurality of nodes includes an object vector for representing the corresponding ground area, the The target vector may be related to at least one of the following: the position information of the corresponding ground area in the topological information; or, the position information of the corresponding ground area in the topological information, and at the point The degree of coincidence (or consistency) between the position information of the corresponding ground area in the cloud data.
  • the objects corresponding to the nodes with connection relationship in the target graph structure are adjacent units, where there are edges between the nodes with connection relationship.
  • the geometric relative position information of the edges can be used as the features of the edges in the graph structure.
  • the nodes that have a connection relationship all correspond to the wall, and the adjacent positions are: the projected line segments of the wall are adjacent and collinear; or,
  • the nodes that have a connection relationship are all corresponding to the wall, and the adjacent positions are: the projected line segment of the wall is used to enclose the same ground area; or,
  • the nodes having a connection relationship correspond to the wall surface and the ground area respectively, and the adjacent positions are: the projection line segment of the wall surface is used to enclose the ground area.
  • the point cloud data further includes: second point cloud data of multiple ceilings of the target building; the multiple ceilings include a first ceiling and a second ceiling; the method further includes include:
  • the boundary of the element can be established for splicing.
  • the roof is composed of multiple primitive planes. For two or two primitives, make intersection lines and project them onto the horizontal plane to form an arrangement diagram (Arrangement).
  • the Arrangement is segmented into atomic units, considering the fitting energy and smoothing energy.
  • the fitting energy item is the distance volume between the roof point cloud and the roof point cloud when a certain patch is classified as a certain plane; for every two patches, the smoothing energy is the difference between the two adjacent patches
  • the energy can be optimized and divided as a whole through the multi-label graph cut algorithm, and the final plane division of the roof can be determined for network construction.
  • the present application provides a building modeling device, the device comprising:
  • An acquisition module configured to acquire point cloud data collected for the target building, where the point cloud data includes first point cloud data of a plurality of walls of the target building;
  • a topological information building module configured to acquire topological information according to the first point cloud data, the topological information includes the position information of the projected line segments of each of the walls, and at least Position information of a ground area; wherein, the plurality of projected line segments are formed by the intersection of straight lines where the projected lines of the plurality of walls on the horizontal plane are located;
  • a modeling module configured to determine whether each of the projected line segments or the ground area is reserved through the target neural network according to the topology information, wherein the reserved projected line segments or the ground area are used to construct A building information model (BIM) of the target building.
  • BIM building information model
  • the plurality of walls include a first wall, and the first projection line segment of the first wall includes a projection line of the first wall on a horizontal plane and a Extension cord.
  • the topology information further includes position information of a point of intersection where projection lines of the plurality of wall surfaces on the horizontal plane intersect.
  • the position information of each projection line segment is represented by the positions of endpoints at both ends of the target projection line, where the endpoints are intersection points between the target projection lines of the plurality of walls.
  • the location information of the at least one ground area is represented by a location of a surrounding projection line segment.
  • the topology information includes the intersection of the lines where the projection lines of the plurality of walls on the horizontal plane intersect; and the arrangement of the intersection, the projection line segment, and the ground area
  • the data structure used by layout relations includes arrangement.
  • the device also includes:
  • a graph structure building module configured to acquire a target graph structure according to the topology information, where the target graph structure includes a plurality of nodes and connection relationships among the plurality of nodes, wherein each node corresponds to an object,
  • the object is the wall or the ground area, and the objects corresponding to the nodes in the connection relationship are adjacent units;
  • the modeling module is specifically used for:
  • the nodes that have a connection relationship all correspond to the wall, and the adjacent positions are: the projected line segments of the wall are adjacent and collinear; or,
  • the nodes that have a connection relationship are all corresponding to the wall, and the adjacent positions are: the projected line segment of the wall is used to enclose the same ground area; or,
  • the nodes having a connection relationship correspond to the wall surface and the ground area respectively, and the adjacent positions are: the projection line segment of the wall surface is used to enclose the ground area.
  • the target neural network includes a graph neural network.
  • each of the plurality of nodes includes a target vector for representing a corresponding object, and the target vector is related to at least one of the following:
  • the point cloud data further includes: second point cloud data of multiple ceilings of the target building; the multiple ceilings include a first ceiling and a second ceiling; the modeling module, also used for:
  • the present application provides a building modeling device, which may include a processor, the processor is coupled to a memory, and the memory stores program instructions.
  • the program instructions stored in the memory are executed by the processor, the above-mentioned first aspect can be realized. described method. For the steps in each possible implementation manner of the processor executing the first aspect, details may refer to the first aspect, which will not be repeated here.
  • the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when it is run on a computer, the computer is made to execute the method described in the above-mentioned first aspect.
  • the present application provides a circuit system, where the circuit system includes a processing circuit configured to execute the method described in the first aspect above.
  • the present application provides a computer program product, including codes, which, when the codes are run on a computer, cause the computer to execute the method described in the first aspect above.
  • the present application provides a chip system, which includes a processor, configured to implement the functions involved in the above aspect, for example, send or process the data and/or information involved in the above method.
  • the chip system further includes a memory, and the memory is used for saving necessary program instructions and data of the server or the communication device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • An embodiment of the present application provides a building modeling method, the method comprising: acquiring point cloud data collected for a target building, the point cloud data including the first of multiple walls of the target building Point cloud data; according to the first point cloud data, topological information is obtained, and the topological information includes position information of each projected line segment of the wall, and at least one ground area surrounded by a plurality of projected line segments position information; wherein, a plurality of the projected line segments are formed by the intersection of the straight lines where the projected lines of the plurality of walls on the horizontal plane are located; according to the topological information, through the target neural network, determine each of the projected Whether the line segment or the ground area is reserved, wherein the reserved projected line segment or the ground area is used to construct the building information model (BIM) of the target building.
  • BIM building information model
  • This application is aimed at the projection line of the wall, which can be extended, and the line segment between the intersection points of the extended projection line (that is, the straight line where the projection line is located) can be used as a candidate projection line segment.
  • This projection line segment may be real. It exists (for example, the point cloud data itself is complete, or the point cloud data itself is missing), or it may not exist (because it is obtained by the intersection between extension lines, it may be true that it does not exist).
  • FIG. 1 is a schematic structural diagram of an application architecture of the present application
  • FIG. 2 is a schematic structural diagram of an application architecture of the present application
  • FIG. 3 is a schematic structural diagram of an application architecture of the present application.
  • FIG. 4 is a schematic diagram of a building modeling method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a construction of topology information provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a construction of topology information provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a construction of topology information provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a construction of topology information provided by an embodiment of the present application.
  • FIG. 9 is a schematic flow chart of a building modeling method provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a building modeling device provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an execution device provided by an embodiment of the present application.
  • Fig. 12 is a schematic structural diagram of a training device provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • Fig. 1 is a schematic diagram of the architecture of the application system of the embodiment of the present application:
  • the application architecture may include point cloud production hardware (4001), main server (4002), and terminal (4003).
  • the point cloud production hardware may include a data acquisition module and a data distribution module, wherein the data acquisition module may exemplarily include: a laser vehicle and an inertial sensor, or a camera group, an RGBD camera, and the like.
  • the data distribution module can store the data collected by the data collection module (such as the point cloud data in the embodiment of the present application) into the memory, and transmit it to the main server through the network card.
  • the hardware of the main server (4002) may include: (a) a network card, used for accepting input point cloud data and distributing output results of building information model (building information model, BIM). (b) Memory, for storage.
  • the terminal (4003) is used to receive the BIM model calculated by the main server, and display the BIM model through the cooperation of memory, GPU, and CPU, so that the user can interact with the terminal.
  • the BIM model can also be analyzed according to business requirements.
  • FIG. 2 is a schematic diagram of the architecture of the application system of the embodiment of the present application. Compared with FIG. 1, FIG. 2 shows the data flow process between various parts in the system architecture:
  • system architecture may include logic modules and hardware modules.
  • the logic module can include a preprocessing module to read the input point cloud data and perform semantic and graphic element segmentation, which is the basis for the successful construction of subsequent BIM main elements.
  • Main element building blocks which include floor plan building blocks, ceiling building blocks, and key structural building blocks such as door and window columns.
  • the main element building module can specifically implement Arrangement-based floor plan construction. Unlike other existing floor plan construction algorithms, it does not need to make assumptions such as Manhattan and room closed loops. It is a general two-dimensional floor plan generation algorithm.
  • the main element building block can specifically realize the ceiling building block based on Arrangement and multi-label graph cut optimization algorithm.
  • the main element building block can specifically realize the door and window column building block based on FCAF3D.
  • the BIM generation module integrates a variety of semantic single elements and uses constructive solid geometry (CSG) to fuse into the final BIM model.
  • CSG constructive solid geometry
  • FIG. 3 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • the system architecture 500 includes an execution device 510 , a training device 520 , a database 530 , a client device 540 , a data storage system 550 and a data collection system 560 .
  • the execution device 510 includes a calculation module 511 , an I/O interface 512 , a preprocessing module 513 and a preprocessing module 514 .
  • a target model/rule 501 may be included in the computation module 511 .
  • the preprocessing module 513 and the preprocessing module 514 can obtain point cloud data, and perform operations such as semantic segmentation and topology construction on the point cloud data.
  • the data collection device 560 is used to collect training samples.
  • the training samples may be point cloud data (the point cloud data may be used to describe the topological information of the building), etc. In the embodiment of the present application, the training samples may be obtained based on the topological information of the building. After collecting the training samples, the data collection device 560 stores these training samples in the database 530 .
  • the training device 520 can obtain the target model/rule 501 based on the training samples maintained in the database 530 and the neural network to be trained (such as the target neural grid in the embodiment of the present application).
  • the training samples maintained in the database 530 are not necessarily collected by the data collection device 560, and may also be received from other devices.
  • the training device 520 does not necessarily perform the training of the target model/rule 501 based entirely on the training samples maintained by the database 530, and it is also possible to obtain training samples from the cloud or other places for model training. Limitations of the Examples.
  • the target model/rule 501 trained according to the training device 520 can be applied to different systems or devices, such as the execution device 510 shown in FIG.
  • Computers, laptops, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) equipment, vehicle-mounted terminals, etc. can also be servers or clouds, etc.
  • the training device 520 may deliver the trained model to the execution device 510 .
  • the execution device 510 is configured with an input/output (input/output, I/O) interface 512 for data interaction with external devices, and the user can input data to the I/O interface 512 through the client device 540 (such as this In the embodiment of the application, it is aimed at point cloud data collected by buildings).
  • I/O input/output
  • the preprocessing module 513 and the preprocessing module 514 are configured to perform preprocessing according to the input data received by the I/O interface 512 . It should be understood that there may be no preprocessing module 513 and preprocessing module 514 or only one preprocessing module. When the preprocessing module 513 and the preprocessing module 514 do not exist, the calculation module 511 may be used directly to process the input data.
  • the execution device 510 When the execution device 510 preprocesses the input data, or in the calculation module 511 of the execution device 510 performs calculation and other related processing, the execution device 510 can call the data, codes, etc. in the data storage system 550 for corresponding processing , the correspondingly processed data and instructions may also be stored in the data storage system 550 .
  • the I/O interface 512 provides the processing result (for example, BIM in the embodiment of the present application) to the client device 540, thereby providing it to the user.
  • the processing result for example, BIM in the embodiment of the present application
  • the user can manually specify input data, and the “manually specify input data” can be operated through the interface provided by the I/O interface 512 .
  • the client device 540 can automatically send the input data to the I/O interface 512 . If the client device 540 is required to automatically send the input data to obtain the user's authorization, the user can set the corresponding authority in the client device 540 .
  • the user can view the results output by the execution device 510 on the client device 540, and the specific presentation form may be specific ways such as display, sound, and action.
  • the client device 540 can also be used as a data collection terminal, collecting input data from the input I/O interface 512 and output results from the output I/O interface 512 as new sample data, and storing them in the database 530 .
  • the data is stored in database 530 .
  • FIG. 3 is only a schematic diagram of a system architecture provided by the embodiment of the present application, and the positional relationship between devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data The storage system 550 is an external memory relative to the execution device 510 , and in other cases, the data storage system 550 may also be placed in the execution device 510 . It should be understood that the above execution device 510 may be deployed in the client device 540 .
  • the calculation module 511 of the execution device 520 can obtain the code stored in the data storage system 550 to implement part or all of the building modeling method in the embodiment of the present application.
  • the calculation module 511 of the execution device 520 may include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, digital signal processing (digital signal processing, DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits, for example, the training device 520 can be a hardware system with the function of executing instructions, such as CPU, DSP, etc. , or a hardware system that does not have the function of executing instructions, such as ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems that do not have the function of executing instructions and hardware systems that have the function of executing instructions.
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • DSP digital signal processing
  • microprocessor or microcontroller etc.
  • the training device 520 can be a hardware system with the function of executing instructions, such as CPU, DSP, etc. , or a hardware system that does not have
  • the computing module 511 of the execution device 520 may be a hardware system capable of executing instructions, part or all of the building modeling method provided in the embodiment of the present application may be software codes stored in the memory, and the execution device 520 The calculation module 511 may acquire software codes from the memory, and execute the acquired software codes to implement part or all of the building modeling method provided in the embodiment of the present application.
  • calculation module 511 of the execution device 520 can be a combination of a hardware system that does not have the function of executing instructions and a hardware system that has the function of executing instructions (such as an AI chip). Or all the steps may also be implemented by a hardware system in the computing module 511 of the execution device 520 that does not have the function of executing instructions, which is not limited here.
  • the neural network can be composed of neural units, and the neural unit can refer to an operation unit that takes xs and intercept 1 as input, and the output of the operation unit can be:
  • Ws is the weight of xs
  • b is the bias of the neuron unit.
  • f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer.
  • the activation function may be a sigmoid function.
  • a neural network is a network formed by connecting many of the above-mentioned single neural units, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected with the local receptive field of the previous layer to extract the features of the local receptive field.
  • the local receptive field can be an area composed of several neural units.
  • the work of each layer in a neural network can be expressed mathematically To describe: From the physical level, the work of each layer in the neural network can be understood as completing the transformation from the input space to the output space (that is, the row space of the matrix to the column space) through five operations on the input space (a collection of input vectors). ), these five operations include: 1. Dimensional up/down; 2. Zoom in/out; 3. Rotate; 4. Translate; 5. "Bend”. Among them, the operations of 1, 2, and 3 are performed by Complete, the operation of 4 is completed by +b, and the operation of 5 is realized by a().
  • W is a weight vector, and each value in the vector represents the weight value of a neuron in this layer of neural network.
  • the vector W determines the space transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space.
  • the purpose of training the neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by the vector W of many layers). Therefore, the training process of the neural network is essentially to learn the way to control the spatial transformation, and more specifically, to learn the weight matrix.
  • Deep Neural Network also known as multi-layer neural network
  • DNN Deep Neural Network
  • the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer.
  • the first layer is the input layer
  • the last layer is the output layer
  • the layers in the middle are all hidden layers.
  • the layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.
  • the coefficient of the kth neuron of the L-1 layer to the jth neuron of the L layer is defined as It should be noted that the input layer has no W parameter.
  • more hidden layers make the network more capable of describing complex situations in the real world. Theoretically speaking, a model with more parameters has a higher complexity and a greater "capacity", which means that it can complete more complex learning tasks.
  • Training the deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by the vector W of many layers).
  • Convolutional Neural Network is a deep neural network with a convolutional structure.
  • a convolutional neural network consists of a feature extractor consisting of a convolutional layer and a subsampling layer.
  • the feature extractor can be seen as a filter, and the convolution process can be seen as using a trainable filter to convolve with an input image or convolutional feature map.
  • the convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network.
  • a neuron can only be connected to some adjacent neurons.
  • a convolutional layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units.
  • Neural units of the same feature plane share weights, and the shared weights here are convolution kernels.
  • Shared weights can be understood as a way to extract image information that is independent of location. The underlying principle is that the statistical information of a certain part of the image is the same as that of other parts. That means that the image information learned in one part can also be used in another part. So for all positions on the image, the same learned image information can be used.
  • multiple convolution kernels can be used to extract different image information. Generally, the more the number of convolution kernels, the richer the image information reflected by the convolution operation.
  • the convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network.
  • the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.
  • the convolutional neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, passing the input signal forward until the output will generate an error loss, and updating the parameters in the initial super-resolution model by backpropagating the error loss information, so that the error loss converges.
  • the backpropagation algorithm is a backpropagation movement dominated by error loss, aiming to obtain the parameters of the optimal super-resolution model, such as the weight matrix.
  • scene vectorization is to automatically convert sensor information into a CAD model that contains semantics, a single instance, and a combination of 3D primitives with very few parameters, which meets and exceeds the quality standard of manual modeling.
  • Scene vectorization is widely used in many industries such as game animation, architectural design, smart factory and digital city. At present, scene vectorization relies heavily on manual processing. Due to high labor costs, low fit with the real environment, and ambiguous semantics, it can only achieve human-computer interaction in limited scenarios, and cannot even achieve further intelligent digital analysis.
  • the BIM is obtained by surface extraction of the data collected by the sensor, which strongly depends on the accuracy of the collected data. If part of the structural plane of the building is not collected by the sensor, the accuracy of the constructed building model lower.
  • the building modeling method provided in the embodiment of the present application can still obtain a relatively high-precision building model when the data collected for the building is not comprehensive.
  • Fig. 4 is a schematic flowchart of a building modeling method provided by the embodiment of the present application. As shown in Fig. 4, the building modeling method provided by the embodiment of the present application includes:
  • the executing device may acquire point cloud data collected for the target building.
  • the collection equipment for collecting the point cloud data of the target building can be a laser scanning vehicle, a camera, a mobile phone, a tablet personal computer, a laptop computer, a digital camera , personal digital assistant (personal digital assistant, PDA), navigation device, mobile internet device (mobile internet device, MID) or wearable device (wearable device), etc., this application does not make specific limitations.
  • the sensor that collects the point cloud data of the target building may be a laser radar, a camera, and the like.
  • the collection device may have real-time upload capability, that is, the collection device may upload the collected data to the server in real time.
  • the collection device may upload the collected data to the server in a unified manner.
  • the collected data is used to execute the device to construct the BIM for the target building.
  • the point cloud data can be obtained in various ways, which may include lidar scanning, stereo camera and multi-view 3D reconstruction and other algorithm recovery.
  • the point cloud data may include multiple point clouds, and the point cloud data may be an Nx6 array representing the three-dimensional coordinates and RGB colors of the N point clouds.
  • the RGB color is optional information.
  • the point cloud data can be used to describe the structural features of the target building (for example, it can be used to describe the structural features of the building's surface).
  • semantic segmentation and primitive segmentation can be performed, for example, a trained semantic segmentation network can be used for semantic segmentation, and a trained primitive segmentation network can be used for primitive segmentation, for example You can use SparseConvNet for semantic segmentation and RegionGrowing for primitive segmentation.
  • each point cloud in the point cloud data can contain corresponding semantic labels and planar primitive parameters.
  • the semantic type can include but not limited to ground, wall, column, door and window, ceiling, table, chair, etc.
  • 0 represents no semantics (for example, for noise points in point cloud data, or point clouds that cannot be recognized by the semantic segmentation network, 0 can be assigned)
  • S represents the total number of semantic types that the system can segment .
  • ⁇ li ⁇ is L
  • the floor can also be determined based on the point cloud data.
  • the scene can be divided into different floors by dividing the primitives whose semantic meaning is the ground. Exemplarily, find the plane semantically as the ground, obtain the median height of the points it contains, and use this number as the height of the horizontal cutting of the floor to segment the point cloud into floors.
  • the point cloud data may include first point cloud data, and the first point cloud data may be point cloud data semantically as a wall in the point cloud data.
  • the first point cloud data may be point cloud data located on the same floor in the point cloud data and semantically referred to as a wall surface.
  • topological information according to the first point cloud data, where the topological information includes position information of each projected line segment of the wall and a position of at least one ground area surrounded by a plurality of projected line segments Information; wherein, the multiple projected line segments are formed by the intersection of straight lines where the projected lines of the multiple walls on the horizontal plane are located.
  • the first point cloud data semantically as a wall can be projected onto the ground (such as a horizontal plane) to become a projection line, and the intersection of the lines where the projection lines of multiple walls on the horizontal plane can form multiple projections Line segments, wherein the endpoints at both ends of each projected line segment may be intersection points between projected lines.
  • the projection line obtained by projecting the point cloud data onto the ground may be obtained by fitting a straight line.
  • the BIM of the building is constructed based on the point cloud data collected from the building and relying on the closed-loop assumption.
  • the collected point cloud data due to omissions in the collected point cloud data, it may affect the It is impossible to collect complete point cloud data on some surfaces, and then a complete closed loop cannot be formed. In this case, some walls or ground areas of buildings in BIM will be missed.
  • Figure 5 The solid line in the figure is the projection result of the point cloud data semantically as the wall in the point cloud data projected to the horizontal plane.
  • the point cloud of the part of the wall corresponding to the projection line 501 Due to the omission of collected data, the point cloud of the part of the wall corresponding to the projection line 501 has not been collected (projection line 501
  • the dotted line in the projection line 502), the point cloud of the part of the wall corresponding to the projection line 502 is not collected (the dotted line in the projection line 502), in this case, because in the closed-loop assumption, only the wall is strictly surrounded by The formed area is the effective ground area. Therefore, the BIM construction of the building is only based on the part of the point cloud data.
  • the ground surrounded by the projection line 501, the projection line 502, the projection line 503 and the projection line 504 in the BIM Region is missing.
  • the projection line on the wall it can be extended, and the line segment between the intersection points of the extended projection line (that is, the straight line where the projection line is located) can be used as a candidate projection line segment.
  • the line segment may be real (for example, the point cloud data itself is complete, or the point cloud data itself is missing), or it may not exist (because it is obtained by the intersection of extension lines, it may be true that it does not exist)
  • the neural network can be used to determine whether to keep the projected line segment obtained by the intersection of the extension line, and then the wall with missing part of the point cloud data can also be restored in BIM, thereby increasing the accuracy of BIM construction.
  • the multiple walls may include a first wall as an example, the first projected line segment of the first wall includes a projected line of the first wall on a horizontal plane and an extension of the projected line.
  • the projection line segment 501 is the intersection point with the extended projection line 501, and the other end of the projection line segment 501 is the intersection point with the projection line 504. Therefore, the projection line segment 504 may include a wall A projected line on the horizontal plane (solid line part) and an extension of said projected line (dotted line part).
  • the structure may include shape and position
  • the projected line segment of the wall on the horizontal plane The structure of the wall surface of the target building is described by the position, and the structure of the ground area of the target building is described by the position of the ground area.
  • the position of the wall surface can be described by the position of the projected line segment of the wall surface on the horizontal plane, and the position of the projected line segment can be determined by the positions of the endpoints at both ends, or the center point of the line segment (or other feature points) combined with directions to describe.
  • the location information of the at least one ground area may also be represented by a location of a surrounding projection line segment.
  • the topology information may include position information of intersection points where the projection lines of the plurality of walls on the horizontal plane intersect.
  • the position information of the intersection point can be described as the coordinate value where the intersection point is located.
  • the position information of each projection line segment is represented by the positions of endpoints at both ends of the target projection line, where the endpoints are intersection points between the target projection lines of the plurality of walls.
  • the position of the projected line segment 501 in FIG. 5 can be described by the positions of the endpoints at both ends of the projected line segment 501 .
  • the location information of the at least one ground area is represented by a location of a surrounding projection line segment.
  • the topology information may be expressed as an arrangement structure (Arrangement).
  • the arrangement layout structure may include the cut atomic line segment (ie, the projection line segment in the embodiment of the present application) and the patch surrounded by the atomic line segment (ie, the ground area in the embodiment of the present application).
  • Arrangement allows isolated points and edges, which is a more general way of expressing planar graphs.
  • the plan prediction based on ArrangementNet not only utilizes explicit geometric data features, but also can learn the potential topological relationship between line segments to generate higher quality plan.
  • the arrangement structure may include the position of each intersection point, the positions of the two end points of each projected line segment, the positions of each ground area and the projected line segments enclosing the ground area.
  • the topological information includes the projected line segment obtained by extending the projected line and the ground area, it is necessary to identify whether this part of the projected line segment and the ground area actually exist, and then determine whether it is necessary to This part of the projected line segment and the ground area are retained in BIM.
  • the topological information can be converted into a graph structure, and based on the trained graph neural network (also including the downstream task network, such as the fully connected layer and the output layer) to identify whether this part needs to be preserved in the BIM Project line segments as well as ground areas.
  • the trained graph neural network also including the downstream task network, such as the fully connected layer and the output layer
  • the target graph structure may be obtained according to the topology information, and the target graph structure includes a plurality of nodes and connection relationships between the nodes, where each node corresponds to a An object, where the object is the wall or the ground area.
  • some nodes in the target graph structure correspond to walls.
  • this part of nodes may have a one-to-one correspondence with walls, that is, one node corresponds to one wall.
  • some of the nodes in the target graph structure correspond to the ground area, and optionally, the part of the nodes may have a one-to-one correspondence with the ground area, that is, one node corresponds to one ground area.
  • each node can be represented as a feature vector, in a possible implementation, each node in the plurality of nodes includes an object vector for representing the corresponding wall, the The target vector may be related to at least one of the following: the position information of the corresponding wall in the topology information (for example, but not limited to, the position of the center point of the projected line segment corresponding to the wall); or, the corresponding The degree of coincidence (or consistency) between the position information of the wall in the topology information and the position information of the corresponding wall in the point cloud data. For example, in FIG. 5, if the dotted line portion of the projected line segment 501 is longer, it means that the coincidence degree between the position information of the corresponding objects in the first point cloud data is lower (or described as the higher the consistency). Difference).
  • each node may be represented as a feature vector, and in a possible implementation, each node in the plurality of nodes includes an object vector for representing the corresponding ground area, the The target vector may be related to at least one of the following: the position information of the corresponding ground area in the topological information; or, the position information of the corresponding ground area in the topological information, and at the point The degree of coincidence (or consistency) between the position information of the corresponding ground area in the cloud data.
  • the objects corresponding to the nodes with connection relationship in the target graph structure are adjacent units, where there are edges between the nodes with connection relationship.
  • the nodes with a connection relationship all correspond to the wall, and the adjacent positions include: projected line segments of the wall are adjacent and collinear.
  • the projection line segment 601 and the projection line segment 602 are adjacent and collinear.
  • the nodes in the connection relationship all correspond to the wall, and the adjacent positions are: the projected line segment of the wall is used to enclose the same ground area.
  • the projection line segment 701 , the projection line segment 702 , the projection line segment 703 and the projection line segment 704 are used to enclose the same ground area.
  • the nodes having a connection relationship respectively correspond to the wall surface and the ground area, and the adjacent positions are: a projected line segment of the wall surface is used to enclose the ground area.
  • the projection line segment 801 , the projection line segment 802 , the projection line segment 803 and the projection line segment 804 are used to surround the same ground area 805 . Therefore, there is a connection relationship between any two nodes in the projected line segment 801 , the projected line segment 802 , the projected line segment 803 , the projected line segment and the ground area 805 .
  • Arrangement can be further constructed into a graph structure as the input of graph neural network.
  • nodes in the graph structure that represent the two elements in the Arrangement: the line segments in the Arrangement represent the wall nodes, and the patches in the Arrangement represent the floor nodes.
  • Three types of edges are constructed in the graph structure for the relationship between two types of nodes:
  • the geometric relative position information of the edges can be used as the features of the edges in the graph structure.
  • the topology information determine whether each of the projected line segments or the ground area is reserved through the target neural network, wherein the reserved projected line segment or the ground area is used to construct the target building building information modeling BIM.
  • the topological information includes the projected line segment obtained by extending the projected line and the ground area, it is necessary to identify whether this part of the projected line segment and the ground area actually exist, and then determine whether it is necessary to This part of the projected line segment and the ground area are retained in BIM.
  • the topological information can be converted into a graph structure, and based on the trained graph neural network (also including the downstream task network, such as the fully connected layer and the output layer) to identify whether this part needs to be preserved in the BIM Project line segments as well as ground areas.
  • the trained graph neural network also including the downstream task network, such as the fully connected layer and the output layer
  • the determining whether each of the projected line segments or the ground area is reserved according to the topology information through the target neural network may specifically include: based on the target graph structure, through the target A neural network that determines whether each node of the plurality of nodes is retained.
  • the input of the graph neural network is the graph structure composed of the basic elements in Arrangement, and the feature vectors of each node and relationship.
  • different network parameters are used for convolution calculation, and the features obtained by edge convolution are gathered back to the adjacent elements themselves through pooling operations.
  • it can include the update of node features and the update of edge features:
  • the features of each node are updated by the features of its adjacent nodes corresponding to all its associated relationships.
  • the formula for calculating the v feature of the k-th layer node is:
  • N(v) represents the neighbor nodes of node v
  • ⁇ represents the proportional coefficient of the original feature in the node feature update
  • f represents the linear projection from the edge feature space to the node feature space
  • ⁇ represents the convolution operation.
  • the edge feature is updated by its two end point features.
  • the update formula of the edge feature composed of uv nodes in the kth layer is:
  • ⁇ represents the proportion coefficient of the original feature in the edge feature update
  • f' represents the linear projection from the node feature space to the edge feature space.
  • the input feature of the initialization element is the coverage (occupancy), size, and center position of the original point cloud, through the 5-layer ArrangementConv, and then through the multi-layer perceptron composed of 3 linear layers to output the Arrangement
  • the binary prediction of each element in is kept as 1 and deleted as 0, and groundtruth supervised learning is used, and the retained elements are the final planar graph.
  • the reserved projected line segment or the ground area is used to construct a building information model (BIM) of the target building.
  • BIM building information model
  • other objects of the target building may also be included in the BIM.
  • the point cloud data further includes: second point cloud data of multiple ceilings of the target building; the multiple ceilings include a first ceiling and a second ceiling;
  • the second point cloud data determine the position information of the first ceiling and the second ceiling; construct an energy equation based on the minimum cut algorithm of the graph, and correct the first ceiling and the second ceiling according to the energy equation
  • the location information of the ceiling, and the corrected location information of the first ceiling and the second ceiling are used to construct a building information model (BIM) of the target building.
  • BIM building information model
  • the boundary of the element can be established for splicing.
  • the roof is composed of multiple primitive planes. For two or two primitives, make intersection lines and project them onto the horizontal plane to form an arrangement diagram (Arrangement).
  • the Arrangement is segmented into atomic units, considering the fitting energy and smoothing energy.
  • the fitting energy item is the distance volume between the roof point cloud and the roof point cloud when a certain patch is classified as a certain plane; for every two patches, the smoothing energy is the difference between the two adjacent patches
  • the energy can be optimized and divided as a whole through the standard Multilabel Graphcut, and the final plane division of the roof can be determined for network construction.
  • FCAF3D For the construction of key structures such as doors, windows and columns, the positions of doors, windows and columns can be learned through FCAF3D. For each object, FCAF3D provides a spatial 3D bounding box.
  • the semantic elements may be fused and modeled in combination with the CSG technology.
  • a set of modeling language is formed, combining various elements (such as layers (floors), floor plans (ground, walls), columns, doors, windows, ceilings) to form the final BIM model.
  • the mathematical description can be:
  • the wall C w and the floor C f can be predicted by the arrangement layout E a function
  • the ceiling C c can be predicted by the E b function (described in S4)
  • the three can be fused.
  • Elements (denoted as function E c ) can be extracted through FCAF3D, including doors, windows and other key semantic elements. Doors and windows are excavated from the first three structures, and the remaining elements are fused through CSG technology.
  • the plane of the ground semantics find the median height of the points it contains, as the ground height, and use the ground height of each layer as the dividing line to cluster the original point cloud to each layer, correspondingly, the semantics and the plane are also clustered class to the corresponding floor. Loop through each floor, construct the corresponding BIM for this floor, and merge to get the final BIM.
  • the preserved line segments are the 2D projections of the real walls that should be preserved, and the preserved patches are the areas that should be filled with floors.
  • FCAF3D Use FCAF3D to extract 3D bounding boxes for key semantics.
  • the 3D bounding box area is subtracted from the intermediate result generated by S4.
  • the CAD models in the model library can be replaced.
  • the three-axis stretching can be directly performed to make the bounding box consistent with the bounding box extracted by FCAF3D.
  • the simplification rate can take 2cm to uniformly thin the point cloud, and output the simplified grid, surface patch and point cloud point number b.
  • Evaluation method Take all indoor scenes, calculate the simplification rate of each scene, and take the average value.
  • Table 1 shows that automatic CAD can achieve a simplification rate very close to that of manual CAD, and the simplification rate of 3D vectorization is much higher than that of network construction.
  • Accuracy ⁇ N indicates the proportion of areas with an error smaller than N.
  • Evaluation method Take the average of the proportion of indoor scenes.
  • Evaluation method Take the average value of the median of the whole scene in the room.
  • An embodiment of the present application provides a building modeling method, the method comprising: acquiring point cloud data collected for a target building, the point cloud data including the first of multiple walls of the target building Point cloud data; according to the first point cloud data, topological information is obtained, and the topological information includes position information of each projected line segment of the wall, and at least one ground area surrounded by a plurality of projected line segments position information; wherein, a plurality of the projected line segments are formed by the intersection of the straight lines where the projected lines of the plurality of walls on the horizontal plane are located; according to the topological information, through the target neural network, determine each of the projected Whether the line segment or the ground area is reserved, wherein the reserved projected line segment or the ground area is used to construct the building information model (BIM) of the target building.
  • BIM building information model
  • This application is aimed at the projection line of the wall, which can be extended, and the line segment between the intersection points of the extended projection line (that is, the straight line where the projection line is located) can be used as a candidate projection line segment.
  • This projection line segment may be real. It exists (for example, the point cloud data itself is complete, or the point cloud data itself is missing), or it may not exist (because it is obtained by the intersection between extension lines, it may be true that it does not exist).
  • the Arrangement structure is used to model the planar graph, and the graph convolutional network based on the Arrangement graph structure is implemented to learn the correct connection relationship of the line segments in the planar graph, and automatically generate a high-quality two-dimensional planar graph.
  • Arrangement-based plan representation does not require Manhattan and contour closure assumptions.
  • the boundary line projection between roof planes is constructed as an Arrangement structure. Construct the energy equation based on the minimum graph cut, and realize the patch segmentation of the arrangement.
  • the fitting energy item is the distance volume between the roof point cloud and the roof point cloud when a certain patch is classified as a certain plane; for every two patches, the smoothing energy is the difference between the two adjacent patches The vertical clearance area caused by the plane.
  • the energy is optimized and divided as a whole, and the final plane division of the roof is determined for network construction.
  • FIG. 10 is a schematic structural diagram of the building modeling device 1000 provided by the embodiment of the present application.
  • the device 1000 includes:
  • An acquisition module 1001 configured to acquire point cloud data collected for a target building, where the point cloud data includes first point cloud data of multiple walls of the target building;
  • step 401 for the specific description of the obtaining module 1001, reference may be made to the description of step 401 in the above embodiment, and details are not repeated here.
  • Topology information construction module 1002 configured to obtain topological information according to the first point cloud data, the topological information includes the position information of the projected line segments of each of the walls, and a plurality of projected line segments surrounded by Position information of at least one ground area; wherein, the multiple projected line segments are formed by the intersection of straight lines where the projected lines of the multiple walls on the horizontal plane are located;
  • topology information construction module 100 For the specific description of the topology information construction module 1002, reference may be made to the description of step 402 in the above embodiment, and details are not repeated here.
  • the modeling module 1003 is configured to determine whether each of the projected line segments or the ground area is reserved according to the topology information through the target neural network, wherein the reserved projected line segment or the ground area is used for A building information model (BIM) of the target building is constructed.
  • BIM building information model
  • the plurality of walls include a first wall, and the first projection line segment of the first wall includes a projection line of the first wall on a horizontal plane and a Extension cord.
  • the topology information further includes position information of a point of intersection where projection lines of the plurality of wall surfaces on the horizontal plane intersect.
  • the position information of each projection line segment is represented by the positions of endpoints at both ends of the target projection line, where the endpoints are intersection points between the target projection lines of the plurality of walls.
  • the location information of the at least one ground area is represented by a location of a surrounding projection line segment.
  • the device also includes:
  • a graph structure building module configured to acquire a target graph structure according to the topology information, where the target graph structure includes a plurality of nodes and connection relationships among the plurality of nodes, wherein each node corresponds to an object,
  • the object is the wall or the ground area, and the objects corresponding to the nodes in the connection relationship are adjacent units;
  • the modeling module is specifically used for:
  • the nodes that have a connection relationship all correspond to the wall, and the adjacent positions are: the projected line segments of the wall are adjacent and collinear; or,
  • the nodes that have a connection relationship are all corresponding to the wall, and the adjacent positions are: the projected line segment of the wall is used to enclose the same ground area; or,
  • the nodes having a connection relationship correspond to the wall surface and the ground area respectively, and the adjacent positions are: the projection line segment of the wall surface is used to enclose the ground area.
  • the target neural network includes a graph neural network.
  • each of the plurality of nodes includes a target vector for representing a corresponding object, and the target vector is related to at least one of the following:
  • the point cloud data further includes: second point cloud data of multiple ceilings of the target building; the multiple ceilings include a first ceiling and a second ceiling; the modeling module, also used for:
  • FIG. 11 is a schematic structural diagram of the execution device provided by the embodiment of the present application. Smart wearable devices, servers, etc. are not limited here.
  • the building modeling apparatus described in the embodiment corresponding to FIG. 10 may be deployed on the executing device 1100 to realize the function of building modeling in the embodiment corresponding to FIG. 11 .
  • the execution device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103, and a memory 1104 (the number of processors 1103 in the execution device 1100 may be one or more, and one processor is taken as an example in FIG. 11 ) , where the processor 1103 may include an application processor 11031 and a communication processor 11032 .
  • the receiver 1101 , the transmitter 1102 , the processor 1103 and the memory 1104 may be connected through a bus or in other ways.
  • the memory 1104 may include read-only memory and random-access memory, and provides instructions and data to the processor 1103 .
  • a part of the memory 1104 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1104 stores processors and operating instructions, executable modules or data structures, or their subsets, or their extended sets, wherein the operating instructions may include various operating instructions for implementing various operations.
  • the processor 1103 controls the operations of the execution device.
  • various components of the execution device are coupled together through a bus system, where the bus system may include not only a data bus, but also a power bus, a control bus, and a status signal bus.
  • the various buses are referred to as bus systems in the figures.
  • the methods disclosed in the foregoing embodiments of the present application may be applied to the processor 1103 or implemented by the processor 1103 .
  • the processor 1103 may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 1103 or instructions in the form of software.
  • the above-mentioned processor 1103 may be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and may further include an application specific integrated circuit (ASIC), field programmable Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • FPGA field programmable Field-programmable gate array
  • the processor 1103 may implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory 1104, and the processor 1103 reads the information in the memory 1104, and completes the steps of the above method in combination with its hardware.
  • the receiver 1101 can be used to receive input digital or character information, and generate signal input related to performing device related settings and function control.
  • the transmitter 1102 can be used to output digital or character information through the first interface; the transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1102 can also include a display device such as a display screen .
  • the processor 1103 is configured to execute the building modeling method in the embodiment corresponding to FIG. 4 .
  • the embodiment of the present application also provides a training device. Please refer to FIG. 12.
  • FIG. 12 Larger differences can be produced due to different configurations or performances, and can include one or more central processing units (central processing units, CPU) 1212 (for example, one or more processors) and memory 1232, one or more storage applications
  • a storage medium 1230 for the program 1242 or data 1244 (such as one or more mass storage devices).
  • the memory 1232 and the storage medium 1230 may be temporary storage or persistent storage.
  • the program stored in the storage medium 1230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the training device.
  • the central processing unit 1212 may be configured to communicate with the storage medium 1230 , and execute a series of instruction operations in the storage medium 1230 on the training device 1200 .
  • the training device 1200 can also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258; or, one or more operating systems 1241, such as Windows ServerTM, Mac OS XTM , UnixTM, LinuxTM, FreeBSDTM and so on.
  • operating systems 1241 such as Windows ServerTM, Mac OS XTM , UnixTM, LinuxTM, FreeBSDTM and so on.
  • the training device may perform the steps related to model training in the foregoing embodiments.
  • the embodiment of the present application also provides a computer program product, which, when running on a computer, causes the computer to perform the steps performed by the aforementioned execution device, or enables the computer to perform the steps performed by the aforementioned training device.
  • An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for signal processing, and when it is run on a computer, the computer executes the steps performed by the aforementioned executing device , or, causing the computer to perform the steps performed by the aforementioned training device.
  • the execution device, training device or terminal device provided in the embodiment of the present application may specifically be a chip.
  • the chip includes: a processing unit and a communication unit.
  • the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, pins or circuits etc.
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chips in the execution device execute the data processing methods described in the above embodiments, or make the chips in the training device execute the data processing methods described in the above embodiments.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM Read-only memory
  • RAM random access memory
  • FIG. 13 is a schematic structural diagram of a chip provided by the embodiment of the present application.
  • the chip can be represented as a neural network processor NPU 1300, and the NPU 1300 is mounted to the main CPU (Host CPU) as a coprocessor. CPU), the tasks are assigned by the Host CPU.
  • the core part of the NPU is the operation circuit 1303, and the operation circuit 1303 is controlled by the controller 1304 to extract matrix data in the memory and perform multiplication operations.
  • the operation circuit 1303 includes multiple processing units (Process Engine, PE).
  • arithmetic circuit 1303 is a two-dimensional systolic array.
  • the arithmetic circuit 1303 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • arithmetic circuit 1303 is a general-purpose matrix processor.
  • the operation circuit fetches the data corresponding to the matrix B from the weight memory 1302, and caches it in each PE in the operation circuit.
  • the operation circuit takes the data of matrix A from the input memory 1301 and performs matrix operation with matrix B, and the obtained partial or final results of the matrix are stored in the accumulator 1308 .
  • the unified memory 1306 is used to store input data and output data.
  • the weight data directly accesses the controller (Direct Memory Access Controller, DMAC) 1305 through the storage unit, and the DMAC is transferred to the weight storage 1302.
  • Input data is also transferred to unified memory 1306 by DMAC.
  • DMAC Direct Memory Access Controller
  • the BIU is the Bus Interface Unit, that is, the bus interface unit 1310, which is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (Instruction Fetch Buffer, IFB) 1309.
  • IFB Instruction Fetch Buffer
  • the bus interface unit 1310 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • BIU Bus Interface Unit
  • the DMAC is mainly used to move the input data in the external memory DDR to the unified memory 1306 , to move the weight data to the weight memory 1302 , or to move the input data to the input memory 1301 .
  • the vector computing unit 1307 includes a plurality of computing processing units, and if necessary, further processes the output of the computing circuit 1303, such as vector multiplication, vector addition, exponent operation, logarithmic operation, size comparison and so on. It is mainly used for non-convolutional/fully connected layer network calculations in neural networks, such as Batch Normalization (batch normalization), pixel-level summation, and upsampling of feature planes.
  • the vector computation unit 1307 can store the vector of the processed output to unified memory 1306 .
  • the vector calculation unit 1307 can apply a linear function; or, a nonlinear function to the output of the operation circuit 1303, such as performing linear interpolation on the feature plane extracted by the convolution layer, and for example, a vector of accumulated values to generate an activation value.
  • the vector computation unit 1307 generates normalized values, pixel-level summed values, or both.
  • the vector of processed outputs can be used as an activation input to operational circuitry 1303, eg, for use in subsequent layers in a neural network.
  • An instruction fetch buffer (instruction fetch buffer) 1309 connected to the controller 1304 is used to store instructions used by the controller 1304;
  • the unified memory 1306, the input memory 1301, the weight memory 1302 and the fetch memory 1309 are all On-Chip memories. External memory is private to the NPU hardware architecture.
  • the processor mentioned above can be a general-purpose central processing unit, microprocessor, ASIC, or one or more integrated circuits for controlling the execution of the above-mentioned programs.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.
  • the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, training device, or network device, etc.) execute the instructions described in various embodiments of the present application method.
  • a computer device which can be a personal computer, training device, or network device, etc.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transferred from a website, computer, training device, or data
  • the center transmits to another website site, computer, training device or data center via wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • wired eg, coaxial cable, fiber optic, digital subscriber line (DSL)
  • wireless eg, infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a training device or a data center integrated with one or more available media.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请实施例公开一种建筑物建模方法，方法包括：获取针对于目标建筑物采集的点云数据，点云数据包含目标建筑物的多个墙面的第一点云数据；根据第一点云数据，获取拓扑信息，拓扑信息包括每个墙面的投影线段的位置信息、以及由多个投影线段包围形成的至少一个地面区域的位置信息；其中，多个投影线段为多个墙面在水平面上的投影线所在的直线相交所形成的；根据拓扑信息，通过目标神经网络，确定每个投影线段或者地面区域是否被保留，其中，被保留的投影线段以及地面区域用于构建目标建筑物的建筑信息模型BIM。本申请可以在对建筑物采集的数据覆盖不全面时，仍然得到较高精度的建筑物模型。

Description

一种建筑物建模方法以及相关装置 技术领域
本申请涉及计算机领域，尤其涉及一种建筑物建模方法以及相关装置。
背景技术
场景矢量化是将传感器信息自动转换为蕴含语义、单体实例、以及极少参数的3D图元组合的CAD模型，达到并超过人工建模的质量标准。场景矢量化被广泛应用于游戏动画，建筑设计，智慧工厂和数字化城市等诸多行业。目前场景矢量化极大依赖于人工处理，由于人工成本过高、与真实环境贴合度低、语义含糊，因此仅能实现有限场景的人机交互，甚至无法实现进一步智能数字化分析。
在现有的实现中，通过对传感器采集的数据进行表面提取来得到建筑信息模型(building information model，BIM)，强依赖于采集的数据的精度，若建筑物的部分结构平面未被传感器采集到，则构建的建筑物模型的精度较低。具体的，现有可以基于对建筑物采集的点云数据，并依赖于闭环假设等来构建建筑物的BIM，在一些场景中，由于采集的点云数据存在遗漏，也就是可能对建筑物的一些表面没办法采集到完整的点云数据，进而，就无法形成一个完整的闭环，在这种情况下，BIM中建筑物的部分墙面或者地面区域会被遗漏。
发明内容
本申请实施例提供的建筑物建模方法，可以在对建筑物采集的数据覆盖不全面时，仍然得到较高精度的建筑物模型。
第一方面，本申请提供了一种建筑物建模方法，所述方法包括：获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
在现有的实现中，基于对建筑物采集的点云数据，并依赖于闭环假设等来构建建筑物的BIM，在一些场景中，由于采集的点云数据存在遗漏，也就是可能对建筑物的一些表面没办法采集到完整的点云数据，进而，就无法形成一个完整的闭环，在这种情况下，BIM中建筑物的部分墙面或者地面区域会被遗漏。本申请实施例中，针对于墙面的投影线，可以对其进行延长，并将延长后的投影线(也就是投影线所在的直线)的交点之间的线段作为候选的投影线段，该投影线段可能是真实存在的(例如点云数据本身是完整的、或者是点云数据本身存在缺失)，也可能是不存在的(由于是延长线之间相交得到的，可能真实就是不存在的)，可以通过神经网络来确定是否保留该由延长线相交得到的投影线段，进而可以将存在点云数据缺失部分的墙面也在BIM中还原出来，进而增加BIM构建的准确性。
在一种可能的实现中，所述多个墙面包括第一墙面，所述第一墙面的第一投影线段包 括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
在一种可能的实现中，所述拓扑信息还包括所述多个墙面在水平面上的投影线所在的直线相交的交点的位置信息。
为了构建目标建筑物的BIM，需要获取到用于描述目标建筑物的墙面以及地面区域的结构(结构可以包括形状和位置)的拓扑信息，例如，可以通过墙面在水平面上的投影线段的位置来描述目标建筑物的墙面的结构，通过地面区域的位置来描述目标建筑物的地面区域的结构。
在一种可能的实现中，其中，墙面的位置可以由墙面在水平面上的投影线段的位置来描述，投影线段的位置可以由两端的端点的位置、或者是线段的中心点(或者其他特征点)结合方向的方式来描述。
在一种可能的实现中，所述至少一个地面区域的位置信息亦可以由包围的投影线段的位置表示。
在一种可能的实现中，拓扑信息可以包括所述多个墙面在水平面上的投影线所在的直线相交的交点的位置信息。其中，交点的位置信息可以描述为交点所在的坐标值。
在一种可能的实现中，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
在一种可能的实现中，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
在一种可能的实现中，所述拓扑信息包括所述多个墙面在水平面上的投影线所在的直线相交的交点；且所述交点、所述投影线段与所述地面区域之间的排列布局关系所用的数据结构包括arrangement。
在一种可能的实现中，拓扑信息可以表示为排列布局结构(Arrangement)。排列布局结构可以包括被切割出的原子线段(即本申请实施例中的投影线段)和原子线段围绕而成的面片(即本申请实施例中的地面区域)。Arrangement中允许包含孤立的点和边，是一种更为通用的平面图表达方式。基于ArrangementNet的平面图预测不仅利用显式的几何数据特征，而且可以学习到潜在的线段间拓扑关系，生成更高质量的平面图。
在一种可能的实现中，在得到拓扑信息之后，由于拓扑信息中包括通过延长投影线得到的投影线段以及地面区域，需要识别出这部分投影线段以及地面区域是否真实存在，进而确定是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，可以将拓扑信息转化为图结构，并基于训练好的图神经网络(还包括下游任务网络，例如全连接层和输出层)来识别是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，可以根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域。
在一种可能的实现中，目标图结构中的部分节点对应于墙面，可选的，该部分节点可以和墙面之间一一对应，也就是一个节点对应一个墙面。
在一种可能的实现中，目标图结构中的部分节点对应于地面区域，可选的，该部分节 点可以和地面区域之间一一对应，也就是一个节点对应一个地面区域。
针对于墙面对应的节点，每个节点可以表示为一个特征向量，在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的墙面的目标向量，所述目标向量可以与如下的至少一种有关：所述对应的墙面在所述拓扑信息中的位置信息(例如但不限于为墙面对应的投影线段的中心点位置)；或者，所述对应的墙面在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的墙面的位置信息之间的重合度(或者称之为一致性)。
针对于地面区域对应的节点，每个节点可以表示为一个特征向量，在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的地面区域的目标向量，所述目标向量可以与如下的至少一种有关：所述对应的地面区域在所述拓扑信息中的位置信息；或者，所述对应的地面区域在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的地面区域的位置信息之间的重合度(或者称之为一致性)。
在一种可能的实现中，目标图结构中存在连接关系的所述节点对应的对象为位置相邻的单元，其中，存在连接关系的所述节点之间存在边。
在一种可能的实现中，可以将边的几何相对位置信息作为图结构中边的特征。
在一种可能的实现中，存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线；或者，
存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域；或者，
存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
在一种可能的实现中，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；所述方法还包括：
根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；
基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
对于屋顶图元，可以确立图元边界，进行拼接。屋顶有多个图元平面组成，对于两两图元，做交线，投影到水平面，形成布局图(Arrangement)。对Arrangement进行面片为原子单位的分割，考虑拟合能量和平滑能量。对于每个原子面片，拟合能量项为将某面片分类为某平面时与屋顶点云的差距体积；对于每两个面片，平滑能量为两个面片相邻处由于所处不同平面引起的垂直间隙面积。可以通过多标签图割算法对能量进行整体优化和分割，确定屋顶最终的平面划分用以构网。
第二方面，本申请提供了一种建筑物建模装置，所述装置包括：
获取模块，用于获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；
拓扑信息构建模块，用于根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括 每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；
建模模块，用于根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
在一种可能的实现中，所述多个墙面包括第一墙面，所述第一墙面的第一投影线段包括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
在一种可能的实现中，所述拓扑信息还包括所述多个墙面在水平面上的投影线所在的直线相交的交点的位置信息。
在一种可能的实现中，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
在一种可能的实现中，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
在一种可能的实现中，所述拓扑信息包括所述多个墙面在水平面上的投影线所在的直线相交的交点；且所述交点、所述投影线段与所述地面区域之间的排列布局关系所用的数据结构包括arrangement。
在一种可能的实现中，所述装置还包括：
图结构构建模块，用于根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域，且存在连接关系的所述节点对应的对象为位置相邻的单元；
所述建模模块，具体用于：
基于所述目标图结构，通过目标神经网络，确定所述多个节点中的各个节点是否被保留。
在一种可能的实现中，存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线；或者，
存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域；或者，
存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
在一种可能的实现中，所述目标神经网络包括图神经网络。
在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的对象的目标向量，所述目标向量与如下的至少一种有关：
所述对应的对象在所述拓扑信息中的位置信息；或者，
所述对应的对象在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的对象的位置信息之间的重合度。
在一种可能的实现中，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；所述建模模块，还用于：
根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；
基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
第三方面，本申请提供了一种建筑物建模装置，可以包括处理器，处理器和存储器耦合，存储器存储有程序指令，当存储器存储的程序指令被处理器执行时实现上述第一方面所述的方法。对于处理器执行第一方面的各个可能实现方式中的步骤，具体均可以参阅第一方面，此处不再赘述。
第四方面，本申请提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。
第五方面，本申请提供了一种电路系统，所述电路系统包括处理电路，所述处理电路配置为执行上述第一方面所述的方法。
第六方面，本申请提供了一种计算机程序产品，包括代码，当代码在计算机上运行时，使得计算机执行上述第一方面所述的方法。
第七方面，本申请提供了一种芯片系统，该芯片系统包括处理器，用于实现上述方面中所涉及的功能，例如，发送或处理上述方法中所涉及的数据和/或信息。在一种可能的设计中，所述芯片系统还包括存储器，所述存储器，用于保存服务器或通信设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。
本申请实施例提供了一种建筑物建模方法，所述方法包括：获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。本申请针对于墙面的投影线，可以对其进行延长，并将延长后的投影线(也就是投影线所在的直线)的交点之间的线段作为候选的投影线段，该投影线段可能是真实存在的(例如点云数据本身是完整的、或者是点云数据本身存在缺失)，也可能是不存在的(由于是延长线之间 相交得到的，可能真实就是不存在的)，可以通过神经网络来确定是否保留该由延长线相交得到的投影线段，进而可以将存在点云数据缺失部分的墙面也在BIM中还原出来，进而增加BIM构建的准确性。
附图说明
图1为本申请的一种应用架构结构示意图；
图2为本申请的一种应用架构结构示意图；
图3为本申请的一种应用架构结构示意图；
图4为本申请实施例提供的一种建筑物建模方法的示意图；
图5为本申请实施例提供的一种拓扑信息构建的示意图；
图6为本申请实施例提供的一种拓扑信息构建的示意图；
图7为本申请实施例提供的一种拓扑信息构建的示意图；
图8为本申请实施例提供的一种拓扑信息构建的示意图；
图9为本申请实施例提供的一种建筑物建模方法的流程示意；
图10为本申请实施例提供的一种建筑物建模装置的示意；
图11为本申请实施例提供的执行设备的一种结构示意图；
图12是本申请实施例提供的训练设备一种结构示意图；
图13为本申请实施例提供的芯片的一种结构示意图。
具体实施方式
下面结合本发明实施例中的附图对本发明实施例进行描述。本发明的实施方式部分使用的术语仅用于对本发明的具体实施例进行解释，而非旨在限定本发明。
下面结合附图，对本申请的实施例进行描述。本领域普通技术人员可知，随着技术的发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
接下来介绍本申请实施例的应用架构：
参照图1，图1为本申请实施例的应用系统的架构示意：
其中，应用架构可以包括点云生产硬件(4001)、主服务器(4002)、终端(4003)。点云生产硬件可以包括数据采集模块和数据分发模块，其中，数据采集模块可以示例性的包括：激光车和惯性传感器，或者相机组、RGBD相机等。数据分发模块可以将数据采集模块采集的数据(例如本申请实施例中的点云数据)存入内存，并通过网卡传输给主服务器。主服务器(4002)的硬件可以包括：(a)网卡，用于接受输入点云数据和分发输出建筑信息模型(building information model，BIM)的结果。(b)内存，用于存储。(c)AI芯片，用于深度学习算法的计算和实现。(d)CPU用于几何处理的算法的计算和实现。其中(c)(d)可以实现本申请实施例中的建筑物建模方法。终端(4003)用于接收主服务器计算完成的BIM模型，通过内存、GPU、CPU进行协同，显示BIM模型，使用户与终端进行交互，可选的，还可以根据业务需求进行BIM模型的分析。
参照图2，图2为本申请实施例的应用系统的架构示意，相比图1，图2示出了系统架构中各部分之间的数据流动过程：
其中，系统架构可以包含逻辑模块和硬件模块。
逻辑模块可以包括预处理模块，用以读取输入点云数据，并执行语义和图元分割，是后续BIM主体元素成功构建的基础。
主体元素构建模块，其中包括平面图构建模块，天花板构建模块和门窗柱体等关键结构的构建模块。示例性的，主体元素构建模块具体可以实现基于Arrangement的平面图构建，不同于现有的其他平面图构建算法，不需要做曼哈顿和房间闭环等假设，是一种通用的二维平面图生成算法。主体元素构建模块具体可以实现基于Arrangement和多标签图割优化算法的天花板构建模块。主体元素构建模块具体可以实现基于FCAF3D的门窗柱体构建模块。
BIM生成模块，融合多种语义化单体元素，使用构造实体几何(constructive solid geometry，CSG)融合成最终的BIM模型。
硬件模块的描述可以参照图1，相似之处不再赘述。
接下来介绍本申请实施例中执行连接关系预测方法的执行主体的更细节的架构。
下面结合图3对本申请实施例提供的系统架构进行详细的介绍。图3为本申请实施例提供的系统架构示意图。如图3所示，系统架构500包括执行设备510、训练设备520、数据库530、客户设备540、数据存储系统550以及数据采集系统560。
执行设备510包括计算模块511、I/O接口512、预处理模块513和预处理模块514。计算模块511中可以包括目标模型/规则501。示例性的，预处理模块513和预处理模块514可以获取到点云数据，并对点云数据进行语义分割、拓扑构建等操作。
数据采集设备560用于采集训练样本。训练样本可以为点云数据(点云数据可以用于描述建筑物的拓扑信息)等等，在本申请实施例中，训练样本可以为基于建筑物的拓扑信息得到的。在采集到训练样本之后，数据采集设备560将这些训练样本存入数据库530。
训练设备520可以基于数据库530中维护训练样本，对待训练的神经网络(例如本申请实施例中的目标神经网格等)，以得到目标模型/规则501。
需要说明的是，在实际应用中，数据库530中维护的训练样本不一定都来自于数据采集 设备560的采集，也有可能是从其他设备接收得到的。另外需要说明的是，训练设备520也不一定完全基于数据库530维护的训练样本进行目标模型/规则501的训练，也有可能从云端或其他地方获取训练样本进行模型训练，上述描述不应该作为对本申请实施例的限定。
根据训练设备520训练得到的目标模型/规则501可以应用于不同的系统或设备中，如应用于图10所示的执行设备510，该执行设备510可以是服务器、终端等，如手机终端，平板电脑，笔记本电脑，增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备，车载终端等，还可以是服务器或者云端等。
具体的，训练设备520可以将训练后的模型传递至执行设备510。
在图10中，执行设备510配置输入/输出(input/output，I/O)接口512，用于与外部设备进行数据交互，用户可以通过客户设备540向I/O接口512输入数据(例如本申请实施例中针对于建筑物采集的点云数据)。
预处理模块513和预处理模块514用于根据I/O接口512接收到的输入数据进行预处理。应理解，可以没有预处理模块513和预处理模块514或者只有的一个预处理模块。当不存在预处理模块513和预处理模块514时，可以直接采用计算模块511对输入数据进行处理。
在执行设备510对输入数据进行预处理，或者在执行设备510的计算模块511执行计算等相关的处理过程中，执行设备510可以调用数据存储系统550中的数据、代码等以用于相应的处理，也可以将相应处理得到的数据、指令等存入数据存储系统550中。
最后，I/O接口512将处理结果(例如本申请实施例中的BIM)提供给客户设备540，从而提供给用户。
在图3所示情况下，用户可以手动给定输入数据，该“手动给定输入数据”可以通过I/O接口512提供的界面进行操作。另一种情况下，客户设备540可以自动地向I/O接口512发送输入数据，如果要求客户设备540自动发送输入数据需要获得用户的授权，则用户可以在客户设备540中设置相应权限。用户可以在客户设备540查看执行设备510输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备540也可以作为数据采集端，采集如图所示输入I/O接口512的输入数据及输出I/O接口512的输出结果作为新的样本数据，并存入数据库530。当然，也可以不经过客户设备540进行采集，而是由I/O接口512直接将如图所示输入I/O接口512的输入数据及输出I/O接口512的输出结果，作为新的样本数据存入数据库530。
值得注意的是，图3仅是本申请实施例提供的一种系统架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在图3中，数据存储系统550相对执行设备510是外部存储器，在其它情况下，也可以将数据存储系统550置于执行设备510中。应理解，上述执行设备510可以部署于客户设备540中。
从模型的推理侧来说：
本申请实施例中，上述执行设备520的计算模块511可以获取到数据存储系统550中存储的代码来实现本申请实施例中的建筑物建模方法中的部分或全部。
本申请实施例中，执行设备520的计算模块511可以包括硬件电路(如专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)、通用处理器、数字信号处理器(digital signal processing，DSP)、微处理器 或微控制器等等)、或这些硬件电路的组合，例如，训练设备520可以为具有执行指令功能的硬件系统，如CPU、DSP等，或者为不具有执行指令功能的硬件系统，如ASIC、FPGA等，或者为上述不具有执行指令功能的硬件系统以及具有执行指令功能的硬件系统的组合。
具体的，执行设备520的计算模块511可以为具有执行指令功能的硬件系统，本申请实施例提供的建筑物建模方法中的部分或全部可以为存储在存储器中的软件代码，执行设备520的计算模块511可以从存储器中获取到软件代码，并执行获取到的软件代码来实现本申请实施例提供的建筑物建模方法中的部分或全部。
应理解，执行设备520的计算模块511可以为不具有执行指令功能的硬件系统以及具有执行指令功能的硬件系统的组合(例如AI芯片)，本申请实施例提供的建筑物建模方法中的部分或全部步骤还可以通过执行设备520的计算模块511中不具有执行指令功能的硬件系统来实现，这里并不限定。
由于本申请实施例涉及大量神经网络的应用，为了便于理解，下面先对本申请实施例涉及的相关术语及神经网络等相关概念进行介绍。
(1)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以xs和截距1为输入的运算单元，该运算单元的输出可以为：
Figure PCTCN2022079311-appb-000001
其中，s＝1、2、……n，n为大于1的自然数，Ws为xs的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，用于将非线性特性引入神经网络中，来将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入。激活函数可以是sigmoid函数。神经网络是将许多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
神经网络中的每一层的工作可以用数学表达式
Figure PCTCN2022079311-appb-000002
来描述：从物理层面神经网络中的每一层的工作可以理解为通过五种对输入空间(输入向量的集合)的操作，完成输入空间到输出空间的变换(即矩阵的行空间到列空间)，这五种操作包括：1、升维/降维；2、放大/缩小；3、旋转；4、平移；5、“弯曲”。其中1、2、3的操作由
Figure PCTCN2022079311-appb-000003
完成，4的操作由+b完成，5的操作则由a()来实现。这里之所以用“空间”二字来表述是因为被分类的对象并不是单个事物，而是一类事物，空间是指这类事物所有个体的集合。其中，W是权重向量，该向量中的每一个值表示该层神经网络中的一个神经元的权重值。该向量W决定着上文所述的输入空间到输出空间的空间变换，即每一层的权重W控制着如何变换空间。训练神经网络的目的，也就是最终得到训练好的神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。因此，神经网络的训练过程本质上就是学习控制空间变换的方式，更具体的就是学习权重矩阵。
因为希望神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网 络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断的调整，直到神经网络能够预测出真正想要的目标值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么神经网络的训练就变成了尽可能缩小这个loss的过程。
(2)深度神经网络
深度神经网络(Deep Neural Network，DNN)，也称多层神经网络，可以理解为具有很多层隐含层的神经网络，这里的“很多”并没有特别的度量标准。从DNN按不同层的位置划分，DNN内部的神经网络可以分为三类：输入层，隐含层，输出层。一般来说第一层是输入层，最后一层是输出层，中间的层数都是隐含层。层与层之间是全连接的，也就是说，第i层的任意一个神经元一定与第i+1层的任意一个神经元相连。虽然DNN看起来很复杂，但是就每一层的工作来说，其实并不复杂，简单来说就是如下线性关系表达式：
Figure PCTCN2022079311-appb-000004
其中，
Figure PCTCN2022079311-appb-000005
是输入向量，
Figure PCTCN2022079311-appb-000006
是输出向量，
Figure PCTCN2022079311-appb-000007
是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量
Figure PCTCN2022079311-appb-000008
经过如此简单的操作得到输出向量
Figure PCTCN2022079311-appb-000009
由于DNN层数多，则系数W和偏移向量
Figure PCTCN2022079311-appb-000010
的数量也就很多了。这些参数在DNN中的定义如下所述：以系数W为例：假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为
Figure PCTCN2022079311-appb-000011
上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。总结就是：第L-1层的第k个神经元到第L层的第j个神经元的系数定义为
Figure PCTCN2022079311-appb-000012
需要注意的是，输入层是没有W参数的。在深度神经网络中，更多的隐含层让网络更能够刻画现实世界中的复杂情形。理论上而言，参数越多的模型复杂度越高，“容量”也就越大，也就意味着它能完成更复杂的学习任务。训练深度神经网络的也就是学习权重矩阵的过程，其最终目的是得到训练好的深度神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。
(3)卷积神经网络
卷积神经网络(CNN，Convolutional Neuron Network)是一种带有卷积结构的深度神经网络。卷积神经网络包含了一个由卷积层和子采样层构成的特征抽取器。该特征抽取器可以看作是滤波器，卷积过程可以看作是使用一个可训练的滤波器与一个输入的图像或者卷积特征平面(feature map)做卷积。卷积层是指卷积神经网络中对输入信号进行卷积处理的神经元层。在卷积神经网络的卷积层中，一个神经元可以只与部分邻层神经元连接。一个卷积层中，通常包含若干个特征平面，每个特征平面可以由一些矩形排列的神经单元组成。 同一特征平面的神经单元共享权重，这里共享的权重就是卷积核。共享权重可以理解为提取图像信息的方式与位置无关。这其中隐含的原理是：图像的某一部分的统计信息与其他部分是一样的。即意味着在某一部分学习的图像信息也能用在另一部分上。所以对于图像上的所有位置，都能使用同样的学习得到的图像信息。在同一卷积层中，可以使用多个卷积核来提取不同的图像信息，一般地，卷积核数量越多，卷积操作反映的图像信息越丰富。
卷积核可以以随机大小的矩阵的形式初始化，在卷积神经网络的训练过程中卷积核可以通过学习得到合理的权重。另外，共享权重带来的直接好处是减少卷积神经网络各层之间的连接，同时又降低了过拟合的风险。
(4)损失函数
在训练深度神经网络的过程中，因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断的调整，直到深度神经网络能够预测出真正想要的目标值或与真正想要的目标值非常接近的值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(los s)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
(5)反向传播算法
卷积神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的超分辨率模型中参数的大小，使得超分辨率模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的超分辨率模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的超分辨率模型的参数，例如权重矩阵。
在一种可能的实现中，场景矢量化是将传感器信息自动转换为蕴含语义、单体实例、以及极少参数的3D图元组合的CAD模型，达到并超过人工建模的质量标准。场景矢量化被广泛应用于游戏动画，建筑设计，智慧工厂和数字化城市等诸多行业。目前场景矢量化极大依赖于人工处理，由于人工成本过高、与真实环境贴合度低、语义含糊，因此仅能实现有限场景的人机交互，甚至无法实现进一步智能数字化分析。
在现有的实现中，通过对传感器采集的数据进行表面提取来得到BIM，强依赖于采集的数据的精度，若建筑物的部分结构平面未被传感器采集到，则构建的建筑物模型的精度较低。本申请实施例提供的建筑物建模方法，可以在对建筑物采集的数据覆盖不全面时，仍然得到较高精度的建筑物模型。
参见图4，图4为本申请实施例提供的一种建筑物建模方法的流程示意，如图4所示，本申请实施例提供的建筑物建模方法，包括：
401、获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多 个墙面的第一点云数据。
在一种可能的实现中，执行设备可以获取到针对于目标建筑物采集的点云数据。
在一种可能的实现中，采集目标建筑物的点云数据的采集设备可以是激光扫描车、相机、以及移动电话、平板电脑(tablet personal computer)、膝上型电脑(laptop computer)、数码相机、个人数字助理(personal digital assistant，PDA)、导航装置、移动上网装置(mobile internet device，MID)或可穿戴式设备(wearable device)等，本申请不做具体限定。
在一种可能的实现中，采集目标建筑物的点云数据的传感器可以是激光雷达、相机等等。
可选的，该采集设备可以具备实时上传能力，即采集设备可以向服务器实时上传采集的数据。也可以是，采集设备将数据采集完成之后，统一将采集的数据向服务器上传。该采集的数据用于执行设备来构建针对目标建筑物的BIM。
示例性的，点云数据可通过多种方式获取，可以可以包括激光雷达扫描、立体摄像机和多视图三维重建等算法恢复。可选的，点云数据可以包括多个点云，点云数据可以是一个Nx6的数组，代表N个点云的三维坐标和RGB颜色。其中，RGB颜色为可选的信息。
在一种可能的实现中，点云数据可以用于描述目标建筑物的结构特征(例如可以用于描述建筑物的表面的结构特征)。
在一种可能的实现中，对于点云数据，可以进行语义分割以及图元分割，例如可以使用训练好的语义分割网络做语义分割、以及使用训练好的图元分割网络做图元分割，例如可以使用SparseConvNet做语义分割和RegionGrowing做图元分割。通过语义分割以及图元分割后，点云数据中的每个点云可以包含对应的语义标签和平面图元参数。
在一种可能的实现中，对于第i个点，可以赋予其一个语义整数值0<＝si<＝S(例如，语义类型可以包含但不限于地面、墙面，柱体，门窗，天花板，桌子，椅子等等)，其中0代表无语义(例如，针对于点云数据中的噪声点、或者是语义分割网络无法识别的点云，可以赋予0)，S代表系统能够分割的语义类型总数。
在一种可能的实现中，每个点云可以赋予一个平面整数值li>＝0，其中0表示未在检测的平面上，被赋予同样平面整数值的点被认为在同样的平面上。假设{li}的最大值是L，则总共有L个平面，每个平面通过平面拟合方法计算其参数<a,b,c,d>，代表平面方程ax+by+cz+d＝0。对于第i个平面(1<＝i<＝L)，可以按如下方法计算它的语义：统计平面所包含的点中各语义所包含的点数最大值，其对应的语义就是平面的语义。
在一种可能的实现中，还可以针对于点云数据进行楼层的确定。具体的，针对于通过分割语义为地面的图元，可以将场景切分成不同的楼层。示例性的，寻找语义为地面的平面，获取其所包含的点的高度中位数，并通过此数作为楼层横向切割的高度，用以切分点云为楼层。
在一种可能的实现中，点云数据可以包括第一点云数据，第一点云数据可以为点云数据中语义为墙面的点云数据。
在一种可能的实现中，第一点云数据可以为点云数据中位于同一楼层且语义为墙面的点云数据。
402、根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影 线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的。
在一种可能的实现中，可以将语义为墙面的第一点云数据投影到地面(例如水平面)成为投影线，多个墙面在水平面上的投影线所在的直线相交可以形成多个投影线段，其中，每个投影线段两端的端点可以为投影线之间的交点。
在一种可能的实现中，点云数据投影到地面(例如水平面)得到的投影线可以为通过直线拟合得到的。
在现有的实现中，基于对建筑物采集的点云数据，并依赖于闭环假设等来构建建筑物的BIM，在一些场景中，由于采集的点云数据存在遗漏，也就是可能对建筑物的一些表面没办法采集到完整的点云数据，进而，就无法形成一个完整的闭环，在这种情况下，BIM中建筑物的部分墙面或者地面区域会被遗漏，参照图5，图5中的实线为点云数据中语义为墙面的点云数据投影到水平面的投影结果，由于采集数据的遗漏，投影线501对应的墙面的部分区域的点云未被采集(投影线501中的虚线部分)，投影线502对应的墙面的部分区域的点云未被采集(投影线502中的虚线部分)，在这种情况下，由于在闭环假设中，只有严格被墙面包围形成的区域才是有效的地面区域，因此，仅仅基于实现部分的点云数据进行建筑物的BIM构建，BIM中由投影线501、投影线502、投影线503以及投影线504包围所形成的地面区域缺失。
本申请实施例中，针对于墙面的投影线，可以对其进行延长，并将延长后的投影线(也就是投影线所在的直线)的交点之间的线段作为候选的投影线段，该投影线段可能是真实存在的(例如点云数据本身是完整的、或者是点云数据本身存在缺失)，也可能是不存在的(由于是延长线之间相交得到的，可能真实就是不存在的)，可以通过神经网络来确定是否保留该由延长线相交得到的投影线段，进而可以将存在点云数据缺失部分的墙面也在BIM中还原出来，进而增加BIM构建的准确性。
以所述多个墙面可以包括第一墙面为例，所述第一墙面的第一投影线段包括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
示例性的，可以参照图5，投影线段501的一端是和延长后的投影线501之间的交点，投影线段501的另一端是和投影线504的交点，因此，投影线段504可以包括墙面在水平面上的投影线(实线部分)以及所述投影线的延长线(虚线部分)。
为了构建目标建筑物的BIM，需要获取到用于描述目标建筑物的墙面以及地面区域的结构(结构可以包括形状和位置)的拓扑信息，例如，可以通过墙面在水平面上的投影线段的位置来描述目标建筑物的墙面的结构，通过地面区域的位置来描述目标建筑物的地面区域的结构。
在一种可能的实现中，其中，墙面的位置可以由墙面在水平面上的投影线段的位置来描述，投影线段的位置可以由两端的端点的位置、或者是线段的中心点(或者其他特征点)结合方向的方式来描述。
在一种可能的实现中，所述至少一个地面区域的位置信息亦可以由包围的投影线段的位置表示。
在一种可能的实现中，拓扑信息可以包括所述多个墙面在水平面上的投影线所在的直 线相交的交点的位置信息。其中，交点的位置信息可以描述为交点所在的坐标值。
在一种可能的实现中，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
例如，参照图5，图5中投影线段501的位置可以由投影线段501两端的端点的位置来描述。
在一种可能的实现中，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
在一种可能的实现中，拓扑信息可以表示为排列布局结构(Arrangement)。排列布局结构可以包括被切割出的原子线段(即本申请实施例中的投影线段)和原子线段围绕而成的面片(即本申请实施例中的地面区域)。Arrangement中允许包含孤立的点和边，是一种更为通用的平面图表达方式。基于ArrangementNet的平面图预测不仅利用显式的几何数据特征，而且可以学习到潜在的线段间拓扑关系，生成更高质量的平面图。
其中，排列布局结构(Arrangement)可以包括各个交点的位置、各个投影线段的两端端点的位置、各个地面区域以及围成地面区域的投影线段的位置。
在一种可能的实现中，在得到拓扑信息之后，由于拓扑信息中包括通过延长投影线得到的投影线段以及地面区域，需要识别出这部分投影线段以及地面区域是否真实存在，进而确定是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，可以将拓扑信息转化为图结构，并基于训练好的图神经网络(还包括下游任务网络，例如全连接层和输出层)来识别是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，可以根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域。
在一种可能的实现中，目标图结构中的部分节点对应于墙面，可选的，该部分节点可以和墙面之间一一对应，也就是一个节点对应一个墙面。
在一种可能的实现中，目标图结构中的部分节点对应于地面区域，可选的，该部分节点可以和地面区域之间一一对应，也就是一个节点对应一个地面区域。
针对于墙面对应的节点，每个节点可以表示为一个特征向量，在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的墙面的目标向量，所述目标向量可以与如下的至少一种有关：所述对应的墙面在所述拓扑信息中的位置信息(例如但不限于为墙面对应的投影线段的中心点位置)；或者，所述对应的墙面在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的墙面的位置信息之间的重合度(或者称之为一致性)。例如，在图5中，若投影线段501的虚线部分越长，则表示在所述第一点云数据中所述对应的对象的位置信息之间的重合度越低(或者描述为一致性越差)。
针对于地面区域对应的节点，每个节点可以表示为一个特征向量，在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的地面区域的目标向量，所述目标向量可以与如下的至少一种有关：所述对应的地面区域在所述拓扑信息中的位置信息；或者，所述对应的地面区域在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的地面 区域的位置信息之间的重合度(或者称之为一致性)。
在一种可能的实现中，目标图结构中存在连接关系的所述节点对应的对象为位置相邻的单元，其中，存在连接关系的所述节点之间存在边。
在一种可能的实现中，存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线。
例如可以参照图6，投影线段601和投影线段602之间位置相邻且共线。
在一种可能的实现中，存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域。
例如可以参照图7，投影线段701、投影线段702、投影线段703以及投影线段704用于包围同一个所述地面区域。
在一种可能的实现中，存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
例如可以参照图8，投影线段801、投影线段802、投影线段803以及投影线段804用于包围同一个所述地面区域805。因此，投影线段801、投影线段802、投影线段803、投影线段和地面区域805中的任意两个节点之间存在连接关系。
以拓扑信息为Arrangement为例，Arrangement可以进一步被构建成图结构，作为图神经网络的输入。图结构中存在两种不同类型的节点表示Arrangement中的两种元素：Arrangement中的线段表示墙节点，Arrangement中的面片表示地板节点。图结构中针对两种节点之间的关系构建三种类型的边：
(1)两条线段相邻共线：相邻具有相同方向向量的线段；
(2)两条线段相邻共面：相邻围成同一个面片的线段；
(3)相邻的线与面：线段与所围绕而成的面片之间的对应关系；
通过计算Arrangement中的元素(线段,面片)与原始点云之间的数据一致性，分别作为图结构中两种节点的特征。可选的，可以将边的几何相对位置信息作为图结构中边的特征。
403、根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
在一种可能的实现中，在得到拓扑信息之后，由于拓扑信息中包括通过延长投影线得到的投影线段以及地面区域，需要识别出这部分投影线段以及地面区域是否真实存在，进而确定是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，可以将拓扑信息转化为图结构，并基于训练好的图神经网络(还包括下游任务网络，例如全连接层和输出层)来识别是否需要在BIM中保留这部分投影线段以及地面区域。
在一种可能的实现中，所述根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，具体可以包括：基于所述目标图结构，通过目标神经网络，确定所述多个节点中的各个节点是否被保留。
接下来给出一个图神经网络的数据处理示意：
在一种可能的实现中，图神经网络的输入为Arrangement中基础元素构成的图结构，以 及每种节点和关系的特征向量。对图结构中每种关系使用不同的网络参数进行卷积计算，并将边卷积得到的特征通过池化操作聚集回边相邻的元素本身。具体可以包括节点特征的更新以及边特征的更新：
(1)节点特征的更新
在一种可能的实现中，每个节点的特征由其所有关联关对应的相邻节点特征更新，例如第k层节点v特征计算公式为：
Figure PCTCN2022079311-appb-000013
其中N(v)表示节点v的邻居节点，λ表示节点特征更新中原有特征所占的比例系数，f表示从边特征空间到节点特征空间的线性投影，⊙表示卷积操作。
(2)边特征的更新
在一种可能的实现中，边特征由其两端点特征更新，例如第k层由uv节点构成的边特征更新公式为：
Figure PCTCN2022079311-appb-000014
其中，γ表示边特征更新中原有特征所占的比例系数，f’表示从节点特征空间到边特征空间的线性投影。
在一种可能的实现中，初始化元素的输入特征为其原始点云的覆盖率(occupancy)、大小、中心位置，通过5层ArrangementConv，再经过3层线性层构成的多层感知机输出对Arrangement中的每个元素的二值预测，保留为1，删除为0，并用groundtruth监督学习，所保留的元素就是最终的平面图。
在一种可能的实现中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
在一种可能的实现中，BIM中还可以包括目标建筑物的其他对象(例如天花板、门窗、柱体等)。
在一种可能的实现中，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；可以根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
对于屋顶图元，可以确立图元边界，进行拼接。屋顶有多个图元平面组成，对于两两图元，做交线，投影到水平面，形成布局图(Arrangement)。对Arrangement进行面片为原子单位的分割，考虑拟合能量和平滑能量。对于每个原子面片，拟合能量项为将某面片分类为某平面时与屋顶点云的差距体积；对于每两个面片，平滑能量为两个面片相邻处由于所处不同平面引起的垂直间隙面积。可以通过标准的Multilabel Graphcut对能量进行整体优化和分割，确定屋顶最终的平面划分用以构网。
对于门窗、柱体等关键结构构建，可以通过FCAF3D学习门窗、柱体的位置。对于每一个物体，FCAF3D提供了一个空间的三维包围盒。
在一种可能的实现中，可以结合CSG技术，将语义元素进行融合建模。根据上述规则，形成一套建模语言，结合多种元素(例如分层(楼层)，平面图(地面、墙)，柱体，门窗，天花板)，构成最终的BIM模型。数学描述可以为：
Figure PCTCN2022079311-appb-000015
具体的，可以通过排列布局E a函数预测墙体C w和地面C f，通过E b函数(描述于S4)预测天花板C c，将三者融合。可以通过FCAF3D提取要素(记为函数E c)，包括门窗以及其他关键语义要素，其中将门窗从前三者结构中挖去，并将其余要素通过CSG技术融合。
接下来结合一个具体示例介绍本申请实施例中的建筑物建模方法：
参照图9，以构建一整个商场为实例，使用Navvis激光车进行商场的扫描，尽可能的覆盖所需要构建的商场区域，从而从激光车中获取Nx6的点云(三维坐标和颜色)。将点云传输至服务器，利用SparseConvNet以及RegionGrowing分别对每个点进行语义标签赋值和平面实例标签赋值；对于每个平面，通过其包含的点拟合出4个参数的平面方程，并找到拥有最大数目点的语义作为每个平面的语义。根据地面语义的平面，找寻其包含的点的中位数高度，作为地面高度，通过每层的地面高度为分界线，将原始点云聚类到每一层，相应的，语义、平面也聚类到对应的楼层中。循环遍历每个楼层，对该楼层构建相应的BIM，并融合得到最后的BIM。
具体的，可以找到语义为墙的该楼层的所有平面，投影到水平面得到二维的线段。通过二维线段的延长相交形成标准的几何排布。通过上述实施例中所描述的方法构成一个几何排布上的图结构，并通过GCN网络对图的元素进行是否保留的二值预测。通过预测得到平面图，具体来说，保留下来的线段为真实应当保留的墙体的二维投影，保留下来的面片为应当有地板填充的区域。
对于语义为天花板的该楼层的所有平面，在3D空间中求得两两相交的交线，将交线投影到2D平面形成几何排布。通过上述实施例中所描述的方法进行几何排布面片的实例分割，对每个面片赋予对应的平面实例。通过平面的参数，可以将面片反投影回3D空间，形成天花板的CAD模型。将平面图的墙体线段拉伸至天花板高度，形成竖直3D墙体。从而得到3D的地、天花、墙的BIM中间结果。
使用FCAF3D对关键语义进行3D包围盒的提取。对于门窗，在S4所生成的中间结果中扣除3D包围盒区域。对于商场其余关键要素，如柱体、楼梯，可以将模型库中的CAD模型进行替换，具体来说，直接进行三个轴的拉伸使其包围盒与FCAF3D提取的包围盒一致即可。通过CSG将这些元素和BIM中间结果融合，形成最终的每楼层BIM。
本申请实施例实施例与现有技术的结果对比可以如下表1：
表1
Figure PCTCN2022079311-appb-000016
其中，简化率可以取2cm均匀抽稀点云，输出简化网格，面片与点云点数b。
评价方法：取所有室内场景，每个场景求简化率，取平均值。
表1表明，自动CAD可以达到与人工CAD极为接近的简化率，并且3D矢量化的简化率 远远高于构网的简化率。
表2
Figure PCTCN2022079311-appb-000017
其中，对于地板(F)、墙(W)、天花板(C)，分别评价：
精度<N表示误差小于N的区域占比。
评价方法：取室内场景占比的平均值。
中位误差：对于上述语义区域取误差中位数。
评价方法：取室全场景中位数的平均值。
本申请实施例提供了一种建筑物建模方法，所述方法包括：获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。本申请针对于墙面的投影线，可以对其进行延长，并将延长后的投影线(也就是投影线所在的直线)的交点之间的线段作为候选的投影线段，该投影线段可能是真实存在的(例如点云数据本身是完整的、或者是点云数据本身存在缺失)，也可能是不存在的(由于是延长线之间相交得到的，可能真实就是不存在的)，可以通过神经网络来确定是否保留该由延长线相交得到的投影线段，进而可以将存在点云数据缺失部分的墙面也在BIM中还原出来，进而增加BIM构建的准确性。
此外，利用Arrangement结构建模平面图，并实现基于Arrangement图结构的图卷积网络学习平面图中线段的正确连接关系，自动生成高质量的二维平面图。基于Arrangement的平面图表达无需曼哈顿以及轮廓封闭性假设。
此外，将屋顶平面之间的边界线投影构建成Arrangement结构。基于最小图割构建能量方程，实现对arrangement的面片分割。对于每个原子面片，拟合能量项为将某面片分类为某平面时与屋顶点云的差距体积；对于每两个面片，平滑能量为两个面片相邻处由于所处不同平面引起的垂直间隙面积。通过标准的多标签图割算法对能量进行整体优化和分割，确定屋顶最终的平面划分用以构网。
接下来从装置的角度对本申请实施例提供的建筑物建模装置，参照图10，图10为本申 请实施例提供的建筑物建模装置1000的结构示意，所述装置1000包括：
获取模块1001，用于获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；
其中，关于获取模块1001的具体描述可以参照上述实施例中步骤401的描述，这里不再赘述。
拓扑信息构建模块1002，用于根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；
其中，关于拓扑信息构建模块1002的具体描述可以参照上述实施例中步骤402的描述，这里不再赘述。
建模模块1003，用于根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
其中，关于建模模块1003的具体描述可以参照上述实施例中步骤403的描述，这里不再赘述。
在一种可能的实现中，所述多个墙面包括第一墙面，所述第一墙面的第一投影线段包括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
在一种可能的实现中，所述拓扑信息还包括所述多个墙面在水平面上的投影线所在的直线相交的交点的位置信息。
在一种可能的实现中，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
在一种可能的实现中，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
在一种可能的实现中，所述装置还包括：
图结构构建模块，用于根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域，且存在连接关系的所述节点对应的对象为位置相邻的单元；
所述建模模块，具体用于：
基于所述目标图结构，通过目标神经网络，确定所述多个节点中的各个节点是否被保留。
在一种可能的实现中，存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线；或者，
存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域；或者，
存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
在一种可能的实现中，所述目标神经网络包括图神经网络。
在一种可能的实现中，所述多个节点中的每个节点包括用于表示对应的对象的目标向量，所述目标向量与如下的至少一种有关：
所述对应的对象在所述拓扑信息中的位置信息；或者，
所述对应的对象在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的对象的位置信息之间的重合度。
在一种可能的实现中，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；所述建模模块，还用于：
根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；
基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
接下来介绍本申请实施例提供的一种执行设备，请参阅图11，图11为本申请实施例提供的执行设备的一种结构示意图，执行设备1100具体可以表现为手机、平板、笔记本电脑、智能穿戴设备、服务器等，此处不做限定。其中，执行设备1100上可以部署有图10对应实施例中所描述的建筑物建模装置，用于实现图11对应实施例中建筑物建模的功能。具体的，执行设备1100包括：接收器1101、发射器1102、处理器1103和存储器1104(其中执行设备1100中的处理器1103的数量可以一个或多个，图11中以一个处理器为例)，其中，处理器1103可以包括应用处理器11031和通信处理器11032。在本申请的一些实施例中，接收器1101、发射器1102、处理器1103和存储器1104可通过总线或其它方式连接。
存储器1104可以包括只读存储器和随机存取存储器，并向处理器1103提供指令和数据。存储器1104的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。存储器1104存储有处理器和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。
处理器1103控制执行设备的操作。具体的应用中，执行设备的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。
上述本申请实施例揭示的方法可以应用于处理器1103中，或者由处理器1103实现。处理器1103可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1103中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1103可以是通用处理器、数字信号处理器(digital signal processing，DSP)、微处理器或微控制器，还可进一步包括专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。该处理器1103可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于 随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1104，处理器1103读取存储器1104中的信息，结合其硬件完成上述方法的步骤。
接收器1101可用于接收输入的数字或字符信息，以及产生与执行设备的相关设置以及功能控制有关的信号输入。发射器1102可用于通过第一接口输出数字或字符信息；发射器1102还可用于通过第一接口向磁盘组发送指令，以修改磁盘组中的数据；发射器1102还可以包括显示屏等显示设备。
本申请实施例中，在一种情况下，处理器1103，用于执行图4对应实施例中的建筑物建模方法。
本申请实施例还提供了一种训练设备，请参阅图12，图12是本申请实施例提供的训练设备一种结构示意图，具体的，训练设备1200由一个或多个服务器实现，训练设备1200可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器(central processing units，CPU)1212(例如，一个或一个以上处理器)和存储器1232，一个或一个以上存储应用程序1242或数据1244的存储介质1230(例如一个或一个以上海量存储设备)。其中，存储器1232和存储介质1230可以是短暂存储或持久存储。存储在存储介质1230的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对训练设备中的一系列指令操作。更进一步地，中央处理器1212可以设置为与存储介质1230通信，在训练设备1200上执行存储介质1230中的一系列指令操作。
训练设备1200还可以包括一个或一个以上电源1226，一个或一个以上有线或无线网络接口1250，一个或一个以上输入输出接口1258；或，一个或一个以上操作系统1241，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM等等。
具体的，训练设备可以执行上述实施例中与模型训练相关的步骤。
本申请实施例中还提供一种包括计算机程序产品，当其在计算机上运行时，使得计算机执行如前述执行设备所执行的步骤，或者，使得计算机执行如前述训练设备所执行的步骤。
本申请实施例中还提供一种计算机可读存储介质，该计算机可读存储介质中存储有用于进行信号处理的程序，当其在计算机上运行时，使得计算机执行如前述执行设备所执行的步骤，或者，使得计算机执行如前述训练设备所执行的步骤。
本申请实施例提供的执行设备、训练设备或终端设备具体可以为芯片，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使执行设备内的芯片执行上述实施例描述的数据处理方法，或者，以使训练设备内的芯片执行上述实施例描述的数据处理方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述无线接入设备端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
具体的，请参阅图13，图13为本申请实施例提供的芯片的一种结构示意图，所述芯片可以表现为神经网络处理器NPU 1300，NPU 1300作为协处理器挂载到主CPU(Host CPU)上，由Host CPU分配任务。NPU的核心部分为运算电路1303，通过控制器1304控制运算电路1303提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路1303内部包括多个处理单元(Process Engine,PE)。在一些实现中，运算电路1303是二维脉动阵列。运算电路1303还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路1303是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器1302中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器1301中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)1308中。
统一存储器1306用于存放输入数据以及输出数据。权重数据直接通过存储单元访问控制器(Direct Memory Access Controller，DMAC)1305，DMAC被搬运到权重存储器1302中。输入数据也通过DMAC被搬运到统一存储器1306中。
BIU为Bus Interface Unit即，总线接口单元1310，用于AXI总线与DMAC和取指存储器(Instruction Fetch Buffer，IFB)1309的交互。
总线接口单元1310(Bus Interface Unit，简称BIU)，用于取指存储器1309从外部存储器获取指令，还用于存储单元访问控制器1305从外部存储器获取输入矩阵A或者权重矩阵B的原数据。
DMAC主要用于将外部存储器DDR中的输入数据搬运到统一存储器1306或将权重数据搬运到权重存储器1302中或将输入数据数据搬运到输入存储器1301中。
向量计算单元1307包括多个运算处理单元，在需要的情况下，对运算电路1303的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。主要用于神经网络中非卷积/全连接层网络计算，如Batch Normalization(批归一化)，像素级求和，对特征平面进行上采样等。
在一些实现中，向量计算单元1307能将经处理的输出的向量存储到统一存储器1306。例如，向量计算单元1307可以将线性函数；或，非线性函数应用到运算电路1303的输出，例如对卷积层提取的特征平面进行线性插值，再例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元1307生成归一化的值、像素级求和的值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路1303的激活输入，例如用于在神经网络中的后续层中的使用。
控制器1304连接的取指存储器(instruction fetch buffer)1309，用于存储控制器1304使用的指令；
统一存储器1306，输入存储器1301，权重存储器1302以及取指存储器1309均为On-Chip存储器。外部存储器私有于该NPU硬件架构。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或一个或多个用于控制上述程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，训练设备，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、训练设备或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、训练设备或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的训练设备、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (25)

  1. 一种建筑物建模方法，其特征在于，所述方法包括：
    获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；
    根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；
    根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
  2. 根据权利要求1所述的方法，其特征在于，所述多个墙面包括第一墙面，所述第一墙面的第一投影线段包括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
  3. 根据权利要求1或2所述的方法，其特征在于，所述拓扑信息还包括所述多个墙面在水平面上的投影线所在的直线相交的交点的位置信息。
  4. 根据权利要求1至3任一所述的方法，其特征在于，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
  5. 根据权利要求1至4任一所述的方法，其特征在于，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
  6. 根据权利要求1至5任一所述的方法，其特征在于，所述拓扑信息包括所述多个墙面在水平面上的投影线所在的直线相交的交点；且所述交点、所述投影线段与所述地面区域之间的排列布局关系所用的数据结构包括arrangement。
  7. 根据权利要求1至6任一所述的方法，其特征在于，所述方法还包括：
    根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域，且存在连接关系的所述节点对应的对象为位置相邻的单元；
    所述根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，包括：
    基于所述目标图结构，通过目标神经网络，确定所述多个节点中的各个节点是否被保留。
  8. 根据权利要求7所述的方法，其特征在于，
    存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线；或者，
    存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域；或者，
    存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
  9. 根据权利要求7或8所述的方法，其特征在于，所述目标神经网络包括图神经网络。
  10. 根据权利要求7至9任一所述的方法，其特征在于，所述多个节点中的每个节点包括用于表示对应的对象的目标向量，所述目标向量与如下的至少一种有关：
    所述对应的对象在所述拓扑信息中的位置信息；或者，
    所述对应的对象在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的对象的位置信息之间的重合度。
  11. 根据权利要求1至10任一所述的方法，其特征在于，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；所述方法还包括：
    根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；
    基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
  12. 一种建筑物建模装置，其特征在于，所述装置包括：
    获取模块，用于获取针对于目标建筑物采集的点云数据，所述点云数据包含所述目标建筑物的多个墙面的第一点云数据；
    拓扑信息构建模块，用于根据所述第一点云数据，获取拓扑信息，所述拓扑信息包括每个所述墙面的投影线段的位置信息、以及由多个所述投影线段包围形成的至少一个地面区域的位置信息；其中，多个所述投影线段为所述多个墙面在水平面上的投影线所在的直线相交所形成的；
    建模模块，用于根据所述拓扑信息，通过目标神经网络，确定每个所述投影线段或者所述地面区域是否被保留，其中，被保留的所述投影线段或者所述地面区域用于构建所述目标建筑物的建筑信息模型BIM。
  13. 根据权利要求12所述的装置，其特征在于，所述多个墙面包括第一墙面，所述第一墙面的第一投影线段包括所述第一墙面在水平面上的投影线以及所述投影线的延长线。
  14. 根据权利要求12或13所述的装置，其特征在于，所述拓扑信息还包括所述多个墙 面在水平面上的投影线所在的直线相交的交点的位置信息。
  15. 根据权利要求12至14任一所述的装置，其特征在于，每个所述投影线段的位置信息由所述目标投影线两端的端点的位置表示，所述端点为所述多个墙面的目标投影线之间的交点。
  16. 根据权利要求12至15任一所述的装置，其特征在于，所述至少一个地面区域的位置信息由包围的投影线段的位置表示。
  17. 根据权利要求12至16任一所述的装置，其特征在于，所述拓扑信息包括所述多个墙面在水平面上的投影线所在的直线相交的交点；且所述交点、所述投影线段与所述地面区域之间的排列布局关系所用的数据结构包括arrangement。
  18. 根据权利要求12至17任一所述的装置，其特征在于，所述装置还包括：
    图结构构建模块，用于根据所述拓扑信息，获取目标图结构，所述目标图结构包括多个节点以及所述多个节点之间的连接关系，其中，每个所述节点对应一个对象，所述对象为所述墙面或者所述地面区域，且存在连接关系的所述节点对应的对象为位置相邻的单元；
    所述建模模块，具体用于：
    基于所述目标图结构，通过目标神经网络，确定所述多个节点中的各个节点是否被保留。
  19. 根据权利要求18所述的装置，其特征在于，
    存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段之间位置相邻且共线；或者，
    存在连接关系的所述节点均对应于所述墙面，所述位置相邻为：所述墙面的投影线段用于包围同一个所述地面区域；或者，
    存在连接关系的所述节点分别对应于所述墙面和所述地面区域，所述位置相邻为：所述墙面的投影线段用于包围所述地面区域。
  20. 根据权利要求18或19所述的装置，其特征在于，所述目标神经网络包括图神经网络。
  21. 根据权利要求18至20任一所述的装置，其特征在于，所述多个节点中的每个节点包括用于表示对应的对象的目标向量，所述目标向量与如下的至少一种有关：
    所述对应的对象在所述拓扑信息中的位置信息；或者，
    所述对应的对象在所述拓扑信息中的位置信息、和在所述点云数据中所述对应的对象的位置信息之间的重合度。
  22. 根据权利要求12至21任一所述的装置，其特征在于，所述点云数据还包括：所述目标建筑物的多个天花板的第二点云数据；所述多个天花板包括第一天花板以及第二天花板；所述建模模块，还用于：
    根据所述第二点云数据，确定所述第一天花板以及所述第二天花板的位置信息；
    基于图的最小割算法，构建能量方程，并根据所述能量方程修正所述第一天花板以及所述第二天花板的位置信息，修正后的所述第一天花板以及所述第二天花板的位置信息用于构建所述目标建筑物的建筑信息模型BIM。
  23. 一种建筑物建模装置，其特征在于，包括：一个或多个处理器和存储器；其中，所述存储器中存储有计算机可读指令；
    所述一个或多个处理器读取所述计算机可读指令，以使所述计算机设备实现如权利要求1至11任一所述的方法。
  24. 一种计算机可读存储介质，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机设备上运行时，使得所述计算机设备执行权利要求1至11任一项所述的方法。
  25. 一种计算机程序产品，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机设备上运行时，使得所述计算机设备执行如权利要求1至11任一所述的方法。
PCT/CN2022/079311 2022-03-04 2022-03-04 一种建筑物建模方法以及相关装置 WO2023164933A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/079311 WO2023164933A1 (zh) 2022-03-04 2022-03-04 一种建筑物建模方法以及相关装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/079311 WO2023164933A1 (zh) 2022-03-04 2022-03-04 一种建筑物建模方法以及相关装置

Publications (1)

Publication Number Publication Date
WO2023164933A1 true WO2023164933A1 (zh) 2023-09-07

Family

ID=87882696

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/079311 WO2023164933A1 (zh) 2022-03-04 2022-03-04 一种建筑物建模方法以及相关装置

Country Status (1)

Country Link
WO (1) WO2023164933A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113256790A (zh) * 2021-05-21 2021-08-13 珠海金山网络游戏科技有限公司 建模方法及装置
CN117454496A (zh) * 2023-12-25 2024-01-26 合肥坤颐建筑科技合伙企业(有限合伙) 消防广播布置方法、装置、设备及存储介质
CN117972357A (zh) * 2024-03-26 2024-05-03 山东科瑞特自动化装备有限责任公司 一种水位测量装置的水位监测数据智能处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110189412A (zh) * 2019-05-13 2019-08-30 武汉大学 基于激光点云的多楼层室内结构化三维建模方法及系统
US20200027266A1 (en) * 2018-07-17 2020-01-23 Uti Limited Partnership Building contour generation from point clouds
CN112070877A (zh) * 2019-05-25 2020-12-11 华为技术有限公司 点云处理方法、装置、设备及计算机可读存储介质
CN112712596A (zh) * 2021-03-29 2021-04-27 深圳大学 一种密集匹配点云建筑物结构化模型精细重建方法
CN112991541A (zh) * 2021-04-06 2021-06-18 中国建筑第二工程局有限公司 基于bim的假山逆向建模方法、装置、设备及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200027266A1 (en) * 2018-07-17 2020-01-23 Uti Limited Partnership Building contour generation from point clouds
CN110189412A (zh) * 2019-05-13 2019-08-30 武汉大学 基于激光点云的多楼层室内结构化三维建模方法及系统
CN112070877A (zh) * 2019-05-25 2020-12-11 华为技术有限公司 点云处理方法、装置、设备及计算机可读存储介质
CN112712596A (zh) * 2021-03-29 2021-04-27 深圳大学 一种密集匹配点云建筑物结构化模型精细重建方法
CN112991541A (zh) * 2021-04-06 2021-06-18 中国建筑第二工程局有限公司 基于bim的假山逆向建模方法、装置、设备及介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HONGXIN ZHANG, FANG YUTONG; LI MING: "Robust Reconstruction Method of 3D Room Layout with Visual-Inertial Module", JOURNAL OF COMPUTER-AIDED DESIGN & COMPUTER GRAPHICS, GAI-KAN BIAN-WEI-HUI, BEIJING, CN, vol. 32, no. 2, 15 February 2020 (2020-02-15), CN , pages 262 - 269, XP093088812, ISSN: 1003-9775, DOI: 10.3724/SP.J.1089.2020.17928 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113256790A (zh) * 2021-05-21 2021-08-13 珠海金山网络游戏科技有限公司 建模方法及装置
CN113256790B (zh) * 2021-05-21 2024-06-07 珠海金山数字网络科技有限公司 建模方法及装置
CN117454496A (zh) * 2023-12-25 2024-01-26 合肥坤颐建筑科技合伙企业(有限合伙) 消防广播布置方法、装置、设备及存储介质
CN117454496B (zh) * 2023-12-25 2024-03-22 合肥坤颐建筑科技合伙企业(有限合伙) 消防广播布置方法、装置、设备及存储介质
CN117972357A (zh) * 2024-03-26 2024-05-03 山东科瑞特自动化装备有限责任公司 一种水位测量装置的水位监测数据智能处理方法
CN117972357B (zh) * 2024-03-26 2024-06-07 山东科瑞特自动化装备有限责任公司 一种水位测量装置的水位监测数据智能处理方法

Similar Documents

Publication Publication Date Title
US11244189B2 (en) Systems and methods for extracting information about objects from scene information
US10297070B1 (en) 3D scene synthesis techniques using neural network architectures
Li et al. Building and optimization of 3D semantic map based on Lidar and camera fusion
US20220414911A1 (en) Three-dimensional reconstruction method and three-dimensional reconstruction apparatus
WO2023164933A1 (zh) 一种建筑物建模方法以及相关装置
CN109559320B (zh) 基于空洞卷积深度神经网络实现视觉slam语义建图功能的方法及系统
CN115456161A (zh) 一种数据处理方法和数据处理系统
WO2022179581A1 (zh) 一种图像处理方法及相关设备
EP4322056A1 (en) Model training method and apparatus
WO2023124676A1 (zh) 3d模型构建方法、装置和电子设备
CN112258565B (zh) 图像处理方法以及装置
CN112801265A (zh) 一种机器学习方法以及装置
CN113449859A (zh) 一种数据处理方法及其装置
WO2023083030A1 (zh) 一种姿态识别方法及其相关设备
US20230401799A1 (en) Augmented reality method and related device
Phalak et al. Scan2plan: Efficient floorplan generation from 3d scans of indoor scenes
CN116310219A (zh) 一种基于条件扩散模型的三维脚型生成方法
WO2022156475A1 (zh) 神经网络模型的训练方法、数据处理方法及装置
WO2022100607A1 (zh) 一种神经网络结构确定方法及其装置
CN114202454A (zh) 图优化方法、系统、计算机程序产品以及存储介质
WO2024017282A1 (zh) 一种数据处理方法及其装置
WO2023273934A1 (zh) 一种模型超参数的选择方法及相关装置
WO2023071658A1 (zh) Ai模型的处理方法、运算方法及装置
WO2023088177A1 (zh) 神经网络模型训练方法、矢量化三维模型建立方法及设备
WO2023051236A1 (zh) 一种偏微分方程求解方法及其相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22929376

Country of ref document: EP

Kind code of ref document: A1