CN117651144A - Deep learning-based building point cloud compression method and system - Google Patents
Deep learning-based building point cloud compression method and system Download PDFInfo
- Publication number
- CN117651144A CN117651144A CN202311639811.1A CN202311639811A CN117651144A CN 117651144 A CN117651144 A CN 117651144A CN 202311639811 A CN202311639811 A CN 202311639811A CN 117651144 A CN117651144 A CN 117651144A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- building point
- local
- feature map
- building
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006835 compression Effects 0.000 title claims abstract description 101
- 238000007906 compression Methods 0.000 title claims abstract description 101
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013135 deep learning Methods 0.000 title claims abstract description 48
- 238000003062 neural network model Methods 0.000 claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 31
- 238000004458 analytical method Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 80
- 230000006870 function Effects 0.000 claims description 24
- 230000004927 fusion Effects 0.000 claims description 24
- 238000013527 convolutional neural network Methods 0.000 claims description 21
- 238000000605 extraction Methods 0.000 claims description 14
- 238000011176 pooling Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 abstract description 10
- 238000010276 construction Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 11
- 238000009826 distribution Methods 0.000 description 8
- 239000013598 vector Substances 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 239000000284 extract Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000006837 decompression Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000005728 strengthening Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/194—Transmission of image signals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Signal Processing (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Image Analysis (AREA)
Abstract
A building point cloud compression method and system based on deep learning are disclosed. Firstly, acquiring a building point cloud to be processed, then, extracting features of the building point cloud to be processed through a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map, then, carrying out global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map, and finally, fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result. Therefore, the construction point cloud compression can be performed more efficiently, accurately and semantically, and an efficient solution is provided for the storage and processing of the point cloud data.
Description
Technical Field
The present disclosure relates to the field of building point cloud compression detection, and more particularly, to a method and system for building point cloud compression based on deep learning.
Background
Building point clouds are a collection of points of a building in three-dimensional space obtained by techniques such as laser scanning or photogrammetry. Since point cloud data typically contains a large number of points, processing and storing such data requires a large amount of computing resources and storage space. Therefore, it is necessary to compress the building point cloud to reduce storage requirements and transmission costs and to improve the processing efficiency of the point cloud data. In addition, the point cloud compression can facilitate data transmission and sharing, so that the point cloud data is easier to transmit and process on a network.
However, conventional point cloud compression schemes are typically based on geometric feature encoding and compression, for example using lossless encoding or mesh-based compression methods. The methods often cannot realize higher compression rate while preserving the geometric information of the point cloud, and still require larger storage space and transmission bandwidth for large-scale building point cloud data. In addition, some existing point cloud compression methods often sample or simplify point cloud data to reduce data volume, and such processing methods may cause important detail and shape information loss, which affects the reconstruction quality of the point cloud data and the accuracy of subsequent analysis tasks.
Accordingly, an optimized deep learning based building point cloud compression scheme is desired.
Disclosure of Invention
The present application has been made in order to solve the above technical problems. The embodiment of the application provides a deep learning-based building point cloud compression method and system, which can perform building point cloud compression more efficiently, accurately and semantically, and provide an efficient solution for the storage and processing of point cloud data.
According to one aspect of the present application, there is provided a deep learning-based building point cloud compression method, including:
acquiring a building point cloud to be processed;
extracting features of the building point cloud to be processed by a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map;
performing global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map; and
and fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
According to another aspect of the present application, there is provided a deep learning-based building point cloud compression system, comprising:
the data acquisition module is used for acquiring the building point cloud to be processed;
the local geometric feature extraction module is used for carrying out feature extraction on the building point cloud to be processed through a local geometric feature extractor based on the deep neural network model so as to obtain a building point cloud local feature map;
the global feature analysis module is used for carrying out global feature analysis on the building point cloud local feature map so as to obtain a building point cloud global feature map; and
and the fusion module is used for fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
Compared with the prior art, the deep learning-based building point cloud compression method and system are characterized in that firstly, the building point cloud to be processed is obtained, then, the feature extraction is carried out on the building point cloud to be processed through a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map, then, global feature analysis is carried out on the building point cloud local feature map to obtain a building point cloud global feature map, and finally, the building point cloud local feature map and the building point cloud global feature map are fused to obtain a building point cloud compression result. Therefore, the construction point cloud compression can be performed more efficiently, accurately and semantically, and an efficient solution is provided for the storage and processing of the point cloud data.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below, which are not intended to be drawn to scale in terms of actual dimensions, with emphasis on illustrating the gist of the present application.
Fig. 1 is a flowchart of a deep learning-based building point cloud compression method according to an embodiment of the present application.
Fig. 2 is a schematic architecture diagram of a deep learning-based building point cloud compression method according to an embodiment of the present application.
Fig. 3 is a block diagram of a deep learning based building point cloud compression system according to an embodiment of the present application.
Fig. 4 is an application scenario diagram of a deep learning-based building point cloud compression method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, are also within the scope of the present application.
As used in this application and in the claims, the terms "a," "an," "the," and/or "the" are not specific to the singular, but may include the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
Flowcharts are used in this application to describe the operations performed by systems according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Aiming at the technical problems, the technical concept of the application is to introduce a data processing and analyzing algorithm based on deep learning to the rear end to analyze and capture local and global characteristics of the building point cloud after the building point cloud is acquired, so that the building point cloud data is effectively compressed, and important characteristics of a building are restored as much as possible during decompression. Therefore, the construction point cloud compression can be performed more efficiently, accurately and semantically, and an efficient solution is provided for the storage and processing of the point cloud data.
Fig. 1 is a flowchart of a deep learning-based building point cloud compression method according to an embodiment of the present application. Fig. 2 is a schematic architecture diagram of a deep learning-based building point cloud compression method according to an embodiment of the present application. As shown in fig. 1 and 2, a deep learning-based building point cloud compression method according to an embodiment of the present application includes the steps of: s110, acquiring a building point cloud to be processed; s120, extracting features of the building point cloud to be processed through a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map; s130, performing global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map; and S140, fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
It should be understood that in step S110, building point cloud data to be processed is collected or generated, the building point cloud being point coordinate data in a three-dimensional space acquired by a laser scan or other sensor for representing the shape and structure of a building. In step S120, feature extraction is performed on the building point cloud to be processed using a depth neural network model-based local geometric feature extractor capable of analyzing local geometric features of each point, such as coordinates, normals, and the like of the points, and converting these features into a form of a local feature map. In step S130, global feature analysis is performed on the building point cloud local feature map, where the global feature analysis may consider features of all points in the point cloud and collect them into a global feature map, and the global feature map may capture global features of the shape and structure of the whole building. In step S140, the local feature map of the building point cloud and the global feature map of the building point cloud are fused to obtain a compression result of the building point cloud, and by fusing the local feature and the global feature, redundancy of the point cloud data can be reduced while important information is maintained, so that effective compression of the building point cloud is realized. By integrating the steps, the deep learning-based building point cloud compression method realizes the compression of building point cloud data by extracting local features and global features and fusing the local features and the global features, can reduce the storage space and transmission bandwidth requirements of the point cloud data, and can maintain the important features of the shape and the structure of the building to a certain extent.
Specifically, in the technical scheme of the application, firstly, a building point cloud to be processed is obtained. It will be appreciated that each point in the building point cloud data contains local geometry around it, such as coordinates, normal vectors, curvature, etc. of the point, which is important to preserve the details and shape of the building. Therefore, in order to extract the local geometric feature information of the building point cloud data, in the technical scheme of the application, the building point cloud to be processed passes through a local geometric feature extractor based on a three-dimensional convolutional neural network model to obtain a building point cloud local feature map. It should be appreciated that the local geometry extractor based on a three-dimensional convolutional neural network model may learn a local feature representation of the point cloud data. That is, by performing operations such as rolling and pooling within a local neighborhood of a building point cloud, local geometry and feature distribution information about the building in the point cloud data can be captured and represented as the building point cloud local feature map. In particular, the building point cloud partial feature map may be regarded as an abstract representation of point cloud data, wherein each feature channel corresponds to feature distribution information of one building partial region in the building point cloud.
Accordingly, in step S120, the deep neural network model is a three-dimensional convolutional neural network model. Specifically, feature extraction is performed on the building point cloud to be processed through a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map, which comprises the following steps: and respectively carrying out three-dimensional convolution processing, mean pooling processing and nonlinear activation processing on input data in forward transfer of layers through each layer of the local geometric feature extractor based on the depth neural network model, wherein the output of the last layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud local feature map, and the input of the first layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud to be processed.
It is worth mentioning that the three-dimensional convolutional neural network (3D Convolutional Neural Network) is a deep learning model and is specially used for processing three-dimensional data, such as point cloud, volume data or video sequence, and the three-dimensional convolutional neural network can retain more spatial information and structural characteristics when processing the three-dimensional data compared with the traditional two-dimensional convolutional neural network (2D CNN). In the building point cloud compression method, a three-dimensional convolutional neural network model is adopted by a local geometric feature extractor based on a deep neural network model. The model extracts the local geometric features of the building point cloud by performing a series of operations on the input data in forward pass of the layers, including three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing. Specifically, the three-dimensional convolution operation performs convolution operation on input data in three directions (width, height, and depth), and local features are extracted by a sliding convolution kernel. Thus, the spatial local structure information in the point cloud data can be captured. The averaging pool operation is used to reduce the spatial resolution of the feature map, reduce the number of parameters, and maintain the translational invariance of the features. It reduces the feature map size by averaging the feature values within each pooling window. Nonlinear activation functions (such as ReLU) introduce nonlinear transformations that increase the expressive power of the model, enabling it to learn more complex feature representations. Through these operations, the three-dimensional convolutional neural network model may extract local geometric features from the building point cloud data, which may capture the shape, structure, and other important local information of the point cloud. These feature maps may be further used for subsequent processing of global feature analysis and point cloud compression. In short, the three-dimensional convolutional neural network model has an important function of extracting local features in the building point cloud processing.
Then, the building point cloud data is considered to contain not only local geometric characteristic information but also overall structural and semantic information. And the global characteristics of the building point cloud can capture the overall shape, distribution and semantic meaning of the building point cloud data, and are very important for reconstruction and analysis tasks of the point cloud. Therefore, in order to fully utilize the feature distribution information of the building point cloud to compress, so as to optimize the feature expression capability and decompression accuracy of the building point cloud, in the technical scheme of the application, the building point cloud local feature map is further processed through a global feature extractor based on a non-local neural network model to obtain a building point cloud global feature map. It should be appreciated that the global feature extractor based on the non-local neural network model may learn a global feature representation of the point cloud data, the model capturing global structure and semantic information of the point cloud by considering relationships between individual points in the point cloud data, such as distances between points, similarities, etc., providing a more comprehensive and rich feature representation for point cloud compression and processing.
Accordingly, in step S130, performing global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map, including: and the building point cloud local feature map is passed through a global feature extractor based on a non-local neural network model to obtain the building point cloud global feature map.
It is worth mentioning that the Non-local neural network model (Non-local Neural Network Model) is a deep learning model for modeling Non-local dependencies, and the Non-local neural network model is capable of capturing long-distance dependencies between input data in a global scope. The core idea of the non-local neural network model is to model global dependencies in the input data by introducing a Self-Attention mechanism (Self-Attention). The self-attention mechanism may calculate the association weights between each element and other elements in the input data to obtain important context information in a global scope. In the building point cloud compression method, a global feature extractor based on a non-local neural network model is used for carrying out global feature analysis on a building point cloud local feature map so as to obtain the global feature map of the building point cloud. By introducing a non-local neural network model, global structure and associated information in the point cloud data can be captured on the basis of local features. Specifically, the global feature extractor processes the local feature map using a non-local neural network model, calculates the associated weights between each point and other points, and applies these weights to the aggregation of the local feature map. Thus, the local feature of each point can be combined with the global information of surrounding points to obtain a global feature map containing global context. The introduction of the non-local neural network model can help capture long-distance dependency relationship among points in point cloud data, extract global features and describe the shape and structure of a building more comprehensively. These global features can be used for subsequent feature fusion and point cloud compression processing to achieve efficient compression and reconstruction of building point cloud data.
It should be understood that the building point cloud local feature map and the building point cloud global feature map capture local geometric feature information, global structure and semantic feature information of building point cloud data respectively. That is, the two feature maps provide point cloud feature representations of different levels and angles. Therefore, in order to comprehensively utilize the local and global feature information of the building point cloud to obtain a more accurate, efficient and semantically rich compression result, in the technical scheme of the application, the building point cloud local feature map and the building point cloud global feature map are further fused to obtain a more comprehensive and accurate building point cloud compression result. Therefore, the compression efficiency of the point cloud of the building can be improved, important information is reserved, and an efficient solution is provided for the storage and processing of the point cloud data.
Accordingly, in step S140, fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result, including: and fusing the building point cloud local feature map and the building point cloud global feature map by using a cascading fusion module structure to obtain the building point cloud compression result.
It should be appreciated that the cascading fusion module structure (Cascade Fusion Module) is a deep learning module for fusing different feature maps. In the building point cloud compression method, a cascading fusion module is used for fusing a local feature map and a global feature map of the building point cloud to obtain a compression result of the point cloud. The main purpose of the cascade fusion module structure is to effectively fuse the features of different scales or different types so as to improve the expression capacity and compression effect of the features. It is typically composed of a plurality of sub-modules, each of which is responsible for feature fusion at a different level or in a different manner. In building point cloud compression, a cascading fusion module structure is used for fusing a local feature map and a global feature map. It can be realized by the following steps: 1. a local feature map and a global feature map are input. 2. And fusing the two feature maps layer by using the sub-modules in the cascade fusion module structure. 3. Each sub-module may employ different fusion strategies such as channel-level fusion, space-level fusion, or attention mechanisms, etc. These policies can effectively combine local and global information and extract important features in the point cloud data. 4. And finally, outputting the fused feature map by the cascading fusion module, namely the compression result of the building point cloud. The complementation between the local features and the global features can be fully utilized by introducing the cascade fusion module structure, and the performance and the effect of the point cloud compression are improved. Through multi-layer and multi-mode fusion, the structure and semantic information in the point cloud data can be better captured, so that a higher-quality point cloud compression result is realized.
Further, in the technical scheme of the application, the building point cloud compression method based on deep learning further comprises a training step: for training the local geometry extractor based on the three-dimensional convolutional neural network model and the global feature extractor based on the non-local neural network model. It should be appreciated that the training step plays a key role in the deep learning-based building point cloud compression method, and by training the local geometric feature extractor and the global feature extractor, they can learn an effective feature representation suitable for building point cloud data, thereby achieving a better point cloud compression effect. Specifically, the training step works as follows: 1. local geometry extractor training: parameters of the local geometry extractor are trained using the labeled building point cloud dataset. In this way, the local geometry extractor can learn how to extract information about local structures and geometry from the raw point cloud data. In the training process, a three-dimensional convolutional neural network model or other network structure suitable for point cloud data processing can be used. 2. Global feature extractor training: parameters of the global feature extractor are trained using the same or different building point cloud data sets. The global feature extractor learns global dependencies and context information in the point cloud data through a non-local neural network model. In the training process, the global feature extractor can effectively capture global features of the point cloud data through methods such as a self-attention mechanism and the like. 3. Parameter optimization: during the training process, the parameters of the local geometric feature extractor and the global feature extractor are optimized through an optimization algorithm (such as random gradient descent), so that the parameters can be better fitted with the building point cloud data set. Through the training step, the local geometry feature extractor and the global feature extractor can learn a feature representation that is appropriate for building point cloud data, thereby providing useful features for subsequent feature fusion and compression steps. The optimization in the training process enables the feature extractors to better capture the structural, geometric and semantic information in the building point cloud data, and improves the quality and efficiency of point cloud compression.
In one example, the training step includes: acquiring training data, wherein the training data comprises building training point clouds to be processed; performing feature extraction on the training point cloud of the building to be processed through the local geometric feature extractor based on the three-dimensional convolutional neural network model to obtain a training building point cloud local feature map; performing global feature analysis on the training building point cloud local feature map through the global feature extractor based on the non-local neural network model to obtain a training building point cloud global feature map; calculating specific loss function values of the training building point cloud local feature map and the training building point cloud global feature map; and training the local geometric feature extractor based on the three-dimensional convolutional neural network model and the global feature extractor based on the non-local neural network model with the specific loss function value.
It should be appreciated that the role of calculating the specific loss function values in the training step is to evaluate the training building point cloud local feature map and the training building point cloud global feature map and to provide feedback signals for optimizing model parameters. Specifically, the function of calculating the specific loss function value is as follows: 1. evaluating the performance of the feature extractor: the specific loss function value may be used to measure the difference between the training building point cloud local feature map and the training building point cloud global feature map and the desired output. By calculating the loss function value, the performance of the feature extractor in the learning process can be evaluated, i.e. whether the feature extractor can accurately extract local and global features related to building training point cloud data. 2. Providing a feedback signal: by calculating the loss function value, information about the performance of the feature extractor and the prediction error can be obtained. This information can be used as part of a back-propagation algorithm to update the parameters of the feature extractor to minimize the loss function value. Through back propagation and parameter updating, the feature extractor can gradually adjust its own parameters to improve the quality and compression effect of the feature representation. 3. Training a guide model: the specific loss function value can be used as an objective function in the training process to guide the learning direction of the model. By minimizing the loss function value, the model can better adapt to the feature distribution and compression requirements of the building training point cloud data. Different loss functions can be designed according to specific tasks and requirements to achieve different optimization objectives, such as minimizing reconstruction errors, maximizing information retention, etc. In summary, calculating a specific loss function value is used in the training step to evaluate the performance of the feature extractor, provide a feedback signal, and guide model training. By optimizing the loss function values, the feature extractor can learn a more efficient representation of features, thereby improving the performance and effect of building point cloud compression.
In particular, in the technical solution of the present application, the training building point cloud local feature map and the training building point cloud global feature map respectively express a local three-dimensional associated feature and a global three-dimensional associated feature of the building training point cloud to be processed, so that when the training building point cloud local feature map and the training building point cloud global feature map are fused, a feature sharing relationship between the local three-dimensional associated feature and the global three-dimensional associated feature is considered, and if the local-global sharing property between key three-dimensional associated features between the training building point cloud local feature map and the training building point cloud global feature map can be improved, the fusion effect of the training building point cloud local feature map and the training building point cloud global feature map can be improved.
That is, in view of feature sharing at a feature sharing angle, feature sharing at a local-global scale that may have a key three-dimensional associated feature between the training building point cloud local feature map and the training building point cloud global feature map, in order to avoid three-dimensional associated feature sharing distribution sparsification when the training building point cloud local feature map and the training building point cloud global feature map are fused, the applicant of the present application introduces a specific loss function for the training building point cloud local feature map and the training building point cloud global feature map in a training process of a model.
Accordingly, in one example, calculating the particular loss function value for the training building point cloud local feature map and the training building point cloud global feature map includes: calculating the specific loss function values of the training building point cloud local feature map and the training building point cloud global feature map according to the following specific loss formulas; wherein the specific loss formula is:
wherein V is 1 Is the training building point cloud local feature vector obtained after the training building point cloud local feature map is unfolded, and V 2 Is the global feature vector of the training building point cloud obtained after the global feature map of the training building point cloud is developed, and II is the same as II 1 And II 2 Respectively a 1-norm and a 2-norm of the feature vector, epsilon is a boundary threshold super-parameter, the feature vectors are all in the form of row vectors,representing vector multiplication, ++>Representing vector subtraction (.) T Representing a transpose operation->Is the specific loss function value.
Specifically, the strengthening of the shared key three-dimensional associated feature between the training building point cloud local feature map and the training building point cloud global feature map can be regarded as the compression of the distribution information of the global feature set, and by performing the distribution sparsification control of the key feature on the basis of reconstructing the relative shape relation of the original feature manifold based on the structural representation between the training building point cloud local feature map and the training building point cloud global feature map, the geometric representation of the sparse but meaningful fusion manifold of the fusion feature can be obtained while strengthening the shared key three-dimensional associated feature between the training building point cloud local feature map and the training building point cloud global feature map, so as to improve the fusion effect of the training building point cloud local feature map and the training building point cloud global feature map. Therefore, the compression of the building point cloud data can be comprehensively carried out based on the local and global characteristics of the building point cloud, so that important characteristics of the building can be restored as much as possible during decompression.
In summary, the deep learning-based building point cloud compression method is explained, which can perform building point cloud compression more efficiently, accurately and semantically, and provides an efficient solution for the storage and processing of point cloud data.
Fig. 3 is a block diagram of a deep learning based building point cloud compression system 100 according to an embodiment of the present application. As shown in fig. 3, a deep learning-based building point cloud compression system 100 according to an embodiment of the present application includes: a data acquisition module 110, configured to acquire a building point cloud to be processed; the local geometric feature extraction module 120 is configured to perform feature extraction on the to-be-processed building point cloud by using a local geometric feature extractor based on a deep neural network model so as to obtain a building point cloud local feature map; the global feature analysis module 130 is configured to perform global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map; and a fusion module 140, configured to fuse the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
In one example, in the deep learning based building point cloud compression system 100 described above, the deep neural network model is a three-dimensional convolutional neural network model.
In one example, in the deep learning-based building point cloud compression system 100, the local geometric feature extraction module 120 is configured to: and respectively carrying out three-dimensional convolution processing, mean pooling processing and nonlinear activation processing on input data in forward transfer of layers through each layer of the local geometric feature extractor based on the depth neural network model, wherein the output of the last layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud local feature map, and the input of the first layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud to be processed.
Here, it will be understood by those skilled in the art that the specific functions and operations of the respective modules in the above-described deep learning-based building point cloud compression system 100 have been described in detail in the above description of the deep learning-based building point cloud compression method with reference to fig. 1 to 2, and thus, repetitive descriptions thereof will be omitted.
As described above, the deep learning-based building point cloud compression system 100 according to the embodiment of the present application may be implemented in various wireless terminals, such as a server or the like having a deep learning-based building point cloud compression algorithm. In one example, the deep learning based building point cloud compression system 100 according to embodiments of the present application may be integrated into a wireless terminal as one software module and/or hardware module. For example, the deep learning based building point cloud compression system 100 may be a software module in the operating system of the wireless terminal or may be an application developed for the wireless terminal; of course, the deep learning based building point cloud compression system 100 may also be one of many hardware modules of the wireless terminal.
Alternatively, in another example, the deep learning based building point cloud compression system 100 and the wireless terminal may also be separate devices, and the deep learning based building point cloud compression system 100 may be connected to the wireless terminal through a wired and/or wireless network and transmit interaction information in a agreed data format.
Fig. 4 is an application scenario diagram of a deep learning-based building point cloud compression method according to an embodiment of the present application. As shown in fig. 4, in this application scenario, first, a building point cloud to be processed (for example, D illustrated in fig. 4) is acquired, and then, the building point cloud to be processed is input into a server (for example, S illustrated in fig. 4) in which a deep learning-based building point cloud compression algorithm is deployed, where the server can process the building point cloud to be processed using the deep learning-based building point cloud compression algorithm to obtain a building point cloud compression result.
Further, the application also provides a lossy point cloud compression method based on learning, specifically, end-to-end training is performed by using the self-encoder. In the encoder stage, two features are designed to obtain efficient compression embedding, point-by-point features are extracted first, then compression and sparsification are carried out to obtain a representation of a low-dimensional local subspace, and meanwhile, in order to improve the accuracy of the reconstructed point cloud, global features and local geometric features of the point cloud are extracted. In the decoder stage, the point cloud can be moved from the aggregation point to a suitable position by a point cloud deconvolution method with point-by-point splitting.
Furthermore, those skilled in the art will appreciate that the various aspects of the invention are illustrated and described in the context of a number of patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.) or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the present application may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present application and is not to be construed as limiting thereof. Although a few exemplary embodiments of this application have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this application. Accordingly, all such modifications are intended to be included within the scope of this application as defined in the claims. It is to be understood that the foregoing is illustrative of the present application and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The application is defined by the claims and their equivalents.
Claims (10)
1. The deep learning-based building point cloud compression method is characterized by comprising the following steps of:
acquiring a building point cloud to be processed;
extracting features of the building point cloud to be processed by a local geometric feature extractor based on a deep neural network model to obtain a building point cloud local feature map;
performing global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map; and
and fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
2. The deep learning-based building point cloud compression method of claim 1, wherein the deep neural network model is a three-dimensional convolutional neural network model.
3. The deep learning-based building point cloud compression method according to claim 2, wherein the feature extraction of the building point cloud to be processed by the local geometric feature extractor based on the deep neural network model to obtain a building point cloud local feature map comprises:
and respectively carrying out three-dimensional convolution processing, mean pooling processing and nonlinear activation processing on input data in forward transfer of layers through each layer of the local geometric feature extractor based on the depth neural network model, wherein the output of the last layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud local feature map, and the input of the first layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud to be processed.
4. The deep learning-based building point cloud compression method of claim 3, wherein performing global feature analysis on the building point cloud local feature map to obtain a building point cloud global feature map comprises:
and the building point cloud local feature map is passed through a global feature extractor based on a non-local neural network model to obtain the building point cloud global feature map.
5. The deep learning-based building point cloud compression method of claim 4, wherein fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result comprises:
and fusing the building point cloud local feature map and the building point cloud global feature map by using a cascading fusion module structure to obtain the building point cloud compression result.
6. The deep learning based building point cloud compression method of claim 5, further comprising the training step of: for training the local geometry extractor based on the three-dimensional convolutional neural network model and the global feature extractor based on the non-local neural network model.
7. The deep learning based building point cloud compression method of claim 6, wherein the training step comprises:
acquiring training data, wherein the training data comprises building training point clouds to be processed;
performing feature extraction on the training point cloud of the building to be processed through the local geometric feature extractor based on the three-dimensional convolutional neural network model to obtain a training building point cloud local feature map;
performing global feature analysis on the training building point cloud local feature map through the global feature extractor based on the non-local neural network model to obtain a training building point cloud global feature map;
calculating specific loss function values of the training building point cloud local feature map and the training building point cloud global feature map; and
training the local geometric feature extractor based on the three-dimensional convolutional neural network model and the global feature extractor based on the non-local neural network model with the specific loss function value.
8. A deep learning-based building point cloud compression system, comprising:
the data acquisition module is used for acquiring the building point cloud to be processed;
the local geometric feature extraction module is used for carrying out feature extraction on the building point cloud to be processed through a local geometric feature extractor based on the deep neural network model so as to obtain a building point cloud local feature map;
the global feature analysis module is used for carrying out global feature analysis on the building point cloud local feature map so as to obtain a building point cloud global feature map; and
and the fusion module is used for fusing the building point cloud local feature map and the building point cloud global feature map to obtain a building point cloud compression result.
9. The deep learning based building point cloud compression system of claim 8, wherein the deep neural network model is a three-dimensional convolutional neural network model.
10. The deep learning based building point cloud compression system of claim 9, wherein the local geometric feature extraction module is configured to:
and respectively carrying out three-dimensional convolution processing, mean pooling processing and nonlinear activation processing on input data in forward transfer of layers through each layer of the local geometric feature extractor based on the depth neural network model, wherein the output of the last layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud local feature map, and the input of the first layer of the local geometric feature extractor based on the depth neural network model is used as the building point cloud to be processed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311639811.1A CN117651144A (en) | 2023-12-01 | 2023-12-01 | Deep learning-based building point cloud compression method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311639811.1A CN117651144A (en) | 2023-12-01 | 2023-12-01 | Deep learning-based building point cloud compression method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117651144A true CN117651144A (en) | 2024-03-05 |
Family
ID=90047326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311639811.1A Pending CN117651144A (en) | 2023-12-01 | 2023-12-01 | Deep learning-based building point cloud compression method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117651144A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118175150A (en) * | 2024-03-11 | 2024-06-11 | 广州思勘测绘技术有限公司 | Historical building cloud data sharing method and system |
-
2023
- 2023-12-01 CN CN202311639811.1A patent/CN117651144A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118175150A (en) * | 2024-03-11 | 2024-06-11 | 广州思勘测绘技术有限公司 | Historical building cloud data sharing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Blueprint separable residual network for efficient image super-resolution | |
| CN110933429B (en) | Video compression sensing and reconstruction method and device based on deep neural network | |
| CN110929622A (en) | Video classification method, model training method, device, equipment and storage medium | |
| CN114616832A (en) | Network-based visual analysis | |
| CN111127538A (en) | Multi-view image three-dimensional reconstruction method based on convolution cyclic coding-decoding structure | |
| CN112597824A (en) | Behavior recognition method and device, electronic equipment and storage medium | |
| CN108898269A (en) | Electric power image-context impact evaluation method based on measurement | |
| CN116579966B (en) | Image reconstruction method and device, electronic equipment and storage medium | |
| CN112950480A (en) | Super-resolution reconstruction method integrating multiple receptive fields and dense residual attention | |
| CN115293986A (en) | Multi-temporal remote sensing image cloud region reconstruction method | |
| CN115545166A (en) | Improved ConvNeXt convolutional neural network and remote sensing image classification method thereof | |
| CN117807269A (en) | Method and system for recommending dado based on video scene analysis | |
| CN117892059A (en) | Electric energy quality disturbance identification method based on multi-mode image fusion and ResNetXt-50 | |
| CN117372617A (en) | Point cloud data reconstruction method based on GCN-converter model and electronic equipment | |
| WO2023174256A1 (en) | Data compression method and related device | |
| CN109558819B (en) | Depth network lightweight method for remote sensing image target detection | |
| CN109063782B (en) | Intelligent fault diagnosis method for self-adaptive pump station | |
| CN114913330B (en) | Point cloud component segmentation method and device, electronic equipment and storage medium | |
| CN113962332B (en) | Salient target identification method based on self-optimizing fusion feedback | |
| CN117651144A (en) | Deep learning-based building point cloud compression method and system | |
| CN115147317A (en) | Point cloud color quality enhancement method and system based on convolutional neural network | |
| CN113628338A (en) | Sampling reconstruction method and device, computer equipment and storage medium | |
| Kasem et al. | DRCS-SR: Deep robust compressed sensing for single image super-resolution | |
| CN115409150A (en) | Data compression method, data decompression method and related equipment | |
| CN113128362A (en) | Yoov 3-based method for rapidly detecting small target under view angle of unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |