CN115631297A

CN115631297A - Urban three-dimensional rapid modeling method and system based on deep learning image recognition

Info

Publication number: CN115631297A
Application number: CN202211183743.8A
Authority: CN
Inventors: 张慎; 张诗慧; 望晓尉; 唐扬
Original assignee: Central South Architectural Design Institute Co Ltd
Current assignee: Central South Architectural Design Institute Co Ltd
Priority date: 2022-09-27
Filing date: 2022-09-27
Publication date: 2023-01-20

Abstract

The invention discloses a city three-dimensional rapid modeling method and system based on deep learning image recognition, belonging to the technical field of building characteristic information extraction and three-dimensional modeling, and the method comprises the following steps: dividing the test area into a training area for selecting a sample and a test area for testing results; extracting an area which accords with the characteristics of the building in the training area as a training sample; training the deep learning model by using the enhanced training sample to obtain related parameters and a pre-training model; extracting the building outline of the test area by using a pre-training model; carrying out small-area processing and outer edge processing on the building outline to obtain an optimized building outline; cutting the original image based on the building outline to obtain building textures; generating a building block by taking the outline as the outline of the bottom surface of the building, and then performing mapping and coloring by using the building texture to perform building batch modeling; the invention provides a new technical support for relevant application scenes such as city three-dimensional live-action modeling, city planning, municipal engineering and the like.

Description

Urban three-dimensional rapid modeling method and system based on deep learning image recognition

Technical Field

The invention belongs to the technical field of building characteristic information extraction and three-dimensional modeling, and particularly relates to a method and a system for urban three-dimensional rapid modeling based on deep learning image recognition.

Background

With the acceleration of the digitization process at the present stage, city modeling is required for both city updating and city planning tasks. Therefore, it is important to establish a digital twin model which is convenient for the operator to edit and process in the later period and is similar to the reality.

There are two main methods for establishing the digital twin model. One is a modeling method based on oblique photography remote sensing images, and the advantages of closer reality of the texture of the remote sensing images, wide coverage range and short updating period support the rapid acquisition of large-area three-dimensional scene information. However, the model obtained by a single oblique photography image has a fixed format and is not editable, and the requirements of post-editing modification and deeper planning processing cannot be met. The other method is a manual modeling method based on traditional design software such as CAD and the like, and the element format in the model can be edited and modified by an operator. However, the manual workload is huge, the model texture and the real texture are different to a certain extent and cannot be completely matched, and the low-efficiency and high-cost data production method cannot meet the requirement of large-scale rapid modeling.

Disclosure of Invention

Aiming at the defects or improvement requirements of the prior art, the invention provides a deep learning image recognition-based urban three-dimensional rapid modeling method, and provides a new technical support for large-range rapid modeling. The invention establishes a set of rapid modeling process method based on deep learning and regularization modeling, which can realize high-precision automatic extraction of the outline and the texture of the building based on a deep learning model and can realize rapid batch modeling of the building under a large scene based on a grammar rule.

In order to achieve the above object, according to an aspect of the present invention, there is provided a city three-dimensional rapid modeling method based on deep learning image recognition, including:

s1: dividing the whole experimental area into a training area for selecting a sample and a testing area for testing a result according to a preset proportion;

s2: building marking is carried out on the training area to form a building training sample set;

s3: training the deep learning model by using the marked building training sample set, and obtaining a pre-training deep learning model with trained parameters after multiple iterations;

s4: testing the test area by using the pre-trained deep learning model, and extracting the building outline of the test area;

s5: performing small-area processing and outer edge regularization processing on the extracted building outline of the test area to obtain an optimized building outline;

s6: cutting the original image based on the optimized building outline to obtain corresponding building texture;

s7: and performing three-dimensional scene modeling by combining the optimized building outline and the building texture.

In some optional embodiments, the method further comprises:

and adding the building training sample set and the optimized building outline into the data set to form a building data set.

The initial sample set is added into a data set library, samples meeting sample characteristics in each test area can be added into the library, and a building data set with rich characteristics can be formed through multiple experiments and used for testing other subsequent test areas. The data set can be used for migrating the sample set in other subsequent applications, and the process of remanufacturing the sample set is omitted.

In some alternative embodiments, in step S1, the labeled sample region should not coincide with the target region sample, i.e. the sample selection region should be different from the test region, the sample set has no intersection with the test region, and the sample features in the training set should contain as many as possible of all the building features in the region.

In some optional embodiments, in step S2, buildings in the training area are marked, building features of the training area are extracted, and the size of the sample set is determined by the area size of the smallest building, so that it should be ensured that the smallest size of the sample includes one building and the sample size is consistent, and then the sample is subjected to sample enhancement operations such as rotation and mirror image, so as to enrich the sample features.

In some optional embodiments, the deep learning model in step S3 selects a unet + + model, which can obtain higher precision under the condition of data imbalance and obtain an optimal segmentation result, and can integrate features of different levels in a feature superposition manner, and in addition, 80% of samples are selected for training, 20% of samples are tested for the model, the model is initialized with parameters, and training is performed for multiple iterations until the loss function reaches the training stop threshold.

In some alternative embodiments, in step S4, the buildings in the test area are automatically extracted using the trained model.

In some optional embodiments, the small region processing in step S5 is small region processing for performing deletion optimization on a region that does not meet the minimum area of the on-house graph, and deleting a result that the minimum area of the on-house graph is too small to meet the minimum area of the building graph, and the edge processing is regularization processing on the extracted region edge to meet the external regularized structure outline of the house.

In some optional embodiments, in step S6, the original image is cut based on the optimized building outline, so as to obtain corresponding building texture information.

In step S6, the building texture extraction may be implemented by using related software or by using a corresponding algorithm.

In some optional embodiments, the parameterized rule modeling in step S7 is specifically implemented by performing three-dimensional scene modeling in combination with the optimized building contour and the building texture, that is, generating a building block in combination with the optimized building contour, controlling the final form of the model by setting parameters such as the volume, the structure, and the appearance, sequentially executing subdivision rules such as stretching and offset conversion, gradually generating small blocks from a large block, and performing charting and coloring by using the extracted building texture to quickly complete the building batch modeling in a large scene.

According to another aspect of the present invention, there is provided a city three-dimensional rapid modeling system based on deep learning image recognition, including:

the area dividing module is used for dividing the whole experimental area into a training area for selecting a sample and a testing area for testing a result according to a preset proportion;

the sample processing module is used for marking the building in the training area to form a building training sample set;

the model training module is used for training the deep learning model by utilizing the marked building training sample set, and obtaining a pre-training deep learning model with trained parameters after multiple iterations;

the building outline extraction module is used for testing the test area by utilizing the pre-training deep learning model and extracting the building outline of the test area;

the contour optimization module is used for carrying out small-area processing and outer edge regularization processing on the extracted building contour of the test area to obtain an optimized building contour;

the building texture extraction module is used for cutting the original image based on the optimized building outline to obtain corresponding building texture;

and the three-dimensional scene modeling module is used for carrying out three-dimensional scene modeling by combining the optimized building outline and the optimized building texture.

In some optional embodiments, the system further comprises:

and the data set construction module is used for adding the building training sample set and the optimized building outline into the data set to form a building data set.

In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:

(1) The invention utilizes the remote sensing image to acquire information, various information of a large-range area can be quickly acquired through the remote sensing image due to the characteristics of wide range, high timeliness, multi-resolution and the like of the remote sensing image, in addition, images at different time intervals can be acquired to meet front-back comparison at different times due to the high updating speed of the remote sensing image, and finally, the remote sensing image has multiple resolutions, different resolutions are selected according to different application scene requirements, namely, the high-resolution image is selected for the area requiring clear textures.

(2) The invention utilizes the deep learning model to extract the building features, is different from the traditional manual building texture extraction, and the deep learning model carries out advanced feature learning on the sample through a plurality of convolution layers, thereby making up for the subjective errors of wrong division, omission and the like possibly caused by manual visual interpretation and quickly, accurately and efficiently extracting the related building features.

(3) The invention utilizes the data set to construct, so that the test data set of each experiment can be correspondingly utilized, the test area which accords with the characteristics can become the training set of other experiments, and the cost (the selection of a training sample is carried out again) and the difficulty (the multi-characteristics can not be completely covered) of the subsequent expansion of the data set are saved.

(4) The method utilizes the parameterization rule for modeling, compared with the traditional manual modeling, the method does not need to operate the buildings one by one, and controls the geometric shape characteristics in batch by adjusting parameter values; in addition, compared with the traditional manual modeling texture, the building texture extracted by the deep learning model is used for mapping, so that the model is more fit with the real building texture.

Drawings

FIG. 1 is a detailed flow chart of a method provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a test area and a training area provided by an embodiment of the present invention;

FIG. 3 is a sample schematic provided by an embodiment of the present invention;

FIG. 4 is a diagram illustrating test results provided by an embodiment of the present invention;

FIG. 5 is a diagram illustrating an optimization result provided by an embodiment of the present invention;

FIG. 6 is a diagram illustrating contour extraction results provided by an embodiment of the present invention;

FIG. 7 is a diagram illustrating texture extraction results according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a three-dimensional modeling result provided by an embodiment of the invention;

fig. 9 is a schematic diagram of an implementation of a system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the respective embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

In recent years, as an important driving force of a new technological revolution, the artificial intelligence technology has become the most influential key common technology in deep learning, and intelligent interpretation combining the deep learning technology and rich remote sensing data sources is applied to various tasks such as ground object target detection, element segmentation and classification. With the rapid development of deep neural networks, the precision of extracting the building related information by utilizing the deep learning convolutional neural network technology is continuously improved, the artificial modeling is assisted, and the efficient and vivid modeling is realized. The invention provides a city three-dimensional rapid modeling method based on deep learning image recognition, aiming at the defects and problems of large-scale three-dimensional scene modeling.

The invention provides a city three-dimensional rapid modeling method based on deep learning image recognition, the specific flow is shown in figure 1, and the specific implementation steps of the method are described below by taking a certain area as an example:

(1) The training area and the test area are selected such that they do not overlap, and the result is shown in fig. 2.

(2) The training samples were labeled to cover as much of the building features in the experimental area as possible and mask files were generated, the results are shown in fig. 3, where white is the building area and black is the non-building area.

(3) And performing parameter training on the model by using the training sample, and performing multiple iterations to obtain a pre-trained deep learning model.

(4) The test area is tested by using the model, and the building automatically extracted in the test area is obtained, and the result is shown in fig. 4, wherein the white area is the extracted building part.

(5) The result is subjected to small-patch optimization processing, and the result is shown in fig. 5, where the adaptive threshold is selected to be 0.9, the white region is the extracted building portion, and the region with the too small area is deleted.

(6) The optimized original image profile is obtained, and the result is shown in fig. 6.

(7) The corresponding texture of the original image is extracted based on the contour, and the result is shown in fig. 7.

(8) The building texture is extracted and the regularized modeling operation is performed, and the obtained model stereo effect is shown in fig. 8.

(9) In summary, as shown in fig. 9, the urban three-dimensional rapid modeling system based on deep learning image recognition disclosed by the invention includes: 1) A sample selection module: selecting a corresponding building feature sample; 2) A model training module: performing model training by using the marked sample; 3) A test result module: automatically extracting a building test for the test area by using the model; 4) A three-dimensional model building module: and carrying out three-dimensional rapid modeling according to the extracted building texture and the parameterized modeling rule. The invention provides a method and a system for rapidly modeling urban three-dimensional based on deep learning image recognition, aiming at the problems of difficult current modeling and unreal scene, which can realize rapid modeling of three-dimensional large-range scenes and provide a new digital technical solution for urban planning and relevant application of municipal engineering.

It should be noted that, according to the implementation requirement, each step/component described in the present application can be divided into more steps/components, and two or more steps/components or partial operations of the steps/components can be combined into new steps/components to achieve the purpose of the present invention.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A city three-dimensional rapid modeling method based on deep learning image recognition is characterized by comprising the following steps:

s4: testing the test area by using the pre-training deep learning model, and extracting the building outline of the test area;

2. The method of claim 1, further comprising:

3. The method according to claim 1, wherein in step S1, the training area and the testing area are not consistent, there is no intersection between the training area and the testing area, and the sample features selected from the training area should contain as many as possible of all building features of the training area.

4. The method of claim 1, wherein in step S2, buildings in the training area are labeled, building features of the training area are extracted, and the size of the building training sample set is determined by the area size of the smallest building, so that the minimum size of the samples in the building training sample set contains one building and the sample size is consistent, and then sample enhancement is performed on the samples.

5. The method according to claim 1, wherein in step S3, the deep learning model selects a unet + + model, selects 80% of samples in the building training sample set for training, performs 20% of model testing, performs parameter initialization on the unet + + model, and performs iterative training for multiple times until the loss function reaches the training stop threshold.

6. The method according to claim 1, wherein the small region processing in step S5 is small region processing for performing deletion optimization on a region that does not conform to the minimum area of the on-house map, and the edge processing is regularization processing on the extracted region edge to conform to the house external regularized structure profile.

7. The method of claim 1, wherein in step S7, building blocks are generated by combining the optimized building contour as a building floor contour, the final form of the three-dimensional scene model is controlled by setting the volume, structure and appearance, then stretching and shifting transformations are performed in sequence, small blocks are gradually generated from large blocks, and the extracted building texture is used for mapping and coloring, thereby rapidly completing the building batch modeling in a large scene.

8. A city three-dimensional rapid modeling system based on deep learning image recognition is characterized by comprising the following components:

the area dividing module is used for dividing the whole experimental area into a training area for selecting a sample and a testing area for testing a test result according to a preset proportion;

9. The system of claim 8, further comprising: