CN116152458A - Three-dimensional simulation building generation method based on images - Google Patents
Three-dimensional simulation building generation method based on images Download PDFInfo
- Publication number
- CN116152458A CN116152458A CN202310244803.0A CN202310244803A CN116152458A CN 116152458 A CN116152458 A CN 116152458A CN 202310244803 A CN202310244803 A CN 202310244803A CN 116152458 A CN116152458 A CN 116152458A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- building
- dimensional building
- image
- nodding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000004088 simulation Methods 0.000 title claims abstract description 43
- 238000003062 neural network model Methods 0.000 claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims description 23
- 238000011176 pooling Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 6
- 238000005034 decoration Methods 0.000 abstract description 14
- 238000013139 quantization Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000005094 computer simulation Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 235000002245 Penicillium camembertii Nutrition 0.000 description 1
- 241000221696 Sclerotinia sclerotiorum Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/04—Architectural design, interior design
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Hardware Design (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Analysis (AREA)
- Computer Graphics (AREA)
- Architecture (AREA)
- Pure & Applied Mathematics (AREA)
- Remote Sensing (AREA)
- Mathematical Optimization (AREA)
- Civil Engineering (AREA)
- Computational Mathematics (AREA)
- Structural Engineering (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a three-dimensional simulation building generation method based on images, which comprises the following steps: performing specification consistency processing on a two-dimensional building nodding image sample to obtain a standardized pattern, and marking and cutting a two-dimensional building in the standardized pattern to obtain a plurality of two-dimensional building training samples; extracting image characteristics of a two-dimensional building training sample, training to obtain a two-dimensional building recognition neural network model, carrying out target recognition on an application image containing a nodding two-dimensional building by using the model, extracting two-dimensional building characteristic data, inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, carrying out three-dimensional building height and exterior decoration simulation layout, and rapidly generating a corresponding three-dimensional simulation building. The method is suitable for carrying out standardized processing on the nodding images of the two-dimensional buildings with various specifications, can accurately identify two-dimensional building objects in the nodding images, and can quickly generate corresponding three-dimensional simulation buildings, and has the advantages of wide application range, low implementation difficulty and high efficiency.
Description
Technical Field
The invention relates to the technical field of computer simulation, in particular to a three-dimensional simulation building generation method based on images.
Background
In the prior art, a ground building image in overlook can be obtained by shooting an image on the ground through satellite remote sensing or based on an aircraft in-plane shooting of the ground image, which is a two-dimensional plane building image.
In computer simulation application, it is hoped that the corresponding three-dimensional building scene can be quickly constructed based on the nodding two-dimensional plane building images, the simulation fidelity is high, the adjacent relation between local building appearances and buildings can be presented, and therefore an effective solution is provided for quickly constructing three-dimensional building simulation scenes with different regional characteristics.
Disclosure of Invention
The invention mainly solves the technical problem of providing a three-dimensional simulation building generation method based on images. The method solves the problem that the prior art lacks a technology for quickly constructing a corresponding three-dimensional building simulation scene based on two-dimensional plane building images.
In order to solve the technical problems, the invention adopts a technical scheme that the invention provides a three-dimensional simulation building generation method based on images, which comprises the following steps:
inputting a two-dimensional building nodding image sample, and performing specification consistency processing on the two-dimensional building nodding image sample to obtain one or more standardized patterns;
labeling and cutting two-dimensional buildings in the standardized patterns to obtain a plurality of two-dimensional building training samples;
extracting image features of the two-dimensional building training sample, inputting the image features into an initial neural network recognition model, and performing recognition training on the two-dimensional building targets in the standardized pattern to finally obtain a two-dimensional building recognition neural network model;
performing target recognition on an application image containing a nodding two-dimensional building by using the two-dimensional building recognition neural network model, and extracting two-dimensional building characteristic data corresponding to the nodding two-dimensional building from the application image;
and inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, taking the two-dimensional building characteristic data as plane projection, and carrying out three-dimensional building height and exterior simulation layout to quickly generate a corresponding three-dimensional simulation building.
Preferably, performing the specification consistency processing on the two-dimensional building nodding image samples includes making the pixel quantity p of different samples 1 And a space value k 1 Ratio p of (2) 1 /k 1 Equal or near equal.
Preferably, the performing the specification consistency processing on the two-dimensional building nodding image sample includes performing a specification consistency processing on the two-dimensional building nodding image sample T 1 Performing segmentation, wherein the transverse segmentation ratio is n 1 The longitudinal split ratio is m 1 A normalized pattern corresponding to the same number of pixels obtained; the two-dimensional building is subjected to nodding of an image sample T 1 The segmentation is divided into a plurality of sets of standardized patterns, namely:
wherein T is 1,1 、T 1,2 、…、/>
Respectively represent the divided normalized patterns having the same number of pixels. />
Preferably, the two-dimensional building training sample is denoted as J i Wherein i represents a sequence number, and the image features of the two-dimensional building training sample
Wherein->
Corresponding to a polygonal feature->
Corresponding to pixel feature>
Corresponding to the spatial orientation feature, < >>
The correspondence is in a picture format.
Preferably, the initial neural network recognition model comprises a first-stage feature extraction network, wherein the first-stage feature extraction network is used for extracting image features in an input image to obtain a feature map, an output result of the first-stage feature extraction network also enters a second-stage region suggestion network to carry out region selection, and then different divided regions are applied to the feature map to carry out region division on the feature map; and then carrying out pooling treatment by a third pooling layer, respectively inputting treatment results into three identification networks in parallel, namely a classification identification network, a frame identification network and a pixel identification network, respectively carrying out type identification on the two-dimensional buildings in the images, carrying out identification marking on the frames of each two-dimensional building, carrying out identification on the pixels in the two-dimensional buildings, and then outputting the identification images subjected to identification marking.
Preferably, training the initial neural network recognition model, including setting network parameters and adjusting network configuration; and loading pre-training weights to start a training network, and storing the weight results obtained by training to finally obtain the required two-dimensional building identification neural network model.
Preferably, the application image is represented as Q 1 The segmentation is divided into a plurality of sets of standardized patterns, namely:
wherein Q is 1,1 、Q 1,2 、…、Q m2,n2 Respectively represent the normalized patterns with the same pixel number after the application image is divided, n 2 Represents the transverse segmentation ratio, m, of the application image 2 The vertical segmentation ratio of the applied image is shown.
Preferably, the method comprises the steps of,the two-dimensional building characteristic data is obtained by taking each divided standardized pattern as an object of an application image and respectively corresponding to one two-dimensional building data set, namely Q 1,1 、Q 1,2 、…、Q m2,n2 Respectively corresponding to two-dimensional building data sets
For any subset q of building data in the set e,r ，e∈(1,m 2 )，r∈(1,n 2 ) The two-dimensional building features extracted from the two-dimensional building features are represented by taking a pixel coordinate system as a reference.
Preferably, the subset q of building data e,r Comprising a plurality of identified two-dimensional building objects j Wherein j represents a sequence number; for any two-dimensional building object l j All of which are represented by data features with reference to a pixel coordinate system
Wherein->
Corresponding to a set of pixel coordinates of the two-dimensional building object, < >>
A set of pixel values corresponding to a plurality of pixels being said two-dimensional building object, +.>
The correspondence is a spatial orientation feature.
Preferably, the two-dimensional building characteristic data is a plane projection basis, and is applied to the two-dimensional building object l in the plane projection basis j Increasing height feature b 4 I.e. in the data features
Is added with +.>
The beneficial effects of the invention are as follows: the invention discloses a three-dimensional simulation building generation method based on images, which comprises the following steps: performing specification consistency processing on a two-dimensional building nodding image sample to obtain a standardized pattern, and marking and cutting a two-dimensional building in the standardized pattern to obtain a plurality of two-dimensional building training samples; extracting image characteristics of a two-dimensional building training sample, training to obtain a two-dimensional building recognition neural network model, carrying out target recognition on an application image containing a nodding two-dimensional building by using the model, extracting two-dimensional building characteristic data, inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, carrying out three-dimensional building height and exterior decoration simulation layout, and rapidly generating a corresponding three-dimensional simulation building. The method is suitable for carrying out standardized processing on the nodding images of the two-dimensional buildings with various specifications, can accurately identify two-dimensional building objects in the nodding images, and can quickly generate corresponding three-dimensional simulation buildings, and has the advantages of wide application range, low implementation difficulty and high efficiency.
Drawings
FIG. 1 is a flow chart of one embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 2 is a schematic illustration of normalized segmentation of an image in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 3 is an illustration of two-dimensional building labeling and cropping of a standardized pattern in another embodiment of an image-based three-dimensional simulated building generation method in accordance with the present invention;
FIG. 4 is a composition diagram of an initial neural network recognition model in another embodiment of an image-based three-dimensional simulated building generation method in accordance with the present invention;
FIG. 5 is a schematic illustration of the outline of two-dimensional building identification in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 6 is a schematic illustration of a two-dimensional building identified outline in combination with pixels in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 7 is a schematic illustration of a white model of a building generated based on two-dimensional building feature data in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 8 is a schematic illustration of a three-dimensional building generated based on a white model of the building in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
FIG. 9 is a schematic representation of a basic form of a building roof in another embodiment of an image-based three-dimensional simulated building generation method according to the present invention;
fig. 10 is a schematic diagram of two implementation forms of the top of a building in another embodiment of the image-based three-dimensional simulation building generation method according to the present invention.
Detailed Description
In order that the invention may be readily understood, a more particular description thereof will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used in this specification includes any and all combinations of one or more of the associated listed items.
FIG. 1 shows a flow chart of an embodiment of an image-based three-dimensional simulated building generation method, comprising the steps of:
step S1: inputting a two-dimensional building nodding image sample, and performing specification consistency processing on the two-dimensional building nodding image sample to obtain one or more standardized patterns;
step S2: labeling and cutting two-dimensional buildings in the standardized patterns to obtain a plurality of two-dimensional building training samples;
step S3: extracting image features of the two-dimensional building training sample, inputting the image features into an initial neural network recognition model, and performing recognition training on the two-dimensional building targets in the standardized pattern to finally obtain a two-dimensional building recognition neural network model;
step S4: performing target recognition on an application image containing a nodding two-dimensional building by using the two-dimensional building recognition neural network model, and extracting two-dimensional building characteristic data corresponding to the nodding two-dimensional building from the application image;
step S5: inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, taking the two-dimensional building characteristic data as a plane projection foundation, and carrying out three-dimensional building height and exterior trim simulation layout to quickly generate a corresponding three-dimensional simulation building.
In step S1, a two-dimensional building is subjected to a nodding image sample T 1 Which comprises at least two parameters, namely the pixel quantity p 1 And a space value k 1 Wherein the pixel quantity represents the number of image pixels of the sample, the space value represents the area of the actual space presented by the sample, and the corresponding representation is T 1 (p 1 ,k 1 ). Typically the desired pixel quantity p 1 As large as possible and spatial value k 1 As small as possible, so that the larger the pixel amount per unit area of the image sample, i.e. the pixel amount p 1 And a space value k 1 Ratio p of (2) 1 /k 1 The larger the image resolution representing the video sample, the higher.
Preferably, the acquired two-dimensional building nodding image sample T 1 There are various sources such as remote sensing images taken by remote sensing satellites, aerial images taken by aerial vehicles, and near-overhead images of unmanned aerial vehicles taken by commercial unmanned aerial vehicles in close-up overhead view. When there are two-dimensional building nodding image samples from multiple sources, it is necessary to perform a specification consistency process on the samples from different sources.
Preferably, in step S1, the two-dimensional building nodding image sample is subjected to a specification consistency process to obtain pixel amounts p of different samples 1 And a space value k 1 Ratio p of (2) 1 /k 1 Equal or near equal. Thus, the first and second substrates are bonded together,in application, classification is usually performed according to different image sources to ensure the requirement of specification consistency processing. And the image can be compressed or interpolated, so as to reduce or increase the pixel quantity of the adjusted image sample; .
In addition, there is also the pixel quantity p 1 And a space value k 1 Ratio p of (2) 1 /k 1 The minimum threshold requirement exists because if the ratio is too small, the resolution and recognition degree of the two-dimensional building are reduced, the features such as the outline, the color, the height and the type of the two-dimensional building are difficult to accurately recognize, and the two-dimensional building cannot be distinguished from ground objects such as roads, rivers, bridges, trees, lines and poles, so that the two-dimensional building cannot be accurately marked and accurately recognized.
Preferably, in step S1, the two-dimensional building nodding image sample is subjected to specification consistency processing, and further includes uniform pixel size processing on the image sample, for example, standardized segmentation is performed on an image with a larger pixel size, and the image is divided into a plurality of standardized patterns with the same pixel size, and at the same time, corresponding spatial values are also guaranteed to be the same or close. The standardized pattern thus obtained is not only the pixel quantity p 1 And a space value k 1 Ratio p of (2) 1 /k 1 The same or close, also the pixel quantity p 1 Is itself identical or close to the corresponding space value k 1 The same or similar in nature.
Preferably, as shown in fig. 2, the standardized segmentation of the image with a large number of pixels is schematically illustrated, and in fig. 2, the two-dimensional building nodding image sample T 1 Is a pixel quantity p of (2) 1 Is formed by the number x of horizontal pixels 1 And the number y of longitudinal pixels 1 Multiplication by, i.e. p 1 ＝x 1 y 1 . Then, the two-dimensional building is subjected to nodding of an image sample T 1 Performing segmentation, wherein the transverse segmentation ratio is n 1 For example n in FIG. 2 1 =7, longitudinal split ratio is m 1 For example m in FIG. 2 1 =3, the number of horizontal pixels x corresponding to the normalized pattern of the same pixel size obtained 2 ＝x 1 /n 1 And the number y of longitudinal pixels 2 ＝y 1 /m 1 . Therefore, the two-dimensional building can be subjected to the nodding image sample T 1 The segmentation is divided into a plurality of sets of standardized patterns, namely:
wherein T is 1,1 、T 1,2 、…、/>
Respectively represent the divided normalized patterns having the same number of pixels.
Since these patterns are presented on a pixel basis, they can be represented entirely by a pixel coordinate system, i.e., coordinate values and pixel values corresponding to each pixel. For example, in fig. 2, the lower left corner of the graph is taken as the origin of the pixel coordinate values, the abscissa is increased rightward in units of one pixel, for example, the maximum value is 1024, which indicates that there are 1024 pixels in the lateral direction, and the ordinate is increased upward in units of one pixel, for example, the maximum value 768, which indicates that there are 768 pixels in the longitudinal direction. Thus, the position coordinates of each pixel in the image of 1024×768 pixels can be determined by such a pixel coordinate value. Then, based on the coordinates, the pixel values are represented by three primary colors (R, G, B).
Preferably, when the two-dimensional building nodding image sample is divided into a plurality of standardized patterns, each standardized pattern is also represented by adopting the pixel coordinate system, so that unified standardized processing is facilitated, each standardized pattern is also facilitated to be processed and then recombined, and the two-dimensional building nodding image sample is processed.
In connection with step S2, fig. 3 shows an example of two-dimensional construction marking and cropping of the standardized pattern, in which a plurality of two-dimensional construction training samples 11 to 18 can be seen. These two-dimensional building training samples are all outlined by polygons, including the number of sides, the length of sides and the shape of the contours of the polygons. The pixel characteristics of the two-dimensional building training samples comprise white pixels, blue pixels, gray-white mixed pixels and the like. The spatial orientation features of the two-dimensional building training samples can be determined according to the spatial orientation of the shot image in the map, for example, the azimuth angle of each side of the training sample is determined by taking the north orientation as 0-angle reference.
Thus, when the two-dimensional building training sample is denoted as J i Where i represents a sequence number indicating that there are a plurality of different two-dimensional building training samples. Further, image features representing the two-dimensional building training sample are correspondingly arranged
Wherein->
Corresponding to a polygonal feature, including a plurality of parameter values, such as the number of sides of the polygon (e.g., single-sided circle, oval, double-sided semicircle, triangle with three sides, etc.), the side length of each side (the side length may be represented by the number of pixels), and the outline shape (e.g., circle, oval, semicircle, triangle, quadrilateral, etc.); />
The corresponding pixel characteristics comprise the number of pixels (or the percentage of pixels, namely the percentage value of the number of pixels of the sample to the total number of pixels of the standardized pattern), the color value of the pixels, the actual space size corresponding to a single pixel and the like; />
The corresponding space azimuth features comprise azimuth angles of all sides; />
The corresponding picture is in a picture format, such as jpg, tiff and other different picture storage formats.
Obtaining more two-dimensional building training samples as much as possible through the step S2, and then extracting the image features of the two-dimensional building training samples in the step S3 to obtain the image features of each two-dimensional building training sample
And then training the initial neural network recognition model by using the two-dimensional building training samples, so that the setting parameters of the initial neural network recognition model are continuously adjusted, and finally, the two-dimensional building target can be accurately recognized from the standardized pattern, thereby obtaining the corresponding two-dimensional building recognition neural network model.
Preferably, as shown in fig. 4, the initial neural network recognition model is an image recognition network mainly constructed by taking a convolutional neural network as a main body, and comprises a first-stage feature extraction network N1 for extracting image features in an input image to obtain a feature map N2, wherein an output result of the first-stage feature extraction network N1 also enters a second-stage region suggestion network N3 to perform region selection, then different divided regions are applied to the feature map N2, namely, the feature map N2 is subjected to region division, and then subjected to pooling treatment by a third-stage pooling layer N4, and the processing results are respectively input to three recognition networks in parallel, namely, a classification recognition network N5, a frame recognition network N6 and a pixel recognition network N7, and are respectively used for performing type recognition on two-dimensional buildings in the image, performing recognition labeling on frames of each two-dimensional building, performing recognition division on pixels in the two-dimensional buildings, and then outputting recognition images subjected to recognition labeling.
Preferably, for the first-stage feature extraction network N1, a RestNet convolutional neural network is used for extracting two-dimensional building features in the image to obtain a segmentation result of the two-dimensional building. For example, resNet18, resNet34, resNet50, resNet101, and ResNet152 networks may be mentioned.
Preferably, the second-level regional suggestion network N3 corresponds to RPN (Region proposal network), and is configured to perform a two-dimensional building target frame regression operation on each candidate region, generate a two-dimensional building target regression frame, and obtain a class likelihood size. Specifically, the second-level regional suggestion network N3 is used to scan the feature map N2 and attempt to determine a region containing a two-dimensional architectural object. For example, the RPN network convolves feature maps of different scales, generating 3 anchor points (anchors) at each location, where 3 x 4 convolution kernels (including building color and background) are generated for a two-dimensional building. And connecting two full-connection layers after the convolution kernel to finish the discrimination of the foreground (target) and the background (background) of each pixel and the regression correction of the two-dimensional building target frame.
Preferably, the N4 correspondence for the third level pooling layer may be a target region pooling network (Region Of Interest Pooling) or an ROI alignment network. Pooling processing corresponding to the ROIPooling network comprises the steps of quantifying and rounding boundary coordinates of a target area; this region is then divided into m x m number of cells and the boundaries of these cells are again quantized and rounded. These quantization operations may deviate the boundary values from the original ones, which will affect the accuracy of the segmentation, although this may not affect the classification.
The pooling process corresponding to the ROI alignment is to cancel the quantization rounding operation of the boundary coordinates, and calculate the feature accurate value corresponding to the boundary in m x m units by using bilinear interpolation, so as to convert the pooling process of the target region into continuous operation. The specific operation is as follows: 1. and traversing each target area, and carrying out no quantization rounding on the boundary. 2. Each target area is divided into m x m units, and the boundary of each unit is not quantized and rounded. 3. The values of the four coordinate positions in each cell are calculated by bilinear interpolation. 4. And carrying out maximum value pooling operation on each unit to obtain a feature map with the size of m.
Preferably, for the class identification network N5, the correspondence may be a full convolutional network (Fully-Convolutional Network, FCN) for predicting two-dimensional building classes.
Preferably, for the frame recognition network N6, the correspondence may be a full convolution network (FCN, fully-Convolutional Network) for generating regression frames corresponding to the two-dimensional building shapes, respectively.
Preferably, the pixel identification network N7 corresponds to a full convolution network (Fully-Convolutional Network, FCN), and may include, for example, 4 consecutive convolution layers and one deconvolution layer, where each convolution layer has a kernel size of 256 channels of 3×3, followed by a 2-up-sampled deconvolution layer for predicting the binary pixel values of each two-dimensional architectural object.
Thus, the final recognition image outputs three recognition results: the class of the object, the regression frame coordinates of the object, and the pixel value of the object.
In step S3, training the initial neural network recognition model by using a two-dimensional building training sample, which mainly includes setting network parameters, adjusting network configuration, such as an initial learning rate, and the number of epochs (which means that a complete data set passes through the neural network model once and returns once, and this process is called once Epoch); and loading pre-training weights to start training a network, and storing weight results obtained by training. And finally obtaining the required two-dimensional building identification neural network model.
In the training process, network configuration and parameters are continuously optimized and adjusted, firstly, a two-dimensional building training sample can be accurately identified, then, an image sample for checking and verifying is input for checking and verifying, and the accuracy and reliability of the two-dimensional building identification neural network model for detecting the two-dimensional building are evaluated. Fig. 5 shows two-dimensional building regression frame coordinates output after detection and recognition of a two-dimensional building training sample in the training process, which shows that the two-dimensional building regression frame coordinates have good recognition accuracy. Fig. 6 shows the two-dimensional building regression frame coordinates and the pixel values in the regression frame output after the two-dimensional building training sample is detected and identified in the training process, and the two-dimensional building regression frame coordinates and the pixel values in the regression frame also show good identification accuracy. The method is only schematically shown, other two-dimensional buildings in the image are accurately identified in practical application, and the method has good coverage rate and can cover more than 96% of two-dimensional buildings in the image.
Preferably, in step S4, the application image to be detected and identified is also required to be divided into a plurality of standardized patterns, and then input into the two-dimensional building identification neural network model for two-dimensional building target identification.
Preferably, the embodiment described with reference to FIG. 2 applies the image representation Q 1 The segmentation is divided into a plurality of sets of standardized patterns, namely:
wherein Q is 1,1 、Q 1,2 、…、/>
Respectively represent the normalized patterns having the same number of pixels after the application image is divided, and the normalized patterns corresponding to the samples in the training process have the same number of pixels. n is n 2 Represents the transverse segmentation ratio, m, of the application image 2 The vertical segmentation ratio of the applied image is shown.
Further, the two-dimensional building feature data obtained in step S4 are respectively corresponding to one two-dimensional building data set, i.e. Q, by using each divided standardized pattern as an object 1,1 、Q 1,2 、…、
Respectively corresponding to two-dimensional building data sets
For any subset q of building data in the set e,r ，e∈(1,m 2 )，r∈(1,n 2 ) The two-dimensional building features extracted from the two-dimensional building features are represented by taking a pixel coordinate system as a reference.
Preferably, continuing with FIG. 6 as an example, if the figure shows a subset q of building data identified by extraction of a normalized pattern after image segmentation is applied e,r The subset q of building data e,r Comprising a plurality of identified two-dimensional building objects j Where j represents a sequence number, as shown in FIG. 6 with l identified 1 To l 8 A two-dimensional building object. For any two-dimensional building object l j All of which are represented by data features with reference to a pixel coordinate system
Wherein->
Corresponding to a set of pixel coordinates of the two-dimensional building object, whichThe combination of the pixel coordinates together forms the contour convenience and the pixel composition inside the contour of the two-dimensional building object, e.g +.>
Can be expressed as +.>
Wherein (x) 1 ',y 1 ') represents the coordinate value corresponding to a pixel in the two-dimensional building object, the other coordinates have the same meaning, and g is shared here 1 Corresponding to g of the coordinates 1 And each pixel. />
Corresponding to a set of pixel values of a plurality of pixels of the two-dimensional building object, each pixel value representing a color value of the pixel, e.g. +.>
Can be expressed as +.>
Wherein->
The pixel value representing a pixel in the two-dimensional building object may comprise three values of the three primary colors red, green and blue, and is thus represented here by a vector, the other pixel values having the same meaning, here g 1 Pixel values of the individual pixels. />
Corresponding to the spatial orientation feature, a clockwise angle between the direction of the line connecting the two pixel coordinates and the north direction can be defined, for example +.>
Representing coordinates (x) 1 ',y 1 ') to (x) 2 ',y 2 ') direction of connection between themThe clockwise included angle between the main body and the north direction is alpha.
Preferably, in step S5, the two-dimensional building feature data obtained based on the two-dimensional building feature data obtained by the method is combined in a divided standardized pattern, and the plurality of identified two-dimensional building objects in the corresponding subset of building data are data features established by taking the pixel coordinate system as a reference, and the data features can be geographically spatially laid out in combination with a specific azimuth, so that the two-dimensional building feature data can be input into a simulation environment of a three-dimensional space for spatial simulation processing.
Preferably, the two-dimensional building characteristic data is taken as a plane projection basis, and the two-dimensional building object l in the plane projection basis is used for j Increasing height feature b 4 I.e. in the data features
Is added with +.>
The height characteristic b 4 Either randomly generated or set according to the type of building. As shown in fig. 7, the building white model 19 generated in the three-dimensional simulation environment after the two-dimensional building feature data is a plane projection basis and the height feature is added is shown.
Further preferably, as shown in fig. 8, the building wall decoration 20 and the building top decoration 21 can be further added to the generated building white mould, and the simulated building exterior decoration can be reasonably selected or a corresponding exterior decoration material database can be constructed according to the geographical environment required by simulation, so as to select and use the building exterior decoration.
Preferably, decoration of the building roof can be achieved by modeling simulation, i.e. constructing a basic form of the building roof, as shown in fig. 9, including a circular roof 31, a planar roof 32, a herringbone roof 33, a four-slope roof 34, a folded roof 35, a single-slope roof 36, etc.
Preferably, the pattern of the building roof can be constructed by combining various forms on the basis of basic forms. As shown in fig. 10, the two-dimensional building projection profile 41, after the building white mold is generated, may have a variety of basic forms of the pattern of the building roof, in which the roof map 42 includes a combination of a herringbone roof and a four-slope roof, and the roof map 43 includes a combination of a herringbone roof and a planar roof. Therefore, various building top decorations can be enriched by constructing various basic forms of the building and then combining the basic forms, so that the building top is adapted to local building styles, building years and the like.
Similarly, the building wall decoration is similar to the building top decoration, and multiple foundation forms can be constructed first and then selected and/or combined for use, so that the diversity of the wall decoration is enriched, and the building wall decoration is matched with the building style and the age of the region where the simulated building is located, so that a more lifelike simulated environment is created.
The invention discloses a three-dimensional simulation building generation method based on images, which comprises the following steps: performing specification consistency processing on a two-dimensional building nodding image sample to obtain a standardized pattern, and marking and cutting a two-dimensional building in the standardized pattern to obtain a plurality of two-dimensional building training samples; extracting image characteristics of a two-dimensional building training sample, training to obtain a two-dimensional building recognition neural network model, carrying out target recognition on an application image containing a nodding two-dimensional building by using the model, extracting two-dimensional building characteristic data, inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, carrying out three-dimensional building height and exterior decoration simulation layout, and rapidly generating a corresponding three-dimensional simulation building. The method is suitable for carrying out standardized processing on the nodding images of the two-dimensional buildings with various specifications, can accurately identify two-dimensional building objects in the nodding images, and can quickly generate corresponding three-dimensional simulation buildings, and has the advantages of wide application range, low implementation difficulty and high efficiency.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structural changes made by the present invention and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The three-dimensional simulation building generation method based on the image is characterized by comprising the following steps of:
inputting a two-dimensional building nodding image sample, and performing specification consistency processing on the two-dimensional building nodding image sample to obtain one or more standardized patterns;
labeling and cutting two-dimensional buildings in the standardized patterns to obtain a plurality of two-dimensional building training samples;
extracting image features of the two-dimensional building training sample, inputting the image features into an initial neural network recognition model, and performing recognition training on the two-dimensional building targets in the standardized pattern to finally obtain a two-dimensional building recognition neural network model;
performing target recognition on an application image containing a nodding two-dimensional building by using the two-dimensional building recognition neural network model, and extracting two-dimensional building characteristic data corresponding to the nodding two-dimensional building from the application image;
and inputting the two-dimensional building characteristic data into a three-dimensional building simulation environment, taking the two-dimensional building characteristic data as plane projection, and carrying out three-dimensional building height and exterior simulation layout to quickly generate a corresponding three-dimensional simulation building.
2. The image-based three-dimensional simulation building generation method according to claim 1, wherein performing specification consistency processing on the two-dimensional building nodding image samples includes causing pixel amounts p of different samples to be calculated 1 And a space value k 1 Ratio p of (2) 1 /k 1 Equal or near equal.
3. The method of claim 1, wherein performing a specification consistency process on the two-dimensional building nodding image sample comprises performing a specification consistency process on the two-dimensional building nodding image sample T 1 Performing segmentation, wherein the transverse segmentation ratio is n 1 The longitudinal split ratio is m 1 Correspondingly obtaining the standardized patterns with the same pixel number;
the two-dimensional building is subjected to nodding shadowImage sample T 1 The segmentation is divided into a plurality of sets of the standardized patterns, namely:
wherein T is 1,1 、T 1,2 、…、/>
Respectively represent the divided normalized patterns having the same number of pixels.
4. The method of claim 3, wherein the two-dimensional building training sample is represented as J i Wherein i represents a sequence number, and the image features of the two-dimensional building training sample
Wherein->
Corresponding to a polygonal feature->
Corresponding to pixel feature>
Corresponding to the spatial orientation feature, < >>
The correspondence is in a picture format.
5. The method for generating the three-dimensional simulation building based on the image according to claim 1, wherein the initial neural network recognition model comprises a first-stage feature extraction network for extracting image features in an input image to obtain a feature map; the output result of the first-stage feature extraction network also enters a second-stage region suggestion network to carry out region selection, and then different divided regions are applied to the feature map to carry out region division on the feature map; and then carrying out pooling treatment by a third pooling layer, respectively inputting treatment results into three identification networks in parallel, namely a classification identification network, a frame identification network and a pixel identification network, respectively carrying out type identification on the two-dimensional buildings in the images, carrying out identification marking on the frames of each two-dimensional building, carrying out identification on the pixels in the two-dimensional buildings, and then outputting the identification images subjected to identification marking.
6. The method for generating the three-dimensional simulation building based on the image according to claim 5, wherein training the initial neural network recognition model comprises setting network parameters and adjusting network configuration; and loading pre-training weights to start a training network, and storing the weight results obtained by training to finally obtain the required two-dimensional building identification neural network model.
7. The method of claim 6, wherein the application image is represented as Q 1 The segmentation is divided into a plurality of sets of standardized patterns, namely:
wherein Q is 1,1 、Q 1,2 、…、/>
Respectively represent the normalized patterns with the same pixel number after the application image is divided, n 2 Represents the transverse segmentation ratio, m, of the application image 2 The vertical segmentation ratio of the applied image is shown.
8. The method of generating three-dimensional simulation building based on image according to claim 7, wherein the two-dimensional building feature data is obtained by applying the application image to each of the divided standardized patterns, respectively toShould be a two-dimensional building data set, i.e. Q 1,1 、Q 1,2 、…、
Respectively corresponding to two-dimensional building data sets->
For any subset q of building data in the set e,r ，e∈(1,m 2 )，r∈(1,n 2 ) The two-dimensional building features extracted from the two-dimensional building features are represented by taking a pixel coordinate system as a reference.
9. The method of claim 8, wherein the subset q of building data e,r Comprising a plurality of identified two-dimensional building objects j Wherein j represents a sequence number; for any two-dimensional building object l j All of which are represented by data features with reference to a pixel coordinate system
Wherein->
Corresponding to a set of pixel coordinates of the two-dimensional building object, < >>
A set of pixel values corresponding to a plurality of pixels of the two-dimensional building object,
the correspondence is a spatial orientation feature.
10. The method of claim 9, wherein the two-dimensional building feature data is a planar projection basis for two-dimensional building object/therein j Increasing height feature b 4 I.e. in the data features
Is added with +.>
/>
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244803.0A CN116152458A (en) | 2023-03-14 | 2023-03-14 | Three-dimensional simulation building generation method based on images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244803.0A CN116152458A (en) | 2023-03-14 | 2023-03-14 | Three-dimensional simulation building generation method based on images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116152458A true CN116152458A (en) | 2023-05-23 |
Family
ID=86356327
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310244803.0A Pending CN116152458A (en) | 2023-03-14 | 2023-03-14 | Three-dimensional simulation building generation method based on images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116152458A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140301633A1 (en) * | 2013-04-09 | 2014-10-09 | Google Inc. | System and Method for Floorplan Reconstruction and Three-Dimensional Modeling |
| CN110363082A (en) * | 2019-06-06 | 2019-10-22 | 武汉易科空间信息技术股份有限公司 | A kind of building extracting method and system based on three-dimensional remote sensing image |
| CN111553303A (en) * | 2020-05-07 | 2020-08-18 | 武汉大势智慧科技有限公司 | Remote sensing ortho image dense building extraction method based on convolutional neural network |
| CN111652892A (en) * | 2020-05-02 | 2020-09-11 | 王磊 | Remote sensing image building vector extraction and optimization method based on deep learning |
| CN112818925A (en) * | 2021-02-26 | 2021-05-18 | 清华大学 | Urban building and crown identification method |
| CN112990086A (en) * | 2021-04-08 | 2021-06-18 | 海南长光卫星信息技术有限公司 | Remote sensing image building detection method and device and computer readable storage medium |
| CN113343346A (en) * | 2021-08-09 | 2021-09-03 | 速度时空信息科技股份有限公司 | Three-dimensional traffic scene rapid modeling method based on high-precision map |
| CN113610032A (en) * | 2021-08-16 | 2021-11-05 | 北京市城市规划设计研究院 | Building identification method and device based on remote sensing image |
| CN114549543A (en) * | 2021-12-30 | 2022-05-27 | 浙江大华技术股份有限公司 | Building three-dimensional model construction method and device, terminal and storage medium |
| CN114820976A (en) * | 2022-04-24 | 2022-07-29 | 奥格科技股份有限公司 | Rural building modeling method, system and storage medium integrating remote sensing image and shot image |
-
2023
- 2023-03-14 CN CN202310244803.0A patent/CN116152458A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140301633A1 (en) * | 2013-04-09 | 2014-10-09 | Google Inc. | System and Method for Floorplan Reconstruction and Three-Dimensional Modeling |
| CN110363082A (en) * | 2019-06-06 | 2019-10-22 | 武汉易科空间信息技术股份有限公司 | A kind of building extracting method and system based on three-dimensional remote sensing image |
| CN111652892A (en) * | 2020-05-02 | 2020-09-11 | 王磊 | Remote sensing image building vector extraction and optimization method based on deep learning |
| CN111553303A (en) * | 2020-05-07 | 2020-08-18 | 武汉大势智慧科技有限公司 | Remote sensing ortho image dense building extraction method based on convolutional neural network |
| CN112818925A (en) * | 2021-02-26 | 2021-05-18 | 清华大学 | Urban building and crown identification method |
| CN112990086A (en) * | 2021-04-08 | 2021-06-18 | 海南长光卫星信息技术有限公司 | Remote sensing image building detection method and device and computer readable storage medium |
| CN113343346A (en) * | 2021-08-09 | 2021-09-03 | 速度时空信息科技股份有限公司 | Three-dimensional traffic scene rapid modeling method based on high-precision map |
| CN113610032A (en) * | 2021-08-16 | 2021-11-05 | 北京市城市规划设计研究院 | Building identification method and device based on remote sensing image |
| CN114549543A (en) * | 2021-12-30 | 2022-05-27 | 浙江大华技术股份有限公司 | Building three-dimensional model construction method and device, terminal and storage medium |
| CN114820976A (en) * | 2022-04-24 | 2022-07-29 | 奥格科技股份有限公司 | Rural building modeling method, system and storage medium integrating remote sensing image and shot image |
Non-Patent Citations (4)
| Title |
|---|
| ALIDOOST, F.; AREFI, H.; TOMBARI, F.: "2D Image-To-3D Model: Knowledge-Based 3D Building Reconstruction (3DBR) Using Single Aerial Images and Convolutional Neural Networks (CNNs)", REMOTE SENSING, vol. 11, no. 19, 23 September 2019 (2019-09-23), pages 2219, XP055855513, DOI: 10.3390/rs11192219 * |
| 傅兴玉;尤红建;付琨;: "单幅高分辨率SAR图像建筑物三维模型重构", 红外与毫米波学报, no. 06, 15 December 2012 (2012-12-15), pages 91 - 98 * |
| 陈健堃;彭凌霄;仇晓兰;丁赤飚;吴一戎;: "基于深度神经网络的SAR建筑目标三维重建方法", 中国科学:信息科学, no. 12, 20 December 2019 (2019-12-20), pages 78 - 97 * |
| 黄榕;林涔涔;武佳;沈雪凡;李亚星;赵真真;: "基于摄影测量技术的三维仿真模型建模方法研究", 福建建筑, no. 10, 1 October 2016 (2016-10-01), pages 112 - 114 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110443842B (en) | Depth map prediction method based on visual angle fusion | |
| CN109523552B (en) | Three-dimensional object detection method based on viewing cone point cloud | |
| CN108596101B (en) | Remote sensing image multi-target detection method based on convolutional neural network | |
| CN108596108B (en) | Aerial remote sensing image change detection method based on triple semantic relation learning | |
| CN110866531A (en) | Building feature extraction method and system based on three-dimensional modeling and storage medium | |
| CN111738111A (en) | Road extraction method of high-resolution remote sensing image based on multi-branch cascade void space pyramid | |
| CN110728707B (en) | Multi-view depth prediction method based on asymmetric depth convolution neural network | |
| CN112084869A (en) | Compact quadrilateral representation-based building target detection method | |
| CN109829476B (en) | End-to-end three-dimensional object detection method based on YOLO | |
| CN107944459A (en) | A kind of RGB D object identification methods | |
| CN111738113A (en) | Road extraction method of high-resolution remote sensing image based on double-attention machine system and semantic constraint | |
| US7778808B2 (en) | Geospatial modeling system providing data thinning of geospatial data points and related methods | |
| CN111640116B (en) | Aerial photography graph building segmentation method and device based on deep convolutional residual error network | |
| CN112907573B (en) | Depth completion method based on 3D convolution | |
| CN111754618B (en) | Object-oriented live-action three-dimensional model multi-level interpretation method and system | |
| CN109063549A (en) | High-resolution based on deep neural network is taken photo by plane video moving object detection method | |
| CN115661404A (en) | Multi-fine-grain tree real scene parametric modeling method | |
| CN111458691B (en) | Building information extraction method and device and computer equipment | |
| CN113223159A (en) | Single remote sensing image three-dimensional modeling method based on target texture virtualization processing | |
| WO2021102948A1 (en) | Image processing method and device | |
| CN111191704A (en) | Foundation cloud classification method based on task graph convolutional network | |
| CN116740261A (en) | Image reconstruction method and device and training method and device of image reconstruction model | |
| CN115953330B (en) | Texture optimization method, device, equipment and storage medium for virtual scene image | |
| CN116152458A (en) | Three-dimensional simulation building generation method based on images | |
| CN113192204B (en) | Three-dimensional reconstruction method for building in single inclined remote sensing image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |