CN115147549A - Urban three-dimensional model generation and updating method based on multi-source data fusion - Google Patents
Urban three-dimensional model generation and updating method based on multi-source data fusion Download PDFInfo
- Publication number
- CN115147549A CN115147549A CN202210801312.7A CN202210801312A CN115147549A CN 115147549 A CN115147549 A CN 115147549A CN 202210801312 A CN202210801312 A CN 202210801312A CN 115147549 A CN115147549 A CN 115147549A
- Authority
- CN
- China
- Prior art keywords
- city
- space
- dimensional
- model
- urban
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Abstract
The invention discloses a method for generating and updating a three-dimensional city model based on multi-source data fusion, which fully utilizes a remote sensing image which is wider in coverage range, better in space precision, faster in updating frequency, lower in acquisition cost and easier to acquire at the same time as a substrate, combines high-definition camera image data which are distributed all over a city as a main data source, further dynamically identifies local space and detail space elements of the city based on the remote sensing image and a video identification image of the city camera, and performs characteristic identification on newly added, lost and form and color changes of the space elements, thereby dynamically updating the three-dimensional city model. On one hand, the construction cost is greatly reduced, on the other hand, the real-time dynamic incremental updating of the urban three-dimensional model is supported, and the urban transition process can be recorded in a full life cycle, so that the high-precision and high-timeliness urban three-dimensional model can be universally applied to a large number of digital twin and intelligent urban digital projects.
Description
Technical Field
The application relates to the technical field of urban three-dimensional model generation and updating, in particular to an urban three-dimensional model generation and updating method based on multi-source data fusion.
Background
The large-scale, high-precision and rapid full-automatic urban three-dimensional modeling technology is always the main direction of research in the fields of photogrammetry and digital cities. The main technical means of urban three-dimensional reconstruction which have proven accurate and effective at present comprise: three-dimensional measurement is carried out on an airborne area array or multi-line array CCD aerial survey camera; airborne LiDAR point cloud measurement is matched with a high-precision two-dimensional image; airborne multi-camera oblique photogrammetry; the method comprises the following steps of measuring an auxiliary panoramic photo by vehicle-mounted LiDAR point cloud, and matching with aerial two-dimensional data; full manual 3Dmax modeling.
In the related application of smart cities, particularly in the application fields with higher requirements on accuracy and real-time performance of space positions such as safety supervision and emergency command, a large amount of large-range high-accuracy three-dimensional city models are often required to be established for constructing digital twin space bases which are highly consistent with the real world.
At present, the industry mainly adopts oblique photogrammetry data and manual three-dimensional modeling, and a mode combining two modes to construct a city three-dimensional model, and the following limitations mainly exist:
the modeling cost is high, the period is long, the oblique photogrammetry needs data field acquisition through equipment such as an unmanned aerial vehicle, and a large amount of data processing is needed after the acquisition is finished; and manual modeling requires on-site investigation at an earlier stage and a professional modeler for organization to make and modify a model by using professional three-dimensional software. Either way, there is a higher cost and longer cycle time for city level modeling.
Data updating is difficult, and timeliness is difficult to guarantee. Because the traditional modeling is usually carried out according to a solidified flow as a whole, the model is not supported to be adjusted in time according to the change of a real city, and the timeliness of the modeling result is difficult to ensure in view of high cost of the whole modeling.
The traditional method can only record the urban space form in one period and cannot record the urban space transition process. Due to the restriction of national laws and regulations and local laws and regulations, many cities and regions cannot acquire oblique photogrammetric data, and the traditional city modeling mode is limited by space conditions to a certain extent.
Disclosure of Invention
Based on the technical problem, a method for generating and updating a three-dimensional city model based on multi-source data fusion is provided.
In a first aspect, a method for generating and updating a three-dimensional city model based on multi-source data fusion, the method comprising:
generating a city three-dimensional model:
collecting a first city high-definition remote sensing image and a first city video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and serves as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
updating the urban three-dimensional model:
when the change rate of the first city high-definition remote sensing image reaches a first threshold value or the change rate of the first city space element reaches a second threshold value, executing the following city three-dimensional model updating process;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the correction record of the urban three-dimensional model, comparing the characteristic label difference of the urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
In the above scheme, optionally, the collecting the first city high-definition remote sensing image and the first city video monitoring image specifically includes:
subscribing a specific area high-resolution remote sensing image of a specific city by an online service, and acquiring a latest image as a first city high-resolution remote sensing image in real time according to the subscription service;
and acquiring a city monitoring video image or installing the first city area and connecting a monitoring camera in real time to acquire the video monitoring image as the first city video monitoring image.
In the foregoing scheme, further optionally, in the extracting of the first city space element and the morphological feature, the space element includes: roads, vegetation, buildings, bodies of water; the morphological characteristics are spatial characteristics, orientation characteristics, shape characteristics, structural characteristics, color characteristics and associated characteristics of the spatial elements.
In the foregoing scheme, further optionally, the pre-acquired second city space element specifically is: and acquiring historical remote sensing images, monitoring images and the existing three-dimensional scene of the existing city, and manufacturing an existing city space element sample library as a second city space element in a manual labeling mode.
In the foregoing scheme, further optionally, the step of performing manual review and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model specifically includes:
selecting various spatial elements from a high-precision city three-dimensional scene by adopting a city three-dimensional model modification tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing the real conditions;
and the third city three-dimensional space model is a final city three-dimensional space model generated by the city three-dimensional model.
In the foregoing scheme, further optionally, the starting of the update process of the urban three-dimensional model when the change rate of the first urban high-definition remote sensing image reaches the first threshold or the change rate of the first urban space element reaches the second threshold specifically includes:
carrying out change detection on the urban high-definition remote sensing image through an image recognition algorithm model, and starting an urban three-dimensional model updating process when the change rate reaches a first threshold value;
and identifying the change of the urban space elements based on the urban video monitoring image, and starting an urban three-dimensional model updating process when the change rate reaches a second threshold value.
In the foregoing scheme, further optionally, the step of performing manual review and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model specifically includes: selecting various spatial elements from the high-precision city three-dimensional scene by adopting a city three-dimensional model modification tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing the real conditions to obtain a modified sixth city three-dimensional space model;
and the six-city three-dimensional space model is a final city three-dimensional space model after the city three-dimensional model is updated.
In a second aspect, a device for generating and updating a three-dimensional city model based on multi-source data fusion, the device comprising:
the city three-dimensional model generation module:
the urban high-definition remote sensing image acquisition system is used for collecting a first urban high-definition remote sensing image and a first urban video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and serves as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
the city three-dimensional model updating module:
the urban three-dimensional model updating method comprises the steps of executing the following urban three-dimensional model updating process when the change rate of a first urban high-definition remote sensing image reaches a first threshold value or the change rate of a first urban space element reaches a second threshold value;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the correction record of the urban three-dimensional model, comparing the characteristic label difference of the urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
In a third aspect, a computer device comprises a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
and (3) generating a city three-dimensional model:
collecting a first city high-definition remote sensing image and a first city video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and used as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
updating the urban three-dimensional model:
when the change rate of the first city high-definition remote sensing image reaches a first threshold value or the change rate of the first city space element reaches a second threshold value, executing the following city three-dimensional model updating process;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the urban three-dimensional model correction record, comparing the characteristic label difference of urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
In a fourth aspect, a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of:
and (3) generating a city three-dimensional model:
collecting a first city high-definition remote sensing image and a first city video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and serves as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
updating the urban three-dimensional model:
when the change rate of the first city high-definition remote sensing image reaches a first threshold value or the change rate of the first city space element reaches a second threshold value, executing the following city three-dimensional model updating process;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the urban three-dimensional model correction record, comparing the characteristic label difference of urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
The invention has at least the following beneficial effects:
based on further analysis and research on the problems in the prior art, the method realizes that the existing modeling cost is high and the period is long, oblique photogrammetry needs to carry out data field acquisition through equipment such as an unmanned aerial vehicle, and a large amount of data processing is needed after the acquisition is finished; and manual modeling requires on-site investigation at an earlier stage and a professional modeler for organization to make and modify a model by using professional three-dimensional software. Either way, there is higher cost and longer period for city level modeling; data updating is difficult, and timeliness is difficult to guarantee. Because the traditional modeling is usually carried out according to a solidified flow by a whole body, the model is not supported to be adjusted in time according to the change of a real city, and the timeliness of a modeling result is difficult to ensure in view of high cost of the whole modeling; the traditional method can only record the urban space form in one period and cannot record the urban space transition process. Due to the restriction of national laws and regulations and local laws and regulations, oblique photogrammetry data acquisition cannot be carried out in many cities and regions, and the traditional city modeling mode is limited by space conditions to a certain extent. The invention fully utilizes the remote sensing image which is wider in coverage range, better in space precision, faster in updating frequency and lower in acquisition cost and is easier to acquire as a base, and combines high-definition camera image data which are distributed throughout the city as a main data source, fully and deeply learns the spatial elements such as multi-scale city spatial element image recognition roads, buildings, vegetation, water systems and the like, and extracts the spatial characteristics; meanwhile, a complete space element sample asset library with a space feature tag is constructed, a city three-dimensional model with higher reality degree and aesthetic degree is rapidly generated by performing patterned matching based on space features, the reality degree and the aesthetic degree of the generated model are optimized by adjusting feature matching parameters at any time, dynamic recognition of city local space and detail space elements is further performed based on remote sensing images and in combination with city camera video recognition images, feature recognition is performed on newly added, disappeared and form and color changes of the space elements, and therefore the city three-dimensional model is dynamically updated. On one hand, the construction cost is greatly reduced, on the other hand, the real-time dynamic incremental updating of the urban three-dimensional model is supported, and the urban transition process can be recorded in a full life cycle, so that the high-precision and high-timeliness urban three-dimensional model can be universally applied to a large number of digital twin and intelligent urban digital projects.
Drawings
Fig. 1 is a schematic flow chart of a method for generating a three-dimensional city model based on multi-source data fusion according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for updating a three-dimensional city model based on multi-source data fusion according to an embodiment of the present invention;
fig. 3 is a schematic specific flowchart of a method for updating a three-dimensional city model based on multi-source data fusion according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
In an embodiment, as shown in fig. 1, a method for generating a city three-dimensional model based on multi-source data fusion is provided, and includes the following steps:
and (3) generating a city three-dimensional model:
collecting a first city high-definition remote sensing image and a first city video monitoring image; the first city is a three-dimensional model city to be generated; carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; based on a pre-acquired second city space element as a sample, constructing a city space element asset library with a multi-dimensional space-time characteristic label; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets.
Performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
as shown in fig. 2, a city three-dimensional model updating method based on multi-source data fusion is provided, which includes the following steps: updating the urban three-dimensional model:
and executing the following urban three-dimensional model updating process when the change rate of the first urban high-definition remote sensing image reaches a first threshold value or the change rate of the first urban space element reaches a second threshold value.
And based on a second city space element which is obtained in advance and used as a sample, identifying and extracting the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm.
And generating the space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element.
Establishing a spatial element feature map based on the feature labels of the spatial elements;
and calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map.
Constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
and carrying out manual review and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model.
And based on the correction record of the urban three-dimensional model, comparing the characteristic label difference of the urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
In the above scheme, optionally, the collecting the first city high-definition remote sensing image and the first city video monitoring image specifically includes:
subscribing a specific area high-resolution remote sensing image of a specific city by an online service, and acquiring a latest image as a first city high-resolution remote sensing image in real time according to the subscription service;
and acquiring a city monitoring video image or installing the first city area and connecting a monitoring camera in real time to acquire the video monitoring image as the first city video monitoring image.
In the foregoing scheme, further optionally, in the extracting of the first city space element and the morphological feature, the space element includes: roads, vegetation, buildings, bodies of water; the morphological characteristics are spatial characteristics, orientation characteristics, shape characteristics, structural characteristics, color characteristics and associated characteristics of the spatial elements.
In the foregoing scheme, further optionally, the pre-acquired second city space element specifically is: and acquiring historical remote sensing images, monitoring images and existing three-dimensional scenes of the existing city, and manufacturing an existing city space element sample library as a second city space element by adopting an artificial labeling mode.
In the foregoing scheme, further optionally, the step of performing manual review and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model specifically includes:
selecting various spatial elements from a high-precision city three-dimensional scene by adopting a city three-dimensional model modifying tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing with the real situation;
and the third city three-dimensional space model is a final city three-dimensional space model generated by the city three-dimensional model. When the change rate of the first city high-definition remote sensing image reaches a first threshold value or the change rate of the first city space element reaches a second threshold value, starting an update process of the city three-dimensional model, which specifically comprises the following steps:
carrying out change detection on the urban high-definition remote sensing image through an image recognition algorithm model, and starting an urban three-dimensional model updating process when the change rate reaches a first threshold value;
and identifying the change of the urban space elements based on the urban video monitoring image, and starting an urban three-dimensional model updating process when the change rate reaches a second threshold value.
The manual rechecking and manual correction are carried out on the fifth city three-dimensional space model, and the generation of the sixth city three-dimensional space model specifically comprises the following steps: and selecting various spatial elements from the high-precision city three-dimensional scene by adopting a city three-dimensional model modifying tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing the real conditions to obtain a modified sixth city three-dimensional space model.
And the six-city three-dimensional space model is a final city three-dimensional space model after the city three-dimensional model is updated.
The embodiment is based on further analysis and research on the problems in the prior art, and realizes that the existing modeling cost is high, the period is long, the oblique photogrammetry needs to carry out data field acquisition through equipment such as an unmanned aerial vehicle, and a large amount of data processing is needed after the acquisition is finished; and manual modeling requires on-site investigation at an earlier stage and a professional modeling engineer for organization to make and modify a model by using professional three-dimensional software. Either way, there is a higher cost and a longer period for city level modeling; data updating is difficult, and timeliness is difficult to guarantee. Because the traditional modeling is usually carried out according to a solidified flow by a whole body, the model is not supported to be adjusted in time according to the change of a real city, and the timeliness of a modeling result is difficult to ensure in view of high cost of the whole modeling; the traditional method can only record the urban space form in one period and cannot record the urban space transition process. Due to the restriction of national laws and regulations and local laws and regulations, many cities and regions cannot acquire oblique photogrammetric data, and the traditional city modeling mode is limited by space conditions to a certain extent. The invention fully utilizes the remote sensing image which is wider in coverage range, better in space precision, faster in updating frequency and lower in acquisition cost and is easier to acquire as a base, and combines high-definition camera image data which are distributed throughout the city as a main data source, fully and deeply learns the spatial elements such as multi-scale city spatial element image recognition roads, buildings, vegetation, water systems and the like, and extracts the spatial characteristics; meanwhile, a complete space element sample asset library with a space feature tag is constructed, a city three-dimensional model with higher reality degree and aesthetic degree is rapidly generated by performing patterned matching based on space features, the reality degree and the aesthetic degree of the generated model are optimized by adjusting feature matching parameters at any time, dynamic recognition of city local space and detail space elements is further performed based on remote sensing images and in combination with city camera video recognition images, feature recognition is performed on newly added, disappeared and form and color changes of the space elements, and therefore the city three-dimensional model is dynamically updated. On one hand, the construction cost is greatly reduced, on the other hand, the real-time dynamic increment updating of the urban three-dimensional model is supported, and the urban transition process can be recorded in a full life cycle, so that the high-precision and high-timeliness urban three-dimensional model can be universally applied to a large number of digital twin and intelligent urban digital projects.
It should be understood that although the various steps in the flow charts of fig. 1-2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-2 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
In one embodiment, a device for generating and updating a three-dimensional city model based on multi-source data fusion is provided, which comprises the following program modules: the city three-dimensional model generation module:
the urban high-definition remote sensing image acquisition system is used for collecting a first urban high-definition remote sensing image and a first urban video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and serves as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
the city three-dimensional model updating module:
the urban three-dimensional model updating method comprises the steps of executing the following urban three-dimensional model updating process when the change rate of a first urban high-definition remote sensing image reaches a first threshold value or the change rate of a first urban space element reaches a second threshold value;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the urban three-dimensional model correction record, comparing the characteristic label difference of urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
For specific limitations of the device for generating and updating the urban three-dimensional model based on the multi-source data fusion, reference may be made to the above limitations of the method for generating and updating the urban three-dimensional model based on the multi-source data fusion, and details are not repeated here. All modules in the device for generating and updating the urban three-dimensional model based on multi-source data fusion can be completely or partially realized through software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 4. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to realize a method for generating and updating a three-dimensional model of a city based on multi-source data fusion. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, which includes a memory and a processor, wherein the memory stores a computer program, and all or part of the procedures in the method of the above embodiment are involved.
In one embodiment, a computer-readable storage medium having a computer program stored thereon is provided, which relates to all or part of the processes of the above-described embodiment methods.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. The non-volatile memory may include Read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, or the like. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or dynamic Random Access memory (D4 semiconductor Random Access memory 4, DRAM), and the like.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A city three-dimensional model generation and updating method based on multi-source data fusion is characterized by comprising the following steps:
and (3) generating a city three-dimensional model:
collecting a first city high-definition remote sensing image and a first city video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and used as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
updating the urban three-dimensional model:
when the change rate of the first city high-definition remote sensing image reaches a first threshold value or the change rate of the first city space element reaches a second threshold value, executing the following city three-dimensional model updating process;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the correction record of the urban three-dimensional model, comparing the characteristic label difference of the urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
2. The method according to claim 1, wherein the collecting of the first city high-definition remote sensing image and the first city video monitoring image specifically comprises:
subscribing the high-resolution remote sensing image of a specific area of a specific city by the online service, and acquiring the latest image as the high-resolution remote sensing image of the first city in real time according to the subscription service;
and acquiring a city monitoring video image or installing the first city area and connecting a monitoring camera in real time to acquire the video monitoring image as the first city video monitoring image.
3. The method of claim 1, wherein the extracting spatial elements of the first city and the morphological features comprises: roads, vegetation, buildings, bodies of water; the morphological characteristics are spatial characteristics, orientation characteristics, shape characteristics, structural characteristics, color characteristics and correlation characteristics of the spatial elements.
4. The method according to claim 1, wherein the pre-acquired second city space elements are specifically: and acquiring historical remote sensing images, monitoring images and the existing three-dimensional scene of the existing city, and manufacturing an existing city space element sample library as a second city space element in a manual labeling mode.
5. The method according to claim 1, wherein the step of performing manual review and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model specifically comprises:
selecting various spatial elements from a high-precision city three-dimensional scene by adopting a city three-dimensional model modification tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing the real conditions;
and the third city three-dimensional space model is a final city three-dimensional space model generated by the city three-dimensional model.
6. The method according to claim 1, wherein the step of starting the update process of the urban three-dimensional model when the change rate of the first urban high-definition remote sensing image reaches a first threshold or the change rate of the first urban space element reaches a second threshold specifically comprises the following steps:
carrying out change detection on the urban high-definition remote sensing image through an image recognition algorithm model, and starting an urban three-dimensional model updating process when the change rate reaches a first threshold value;
and identifying the change of the urban space elements based on the urban video monitoring image, and starting an urban three-dimensional model updating process when the change rate reaches a second threshold value.
7. The method according to claim 1, wherein the step of performing manual review and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model specifically comprises: selecting various spatial elements from the high-precision city three-dimensional scene by adopting a city three-dimensional model modification tool, and modifying the characteristics, the position, the direction, the size and the style of the spatial elements by comparing the real conditions to obtain a modified sixth city three-dimensional space model;
and the six-city three-dimensional space model is a final city three-dimensional space model after the city three-dimensional model is updated.
8. A device for generating and updating a city three-dimensional model based on multi-source data fusion is characterized by comprising:
the city three-dimensional model generation module:
the urban high-definition remote sensing image acquisition system is used for collecting a first urban high-definition remote sensing image and a first urban video monitoring image; the first city is a three-dimensional model city to be generated;
carrying out pattern recognition on the city high-definition remote sensing image and the city video monitoring image by adopting a deep neural network, and extracting a first city space element and morphological characteristics; obtaining a first city three-dimensional space model based on the first city high-definition remote sensing image and the geographic space attribute of the first city video monitoring image;
constructing an urban space element asset library with a multi-dimensional space-time characteristic label based on a second urban space element which is acquired in advance and used as a sample; wherein the spatiotemporal features include: temporal features, spatial features, orientation features, shape features, structural features, color features, association features; the space element asset library includes: building assets, vegetation assets, road assets, water assets;
performing feature matching on the first city space element and the space asset of the city space element asset library, and fusing the first city space element and the space asset of the city space element asset library into a first city three-dimensional space model to obtain a second city three-dimensional space model;
carrying out manual rechecking and manual correction on the second city three-dimensional space model to generate a third city three-dimensional space model;
the city three-dimensional model updating module:
the urban three-dimensional model updating method comprises the steps of executing the following urban three-dimensional model updating process when the change rate of a first urban high-definition remote sensing image reaches a first threshold value or the change rate of a first urban space element reaches a second threshold value;
based on a second city space element which is obtained in advance and used as a sample, identifying and extracting features of the space elements in the first city high-definition remote sensing image and the city video monitoring image through a deep neural network algorithm;
generating a space element specific label according to the time, space position, orientation, shape, structure, color and space relation characteristics of the identified space element;
establishing a spatial element feature map based on the feature labels of the spatial elements;
calculating the spatial distribution of the identified first city space elements according to the shooting positions and the geographic space information of the first city high-definition remote sensing image and the first city video monitoring image, and generating a fourth city three-dimensional space model based on a space element feature map;
constructing a fourth city three-dimensional space model city space element asset library, and carrying out feature labeling on the assets according to the space position, orientation, shape, structure, color and space relation of the space element assets; performing feature matching on the space elements subjected to feature labeling and the space assets of the city space element asset library, and fusing the space elements subjected to feature labeling and the space assets of the city space element asset library into a fourth city three-dimensional space model to obtain a fifth city three-dimensional space model;
carrying out manual rechecking and manual correction on the fifth city three-dimensional space model to generate a sixth city three-dimensional space model;
and based on the correction record of the urban three-dimensional model, comparing the characteristic label difference of the urban space elements before and after correction, and re-correcting the space element characteristic map and the space element sample.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210801312.7A CN115147549A (en) | 2022-07-08 | 2022-07-08 | Urban three-dimensional model generation and updating method based on multi-source data fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210801312.7A CN115147549A (en) | 2022-07-08 | 2022-07-08 | Urban three-dimensional model generation and updating method based on multi-source data fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115147549A true CN115147549A (en) | 2022-10-04 |
Family
ID=83412794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210801312.7A Pending CN115147549A (en) | 2022-07-08 | 2022-07-08 | Urban three-dimensional model generation and updating method based on multi-source data fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147549A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117216039A (en) * | 2023-10-31 | 2023-12-12 | 重庆市规划和自然资源信息中心 | Method for building three-dimensional base of building based on three-dimensional cadastral database |
-
2022
- 2022-07-08 CN CN202210801312.7A patent/CN115147549A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117216039A (en) * | 2023-10-31 | 2023-12-12 | 重庆市规划和自然资源信息中心 | Method for building three-dimensional base of building based on three-dimensional cadastral database |
| CN117216039B (en) * | 2023-10-31 | 2024-04-09 | 重庆市规划和自然资源信息中心 | Method for constructing three-dimensional base of building based on three-dimensional cadastral database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liao et al. | Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d | |
| CN109059954B (en) | Method and system for supporting high-precision map lane line real-time fusion update | |
| CN111144388B (en) | Monocular image-based road sign line updating method | |
| CN113065000B (en) | Multisource heterogeneous data fusion method based on geographic entity | |
| KR20200121274A (en) | Method, apparatus, and computer readable storage medium for updating electronic map | |
| US20200401617A1 (en) | Visual positioning system | |
| US8359156B2 (en) | Map generation system and map generation method by using GPS tracks | |
| KR102199940B1 (en) | Method of constructing 3D map of mobile 3D digital twin using 3D engine | |
| KR102200299B1 (en) | A system implementing management solution of road facility based on 3D-VR multi-sensor system and a method thereof | |
| CN113989450A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| CN113034566A (en) | High-precision map construction method and device, electronic equipment and storage medium | |
| CN113804100B (en) | Method, device, equipment and storage medium for determining space coordinates of target object | |
| Zhang et al. | 3D urban buildings extraction based on airborne lidar and photogrammetric point cloud fusion according to U-Net deep learning model segmentation | |
| CN116129066A (en) | Automatic driving high-precision map model based on digital twin and high-precision static map manufacturing method | |
| CN117268350A (en) | Mobile intelligent mapping system based on point cloud data fusion | |
| CN115147549A (en) | Urban three-dimensional model generation and updating method based on multi-source data fusion | |
| CN114758086A (en) | Method and device for constructing urban road information model | |
| Wu et al. | A stepwise minimum spanning tree matching method for registering vehicle-borne and backpack LiDAR point clouds | |
| Zhang et al. | Bev-locator: An end-to-end visual semantic localization network using multi-view images | |
| US20220196432A1 (en) | System and method for determining location and orientation of an object in a space | |
| CN115830073A (en) | Map element reconstruction method, map element reconstruction device, computer equipment and storage medium | |
| CN115797438A (en) | Object positioning method, device, computer equipment, storage medium and program product | |
| CN116295457A (en) | Vehicle vision positioning method and system based on two-dimensional semantic map | |
| CN114238541A (en) | Sensitive target information acquisition method and device and computer equipment | |
| CN114116933A (en) | Semantic topology joint map building method based on monocular image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |