CN111383335B - Crowd funding photo and two-dimensional map combined building three-dimensional modeling method - Google Patents
Crowd funding photo and two-dimensional map combined building three-dimensional modeling method Download PDFInfo
- Publication number
- CN111383335B CN111383335B CN202010146833.4A CN202010146833A CN111383335B CN 111383335 B CN111383335 B CN 111383335B CN 202010146833 A CN202010146833 A CN 202010146833A CN 111383335 B CN111383335 B CN 111383335B
- Authority
- CN
- China
- Prior art keywords
- building
- photos
- street view
- photo
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/206—Drawing of charts or graphs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/04—Architectural design, interior design
Abstract
The invention relates to a three-dimensional building modeling method combining crowd funding photos and a two-dimensional map, which comprises the following steps of: downloading and organizing street view data; extracting street view photos in non-building areas; searching and grouping photos; screening photos; performing crowd funding photo-based three-dimensional modeling; point cloud registration based on 2D vector data. The invention adopts the crowd funding photo data which is free and easy to obtain, has low cost and can be obtained in a large range; the invention introduces a new building photo data source, is a three-dimensional modeling method oriented to multi-source data integration, and uses a three-step strategy: screening and grouping photos; performing crowd funding photo-based three-dimensional modeling; point cloud registration based on 2D vector data. The invention excavates the potential value of street view data, increases the availability of street view data and provides a data source for three-dimensional modeling of large-scale urban buildings. The invention skillfully combines the photo retrieval method, and realizes the rapid grouping of massive street view data by introducing photos from other sources.
Description
Technical Field
The invention relates to a three-dimensional building modeling method, in particular to a large-scale three-dimensional building modeling method combining crowd funding photos and two-dimensional map data.
Background
With the continuous development of digital cities, the demand for building three-dimensional models is gradually increased. The establishment of the three-dimensional model of the building is a research hotspot in the fields of geographic information science, remote sensing and the like. In recent years, the rapid development of computer vision technology and the emergence of high-resolution and high-precision photogrammetry technology, unmanned aerial vehicle technology and the like provide a new method and technology for building high-quality three-dimensional models of buildings. The building three-dimensional modeling technology is widely applied to other fields such as city planning, three-dimensional live-action navigation, city virtual tourism and the like. The large-scale three-dimensional building modeling has practical significance for multi-azimuth geographic information analysis application (such as solar potential estimation) and the like.
Currently, research on three-dimensional modeling of buildings can be roughly divided into three categories. One is the use of laser point cloud data, including three-dimensional modeling of buildings based on ground laser scan data, airborne laser scan data, and integrated multi-platform LiDAR data. Shi et al propose a method for reconstructing a building facade model using ground laser scan data by segmenting the original high density point cloud, extracting planar features and dividing them into various semantics (walls, doors, windows, etc.) to form a building polyhedral model. However, the model generated by the method lacks texture, and the large amount of work in the data acquisition and processing stage makes the ground laser data unsuitable for large-scale modeling. Elberink et al propose a theoretical and empirical method for quality analysis of advantages and disadvantages of three-dimensional building models reconstructed from airborne laser scanning data. The airborne LiDAR data is difficult to obtain the elevation information of the ground object, and the fine building elevation modeling under all directions and multiple viewing angles cannot be realized. Chen et al propose an integration method based on vehicle-mounted LiDAR data and airborne LiDAR data, which enables multi-view modeling of building rooftop and facade models.
Another category uses remote sensing satellite imagery data, mostly by shadow measurement, to achieve three-dimensional modeling of buildings (Liasis, 2016 zhang,2011 hartl, 1995. Mainly comprises the steps of adopting InSAR data and optical image data. Still et al analyzed the potential and limitations of using InSAR data to model buildings in built areas. UWE et al use high resolution synthetic aperture radar images, combine and match 2D image objects, and perform building height estimation by 3D clustering to complete stereo analysis. However, inSAR data is noisy and in dense building areas, it is not suitable for the recovery of building facade information because the building signals interfere with each other to affect the building modeling. Huang et al propose a semi-automatic method for building extraction and height measurement of a single IKONOS image using building top, base and shadow information, which is suitable for elevation information extraction, but cannot achieve restoration of elevation details.
A third type of method is three-dimensional modeling of buildings based on image data. The method mainly comprises three methods of space photo matching, image or image sequence analysis and photo and vector data integration. With the development of imaging technology, aerial images and ground panoramic images are the major data sources at present. Given the complementarity between the airborne and terrestrial data sets, many studies have explored joint processing of the two. Wu et al unite the aerial oblique images with the image data acquired by the ground mobile mapping system by automatic feature matching based on base surface fitting and image correction and bundle adjustment, thus realizing three-dimensional modeling and optimization of urban areas. However, the method has the disadvantages of difficult data acquisition and complicated modeling process. In addition, the problem of three-dimensional modeling of landmarks such as buildings in image sequences and video data has received much attention. Tian et al integrate building structure knowledge into a video sequence for building three-dimensional modeling to obtain the main structure of the building. Most of the methods present the building structure by polyhedral models, and the fine modeling of the building is difficult to realize. Xiao et al reconstructs the image sequence by multi-semantic image segmentation and motion structure recovery algorithm to generate a set of semi-dense point clouds and camera poses, however, the linear structure assumption in the research has certain limitations for more complex buildings. Lee et al estimate rotation parameters from the motion of the sky region in the cylindrical projected panoramic image, and combine SfM algorithm and bundle adjustment to perform three-dimensional building modeling. This method does not recover a wide range of building facade information. Fan et al propose three-dimensional modeling using OSM data and VGI photo data, however photo data coverage is small and photos from different angles of the same building are lacking.
Therefore, how to realize the rapid establishment of the three-dimensional model of the large-scale building under the condition of lower cost is a problem to be solved urgently at present.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method overcomes the defects in the prior art, and provides a three-dimensional building modeling method combining crowd funding photos and two-dimensional maps. Street view data can provide a street level view including buildings, road surfaces, street trees, traffic facilities, etc. on both sides of a road. In addition, there are a large number of photographs of buildings in data from other sources (hundredth picture library, flickr, microblog, cell phone photography, etc.). The photo data is characterized by convenience, low cost and high volume of acquisition, and can be used as an effective means and a new way for three-dimensional modeling of large-scale buildings. Therefore, the invention takes building three-dimensional modeling as a target, introduces new building crowd-funded photo data, takes tengcong street view photos as a main part, and takes other source photos as an auxiliary part. And the method is combined with two-dimensional vector contour data of a building, so that a new three-dimensional modeling method facing multi-source data integration is developed, and a point cloud of the building is converted into a real world coordinate system through point cloud registration, so that a brand-new three-dimensional modeling technical system is established.
In order to solve the technical problem, the invention provides a three-dimensional building modeling method combining crowd funding photos and two-dimensional maps, which comprises the following steps:
the three-dimensional building modeling method combining the crowd funding photo and the two-dimensional map comprises the following steps of:
the screening method comprises the following steps: the network photos and/or the shot photos in each group are required to be high in quality, the shielding condition of a target building is small, otherwise, the photos are removed, and only street view photos are adopted for modeling; the overlapping degree of two adjacent street view photos is more than 60 percent; the street view picture contains both sides of the building;
6, point cloud registration based on 2D vector data, namely downloading a 2D building vector outline and extracting the external outline of the 2D building vector outline, and amplifying or reducing the proportion of the point cloud of the building facade in the step 5 to enable the external outline size to be similar to the building external outline size in the vector diagram; the building vector external contour is scattered, the building vector external contour stored in a surface element form is converted into line elements, the line elements are scattered into scattered points according to intervals, XY coordinates of each point are calculated, and then elevation inversion is carried out to obtain three-dimensional inversion point cloud; and registering the building facade point cloud and the three-dimensional inversion point cloud to realize the conversion of the three-dimensional modeling building point cloud into a real geographic coordinate system.
The invention also has the following further features:
1. in step 2, the non-building area mainly comprises a green plant area and a sky area in the street view picture, and the preset threshold value is 0.5.
The method comprises the steps of firstly adopting Gaussian filtering to preprocess street view pictures shot under natural conditions. The specific operation is to utilize each pixel point in the template scanning image to calculate the weighted average value of the pixels in the template range.
The green plant area in the street view photograph is extracted as follows.
The green visual index GVI at a certain point is defined as the proportion of the area of a green area in a photo of the point to the area of the whole photo, and the calculation formula is as follows:
wherein, area green Is the number of green pixels in the street view picture, area total Is the total number of picture pixels.
The GVI of street view photos is calculated mainly by the following three steps: and (1) roughly extracting green pixels in the image. The green vegetation in the picture is extracted by adopting a wave band operation method proposed by Li and the like. As shown in the following equation:
G-R＝diff1
G-B＝diff2
diff1×diff2＝diff3
wherein diff1 and diff2 are a green-red difference image (G-R) and a green-blue difference image (G-B), respectively. For each pixel (i, j) in the image, if the pixel satisfies diff3 >0 and diff1>0, the pixel is green vegetation, and a foreground area obtained by binarizing the street view photo according to the rule is a crude extraction result of the green vegetation. And (2) optimizing. Mainly comprises morphological filtering and image reconstruction. Stray points of a crude extraction result in the step (1) are removed by adopting a morphological filtering method proposed by Jayaraman et al, and an image is corroded and then expanded by adopting morphological opening operation. And finally, further recovering the image through a morphological reconstruction algorithm to obtain a green plant region.
Sky region extraction is performed as follows.
Firstly, the street view image is segmented, the invention adopts a segmentation algorithm of a color image proposed by Ye et al, and the specific flow comprises the following steps: (1) intra-image color quantization; (2) carrying out initial segmentation by adopting a region growing algorithm; (3) merging similarity regions; (4) And (4) formulating a region merging termination rule by measuring the different quality degrees of the colors of different regions in the merged street view image.
Second, the SOI index proposed by Cheng et al is used to characterize the sky region in street view photos. The calculation formula of SOI is as follows:
wherein N is sk6 Is the number of regions classified as sky within the street view image after image segmentation, r i Is the number of pixels in the ith sky region, and N is the total number of pixels in the street view image.
2. In the step 2, the street view photo is screened by adopting an iterative method: and selecting a part of samples to perform three-dimensional modeling, respectively counting the area of a street view picture corresponding to a building which completes the three-dimensional modeling and the area of a non-building area in a picture corresponding to a building which fails to perform the modeling, and completing the screening of the street view pictures by removing the street view pictures of which the area of the non-building area is larger than a threshold value of 0.5.
3. In step 3, the crowd funding photo grouping method comprises the following steps: (1) for each other source photo (network photo and/or shot photo) and the street view photo data set, firstly, SIFT descriptors are used for feature extraction, secondly, nearest neighbor feature retrieval is carried out on the features of each other source photo, and the most similar street view photo corresponding to each other source photo is obtained through a voting method, dynamic pruning and smoothing processing (ZAMIR, 2010); (2) and constructing a buffer zone by taking the sampling point where the most similar street view photos are located as the center, and grouping the street view photos and the corresponding other source photos in the range of the buffer zone into a group.
4. And 5, recovering 3D information from the 2D picture by using an SfM algorithm to obtain the building facade point cloud.
The SfM three-dimensional modeling method comprises the following steps: calibrating a camera; extracting the features of the photo by using a feature extraction operator MSER and an optimal description operator SIFT, and calculating Nearest neighbor matching by using an ANN (adaptive Nearest Neighbors) algorithm; selecting a group of photos with the most matching pairs, and performing three-dimensional intersection by adopting a beam adjustment algorithm (Lourakis, 2009) to generate an initial sparse point cloud; based on the initial sparse point cloud, a CMVS (clustering multi-view stereo) algorithm and a PMVS (batch-based multi-view stereo) algorithm (Furukawa, 2008) are used to complete the final dense matching.
5. And 6, before registration, preprocessing is firstly carried out, the building vector outline is combined with the corresponding building outline edge length in the street view modeling point cloud, the initial scale is roughly estimated, then the building outline vector diagram is laid down according to the building outline of the Tencent satellite image diagram, and only the external outline is reserved.
6. In step 6, the facade point cloud inversion of the 2D vector outline firstly scatters the building vector outline, converts the building outline stored in a surface element form into line elements, beats the line elements into scattered points according to 0.5m intervals, calculates the XY coordinates of each point, and then carries out facade inversion. Assuming that the horizontal plane Z =0 of the top of the building is provided, a 2D building outline plane point set is vertically extended downwards from Z =0 along the Z axis in the opposite direction to a certain height at certain intervals to obtain a virtual facade point cloud.
7. In step 6, an Iterative Closest Point (ICP) algorithm proposed by Cheng et al is adopted to register the streetscape three-dimensional point cloud and the virtual point cloud. The method comprises two steps of initial registration and ICP fine registration.
The effective benefits of the invention are as follows:
(1) The invention provides a method for performing large-scale building three-dimensional modeling by combining crowd-funded photos (mainly Tengchin street view photos and secondarily photos from other sources) with two-dimensional vector data on the top surface of a building, which adopts freely accessible crowd-funded photo data, has low cost and can be obtained in a large range.
(2) The invention introduces a new building photo data source, further develops a new three-dimensional modeling method facing multi-source data integration, provides a set of complete building point cloud modeling system, and uses a three-step strategy: screening and grouping photos; performing crowd funding photo-based three-dimensional modeling; point cloud registration based on 2D vector data.
(3) The invention excavates the potential value of street view data, increases the availability of street view data and provides a data source for three-dimensional modeling of large-scale urban buildings.
(4) The method skillfully combines the photo retrieval method, and realizes the quick grouping of massive street view data by introducing photos from other sources.
Drawings
The invention will be further described with reference to the accompanying drawings.
FIG. 1 is a general flow diagram of an embodiment of the present invention.
FIG. 2 is a schematic diagram of point cloud registration according to an embodiment of the present invention.
FIG. 3 is a schematic of the study area and data of the present invention.
Fig. 4 is a schematic diagram of the distribution of the positions of the buildings modeled in three dimensions according to the experimental results of the embodiment of the present invention.
Fig. 5 is a schematic diagram of extracting non-architectural areas of a street view photo according to an embodiment of the invention.
FIG. 6 is a schematic diagram of the three-dimensional modeling result of the building according to the embodiment of the invention.
FIG. 7 is a diagram illustrating the contribution of photos from other sources according to an embodiment of the present invention.
FIG. 8 is a diagram illustrating accuracy verification according to an embodiment of the present invention.
Detailed Description
The technical route and the operation steps of the present invention will be more clearly understood from the following detailed description of the present invention with reference to the accompanying drawings.
In this example, the area of 37050is selected from Nanjing as experimental area, as shown in FIG. 3. Jian 37050where the district is one of the main urban areas of Nanjing, is located in the southwest of the Nanjing urban area, and is adjacent to the outer Qinhuai river and Qinhuai new river in east and south, the west faces the Yangtze river, and north ends the great street of Hanzhongmen. 37050area includes old city area with low and dense buildings and new city building area with high buildings, and has diversified building types and facade structure characteristics, is suitable for three-dimensional modeling and result analysis of buildings on two sides of street, and has typicality and representativeness. The test area coverage area is about 29.0km 2 The total number of buildings is about 14274. The downloaded street view data covers nearly 286 kilometers of city roads, the data volume of street view photos is up to 20 ten thousand, and the number of photos from other sources (network photos and/or taken photos) is 655.
In this embodiment, a three-dimensional building modeling method combining crowd funding photos and two-dimensional maps is described by taking the test area as an example, and as shown in a flowchart in fig. 1, the method specifically includes the following steps:
In the step, firstly, a road network in a test area is digitized, the road with line characteristics is resampled into discrete points by ArcGIS 10.4 software, the sampling interval is 12 meters, and the longitude and latitude coordinates of each sampling point are obtained.
For each sampling point with known longitude and latitude, calling an update map API (application programming interface), crawling a photo every 30 degrees from an initial yaw angle along the clockwise direction, downloading 12 photos of each sampling point (not only can ensure a certain content overlapping degree between two adjacent photos, but also can meet a more appropriate data storage amount), setting the pitch angle as 20 degrees, and then numbering and storing street view photos according to a uniform naming rule, as shown in FIG. 3.
And 2, extracting non-building areas of the street view photos, namely extracting all non-building areas for downloading the street view photos, and if the area ratio of the non-building areas is greater than a certain threshold (0.5 is taken in the embodiment), eliminating the street view photos without participating in three-dimensional modeling so as to obtain the street view photos after primary screening. The non-architectural area includes green vegetation and a sky area.
The green vegetation extraction process is as follows: the green visual index GVI at a certain point is defined as the proportion of the area of a green area in a photo of the point to the area of the whole photo, and the calculation formula is as follows:
wherein, area green Is the number of green pixels in the street view picture, area total Is the total number of picture pixels.
The GVI of the street view photo is calculated mainly by the following three steps: and (1) roughly extracting green pixels in the image. The green vegetation in the picture is extracted by adopting a wave band operation method proposed by Li and the like. As shown in the following equation:
G-R＝diff1
G-B＝diff2
diff1×diff2＝diff3
where diff1 and diff2 are difference images. For each pixel (i, j) in the image, if the pixel meets diff3 > 0&diff1> -0, the pixel is green vegetation, and a foreground region obtained by binarizing the street view picture according to the rule is a green vegetation crude extraction result. And (2) optimizing. Mainly comprises morphological filtering and image reconstruction. Stray points of the crude extraction result in the step (1) are removed by adopting a morphological filtering method proposed by Jayaraman and the like, and the image is corroded and then expanded by adopting morphological opening operation. And finally, further restoring the image through a morphological reconstruction algorithm.
Sky region extraction is performed according to the following method:
firstly, the street view image is segmented, and a color image segmentation algorithm proposed by Ye et al is adopted, and the specific flow comprises the following steps: (1) intra-image color quantization; (2) carrying out initial segmentation by adopting a region growing algorithm; (3) merging similarity areas; (4) And (4) formulating a region merging termination rule by measuring the different quality degrees of the colors of different regions in the merged street view image.
Second, the sky region in the street view photo is extracted using the SOI index proposed by Cheng et al. The calculation formula of SOI is as follows:
wherein N is 4k6 Is the number of regions classified as sky within the street view image after image segmentation, r i Is the number of pixels in the ith sky region, and N is the total number of pixels in the street view image.
Extracting non-building areas from all street view photos, and if the area ratio of the non-building areas is greater than a threshold value, removing the non-building areas to obtain street view photos after primary screening, wherein a group of street view photos obtained after primary screening at a certain sampling point are shown in fig. 1 (a). The initial screening of street view photos is an iterative process. Firstly, 260 street view photos are randomly selected for three-dimensional modeling, and non-building areas are extracted. And analyzing street view photos corresponding to the buildings which are subjected to three-dimensional modeling and failed in modeling respectively. Statistics shows that when the threshold value is set to be 0.5, street view pictures with the area larger than 0.5 in the non-building area can be removed to screen out street view pictures capable of achieving three-dimensional modeling, so that the number of pictures participating in modeling is reduced, and crowd funding of the pictures is facilitated. Taking two buildings as an example, as shown in fig. 5, the three-dimensional modeling of the buildings in the street view picture (a) fails, the three-dimensional modeling of the buildings in the (b) succeeds, and fig. 5 shows the extraction results of green and sky areas and the total area calculation result of the non-building area in the two pictures.
And 3, photo retrieval and grouping, namely because the street view photos after primary screening are large in quantity, and other source photos are arranged in a disordered way, and the time for directly grouping the street view photos is long, the image retrieval method provided by Cheng et al is adopted, the street view photos after primary screening are grouped by using other source photos, and crowd funded photos containing the same building are grouped into one group.
The specific process is as follows: (1) for each other source photo and street view photo data set, firstly, SIFT descriptors are used for feature extraction, secondly, nearest neighbor feature retrieval is carried out on the features of each other source photo, and the most similar street view corresponding to each other source photo is obtained through a voting method and dynamic pruning and smoothing processing (ZAMIR, 2010); (2) and (3) constructing a buffer zone by taking the sampling point where the most similar street view is located as the center, wherein the range is 200m, and dividing the street view photos in the range of the buffer zone and the corresponding photos from other sources into a group.
And 4, screening the photos, namely screening each group of photos again for carrying out three-dimensional modeling on the building, wherein each group of photos is a data source of the building to be modeled, and is shown in fig. 1 (a).
The screening rules are as follows: (1) and for the photos from other sources in each group, ensuring that the quality of the photos is high and the shielding condition of the target building is small, otherwise, removing the photos and modeling by only adopting street view photos. (2) For each group of street view photos, the overlapping degree of two adjacent photos is more than 60%; photographs that are too far from the target or too close are not suitable; the street view picture should contain both sides of the building.
The specific method comprises the following steps: (1) calibrating a camera; (2) Extracting and matching features, namely extracting the features of the picture by using a feature extraction operator MSER and an optimal description operator SIFT to obtain a 128-dimensional feature vector, and calculating nearest neighbor matching by using an ANN algorithm; (3) Recovering a motion structure and generating a sparse point cloud, selecting a group of photos with the most matched pairs, carrying out triangulation by adopting a 5-point method and a track to obtain an initial point cloud, carrying out first beam method adjustment (BA) on the initialized photo pair, and carrying out BA once introducing a new photo to generate the initial sparse point cloud; (4) And (4) generating dense point cloud, and finishing final dense matching by adopting a CMVS algorithm and a PMVS algorithm.
And 6, point cloud registration based on 2D vector data, wherein the point cloud of the building generated by using SfM is defined under a local coordinate system. The essence of the streetscape three-dimensional point cloud and 2D building outline vector data registration is to realize the absolute orientation of the streetscape three-dimensional point cloud. And the three-dimensional modeling point cloud is converted into a real geographic coordinate system by registering the three-dimensional modeling building point cloud and the point cloud inverted according to the 2D contour.
The specific flow is shown in fig. 2. Before registration, firstly, combining the building outline vector diagram with the corresponding building outline edge length in the street view modeling point cloud, roughly estimating an initial scale, and according to the building outline of the Tencent satellite image diagram, restituting the 2D building outline vector diagram corresponding to the street view facade point cloud, and keeping the external outline. Fig. 2 (a) shows the satellite image of the original building (aosoma) and the 2D building vector contour before and after the rest.
The virtual facade point cloud inversion of the 2D vector outline firstly scatters the building vector outline, converts the building outline stored in a surface element form into line elements, beats the line elements into scattered points according to 0.5m intervals, calculates XY coordinates of each point, and then carries out facade inversion. And assuming that the horizontal plane Z =0 at the top of the building is provided, extending a 2D building outline plane point set from Z =0 to a certain height h along the Z axis in a vertical and downward direction at certain intervals to obtain a virtual facade point cloud. The present invention uses the floor information provided by the height data as a reference, and assumes that the height of one floor of the height data is 3m, and the extension height h is floor x 3m. Still taking the auspicious name seat as an example, fig. 2 (b) shows the spatial position relationship between the street view point cloud and the vector outline virtual point cloud after the scale is restored, and then the two are registered.
And performing registration by adopting ICP (inductively coupled plasma), wherein the ICP algorithm has higher requirement on the initial state of the point cloud, the initial registration is performed firstly, so that the positions of the two points of the cloud are closer, and then the ICP algorithm is used for fine registration.
By selecting three pairs of homonymous points for rough registration, the selection of the third pair of homonymous points should be distributed in the middle of the whole facade point cloud as much as possible due to the possible top or bottom point cloud missing condition of the point cloud modeled by the SfM. After the rough registration, the two point clouds have good initial positions, and then the fine registration is carried out by using an ICP algorithm.
The selection of the same name point and the top view after the registration of the two are shown in fig. 2 (c). The coordinate system of the three-dimensional modeling point cloud can be transformed to the real geospatial coordinate system using the registration parameters.
Verification of the examples:
the following example is continued to verify the accuracy and reliability of the method of the present invention.
The number of building models with good modeling effect is 53, and the position distribution is shown in fig. 4, wherein the number of buildings for introducing the photo joint modeling from other sources is 25. FIG. 6 illustrates a partial building three-dimensional modeling point cloud.
In order to analyze the contribution degree of the photos from other sources to the modeling effect, 6 representative buildings (group B) are selected, and only the corresponding street view photos are used for three-dimensional modeling (group B-1), and the result is shown in FIG. 7: (1) According to the building, the shape change condition of the building is improved after other source pictures are introduced, the point cloud integrity of a square frame area is improved, and the texture of a window is clearer; (2) the inclination condition of the dense point cloud of the building is improved; (3) (4) building point cloud integrity is improved at the bottom and upper part respectively, as shown by the area in the frame; for the (5) and (6) good building point cloud modeling conditions, no obvious change exists after other source photos are introduced for modeling, and the quantity of the dense point clouds is analyzed to be changed, wherein (6) the quantity of the dense point clouds is changed from 47562 to 88118 after the other source photos are introduced, and the quantity is obviously increased.
And carrying out precision evaluation by adopting vehicle-mounted LiDAR data, and calculating the absolute offset distance between the point cloud of the three-dimensional model of each building and the true value data. Taking a single building as an example, analyzing the error condition of the modeling point cloud and the vehicle-mounted LIDAR point cloud on the spatial position, and matching colors from deep to shallow according to the error from small to large. As shown in fig. 8, the overall distribution of the error is relatively uniform and the error is in a low value region; in the figure, an area 1 shows a distribution area with a large error portion, which is mainly distributed in a top area and a bottom area, and since a point cloud generated by photo data contains a sky area, a noise residue still exists in a filtering process to cause a large top error, and in a scanning process of vehicle-mounted LIDAR data, due to shielding of bottom trees, vehicles and the like, bottom data is sparse or missing to cause a large error, as shown in an area 2 in fig. 8. The error of the middle part (area 3) of the wall body is mainly caused by the complex structure of the wall body, the building has an inner recess of about 4.8m, the noise data is large, and the point cloud data of the three-dimensional modeling has certain position offset under the condition. Therefore, the method can realize high-precision recovery of the point cloud of the facade of the building.
In addition to the above embodiments, the present invention may have other embodiments. All technical solutions formed by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the present invention.
Claims (7)
1. A three-dimensional building modeling method combining crowd funding photos and two-dimensional maps comprises the following steps:
step 1, downloading and organizing street view data, namely digitizing and resampling roads in a test area into discrete points, wherein the sampling interval of resampling is consistent with the shooting interval of the street view data, acquiring longitude and latitude coordinates of each sampling point, and downloading an all-dimensional street view picture of the sampling point;
step 2, extracting non-building areas of the street view photos, namely extracting all non-building areas for downloading the street view photos, and if the proportion of the area of the non-building areas to the street view photos is larger than a preset threshold value, rejecting the street view photos to not participate in three-dimensional modeling, so as to obtain the street view photos after primary screening;
step 3, photo retrieval and grouping, namely finding out a picture most similar to the network photo and/or the shot photo from the street view photo by using an image retrieval method; constructing a buffer area by taking the sampling point where the most similar street view photos are located as the center, and dividing the street view photos in the range of the buffer area and the corresponding network photos and/or shot photos into a group;
step 4, screening photos, namely screening each group of photos again for carrying out three-dimensional modeling on the building, wherein each group of photos is a data source of the building to be modeled;
the screening method comprises the following steps: the network photos and/or the shot photos in each group are required to ensure higher quality of the photos and smaller shielded situation of a target building, otherwise, the photos are removed, and only street view photos are adopted for modeling; the overlapping degree of two adjacent street view photos is more than 60 percent; the street view picture contains both sides of the building;
step 5, performing three-dimensional modeling based on crowd funding photos, namely recovering 3D information from the 2D photos to obtain building facade point clouds;
6, point cloud registration based on 2D vector data, namely downloading a 2D building vector outline and extracting the external outline of the building vector outline, and amplifying or reducing the proportion of the point cloud of the building facade in the step 5 to enable the external outline size to be similar to the building external outline size in the vector diagram; the building vector external contour is scattered, the building vector external contour stored in a surface element form is converted into line elements, the line elements are scattered into scattered points according to intervals, XY coordinates of each point are calculated, and then elevation inversion is carried out to obtain three-dimensional inversion point cloud; and registering the building facade point cloud and the three-dimensional inversion point cloud to realize the conversion of the three-dimensional modeling building point cloud into a real geographic coordinate system.
2. The crowd funding picture and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: in step 2, the non-building area is a green planting area and a sky area, and the preset threshold value is 0.5.
3. The crowd funding photo and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: in step 3, for each network photo and/or shot photo and street view photo, SIFT descriptors are used for feature extraction firstly, nearest neighbor feature retrieval is carried out on the features of each network photo and/or shot photo, and the street view photo with the most similar network photo and/or shot photo is obtained through voting method, dynamic pruning and smoothing processing.
4. The crowd funding photo and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: and 5, recovering 3D information from the 2D picture by using an SfM algorithm to obtain the building facade point cloud.
5. The crowd funding picture and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: and 6, amplifying or reducing the proportion of the point cloud of the building facade to enable the difference between the outer contour size of the building facade point cloud and the outer contour size of the building in the vector diagram to be less than 5%.
6. The crowd funding photo and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: in step 6, the line elements are broken into scattered points at 0.5m intervals.
7. The crowd funding picture and two-dimensional map combined building three-dimensional modeling method of claim 1, wherein: and 6, registering the building facade point cloud obtained in the step 5 and the three-dimensional inversion point cloud by adopting an ICP (inductively coupled plasma) algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010146833.4A CN111383335B (en) | 2020-03-05 | 2020-03-05 | Crowd funding photo and two-dimensional map combined building three-dimensional modeling method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010146833.4A CN111383335B (en) | 2020-03-05 | 2020-03-05 | Crowd funding photo and two-dimensional map combined building three-dimensional modeling method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111383335A CN111383335A (en) | 2020-07-07 |
| CN111383335B true CN111383335B (en) | 2023-03-21 |
Family
ID=71221381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010146833.4A Active CN111383335B (en) | 2020-03-05 | 2020-03-05 | Crowd funding photo and two-dimensional map combined building three-dimensional modeling method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111383335B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287061A (en) * | 2020-11-17 | 2021-01-29 | 深圳市泰同科技有限公司 | Method for splicing street view elevation map by utilizing network open data |
| CN113920266B (en) * | 2021-11-03 | 2022-10-21 | 泰瑞数创科技(北京)股份有限公司 | Artificial intelligence generation method and system for semantic information of city information model |
| CN114202622B (en) * | 2022-02-18 | 2022-05-31 | 腾讯科技（深圳）有限公司 | Virtual building generation method, device, equipment and computer readable storage medium |
| CN116106904B (en) * | 2023-02-22 | 2023-07-18 | 深圳大学 | Facility deformation monitoring method and facility deformation monitoring equipment for object MT-InSAR |
| CN116918593B (en) * | 2023-09-14 | 2023-12-01 | 众芯汉创(江苏)科技有限公司 | Binocular vision unmanned image-based power transmission line channel tree obstacle monitoring system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6895126B2 (en) * | 2000-10-06 | 2005-05-17 | Enrico Di Bernardo | System and method for creating, storing, and utilizing composite images of a geographic location |
| CN103247225B (en) * | 2012-02-13 | 2015-04-29 | 联想(北京)有限公司 | Instant positioning and map building method and equipment |
| CN107133325B (en) * | 2017-05-05 | 2020-01-07 | 南京大学 | Internet photo geographic space positioning method based on street view map |
| CN109858361B (en) * | 2018-12-28 | 2023-04-18 | 南京航空航天大学 | Positioning method for repeated structure building based on street view image matching |
-
2020
- 2020-03-05 CN CN202010146833.4A patent/CN111383335B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111383335A (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111383335B (en) | Crowd funding photo and two-dimensional map combined building three-dimensional modeling method | |
| Balsa-Barreiro et al. | Generation of visually aesthetic and detailed 3D models of historical cities by using laser scanning and digital photogrammetry | |
| Cheng et al. | 3D building model reconstruction from multi-view aerial imagery and lidar data | |
| CN107133325B (en) | Internet photo geographic space positioning method based on street view map | |
| Zhang | Automatic digital surface model (DSM) generation from linear array images | |
| US7509241B2 (en) | Method and apparatus for automatically generating a site model | |
| CN100371952C (en) | Video object recognition device and recognition method, video annotation giving device and giving method, and program | |
| WO2018061010A1 (en) | Point cloud transforming in large-scale urban modelling | |
| Pylvanainen et al. | Automatic alignment and multi-view segmentation of street view data using 3d shape priors | |
| CA2702729A1 (en) | Geospatial modeling system and related method using multiple sources of geographic information | |
| Pepe et al. | Use of SfM-MVS approach to nadir and oblique images generated throught aerial cameras to build 2.5 D map and 3D models in urban areas | |
| KR100904078B1 (en) | A system and a method for generating 3-dimensional spatial information using aerial photographs of image matching | |
| KR101079475B1 (en) | A system for generating 3-dimensional urban spatial information using point cloud filtering | |
| CN113240755B (en) | City scene composition method and system based on street view image and vehicle-mounted laser fusion | |
| KR101079531B1 (en) | A system for generating road layer using point cloud data | |
| Nakagawa et al. | Fusing stereo linear CCD image and laser range data for building 3D urban model | |
| Zhu | A pipeline of 3D scene reconstruction from point clouds | |
| JP4231279B2 (en) | Digital image processing device | |
| KR101083902B1 (en) | A system for generating 3-dimensional spatial information using an aerial lidar surveying data | |
| KR101079359B1 (en) | A system for generating digital map using an aerial photograph and aerial light detection of ranging data | |
| KR101114904B1 (en) | A system and method for generating urban spatial information using a draft map and an aerial laser measurement data | |
| Mathews et al. | Three-dimensional building modeling using structure from motion: improving model results with telescopic pole aerial photography | |
| Wu et al. | Building Facade Reconstruction Using Crowd-Sourced Photos and Two-Dimensional Maps | |
| Boulaassal et al. | 3D modelling of facade features on large sites acquired by vehicle based laser scanning | |
| Yu et al. | A cue line based method for building modeling from LiDAR and satellite imagery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |