CN117115243B - Building group outer facade window positioning method and device based on street view picture - Google Patents
Building group outer facade window positioning method and device based on street view picture Download PDFInfo
- Publication number
- CN117115243B CN117115243B CN202311370849.3A CN202311370849A CN117115243B CN 117115243 B CN117115243 B CN 117115243B CN 202311370849 A CN202311370849 A CN 202311370849A CN 117115243 B CN117115243 B CN 117115243B
- Authority
- CN
- China
- Prior art keywords
- building
- view
- facade
- target
- window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000004927 fusion Effects 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims abstract description 8
- 238000012937 correction Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000015654 memory Effects 0.000 claims description 16
- 238000013135 deep learning Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 238000010276 construction Methods 0.000 abstract description 5
- 239000002131 composite material Substances 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 5
- 230000002411 adverse Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a method for positioning a building group outer facade window based on a street view picture, which is applied to the technical field of urban three-dimensional modeling and comprises the following steps: collecting a multi-view street view picture set of each building outer elevation of the target building group, and carrying out view fusion on the multi-view street view picture set to obtain a synthesized street view front view of each building outer elevation of the target building group; identifying pixel coordinates for each window location for each building facade of the target building group based on the composite street view elevation; and obtaining world coordinates of each window position of each building facade of the target building group through coordinate conversion based on the pixel coordinates of each window position of each building facade of the target building group. The method can fully utilize the advantages of the street view picture in the aspect of urban information acquisition, solve the technical problem of identifying the outer facade window position of the building group from the street view picture, and promote the rapid construction of the high-detail urban three-dimensional model.
Description
Technical Field
The invention relates to the technical field of urban three-dimensional modeling, in particular to a method and a device for positioning a building group outer facade window based on street view pictures.
Background
With the rapid development of ideas such as smart cities and digital earth, the demand for three-dimensional fine modeling of cities is increasing. The spatial position of the building group outer elevation window in the city can be rapidly and accurately determined, and rapid construction of a high-detail city three-dimensional model can be promoted. The street view picture has the advantages of complete coverage, low acquisition cost and the like, and a user can flexibly and freely download the street view picture of the geographical position of interest from the map website through the corresponding API interface. The position information of the outer facade window of the urban building group can be rapidly identified through the street view picture, so that the defects of the traditional means in terms of automation, convenience and cost can be overcome, and the method has wide application prospects in the field of urban three-dimensional modeling. However, street view pictures of building facades are often obscured by surrounding trees, debris, etc., affecting the accuracy of identifying window information from the building facades. There is currently no mature method for eliminating the adverse effect of such shielding as much as possible, and realizing high-precision identification of building group facade window position information based on street view pictures.
Disclosure of Invention
The embodiment of the invention provides a method and a device for positioning a building group outer elevation window based on a street view picture, which are used for fully utilizing the advantages of the street view picture in the aspect of urban information acquisition, solving the technical problem of identifying the building group outer elevation window position from the street view picture and promoting the rapid construction of a high-detail urban three-dimensional model. The technical scheme is as follows:
on the one hand, the embodiment of the invention provides a method for positioning a building group outer facade window based on street view pictures, which comprises the following steps:
s1: collecting a multi-view street view picture set of the outer elevation of each building of the target building group to obtain a multi-view street view picture data set;
s2: performing view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of the target building group;
s3: identifying pixel coordinates for each window position for each building facade of a target building group based on the synthetic street view elevation;
s4: and obtaining world coordinates of each window position of each building facade of the target building group through coordinate conversion based on the pixel coordinates of each window position of each building facade of the target building group.
Preferably, in the step S1, collecting a multi-view street view picture set of each building facade of the target building group to obtain a multi-view street view picture data set, including:
s11: road network data in the area range of the target building group and bottom surface outline polygon data of each building are obtained;
s12: identifying each building facade and its position in the target building group based on the floor profile polygon data;
s13: based on the position of each building facade in the target building group and the road network data, identifying road sections with the distance of each building facade in the target building group within a preset range, and arranging a series of observation points corresponding to each building facade along the road sections according to preset intervals;
s14: taking the ray direction from each observation point in the series of observation points corresponding to each building facade to the corresponding facade as a sight line direction; extracting serial shooting azimuth angle data corresponding to each building facade based on the sight line direction;
s15: and capturing street view pictures of the outer facades of the buildings corresponding to the observation points of the outer facades of the buildings from a preset map website according to the series observation points and series shooting azimuth angle data, and obtaining multi-view street view picture sets of the outer facades of the buildings to form multi-view street view picture data sets.
Preferably, the road network data and the bottom surface outline polygon data of each building within the area where the acquisition target building group in S11 is located include:
s111: acquiring range data of a target building group, wherein the range data comprises a longitude range and a latitude range;
s112: obtaining road network data in a range according to the range data, wherein the road network data comprises position coordinates of each node forming a road network central line;
s113: and acquiring the bottom surface outline polygonal data of each building in the range according to the range data, wherein the bottom surface outline polygonal data of each building comprises the position coordinates of each node of the bottom surface outline polygonal.
Preferably, in the step S2, the view merging is performed on the multi-view street view picture dataset to obtain a synthesized street view front view of each building facade of the target building group, including:
s21: based on the multi-view street view picture data set, a building outer elevation object framed by a target mask is segmented from each street view picture in the multi-view street view picture data set through a deep learning algorithm to obtain a building target mask map, and a building target mask map set of each building outer elevation is formed;
s22: the correction parameters of each street view picture in the multi-view street view picture data set are obtained through a correction algorithm, each street view picture is corrected into an original front view according to the correction parameters, and an original front view picture set of each building facade is obtained;
s23: correcting a corresponding building target mask image into a target mask front view according to the correction parameters of each street view image to obtain a target mask front view set of each building outer elevation;
s24: based on the target mask front view set of each building outer elevation, establishing a pixel corresponding relation between every two target mask front views as a pixel corresponding relation between every two original front views of each building outer elevation;
s25: in each target mask front view of a target mask front view set of each building outer elevation, calculating a target mask which is not in a null proportion, and selecting an original front view corresponding to a target mask front view with the target mask which is not in a highest null proportion as a reference front view of each building outer elevation, wherein the target mask which is not in a null proportion corresponds to an unobstructed part of images in the original front view;
s26: identifying pixels of an occluded part in the reference elevation view based on the reference elevation view of each building facade and the target mask elevation view corresponding to the reference elevation view;
s27: and filling pixels of the shielded part in the reference front view of each building facade according to the pixel corresponding relation between every two original front views of each building facade based on the original front view set of each building facade, so as to obtain the synthesized street view front view of each building facade.
Preferably, in the step S26, the identifying the pixels of the blocked portion in the reference front view based on the reference front view of each building facade and the target mask front view corresponding to the reference front view includes:
s261: for each pixel of the blocked part in the reference front view of each building facade, acquiring corresponding pixel values in the original front views of other non-reference front views based on the pixel correspondence between every two original front views of each building facade, and forming a pixel value sequence [ a1, a2, … an ];
s262: sequence of pixel values [ , /> ,…/>]Inputting formula (1) to obtain an estimated pixel value:
（1）
wherein,for the estimated pixel value +.>Is the original elevation weighting;
s263: and filling each pixel of the shielded part in the reference front view of each building facade based on the estimated pixel value to obtain a synthesized street view front view of each building facade.
Preferably, identifying pixel coordinates of each window position of each building facade of the target building group based on the synthesized street view front view at the S3 includes:
s31: dividing each window framed by a target mask through a deep learning algorithm based on the synthesized street view front view of each building facade to obtain a window target mask map of each building facade;
s32: performing binarization processing on the window target mask map of each building facade, and then extracting the outline of each window of each building facade by using a contrast algorithm;
s33: pixel coordinates of vertices of each window of each building facade are extracted based on the contour of each window of each building facade.
Preferably, in the step S4, the pixel coordinates of each window position of each building facade of the target building group are obtained by converting coordinates, and the world coordinates of each window position of each building facade of the target building group are included:
s41: establishing a rotation matrix of a pixel coordinate system and a world coordinate system of each building facade, wherein the rotation matrix is a transformation matrix from the world coordinate system to a front view shooting camera coordinate system;
s42: multiplying the pixel coordinates of the vertexes of each window of each building facade by the rotation matrix to obtain projected vertex coordinates of each window of each building facade;
s43: adding the projected vertex coordinates of each window and the central coordinate point of the building facade to obtain the world coordinates of each window position, and forming the world coordinate data of the building facade window position of the target building group.
In a second aspect, an embodiment of the present invention provides a positioning device for a building group facade window based on street view pictures, including the following steps:
data unit: the multi-view street view picture collection method comprises the steps of collecting multi-view street view picture sets of outer facades of each building of a target building group to obtain multi-view street view picture data sets;
a synthesis unit: the method comprises the steps of performing view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of a target building group;
pixel coordinate unit: pixel coordinates for identifying each window position of each building facade of a target building group based on the synthetic street view elevation;
world coordinate unit: the world coordinate of each window position of each building facade of the target building group is obtained through coordinate conversion based on the pixel coordinate of each window position of each building facade of the target building group.
In a third aspect, an embodiment of the present invention provides an electronic device, including: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space surrounded by the shell, and the processor and the memory are arranged on the circuit board; a power supply circuit for supplying power to each circuit or device of the electronic apparatus; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory for performing the method as described above.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium, where one or more programs are stored, and the one or more programs are executable by one or more processors to implement the method as described above.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
compared with the prior art, the technical scheme has at least the following beneficial effects: the method and the device for positioning the building group outer elevation window based on the street view picture fully utilize the advantages of easy acquisition, complete coverage and the like of the street view picture, eliminate adverse effects on subsequent window position identification caused by shielding of the building elevation as much as possible through fusion of the multi-view street view picture, and realize rapid determination of the urban building group outer elevation window position information from the street view picture, thereby providing technical support for rapid construction of a high-detail urban three-dimensional model.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an execution flow of a method for positioning windows on an outer facade of a building group based on street view pictures according to an embodiment of the present invention;
FIG. 2 is a schematic view of the road network shape and the bottom profile polygon of each building within the area of the target building group according to the embodiment of the present invention;
FIG. 3 is a front view of a target mask for a window of an facade provided in an embodiment of the invention;
fig. 4 is a block diagram of a building group outer elevation window positioning device based on street view pictures, which is provided by the embodiment of the invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides a method for positioning a building group outer elevation window based on a street view picture, which can be realized by electronic equipment, wherein the electronic equipment can be a terminal or a server. The process flow of the method for positioning the outer facade window of the building group based on the street view picture as shown in fig. 1 can comprise the following steps:
s1: collecting a multi-view street view picture set of the outer elevation of each building of the target building group to obtain a multi-view street view picture data set;
preferably, the S1 includes:
s11: road network data in the area range of the target building group and bottom surface outline polygon data of each building are obtained;
preferably, the step S111 includes:
s111: acquiring range data of a target building group, wherein the range data comprises a longitude range and a latitude range;
s112: obtaining road network data in a range according to the range data, wherein the road network data comprises position coordinates of each node forming a road network central line;
s113: acquiring the bottom surface outline polygonal data of each building in the range according to the range data, wherein the bottom surface outline polygonal data of each building comprises the position coordinates of each node of the bottom surface outline polygonal;
s12: identifying each building facade and its position in the target building group based on the floor profile polygon data;
s13: based on the position of each building facade in the target building group and the road network data, identifying road sections with the distance of each building facade in the target building group within a preset range, and arranging a series of observation points corresponding to each building facade along the road sections according to preset intervals;
s14: taking the ray direction from each observation point in the series of observation points corresponding to each building facade to the corresponding facade as a sight line direction; extracting serial shooting azimuth angle data corresponding to each building facade based on the sight line direction;
s15: and capturing street view pictures of the outer facades of the buildings corresponding to the observation points of the outer facades of the buildings from a preset map website according to the series observation points and series shooting azimuth angle data, and obtaining multi-view street view picture sets of the outer facades of the buildings to form multi-view street view picture data sets.
In some embodiments, the embodiments of the present invention target building groups in a university campus in north China. Fig. 2 shows a schematic diagram of a road network shape and a bottom outline polygon of each building within an area where a target building group is located according to an embodiment of the present invention.
It should be noted that, the multi-view street view picture of a certain facade of a certain building in the target building group provided by the embodiment of the invention is obtained by shooting a series of observation points which are arranged along a road network with the nearest distance to the facade according to a preset distance of 20 meters against the facade.
S2: performing view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of the target building group;
preferably, the S2 includes:
s21: based on the multi-view street view picture data set, a building outer elevation object framed by a target mask is segmented from each street view picture in the multi-view street view picture data set through a deep learning algorithm to obtain a building target mask map, and a building target mask map set of each building outer elevation is formed;
s22: the correction parameters of each street view picture in the multi-view street view picture data set are obtained through a correction algorithm, each street view picture is corrected into an original front view according to the correction parameters, and an original front view picture set of each building facade is obtained;
s23: correcting a corresponding building target mask image into a target mask front view according to the correction parameters of each street view image to obtain a target mask front view set of each building outer elevation;
s24: based on the target mask front view set of each building outer elevation, establishing a pixel corresponding relation between every two target mask front views as a pixel corresponding relation between every two original front views of each building outer elevation;
s25: in each target mask front view of a target mask front view set of each building outer elevation, calculating a target mask which is not in a null proportion, and selecting an original front view corresponding to a target mask front view with the target mask which is not in a highest null proportion as a reference front view of each building outer elevation, wherein the target mask which is not in a null proportion corresponds to an unobstructed part of images in the original front view;
s26: identifying pixels of an occluded part in the reference elevation view based on the reference elevation view of each building facade and the target mask elevation view corresponding to the reference elevation view;
preferably, the step S226 includes:
s261: for each pixel of the blocked part in the reference front view of each building facade, acquiring corresponding pixel values in the original front views of other non-reference front views based on the pixel correspondence between every two original front views of each building facade, and forming a pixel value sequence [ a1, a2, … an ];
s262: inputting the sequence of pixel values [ a1, a2, … an ] into equation (1) to obtain an estimated pixel value:
（1）
wherein,for the estimated pixel value +.>Is the original elevation weighting;
s263: filling each pixel of the shielded part in the reference front view of each building facade based on the estimated pixel value to obtain a synthesized street view front view of each building facade;
s27: and filling pixels of the shielded part in the reference front view of each building facade according to the pixel corresponding relation between every two original front views of each building facade based on the original front view set of each building facade, so as to obtain the synthesized street view front view of each building facade.
In some embodiments, correcting the building elevation picture of each view angle into elevation views to obtain a series of elevation views of the building elevation, wherein the differences exist in the blocked portions of the building elevation in each elevation view;
preferably, for example, a front view corrected from a building elevation picture at one view angle is taken as a reference picture of the building elevation, for a pixel of An occluded part in the reference picture, it is determined whether the pixel in other front views belongs to the occluded part, if the pixel of the pictures A1, A2 …, an in other front views does not belong to the occluded part, the pixel of the pictures A1, A2 …, an in the same position is extracted, and the pixel of the occluded part in the reference picture is replaced by the estimated pixel value, which is brought into the formula (1). The weights of the pictures can be estimated according to the shielding condition, or only one picture can be selected to have the weight of 1, and the weights of other pictures are zero. And finally, a synthesized street view front view of each building elevation of the target building group is obtained, wherein the blocked area of the building elevation object in the synthesized street view front view is smaller than that of the building elevation object in any one of the multi-view street view pictures.
It should be further noted that, in the embodiment of the present invention, a Mask R-CNN deep neural network model is used to detect a building facade, so as to segment a building facade object from street view pictures under each view angle. Compared with the front view after the correction of the building elevation, the area of the building elevation shielded in the front view of the synthesized street view is obviously reduced, and a high-quality building elevation picture is provided for the subsequent identification of window pixels.
S3: identifying pixel coordinates for each window position for each building facade of a target building group based on the synthetic street view elevation;
preferably, the S3 includes:
s31: dividing each window framed by a target mask through a deep learning algorithm based on the synthesized street view front view of each building facade to obtain a window target mask map of each building facade;
s32: performing binarization processing on the window target mask map of each building facade, and then extracting the outline of each window of each building facade by using a contrast algorithm;
s33: pixel coordinates of vertices of each window of each building facade are extracted based on the contour of each window of each building facade.
In some embodiments, in embodiments of the present invention, a Mask R-CNN deep neural network model is used to segment window pixels on a building facade. Fig. 3 shows a schematic view of a window pixel segmentation result of a certain facade of a certain building according to an embodiment of the present invention. White pixels in the figure are pixels belonging to a window.
S4: and obtaining world coordinates of each window position of each building facade of the target building group through coordinate conversion based on the pixel coordinates of each window position of each building facade of the target building group.
Preferably, the S4 includes:
s41: establishing a rotation matrix of a pixel coordinate system and a world coordinate system of each building facade, wherein the rotation matrix is a transformation matrix from the world coordinate system to a front view shooting camera coordinate system;
s42: multiplying the pixel coordinates of the vertexes of each window of each building facade by the rotation matrix to obtain projected vertex coordinates of each window of each building facade;
s43: adding the projected vertex coordinates of each window and the central coordinate point of the building facade to obtain the world coordinates of each window position, and forming the world coordinate data of the building facade window position of the target building group.
It should be noted that, in the embodiment of the present invention, for any window of any facade of any building, based on the pixel coordinates of the window, a constant 1 is added after x and y pixel coordinates of each vertex of an outline rectangle of a pixel block occupied by the window, so that the pixel coordinates of each vertex are formed into a 3×1 coordinate vector of (x, y, 1), then a new 3×1 coordinate vector is obtained by multiplying the 3×1 coordinate vector of each vertex by a 3×3 rotation matrix, and then a 4×1 coordinate vector is obtained by multiplying the new 3×1 coordinate vector of each vertex by an inverse matrix of the 3×4 rotation matrix and an inverse matrix of the 4×4 viewing angle matrix in order, where the first three elements of the coordinate vector are world coordinates of the corresponding vertex
The above description of the method embodiments further describes the solution of the present invention by means of device embodiments.
As shown in fig. 4, the embodiment of the invention provides a building group outer elevation window positioning device based on street view pictures, which comprises the following steps:
data unit 410: the multi-view street view picture collection method comprises the steps of collecting multi-view street view picture sets of outer facades of each building of a target building group to obtain multi-view street view picture data sets;
the synthesizing unit 420: the method comprises the steps of performing view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of a target building group;
pixel coordinate unit 430: pixel coordinates for identifying each window position of each building facade of a target building group based on the synthetic street view elevation;
world coordinates unit 440: the world coordinate of each window position of each building facade of the target building group is obtained through coordinate conversion based on the pixel coordinate of each window position of each building facade of the target building group.
The invention provides a method and a device for positioning a building group outer facade window based on a street view picture. Meanwhile, the method can rapidly position the outer facade window of the urban building group and provide contour information, so that the technical problem of identifying the position of the outer facade window of the building group from the street view picture is solved, and rapid construction of a high-detail urban three-dimensional model is promoted.
The invention provides an electronic device, which is characterized by comprising: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space surrounded by the shell, and the processor and the memory are arranged on the circuit board; a power supply circuit for supplying power to each circuit or device of the electronic apparatus; the memory is used for storing executable program codes; the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory for performing the method as described above.
The present invention provides a computer-readable storage medium storing one or more programs executable by one or more processors to implement the method as described above.
Fig. 5 is a schematic structural diagram of an electronic device 500 according to an embodiment of the present invention, where the electronic device 500 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 and one or more memories 502, where at least one instruction is stored in the memories 502, and the at least one instruction is loaded and executed by the processors 501 to implement the steps of the above-mentioned chinese text spell checking method.
In an exemplary embodiment, a computer readable storage medium, such as a memory including instructions executable by a processor in a terminal to perform the above-described chinese text spell checking method, is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The following points need to be described:
(1) The drawings of the embodiments of the present invention relate only to the structures related to the embodiments of the present invention, and other structures may refer to the general designs.
(2) In the drawings for describing embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced for clarity, i.e., the drawings are not drawn to actual scale. It will be understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, it can be "directly on" or "under" the other element or intervening elements may be present.
(3) The embodiments of the invention and the features of the embodiments can be combined with each other to give new embodiments without conflict.
The foregoing description of the preferred embodiments of the present invention should not be taken as limiting the scope of the invention, which is defined by the appended claims, and any modifications, equivalents, improvements or the like that fall within the spirit and scope of the invention are intended to be embraced by the claims.
Claims (8)
1. A method for positioning a building group outer facade window based on street view pictures is characterized by comprising the following steps:
s1: collecting a multi-view street view picture set of the outer elevation of each building of the target building group to obtain a multi-view street view picture data set;
s2: performing view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of the target building group:
s2 carries out view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of the target building group, and the method comprises the following steps:
s21: based on the multi-view street view picture data set, dividing building outer elevation objects framed by target masks from each street view picture in the multi-view street view picture data set through a deep learning algorithm to obtain building target mask patterns, and forming building target mask patterns of each building outer elevation;
s22: the correction parameters of each street view picture in the multi-view street view picture data set are obtained through a correction algorithm, each street view picture is corrected into an original front view according to the correction parameters, and an original front view picture set of each building facade is obtained;
s23: correcting a corresponding building target mask image into a target mask front view according to the correction parameters of each street view image to obtain a target mask front view set of each building outer elevation;
s24: based on the target mask front view set of each building outer elevation, establishing a pixel corresponding relation between every two target mask front views as a pixel corresponding relation between every two original front views of each building outer elevation;
s25: in each target mask front view of the target mask front view set of each building outer elevation, calculating that the target mask is not in a null proportion, selecting an original front view corresponding to the target mask front view with the highest target mask not in the null proportion as a reference front view of each building outer elevation, wherein the target mask is not in a null proportion and is a part of images which are not blocked in the original front view;
s26: identifying pixels of an occluded part in the reference elevation view based on the reference elevation view of each building facade and the target mask elevation view corresponding to the reference elevation view;
s27: filling pixels of the shielded part in the reference front view of each building facade according to the pixel corresponding relation between every two original front views of each building facade based on the original front view set of each building facade to obtain a synthesized street view front view of each building facade;
s3: identifying pixel coordinates for each window position for each building facade of a target building group based on the synthetic street view elevation:
the step S3 of identifying pixel coordinates of each window position of each building facade of a target building group based on the synthesized street view front view includes:
s31: dividing each window framed by a target mask through a deep learning algorithm based on the synthesized street view front view of each building facade to obtain a window target mask map of each building facade;
s32: performing binarization processing on the window target mask map of each building facade, and then extracting the outline of each window of each building facade by using a contrast algorithm;
s33: extracting pixel coordinates of vertices of each window of each building facade based on the contour of each window of each building facade;
s4: and obtaining world coordinates of each window position of each building facade of the target building group through coordinate conversion based on the pixel coordinates of each window position of each building facade of the target building group.
2. The method for positioning a building group facade window based on a street view picture according to claim 1, wherein the step of S1 of collecting the multi-view street view picture set of each building facade of the target building group to obtain the multi-view street view picture data set comprises the following steps:
s11: road network data in the area range of the target building group and bottom surface outline polygon data of each building are obtained;
s12: identifying each building facade and its position in the target building group based on the floor profile polygon data;
s13: based on the position of each building facade in the target building group and the road network data, identifying road sections with the distance of each building facade in the target building group within a preset range, and arranging a series of observation points corresponding to each building facade along the road sections according to preset intervals;
s14: taking the ray direction from each observation point in the series of observation points corresponding to each building facade to the corresponding facade as a sight line direction; extracting serial shooting azimuth angle data corresponding to each building facade based on the sight line direction;
s15: and capturing street view pictures of the outer facades of the buildings corresponding to the observation points of the outer facades of the buildings from a preset map website according to the series observation points and series shooting azimuth angle data, and obtaining multi-view street view picture sets of the outer facades of the buildings to form multi-view street view picture data sets.
3. The method for positioning a building group facade window based on a street view picture according to claim 2, wherein the step S11 of obtaining road network data and bottom surface outline polygon data of each building in the area where the target building group is located includes:
s111: acquiring range data of a target building group, wherein the range data comprises a longitude range and a latitude range;
s112: obtaining road network data in a range according to the range data, wherein the road network data comprises position coordinates of each node forming a road network central line;
s113: and acquiring the bottom surface outline polygonal data of each building in the range according to the range data, wherein the bottom surface outline polygonal data of each building comprises the position coordinates of each node of the bottom surface outline polygonal.
4. The method for positioning a group facade window based on street view pictures according to claim 1, wherein the step S26 of identifying pixels of the blocked portion in the reference elevation based on the reference elevation view of each building facade and the target mask elevation view corresponding to the reference elevation view comprises:
s261: for each pixel of the blocked part in the reference front view of each building facade, acquiring corresponding pixel values in the original front views of other non-reference front views based on pixel correspondence between every two original front views of each building facade, and forming a pixel value sequence [ a1, a2, & gt an ];
s262: inputting a sequence of pixel values [ a1, a2,..an ] into equation (1), resulting in an estimated pixel value:
（1）
wherein,for the estimated pixel value +.>Is the original elevation weighting;
s263: and filling each pixel of the shielded part in the reference front view of each building facade based on the estimated pixel value to obtain a synthesized street view front view of each building facade.
5. The method for positioning windows on building group facades based on street view pictures according to claim 1, wherein the step S4 of obtaining the world coordinates of each window position of each building facade of the target building group by coordinate conversion based on the pixel coordinates of each window position of each building facade of the target building group comprises the following steps:
s41: establishing a rotation matrix of a pixel coordinate system and a world coordinate system of each building facade, wherein the rotation matrix is a transformation matrix from the world coordinate system to a front view shooting camera coordinate system;
s42: multiplying the pixel coordinates of the vertexes of each window of each building facade by the rotation matrix to obtain projected vertex coordinates of each window of each building facade;
s43: adding the projected vertex coordinates of each window and the central coordinate point of the building facade to obtain the world coordinates of each window position, and forming the world coordinate data of the building facade window position of the target building group.
6. A group facade window positioning device based on street view pictures, which is suitable for the group facade window positioning method based on street view pictures as claimed in any one of the claims 1-5, and comprises the following steps:
data unit: the multi-view street view picture collection method comprises the steps of collecting multi-view street view picture sets of outer facades of each building of a target building group to obtain multi-view street view picture data sets;
a synthesis unit: the method is used for carrying out view fusion on the multi-view street view picture data set to obtain a synthesized street view front view of each building outer elevation of the target building group, and specifically comprises the following steps:
s21: based on the multi-view street view picture data set, dividing building outer elevation objects framed by target masks from each street view picture in the multi-view street view picture data set through a deep learning algorithm to obtain building target mask patterns, and forming building target mask patterns of each building outer elevation;
s22: the correction parameters of each street view picture in the multi-view street view picture data set are obtained through a correction algorithm, each street view picture is corrected into an original front view according to the correction parameters, and an original front view picture set of each building facade is obtained;
s23: correcting a corresponding building target mask image into a target mask front view according to the correction parameters of each street view image to obtain a target mask front view set of each building outer elevation;
s24: based on the target mask front view set of each building outer elevation, establishing a pixel corresponding relation between every two target mask front views as a pixel corresponding relation between every two original front views of each building outer elevation;
s25: in each target mask front view of the target mask front view set of each building outer elevation, calculating that the target mask is not in a null proportion, selecting an original front view corresponding to the target mask front view with the highest target mask not in the null proportion as a reference front view of each building outer elevation, wherein the target mask is not in a null proportion and is a part of images which are not blocked in the original front view;
s26: identifying pixels of an occluded part in the reference elevation view based on the reference elevation view of each building facade and the target mask elevation view corresponding to the reference elevation view;
s27: filling pixels of the shielded part in the reference front view of each building facade according to the pixel corresponding relation between every two original front views of each building facade based on the original front view set of each building facade to obtain a synthesized street view front view of each building facade;
pixel coordinate unit: pixel coordinates for identifying each window position of each building facade of a target building group based on the synthetic street view front view, specifically comprising:
s31: dividing each window framed by a target mask through a deep learning algorithm based on the synthesized street view front view of each building facade to obtain a window target mask map of each building facade;
s32: performing binarization processing on the window target mask map of each building facade, and then extracting the outline of each window of each building facade by using a contrast algorithm;
s33: extracting pixel coordinates of vertices of each window of each building facade based on the contour of each window of each building facade;
world coordinate unit: the world coordinate of each window position of each building facade of the target building group is obtained through coordinate conversion based on the pixel coordinate of each window position of each building facade of the target building group.
7. An electronic device, the electronic device comprising: the device comprises a shell, a processor, a memory, a circuit board and a power circuit, wherein the circuit board is arranged in a space surrounded by the shell, and the processor and the memory are arranged on the circuit board; a power supply circuit for supplying power to each circuit or device of the electronic apparatus; the memory is used for storing executable program codes; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory for performing the street view picture based building group facade window positioning method according to any one of the preceding claims 1 to 5.
8. A computer readable storage medium storing one or more programs executable by one or more processors to implement the street view picture based building group facade window positioning method of any one of the preceding claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311370849.3A CN117115243B (en) | 2023-10-23 | 2023-10-23 | Building group outer facade window positioning method and device based on street view picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311370849.3A CN117115243B (en) | 2023-10-23 | 2023-10-23 | Building group outer facade window positioning method and device based on street view picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117115243A CN117115243A (en) | 2023-11-24 |
| CN117115243B true CN117115243B (en) | 2024-02-09 |
Family
ID=88809500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311370849.3A Active CN117115243B (en) | 2023-10-23 | 2023-10-23 | Building group outer facade window positioning method and device based on street view picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117115243B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117931979B (en) * | 2024-03-22 | 2024-07-05 | 腾讯科技（深圳）有限公司 | Building display method and related device in electronic map |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101763656A (en) * | 2010-01-28 | 2010-06-30 | 北京航空航天大学 | Construction and display control method for floor and house division model of three-dimensional urban building |
| CN110956196A (en) * | 2019-10-11 | 2020-04-03 | 东南大学 | Automatic recognition method for window-wall ratio of urban building |
| WO2020192354A1 (en) * | 2019-03-28 | 2020-10-01 | 东南大学 | Blended urban design scene simulation method and system |
| CN112507444A (en) * | 2021-02-03 | 2021-03-16 | 四川见山科技有限责任公司 | AI-construction-based digital urban building night scene generation method and system |
| CN115482355A (en) * | 2022-09-21 | 2022-12-16 | 南京国图信息产业有限公司 | Many-source data driven LOD 2-level city building model enhanced modeling algorithm |
| CN115739438A (en) * | 2022-12-12 | 2023-03-07 | 中大智能科技股份有限公司 | Unmanned aerial vehicle-based method and system for repairing appearance of outer facade of building |
| CN115841542A (en) * | 2022-10-10 | 2023-03-24 | 阿里巴巴（中国）有限公司 | Building modeling method and device, computer equipment and storage medium |
-
2023
- 2023-10-23 CN CN202311370849.3A patent/CN117115243B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101763656A (en) * | 2010-01-28 | 2010-06-30 | 北京航空航天大学 | Construction and display control method for floor and house division model of three-dimensional urban building |
| WO2020192354A1 (en) * | 2019-03-28 | 2020-10-01 | 东南大学 | Blended urban design scene simulation method and system |
| CN110956196A (en) * | 2019-10-11 | 2020-04-03 | 东南大学 | Automatic recognition method for window-wall ratio of urban building |
| CN112507444A (en) * | 2021-02-03 | 2021-03-16 | 四川见山科技有限责任公司 | AI-construction-based digital urban building night scene generation method and system |
| CN115482355A (en) * | 2022-09-21 | 2022-12-16 | 南京国图信息产业有限公司 | Many-source data driven LOD 2-level city building model enhanced modeling algorithm |
| CN115841542A (en) * | 2022-10-10 | 2023-03-24 | 阿里巴巴（中国）有限公司 | Building modeling method and device, computer equipment and storage medium |
| CN115739438A (en) * | 2022-12-12 | 2023-03-07 | 中大智能科技股份有限公司 | Unmanned aerial vehicle-based method and system for repairing appearance of outer facade of building |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117115243A (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9942535B2 (en) | Method for 3D scene structure modeling and camera registration from single image | |
| EP1242966B1 (en) | Spherical rectification of image pairs | |
| US8787700B1 (en) | Automatic pose estimation from uncalibrated unordered spherical panoramas | |
| Xie et al. | Study on construction of 3D building based on UAV images | |
| CN109255808B (en) | Building texture extraction method and device based on oblique images | |
| CN117115243B (en) | Building group outer facade window positioning method and device based on street view picture | |
| CN112489099B (en) | Point cloud registration method and device, storage medium and electronic equipment | |
| KR100904078B1 (en) | A system and a method for generating 3-dimensional spatial information using aerial photographs of image matching | |
| CN108629742B (en) | True ortho image shadow detection and compensation method, device and storage medium | |
| CN113034347B (en) | Oblique photography image processing method, device, processing equipment and storage medium | |
| Yoo et al. | True orthoimage generation by mutual recovery of occlusion areas | |
| CN116106904B (en) | Facility deformation monitoring method and facility deformation monitoring equipment for object MT-InSAR | |
| Deng et al. | Automatic true orthophoto generation based on three-dimensional building model using multiview urban aerial images | |
| Ma et al. | Low‐Altitude Photogrammetry and Remote Sensing in UAV for Improving Mapping Accuracy | |
| CN114140593B (en) | Digital earth and panorama fusion display method and device | |
| CN117095098A (en) | Image rendering method, device, electronic equipment and computer readable storage medium | |
| CN115131504A (en) | Multi-person three-dimensional reconstruction method under wide-field-of-view large scene | |
| Wu et al. | Building Facade Reconstruction Using Crowd-Sourced Photos and Two-Dimensional Maps | |
| CN113642395B (en) | Building scene structure extraction method for city augmented reality information labeling | |
| Fan et al. | Pano2Geo: An efficient and robust building height estimation model using street-view panoramas | |
| CN116805277B (en) | Video monitoring target node pixel coordinate conversion method and system | |
| Zhou et al. | Object detection and spatial location method for monocular camera based on 3D virtual geographical scene | |
| CN118097339B (en) | Deep learning sample enhancement method and device based on low-altitude photogrammetry | |
| CN113870365B (en) | Camera calibration method, device, equipment and storage medium | |
| CN112767469B (en) | Highly intelligent acquisition method for urban mass buildings |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |