CN117095300A

CN117095300A - Building image processing method, device, computer equipment and storage medium

Info

Publication number: CN117095300A
Application number: CN202311355277.1A
Authority: CN
Inventors: 张云东; 尚鸿; 孙钟前
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-10-19
Filing date: 2023-10-19
Publication date: 2023-11-21
Anticipated expiration: 2043-10-19
Also published as: CN117095300B

Abstract

The present application relates to a building image processing method, apparatus, computer device, storage medium and computer program product. The method comprises the following steps: acquiring a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation; detecting a facade unit based on a front view of the building facade, and generating a building semantic graph according to a detection result; each elevation unit is marked in the building semantic graph; grouping all the elevation units in the building semantic graph according to columns to obtain a plurality of column groups; and carrying out intra-column regularization treatment on the vertical face units belonging to the same column group, and carrying out inter-column regularization treatment on different column groups to obtain a regularized building vertical face layout. By adopting the method, more regular and realistic elevation units can be obtained, so that the finally obtained regular building elevation layout is closer to the realistic building law, and the real situation of the building can be better reflected.

Description

Building image processing method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to a building image processing method, apparatus, computer device, storage medium, and computer program product.

Background

Building facade restoration refers to extracting facade units from a building picture to obtain the facade layout of a building. By further analyzing the facade layout of the obtained building, the structure of one building can be known, so that the building facade restoration has wide application in urban design, efficiency estimation, building use estimation, digital twinning and other applications. For how to obtain a facade layout closer to the reality building law based on building pictures, the method also becomes a research direction of comparison hotspots.

At present, in the process of restoring the building elevation based on the building picture, a template matching mode is mostly adopted, namely, a user inputs the acquired building picture into a neural network model, the neural network model determines the simplified expression of the elevation layout, a layout template which is most matched with the simplified expression of the elevation layout is predicted based on the simplified expression of the elevation layout, and the predicted most matched layout template is used as the final elevation layout of the building.

However, in this way, the reduction degree and the degree of freedom are limited due to the small number of layout templates, and the generalization and expansion are poor, so that the reduction effect of the final building elevation is poor.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a building image processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can obtain a building elevation layout that more closely approximates a realistic building law and better reflects the reality of a building.

In a first aspect, the present application provides a building image processing method. The method comprises the following steps:

acquiring a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation;

detecting a facade unit based on a front view of a building facade, and generating a building semantic graph according to a detection result; marking each elevation unit in the building semantic graph;

grouping all the elevation units in the building semantic graph according to columns to obtain a plurality of column groups;

and carrying out intra-column regularization treatment on the vertical face units belonging to the same column group, and carrying out inter-column regularization treatment on different column groups to obtain a regularized building vertical face layout.

In a second aspect, the application also provides a building image processing device. The device comprises:

the correction module is used for acquiring a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation;

The detection module is used for detecting the elevation unit based on the elevation of the building elevation and generating a building semantic graph according to the detection result; marking each elevation unit in the building semantic graph;

the grouping module is used for grouping all the elevation units in the building semantic graph according to columns to obtain a plurality of column groups;

and the regularization module is used for carrying out intra-column regularization treatment on the facade units belonging to the same column group and carrying out inter-column regularization treatment on different column groups to obtain a regularized building facade layout.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the building image processing method described above when the processor executes the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the building image processing method described above.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when being executed by a processor, implements the steps of the building image processing method described above.

The building image processing method, the device, the computer equipment, the storage medium and the computer program product are used for correcting the building elevation to obtain the elevation of the building elevation, and detecting the elevation units based on the elevation of the building elevation to generate the building semantic graph marked with the elevation units. And then vertical face units in the building semantic graph are grouped according to columns to obtain a plurality of column groups. For a plurality of obtained column groups, intra-column regularization processing is performed on each vertical face unit belonging to the same column group, inter-column regularization processing is performed on different column groups, so that the influence of noise and shielding in a building vertical face image can be reduced, more regular and realistic vertical face units are obtained, the finally obtained regularized building vertical face layout image is closer to the realistic building rule, and the real situation of a building can be reflected better.

Drawings

FIG. 1 is an application environment diagram of a building image processing method in one embodiment;

FIG. 2 is a flow chart of a method of processing a building image in one embodiment;

FIG. 3 is a schematic diagram of a perspective distortion correction step in one embodiment;

FIG. 4 is a schematic diagram of a perspective distortion correction step in another embodiment;

FIG. 5 is a schematic diagram of steps for determining a chunking line in one embodiment;

FIG. 6 is a schematic diagram of a regularization process step in one embodiment;

FIG. 7 is a schematic diagram of an intra-column normalized front-to-back column group layout in one embodiment;

FIG. 8 is a schematic diagram of an inter-column normalized front-to-back column group layout in one embodiment;

fig. 9 is a schematic structural diagram of matrix decomposition for stitched pictures in an embodiment:

FIG. 10 is a flow diagram of a method of building image processing in one embodiment;

FIG. 11 is a block diagram showing the construction of a construction image processing apparatus in one embodiment;

fig. 12 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The building image processing method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 and the server 104 may be used alone to perform the building image processing method in the present application, and the terminal 102 and the server 104 may be used cooperatively to perform the building image processing method in the present application. Taking the example of the present application being cooperatively executed by the terminal 102 and the server 104, when the building image processing is specifically performed, the user can acquire a building elevation through the terminal 102 and send the building elevation to the server 104. The server 104 may obtain the building elevation sent by the terminal 102, and correct the building elevation to obtain a front view of the building elevation; the server 104 performs elevation unit detection based on the elevation of the building elevation, and generates a building semantic graph according to the detection result; marking each elevation unit in the building semantic graph; the server 104 groups the units of each vertical face in the building semantic graph according to columns to obtain a plurality of column groups; the server 104 performs intra-column regularization on the facade units belonging to the same column group, and performs inter-column regularization on different column groups to obtain a regularized building facade layout.

The terminal 102 may be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, internet of things devices, portable wearable devices, intelligent voice interaction devices, intelligent home appliances, vehicle terminals, aircrafts, etc. The internet of things equipment can be an intelligent sound box, an intelligent television, an intelligent air conditioner, intelligent vehicle-mounted equipment and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers. The server can be realized by an independent server or a server cluster formed by a plurality of servers, and can also be a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication.

The embodiment of the invention can be applied to various scenes, including but not limited to cloud technology, artificial intelligence, intelligent transportation, auxiliary driving and the like.

Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Before describing the present application in detail, some of the terms involved will be explained:

building facade: the building can be roughly divided into three parts, namely a building roof, a building elevation and a building bottom layer; the building elevation can be abbreviated as elevation, and is a main structure of the building.

Elevation unit: refers to structural units for constructing building facades such as windows, balconies, cladding etc. on building facades.

Vertical face layout: the combination of all facade elements on a building facade is called a facade layout.

The building image processing method of the present application is described in detail below:

in an exemplary embodiment, as shown in fig. 2, a building image processing method is provided, which is described as an example of application to a computer device (the computer device may be specifically the terminal 102 or the server 104 in fig. 1), and includes the following steps:

step 202, obtaining a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation.

The building elevation is an image including a building elevation, and may be an image obtained when a building is photographed. The building elevation view can be obtained by shooting or collecting the building through a camera or other image collecting equipment, and can be a street view image, a trip image and the like.

It will be appreciated that due to the different imaging principles of the camera and the visual principles of the human eye, there may be problems with perspective distortion in the resulting original building elevation. Perspective distortion refers to the problem of image distortion in photographic or visual applications due to the projected relationship between the three-dimensional world and the two-dimensional image. Therefore, in order to improve the processing accuracy, the building elevation view can be corrected, and then the next processing is performed based on the corrected image.

The front view of the building elevation is an image formed by orthographic projection of an image of the building elevation from the front to the rear. In this embodiment, the front view of the building facade is an image obtained by correcting the building facade, which may also be referred to as a parallel front view.

In particular, the computer device may use the image homography transformation to transform the building elevation into a parallel elevation view that includes only elevation by selecting a predetermined number (e.g., 4) of correction points in the building elevation.

In some embodiments, the computer device may obtain the building elevation view photographed and uploaded by the user based on the terminal, determine the object to be corrected in the building elevation view, and further perform geometric transformation on the object to be corrected to obtain a front view of the building elevation.

In some embodiments, the object to be corrected refers to a target object or area in the building elevation that is to be corrected, such as a building elevation, or a portion of a building elevation, such as a window, balcony, and cladding, etc.

Wherein, the correction is to correct perspective distortion aiming at the problem of perspective distortion. Perspective distortion correction is a processing method for correcting perspective distortion caused by shooting angles and the like, and corrects a distorted part in an image by applying geometric transformation so that the distorted part is more realistic and visual.

In some embodiments, the object to be rectified is part of a building elevation, in particular an image taken from a front view of the building elevation. For example, a 3-5 floor building facade area can be selected, the left and right boundary lines of the area are parallel to the left and right boundaries of the building, and the upper and lower boundary lines of the area are parallel to the upper and lower boundaries of the window balcony.

In some embodiments, correcting the building elevation results in a front view of the building elevation, comprising: intercepting an object to be corrected from a building elevation view; determining a correction transformation matrix based on the object to be corrected; and performing perspective distortion correction on the object to be corrected according to the correction transformation matrix to obtain a front view of the building elevation.

Specifically, after obtaining the building elevation, the computer device determines the left and right boundaries and the upper and lower boundaries of the building elevation in the building elevation, intercepts the object to be corrected from the original building image based on the left and right boundaries and the upper and lower boundaries, obtains the coordinates of the correction points in the object to be corrected, determines the correction transformation matrix from the building elevation to the front view of the obtained building elevation after correction based on the coordinates of the correction points, and transforms the object to be corrected according to the determined correction transformation matrix to obtain the front view of the building elevation.

In some embodiments, the computer device may be configured to intercept the image to be rectified from the building elevation based on coordinates of the correction points, which may be set by the user, or adaptively set by the computer device in conjunction with actual geometric parameters of the building elevation, such as width, height, etc., without limitation.

In some embodiments, the correction point may be a target pixel point selected from among the pixel points in the building elevation view. For example, the selected target pixel points can be four vertexes of the building elevation view; or the pixel point and the vertex point of the midpoint of the left boundary line and the right boundary line of the building elevation view are determined together; the pixel point and the vertex point of the midpoint of the upper boundary line and the lower boundary line of the building image can be determined together; the selection of specific correction points can be adaptively adjusted according to the actual condition of the building elevation and the application requirements of users.

In some embodiments, as shown in fig. 3, a front view of a building facade is obtained by correcting the building facade. As can be seen from fig. 3, the image to be corrected is selected by selecting 1, 2, 3 and 4 correction points for the building elevation, and combining the correction points. The computer device may further correct perspective distortion in the building elevation view resulting in an elevation view of the building elevation. Through the front view of the building elevation that obtains, can draw the facade unit of building better, and then when carrying out regularization processing based on the facade monomer that draws, can improve the accuracy to the regularization processing of building elevation map for the building elevation layout map that obtains is more true accurate.

In some embodiments, as shown in fig. 4, fig. 4 is a schematic diagram illustrating correction for an elevation view of a building in one embodiment:

the computer equipment intercepts the image to be corrected from the building elevation by selecting an area, and particularly can be a trapezoid image obtained by interception. The truncated trapezoid image is corrected into a regular rectangular image, namely a front view of the building elevation, through perspective transformation.

The perspective transformation is specifically as follows:

The homogeneous coordinates of the four vertices of the trapezoid image shown in FIG. 4 are respectivelyI.e. +.>、/>、/>And +.>Wherein->. The homogeneous coordinates of the four vertexes of the conventional rectangular image after the correction can be preset and respectively are thatI.e. +.>、/>、/>And +.>Let->。

The computer device may calculate a correction transformation matrix based on the homogeneous coordinates of the four vertices of the trapezoid image and the homogeneous coordinates of the four vertices of the preset corrected regular rectangular image in combination with a general transformation formula. The computer equipment can specifically substitute each homogeneous coordinate into the following general transformation formula of perspective transformation, and solve the general transformation formula of perspective transformation to obtain a correction transformation matrix:

the general transformation formula for perspective transformation is:

wherein,is the coordinates on the trapezoid image, +.>Is the coordinates of a conventional rectangular image,is a transformation matrix. Thus, the computer device can bring the four vertex coordinates of the trapezoid image and the four vertex coordinates of the final transformed image into the transformation formula, resulting in the following 8 equations:

solving the equations in parallel can obtain the correction transformation matrix. After the correction transformation matrix is obtained, the computer device can acquire the coordinate value of each pixel point on the trapezoid image, substitutes the acquired coordinate value of each pixel point into a general transformation formula of the known correction transformation matrix, and solves the coordinate value, so as to acquire the corresponding pixel value of each point on the regular rectangle image from the trapezoid image. So that the computer equipment can complete perspective transformation (namely perspective distortion correction) of the object to be corrected, and a front view of the building elevation is obtained.

Step 204, detecting elevation units based on the elevation of the building elevation, and generating a building semantic graph according to the detection result; each facade element is marked in the building semantic graph.

The building semantic graph is an image for describing or marking each identified elevation unit by adopting natural language or symbolic language and the like.

Specifically, the computer device may perform elevation unit detection on a front view of the building elevation, and identify each elevation unit in the front view, and attribute information corresponding to each elevation unit, such as size, position, and category of the elevation unit. The computer equipment can further generate a building semantic graph according to each facade unit identified by the target detection and the attribute information corresponding to each facade unit.

In some embodiments, the computer device may be developed based on a pre-trained facade element detection model or other algorithm that may be used for target detection when performing facade element detection.

In some embodiments, the facade element detection model can be a network model built based on artificial intelligence algorithms. When the facade unit detection model is constructed, the construction can be performed based on a plurality of algorithms such as a supervised learning algorithm, an unsupervised learning algorithm and the like. When the facade unit detection model is constructed based on the supervised learning algorithm, the facade unit detection model can be constructed based on various target detection models such as DETR (DEtection Transformer, end-to-end target detection network), faster-RCNN (Faster Regions with CNN features, quick target detection algorithm), YOLO (YouOnly Look Once, one target detection algorithm) and the like.

In some embodiments, the facade element detection model is based on YOLO training. In training the facade element detection model based on YOLO, the training step may include: acquiring labeling information corresponding to a sample building elevation view and a sample building elevation view; carrying out data enhancement processing on the sample building elevation to obtain a sample building elevation after the data enhancement processing; based on the sample building elevation graph and the labeling information after the data enhancement processing, the elevation unit detection model to be trained is updated and then continues to be trained until the training is finished, and the elevation unit detection model is obtained.

Wherein the sample building facade atlas is a picture set related to a building image obtained from a plurality of published data sources. The disclosed data sources may include, for example, facandWHO (street view data annotated semantic wall model training set), COCO (Common Objects in Context, image recognition data set), ECP (object detection data set), and the like.

The labeling information is information obtained after labeling the object to be identified in the sample building image set. The labeling information can generally comprise the coordinates (such as the abscissa, the ordinate, the width, the height and the like) of the boundary box of the object to be identified, corresponding class labels and the like, and can be automatically generated by a special labeling tool or manually labeled.

The data enhancement processing is to perform various strategies on the sample building images in the sample building image set, for example, random color transformation, image contrast transformation, exposure transformation and random rotation can be included, so that the model can better cope with complex shooting conditions in reality.

Specifically, the computer device obtains sample building elevation views from a plurality of data sources, obtains a sample building elevation view set, and marks the sample building elevation views to obtain marking information. And further performing data enhancement processing on the sample building elevation graph, and then performing unfolding training to obtain an elevation unit detection model. The generalization of the model can be effectively enhanced by carrying out data enhancement on the sample building elevation in the training process. When elevation unit detection is carried out on the elevation of the building elevation in the follow-up process, the trained elevation unit detection model with strong generalization can be unfolded, and the accuracy of target detection can be improved.

In some embodiments, the computer device may tag all identified categories of facade elements to obtain a building semantic map. Of course, the computer device may also select only a part of the category of facade units for marking according to the actual scene requirement. For example, when the identified facade unit includes a window, a balcony, and an air conditioner, the computer device may simply mark the window, the balcony, to obtain a building semantic map; of course, the computer device may also use other types of facade units or combinations of facade units to make a building semantic graph, which is not described herein. By only selecting part of the elevation units for marking, the computing resource can be saved and the building data processing efficiency of the computer equipment can be improved on the basis that the effect of the final building elevation layout is not affected.

And 206, grouping the elevation units in the building semantic graph according to columns to obtain a plurality of column groups.

The column groups are composed of vertical face units in the building semantic graph, the types of the vertical face units included in the same column group are the same, and the types of the vertical face units included in different column groups can be the same or different. The facade elements, which are typically located in the same column, can be considered to be in the same column group.

Specifically, the computer device determines that the facade units in the same column are identical in combination with the layout priori of the real building, so that the computer device can group the facade units in the building semantic graph according to the columns to obtain a plurality of column groups. In connection with the actual situation of the building elevation, the obtained array may comprise an array of windows, an array of balconies, etc.

In some embodiments, due to the extensive noise, occlusion, etc. problems in the initially obtained building elevation map, irregular elevation units marked in the building semantic map may exist, i.e. the types of elevation units belonging to the same column group are the same, but there may be problems of inconsistent or misaligned elevation unit sizes in part or all.

In some embodiments, after grouping by column for each facade element in the building semantic graph, a 1, 2..N group of columns may be obtained, e.g., group 1, 2, 3 may be a column group of window-like facade elements and group 4 may be a column group of balcony-like facade elements.

Step 208, intra-column regularization is performed on the facade units belonging to the same column group, and inter-column regularization is performed on different column groups, so as to obtain a regularized building facade layout.

The regularization treatment is a treatment process of rearranging objects to be regularized, so that the obtained building elevation layout is more regular and the reality is restored. The objects to be regularized are e.g. facade elements, column groups etc. The regularization process may specifically refer to normalization process, by which the finally obtained regularized building elevation layout may be made closer to the elevation layout of the actual building law.

The in-column regularization processing is to normalize all the facade units belonging to the same column, so that all the facade units in the same column have consistency, and particularly, all the facade units in the same column group can be kept consistent, for example, the sizes of all the facade units are consistent, and the alignment effect is shown in the vertical direction.

The inter-column regularization processing refers to normalization post-processing for different column groups, so that the column groups subjected to normalization processing have consistency, and particularly can be that the vertical face units in different column groups are consistent in size and have alignment effects in the horizontal direction and the vertical direction.

The building elevation layout is an elevation layout formed by elevation units which are regularly and consistently arranged. The building elevation layout is closer to the reality building law, so that the building elevation layout can facilitate downstream analysis, such as helping to analyze layout design of reality buildings, building restoration in game making, construction analysis in city planning, simulation in automatic driving and the like.

Specifically, the computer device performs in-column regularization processing for each column group based on the obtained plurality of column groups, that is, normalizes each facade unit in each column group, respectively, to obtain an in-column regularization processing result for each column group. On the basis of obtaining the intra-column regularization result, the computer equipment performs inter-column regularization on each column group to obtain a final building elevation layout. Of course, the computer device may just perform the intra-column regularization processing, and then directly obtain the building elevation layout based on the intra-column regularization processing result. The computer device may also synchronize the in-column regularization process with the inter-column regularization process, without limitation.

In some embodiments, after the in-column regularization processing is performed on the building elevation unit, the computer device may replace a corresponding column group in the building semantic graph according to the in-column regularization processing result of each column, so as to obtain a new building semantic graph, where the new building semantic graph may be directly used as a building elevation layout graph.

In some embodiments, the computer device may also temporarily not update the building semantic graph after in-line regularization of the building facade elements. And after the computer equipment performs inter-column regularization processing on different column groups to obtain inter-column regularization processing results, uniformly combining the intra-column regularization processing results and the inter-column regularization processing results, and updating the building semantic graph to obtain a regularized building elevation layout graph.

In the building image processing method, the building elevation view is corrected to obtain the elevation view of the building elevation, elevation unit detection is carried out based on the elevation view of the building elevation, and the building semantic view marked with each elevation unit is generated. And then vertical face units in the building semantic graph are grouped according to columns to obtain a plurality of column groups. For a plurality of obtained column groups, intra-column regularization processing is performed on each vertical face unit belonging to the same column group, inter-column regularization processing is performed on different column groups, so that the influence of noise and shielding in a building vertical face image can be reduced, more regular and realistic vertical face units are obtained, the finally obtained regularized building vertical face layout image is closer to the realistic building rule, and the real situation of a building can be reflected better.

In some embodiments, facade element detection is performed based on a front view of a building facade, and a building semantic graph is generated according to detection results, including: identifying each facade unit in the front view of the building facade and the category to which each facade unit belongs; generating an initial semantic graph which is equivalent to a front view of a building elevation; for any category, marking the facade units belonging to the category by the identification matched with the category in the initial semantic graph to obtain the building semantic graph.

The initial semantic graph is the same as the front view of the building elevation, and the initial semantic graph can be an image which is consistent with the front view of the building elevation and blank.

The marks are marks, symbols or markers used for describing the facade units, and the marks can be used for distinguishing different types of facade units, namely, the marks matched with the facade units can be corresponding to the facade units of different types.

Specifically, the computer device may input a front view of the building facade to a pre-trained facade element detection model, and identify each facade element in the front view of the building facade and a category to which each facade element belongs from the facade element detection model. Further, the computer equipment generates an initial semantic graph which is consistent with the front view of the building elevation in size and blank, marks the elevation units in the initial semantic graph based on the identification matched with each elevation unit, and therefore the building semantic graph is obtained after the marking of each elevation unit in the initial semantic graph is completed.

In some embodiments, when determining the identity of the facade element match, the computer device may query for the identity of the facade element matching the category of the facade element directly according to the category of the facade element. The computer device may store the association relationship between the category of the facade unit and the identifier in advance. And the computer equipment can directly develop the query based on the association relationship when determining the identification aiming at a certain type.

In some embodiments, elements that may represent facade elements, such as geometric figures, text, mathematical symbols, etc., are identified as classes. For example, when identified as a geometric figure, the geometric figure may be a circular, oval, triangular, or rectangular shape, among others. For a closer fit to the facade elements of a conventional building facade, a rectangle may be used as an identification.

In some embodiments, the computer device may employ rectangles as identifiers to tag facade elements belonging to different types in the initial semantic graph. For different categories of facade elements, the color of the rectangle may differ, and the color may correspond to the pixel value, i.e. the pixel value is different for different categories of facade elements.

In some embodiments, the computer device may further mark other areas in the initial semantic graph, except for the facade unit, where the pixel values of the other areas are different from the pixel values of the area where the facade unit is located, so as to facilitate subsequent regularization operations. For example, the computer device may mark the area of the facade element that is a window with a blue rectangle; aiming at the area of which the elevation unit is a balcony, the area is marked by adopting an orange rectangle, and other areas are marked by adopting black.

In the above embodiment, the computer device marks the facade units belonging to the aimed category based on the identification matched with the aimed category in the initial semantic map to obtain the building semantic map, so as to determine the initial layout of the building facade and lay a foundation for the subsequent regularization processing. And regularization processing is carried out on the basis of the building semantic graph, so that the influence of noise and shielding in the image is reduced, and the finally obtained building elevation layout graph is more regular and is reduced to reality.

In some embodiments, grouping the facade elements in the building semantic graph by columns results in a plurality of column groups, including: determining a plurality of block lines in the building semantic graph; based on any two adjacent block lines in the plurality of block lines to form sub-areas, each sub-area is cut out from the building semantic graph to obtain a plurality of column groups.

The block lines are determined vertical lines for cutting the building semantic graph. The subarea is an area formed by facade units on the building semantic graph, and the subarea can be determined by two adjacent block lines.

Specifically, the computer device may determine a plurality of block lines according to the block line characteristics required to be provided by the set block line, and form a sub-area according to the criterion that two adjacent block lines may form a region based on the determined plurality of block lines, and further cut the formed sub-area, thereby obtaining a plurality of column groups.

In some embodiments, the set chunking line characteristics may be: the blocking lines exist between adjacent column units; there is a certain distance between the segment lines. The distance may be set in units of pixels, such as 5 pixels, 10 pixels, 15 pixels, and the like; the distance may also be set directly in terms of length measurement units, such as 1 cm, 0.5 cm, etc. The distance may be set adaptively in combination with the actual structure of the building elevation, the size of the building elevation, and the like.

In some embodiments, the computer device may also determine the boundary of the building semantic graph itself as a partition line, specifically may use the left and right boundaries of the building semantic graph as partition lines, and further combine with other partition lines except the left and right boundaries to form the sub-region.

In some embodiments, the block lines determined by the computer device include a plurality of block lines such as block line a, block line b, block line c. The computer device obtains column group 1, column group 2.

In the above embodiment, the computer device may form the sub-area based on the determined two adjacent column block lines, and cut the sub-area to obtain a plurality of column groups, so that the following may use the column groups as units, and perform expansion processing according to the column groups and the column groups to obtain more regular and realistic elevation units, so that the finally obtained regularized building elevation layout is closer to the realistic building rule, and the real situation of the building can be better reflected.

In some embodiments, to determine the block lines, as shown in fig. 5, determining a plurality of block lines in a building semantic graph includes:

Step 502, determining a plurality of vertical lines in the building semantic graph in the current iteration process.

The vertical line is a straight line which is determined from the building semantic graph and is in a vertical state.

Specifically, the computer device may determine a plurality of vertical lines in the building semantic graph in the current iteration process by combining geometric parameters of the building semantic graph, the situation of the block line determined in the iteration process before the current iteration process, and the like.

In some embodiments, the computer device may enumerate a plurality of vertical lines in the current iteration process based on the width of the building semantic graph, so long as the enumerated vertical lines satisfy the following formula:

wherein,for the width of the building semantic graph, < >>The vertical line at x=i, i may refer to a coordinate value of the vertical line in a horizontal direction, and may be used to represent a distance between the determined vertical line and a left boundary line of the building semantic graph in the horizontal direction, or a distance between the determined vertical line and a right boundary line of the building semantic graph in the horizontal direction. Wherein, the range of i can be set to be more than 0 and less than or equal to the width of the building semantic graph. Of course, the range of i may also be set to be greater than 1 and less than the width of the building semantic graph. The specific value range of i can be adaptively adjusted by combining the structure of the building elevation diagram, the characteristics of the vertical lines to be enumerated and the like.

In some embodiments, the computer device, when determining the plurality of vertical lines in the current iteration process, may determine from the vertical lines determined in the last iteration process of the current iteration process. For example, the computer device may take the remaining vertical lines that were not determined to be patch lines in the last iteration process directly as the vertical lines in the current iteration process; for another example, the computer device may also obtain the vertical line in the current iteration process after adding a portion of the new vertical line, or after subtracting a portion of the vertical line, based on the remaining vertical line. The computer device may still determine a vertical line within the width of the building semantic graph based on the width of the building semantic graph as new vertical lines are added.

Step 504, traversing a plurality of vertical lines, and determining a score of the traversed vertical line according to a vertical line before the traversed vertical line and the current block line set for any traversed vertical line.

The block line set is used for storing the determined block lines, and the block lines determined in the iteration process before the current iteration process are stored in the block line set. When the current iteration process is the first iteration, that is, when the current iteration process does not exist in the previous iteration process, the number of the partition lines recorded in the partition line set is 0.

The score of a vertical line is the score when the vertical line is taken as a patch line, the higher the score of the vertical line means that it is more likely to be a patch line, and the lower the score of the vertical line means that it is less likely to be a patch line.

Specifically, the computer device may sequentially process the plurality of vertical lines according to a preset traversal sequence, obtain a score of each vertical line, and determine the partition line based on the obtained score of each vertical line. The preset traversal sequence can be a sequence which takes the left boundary of the building semantic graph as a starting point and the right boundary of the building semantic graph as an end point and is arranged from left to right; the preset traversal sequence can also be a sequence which takes the right boundary of the building semantic graph as a starting point and the left boundary of the building semantic graph as an end point and is arranged from right to left; the preset traversal order may also be an order set based on the characteristics of the vertical line. The specific traversing sequence can be adaptively set in combination with the actual situation of the block lines and the like.

And step 506, determining a target vertical line with scores meeting preset conditions from the plurality of vertical lines, taking the target vertical line as a blocking line and adding the blocking line to the blocking line set.

The preset condition is a set condition for judging a score of the vertical line to determine whether the vertical line corresponding to the score can be determined as a target vertical line.

Specifically, the computer device may compare the score of each vertical line with a preset condition, determine whether the score of each vertical line satisfies the preset condition, determine the score satisfying the preset condition as a target vertical line, and add the target vertical line as a partition line to the partition line set. The number of the block lines determined in the iterative process can be only one or a plurality of, and the specific number is related to the preset condition.

In some embodiments, the preset condition may be a set score threshold, i.e. may be determined as a chunking line as long as the score reaches the score threshold. For example, the score threshold may be set to 90 minutes, 95 minutes, or the like, and when the score threshold is set to 90 minutes, the vertical lines having a score of 90 minutes may be all taken as the patch lines.

In other embodiments, the preset condition may be that the highest score in the scores of all the vertical lines is determined to be the block line, and the computer device may compare the scores of the vertical lines, determine the highest score in the scores of the vertical lines, and determine the vertical line corresponding to the highest score to be the block line.

And step 508, returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process, and continuing to execute until a preset iteration stop condition is reached, so as to obtain a plurality of block lines in the building semantic graph.

The preset iteration stopping condition is a set condition for stopping the iteration process, and can be determined by combining the calculated score of each vertical line and the score of the determined partition line in the partition line set; the preset iteration stop condition may also be preset iteration times, and when the iteration times reach the preset iteration times, it is determined that the preset iteration stop condition is met.

In particular, the computer device may repeat the iterative process to determine a patch line during each iteration, and stop the iteration when the computer device determines that a preset iteration stop condition is met. The computer device may use the partition lines in the set of partition lines determined in the iteration process preceding the current iteration process as a plurality of partition lines in the building semantic graph. The computer device may also use the partition line determined in the current iteration process and the partition line determined in the iteration process before the current iteration process together as a plurality of partition lines in the building semantic graph. The specific block lines can be determined by combining the scoring condition of the block lines and the actually determined quantity.

In the above embodiment, the computer device may calculate the score of each determined vertical line in each iterative process through an iterative manner, so as to determine the blocking line according to the score of each vertical line, so that the accuracy of determining the blocking line may be improved, and the column group obtained by clipping is more regular and reasonable, and the accuracy of building image processing is improved.

In some embodiments, determining the score of the traversed vertical line from the vertical line preceding the traversed vertical line and the current set of patch lines includes: determining a pixel difference between a vertical line preceding the traversed vertical line and the traversed vertical line; determining the distance between the traversed vertical line and each block line in the current block line set; a score of the traversed vertical line is determined based on the pixel differences, and a minimum in distance.

Wherein the pixel difference is a difference in pixel value between a traversed vertical line and a vertical line preceding the traversed vertical line. For each vertical line, a plurality of pixel points can be included, and the pixel difference refers to the difference of the pixel values of the pixel points.

Specifically, the computer device may take a plurality of current pixel points for the traversed vertical line and determine a previous pixel point that matches each current pixel point based on the previous vertical line of the traversed vertical line. And further obtaining a pixel value of each current pixel point, a pixel value of each previous pixel point, and determining a pixel difference based on the pixel value of the current pixel point and the pixel value of the previous pixel point matched with the current pixel point. Further, the computer device also calculates a distance between the traversed vertical line and each of the current set of patch lines, and finally determines a score for the traversed vertical line based on a product of the pixel difference and a minimum value in the distance.

In some embodiments, when determining the current pixel point, the computer device may determine each pixel point forming the vertical line as the current pixel point, so that the calculated pixel difference is more accurate.

In other embodiments, the computer device may also select a portion of the key pixels from all the pixels that form the vertical line as the current pixel. The key pixel points can be randomly selected according to the set quantity, and can also be selected pixel points capable of obviously expressing the characteristics of the vertical line. By processing only part of the pixel points, the calculated amount can be reduced, and the data processing efficiency of the computer equipment can be improved.

In some embodiments, a previous pixel that matches a current pixel may refer to a pixel that is on the same horizontal line as the previous pixel, i.e., different from the abscissa and the same ordinate.

In some embodiments, the computer device, when determining the pixel differences, may obtain a plurality of initial pixel differences based on the pixel value of each current pixel point and differences in pixel values of previous pixel points that match the current pixel point, and determine the pixel differences based on the initial pixel differences.

In some embodiments, the computer device may determine the pixel difference based on the mode, the median, or the coincidence of the pixel value distributions in the initial pixel difference, or the computer device may calculate the average value of the initial pixel difference, and the average value is used as the pixel difference, and the manner of specifically calculating the pixel difference may be adaptively selected in combination with the actual precision requirement, the computing resource, and the like.

In some embodiments, the score of any one vertical line is obtained according to the product of the minimum value of the distance and the pixel difference, wherein the pixel difference is the average difference of the initial pixel differences, and the specific calculation formula is as follows: wherein,，/>representing the average difference between the pixels in the ith column and the i-1 th column, h being the number of pixels, < >>Pixel value of pixel point representing the j-th point on the vertical line of the i-th column,/>A pixel value representing a pixel point of a j-th point on a vertical line of the i-1 th column.

The calculation formula of the minimum distance is as follows:

wherein,to step s (i.e., the current iterative process), all of the determined patch lines, k, are any of the determined patch lines.

Can be understood as +.>(i-th vertical line) and the distance of the nearest block line from all block lines determined in steps 0 to s-1 (i.e., the previous iteration of the current iteration). When a column of pixels jumps and there is no patch line near that column, that column will get a higher score, meaning that it is more likely to be a patch line.

In the above embodiment, the computer device may accurately calculate the score of each vertical line based on the pixel difference and the minimum distance, and may accurately determine the block line based on the score of the vertical line, so that the column group obtained by clipping is more regular and reasonable, and the accuracy of building image processing is improved.

In some embodiments, after the target vertical line is taken as the patch line and added to the set of patch lines, the building image processing method further comprises: determining the score corresponding to each block line in the current block line set; determining the difference between the maximum score in the scores corresponding to the block lines and the score of the target vertical line; and when the preset iteration stop condition is not met based on the difference, executing the step of returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process.

Specifically, the computer may determine a score of each partition line in the partition line set, select a maximum score from the scores, compare the score of the target vertical line with the maximum score, determine a difference between the two, determine whether a preset iteration stop condition is reached according to the difference, and set the preset iteration stop condition in a manner of calculating the difference. When the preset iteration stop condition is not reached, the computer equipment continues to return to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process. When the preset iteration stop condition has been reached, the computer device determines a plurality of chunking lines.

In some embodiments, the computer device may directly calculate the difference between the score of the target vertical line and the maximum score and determine whether a preset iteration stop condition is met with the difference as the difference. At this time, the preset iteration stop condition may be whether the difference value reaches a difference threshold value, and the difference threshold value may be set to a larger value such as 75, 80, etc. The larger the difference value between the two is, the lower the score of the target vertical line is, and when the score of the target vertical line is low to a certain degree, the difference value threshold is reached, and at the moment, the iteration process is not needed. When the difference between the score of the newly determined target vertical line and the maximum score is too large, the fact that all the block lines in the building semantic graph are determined completely is indicated, and no block line exists in the vertical lines determined in the current iteration process.

In some embodiments, the computer device may further directly compare the obtained score of the target vertical line with a maximum score of a preset multiple to obtain a comparison result, and use the comparison result as a difference. At this time, the preset iteration stop condition may be whether the score of the target vertical line is smaller than the maximum score of the preset multiple, the preset multiple may be set to 0.1, 0.15, and 0.2, and when the score of the target vertical line is smaller than the maximum score of the preset multiple, it indicates that the score of the target vertical line is very low, thereby meeting the preset iteration stop condition, and no iteration process is required.

In some embodiments, the computer device compares the scores of the target vertical linesf _s And maximum scoref _max If less than 0.1f _max The iteration is terminated. Otherwise the first set of parameters is selected,i.e. from scoref _s Andf _max the maximum score for the next comparison is determined and the step of returning to the determination of the plurality of vertical lines in the building semantic graph during the current iteration is performed.

In the above embodiment, the computer device determines the difference between the maximum score and the score of the target vertical line, and determines whether to stop iteration based on whether the preset iteration stop condition is satisfied by setting the preset iteration stop condition, so that on one hand, each column in the building semantic graph can be independently formed into a sub-region by the obtained partitioning line; on the other hand, the situation that the searching of the block lines is continued after the sufficient block lines are determined and the waste of calculation resources is avoided.

In some embodiments, as shown in fig. 6, the intra-column regularization is performed on the facade units belonging to the same column group, and the inter-column regularization is performed on different column groups, so as to obtain a regularized building facade layout, which includes the following steps:

step 602, performing intra-row normalization on the facade units in each row group to obtain each first middle row group.

The normalization in the columns is a processing procedure for normalizing the facade units in the column group so that the facade units in the same column group show consistency. The first middle column group is a column group obtained by performing intra-column normalization, and each vertical face unit in the first middle column group has consistency, specifically, the sizes of each vertical face unit in the first middle column group are consistent, and an alignment effect is shown in the vertical direction.

Specifically, the computer device may perform intra-column normalization on the facade units in each column group to obtain a first middle column group corresponding to each column group, where each facade unit in each column group is obtained to be more regular.

In some embodiments, as shown in fig. 7, a schematic diagram of the structure is shown before the in-column normalization for a certain column group and after the in-column normalization. As can be seen from fig. 7, before in-column normalization, the dimensions of each facade unit in the same column group are different and not aligned, after in-column normalization, the dimensions of each facade unit are the same and aligned, so that the overall facade layout of the building is more consistent with the real building, and the real situation of the building can be better reflected.

Step 604, determining at least one column group set based on the similarity between the first intermediate column groups; each column group includes a plurality of first intermediate column groups satisfying a preset similarity condition.

Wherein the degree of similarity is a parameter for describing the similarity between the first intermediate column groups. The preset similarity condition is a condition set for judging whether the first intermediate column group can perform inter-column normalization.

Specifically, the computer device may determine, for any two first intermediate column groups in each first intermediate column group, a degree of similarity between any two first intermediate column groups, and when the degree of similarity between any two first intermediate column groups satisfies a preset similarity condition, indicate that the degree of similarity between the two first intermediate column groups is higher, where the two first intermediate column groups may belong to the same column group set.

In some embodiments, the preset similarity condition is set according to a similarity threshold, and when the similarity between the first column groups reaches the similarity threshold, it is determined that the preset similarity condition is satisfied. The similarity threshold can be adaptively adjusted in combination with actual application scenes, precision requirements and the like, such as 0.9, 0.95 and the like which can be set.

Step 606, performing inter-column normalization on the column groups in each column group set to obtain a second intermediate column group.

The normalization between columns is a processing procedure of normalizing by taking column groups as a unit, so that column groups in the same column group set show consistency, the consistency can be shown that the vertical face units of all column groups in the same column group set have the same size, and all vertical face units are aligned in the horizontal and vertical directions.

Specifically, the computer device may perform inter-column normalization on each first intermediate column group in the same column group set, to obtain a second intermediate column group corresponding to each column group set respectively.

Step 608, obtaining a regularized building elevation layout corresponding to the building elevation from the column group not subjected to inter-column normalization in the first intermediate column group and the second intermediate column group.

Specifically, the column group in the first intermediate column group, for which inter-column normalization is not performed, is a column group that does not satisfy a preset similarity condition. The computer device may combine the results of the intra-column normalization with the results of the inter-column normalization to obtain a final regularized building facade layout.

In some embodiments, obtaining a regularized building elevation layout corresponding to the building elevation map from the set of columns in the first intermediate set of columns that are not inter-column normalized, and the second intermediate set of columns, includes: and replacing the corresponding column group in the building semantic graph by the column group which does not perform inter-column normalization in the first intermediate column group and the second intermediate column group to obtain a regularized building elevation layout graph corresponding to the building elevation graph.

Specifically, when the computer device replaces the column groups in the building semantic graph, the column groups matched with the column groups which are not subjected to regularization processing in the building semantic graph can be searched from the column groups which are not subjected to inter-column normalization in the first middle column group and the second middle column group, so that each column group in the replacing building semantic graph is subjected to replacement processing, and a regularized building elevation layout graph corresponding to the building elevation graph is obtained.

In the above embodiment, the computer device performs intra-row normalization on each of the facade units belonging to the same row group for the obtained plurality of row groups, to obtain each of the first intermediate row groups. And determining a column group set based on the similarity between the first middle column groups, and performing inter-column regularization processing on the column groups in the column group set, so that the influence of noise and shielding in the building elevation can be reduced, and a more regular and realistic elevation unit is obtained. The computer equipment combines the in-column normalization results and the inter-column normalization results to obtain a final regularized building elevation layout, and the final regularized building elevation layout is closer to the actual building rules, so that the real situation of the building can be reflected better.

In some embodiments, performing intra-column normalization on the facade elements in each column group to obtain each first middle column group includes: for any column group, generating a plurality of approximate column groups corresponding to the column group according to each facade unit included in the column group; screening out a target approximate column group from a plurality of approximate column groups, and taking a vertical face unit from which the target approximate column group is derived as a representative vertical face unit; the representative facade elements are replicated to obtain a first intermediate column group corresponding to the targeted column group.

Wherein, the approximate column group is a column group formed by arranging and combining vertical face units according to rows and columns. For any column group, its corresponding approximate column group can be obtained by multiplying the elevation cells in the column group by a column-wise block matrix (or vector) and a row-wise block matrix (or vector).

The target approximate column group is a column group determined from the approximate column group. For any column group, the target approximate column group may specifically be an approximate column group with a small difference from the column group.

The representative facade units are representative units in the same column group and can be used for replacing other facade units. For each column group there is one representative facade element.

Specifically, the computer device may perform row-column permutation and combination on each facade unit included in any column group to obtain a plurality of approximate column groups. The computer device can further calculate the column group difference between each approximate column group and the corresponding column group, and judge whether the column group difference meets the preset column group difference condition. The preset column group difference condition is a condition set for judging the calculated column group difference to determine whether or not an approximate column group corresponding to the column group difference can be determined as a target approximate column group. The preset column group difference conditions can be adaptively set in combination with the accuracy requirements of actual building image processing, actual application scenes and the like.

Further, the computer device may copy the representative facade units for each column group, that is, use the representative facade units, replace other facade units except the representative facade units in the column group to obtain a first middle column group corresponding to the column group.

In some embodiments, the computer device, when generating the approximated column group based on the facade elements, expands in particular according to the actual column arrangement of the column group for which it is intended. For example, when a certain vertical face unit in a column group is adopted to obtain an approximate column group, it is indicated that the vertical face unit needs to be arranged three times in the column direction and arranged once in the row direction, and the approximate column group can be obtained.

In some embodiments, the computer device may calculate the euclidean distance, cosine similarity, or pearson similarity, etc. between the approximated column group and the targeted column group to determine a column group difference between the column group and its corresponding approximated column group.

In some embodiments, the preset column group variance condition may be set based on a column group variance threshold; for any column group, when the column group difference between the column group and the corresponding approximate column group is smaller than or equal to the column group difference threshold, the approximate column group corresponding to the column group difference is determined as the target approximate column group. When a plurality of target approximate column groups are determined for the same column group, a vertical face unit corresponding to any one target approximate column group can be selected as a representative vertical face unit, and of course, a voting mechanism can be set, and one of the vertical face units corresponding to the plurality of target approximate column groups can be selected as the representative vertical face unit.

In other embodiments, the preset column group difference condition may be that the condition is satisfied when the column group difference is minimum. The computer device may determine, for any one of the column groups, a respective column group difference between the column group to which it is directed and the corresponding approximate column group, and determine the approximate column group corresponding to the smallest column group difference as the target approximate column group.

In some embodiments, a computer device may group columnsThe inner facade units are normalized. The computer device may be cycled->All units in->For any->Solving a block column matrix +.>And a blocking row matrix->So that the following conditions are satisfied:

wherein,to solve the result for an approximate set of raw sets of columns, the final computer device may takeCan find the column group +.>Representing units +.>Approximate result->。

In the above embodiment, the computer device may normalize the facade units in each column group, so as to obtain a representative facade unit of each column group, and duplicate the representative facade unit to obtain a first middle column group corresponding to the column group. Whereby the facade elements in the resulting first intermediate column group are regularly identical.

In some embodiments, generating a plurality of approximated column groups corresponding to the targeted column groups from the facade elements included in the targeted column groups includes: acquiring a first blocking matrix matched with the aimed column group; traversing each facade unit in the aimed column group, and determining an approximate column group corresponding to the aimed column group according to the traversed facade unit and the first block matrix; after the traversal is completed, a plurality of approximate column groups corresponding to the targeted column group are obtained.

Wherein the first blocking matrix is matched with the element composition of the column group in the horizontal direction and the element composition in the vertical direction. When the elements of the column group are vertical face units, the first block matrix is related to the vertical face unit composition in the horizontal direction and the vertical face unit composition in the vertical direction in the column group. Thus, the first blocking matrix will also be different for column groups where there are different elevational cell compositions.

The first blocking matrix comprises a first blocking row matrix and a first blocking column matrix, and the dimension, the size, the structure and the like of the first blocking row matrix depend on the number of rows of elements in the column group, namely the number of rows of the vertical face units in the column group; the dimensions, size, structure, etc. of the first segmented column matrix depend on the number of columns of elements in the column set, i.e. the number of columns of the facade elements in the column set.

Specifically, for any one column group, the computer equipment acquires a first blocking matrix matched with the aimed column group; for each facade element in the aimed column group, multiplying the facade element by the obtained first blocking matrix in turn to obtain a plurality of approximate column groups matched with the aimed column group.

In some embodiments, the first segmented row matrix and the first segmented column matrix are combined from a zero matrix and an identity matrix, and the number of the zero matrix and the identity matrix depends on the distribution of the planar units in the column group. The distribution may be specifically the number of columns and rows of the cells in the column group.

In some embodiments, the column group a is a column group formed by three vertical plane units, and the first blocking matrix matched with the column group comprises a first blocking row matrix and a first blocking column matrix, wherein the first blocking row matrix can be a matrix formed by combining a zero matrix and an identity matrix; the first blocking column matrix may be a matrix composed of three zero matrices and three identity matrices combined.

In some embodiments, the column group B is a column group formed by five vertical plane units, and the first blocking matrix matched with the column group includes a first blocking row matrix and a first blocking column matrix, where the first blocking row matrix may be a matrix formed by combining a zero matrix and an identity matrix; the first blocking column matrix may be a matrix composed of five zero matrices and five identity matrices combined.

In the above embodiment, the computer device obtains the first blocking matrix matched with the aimed column group, and multiplies each facade unit in the aimed column group by the obtained first blocking matrix in turn, so that a plurality of approximate column groups matched with the column group can be accurately restored.

In some embodiments, determining at least one set of column groups based on a degree of similarity between the first intermediate column groups comprises: determining the similarity between any two first intermediate column groups in each first intermediate column group; connecting the two first middle column groups with the similarity meeting the preset similarity condition through the connecting edge to obtain at least one connected domain; and taking the first middle column group belonging to the same connected domain as a column group set.

The connection edge is a line for connecting the first middle column group, and may specifically be a line for connecting the first middle column group with an association relationship. When the similarity in any two first intermediate column groups meets a preset similarity condition, the fact that the two first intermediate column groups have an association relationship is indicated, and connection can be performed by using a connection edge.

The connected domain is a column group set formed by a first middle column group connected by connecting edges, and when the connecting edges are adopted to connect the first middle column group, at least one connected domain can be formed; there is no connecting edge between the first intermediate column groups in the different communication domains.

Specifically, the computer device may calculate the similarity between any two first intermediate column groups, model the relationship between the first intermediate column groups as a connection graph, that is, connect the first intermediate column groups whose similarity meets a preset similarity condition with a connection edge until the computer device performs similarity calculation on all the first intermediate column groups, obtain at least one connected domain based on the connection graph generated by the connection edge, and use the first intermediate column groups belonging to the same connected domain as a column group set.

In some embodiments, the first middle column group belonging to the connected domain may be indirectly connected. Such as: the first middle column group A is connected with the first middle column group B through a connecting edge, the first middle column group B is connected with the first middle column group C through a connecting edge, the first middle column group A and the first middle column group C are not in direct connection, but the first middle column group A, the first middle column group B and the first middle column group C belong to column groups in the same communication domain. In this way, the column groups in the column group set can be as comprehensive as possible, and the obtained building elevation layout is more regular by normalizing as many column groups as possible.

In some embodiments, the connected domain obtained by the computer device is:。

wherein,is->All first middle column groups in the inner part, each first middle column group is directly connected or inter-connected through a connecting edgeAnd (5) connecting.

In some embodiments, the similarity threshold is set to 0.9, and the computer device may connect the first middle column groups having a similarity between any two first middle column groups of 0.9 with a connecting edge.

In some embodiments, the computer device may analyze, based on various algorithms such as a Floyd (Floyd algorithm), a Dijkstra (single-source shortest path) algorithm, a Bellman-Ford (Bellman Ford algorithm), and the like, for the first middle column group connected by the connection edge, to obtain at least one connected domain.

In the above embodiment, the computer device may connect the connection sides through which the similarity meets the preset similarity condition to obtain at least one connected domain, and determine the column group set based on the connected domain, so as to ensure that the normalization between the column groups is expanded between the first middle column groups that are approximately the same, and promote the rationality of the normalization between the columns, so that the finally obtained building elevation layout accords with the real building elevation layout.

In some embodiments, determining the similarity between any two first intermediate column groups of the first intermediate column groups comprises: combining any two first intermediate column groups in each first intermediate column group into a combination; for each combination, determining content similarity and geometric similarity between two first intermediate column groups in the combination; and obtaining the similarity between the two first middle column groups in the combination according to the content similarity and the geometric similarity.

The content similarity is an index for describing the similarity degree of the picture content, and can be used for measuring the similarity of the picture content. The geometric similarity is a similarity determined based on the width, area, etc. of the cells in the first intermediate column group.

Specifically, the computer device may determine, for a combination of any two first intermediate column groups, a structural similarity and a geometric similarity between the first intermediate column groups in the combination, and determine, based on a sum of the structural similarity and the geometric similarity, a similarity between the two first intermediate column groups in the combination.

In some embodiments, the first middle column groups are combined two by two according to the type of the middle unit in each first middle column group or the identification of each first middle column group, and of course, the computer device may also combine each first middle column group randomly.

In some embodiments, if the computer device performs the combination of the first middle column groups according to the types of the middle units in the first middle column groups, the computer device may perform the calculation of expanding the first middle column groups where the same type of facade units are located when performing the similarity calculation for the two first middle column groups in the combination, so that the efficiency of the similarity calculation may be improved, and the problem of computing resource waste caused by the expansion calculation of the first middle column groups which obviously cannot meet the preset similarity condition is avoided.

In some embodiments, the computer device may calculate the content similarity based on SSIM (Structure Similarity Index Measure, structural metrics) operator expansion calculations, and may also calculate the content similarity using methods such as mean square error, peak signal-to-noise ratio, and multi-scale similarity metrics.

In some embodiments, the computer device primarily considers brightness, contrast, and structure between pictures when performing content similarity calculations based on SSIM operators.

In some embodiments, for any pair of first intermediate column groups，/>The similarity between them can be calculated as follows:

wherein, Respectively->Width and total area in vertical units, +.>Respectively->Width and total area in vertical units, +.>Operators are evaluated for structural similarity (Structural Similarity, SSIM). Intuitively, the similarity score considers the similarity of the picture content (SSIM) as well as the shape and size (width and area) of the column group. When->When the similarity score is greater than a preset similarity score threshold, the similarity score threshold may be set to 2, 2.2, 2.5, etc., and if the similarity score threshold is 2, the computer device may determine that the first middle column group->，/>Belonging to the same column group set.

In the above embodiment, the computer device determines the similarity between the first column groups from multiple aspects by calculating the content similarity and the geometric similarity, so as to more accurately evaluate the similarity between the two first middle column groups, thereby improving the accuracy of subsequent normalization between the column groups, and obtaining a regular facade layout closer to the reality building rule.

In some embodiments, intercolumn normalization is performed on the set of column groups to obtain a second intermediate column group, comprising: for any column group set, generating a plurality of approximate column group sets corresponding to the column group set according to each first intermediate column group included in the column group set; screening a target approximate column group set from a plurality of approximate column group sets, and taking a first intermediate column group from which the target approximate column group set is derived as a representative column group; the representative column group is replicated to obtain a plurality of second intermediate column groups corresponding to the set of column groups.

The approximate column group set is a column group set obtained by performing column and row arrangement combination on the first intermediate column groups, namely, for any one column group set, a plurality of approximate column group sets corresponding to the column group set can be obtained by performing column and row arrangement combination on each first intermediate column group in the column group set.

The target set of approximate column groups is a column group determined from the set of approximate column groups. For any column group set, its target approximate column group set may specifically be an approximate column group set with small differences from the column group set.

The representative column group is a column group representative of each first intermediate column group in the same column group set that can be used to replace other first intermediate column groups. For each column group set, there is one representative column group.

Specifically, the computer device may perform row-column permutation and combination on each first intermediate column group included in any column group set to obtain a plurality of approximate column group sets. The computer device may then calculate a column set difference between each of the approximated column set and the targeted column set, and determine whether the column set difference satisfies a preset column set difference condition. The preset column group set difference condition is a condition set for judging the calculated column group set difference to determine whether or not an approximate column group set corresponding to the column group set difference can be determined as a target approximate column group set. The preset column group set difference conditions can be adaptively set by combining the accuracy requirements of actual building image processing, actual application scenes and the like.

Further, the computer device may copy the representative column group for each column group set, that is, use the representative column group, replace the first intermediate column groups other than the representative column group in the column group set to be referred to, and obtain a plurality of second intermediate column groups corresponding to the column group set to be referred to.

In some embodiments, the computer device, when generating the approximate column group set based on the first intermediate column group, expands in particular according to the actual column-row arrangement of the column group set for which it is intended. For example, a certain column group set is composed of three first middle column groups, and each first middle column group includes three facade units. When the computer equipment adopts any first middle column group in the column group to obtain the approximate column group set, any vertical face unit in the first middle column group can be arranged for three times in the column direction, and any vertical face unit in the first middle column group can be arranged for three times in the row direction, so that the approximate column group set can be obtained.

In some embodiments, the computer device may calculate a euclidean distance, cosine similarity, or pearson similarity, etc. between the approximated column group set and the column group set for which it is intended to determine the differences between the column group set and its corresponding approximated column group set.

In some embodiments, the preset column group set difference condition may be set based on a column group set difference threshold; and determining the approximate column group set corresponding to the column group set difference as a target approximate column group set when the column group set difference between the target column group set and the corresponding approximate column group set is smaller than or equal to a column group set difference threshold value for any column group set. When determining a plurality of target approximate column group sets for the same column group set, a first intermediate column group corresponding to any one target approximate column group set can be selected as a representative column group, and of course, a voting mechanism can be set, and one of the first intermediate column groups corresponding to the plurality of target approximate column group sets can be selected as the representative column group.

In other embodiments, the preset column group set difference condition may be that the condition is satisfied when the column group set difference is minimum. The computer device may determine, for any column group set, a respective column group set difference between the column group set for which it is intended and the corresponding approximate column group set, and determine the approximate column group set corresponding to the smallest column group set difference as the target approximate column group set.

In some embodiments, as shown in FIG. 8, there is a schematic diagram of the structure before inter-column normalization for the three first intermediate column groups in the column group set, and after inter-column normalization. As can be seen from fig. 8, the column group set includes a first intermediate column group a, a first intermediate column group b, and a first intermediate column group c, and the computer device determines that the representative column group is the first intermediate column group c. The computer device may then copy the representative column group for the column group set, that is, replace the first intermediate column group b and the first intermediate column group a in the column group set with the first intermediate column group c, to obtain a plurality of second intermediate column groups corresponding to the column group set: a second intermediate column group a, a second intermediate column group b, and a second intermediate column group c. The dimension of each vertical face unit between the second middle column groups is the same, and the vertical face units are aligned in the horizontal direction and the vertical direction.

In the above embodiment, the computer device may normalize the column groups according to the column groups, so that all column groups in the same column group set maintain consistency, which is favorable for obtaining a regularized building elevation layout, and the regularized building elevation layout is closer to the actual building rule, and can better reflect the real situation of the building, and effectively improve the accuracy of building image processing.

In some embodiments, generating a plurality of approximated column group sets corresponding to the targeted column group set from each first intermediate column group included in the targeted column group set includes: splicing each first middle column group in the targeted column group set to obtain a spliced picture; acquiring a second block matrix matched with the spliced picture; traversing each first intermediate column group in the spliced picture, and generating an approximate column group set corresponding to the column group set according to the traversed first intermediate column group and the traversed second block matrix; after the traversal is completed, a plurality of approximate column group sets corresponding to the targeted column group set are obtained.

The first middle column group is to splice each first middle column group belonging to the same column group set into a picture.

The second blocking matrix is matched with the element composition of the spliced picture in the horizontal direction and the element composition of the spliced picture in the vertical direction. When the elements of the spliced picture are vertical face units, the second block matrix is related to the vertical face unit composition in the horizontal direction and the vertical face unit composition in the vertical direction in the spliced picture. Thus, the second tile matrix will also be different for stitched pictures where there are different facade element compositions.

The second blocking matrix comprises a second blocking row matrix and a second blocking column matrix, and the dimension, the size, the structure and the like of the second blocking row matrix depend on the number of lines of elements in the spliced picture, namely the number of lines of the facade units in the spliced picture; the dimension, size, structure, etc. of the second block column matrix depend on the number of columns of elements in the stitched picture, i.e. the number of columns of the facade elements in the stitched picture.

Specifically, for any spliced picture, the computer equipment acquires a second block matrix matched with the spliced picture; for each first middle column group in the aimed spliced picture, the computer equipment can take any one elevation unit from each first middle column group, and multiply the elevation unit with the obtained second block matrix respectively to obtain a plurality of approximate column group sets matched with the aimed column group sets.

In some embodiments, the second block row matrix and the second block column matrix are obtained by combining a zero matrix and an identity matrix, and the number of the zero matrix and the identity matrix depends on the distribution condition of the facade units in the spliced picture.

In some embodiments, the spliced picture a is a picture spliced by three first middle column groups, each first middle column group may be formed by 4 facade units, and the second block matrix matched with the spliced picture a includes a second block row matrix and a second block column matrix, where the second block row matrix is a matrix formed by combining at least three zero matrices and three identity matrices; the second block column matrix is a matrix formed by combining at least four zero matrices and four unit matrices.

In some embodiments, as shown in fig. 9, a schematic structural diagram of matrix decomposition is performed for a stitched picture:

the spliced picture L may be a picture spliced by 4 first intermediate column groups, where each of the first intermediate column groups has completely identical facade units. The computer equipment can acquire a second block column matrix and a second block row matrix which are matched with the spliced picture L, wherein the second block row matrix and the second block column matrix are matrices formed by an identity matrix and a zero matrix; for any first middle column group in the spliced picture L, the computer equipment can take any facade unit R from the first middle column group to be aimed at, and multiply the facade unit R with the obtained second block column matrix and the second block row matrix respectively to obtain a plurality of approximate column group sets matched with the aimed at column group sets.

In some embodiments, when the computer device obtains the stitched pictures based on the first middle column group, the computer device may expand the stitching according to the column group intervals in the building semantic graph, or may perform the stitching according to the set column group intervals. When the interval is set, the blank area is reduced, the picture width is reduced, and the adjacent first middle column group is ensured to be set on the premise. The set column group interval may be a plurality of pixel intervals of 8 pixels, 10 pixels, 15 pixels, and the like.

In some embodiments, the computer device splices first intermediate column groups in the same column group into a spliced picture, the first intermediate column groups being blank blocks of 10 pixels in widthFor separation, there are。

Wherein,in order to splice the pictures together,Bis a blank block which is used for the display of the display screen,R ^10×H representing the width range of the blank block B, R is the real number domain, H is the height of the building semantic graph, < ->Each first intermediate column group is represented. Here with a blank block of 10 pixelsBThe array groups are separated instead of the original intervals of the array groups in the original image, the width of the spliced pictures is obviously reduced, the calculated amount is reduced, and the algorithm efficiency is quickened.

In some embodiments, a computer device may be directed toPerforming inter-column normalization to obtain new graph with all vertical face units normalized and all column groups completely identical ∈ ->At this time, the computer device can conveniently extract a column group from the column groupUse +.>Replace->And (3) all the column groups in the list, thereby finishing the normalization among columns and obtaining the final regularization result.

In the above embodiment, the computer device may calculate the approximate column group set by acquiring the second block matrix matched with the aimed column group set, for each first intermediate column group in the aimed column group, in combination with the acquired second block matrix, so as to accurately restore a plurality of approximate column group sets matched with the column group set.

In some embodiments, further comprising: modeling a virtual scene according to the building elevation layout; or performing building analysis according to the building elevation layout, wherein the building analysis at least comprises any one of lighting analysis, heat dissipation analysis or style analysis; or, performing simulation application according to the building elevation layout; the simulation application includes at least one of an autopilot simulation, or a virtual city simulation.

The virtual scene modeling is to simulate a virtual environment with high simulation degree and strong sense of reality based on a building elevation layout diagram in a virtual digital space by utilizing a virtual reality technology, thereby playing an auxiliary role in the production of games in cities and the production of various videos.

In the building analysis, lighting, heat dissipation, regional building style and the like of a building are analyzed through a building elevation layout diagram in a smart city project.

The simulation application is an application in the aspects of automatic driving simulation, virtual smart city simulation and the like based on the building elevation view.

Specifically, based on the obtained building elevation layout, the computer device may perform virtual scene modeling, building analysis, simulation application, and the like, and thus, the building elevation layout may play an important role in applications such as city design, efficiency estimation, building usage evaluation, digital twinning, and the like.

In the above embodiment, the computer device may perform virtual scene modeling, building analysis and simulation application based on the building elevation layout, and since the obtained regularized building elevation layout is close to the real building rule, the real situation of the building may be reflected, and the obtained building elevation layout may also play a better auxiliary role when applied in various large fields.

The application also provides an application scene, which applies the building image processing method. Specifically, the application of the building image processing method in the application scene is as follows:

as shown in fig. 10, the building image processing method provided by the embodiment of the application includes: the method comprises the steps of 1, data preparation and preprocessing; 2. detecting a vertical face unit; 3. regularization in columns; 4. four parts are regularized between columns.

In the data preparation and preprocessing section, a building elevation map is acquired, which requires extraction of the building elevation layout map. The user can select four correction points of the building elevation on the picture, and in the data preparation and preprocessing, a parallel elevation view of the building elevation can be obtained.

Then, the facade unit detection module extracts the facade units from the front view, wherein the facade units comprise windows and balconies, and a user can enable an algorithm to automatically extract the facade units and manually select the facade units.

Further, the intra-column regularization module detects column block lines, determines column block lines, groups the vertical face units according to columns based on the column block lines, and obtains a plurality of column groups. And the vertical face units in each column group are subjected to in-column normalization, so that the vertical face units in the same column group are identical, and the layout prior of a real building can be utilized: the elements of the same column are considered identical.

Finally, the inter-column regularization module finds the same column group by calculating the similarity of different column groups, performs inter-column normalization on the same column group to obtain a final regular facade layout result, and a user can modify an evaluation threshold of the inter-column similarity so as to meet own requirements for different scenes.

According to the building image processing method provided by the embodiment of the application, for any one input building elevation image, the elevation layout of the building is obtained, wherein the elevation layout comprises building units, the sizes and the positions of the units and the types of the units exist in an elevation. Using the output layout results, the user can either restore the building facade in a real building with high fidelity or help analyze the layout design of the real building. Typical application scenarios include building restoration in game making, building analysis in city planning, simulation in autopilot, etc.

The building image processing method provided by the embodiment of the application has good universality, and the subsequent calculation does not need manual intervention except the initial correction module, so that the building image processing method has the advantages of low cost, simple deployment and high operation efficiency. The building image processing method provided by the embodiment of the application has extremely high expansibility because the modules are mutually decoupled.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a building image processing device for realizing the building image processing method. The implementation of the solution provided by the apparatus is similar to the implementation described in the above method, so the specific limitations in the embodiments of one or more building image processing apparatuses provided below may be referred to the limitations of the building image processing method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 11, there is provided a building image processing apparatus 1100 including: a rectification module 1102, a detection module 1104, a grouping module 1106, and a regularization module 1108, wherein:

and the correction module 1102 is used for acquiring a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation.

The detection module 1104 is used for detecting the elevation unit based on the elevation of the building elevation and generating a building semantic graph according to the detection result; each facade element is marked in the building semantic graph.

Grouping module 1106 is configured to group each facade unit in the building semantic graph according to columns, so as to obtain a plurality of column groups.

The regularization module 1108 is configured to perform intra-column regularization on the facade units belonging to the same column group, and perform inter-column regularization on different column groups, so as to obtain a regularized building facade layout.

In some embodiments, the detection module 1104 is further configured to identify each facade element in a front view of the building facade, and a category to which each facade element belongs; generating an initial semantic graph which is equivalent to a front view of a building elevation; for any category, marking the facade units belonging to the category by the identification matched with the category in the initial semantic graph to obtain the building semantic graph.

In some embodiments, grouping module 1106 is further configured to determine a plurality of chunking lines in the architectural semantic map; based on any two adjacent block lines in the plurality of block lines to form sub-areas, each sub-area is cut out from the building semantic graph to obtain a plurality of column groups.

In some embodiments, grouping module 1106 is further configured to determine a plurality of vertical lines in the building semantic graph during the current iteration; traversing a plurality of vertical lines, and determining the score of the traversed vertical line according to the vertical line before the traversed vertical line and the current block line set aiming at any traversed vertical line; determining a target vertical line with scores meeting preset conditions from a plurality of vertical lines, taking the target vertical line as a block line and adding the block line into a block line set; and returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process, and continuing to execute until the preset iteration stop condition is reached, so as to obtain a plurality of block lines in the building semantic graph.

In some embodiments, grouping module 1106 includes a score computation module; a score calculation module for determining a pixel difference between a vertical line preceding the traversed vertical line and the traversed vertical line; determining the distance between the traversed vertical line and each block line in the current block line set; a score of the traversed vertical line is determined based on the pixel differences, and a minimum in distance.

In some embodiments, the grouping module 1106 is further configured to determine a score corresponding to each partition line in the current set of partition lines; determining the difference between the maximum score in the scores corresponding to the block lines and the score of the target vertical line; and when the preset iteration stop condition is not met based on the difference, executing the step of returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process.

In some embodiments, the regularization module 1108 is further configured to normalize the facade units in each column group in a column to obtain each first middle column group; determining at least one column group set based on a degree of similarity between the first intermediate column groups; each column group comprises a plurality of first middle column groups meeting preset similar conditions; performing inter-column normalization on the column groups in each column group set to obtain a second middle column group; and obtaining a regularized building elevation layout corresponding to the building elevation according to the column group which does not perform inter-column normalization in the first middle column group and the second middle column group.

In some embodiments, regularization module 1108 includes an intra-column regularization module; the in-column regularization module is used for generating a plurality of approximate column groups corresponding to the column groups according to each vertical face unit included in the column group; screening out a target approximate column group from a plurality of approximate column groups, and taking a vertical face unit from which the target approximate column group is derived as a representative vertical face unit; the representative facade elements are replicated to obtain a first intermediate column group corresponding to the targeted column group.

In some embodiments, the intra-column regularization module is further configured to obtain a first blocking matrix that matches the targeted column group; traversing each facade unit in the aimed column group, and determining an approximate column group corresponding to the aimed column group according to the traversed facade unit and the first block matrix; after the traversal is completed, a plurality of approximate column groups corresponding to the targeted column group are obtained.

In some embodiments, regularization module 1108 includes an inter-column regularization module; the inter-column regularization module is used for determining the similarity between any two first intermediate column groups in each first intermediate column group; connecting the two first middle column groups with the similarity meeting the preset similarity condition through the connecting edge to obtain at least one connected domain; and taking the first middle column group belonging to the same connected domain as a column group set.

In some embodiments, the inter-column normalization module is further configured to combine any two first intermediate column groups of the first intermediate column groups into a combination; for each combination, determining content similarity and geometric similarity between two first intermediate column groups in the combination; and obtaining the similarity between the two first middle column groups in the combination according to the content similarity and the geometric similarity.

In some embodiments, the inter-column normalization module is further configured to, for any one of the column group sets, generate a plurality of approximate column group sets corresponding to the column group set according to each first intermediate column group included in the column group set; screening a target approximate column group set from a plurality of approximate column group sets, and taking a first intermediate column group from which the target approximate column group set is derived as a representative column group; the representative column group is replicated to obtain a plurality of second intermediate column groups corresponding to the set of column groups.

In some embodiments, the inter-column normalization module is further configured to splice each first middle column group in the set of targeted column groups to obtain a spliced picture; acquiring a second block matrix matched with the spliced picture; traversing each first intermediate column group in the spliced picture, and generating an approximate column group set corresponding to the column group set according to the traversed first intermediate column group and the traversed second block matrix; after the traversal is completed, a plurality of approximate column group sets corresponding to the targeted column group set are obtained.

In some embodiments, the building image processing apparatus further comprises a map determination module; and the layout diagram determining module is used for replacing the corresponding column group in the building semantic diagram by the column group which does not perform inter-column normalization in the first middle column group and the second middle column group to obtain a regularized building elevation layout diagram corresponding to the building elevation diagram.

In some embodiments, the correction module 1102 is further configured to intercept the object to be corrected from the building elevation; determining a correction transformation matrix based on the object to be corrected; and performing perspective distortion correction on the object to be corrected according to the correction transformation matrix to obtain a front view of the building elevation.

In some embodiments, the building image processing apparatus further comprises a map processing module; the layout diagram processing module is used for carrying out virtual scene modeling according to the building elevation layout diagram; or performing building analysis according to the building elevation layout, wherein the building analysis at least comprises any one of lighting analysis, heat dissipation analysis or style analysis; or, performing simulation application according to the building elevation layout; the simulation application includes at least one of an autopilot simulation, or a virtual city simulation.

The respective modules in the above building image processing apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server or a terminal, and the internal structure of which may be as shown in fig. 12. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing building image data. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a building image processing method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 12 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method of building image processing, the method comprising:

detecting a facade unit based on a front view of the building facade, and generating a building semantic graph according to a detection result; each elevation unit is marked in the building semantic graph;

2. The method according to claim 1, wherein the performing facade element detection based on the front view of the building facade and generating the building semantic graph according to the detection result includes:

identifying each facade unit in the front view of the building facade and the category to which each facade unit belongs;

generating an initial semantic graph that is equivalent to a front view of the building facade;

and marking the facade units belonging to the aimed category in the initial semantic graph through the identification matched with the aimed category aiming at any category to obtain the building semantic graph.

3. The method of claim 1, wherein grouping the facade elements in the building semantic graph into columns results in a plurality of column groups, comprising:

determining a plurality of block lines in the building semantic graph;

based on any two adjacent block lines in the plurality of block lines to form sub-areas, cutting each sub-area out of the building semantic graph to obtain a plurality of column groups.

4. A method according to claim 3, wherein said determining a plurality of chunking lines in the building semantic graph comprises:

determining a plurality of vertical lines in the building semantic graph in the current iteration process;

traversing the plurality of vertical lines, and determining the score of the traversed vertical line according to the vertical line before the traversed vertical line and the current block line set aiming at any traversed vertical line;

determining a target vertical line with scores meeting preset conditions from the plurality of vertical lines, taking the target vertical line as a blocking line and adding the blocking line into a blocking line set;

and returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process, and continuing to execute until a preset iteration stopping condition is reached, so as to obtain a plurality of block lines in the building semantic graph.

5. The method of claim 4, wherein determining the score of the traversed vertical line from the vertical line preceding the traversed vertical line and the current set of patch lines comprises:

determining a pixel difference between a vertical line preceding the traversed vertical line and the traversed vertical line;

Determining the distance between the traversed vertical line and each block line in the current block line set;

a score of the traversed vertical line is determined based on the pixel difference, and a minimum of the distances.

6. The method of claim 4, wherein after the target vertical line is taken as a patch line and added to a set of patch lines, the method further comprises:

determining the score corresponding to each block line in the current block line set;

determining the difference between the maximum score in the scores corresponding to the block lines and the score of the target vertical line;

and when the fact that the preset iteration stop condition is not met is determined based on the difference, executing the step of returning to the step of determining a plurality of vertical lines in the building semantic graph in the current iteration process.

7. The method of claim 1, wherein the performing intra-column regularization on facade elements belonging to a same column group and performing inter-column regularization on different column groups to obtain a regularized building facade layout comprises:

performing in-row normalization on the vertical face units in each row group to obtain each first middle row group;

Determining at least one column group set based on a degree of similarity between each of the first intermediate column groups; each column group comprises a plurality of first middle column groups meeting preset similar conditions;

performing inter-column normalization on the column groups in each column group set to obtain a second middle column group;

and obtaining a regularized building elevation layout corresponding to the building elevation according to the column group which does not perform inter-column normalization in the first intermediate column group and the second intermediate column group.

8. The method of claim 7, wherein said in-column normalizing the facade elements in each column group to obtain each first intermediate column group comprises:

for any column group, generating a plurality of approximate column groups corresponding to the column group according to each facade unit included in the column group;

screening out a target approximate column group from the plurality of approximate column groups, and taking a vertical face unit from which the target approximate column group is derived as a representative vertical face unit;

the representative facade elements are replicated to obtain a first intermediate column group corresponding to the targeted column group.

9. The method of claim 8, wherein generating a plurality of approximated column groups corresponding to the column group from the facade elements included in the column group comprises:

Acquiring a first blocking matrix matched with the aimed column group;

traversing each facade unit in the aimed column group, and determining an approximate column group corresponding to the aimed column group according to the traversed facade unit and the first block matrix;

after the traversal is completed, a plurality of approximate column groups corresponding to the targeted column group are obtained.

10. The method of claim 7, wherein said determining at least one set of column groups based on a degree of similarity between each of said first intermediate column groups comprises:

determining the similarity between any two first intermediate column groups in each first intermediate column group;

connecting the two first middle column groups with the similarity meeting the preset similarity condition through the connecting edge to obtain at least one connected domain;

and taking the first middle column group belonging to the same connected domain as a column group set.

11. The method of claim 10, wherein determining the similarity between any two first intermediate column groups in each of the first intermediate column groups comprises:

combining any two first intermediate column groups in each first intermediate column group into a combination;

for each combination, determining content similarity and geometric similarity between two first intermediate column groups in the combination;

And obtaining the similarity between the two first middle column groups in the combination according to the content similarity and the geometric similarity.

12. The method of claim 7, wherein intercolumn normalizing the column groups in each column group to obtain a second intermediate column group, comprises:

for any column group set, generating a plurality of approximate column group sets corresponding to the column group set according to each first intermediate column group included in the column group set;

screening a target approximate column group set from the plurality of approximate column group sets, and taking a first middle column group from which the target approximate column group set is derived as a representative column group;

the representative column group is replicated to obtain a plurality of second intermediate column groups corresponding to the set of column groups.

13. The method of claim 12, wherein the generating a plurality of approximate column group sets corresponding to the column group set for each first intermediate column group included in the column group set for each first intermediate column group includes:

splicing each first middle column group in the targeted column group set to obtain a spliced picture;

acquiring a second block matrix matched with the spliced picture;

traversing each first intermediate column group in the spliced picture, and generating an approximate column group set corresponding to the column group set according to the traversed first intermediate column group and the second block matrix;

After the traversal is completed, a plurality of approximate column group sets corresponding to the targeted column group set are obtained.

14. The method of claim 7, wherein the obtaining a regularized building elevation layout corresponding to the building elevation from the set of columns in the first intermediate set of columns that are not inter-column normalized and the second intermediate set of columns comprises:

and replacing the corresponding column group in the building semantic graph by the column group which does not perform inter-column normalization in the first intermediate column group and the second intermediate column group to obtain a regularized building elevation layout graph corresponding to the building elevation graph.

15. The method of claim 1, wherein said correcting the building elevation results in a front view of the building elevation, comprising:

intercepting an object to be corrected from the building elevation;

determining a correction transformation matrix based on the object to be corrected;

and performing perspective distortion correction on the object to be corrected according to the correction transformation matrix to obtain a front view of the building elevation.

16. The method according to any one of claims 1 to 15, further comprising:

performing virtual scene modeling according to the building elevation layout; or,

Building analysis is carried out according to the building elevation layout, wherein the building analysis at least comprises any one of lighting analysis, heat dissipation analysis or style analysis; or,

performing simulation application according to the building elevation layout; the simulation application includes at least one of an autopilot simulation, or a virtual city simulation.

17. A building image processing apparatus, characterized in that the apparatus comprises:

the correction module is used for obtaining a building elevation view, and correcting the building elevation view to obtain a front view of the building elevation;

the detection module is used for detecting the elevation unit based on the elevation of the building elevation and generating a building semantic graph according to the detection result; each elevation unit is marked in the building semantic graph;

18. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 16 when the computer program is executed.

19. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 16.

20. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 16.