CN117764896A - Building integrity detection method and device, electronic equipment and storage medium - Google Patents

Abstract

The application provides a building integrality detection method, device, electronic equipment and storage medium, has related to image recognition technical field, has solved at present through the manual work to the problem that building outer wall integrality periodic investigation is with high costs and inefficiency, includes: acquiring a building video containing a target building; identifying a target building outer elevation in a building video to obtain a building outer elevation image of the target building; the method and the device can automatically realize the integrity analysis of the target building without needing additional personnel to periodically observe the target building on site, reduce the integrity analysis cost of the target building and improve the integrity analysis efficiency of the target building.

Description

Building integrity detection method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method and apparatus for detecting building integrity, an electronic device, and a storage medium.

Background

At present, according to incomplete statistics, accidents such as casualties, automobile damage and the like caused by falling of building exterior wall facing can occur each year, and in order to reduce accidents caused by falling of building exterior walls, safety hidden danger investigation is carried out on building exterior surfaces in jurisdictions by arranging personnel regularly at present, and safety liability people or management units are supervised and urged to carry out daily maintenance, safety inspection, identification and safety hidden danger remediation on the existing building exterior surfaces.

Only rely on personnel to examine regularly, have with high costs, ageing poor, personnel arrange not good, can cover the incomplete and can not accurately analyze out the problem of building outer wall integrality in region.

Disclosure of Invention

The building integrity detection method, device, electronic equipment and storage medium can rapidly analyze the integrity of the target building, and does not need additional personnel to periodically observe and analyze the target building, so that the cost of analyzing the integrity of the target building is reduced, and the efficiency of analyzing the integrity of the target building is improved.

In one aspect, the present application provides a building integrity detection method, comprising:

acquiring a building video containing a target building;

identifying the outer facade of the target building in the building video to obtain a building outer facade image of the target building;

Determining an edge of the target building in the building facade image, an

And determining whether the target building is complete in the building facade image according to the edge.

In one possible implementation manner of the present application, the identifying the outer facade of the target building in the building video, and obtaining a building outer facade image of the target building includes:

detecting the target building for the building video to generate a building detection frame;

intercepting an initial image of a building in a corresponding area according to the building detection frame;

determining a plurality of candidate corner points corresponding to the target building in the initial building image;

and generating the building outer elevation image according to the candidate corner points.

In one possible implementation manner of the present application, the determining a plurality of candidate corner points corresponding to the target building in the initial image of the building includes:

detecting a plurality of initial corner points in the initial image of the building by using a Harris corner point detector;

and filtering a plurality of initial corner points which are not near the plurality of vertexes of the building detection frame in the initial image of the building according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the building detection frame, and taking the rest of the initial corner points in the initial image of the building as a plurality of candidate corner points.

In one possible implementation manner of the present application, the generating the building facade image according to the candidate corner points includes:

affine transformation is carried out on an irregular image area surrounded by the candidate angular points, so as to obtain a regular image area;

and taking the regular image as the building outer elevation image.

In one possible implementation of the present application, the determining an edge of the target building in the building facade image includes:

binarizing the building facade image; and

and performing edge detection according to the binarized building outer elevation image to obtain the edge of the target building in the building outer elevation image.

In one possible implementation manner of the present application, the determining, according to the edge, whether the target building is complete in the building facade image includes:

calculating the graphic area of a graphic formed by the edges of the target building in the building outer elevation image;

performing a straight line fit to the edge to obtain a plurality of straight line equations representing the edge of the target building in the building facade image;

Calculating the area of the outer elevation area formed by the straight lines expressed by the plurality of straight line equations;

and determining whether the target building is complete in the building facade image according to the graph area and the facade area.

In one possible implementation manner of the present application, the determining, according to the graphic area and the facade area, whether the target building is complete in the building facade image includes:

calculating an area ratio of the graphic area to the facade area

And when the area ratio is greater than a preset threshold value, determining that the target building is complete in the building facade image.

In one possible implementation of the present application, after the determining whether the target building is complete in the building facade image according to the edge, the method further includes:

when the target building is complete in the building facade image, storing the building facade image into a preset facade complete sample library;

and when the target building is not complete in the building outer elevation image, sending warning information of potential safety hazards of the target building to a target receiving terminal.

In another aspect, the present application provides a building integrity detection apparatus, the apparatus comprising:

the acquisition module is used for acquiring building videos containing the target building;

the identification module is used for identifying the outer facade of the target building in the building video to obtain a building outer facade image of the target building;

an edge determination module for determining an edge of the target building in the building facade image, an

And the integrity determining module is used for determining whether the target building is complete in the building facade image according to the edge.

In another aspect, the present application also provides a computer readable storage medium having stored thereon a computer program to be loaded by a processor for performing the steps of the building integrity detection method.

After the building video containing the target building is acquired, automatically identifying the outer elevation of the target building in the building video to obtain a building outer elevation image of the target building, determining the edge of the target building in the building outer elevation image, and determining whether the target building is complete in the building outer elevation image according to the edge; therefore, the method and the system can automatically identify the building outer elevation image based on the building video by collecting the building video of the target building, can automatically realize the integrity analysis of the target building based on the building outer elevation image, and can reduce the integrity analysis cost of the target building and improve the integrity analysis efficiency of the target building without needing additional personnel to periodically observe the target building on site.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is an application scenario diagram of a building integrity detection system provided in an embodiment of the present application;

FIG. 2 is a flow diagram of one embodiment of a method for building integrity detection provided in an embodiment of the present application;

FIG. 3 is a schematic structural view of one embodiment of a building integrity detection device provided in an embodiment of the present application;

fig. 4 is a schematic structural diagram of an embodiment of a server provided in an embodiment of the present application.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.

In the description of the present invention, it should be understood that, furthermore, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or an implicit indication of the number of features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

In this application, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the invention. In the following description, details are set forth for purposes of explanation. It will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the invention with unnecessary detail. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiment of the application provides a building integrity detection method, a device, an electronic device and a storage medium, and the method, the device, the electronic device and the storage medium are respectively described in detail below.

The execution body of the building integrity detection method in the embodiment of the present application may be a building integrity detection device provided in the embodiment of the present application, or different types of electronic devices such as a server device, a physical host, or a User Equipment (UE) integrated with the building integrity detection device, where the building integrity detection device may be implemented in a hardware or software manner, and the UE may specifically be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer, or a personal digital assistant (Personal Digital Assistant, PDA).

The electronic device may be operated in a single operation mode, or may also be operated in a device cluster mode.

As shown in fig. 1, fig. 1 is a schematic view of a building integrity detection system according to an embodiment of the present application. The building integrity detection system may include a video acquisition device for capturing a target building and an electronic device 100 for completing a building integrity detection method, where the electronic device 100 is integrated with a building integrity detection device. For example, the electronic device may acquire a building video that includes a target building; identifying a target building outer elevation in a building video to obtain a building outer elevation image of the target building; determining an edge of the target building in the building facade image, and determining whether the target building is complete in the building facade image according to the edge.

In addition, as shown in fig. 1, the building integrity detection system may further include a memory 200 for storing data, such as building video data, image data, and device data of a video capture device for capturing building video, etc.

It should be noted that, the schematic view of the building integrity detection system shown in fig. 1 is only an example, and the building integrity detection system and the scene described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and as one of ordinary skill in the art can know, with evolution of the building integrity detection system and appearance of a new service scenario, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

Next, a description will be given initially of a building integrity detection method provided in an embodiment of the present application, in which an electronic device is used as an execution body, and in order to simplify and facilitate description, in a subsequent method embodiment, the execution body is omitted, and the building integrity detection method includes:

acquiring a building video containing a target building; identifying a target building outer elevation in a building video to obtain a building outer elevation image of the target building; determining an edge of the target building in the building facade image, and determining whether the target building is complete in the building facade image according to the edge.

According to the method and the system, the building video of the target building can be collected, the building outer elevation image is identified based on the building video, the integrity analysis of the target building can be realized based on the building outer elevation image, no extra personnel are needed to periodically observe the target building on site, the integrity analysis cost of the target building is reduced, and the integrity analysis efficiency of the target building is improved.

Fig. 2 is a schematic flow chart of an embodiment of a building integrity detection method according to an embodiment of the present application, and fig. 2 is a schematic flow chart of a building integrity detection method according to an embodiment of the present application. It should be noted that although a logical order is depicted in the flowchart, in some cases the steps depicted or described may be performed in a different order than presented herein. The building integrity detection method specifically comprises the following steps 201 to 204:

201. building video is acquired that includes the target building.

The target building may be any building in a street view, where integrity detection needs to be performed, the building video is a video obtained by shooting at any angle outside the target building through the video capturing device, in an application process, the video capturing device may be a fixed shooting device, such as a thunder integrated machine, a camera, and the like, which is arranged near the target building, and the video capturing device may also be a mobile shooting device, such as a vehicle-mounted shooting device, an unmanned aerial vehicle, and the like, which is not limited in this embodiment.

In this embodiment, after the video acquisition device acquires the building video of the target building, a connection channel is established between the video acquisition device and the electronic device for executing the building integrity detection method through the network transmission module, and the building video or the image acquired by the video acquisition device is sent to the electronic device in a message form, so as to acquire the building video of the target building.

In this embodiment, in order to record the geographic information and shooting time information of the shot target building, when the video acquisition device transmits the building video to the electronic device, the spatiotemporal geographic information of the video acquisition device is transmitted at the same time, and the spatiotemporal geographic information includes information such as coordinate position information, attitude angle information, timestamp information and the like when the video acquisition device acquires the building video of the target building, and the data content transmitted by the video acquisition device is not specifically limited in this embodiment.

202. And identifying the outer facade of the target building in the building video to obtain a building outer facade image of the target building.

The outer elevation of the target building is the outer surface of the target building, and because the video acquisition equipment acquires pictures of other elements except the target building when acquiring the building video, in order to avoid interference of the pictures of the other elements in the building video on the integrity detection of the target building, before the integrity detection of the target building in the building video, the area of the target building, namely the outer elevation image of the building of the target building, needs to be determined from the building video.

Illustratively, identifying a target building facade in a building video to obtain a building facade image of the target building, including steps 301-304:

301. and detecting the target building for the building video to generate a building detection frame.

In this embodiment, a set of image frames of a building video is taken as input, a building video is detected by a trained target building detection model, each image frame is output with a set of image frames of a target building and a building detection frame, and then the integrity analysis of the target building is based on one of the image frames in the set of image frames; in the application process, the integrity analysis of the target building may be based on an analysis of an image frame with the target building and the building detection frame of any frame in the image frame assembly, so as to obtain a more complete and complete detection of whether the target building is complete in the building facade image, or may be based on a multi-frame image frame or each frame image frame in the image frame assembly, which is not limited in particular in this embodiment.

302. And intercepting an initial image of the building in the corresponding area according to the building detection frame.

Because each image frame includes the target building and other irrelevant areas, in order to avoid interference of the other irrelevant areas in the image frame to the integrity detection of the target building, in this embodiment, based on the area of the target building marked by the building detection frame, the image of the area of the target building is intercepted, so as to obtain an initial building image, and then, based on the initial building image, the integrity analysis of the target building is performed.

303. A plurality of candidate corner points corresponding to the target building in the initial image of the building are determined.

In this embodiment, after the initial building image is obtained, in order to restore the initial building image as clearly as possible, so as to extract a plurality of candidate corner points corresponding to the target building in the initial building image, firstly, converting the initial building image from RGB color space to Lab color space by adopting cvtColor function in opencv, extracting L texture information of the initial building image, and then determining a plurality of candidate corner points corresponding to the target building in the initial building image based on the L texture information of the initial building image. In this embodiment, the L texture information of the building initial image may be extracted by using a gray level co-occurrence matrix, gray level stroke statistics, gray level difference statistics, local gray level statistics, half-square difference map, autocorrelation function, and the like, which is not limited in this embodiment.

In this embodiment, determining a plurality of candidate corner points corresponding to a target building in an initial image of the building specifically includes 3031 to 3032:

3031. a Harris corner detector is used to detect a plurality of initial corner points in an initial image of a building.

Specifically, a Harris angle point detector is adopted to slide in any direction on an initial image of a building, the two conditions before and after sliding are compared, and if the sliding in any direction has larger gray level change, interest points, which are also called corner points, exist in the sliding window;

the Harris corner point detector is adopted for corner point detection in the initial image of the building for the first time, a plurality of initial corner points in the initial image of the building can be obtained, but not every initial corner point in the initial image of the building can help to determine the edge of the target building in the outer elevation image of the building, so that the needed candidate corner points need to be determined from the initial corner points.

3032. And filtering a plurality of initial corner points which are not near the plurality of vertexes of the building detection frame in the initial image of the building according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the building detection frame, and taking the rest of the plurality of initial corner points in the initial image of the building as a plurality of candidate corner points.

According to Euler distances between a plurality of initial corner points and a plurality of vertexes of a building detection frame, the method specifically comprises the following steps:

calculating Euler distances between each initial corner point of the plurality of initial corner points and each vertex of the building detection frame, namely calculating the distances between the initial corner points and the vertices of the building detection frame in a specific plane rectangular coordinate system to obtain a plurality of groups of distance calculation value groups corresponding to the plurality of initial corner points respectively, wherein each group of distance calculation value groups comprises a plurality of distance calculation values, and the number of the distance calculation values in each group of distance calculation value groups is the same as that of the vertices of the building detection frame;

filtering a plurality of initial corner points which are not near a plurality of vertexes of a building detection frame in an initial image of a building, taking the rest of the plurality of initial corner points in the initial image of the building as a plurality of candidate corner points, and specifically comprising the following steps:

all the calculated distance calculation values are ordered, whether the order is ascending or descending, initial corner points exceeding a preset distance threshold value in the distance calculation values (namely, a plurality of initial corner points which are not near a plurality of vertexes of a building detection frame) are filtered, initial corner points smaller than the preset distance threshold value in the distance calculation values (namely, a plurality of initial corner points which are near a plurality of vertexes of the building detection frame) are reserved, and the reserved initial corner points are used as a plurality of candidate corner points.

304. And generating a building facade image according to the plurality of candidate corner points.

In this embodiment, according to a plurality of candidate corner points, a building facade image is generated, which specifically includes:

carrying out affine transformation on an irregular image area surrounded by a plurality of candidate corner points to obtain a regular image area; the regular image is taken as a building facade image.

And carrying out translation, rotation, scale transformation and the like on an irregular image area surrounded by a plurality of candidate corner points so as to carry out image correction and texture correction on the irregular image area and obtain a regular image area.

Other methods may be used to identify the outer facade of the target building in the building video, and a building outer facade image of the target building is obtained.

203. Determining an edge of the target building in the building facade image;

the edge of the target building in the building outer elevation image refers to the outline of the target building in the building outer elevation image, and for the target building, the photographed building outer elevation image is different due to the position and the pose of the video acquisition equipment, namely the edge of the target detection object in different building outer elevation images is also different.

In this embodiment, determining an edge of a target building in a building facade image specifically includes:

binarizing the building outer elevation image; and performing edge detection according to the binarized building outer elevation image to obtain the edge of the target building in the building outer elevation image.

The binarization process of the building outer elevation image is an image segmentation process of the building outer elevation image, the binarization of the building outer elevation image is that the gray value of a pixel point on the building outer elevation image is set to be 0 or 255, the building outer elevation image shows obvious black and white effect, and each pixel of the binarized building outer elevation image has only two values: either pure black or pure white;

in the present embodiment, the building facade image may be binarized by using a threshold-based segmentation algorithm, a region-based segmentation algorithm, or the like, which is not particularly limited in this application;

according to the obtained binarized building facade image, a connected component can be searched in the image to obtain a color block where the target building is located, and edge detection is performed on the color block obtained in the binarized building facade image to obtain an edge of the target building in the building facade image.

Binarization is carried out on the building outer elevation image, then edge detection is carried out, and the edge of the target building in the building outer elevation image, which is more reasonable and has no burrs, can be obtained, so that the accuracy of the integrity detection of the target building is ensured.

204. And determining whether the target building is complete in the building facade image according to the edges.

In this embodiment, after determining the edge of the target building in the building facade image, the external contour shape of the target building can be constructed based on the edge of the target building in the building facade image, and whether the target building is complete in the building facade image can be determined according to the external contour shape;

in this embodiment, the building facade image may be used as input, and whether the target building is complete in the building facade image is detected by the trained building facade integrity detection model, and a result of whether the target building is complete in the building facade image is output.

In another embodiment, it is also possible to determine whether the target building is complete in the building facade image according to the edges in a manually discernable manner, in particular: when the target building is set to be a regular cuboid building and the outer surface of the target building has no other decorations, the outer contour shape of the target building is linear; if the external contour shape of the target building in the building outer elevation image is detected to be linear according to the edges, the external contour shape of the target building can be judged to be normal, namely the target building is determined to be complete in the building outer elevation image; if the middle of the outer contour shape of the target building in the building outer elevation image is detected to be in an irregular bending shape according to the edges, and the outer contour shape of the target building can be judged to be abnormal if the irregular bending shape is in a straight line shape, namely the target building is determined to be incomplete in the building outer elevation image;

In this embodiment, the integrity determination result of the target building still needs to be manually verified whether the integrity determination result is accurate later, if so, the integrity determination result is directly used as a detection result, if not, the integrity determination is still needed to be manually performed until the accurate detection result is detected, but compared with a mode of checking the integrity of the target building by a gridding member periodically to the site, the efficiency of the scheme provided by this embodiment is still higher.

In this embodiment, in order to further improve the integrity detection efficiency of the target building, determining whether the target building is complete in the building facade image according to the edges may specifically include the following steps 401 to 404:

401. and calculating the graphic area of the graphic formed by the edges of the target building in the building facade image.

The graphic area of the graphic formed by the edge in the building facade image represents the area actually occupied by the target building in the building facade image, and the graphic area can be expressed by the number of pixels in the graphic enclosed by the edge in the building facade image.

402. The edges are line fitted to obtain a plurality of line equations representing the edges of the target building in the building facade image.

In this embodiment, the edge of the target building in the building facade image is typically formed by N straight line segments in succession, N being a natural number; in this embodiment, when the edge of the target building in the building facade image is an N-polygon, N straight-line segments in the edge may be represented by N straight-line equations, specifically, N intersecting points between the N straight-line equations may be determined and selected by the angle and the positional relationship between the N straight-line equations, the N intersecting points may form N vertices of an N-polygon edge, and N straight-line equations corresponding to the edge are represented by coordinates of the N intersecting points, where the number of straight-line equations of the edge is the same;

illustratively, in one edge, when coordinates of two intersections between one straight line segment and two adjacent straight line segments of the edge are (x 1, y 1) and (x 2, y 2), respectively, then the straight line equation corresponding to the straight line segment is (y-y 2)/(y 1-y 2) = (x-x 2)/(x 1-x 2); the present embodiment may also perform straight line fitting on the edges in other manners to obtain a plurality of straight line equations representing the edges of the target building in the building facade image, which is not particularly limited.

403. The area of the facade area formed by the straight lines expressed by the straight line equations is calculated.

When the edge is represented by N straight-line equations, calculating the area of an N-sided polygon surrounded by the straight lines represented by the N straight-line equations, and when the N straight-line equations are represented by coordinates of N intersection points, the N intersection points form an N-sided polygon taking the N-sided polygon as a vertex, namely the N straight-line equations represent the area surrounded by the N-sided polygon; and calculating the area of the outer elevation area of the N-sided polygon surrounded by the N linear equations.

404. And determining whether the target building is complete in the building facade image according to the graph area and the facade area.

In this embodiment, after the graphic area of the graphic formed by the edges in the building facade image and the facade area of the edges in the building facade image are calculated, the graphic area and the facade area are compared, and whether the areas are the same is judged, if the areas are the same or are within the preset area ratio range, the target building is shown to be complete in the building facade image, or else, the target building is not complete.

Therefore, according to the graphic area and the facade area, determining whether the target building is complete in the building facade image specifically comprises the following steps:

Calculating the area ratio of the graphic area and the facade area, and determining that the target building is complete in the building facade image when the area ratio is greater than a preset threshold.

In the embodiment, the preset threshold value can be set according to actual needs, and the preset threshold value is set to any value of 80% -90% by way of example; when the area ratio of the graphic area to the facade area is larger than any one of 80% -90%, determining that the target building is complete in the building facade image, otherwise, determining that the target building is incomplete.

In another embodiment of the present application, whether the target building is complete is determined based on an edge in any one of the image frames in the image frame set, or whether the target building is complete is determined based on an edge in the or each of the plurality of image frames in the image frame set, after determining whether the target building is complete in the building facade image based on the edge, the method further comprises:

when the target building is complete in the building facade image, storing the building facade image into a preset facade complete sample library;

the building exterior complete sample library is used for storing sample image frame integration sets of building sample videos, the sample image frame integration sets in the building exterior complete sample library can be used for training a target building detection model and a building exterior integrity detection model, when a target building is complete in building exterior images, the building exterior images are stored in the building exterior complete sample library, continuous accumulation of model training data is achieved, and therefore the target building detection model and the building exterior integrity detection model can be iterated continuously. And when the target building is not complete in the building outer elevation image, sending warning information of potential safety hazards of the target building to the target receiving terminal.

When the fact that the target building is not complete in the building outer elevation image is determined, warning information of potential safety hazards is sent to the target receiving terminal, so that potential safety hazards of the target building of the opposite side can be informed in time, and potential safety hazard investigation efficiency of the target building is improved; in this embodiment, the target receiving terminal may be another electronic device of a different type, such as a server device, a physical host, or a user device, for receiving the warning information, where the electronic device for receiving the warning information and the electronic device for implementing the building facade integrity detection method may perform information transmission through any existing information transmission manner, which is not limited in this embodiment.

In another embodiment of the present application, the target building detection model may be trained by:

training by taking an efficientnet as a backbone network and taking a yolox or anchor free target detection model as a model to be trained, taking a sample image frame set of a building sample video stored in an outer vertical surface complete sample library as input of the model to be trained, and taking an image frame containing a building and a building detection frame as output to perform model training to obtain a target building detection model;

In the training process, the modeling capability of the model to be trained can be improved by adopting a data enhancement mode, wherein the data enhancement mode specifically comprises the following modes but not limited to the following modes:

(1) Random cropping (Random Crop) is performed on the sample image frames in the sample image frame set, specifically, cropping is performed on the sample image frames in a region with a Random ratio of 0.6-1.0, and the cropped sample image frames are used as input of a model to be trained;

(2) Embedding a Dropblock layer in a network, namely, a backbone network comprises a plurality of inner winding layers, a pooling layer and a Dropblock layer, wherein in the Dropblock layer, the discarding probability of a neighborhood space pixel point with the area size of K multiplied by R in a discarding feature map is p, and the Dropblock layer is used for setting the discarding probability of a neighborhood space pixel point with the area size of 3 multiplied by 3 in the discarding feature map to be 0.1;

(3) The Mosaic mosaics are adopted to realize data enhancement, specifically, four sample image frames in a sample image frame set are spliced into one Mosaic image randomly, and a new sample image obtained by splicing is used as training data to be used as input for model training.

In this embodiment, multi-scale training (Multi Scale Training, MST) is used to reduce the risk of model overfitting and enhance the robustness of the target building detection model.

In another embodiment of the present application, the building facade integrity test model may be trained by the following steps:

training by taking an efficientnet as a backbone network and taking a yolox or anchor free target detection model as a model to be trained, taking a building outer elevation sample image aggregate as input of the model to be trained, taking a building outer elevation sample image and a building complete label as output, and performing model training to obtain a building outer elevation integrity detection model;

in the training process, the modeling capability of the model to be trained can be improved by adopting a data enhancement mode, wherein the data enhancement mode specifically comprises the following modes but not limited to the following modes:

(1) Random cropping (Random Crop) is performed on the building outer elevation sample images in the building outer elevation sample image collection, specifically, cropping is performed on the area with the Random ratio of 0.6-1.0 on the building outer elevation sample images, and the cropped building outer elevation sample images are used as input of a model to be trained;

(2) Embedding a Dropblock layer in a network, namely, a backbone network comprises a plurality of inner winding layers, a pooling layer and a Dropblock layer, wherein in the Dropblock layer, the discarding probability of a neighborhood space pixel point with the area size of K multiplied by R in a discarding feature map is p, and the Dropblock layer is used for setting the discarding probability of a neighborhood space pixel point with the area size of 3 multiplied by 3 in the discarding feature map to be 0.1;

(3) The Mosaic mosaics are adopted to realize data enhancement, specifically, four building facade sample images in a building facade sample image collection set are spliced into one Mosaic image randomly, and a new sample facade image obtained by splicing is used as training data to be used as input for model training.

In this embodiment, multi-scale training (Multi Scale Training, MST) is employed to reduce the risk of model overfitting and enhance the robustness of the building facade integrity detection model.

It should be noted that, besides the integrity detection of the target building provided by the present application, in many practical application scenarios, it is required to perform the integrity detection on a certain target object to be detected, that is, determine whether a certain target object to be detected is complete in a video or an image in many practical application scenarios, where the certain target object to be detected may be a plant branch, a road sign board, etc. in a street view, or may be various articles in daily life, such as a vase, a cabinet, a display, etc., so in an application process, the integrity detection of more target objects to be detected may be completed by the building integrity detection method provided by the present application.

In order to better implement the building integrity detection method according to the embodiment of the present application, on the basis of the building integrity detection method, a building integrity detection device is further provided in the embodiment of the present application, as shown in fig. 3, where the building integrity detection device 500 includes:

an acquisition module 501 for acquiring a building video including a target building;

the identifying module 502 is configured to identify a building facade of a target building in a building video, and obtain a building facade image of the target building;

an edge determination module 503 for determining an edge of the target building in the building facade image, an

An integrity determination module 504 is configured to determine whether the target building is complete in the building facade image based on the edges.

The identification module 502 is also specifically:

the method comprises the steps of detecting a target building for a building video to generate a building detection frame;

the method comprises the steps of intercepting an initial image of a building in a corresponding area according to a building detection frame;

the method comprises the steps of determining a plurality of candidate corner points corresponding to a target building in an initial image of the building;

for generating a building facade image from a plurality of candidate corner points.

The identification module 502 is also specifically:

Detecting a plurality of initial corner points in an initial image of a building by using a Harris corner point detector;

the method is used for filtering a plurality of initial corner points which are not near the plurality of vertexes of the building detection frame in the initial image of the building according to the Euler distances of the plurality of vertexes of the building detection frame, and taking the rest of the plurality of initial corner points in the initial image of the building as a plurality of candidate corner points.

The identification module 502 is also specifically:

affine transformation is carried out on an irregular image area surrounded by a plurality of candidate angular points to obtain a regular image area;

for taking the regular image as a building facade image.

The edge determining module 503 is also specifically:

the method is used for binarizing the building outer elevation image; and

and the edge detection is used for carrying out edge detection according to the binarized building outer elevation image to obtain the edge of the target building in the building outer elevation image.

The integrity determination module 504 is also specifically:

the method comprises the steps of calculating a graphic area of a graphic formed by edges of a target building in a building facade image;

for straight-line fitting the edges to obtain a plurality of straight-line equations representing the edges of the target building in the building facade image;

The method comprises the steps of calculating the area of an outer elevation area formed by straight lines expressed by a plurality of straight line equations;

and the method is used for determining whether the target building is complete in the building facade image according to the graphic area and the facade area.

The integrity determination module 504 is also specifically:

for calculating the area ratio of the graphic area to the area of the facade area

For determining that the target building is complete in the building facade image when the area ratio is greater than a preset threshold.

The building integrity detection device 500 further comprises a storage module 505 and a notification module 506,

a storage module 505, configured to store the building facade image to a preset facade complete sample library when the target building is complete in the building facade image;

and the notification module 506 is configured to send a warning message that the target building has a potential safety hazard to the target receiving terminal when the target building is not complete in the building facade image.

In another embodiment of the present application, as shown in fig. 4, the present application further provides an electronic device 600, which shows a schematic structural diagram of the electronic device according to the embodiment of the present application, specifically:

the electronic device may include one or more processing cores 'processors 601, one or more computer-readable storage media's memory 602, power supply 603, and input unit 604, among other components. Those skilled in the art will appreciate that the electronic device structure shown in fig. 4 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:

The processor 601 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 602, and calling data stored in the memory 602, thereby performing overall monitoring of the electronic device. Optionally, the processor 601 may include one or more processing cores; the processor 601 may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and preferably, the processor 601 may integrate an application processor primarily handling operating systems, user interfaces, application programs, and the like, with a modem processor primarily handling wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 601.

The memory 602 may be used to store software programs and modules, and the processor 601 may execute various functional applications and data processing by executing the software programs and modules stored in the memory 602. The memory 602 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide access to the memory 602 by the processor 601.

The electronic device further comprises a power supply 603 for supplying power to the various components, preferably the power supply 603 may be logically connected to the processor 601 by a power management system, so that functions of charge, discharge, power consumption management and the like are managed by the power management system. The power supply 603 may also include one or more of any components, such as a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may further comprise an input unit 604, which input unit 604 may be used for receiving input digital or character information and for generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 601 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 executes the application programs stored in the memory 602, so as to implement various functions as follows:

acquiring a building video containing a target building;

identifying a target building outer elevation in a building video to obtain a building outer elevation image of the target building;

determining edges of a target building in a building facade image, and

and determining whether the target building is complete in the building facade image according to the edges.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

In some embodiments of the present application, the present application also provides a computer-readable storage medium, which may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like. On which a computer program is stored which is loaded by a processor for performing the steps in the building integrity detection method provided by the embodiments of the present application. For example, the loading of the computer program by the processor may perform the steps of:

acquiring a building video containing a target building;

identifying a target building outer elevation in a building video to obtain a building outer elevation image of the target building;

determining edges of a target building in a building facade image, and

and determining whether the target building is complete in the building facade image according to the edges.

In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of one embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description of other embodiments, which are not described herein again.

The foregoing has described in detail a method, apparatus, electronic device and storage medium for detecting building integrity provided by the embodiments of the present application, and specific examples have been applied herein to illustrate the principles and embodiments of the present invention, where the foregoing examples are provided to assist in understanding the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.

Claims (10)

1. A method of building integrity testing comprising:

acquiring a building video containing a target building;

identifying the outer facade of the target building in the building video to obtain a building outer facade image of the target building;

determining an edge of the target building in the building facade image, an

And determining whether the target building is complete in the building facade image according to the edge.

2. The building integrity detection method of claim 1 wherein said identifying said target building facade in said building video results in a building facade image of said target building comprising:

detecting the target building for the building video to generate a building detection frame;

intercepting an initial image of a building in a corresponding area according to the building detection frame;

determining a plurality of candidate corner points corresponding to the target building in the initial building image;

and generating the building outer elevation image according to the candidate corner points.

3. The building integrity detection method of claim 2, wherein the determining a plurality of candidate corner points in the initial image of the building corresponding to the target building comprises:

Detecting a plurality of initial corner points in the initial image of the building by using a Harris corner point detector;

and filtering a plurality of initial corner points which are not near the plurality of vertexes of the building detection frame in the initial image of the building according to Euler distances between the plurality of initial corner points and the plurality of vertexes of the building detection frame, and taking the rest of the initial corner points in the initial image of the building as a plurality of candidate corner points.

4. A building integrity detection method according to claim 3, wherein said generating said building facade image from said plurality of candidate corner points comprises:

affine transformation is carried out on an irregular image area surrounded by the candidate angular points, so as to obtain a regular image area;

and taking the regular image as the building outer elevation image.

5. The building integrity detection method of claim 1 wherein said determining edges of said target building in said building facade image comprises:

binarizing the building facade image; and

and performing edge detection according to the binarized building outer elevation image to obtain the edge of the target building in the building outer elevation image.

6. The building integrity detection method of claim 2 wherein said determining from said edge whether said target building is complete in said building facade image comprises:

calculating the graphic area of a graphic formed by the edges of the target building in the building outer elevation image;

performing a straight line fit to the edge to obtain a plurality of straight line equations representing the edge of the target building in the building facade image;

calculating the area of the outer elevation area formed by the straight lines expressed by the plurality of straight line equations;

and determining whether the target building is complete in the building facade image according to the graph area and the facade area.

7. The building integrity detection method of claim 6 wherein said determining whether said target building is complete in said building facade image based on said graphic area and said facade area comprises:

calculating an area ratio of the graphic area to the facade area

And when the area ratio is greater than a preset threshold value, determining that the target building is complete in the building facade image.

8. The building integrity detection method of claim 1, wherein after said determining from said edge whether said target building is complete in said building facade image, said method further comprises:

when the target building is complete in the building facade image, storing the building facade image into a preset facade complete sample library;

and when the target building is not complete in the building outer elevation image, sending warning information of potential safety hazards of the target building to a target receiving terminal.

9. A building integrity testing apparatus, the apparatus comprising:

the acquisition module is used for acquiring building videos containing the target building;

the identification module is used for identifying the outer facade of the target building in the building video to obtain a building outer facade image of the target building;

an edge determination module for determining an edge of the target building in the building facade image, an

And the integrity determining module is used for determining whether the target building is complete in the building facade image according to the edge.

10. A computer readable storage medium, having stored thereon a computer program, the computer program being loaded by a processor to perform the steps of the building integrity detection method of any one of claims 1 to 8.