CN114463624A

CN114463624A - Method and device for detecting illegal buildings applied to city management supervision

Info

Publication number: CN114463624A
Application number: CN202111598817.XA
Authority: CN
Inventors: 尹锴; 田亦陈; 袁超; 陈芳淼; 余堃; 郑家祥
Original assignee: Aerospace Information Research Institute of CAS
Current assignee: Aerospace Information Research Institute of CAS
Priority date: 2021-12-24
Filing date: 2021-12-24
Publication date: 2022-05-10

Abstract

The invention provides a method and a device for detecting illegal buildings applied to city management supervision, wherein the method comprises the following steps: inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model; acquiring a violation building detection result of a target area based on a building identification result of each remote sensing image to be identified; the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, and the difference feature map is obtained based on a result of image subtraction of the foreground enhancement feature maps of the target scales of every two remote sensing images to be identified. The method and the device for detecting the illegal buildings applied to city management supervision can detect the illegal buildings in a target area more accurately and efficiently.

Description

Method and device for detecting illegal buildings applied to city management supervision

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a method and a device for detecting a violation building, which are applied to city management supervision.

Background

With the development of social economy, the phenomenon of illegal construction is endless, and illegal buildings for illegal construction erode public land resources, which brings serious threats to the living environment, the safe environment, the ecological environment, the public security environment, the social environment and the like of people. Based on the development and application requirements of serial high-resolution satellite remote sensing data produced in China, illegal buildings are timely and accurately found, effective data support is provided for urban management supervision industry departments, and the method has very important significance.

In the existing method for detecting the illegal buildings, image data can be obtained based on various modes such as satellite remote sensing, aerial remote sensing, unmanned aerial vehicle remote sensing and the like, and further, the detection of the illegal buildings is realized by using a deep learning method. However, since the content of the image data is complex, and the image data has large intra-class difference and small inter-class difference, it is difficult to accurately obtain the detection result of the illegal building based on the conventional illegal building detection method.

Disclosure of Invention

The invention provides a method and a device for detecting a violation building, which are applied to city management and supervision and are used for solving the defect that the detection result of the violation building is difficult to accurately obtain in the prior art and realizing more accurate acquisition of the detection result of the violation building.

The invention provides a method for detecting a violation building, which is applied to city management supervision and comprises the following steps:

acquiring a plurality of remote sensing images to be identified in different time phases of a target area;

inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model;

acquiring a violation building detection result of the target area based on the building identification result of each remote sensing image to be identified;

the building identification model is obtained by training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images;

the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction performed on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

According to the illegal building detection method applied to city management supervision provided by the invention, the building identification model comprises the following steps: the system comprises a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer;

correspondingly, the inputting each remote sensing image to be identified into a building identification model, and obtaining the building identification result of each remote sensing image to be identified output by the building identification model specifically comprises:

inputting each remote sensing image to be identified into the feature extraction layer, and acquiring a first feature map and a second feature map of each remote sensing image to be identified in target scale, which are output by the feature extraction layer;

inputting the first characteristic diagram and the second characteristic diagram of each target scale of the remote sensing image to be recognized into the semantic segmentation layer, and acquiring a foreground enhancement characteristic diagram of each target scale of the remote sensing image to be recognized, which is output by the semantic segmentation layer;

inputting the foreground enhancement feature map of each target scale of the remote sensing image to be identified into the change detection layer, and generating a difference feature map of the target scale corresponding to each two remote sensing images to be identified by the result of image subtraction based on the foreground enhancement feature maps of each two remote sensing images to be identified by the change detection layer, so as to obtain the difference feature map of the target scale corresponding to each two remote sensing images to be identified, which is output by the change detection layer;

and inputting the foreground enhancement feature map of each target scale of the remote sensing images to be identified and the difference feature map of the target scale corresponding to each two remote sensing images to be identified into the result output layer, and acquiring the building identification result of each remote sensing image to be identified, which is output by the result output layer.

According to the illegal building detection method applied to city management supervision, provided by the invention, a loss function dynamic weighting loss function, a semantic segmentation loss function and a change detection loss function of a building identification model are adopted;

the dynamic weighting loss function is determined based on a preset maximum iteration number and a current iteration number of a building identification model in training, a predetermined highest weight and a predetermined lowest weight, the semantic segmentation loss function and the change detection loss function;

the semantic segmentation loss function and the change detection loss function are determined based on a predicted building recognition result of each sample image and a building recognition result label of each sample image, wherein the predicted building recognition result of each sample image is input into the building recognition model in training and output by a result output layer in the building recognition model in training;

the dynamic weighting loss function is used for adjusting a first weight corresponding to the semantic segmentation loss function to be reduced along with the increase of the iteration number of the building identification model in the training process of the building identification model, and a second weight corresponding to the change detection loss function to be increased along with the increase of the iteration number of the building identification model in the training process.

According to the illegal building detection method applied to city management supervision provided by the invention, whether illegal buildings exist in the target area or not is judged based on the building identification result, and the method specifically comprises the following steps:

for each two remote sensing images to be identified, judging whether a building is newly added in the compared remote sensing images to be identified based on the building identification result and a GIS space analysis method;

under the condition that buildings are newly added in the remote sensing images to be identified after the comparison between every two remote sensing images to be identified, judging whether each newly added building is a violation building or not based on the functional area element map layer corresponding to the target area;

and the functional area element layer is obtained by performing buffer area analysis based on the original functional area element layer obtained in advance.

According to the method for detecting the illegal buildings applied to city management supervision provided by the invention, whether each newly added building is the illegal building is judged based on the functional area element map layer corresponding to the target area, and the method specifically comprises the following steps:

acquiring the probability that each newly added building is a violation building based on the functional area element layer corresponding to the target area;

and under the condition that the probability that any newly added building is a violation building is greater than a preset threshold value, determining that the newly added building is the violation building.

According to the violation building detection method applied to city management supervision provided by the invention, the semantic segmentation layer comprises the following steps: a foreground feature enhancement layer and a spatial detail recovery layer;

correspondingly, the inputting the first feature map and the second feature map of each target scale of the remote sensing image to be recognized into the semantic segmentation layer to obtain the foreground enhancement feature map of each target scale of the remote sensing image to be recognized, which is output by the semantic segmentation layer, specifically includes:

inputting the first feature map and the second feature map of each target scale of the remote sensing image to be recognized into the foreground feature enhancement layer, generating a target vector corresponding to each remote sensing image to be recognized by the foreground feature enhancement layer based on the first feature map of each remote sensing image to be recognized, acquiring an inner product of the target vector and the second feature map of the target scale of the remote sensing image to be recognized as a semantic segmentation feature map of the target scale of the remote sensing image to be recognized, and further acquiring a semantic segmentation feature map of each target scale of the remote sensing image to be recognized, which is output by the foreground feature enhancement layer;

and inputting the semantic segmentation feature map of each target scale of the remote sensing image to be recognized into the spatial detail recovery layer, and generating a foreground enhancement feature map of each target scale of the remote sensing image to be recognized by the spatial detail recovery layer through recovering the spatial detail of the semantic segmentation feature map of each target scale of the remote sensing image to be recognized, thereby obtaining the foreground enhancement feature map of each remote sensing image to be recognized, which is output by the spatial detail recovery layer.

According to the illegal building detection method applied to city management supervision, the target scale comprises a plurality of scales.

The invention also provides a violation building detection device applied to city management supervision, which comprises the following components:

the data acquisition module is used for acquiring a plurality of remote sensing images to be identified in different time phases of the target area;

the image identification module is used for inputting each remote sensing image to be identified into a building identification model and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model;

the illegal building detection module is used for acquiring illegal building detection results of the target area based on the building identification results of the remote sensing images to be identified;

The invention also provides electronic equipment comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the illegal building detection method applied to city management supervision.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of any of the above-described method for detecting a violation building for use in city management supervision.

The invention also provides a computer program product comprising a computer program which when executed by a processor carries out the steps of any of the above mentioned violation building detection methods applied in city management supervision.

The invention provides a method and a device for detecting illegal buildings applied to city management supervision, which are used for respectively identifying buildings for every two remote sensing images to be identified by acquiring a foreground enhanced feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified based on a trained building identification model, acquiring building identification results of each remote sensing image to be identified including building identification results of every two remote sensing images to be identified, and acquiring illegal building detection results of a target area based on the building identification results of each remote sensing image to be identified, thereby realizing automatic extraction of buildings and changed buildings in the remote sensing images to be identified in front and back time phases, more accurately detecting illegal buildings in the target area, and simplifying the process of detecting illegal buildings, The method is more efficient, can reduce the investment of manpower and material resources, and reduces the cost investment of illegal building detection.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart of a violation building detection method applied to city management supervision provided by the invention;

FIG. 2 is a second schematic flow chart of the illegal building detection method applied to city management supervision according to the present invention;

FIG. 3 is one of schematic diagrams of building identification results of remote sensing images to be identified in the violation building detection method applied to city management supervision according to the present invention;

FIG. 4 is a second schematic diagram of the building identification result of each remote sensing image to be identified in the violation building detection method applied to city management supervision provided by the present invention;

FIG. 5 is a schematic structural diagram of a building identification model in a violation building detection method applied to city management supervision according to the present invention;

FIG. 6 is a schematic diagram of a structure of a building identification model in the prior art;

FIG. 7 is a schematic structural diagram of a violation building detection device applied to city management supervision provided by the invention;

fig. 8 is a schematic structural diagram of an electronic device provided in the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the description of the invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

In general, conventional violation building detection methods may include the following:

firstly, manual regular inspection. A specially-assigned person regularly carries out manual inspection at a fixed point, and if illegal construction conditions are found, field recording is carried out;

and secondly, assisting visual judgment by utilizing a satellite remote sensing technology. The method comprises the steps of obtaining a high-resolution remote sensing image regularly, carrying out manual visual judgment on two remote sensing images in different periods under the same screen based on a remote sensing image display platform, carrying out vectorization drawing on a changed building serving as a suspected violation building, generating a suspected violation building pattern spot and synchronously uploading the pattern spot to a server;

and thirdly, assisting visual discrimination by utilizing the unmanned aerial vehicle technology. Judging whether a suspected violation building exists in the image or the video through manual visual observation by using the image or the video acquired in real time by the unmanned aerial vehicle aviation flight, and if so, automatically capturing a picture and uploading the picture to a server;

extracting buildings in the remote sensing images of the front time phase and the rear time phase by using a traditional remote sensing interpretation method, obtaining a changed building pattern spot through manual visual observation or superposition subtraction, comparing the remote sensing images of the front time phase and the rear time phase to determine a newly added building appearing in the rear time phase, and judging whether the newly added building is a violation building;

fifthly, extracting buildings in the remote sensing images of the front time phase and the rear time phase by using a traditional deep learning method, obtaining changed building pattern spots through manual visual observation or superposition subtraction, performing GIS space analysis on the extracted buildings in the remote sensing images of the front time phase and the rear time phase to obtain new buildings of the changed building pattern spots, and judging whether the new buildings are illegal buildings or not;

and sixthly, other modes. Such as encouraging public inspection, unmanned aerial vehicle visual comparison, etc.

However, the above method has the following drawbacks:

firstly, the manual regular inspection and other technologies consume a large amount of manpower and material resources, and the workload is huge;

secondly, the remote sensing satellite technology or the unmanned aerial vehicle technology is used for assisting visual judgment, so that the workload of workers is reduced, the workers still need to visually interpret, and a large amount of time is needed for finding newly added buildings in the images;

thirdly, the method for extracting the buildings in the remote sensing images of the front time phase and the rear time phase by the traditional remote sensing interpretation method is low in precision, and further automatic comparison of changes among different buildings is difficult to perform, so that the method cannot be applied to practice;

on one hand, the remote sensing image content is complex, the intra-class difference is large, the inter-class difference is small, the false detection phenomenon is serious, and the explicit enhancement foreground characteristic and the difference between the foreground characteristics of different time phases are not considered; on the other hand, the recognition result is greatly influenced by registration, illumination, viewing angle, antenna, sensor difference and other factors, and the recognition result may have "pseudo-change" of the building in the remote sensing image of the front and rear time phases.

Therefore, the method for detecting the illegal buildings applied to city management supervision can explicitly enhance the foreground characteristics of the remote sensing images of the front time phase and the rear time phase and highlight the difference between the foreground characteristics of the remote sensing images of the front time phase and the rear time phase, so that the detection result of the illegal buildings can be obtained more accurately.

Fig. 1 is a schematic flow chart of a method for detecting a violation building applied to city management supervision provided by the invention. The following describes a violation building detection method applied to city management supervision according to the present invention with reference to fig. 1. As shown in fig. 1, the method includes: step 101, obtaining a plurality of remote sensing images to be identified in different time phases of a target area.

The violation building detection method applied to city management supervision detects whether a violation building exists in the target area to obtain a violation building detection result of the target area.

Optionally, the remote sensing technology has the advantages of wide observation range, large information amount, quick information acquisition, short updating period, manpower and material resource saving, few man-made interference factors and the like, can solve the problem of difficult evidence obtaining of historical violation construction, and improves the timeliness and accuracy of finding the violation buildings. The remote sensing image to be identified in the embodiment of the invention can be a long-time sequence high-resolution remote sensing image with high revisit.

Optionally, at least two remote sensing images to be identified of different phases of the target area can be acquired through the high-resolution satellite. The high resolution satellite may be: the domestic 'high-score' series satellites in China can also be IKONOS satellites, GeoEye satellites, Quickbird satellites, WorldView series foreign commercial satellites and the like.

102, inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model; the building identification model is obtained after training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images; the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

It should be noted that, when the number of the remote sensing images to be identified is greater than two, the building identification model may obtain the building identification result of each two remote sensing images to be identified, and output the building identification result as the building identification result of each remote sensing image to be identified.

For each two remote sensing images to be identified, the remote sensing image to be identified which is to be identified and is in the front of the time phase in the two remote sensing images to be identified can be called a first remote sensing image to be identified, and the remote sensing image to be identified and is in the back of the time phase can be called a second remote sensing image to be identified.

Fig. 2 is a second schematic flow chart of the illegal building detection method applied to city management supervision according to the present invention. As shown in fig. 2, before each remote sensing image to be recognized is input into the building recognition model and the building recognition result of each remote sensing image to be recognized output by the building recognition model is obtained, the building recognition model may be trained in advance to obtain a trained building recognition model.

Optionally, the building identification model may be trained by first obtaining multiple high-resolution remote sensing images of different sample regions in different time phases, where the resolution of each high-resolution remote sensing image is the same, processing each high-resolution remote sensing image, obtaining multiple sample images of different sample regions in different time phases and building identification result labels of each sample image, and constructing a building sample library. The changed buildings may include an extension/reconstruction building, an additional building, and a lost building. Then, the building identification model can be trained based on the sample images of the sample regions in different time phases and the building identification result labels of the sample images, so as to obtain the trained building identification model.

It should be noted that the building identification model may be constructed based on a twin neural network (Siamese). The building identification model is of a U-shaped structure.

Optionally, the specific process of obtaining the sample image by processing the high-resolution remote sensing image of the sample area may include: image preprocessing and sample preparation. The image preprocessing can comprise preprocessing such as geometric correction, fusion, registration, mosaic and cutting of the high-resolution remote sensing image. The sample preparation may include labeling the building vector layer and the changed building vector layer on the preprocessed high-resolution remote sensing image, and cutting the labeled high-resolution remote sensing image to the same size as the unmarked high-resolution remote sensing image. Preferably, the cropping size may be 512 pixels by 512 pixels. The high-resolution remote sensing images with good quality can be selected for cleaning, and the cleaned high-resolution remote sensing images are subjected to horizontal overturning, vertical overturning, rotating, transposing, random cutting and resampling, random brightness, random contrast, Gaussian noise, Gaussian blur and the like, so that sample images of a plurality of sample areas in different time phases and building identification result labels of the sample images are obtained.

After the trained building identification model is obtained, the remote sensing images to be identified can be input into the trained building identification model, and the trained building identification model can output the building identification result of the remote sensing images to be identified from end to end.

For every two remote sensing images to be identified, the trained building identification model can be used for obtaining a feature map of a first remote sensing image target scale to be identified and a feature map of a second remote sensing image target scale to be identified, obtaining a foreground enhancement feature map of the first remote sensing image target scale by performing foreground feature enhancement on the feature map of the first remote sensing image target scale to be identified, obtaining a foreground enhancement feature map of the second remote sensing image target scale to be identified by performing foreground feature enhancement on the feature map of the second remote sensing image target scale to be identified, and obtaining a difference feature map of the target scale corresponding to the two remote sensing images to be identified based on the result of image subtraction between the foreground enhancement feature map of the first remote sensing image target scale and the foreground enhancement feature map of the second remote sensing image target scale to be identified, and further, building identification can be carried out on the two remote sensing images to be identified based on the foreground enhancement feature map of the first remote sensing image target scale to be identified, the foreground enhancement feature map of the second remote sensing image target scale to be identified and the difference feature maps of the target scales corresponding to the two remote sensing images to be identified, and building identification results of the two remote sensing images to be identified are obtained and output.

It should be noted that the target dimension may include one or more dimensions. And under the condition that the target scale comprises a plurality of scales, the scales of the feature map of the remote sensing image to be identified, the foreground feature enhancement feature map and the difference feature map corresponding to every two remote sensing images to be identified have a corresponding relation.

It should be noted that the building identification result of each remote sensing image to be identified may include the building identification result of every two remote sensing images to be identified. The building identification result of each two remote sensing images to be identified can comprise the shape and the position information of each building in the first remote sensing image to be identified, the shape and the position information of each building in the second remote sensing image to be identified, and the shape and the position information of the building which is changed in the first remote sensing image to be identified and the second remote sensing image to be identified.

It can be understood that the building identification result of each two remote sensing images to be identified output by the building identification model can comprise a building pattern spot in the first remote sensing image to be identified, a building pattern spot in the second remote sensing image to be identified, and a building pattern spot with a change in the first remote sensing image to be identified and the second remote sensing image to be identified. The building pattern spots in the first remote sensing image to be recognized can indicate the shape and the position information of each building in the first remote sensing image to be recognized, the building pattern spots in the second remote sensing image to be recognized can be used for indicating the shape and the position information of each building in the second remote sensing image to be recognized, and the building pattern spots changed in the first remote sensing image to be recognized and the second remote sensing image to be recognized can be used for indicating the shape and the position information of each building changed in the first remote sensing image to be recognized and the second remote sensing image to be recognized.

Fig. 3 is one of schematic diagrams of building identification results of remote sensing images to be identified in the violation building detection method applied to city management supervision provided by the present invention. As shown in fig. 3, a white spot may represent a building. The graph (a) is a building pattern spot in the first remote sensing image to be identified, the graph (b) is a building pattern spot in the second remote sensing image to be identified, and the graph (c) is a building pattern spot changed in the first remote sensing image to be identified and the second remote sensing image to be identified. As shown in fig. 3, compared to the first remote sensing image to be recognized, there is no building that has changed in the second remote sensing image to be recognized.

Fig. 4 is a second schematic diagram of the building identification result of each remote sensing image to be identified in the violation building detection method applied to city management supervision provided by the invention. As shown in fig. 4, a white spot may represent a building. The graph (a) is a building pattern spot in the first remote sensing image to be identified, the graph (b) is a building pattern spot in the second remote sensing image to be identified, and the graph (c) is a building pattern spot changed in the first remote sensing image to be identified and the second remote sensing image to be identified. As shown in fig. 4, the second remote sensing image to be recognized has more buildings changed compared with the first remote sensing image to be recognized.

And 103, acquiring a violation building detection result of the target area based on the building identification result of each remote sensing image to be identified.

Specifically, the illegal building detection result of the target area can be obtained through various methods based on the building identification result of each remote sensing image to be identified, for example: and acquiring the detection result of the illegal building in the target area based on the building identification result of each remote sensing image to be identified by a GIS space analysis method.

The embodiment of the invention acquires and carries out building identification on every two remote sensing images to be identified respectively based on the foreground enhanced feature map of the target scale of each remote sensing image to be identified and the difference feature map of the target scale corresponding to every two remote sensing images to be identified based on the trained building identification model, acquires the building identification result of each remote sensing image to be identified including the building identification result of every two remote sensing images to be identified, acquires the illegal building detection result of the target area based on the building identification result of each remote sensing image to be identified, can realize automatic extraction of buildings and changed buildings in the remote sensing images to be identified in front and back time phases, can more accurately carry out illegal building detection on the target area, has simpler and more efficient process for detecting the illegal buildings, and can reduce the investment of manpower and material resources, the cost investment of illegal building detection is reduced.

Based on the content of the above embodiments, the target scale includes a plurality of scales.

Specifically, the remote sensing image to be identified may include buildings of a plurality of different dimensions, for example: the building targets a low building and a closely-packed row of large plants. In order to improve the accuracy of building identification on each remote sensing image to be identified, the target scale in the embodiment of the invention can comprise a plurality of scales, and the trained building identification model can obtain the foreground enhancement feature map of each remote sensing image to be identified in a corresponding scale by obtaining the feature maps of each remote sensing image to be identified in a plurality of scales, so that the difference feature map of each remote sensing image to be identified in a corresponding scale can be obtained, and the building identification on each remote sensing image to be identified can be more accurately realized.

According to the embodiment of the invention, the characteristic maps of multiple scales of each remote sensing image to be recognized are obtained based on the trained building recognition model, the foreground enhancement characteristic map of the corresponding scale of each remote sensing image to be recognized is obtained, and the difference characteristic map of the corresponding scale corresponding to each two remote sensing images to be recognized is further obtained, so that the condition that a building with a small scale or a building with a larger scale cannot be recognized when the building recognition is carried out on each remote sensing image to be recognized is avoided, and the building recognition of each remote sensing image to be recognized is more accurate.

Fig. 5 is a schematic structural diagram of a building identification model in a violation building detection method applied to city management supervision provided by the invention. As shown in fig. 5, the building recognition model includes: the system comprises a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer.

Correspondingly, inputting each remote sensing image to be identified into the building identification model, and obtaining the building identification result of each remote sensing image to be identified output by the building identification model, wherein the building identification result specifically comprises the following steps: and inputting each remote sensing image to be identified into the feature extraction layer, and acquiring a first feature map and a second feature map of each remote sensing image to be identified target scale output by the feature extraction layer.

It should be noted that, in the embodiment of the present invention, an example in which the target scale includes five scales is described. J can be used as the identifier of the target scale, and j is 1,2,3,4, 5.

Specifically, for each two remote sensing images to be identified, for buildings with different scales in the two remote sensing images to be identified, the Feature extraction layer in the embodiment of the present invention may generate and output a Pyramid-shaped first Feature map of each remote sensing image to be identified by using a top-down path and a transverse connection method of a Feature Pyramid Network (FPN). The first feature map can be enhanced through shallow spatial detail information and deep semantic information, so that spatial information, semantic information and multi-scale context information are better fused, a pyramid-shaped second feature map of each remote sensing image to be recognized is obtained, and building recognition results of the two remote sensing images to be recognized can be more accurately obtained based on the first feature maps and the second feature maps.

Optionally, the feature extraction layer may include a first encoder and a second encoder. The first encoder and the second encoder have the same structure. Compared with a common Resnet network, the first feature extraction layer and the second feature extraction layer in the embodiment of the invention can adopt a Resnest-50 network as an encoder, so that the calculation accuracy of the first encoder and the second encoder can be improved by introducing an attention mechanism into the first encoder and the second encoder.

In case the target scale comprises five scales, the first encoder and the second encoder comprise one layer of semantic header and four layers of residual block consisting of Conv-Relu-BatchNormalization, respectively.

Alternatively, for every two remote sensing images to be identified, I can be used_iRepresenting the two remote sensing images to be identified, wherein I represents the identification of the remote sensing image to be identified, I is 1,2, and I₁Representing a first remote sensing image to be identified, I₂Representing the second remote sensing image to be identified.

The first remote sensing image I to be identified₁Inputting the first encoder, and acquiring a first to-be-identified remote sensing image I output by the first encoder₁First feature map of target scale {

C

_(1,j)1,2,3,4,5 }. Wherein, { C_(1,j)And | j | -1, 2,3,4,5} is output by a layer of semantic header and a four-layer residual block in the first encoder.

The second remote sensing image I to be identified₂Inputting a second encoder, and acquiring a second remote sensing image I to be recognized output by the second encoder₂First feature map of target scale {

C

_(2,j)1,2,3,4,5 }. Wherein, { C_(2,j)And | j | -1, 2,3,4,5} is output by a layer of semantic header and a four-layer residual block in the second encoder.

It should be noted that, because the feature maps extracted from the semantic headers in the first layer of the first encoder and the second encoder have poor expression capability, the first feature map C extracted from the semantic header in the first layer of the first encoder is not used_(1,1)As input to the first decoder, the first feature map C extracted without the first layer semantic header of the second encoder_(2,1)As input to a second decoder.

Alternatively,the feature extraction layer may include a first decoder and a second decoder. The first decoder and the second decoder have the same structure. The first decoder and the second decoder can adopt the top-down converged context information of the FPN network and can adopt the jumper connection method of the Unet network to recover the {

C

_(1,j)2,3,4,5 and { C | j |_(2,j)J — 2,3,4,5 }.

Will { C_(1,j)If j is 2,3,4,5, the first to-be-identified remote sensing image I output by the first decoder can be obtained₁Second feature map of target scale { P_(1,j)|j＝2,3,4,5}。

Will { C_(2,j)If j is 2,3,4,5, the second to-be-recognized remote sensing image I output by the second decoder can be obtained₂Second feature map of target scale { P_(2,j)|j＝2,3,4,5}。

P_(i,j)Can be expressed by the following formula:

where ζ represents the jumper cross-connections achieved by 1 × 1 convolutional layers; Γ denotes a bilinear interpolation with an upsampling factor of 2.

And inputting the first characteristic diagram and the second characteristic diagram of each target scale of the remote sensing image to be recognized into the semantic segmentation layer, and acquiring the foreground enhancement characteristic diagram of each target scale of the remote sensing image to be recognized, which is output by the semantic segmentation layer.

Specifically, for every two remote sensing images to be identified, aiming at the characteristics of complex content, large intra-class difference and small inter-class difference in the remote sensing images to be identified, the semantic segmentation layer in the building identification model can adopt high-level semantic information to enhance the foreground characteristics, and improves the discrimination of the foreground characteristics by associating the context related to the geographic space scene, so that the first characteristic diagram { C of the target scale of the two remote sensing images to be identified can be based on_(i,j)1,

2j

2,3,4,5 and a second profile { P }_(i,j)Reinforcing foreground characteristic with 1,

2j

2,3,4,5 to reinforce the above twoAnd generating and outputting a foreground enhancement feature map { Z } of the target scale of the two remote sensing images to be identified according to the difference between the foreground feature and the background feature of the remote sensing images to be identified_(i,j)|i＝1,2j＝2,3,4,5}。

And inputting the foreground enhancement characteristic map of each to-be-identified remote sensing image target scale into a change detection layer, generating a difference characteristic map of the target scale corresponding to each two to-be-identified remote sensing images by the result of image subtraction based on the foreground enhancement characteristic maps of each two to-be-identified remote sensing image target scales through the change detection layer, and further acquiring the difference characteristic map of the target scale corresponding to each two to-be-identified remote sensing images output by the change detection layer.

Specifically, for every two remote sensing images to be identified, in order to highlight the difference between buildings in the first remote sensing image to be identified and the second remote sensing image to be identified, the change detection layer may adopt Sigmoid function to enhance the foreground characteristic map { Z ] of the target scale of the two remote sensing images to be identified_(i,j)Normalize i | 1,2j ═ 2,3,4,5} and obtain normalized { Z ] for each

target scale

_(1,j)2,3,4,5 and normalized Z_(2,j)Taking the subtraction result of the | j ═ 2,3,4,5} images as the difference feature map { D ] of the two remote sensing images to be identified at each target scale_j|j＝2,3,4,5}。D_jCan be expressed by the following formula:

D_j＝S[ζ(Z_(1,j))]-S[ζ(Z_(2,j))]j＝2,3,4,5

wherein S denotes a Sigmoid function.

Fig. 6 is a schematic structural diagram of a building identification model in the prior art. Fig. 6 shows a single-task learning framework in each of the diagrams (a), (b), (c), and (d), and a multi-task learning framework in each of the diagrams (e). As shown in fig. 6, the building identification model in the prior art does not consider the explicit enhancement foreground features and the differences between the foreground features in different time phases, but the building identification model in the present application may explicitly enhance the foreground features of each remote sensing image to be identified in the front and rear time phases based on the semantic segmentation layer and the change detection layer, and highlight the differences between the foreground features of each remote sensing image to be identified in the front and rear time phases, thereby implementing more accurate building identification on each remote sensing image to be identified, and further acquiring the detection result of the violating buildings in the target area based on the more accurate building identification result of each remote sensing image to be identified.

And inputting the foreground enhancement feature map of each target scale of the remote sensing image to be identified and the difference feature map of the target scale corresponding to each two remote sensing images to be identified into a result output layer, and acquiring the building identification result of each remote sensing image to be identified, which is output by the result output layer.

Specifically, for every two remote sensing images to be identified, the result output layer can adopt a dynamic weighting pyramid strategy to realize the foreground enhancement feature maps { Z ] of the two remote sensing images to be identified in each target scale _(i,j)1,

2j

2,3,4,5 and a difference feature map { D } corresponding to the two remote sensing images to be identified_jThe characteristics of 2,3,4 and 5 are fused, so that the loss of the buildings with smaller dimensions and the loss of the buildings with larger dimensions in the two remote sensing images to be identified can be avoided.

Optionally, Conv-Relu-BatchNormalization-Ubosample (upsilon-beta) may be adopted as the result output layer_j) Step by step recovery of { Z }_(i,j)1, | 2, 4,5} and { D |_jSpatial resolution of 2,3,4,5 | j | to Z_(1,2)And Z_(2,2)After the same, { Z ] is checked by 1 x 1 convolution _(i,j)1, | 2, 4,5} and { D |_jConvolving | j | 2,3,4,5} and performing feature fusion based on the weight corresponding to each target scale to obtain a fusion feature O of the first remote sensing image to be identified₁And the fusion characteristic O of the second remote sensing image to be identified₂And fusion characteristic O of the difference characteristic graph of the target scale corresponding to the two remote sensing images to be identified_1/2. Fusion characteristic O of remote sensing image to be recognized_iCan be expressed by the following formula:

fusion characteristic O of difference characteristic graph of target scale corresponding to every two remote sensing images to be identified_1/2Can be expressed by the following formula:

wherein, ω is_jRepresenting the weight corresponding to the target scale j; j denotes the number of scales included in the target scale, and in the case where the target scale includes 5 scales, J is 5.

Optionally, the result output layer may also adopt a Conv-Relu-BatchNormalization mode to construct a classifier for classifying O_iAnd O_1/2The number of channels in (2) is converted into two categories, representing buildings and non-buildings at a semantic segmentation level, and representing changed buildings and unchanged buildings at a change detection level. Using bilinear interpolation with an upsampling factor of 4_iAnd O_1/2Is recovered to₁Or I₂And outputting the building identification results of the two remote sensing images to be identified after the same spatial resolution.

It should be noted that, the result output layer may further perform post-processing on the building identification result of each to-be-identified remote sensing image, and then obtain and output the building identification result of each to-be-identified remote sensing image after the post-processing, so as to avoid occurrence of a salt and pepper phenomenon, and reduce false detection rate and false detection rate of the building identification model of each to-be-identified remote sensing image.

The post-processing of the building identification result of each remote sensing image to be identified by the result output layer may include: and (3) adopting a fully connected conditional random field and hole filling to optimize the boundary of each identified building outline and fill holes with smaller areas in the foreground so as to reduce the missing detection rate and the false detection rate of the building identification model.

The embodiment of the invention obtains the first characteristic diagram and the second characteristic diagram of each target scale of the remote sensing image to be recognized through the characteristic extraction layer in the building recognition model, obtains the foreground enhancement characteristic diagram of each target scale of the remote sensing image to be recognized based on the first characteristic diagram and the second characteristic diagram of each target scale of the remote sensing image to be recognized through the semantic segmentation layer in the building recognition model, obtains the difference characteristic diagram of each target scale corresponding to two remote sensing images to be recognized through the change detection layer in the building recognition model, obtains the building recognition result of the remote sensing image to be recognized based on the foreground enhancement characteristic diagram of each target scale of the remote sensing image to be recognized and the difference characteristic diagram of each target scale corresponding to two remote sensing images to be recognized through the result output layer in the building recognition model, and can pay more effective attention to the foreground characteristic difference in the remote sensing image to be recognized, the background characteristic difference in the remote sensing images to be recognized is ignored, the context information can be better considered, the accuracy of the obtained building recognition result of each remote sensing image to be recognized is improved, and the accuracy of detecting the buildings against the regulations in the target area is further improved.

Based on the content of the foregoing embodiments, the semantic division layer includes: a foreground feature enhancement layer and a spatial detail recovery layer.

Correspondingly, inputting the first feature map and the second feature map of each target scale of the remote sensing image to be recognized into the semantic segmentation layer, and acquiring the foreground enhancement feature map of each target scale of the remote sensing image to be recognized, which is output by the semantic segmentation layer, wherein the foreground enhancement feature map specifically comprises the following steps: inputting the first characteristic diagram and the second characteristic diagram of each target scale of the remote sensing image to be recognized into a foreground characteristic enhancement layer, generating a target vector corresponding to each remote sensing image to be recognized by the foreground characteristic enhancement layer based on the first characteristic diagram of each remote sensing image to be recognized, acquiring an inner product of the target vector and the second characteristic diagram corresponding to the target scale of the remote sensing image to be recognized as a semantic segmentation characteristic diagram of the target scale of the remote sensing image to be recognized, and further acquiring a semantic segmentation characteristic diagram of each target scale of the remote sensing image to be recognized, which is output by the foreground characteristic enhancement layer.

Specifically, for every two remote sensing images to be identified, the first feature map { C } of the target scale of each remote sensing image to be identified is used_(i,j)1,

2j

2,3,4,5 and a second profile { P }_(i,j)Inputting a foreground feature enhancement layer with i ═ 1,2j ═ 2,3,4,5 |, and pooling { C with richer semantic information by the foreground feature enhancement layer using global averaging_(i,5)Converting | i ═ 1,2} into each remote sensing image to be identified corresponding to each remote sensing imageTarget vector of { C_(i,6)|i＝1,2}。

Will { P_(i,j)The i | -1, 2j | -2, 3,4,5} is converted into the geographic scene space by convolution with 1 × 1 to obtain { P ″._(i,j)1,

2j

2,3,4,5, and { P'_(i,j)1, | 2j ═ 2,3,4,5} and { C_(i,6)Taking the inner product of 1 and 2 as a semantic segmentation feature map { R } of each remote sensing image to be recognized_(i,j)1,

2j

2,3,4,5, and then obtaining a semantic segmentation feature map { R } of each target scale of the remote sensing image to be recognized output by the foreground feature enhancement layer_(i,j)|i＝1,2j＝2,3,4,5}。R_(i,j)Can be expressed by the following formula:

R_(i,j)＝C_(i,6)⊙ζP′_(i,j)i＝1,2j＝2,3,4,5

wherein, an indicates an inner product.

And inputting the semantic segmentation feature map of each target scale of the remote sensing image to be identified into a spatial detail recovery layer, generating a foreground enhancement feature map of each target scale of the remote sensing image to be identified by restoring the spatial detail of the semantic segmentation feature map of each target scale of the remote sensing image to be identified through the spatial detail recovery layer, and further acquiring the foreground enhancement feature map of each target scale of the remote sensing image to be identified, which is output by the spatial detail recovery layer.

Specifically, semantic segmentation feature map { R ] of each target scale of the remote sensing image to be recognized is obtained_(i,j)Inputting i | -1, 2j | -2, 3,4,5} into a spatial detail recovery layer, and converting { R } into a spatial detail recovery layer_(i,j)The | -1, 2j ═ 2,3,4,5} is converted from the geographic scene space to the feature space by convolution with 1 × 1 to obtain { R ″._(i,j)1,2 j-2, 3,4,5, and adopting Sigmoid function pair { R'_(i,j)Normalizing | -1, 2j | -2, 3,4,5} to obtain normalized { R'_(i,j)1,2j 2,3,4,5, and using Sigmoid function pair { C_(i,j)Normalize i-1, 2 j-2, 3,4,5, and change normalized { C by jumper connection_(i,j)The channel number of | i ═ 1,2j ═ 2,3,4,5} is then normalized with { R'_(i,j)By adding i to 1,2j to 2,3,4,5, { C } can be recovered_(i,j)Obtaining spatial details of each target scale, | i ═ 1,2j ═ 2,3,4,5}, eachForeground enhancement map { Z) of target scale of remote sensing image to be identified_(i,j)|i＝1,2j＝2,3,4,5}。Z_(i,j)Can be expressed by the following formula:

Z_(i,j)＝S[ζ(R_(i,j))]+S[ζ(C_(i,j))]

wherein S denotes a Sigmoid function.

After the embodiment of the invention obtains the target vector corresponding to each remote sensing image to be identified based on the first characteristic diagram of the target scale of each remote sensing image to be identified through the foreground characteristic enhancement layer in the semantic segmentation layer, acquiring the inner product of the target vector and a second feature map corresponding to the target scale of the remote sensing image to be recognized as a semantic segmentation feature map of the target scale of the remote sensing image to be recognized, the space detail in the semantic segmentation feature map of each target scale of the remote sensing image to be recognized is recovered through the space detail recovery layer in the semantic segmentation layer, the foreground enhancement feature map of each target scale of the remote sensing image to be recognized is obtained, the foreground enhancement feature map of each target scale of the remote sensing image to be recognized can be obtained more efficiently and more accurately, and the accuracy of the building recognition result of each remote sensing image to be recognized, which is output by the building recognition model, can be further improved.

Based on the disclosure of the various embodiments described above, the loss function of the building identification model comprises a dynamically weighted loss function.

The dynamic weighting loss function is determined based on the preset maximum iteration times and the current iteration times of the building identification model in training, the preset highest weight and the preset lowest weight, the semantic segmentation loss function and the change detection loss function.

The semantic segmentation loss function and the change detection loss function are determined based on the predicted building recognition result of each sample image and the building recognition result label of each sample image, and the predicted building recognition result of each sample image is output by a result output layer in the building recognition model in training after each sample image is input into the building recognition model in training.

And the dynamic weighting loss function is used for adjusting the first weight corresponding to the semantic segmentation loss function to be reduced along with the increase of the iteration times of the building identification model in the training process of the building identification model, and the second weight corresponding to the change detection loss function to be increased along with the increase of the iteration times of the building identification model in the training process.

Specifically, in order to improve the accuracy of the building identification result of each remote sensing image to be identified output by the building identification model, the building identification model may be trained based on a dynamic weighted loss function, and the training target is the minimization of the dynamic weighted loss function.

The dynamic weighting loss function is determined based on a semantic segmentation loss function, a change detection loss function, predetermined highest and lowest weights, and a maximum number of iterations and a current number of iterations of the building identification model under training.

Optionally, the semantic segmentation loss function and the change detection loss function are constructed based on a DiceLoss function and a FocalLoss function. The DiceLoss function can be expressed by the following formula:

wherein X represents the predicted building identification result of each sample image; y represents the building identification result label of each sample image. The predicted building recognition result of each sample image is output from a result output layer in the building recognition model under training by inputting each sample image into the building recognition model under training.

The focallloss function can be expressed by the following formula:

L_focal＝-α(1-p_t)^γlog(p_t)

wherein α represents a weight controlling the positive and negative samples; gamma represents the weight of the control difficulty sample; p is a radical of_tThe probability is represented.

Semantic segmentation loss function L_segCan be expressed by the following formula:

L_seg＝L_dice+L_focal

change detection loss function L_cdCan be expressed by the following formula:

L_cd＝L_dice+L_focal

dynamic weighted loss function L_dwmCan be expressed by the following formula:

L_dwm＝Ψ_segL_cd+Ψ_cdL_seg

therein, Ψ_segRepresenting a semantic segmentation loss function L_segA corresponding first weight; Ψ_cdRepresenting a dynamic weighting loss function L_dwmA corresponding second weight; epoch_maxRepresenting a maximum number of iterations of the building identification model under training; epoch represents the current number of iterations of the building identification model under training; Ψ_maxRepresents a predetermined highest weight; Ψ_minRepresenting the lowest predetermined weight. Preferably Ψ_max＝1.5，Ψ_min＝0.5。

In the process of training the building identification model based on the dynamic weighting loss function, the semantic segmentation loss function L increases along with the increase of the current iteration times epoch of the building identification model in training_segCorresponding first weight Ψ_segFrom the highest weight Ψ_maxGradually decrease to the lowest weight Ψ_minChange detection loss function L_cdCorresponding second weight Ψ_cdFrom the lowest weight Ψ_minGradually increasing to the highest weight Ψ_max. The building recognition model is trained based on the dynamic weighting loss function, so that the gravity center of the training is gradually divided by the semantic segmentation loss function L_segBias to change detection loss function L_cdCapable of improving building recognition modelAnd (4) performance.

The building identification model is trained based on the dynamic weighting loss function, the training gravity center deviation semantic segmentation loss function can be ensured in the early stage of the training of the building identification model, the training gravity center deviation change detection loss function is ensured in the training candidate of the building identification model, and therefore the performance of the building identification model can be improved, the multi-task learning model formed by integrating a plurality of single-task learning models can be effectively avoided, and the accuracy of the single-task learning model is reduced.

Based on the content of each embodiment, whether a violation building exists in the target area is judged based on the building identification result, and the method specifically comprises the following steps: for each two remote sensing images to be identified, judging whether a building is newly added in the remote sensing images to be identified after the comparison in each two remote sensing images to be identified based on a building identification result and a GIS space analysis method; the functional area element layer is obtained by performing buffer area analysis based on a pre-obtained original functional area element layer.

As shown in fig. 2, the original functional area element map layer may include, but is not limited to, a cultivated land protected area element map layer, a scenic spot protected area element map layer, a river and lake protected area element map layer, an ecological protected area element map layer, and the like. The original function region element layer may be obtained from a related database in advance.

The buffer area analysis is performed on the original functional area element layer, so that different types of functional area element layers can be obtained, for example: ploughing land, scenic spot, river and lake, ecological protection area, etc. Based on the type of the target area, a functional area element layer corresponding to the target area can be determined.

Performing buffer analysis on the original function region element layer may include determining a buffer distance based on a level of a function region in the original function region element layer, and then performing buffer analysis on the original function region element layer based on the buffer distance. For example: when the buffer area analysis is carried out on the element layer of the river and lake protection area, the buffer distance can be set according to the grade of a river, and the buffer area analysis is carried out on the element layer of the river and lake protection area based on the buffer distance; alternatively, when performing buffer analysis on the map layer of the scenic-site protected area, the buffer distance may be set according to the level of the scenic-site name protected area, and the buffer analysis may be performed on the map layer of the scenic-site protected area based on the buffer distance.

After the building identification result of each remote sensing image to be identified output by the building identification model is obtained, for the building identification result of each two remote sensing images to be identified, each element in the building pattern spots changing in the first remote sensing image to be identified and the second remote sensing image to be identified can be traversed based on intersection analysis in superposition analysis in GIS space analysis, intersection analysis is carried out on each element and the building pattern spots in the second remote sensing image to be identified, elements with the intersection area larger than a preset first area threshold value are added into a newly added building pattern layer, and elements with the intersection area smaller than a preset second area threshold value are added into a vanishing building pattern layer.

And obtaining a new building layer and a lost building layer after all the elements are traversed. Based on the newly added building map layer, whether a newly added building is added in the second remote sensing image to be recognized can be judged, and the shape and position information of the newly added building can be obtained under the condition that the newly added building in the second remote sensing image to be recognized is determined.

It should be noted that the first area threshold and the second area threshold may be determined according to actual situations. In the embodiment of the present invention, the first area threshold and the second area threshold are not particularly limited.

And under the condition that the newly added buildings in the remote sensing images to be identified are compared when every two remote sensing images to be identified are determined, judging whether each newly added building is a violation building or not based on the functional area element map layer corresponding to the target area.

Specifically, under the condition that a new building in the second remote sensing image to be recognized is determined, the new building layer and the functional area element layer corresponding to the target area may be subjected to overlay analysis, and whether the new building in the second remote sensing image to be recognized is a violation building or not may be determined.

According to the embodiment of the invention, after determining the newly-added buildings in the remote sensing images to be identified after being compared in each two remote sensing images to be identified based on the building identification result of each remote sensing image to be identified, whether the newly-added buildings are illegal buildings is judged based on the functional area element layer corresponding to the target area, a scientific basis is provided for detecting the illegal buildings in the target area from the perspective of the functional area, whether the newly-added buildings are newly-added in the remote sensing images to be identified after being compared in each two remote sensing images to be identified can be determined only through one-time GIS space analysis, the influence of accumulated errors generated by multiple GIS space analyses can be avoided, and the accuracy of detecting the illegal buildings in the target area can be further improved.

Based on the content of each embodiment, whether the newly added building is a violation building is judged based on the functional area element map layer corresponding to the target area, which specifically includes: and acquiring the probability that each newly added building is a violation building based on the functional area element layer corresponding to the target area.

Specifically, based on the superposition analysis result of the newly added building layer and the functional area element layer corresponding to the target area, the probability that each newly added building in the second remote sensing image to be identified is a violation building can be obtained.

And under the condition that the probability that any newly added building is a violation building is greater than a preset threshold value, determining the newly added building as the violation building.

Specifically, after the probability that each newly added building in the second remote sensing image to be recognized is a violation building is obtained, for any newly added building, the probability that the newly added building is a violation building can be compared with a preset probability threshold.

And under the condition that the probability is larger than a preset probability threshold value, determining that the newly added building is a violation building.

It should be noted that the probability threshold in the embodiment of the present invention may be determined according to actual situations. The embodiment of the invention does not limit the specific value of the probability threshold.

According to the embodiment of the invention, the probability that each newly added building in each two remote sensing images to be identified is a violation building is obtained based on the functional area element image layer corresponding to the target area, and under the condition that the probability that any newly added building is a violation building is greater than the preset probability threshold value, the newly added building is determined to be the violation building, the detection result of the violation building in the target area can be quantized, the detection result of the violation building in the target area can be more intuitively represented, a worker can focus on the newly added building with higher probability, and the management efficiency of the worker on violation construction is improved.

Fig. 7 is a schematic structural diagram of a violation building detection device applied to city management supervision provided by the invention. The violation building detection device applied to city management supervision provided by the invention is described below with reference to fig. 7, and the violation building detection device applied to city management supervision described below and the violation building detection method applied to city management supervision provided by the invention described above can be referred to correspondingly. As shown in fig. 7, the apparatus includes: a data acquisition module 701, an image recognition module 702, and a violation detection module 703.

The data acquisition module 701 is used for acquiring a plurality of remote sensing images to be identified in different time phases of the target area.

And the image identification module 702 is used for inputting each remote sensing image to be identified into the building identification model and obtaining the building identification result of each remote sensing image to be identified output by the building identification model.

And the violation detection module 703 is configured to obtain a violation building detection result of the target area based on the building identification result of each remote sensing image to be identified.

The building identification model is obtained after training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images.

The building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

Specifically, the data acquisition module 701, the image recognition module 702, and the violation detection module 703 are electrically connected.

The embodiment of the invention acquires and carries out building identification on every two remote sensing images to be identified respectively based on the foreground enhanced feature map of the target scale of each remote sensing image to be identified and the difference feature map of the target scale corresponding to every two remote sensing images to be identified based on the trained building identification model, acquires the detection result of the buildings against the regulations in the target area based on the building identification result of each remote sensing image to be identified after acquiring the building identification result of each remote sensing image to be identified including the building identification result of each remote sensing image to be identified, can realize automatic extraction of the buildings and the changed buildings in the remote sensing images to be identified in front and back time phases, can carry out detection of the buildings against the regulations in the target area more accurately, has simpler and more efficient process of detecting the buildings against the regulations, and can reduce the investment of manpower and material resources, the cost investment of illegal building detection is reduced.

Fig. 8 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 8: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform a violation building detection method for city management supervision, the method comprising: acquiring a plurality of remote sensing images to be identified in different time phases of a target area; inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model; acquiring a violation building detection result of a target area based on the building identification result of each remote sensing image to be identified; the building identification model is obtained after training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images; the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program, the computer program being stored on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing a method for detecting a violation building applied to city management supervision provided by the above methods, the method comprising: acquiring a plurality of remote sensing images to be identified in different time phases of a target area; inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model; acquiring a violation building detection result of a target area based on the building identification result of each remote sensing image to be identified; the building identification model is obtained after training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images; the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a method for detecting a violation building applied to city management supervision provided by the above methods, the method comprising: acquiring a plurality of remote sensing images to be identified in different time phases of a target area; inputting each remote sensing image to be identified into a building identification model, and obtaining a building identification result of each remote sensing image to be identified, which is output by the building identification model; acquiring a violation building detection result of a target area based on the building identification result of each remote sensing image to be identified; the building identification model is obtained after training based on sample images of a plurality of sample regions in different time phases and building identification result labels of the sample images; the building identification model is used for obtaining and identifying buildings of the remote sensing images to be identified based on a foreground enhancement feature map of a target scale of each remote sensing image to be identified and a difference feature map of the target scale corresponding to every two remote sensing images to be identified, the foreground enhancement feature map of the target scale of the remote sensing images to be identified is obtained by performing foreground feature enhancement on the feature map of the target scale of the remote sensing images to be identified, and the difference feature map of the target scale corresponding to every two remote sensing images to be identified is obtained based on a result of image subtraction on the foreground enhancement feature map of the target scale of every two remote sensing images to be identified.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting illegal buildings applied to city management supervision is characterized by comprising the following steps:

2. The illegal building detection method for city management supervision according to claim 1 is characterized in that the building identification model comprises: the system comprises a feature extraction layer, a semantic segmentation layer, a change detection layer and a result output layer;

3. The method for detecting illegal buildings applied to city management supervision as claimed in claim 2, characterized in that the loss function of the building identification model is a dynamic weighting loss function, a semantic segmentation loss function and a change detection loss function;

4. The method for detecting illegal buildings applied to city management supervision according to claim 1 is characterized in that the step of judging whether illegal buildings exist in the target area based on the building identification result specifically comprises the following steps:

5. The method for detecting illegal buildings applied to city management supervision according to claim 4 is characterized in that whether each newly added building is a illegal building is judged based on the functional area element map layer corresponding to the target area, and the method specifically comprises the following steps:

6. The method for detecting illegal buildings applied to city management supervision of claim 2 is characterized in that the semantic segmentation layer comprises the following steps: a foreground feature enhancement layer and a spatial detail recovery layer;

7. The method for detecting the illegal building applied to city management supervision according to any one of claims 1 to 6, characterized in that the target scale comprises a plurality of scales.

8. The utility model provides a be applied to city management and supervise's illegal building detection device which characterized in that includes:

the violation detection module is used for acquiring violation building detection results of the target area based on the building identification results of the remote sensing images to be identified;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of the violation building detection method for city management supervision according to any of claims 1-7.

10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method for detecting a violation building for city management supervision according to any of claims 1 to 7.