CN111368615A

CN111368615A - Violation building early warning method and device and electronic equipment

Info

Publication number: CN111368615A
Application number: CN201910563579.5A
Authority: CN
Inventors: 吕军; 户军; 唐槐
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2019-06-26
Filing date: 2019-06-26
Publication date: 2020-07-03
Anticipated expiration: 2039-06-26
Also published as: CN111368615B

Abstract

The embodiment of the invention provides a method and a device for early warning of illegal buildings and electronic equipment. The method comprises the following steps: acquiring an image to be detected acquired by monitoring equipment; inputting an image to be detected into a preset default detection model to obtain a detection result output by the default detection model; when the detection result represents that the illegal buildings in the preset construction state exist in the image to be detected, obtaining a comparison result of the image to be detected and a preset reference image, and calculating a first early warning value of the image to be detected based on the comparison result and the detection result; and when the first early warning value is larger than the first early warning threshold value, generating first early warning information. Compared with the prior art, the scheme provided by the embodiment of the invention can find the illegal buildings existing on the actual buildings in time and effectively early warn the illegal buildings in the preset construction state.

Description

Violation building early warning method and device and electronic equipment

Technical Field

The invention relates to the technical field of video monitoring, in particular to a method and a device for early warning of buildings against regulations and electronic equipment.

Background

In recent years, various illegal buildings not only occupy public social resources and urban spaces and destroy the good images of cities, but also bring certain harm to the economic order and social management of markets, and are important problems to be solved urgently in the urban management process.

Once the violation buildings cannot be stopped and dismantled in time at the initial construction stage, the difficulty of subsequent dismantling work is increased, and higher dismantling cost is brought, and particularly for the established violation buildings, the dismantling difficulty and the dismantling cost are higher than those of the violation buildings, so that the requirement of early warning in time on the violation buildings existing on actual buildings exists.

Therefore, how to early warn the illegal buildings in time so as to reduce the dismantling difficulty and the dismantling cost becomes a problem to be solved urgently at present.

Disclosure of Invention

The embodiment of the invention aims to provide a violation building early warning method, a violation building early warning device and electronic equipment, so as to find the violation building existing in an actual building in time. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for early warning of a violation building, where the method includes:

acquiring an image to be detected acquired by monitoring equipment;

inputting the image to be detected into a preset default detection model to obtain a detection result output by the default detection model; wherein the default detection model is obtained by training based on a training sample and a sample label of the training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels of the training samples are: the construction state of the illegal building in the training sample;

when the detection result represents that the illegal building in the preset construction state exists in the image to be detected, obtaining a comparison result of the image to be detected and a preset reference image, and calculating a first early warning value of the image to be detected based on the comparison result and the detection result; wherein the comparison result is: comparing the image content of the image to be detected with the image content of the reference image to determine a first position coordinate of an area with image content difference meeting a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

when the first early warning value is larger than a first early warning threshold value, generating first early warning information; wherein the first early warning information is: and the early warning information about the existence of illegal buildings in the preset construction state in the target buildings included in the image to be detected.

Optionally, in a specific implementation manner, the determining manner of the comparison result includes:

respectively cutting the image to be detected and the reference image to obtain a plurality of pairs of image region blocks corresponding to the positions;

and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold or not aiming at the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area block corresponding to the pair of positions as a comparison result.

Optionally, in a specific implementation manner, the detection result includes a second position coordinate and a confidence of an image area where the illegal building in the preset construction state exists in the image to be detected;

the step of calculating the first early warning value of the image to be detected based on the comparison result and the detection result comprises the following steps:

and calculating a first early warning value of the image to be detected by using the area overlapping rate of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate and the confidence coefficient.

Optionally, in a specific implementation manner, the image to be detected is an image to be detected obtained by shooting the target shooting range by the monitoring device for the nth time; the target shooting range is a shooting range corresponding to the image to be detected;

the step of calculating the first early warning value of the image to be detected by using the area overlapping rate of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate and the confidence coefficient comprises the following steps:

calculating a first early warning value of the image to be detected by using a first formula; wherein the first formula is:

wherein, W is a first early warning value of the image to be detected, x (i) is a region overlapping rate of a first position coordinate corresponding to an ith image and a second position coordinate corresponding to the image, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is an image obtained by shooting the target shooting range for the ith time by the monitoring equipment.

Optionally, in a specific implementation manner, the calculating manner of the area overlapping ratio includes:

calculating the region overlapping rate by using a second formula; wherein the second formula is:

x(i)＝Area_i/(Area1_i+Area2_i-Area_i)

wherein, Area_iThe area of an overlapping area of a first position coordinate corresponding area and a second position coordinate corresponding area corresponding to the ith image; area1_iThe area of a region corresponding to the first position coordinate corresponding to the ith image is obtained; area2_iAnd the area of the area corresponding to the second position coordinate corresponding to the ith image is obtained.

Optionally, in a specific implementation manner, the predetermined construction state is: establishing a state;

the violation detection model is specifically trained on the basis of a training sample, an auxiliary sample, a sample label of the training sample and a sample label of the auxiliary sample, wherein the auxiliary sample is an image of a violation building in an established state and marked with an image area where the violation building is located, and the sample label of the auxiliary sample is in the established state;

the method further comprises the following steps:

when the detection result represents that the violation buildings in the established state exist in the image to be detected, calculating a second early warning value of the image to be detected based on the detection result;

when the second early warning value is larger than a second early warning threshold value, second early warning information is generated; wherein the second early warning information is: and the early warning information about the existence of the illegal buildings in the established state in the target buildings included in the image to be detected.

Optionally, in a specific implementation manner, the detection result includes a confidence level;

the step of calculating a second early warning value of the image to be detected based on the detection result comprises the following steps:

and calculating a second early warning value of the image to be detected by using the confidence coefficient.

the step of calculating the second early warning value of the image to be detected by using the confidence coefficient comprises the following steps:

calculating a second early warning value of the image to be detected by using a third formula; wherein the third formula is:

and E is a second early warning value of the image to be detected, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is an image obtained by shooting the target shooting range for the ith time by the monitoring equipment.

In a second aspect, an embodiment of the present invention provides a violation building early warning device, where the device includes:

the image acquisition module is used for acquiring an image to be detected, which is acquired by the monitoring equipment;

the image detection module is used for inputting the image to be detected into a preset illegal building detection model to obtain a detection result output by the illegal building detection model; wherein the default detection model is obtained by training based on a training sample and a sample label of the training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels of the training samples are: the construction state of the illegal building in the training sample;

the first early warning value calculation module is used for acquiring a comparison result of the image to be detected and a preset reference image when the detection result represents that the illegal building in the preset construction state exists in the image to be detected, and calculating a first early warning value of the image to be detected based on the comparison result and the detection result; wherein the comparison result is: comparing the image content of the image to be detected with the image content of the reference image to determine a first position coordinate of an area with image content difference meeting a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

the first information generation module is used for generating first early warning information when the first early warning value is larger than a first early warning threshold value; wherein the first early warning information is: and the early warning information about the existence of illegal buildings in the preset construction state in the target buildings included in the image to be detected.

Optionally, in a specific implementation manner, the apparatus further includes an image comparison module;

the image comparison module is used for determining the comparison result; the image comparison module is specifically configured to:

respectively cutting the image to be detected and the reference image to obtain a plurality of pairs of image region blocks corresponding to the positions; and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold or not aiming at the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area block corresponding to the pair of positions as a comparison result.

the first early warning value calculation module is specifically configured to: and calculating a first early warning value of the image to be detected by using the area overlapping rate of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate and the confidence coefficient.

the first early warning value calculation module is specifically configured to:

x(i)＝Area_i/(Area1_i+Area2_i-Area_i)

the device further comprises:

the second early warning value calculation module is used for calculating a second early warning value of the image to be detected based on the detection result when the detection result represents that the illegal building in the established state exists in the image to be detected;

the second information generation module is used for generating second early warning information when the second early warning value is larger than a second early warning threshold value; wherein the second early warning information is: and the early warning information about the existence of the illegal buildings in the established state in the target buildings included in the image to be detected.

the second early warning value calculation module is specifically configured to:

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

the processor is configured to implement any of the method steps of the violation early warning method provided by the first aspect when executing the program stored in the memory.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements any one of the method steps in the violation early warning method provided in the first aspect.

Therefore, by applying the scheme provided by the embodiment of the invention, the area which needs to be subjected to the violation building early warning can be monitored by the monitoring equipment, so that the image to be detected, which is acquired by the monitoring equipment, can be acquired. Furthermore, when a detection result obtained by detecting the image to be detected by using the preset default building detection model represents that the illegal building in the preset construction state exists in the image to be detected, a comparison result of the image to be detected and a preset reference image with the same shooting range as that of the image to be detected can be obtained, a first early warning value of the image to be detected is calculated according to the comparison result obtained by comparing and analyzing the image to be detected and the reference image and the detection result, and early warning information about the illegal building in the preset construction state existing in a target building included in the image to be detected is generated when the first early warning value is larger than a preset threshold value. Therefore, after the image to be detected acquired by the monitoring equipment is obtained, the violation buildings existing on the actual building can be found in time through the fusion application of the image contrast analysis method and the model detection method, and the early warning information aiming at the preset construction state is generated, so that the effective early warning of the violation buildings is realized, the dismantling difficulty is reduced, and the dismantling cost is reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a violation building early warning method provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of a panoramic image corresponding to a high altitude dome camera;

FIG. 3 is a schematic diagram of a corresponding relationship between preset points and actual buildings;

FIG. 4 is a schematic flow chart of a comparison result determination method;

FIGS. 5(a) - (c) are a training sample of the presence of a building violating a traffic violation in an established state, respectively;

FIGS. 6(a) - (c) are respectively a training sample of an existing violation building under construction;

fig. 7 is a schematic flow chart of another violation building early warning method according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a violation building early warning device provided by the embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In recent years, various illegal buildings not only occupy public social resources and urban spaces and destroy the good images of cities, but also bring certain harm to the economic order and social management of markets, and are important problems to be solved urgently in the urban management process. How to early warn the illegal buildings in time so as to reduce the dismantling difficulty and the dismantling cost becomes a problem to be solved urgently at present. In order to solve the technical problem, the embodiment of the invention provides a violation building early warning method.

First, a method for warning against illegal buildings according to an embodiment of the present invention is described below.

Fig. 1 is a schematic diagram of a violation building early warning method according to an embodiment of the present invention, where the method may be applied to any electronic device that needs to perform violation building early warning, such as a notebook computer, a desktop computer, a tablet computer, a mobile phone, a server, and the like. The electronic equipment is in communication connection with the monitoring equipment, so that the image to be detected acquired by the monitoring equipment can be acquired, subsequent related steps are further executed, and early warning of illegal buildings is realized.

As shown in fig. 1, the method for warning against illegal buildings according to the embodiment of the present invention may include the following steps:

s101: acquiring an image to be detected acquired by monitoring equipment;

s102: inputting an image to be detected into a preset default detection model to obtain a detection result output by the default detection model;

wherein, it obtains to break away to establish the detection model based on the sample label training of training sample and training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels for the training samples were: training the construction state of the illegal building in the sample;

s103: when the detection result represents that the illegal buildings in the preset construction state exist in the image to be detected, obtaining a comparison result of the image to be detected and a preset reference image, and calculating a first early warning value of the image to be detected based on the comparison result and the detection result;

wherein, the comparison result is: comparing the image content of the image to be detected with the image content of the reference image to determine a first position coordinate of an area with the image content difference meeting a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

s104: when the first early warning value is larger than a first early warning threshold value, generating first early warning information;

wherein, the first early warning information is: and the early warning information about the existence of illegal buildings in a preset construction state in the target building included in the image to be detected.

Therefore, by applying the scheme provided by the embodiment of the invention, the area which needs to be subjected to the violation building early warning can be monitored by the monitoring equipment, so that the image to be detected, which is acquired by the monitoring equipment, can be acquired. Furthermore, when a detection result obtained by detecting the image to be detected by using the preset default building detection model represents that the illegal building in the preset construction state exists in the image to be detected, a comparison result of the image to be detected and a preset reference image with the same shooting range as that of the image to be detected can be obtained, a first early warning value of the image to be detected is calculated according to the comparison result obtained by comparing and analyzing the image to be detected and the reference image and the detection result, and early warning information about the illegal building in the preset construction state existing in a target building included in the image to be detected is generated when the first early warning value is larger than a preset threshold value. Therefore, after the image to be detected acquired by the monitoring equipment is obtained, the illegal buildings existing on the actual buildings can be found in time through the fusion application of the image contrast analysis method and the model detection method, so that the dismantling difficulty is reduced, and the dismantling cost is reduced.

In step S101, after the monitoring device collects the image to be detected, the monitoring device may send the collected image to be detected to the electronic device. Therefore, the electronic equipment can acquire the image to be detected acquired by the monitoring equipment.

Wherein, above-mentioned supervisory equipment can be high altitude ball machine, also can be the supervisory equipment who erects on unmanned aerial vehicle, for example, erects camera etc. on unmanned aerial vehicle. Of course, the monitoring device may be other types of image capturing devices. This is all reasonable.

Next, the monitoring device in the above step S101 will be described by taking a high altitude dome camera as an example. Specifically, the method comprises the following steps:

when the monitoring device is a high-altitude dome camera, before the violation building early warning method provided by the embodiment of the invention is executed, the shooting range of the high-altitude dome camera can be determined firstly.

For example, firstly, the horizontal visual angle of the high-altitude dome camera is kept at 0 degree, then the high-altitude dome camera is rotated by 180 degrees in the vertical direction, and an image shot by the high-altitude dome camera is acquired every time the high-altitude dome camera rotates by 30 degrees; then, rotating the high-altitude dome camera in the horizontal direction, keeping the horizontal visual angle of the high-altitude dome camera at 30 degrees, further rotating the high-altitude dome camera in the vertical direction for 180 degrees again, and acquiring an image shot by the high-altitude dome camera every time the high-altitude dome camera rotates for 30 degrees; then, continuing to rotate the high-altitude dome camera in the horizontal direction, keeping the horizontal visual angle of the high-altitude dome camera at 60 degrees, further rotating the high-altitude dome camera in the vertical direction for 180 degrees, and acquiring an image shot by the high-altitude dome camera every time the high-altitude dome camera rotates for 30 degrees; and repeating the steps until the horizontal visual angle of the high-altitude dome camera is kept at 90 degrees, further rotating the high-altitude dome camera by 180 degrees in the vertical direction again, and acquiring an image shot by the high-altitude dome camera every time the high-altitude dome camera rotates by 30 degrees.

Therefore, the acquired images can be spliced by using an image splicing algorithm, so that a panoramic image corresponding to the high-altitude dome camera is obtained, and the content included in the panoramic image is the shooting range of the high-altitude dome camera. Wherein each building in the panoramic image may have address or number information, forming a specific location sample tag. The address or the number information can be directly obtained from the GIS, or can be manually marked, which is reasonable.

For example, as shown in fig. 2, a schematic diagram of a panoramic image corresponding to a high-altitude dome camera is shown, where 210 is the high-altitude dome camera, a position sample tag of each building is number information, and through the panoramic image, it can be determined that the shooting range of the high-altitude dome camera is from 1 to 15 of a residential building. Further, the address information of the shooting range of the high dome camera can be further determined based on the correspondence between the number and the real address of each residential building determined in advance.

Of course, for different high-altitude dome cameras and different positions where the high-altitude dome cameras are installed, other modes can be adopted to determine the shooting range of the high-altitude dome cameras, for example, the change rule of the angle of view in the above modes is changed. This is all reasonable.

In addition, the shooting range of the high-altitude dome camera can be directly determined by the electronic equipment, or the shooting range of the high-altitude dome camera can be determined by other electronic equipment in communication connection, and then the electronic equipment acquires the determined shooting range of the high-altitude dome camera from the other electronic equipment.

Similarly, when the monitoring device is a monitoring device erected on an unmanned aerial vehicle, before the violation building early warning method provided by the embodiment of the invention is executed, the shooting range of the monitoring device erected on the unmanned aerial vehicle can be determined at first.

For example, according to the preset cruising route of the unmanned aerial vehicle, after the unmanned aerial vehicle flies for a preset time or flies for a preset distance, the image collected by the monitoring equipment erected on the unmanned aerial vehicle is acquired. Like this, accomplish the route of cruising at unmanned aerial vehicle and fly the back, alright splice with the image that above-mentioned flight in-process obtained to obtain a panoramic picture that erects the supervisory equipment on unmanned aerial vehicle and correspond, the content that this panoramic picture includes is this the shooting scope of the supervisory equipment who erects on unmanned aerial vehicle promptly. The preset cruising route, the preset time and the preset distance can be limited according to the requirements in practical application.

Further, it can be understood that the image to be detected collected by the monitoring device may include a plurality of actual buildings, and based on the physical parameters of the monitoring device itself, the monitoring device may not be able to capture details of each actual building.

Based on the above, when the early warning information is generated, the buildings with the illegal buildings can be quickly determined without analyzing the image to be detected again, or the address information of the buildings with the illegal buildings is determined through field verification, so that the early warning efficiency of the illegal buildings is improved, and the problem that the address information of the buildings with the illegal buildings cannot be distinguished from the image to be detected is solved.

Optionally, in a specific implementation manner, a plurality of preset points may be preset, and a corresponding relationship is established between each preset point and address information of an actual building in a shooting range corresponding to the monitoring device at the preset point. In the above correspondence, the actual building in the shooting range corresponding to the monitoring device corresponding to each preset point is the actual building in which the monitoring device can shoot the detailed information at the preset point. When a plurality of actual buildings exist in the image to be detected shot at a certain preset point, a specific actual building area corresponding to the preset point can be further configured.

As shown in fig. 3, the actual building area included in the image frame 310 is a specific actual building area corresponding to the preset point where the monitoring device is located when the image to be detected shown in fig. 3 is captured. Furthermore, areas in the image to be detected that do not require analysis or that may involve privacy also need to be configured to protect the privacy of persons associated with the actual building area being captured. In particular, the region blocking can be performed on the region which does not need to be analyzed or may relate to privacy in the image to be detected. Such as the actual building area included in image box 320 in fig. 3.

Based on this, it can be understood that, when the region that does not need to be analyzed or may involve privacy in the image to be detected is configured, then, for the reference image in the same shooting range as the image to be detected, the region that does not need to be analyzed or may involve privacy in the reference image is also configured.

Thus, in this specific implementation manner, in step S101, the image to be detected acquired by the monitoring device and acquired by the electronic device is the image to be detected and captured by the monitoring device at a certain preset point. Furthermore, when the electronic device generates early warning information for the existence of the illegal building in the image to be detected, the electronic device can determine the preset point where the monitoring device is located when the monitoring device shoots the image to be detected, and further determine the address information of the actual building with the illegal building in the image to be detected through the corresponding relation between each preset point and the position sample label of the actual building.

When the monitoring equipment sends the collected image to be detected to the monitoring equipment, the monitoring equipment can send the information of the preset point where the monitoring equipment is located when the detection image is shot to the monitoring equipment. For example, the number of the preset point where the monitoring device itself is located when the detection image is captured is simultaneously sent to the monitoring device. This is all reasonable.

Further, in this specific implementation manner, in the step S101, the reference image acquired by the electronic device in the same shooting range as the image to be detected may be: and before the preset period, the monitoring equipment shoots the image to be detected at the preset point. Obviously, the reference image has a corresponding relationship with the address information of the actual building within the shooting range corresponding to the preset point.

For example, if the image to be detected is an image captured by the monitoring apparatus at the preset point 1, the reference image may be an image captured by the monitoring apparatus at the preset point 1 30 days ago.

Based on the above description, when the monitoring device is an overhead dome camera, the preset points are shooting angles selected from a shooting time range which can be reached by the overhead dome camera. When the shooting visual angle of the high-altitude dome camera rotates to the shooting visual angle corresponding to a certain preset point, the pixels of the shot image to be detected can be adjusted through focal length adjustment, so that the pixels of the shot image of the actual building corresponding to the preset point can meet the requirements of subsequent image comparison and image detection.

Further, the staying time and the training interval of the high-altitude dome camera at each preset point can be preset.

For example, if 10 preset points are preset, and the high-altitude dome camera stays for 20 minutes at each preset point, the training period of the 10 preset points is 200 minutes, and the training interval is set to be 100 minutes.

In order to avoid image blurring caused by rotation of the high-altitude dome camera in the to-be-detected image shot by the high-altitude dome camera at each preset point, when the high-altitude dome camera rotates to the shooting visual angle corresponding to each preset point, the high-altitude dome camera can take the collected image as the to-be-detected image when the high-altitude dome camera is at the shooting visual angle for a certain length and a certain length of time is also available after the high-altitude dome camera stays for a certain time, and the collected image is sent to the electronic equipment. Therefore, the image blurring caused by the rotation of the high altitude dome camera when the high altitude dome camera rotates from other shooting visual angles to the shooting visual angle corresponding to each preset point or rotates from the shooting visual angle corresponding to each preset point to other shooting visual angles is avoided.

For example, when the high-altitude dome camera stays at each preset point for 20 minutes, the high-altitude dome camera may take the acquired image as the image to be detected and send the image to the electronic device when the duration of the high-altitude dome camera at the shooting visual angle reaches 10 minutes. Of course, when the duration of the photographing view angle reaches 8 minutes, 9 minutes, 30 seconds or 12 minutes, the acquired image may be sent to the electronic device as the image to be detected. The high altitude dome camera can be used only if the duration of the high altitude dome camera at the shooting visual angle reaches a certain length and a certain duration exists after the stopping time is over.

When the monitoring equipment is the monitoring equipment erected on the unmanned aerial vehicle, a plurality of cruising positions on the cruising route can be set as preset points according to the preset cruising route and the flight period. Of course, the cruising interval of the drone may also be set.

Like this, erect supervisory equipment on unmanned aerial vehicle alright in order when unmanned aerial vehicle flies the cruise position that every preset point corresponds, the image that will be in shooting when this cruise position is as waiting to detect the image to send for electronic equipment.

After the step S101 is executed, and the image to be detected is obtained, the electronic device may continue to execute the step S102, and input the image to be detected into the preset default detection model, so as to obtain the detection result output by the default detection model.

Wherein, it obtains to break away to establish the detection model based on the sample label training of training sample and training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels for the training samples were: training the construction status of the offending building present in the sample.

Optionally, in a specific implementation manner, the predetermined construction state may be a construction state and a built state, where the built state refers to that the construction of the illegal building is completed, and as shown in fig. 5(a) - (c), the predetermined construction state is a training sample of the illegal building existing in the built state. The under-construction state refers to that the illegal construction is in the process of construction and is not completed, and as shown in fig. 6(a) - (c), the under-construction state is a training sample of the illegal construction in the under-construction state.

Based on the above, a large number of training samples can be obtained, the training samples are images of the illegal buildings in the under-construction state or the established state, the image areas of the illegal buildings are labeled in the training samples, and the construction states of the illegal buildings included in the training samples are determined as sample labels of the training samples.

Wherein, for the image comprising the violation buildings in the established state, the image area where the existing violation buildings are located can be determined according to the characteristics of various types of established violation buildings. However, since the illegal building is not formed, the image area where the illegal building exists cannot be determined by the characteristics of various types of established illegal buildings, and the image area where the illegal building exists needs to be determined by the characteristics related to the construction process, for example, continuous residence of 3 or more persons, existence of construction building materials (for example, semi-formed steel bars, reinforced concrete, bricks, etc.), existence of construction covering, and the like.

For example, for fig. 5(a), 510 is the label of the image area where the established paintroom violation building is located, and the sample label is the established state; for fig. 5(b), 520 is the label of the image area where the constructed glasshouse violation building is located, and the sample label is the constructed state; with respect to FIG. 5(c), 530 is an annotation of the image area where the established story violation building is located and the sample label is established.

For another example, in fig. 6(a), 610 is a label of an image area where the tarpaulin for construction covering is located, and the sample label is in a construction state; for FIG. 6(b), 620 is the label of the image region where the constructor is located, and the sample label is in the under-construction state; for FIG. 6(c), 630 is the label for the image region where the construction scene is located, and the sample label is in the under-built state.

Thus, in this specific implementation manner, after obtaining a plurality of training samples and a sample label of each training sample, a preset initial model may be trained by using the obtained plurality of training samples and the sample label of each training sample, so that the preset initial model learns the features of the buildings violating regulations and the coordinates of the image areas where the buildings violating regulations are located in the training samples, and the sample labels of the training samples, and gradually establishes the corresponding relationship between the features of the buildings violating regulations and the coordinates of the image areas where the buildings violating regulations are located in the training samples, and the sample labels of the training samples through the learning of a large number of training samples. And when the preset convergence condition is met, stopping training to obtain the trained illegal building detection model.

Furthermore, in this specific implementation manner, when the step S103 is executed, the electronic device may input the image to be detected into the trained illegal building detection model, further detect whether there is an illegal building in the image to be detected by using the illegal building detection model, and detect the building type of the existing illegal building and the image area where the existing illegal building is located in the image to be detected when detecting that there is an illegal building in the image to be detected, thereby obtaining the detection result output by the illegal building detection model.

The detection result output by the illegal building detection model can comprise the construction state of the illegal building, wherein the construction state of the illegal building is any one of the following three states: the violation building state does not exist, the violation building state is established and the violation building state is established. Wherein the condition of the illegal buildings does not exist, and the condition indicates that the illegal buildings do not exist in the image to be detected; the established illegal building state represents that the illegal building in the established state exists in the image to be detected; and establishing a violation building state representation, wherein the violation building in the established state exists in the image to be detected.

Obviously, in this specific implementation manner, the predetermined construction state is a under-construction state and an established state, and when the detection result output by the violation detection model includes the established violation building state or the under-construction violation building state, the detection result may represent that the violation building in the predetermined construction state exists in the image to be detected, and further, the electronic device may continue to perform the subsequent step S103.

In addition, because the violation detection model can establish the corresponding relation between the characteristics of the violation buildings and the coordinates of the image areas where the violation buildings are located in the training samples and the sample labels of the training samples in the training process, when the detection result output by the violation detection model comprises the established violation building state or the established violation building state, the detection result can also comprise the second position coordinates of the image areas where the violation buildings in the established violation building state or the established violation building state are located in the image to be detected.

In another specific implementation, the predetermined construction state may be a construction state and a built state; in yet another specific implementation, the predetermined construction state may be a constructed state. Obviously, in these two specific implementations, the manner in which the electronic device executes the step S102 is similar to that in the above specific implementation, and the training manner of the violation detection model is also similar to that in the above specific implementation, and therefore, the details are not repeated here.

The training process of the violation detection model can be completed in the electronic device, or can be completed in other electronic devices in communication connection with the electronic device, and the training process is reasonably sent to the electronic device by the other electronic devices. In addition, the embodiment of the invention also does not limit the source of the training sample used for training the above default detection model.

In addition, in one embodiment, when training the violation detection model, the type of the violation building in the established state, such as a colored board house, a glass house, a tile house, etc., can be marked in the training sample. Therefore, when the output result of the violation detection model represents that the violation buildings in the established state exist in the image to be detected, the detection result can also comprise the types of the violation buildings in the established state.

Similarly, when the violation detection model is trained, the construction information of the violation building in the construction state, such as constructors, construction scenes, construction coverage, construction building materials, and the like, may be marked in the training sample. Therefore, when the output result of the violation detection model represents that the violation buildings in the established state exist in the image to be detected, the detection result can also comprise the construction information of the violation buildings in the established state.

After the step S102 is executed, and the detection result output by the violation detection model is obtained, the electronic device may determine whether there is a violation building in the predetermined construction state in the image to be detected according to the construction state of the violation building included in the detection result. Furthermore, when the electronic device determines that the detection result output by the default detection model indicates that the illegal building in the preset construction state exists in the image to be detected, the electronic device may continue to execute step S103, that is, obtain the comparison result between the image to be detected and the preset reference image, and calculate the first warning value of the image to be detected based on the obtained comparison result and the detection result obtained in S102.

Wherein, the obtained comparison result is as follows: and comparing the image contents of the image to be detected and the reference image to obtain the first position coordinates of the area with the difference of the image contents meeting the preset difference rule. Therefore, the electronic device executes the step S103 specifically as follows: and calculating a first early warning value of the image to be detected based on the first position coordinate of the region, which is obtained by comparing the image content of the image to be detected with the image content of the reference image and has the difference meeting the preset difference rule, and the detection result obtained in the step S103.

Preferably, in step S103, the comparison result obtained by the electronic device may be determined only when the detection result indicates that the illegal building in the predetermined construction state exists in the image to be detected. That is to say, when it is determined that the detection result represents that the illegal building in the preset construction state exists in the image to be detected, the electronic device obtains the preset reference image with the same shooting range as that of the image to be detected, and then compares the image content of the image to be detected with the image content of the preset reference image, and obtains the first position coordinate of the area with the difference of the image content meeting the preset difference rule, so as to obtain the comparison result.

Preferably, in step S103, the comparison result obtained by the electronic device may be predetermined. That is to say, after the image to be detected is acquired, the electronic device may acquire a preset reference image having the same shooting range as that of the image to be detected, compare the image contents of the image to be detected and the preset reference image, and obtain a first position coordinate of an area where the difference between the image contents meets a preset difference rule, i.e., obtain the comparison result.

The electronic device may acquire the reference image in various ways whenever the comparison result is determined, for example, the electronic device may acquire the reference image from a plurality of images stored locally, or the electronic device may acquire the reference image from other electronic devices connected in communication, and the embodiment of the present invention is not limited in particular.

In addition, the electronic device may determine the comparison result in multiple ways, that is, the electronic device may compare the image contents of the image to be detected and the reference image in multiple ways to obtain the first position coordinate of the region where the difference between the image contents meets the preset difference rule, which is not limited in the embodiment of the present invention.

Optionally, in a specific implementation manner, as shown in fig. 4, the manner in which the electronic device determines the comparison result may include the following steps:

s401: respectively cutting the image to be detected and the reference image to obtain a plurality of pairs of image area blocks corresponding to the positions;

for the obtained image to be detected and the reference image, the electronic device wants to determine the region with smaller similarity in the image to be detected and the reference image, so that the electronic device can respectively cut the image to be detected and the reference image, and respectively cut the image to be detected and the reference image into a plurality of image region blocks, thereby obtaining a plurality of pairs of image region blocks corresponding to the positions.

Specifically, the method comprises the following steps: and the shooting range corresponding to each image area block corresponding to the image to be detected is a sub-range of the shooting range of the image to be detected. Similarly, since the shooting range of the reference image is the same as the shooting range of the image to be detected, the shooting range corresponding to each image block corresponding to the reference image is also a sub-range of the shooting range of the image to be detected.

In order to determine the region with smaller similarity between the image to be detected and the reference image, for each image region block corresponding to the image to be detected, the shooting range corresponding to the image region block should be the same as the shooting range corresponding to a certain image block in the plurality of image blocks corresponding to the reference image. That is, for each image area block corresponding to the image to be detected, the position of the image area block in the image to be detected should correspond to the position of a certain image block in the reference image in the plurality of image blocks corresponding to the reference image.

Therefore, the electronic equipment can respectively cut the image to be detected and the reference image to obtain a plurality of pairs of image area blocks corresponding to the positions. Therefore, the number of image region blocks obtained by cutting the image to be detected by the electronic device is the same as the number of image region blocks obtained by cutting the reference image, and for each image region block corresponding to the image to be detected, a plurality of image region blocks corresponding to the reference image exist and only one image region block corresponding to one position exists, so that a pair of image region blocks corresponding to one position can be obtained.

The electronic device may respectively cut the image to be detected and the reference image in various manners such as a gray threshold segmentation method, an edge-based segmentation method, a clustering-analysis-based segmentation method, and the like, which is not particularly limited in the embodiments of the present invention.

In addition, in order to reduce shooting range errors caused by image sizes, position deviations and the like of the image to be detected and the reference image, the shooting ranges corresponding to the obtained image area blocks corresponding to the multiple pairs of positions are more similar, the accuracy of the comparison result is improved, and finally the accuracy of the early warning result is improved.

Optionally, before performing step S401, the electronic device may perform image alignment on the image to be detected and the reference image by using an image analysis algorithm, and correct the position deviation between the two images. Furthermore, the electronic device can respectively cut the image to be detected and the reference image obtained after the image alignment and the position deviation correction are performed, so as to obtain a plurality of pairs of image area blocks corresponding to the positions.

The electronic device may perform image alignment on the image to be detected and the reference image through various image analysis algorithms, and correct the position deviation between the two images, which is not particularly limited in the embodiments of the present invention. For example, Harris corner detection and feature matching algorithm, etc.

S402: and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold or not aiming at the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area block corresponding to the pair of positions as a comparison result.

After the step S402 is executed, the image area blocks corresponding to the plurality of pairs of positions are obtained, and for each pair of image area blocks corresponding to each pair of positions, the electronic device may determine whether the similarity of the image area blocks corresponding to the pair of positions is smaller than a preset similarity threshold, and if the determination result is yes, obtain the first position coordinates of the image area blocks corresponding to the pair of positions as the comparison result.

The electronic device may compare each pair of image area blocks corresponding to the positions by using various image comparison algorithms, so as to obtain the similarity of the image area blocks corresponding to the pair of positions. The embodiment of the present invention is not particularly limited. For example, a similarity discrimination algorithm based on 2-channel network (two-channel network structure).

In addition, the specific value of the preset similarity threshold may be defined according to requirements in practical applications, and the specific value of the preset similarity threshold is not defined in the embodiment of the present invention.

Optionally, when there is a similarity of a pair of image region blocks corresponding to the position smaller than a preset similarity threshold, the electronic device may use the first position coordinates of the pair of image region blocks corresponding to the position as the comparison result.

Optionally, when there are multiple pairs of image area blocks with similarity smaller than a preset similarity threshold, the electronic device may use the first position coordinates of the image area blocks corresponding to the multiple pairs of positions as the comparison result.

Optionally, when there are a plurality of pairs of image region blocks with similarity smaller than a preset similarity threshold, the electronic device may use the first position coordinates of the image region block corresponding to the pair with the minimum similarity in the image region blocks corresponding to the plurality of pairs as the comparison result.

Optionally, when the similarity of the image region blocks corresponding to the multiple pairs of positions is smaller than a preset similarity threshold, the electronic device determines a first position coordinate of the image region block corresponding to the multiple pairs of positions, further determines, according to the determined multiple first position coordinates, a minimum image region range capable of containing the image region block belonging to the image to be detected in the image to be detected, and uses the determined coordinate of the minimum image region range as a comparison result. Of course, the electronic apparatus also determines, in the reference image, a minimum image area range capable of containing an image area block belonging to the reference image among the image area blocks corresponding to the plurality of pairs of positions, based on the determined plurality of first position coordinates, and uses coordinates of the determined minimum image area range as a comparison result. The coordinate of the minimum image area range determined in the image to be detected is the same as the coordinate of the minimum image area range determined in the reference image.

Thus, after the step S103 is executed, and the first early warning value of the image to be detected is obtained through calculation, the electronic device may continue to execute the step S104, that is, determine whether the calculated first early warning value is greater than the first early warning threshold value, and generate early warning information about the existence of the illegal building in the predetermined construction state in the target building included in the image to be detected, that is, generate the first early warning information when the determination result is that the first early warning value is greater than the first early warning threshold value.

The target building is a building which is included in the image to be detected and has the illegal building detected in the step S102 and in the preset construction state; the specific value of the first early warning threshold value may be set according to the requirements of practical application, and the embodiment of the present invention does not limit the specific value of the first early warning threshold value.

When the image to be detected is an image shot by the monitoring device at a certain preset point, the first early warning information generated in the step S104 is early warning information about an actual building corresponding to the preset point, namely, an illegal building in a preset construction state.

The electronic device may perform the step S103 in various ways, that is, calculate the first warning value of the image to be detected based on the comparison result and the detection result in various ways. The embodiment of the present invention is not particularly limited.

Optionally, in a specific implementation manner, the detection result output by the violation detection model may include a second position coordinate and a confidence of an image area where a violation building in a predetermined construction state exists in the image to be detected; therefore, in this specific implementation manner, the manner in which the electronic device executes step S103 may include the following steps:

and calculating a first early warning value of the image to be detected by utilizing the area overlapping rate and the confidence coefficient of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate.

The electronic device may calculate the first warning value of the image to be detected in a plurality of ways by using the area overlapping rate and the confidence of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate, which is not limited in the embodiments of the present invention.

Optionally, in an embodiment, the image to be detected may be an image to be detected obtained by shooting the target shooting range by the monitoring device for the nth time; the target shooting range is a shooting range corresponding to an image to be detected; therefore, in this embodiment, the manner of calculating the first warning value of the image to be detected by the electronic device using the area overlapping rate of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate and the confidence coefficient may include the following steps:

wherein, W is a first early warning value of the image to be detected, x (i) is a region overlapping rate of an image region corresponding to a first position coordinate and an image region corresponding to a second position coordinate corresponding to an ith image, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is the image to be detected obtained by shooting a target shooting range for the ith time by the monitoring equipment.

Further, optionally, the calculating method of the area overlapping ratio in the first formula includes:

x(i)＝Area_i/(Area1_i+Area2_i-Area_i)

wherein, Area_iThe area of an overlapping area of a first position coordinate corresponding area and a second position coordinate corresponding area corresponding to the ith image; area1_iThe area of the area corresponding to the first position coordinate corresponding to the ith image; area2_iThe area of the area corresponding to the second position coordinate corresponding to the ith image.

Obviously, in this embodiment, when the image to be detected is an image captured by the monitoring device at a certain preset point, the image to be detected may be an image captured by the monitoring device at the preset point for the nth time, and the ith image may be an image captured by the monitoring device at the preset point for the ith time.

In addition, because some or several images to be detected may have a plurality of illegal buildings in a preset construction state in the plurality of images to be detected obtained by shooting the target shooting range by the monitoring device, the image to be detected may correspond to a plurality of second position coordinates and a plurality of confidence degrees, and each second position coordinate corresponds to one confidence degree.

Therefore, for the image to be detected, the highest confidence corresponding to the image to be detected can be determined as the confidence utilized in the first formula, and the region overlapping rate in the first formula is the region overlapping rate of the image region corresponding to the first position coordinate corresponding to the image to be detected and the image region corresponding to the second position coordinate corresponding to the highest confidence corresponding to the image to be detected. Furthermore, the area of the area corresponding to the second position coordinate corresponding to the image to be detected, which is utilized in the second formula, is the area of the image area corresponding to the second position coordinate corresponding to the highest confidence corresponding to the image to be detected.

Further, for the second formula:

preferably, when the electronic device includes a first position coordinate in the comparison result determined in the step S102 for the ith image, the Area1 in the second formula is_iNamely the area of the area corresponding to the first position coordinate corresponding to the determined ith image.

Preferably, when the electronic device includes a plurality of first position coordinates in the comparison result determined in the step S102 for the ith image, the electronic device may use the Area of the corresponding region of each first position coordinate as Area1 in the second formula_iThus, a plurality of x (i) s are calculated, and further, the calculated maximum x (i) s may be determined as the area overlapping ratio used in the first formula.

Preferably, when the electronic device includes a plurality of first position coordinates in the comparison result determined in step S102 for the ith image, the electronic device may further obtain a similarity corresponding to each first position coordinate, and further apply the Area1 in the second formula_iAnd determining the area of the area corresponding to the first position coordinate corresponding to the minimum similarity.

In addition, in the first formula, the specific value of N may be defined according to the requirement of practical application, and the specific value of N is not defined in the embodiment of the present invention.

Obviously, when N is 1, the above first formula may be modified as follows:

W＝(x+y)/2

w is a first early warning value of the image to be detected, x is the area overlapping rate of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate, which correspond to the image to be detected, and y is the confidence coefficient corresponding to the image to be detected.

Further, the second formula may be modified as follows:

x＝Area/(Area1+Area2-Area)

wherein, Area is the Area of the overlapping Area of the Area corresponding to the first position coordinate and the Area corresponding to the second position coordinate corresponding to the image to be detected, and Area1 is the Area of the Area corresponding to the first position coordinate corresponding to the image to be detected; area2 is the Area of the Area corresponding to the second position coordinate corresponding to the image to be detected.

Further, in order to better distinguish whether the image to be detected comprises the violation buildings in the construction state or the violation buildings in the established state, different violation building early warning methods can be adopted for the violation buildings in different construction states.

Optionally, in a specific implementation manner, the predetermined construction state may be: establishing a state;

thus, the violation detection model applied in step S102 above: the violation detection model is trained based on a training sample, an auxiliary sample, a sample label of the training sample and a sample label of the auxiliary sample, the auxiliary sample is an image of a violation building in an established state and marked with an image area where the violation building is located, and the sample label of the auxiliary sample is in the established state.

In the specific implementation manner, the preset construction state can be a construction state, so that the violation detection model is trained by using an image in which the adopted training sample is an image in which the violation building in the construction state exists and the image area where the violation building is marked; the sample labels for the training samples were: and establishing a state. Therefore, in order to enable the violation detection model to detect the violation buildings in the established state in the image to be detected, the violation detection model can be obtained by utilizing the training sample, the auxiliary sample, the sample label of the training sample and the sample label of the auxiliary sample for training. Obviously, the auxiliary sample is an image of an image area where the illegal building exists in the established state and the illegal building is marked, and the sample label of the auxiliary sample is the established state.

Thus, in this specific implementation manner, as shown in fig. 7, another violation building early-warning method provided by the embodiment of the present invention may further include the following steps:

s105: when the detection result represents that the illegal building in the established state exists in the image to be detected, calculating a second early warning value of the image to be detected based on the detection result;

s106: when the second early warning value is larger than a second early warning threshold value, second early warning information is generated;

wherein the second early warning information is: and the early warning information about the existence of the illegal buildings in the established state in the target buildings included in the image to be detected.

That is, when the detection result output by the violation detection model indicates that there is a violation building in the established state in the image to be detected, since the established state does not belong to the predetermined establishment state in the present implementation manner, the electronic device cannot execute the steps S103 and S104. Based on this, the electronic device may continue to perform the step S105, that is, calculate the second warning value of the image to be detected based on the detection result obtained in the step S102.

Thus, after the step S105 is executed, and the second early warning value of the image to be detected is obtained through calculation, the electronic device may continue to execute the step S106, that is, determine whether the calculated second early warning value is greater than the second early warning threshold value, and generate early warning information about that the target building included in the image to be detected has a violation building in the established state, that is, generate second early warning information when the determination result is that the second early warning value is greater than the second early warning threshold value.

The specific value of the second early warning threshold value may be set according to the requirements of practical application, and the specific value of the second early warning threshold value is not limited in the embodiment of the present invention.

When the image to be detected is an image shot by the monitoring device at a certain preset point, the second early warning information generated in the step S106 is early warning information about an actual building corresponding to the preset point, namely an illegal building existing in a built state.

In this specific implementation manner, steps S101 to S104 are the same as the specific implementation manner of steps S101 to S104 in the embodiment shown in fig. 1, and are not described herein again.

The electronic device may perform the step S105 in various ways, that is, calculate the second warning value of the image to be detected based on the detection result in various ways. The embodiment of the present invention is not particularly limited.

Optionally, in an embodiment, the detection result output by the violation detection model may include a confidence level; therefore, the manner in which the electronic device performs step S105 may include the following steps:

Preferably, the image to be detected can be an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring equipment; the target shooting range is the shooting range corresponding to the image to be detected, so that the mode of calculating the second early warning value of the image to be detected by the electronic equipment by using the confidence coefficient can comprise the following steps:

and E is a second early warning value of the image to be detected, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is the image to be detected obtained by shooting the target shooting range for the ith time by the monitoring equipment.

In addition, because some or several images to be detected may have a plurality of illegal buildings in a preset construction state in the plurality of images to be detected obtained by shooting the target shooting range by the monitoring device, the image to be detected may correspond to a plurality of second position coordinates and a plurality of confidence degrees, and each second position coordinate corresponds to one confidence degree. Therefore, for such an image to be detected, the highest confidence corresponding to the image to be detected can be determined as the confidence utilized in the third formula.

In addition, in the third formula, the specific value of N may be defined according to the requirement of practical application, and the specific value of N is not defined in the embodiment of the present invention.

Obviously, when N is 1, the above first formula may be modified as follows:

E＝y(i)

namely, the second early warning value of the image to be detected is the confidence corresponding to the image to be detected.

In addition, based on the above descriptions of the embodiments of the present invention, in a specific implementation manner, the predetermined construction state may be a constructed state and a constructed state, and a detection result output by the violation detection model includes a second position coordinate and a confidence of an image region in the constructed state or the constructed state, which is present in the image to be detected; thus, in this particular implementation:

when the detection result output by the violation detection model represents that the violation buildings in the established state exist in the image to be detected, the electronic equipment can calculate a first early warning value of the image to be detected by utilizing the area overlapping rate and the confidence coefficient of the image area corresponding to the first position coordinate in the comparison result and the image area corresponding to the second position coordinate in the detection result;

the electronic device can also calculate the first warning value by using the first formula and the second formula.

When the detection result output by the violation detection model represents that the violation building in the established state exists in the image to be detected, the electronic equipment can calculate a second early warning value of the image to be detected by using the image area corresponding to the first position coordinate in the comparison result and the confidence coefficient in the detection result;

the electronic device may further calculate the second warning value by using the third formula.

In this specific implementation manner, the specific implementation manner of each step is the same as the specific implementation manner of the same step provided in each embodiment of the present invention, and is not described herein again.

Obviously, in the present specific implementation, although the predetermined construction state is the under-construction state and the established state, it is still possible to distinguish whether the image to be detected includes the illegal building in the under-construction state or the illegal building in the established state, and different illegal building early warning methods are adopted for the illegal buildings in different construction states.

In addition, after the first early warning information or the second early warning information is obtained, the monitoring device can be a video acquisition device, and the monitoring device can shoot videos in a shooting range, so that a user can call videos in a period of time before and after the moment when the monitoring device shoots the image to be detected in the videos shot by the monitoring device. Furthermore, the user can further determine the specific situation of the illegal building in the under-construction state or the established state and the specific situation of the actual building with the illegal building in the image to be detected by observing the image information of the actual building in the video. Meanwhile, the user can also acquire more specific information of the actual building from the recorded information about the actual building. For example, the owner of the actual building, the user of the actual building, the construction time of the actual building, whether a construction violation has occurred before the actual building, etc.

Further, after the information is acquired, the user can archive and pack the related information, so that the information can be pushed to related government workers, and subsequent processing, such as dismantling of illegal buildings, can be facilitated.

Compared with the illegal building early warning method provided by the embodiment of the invention, the embodiment of the invention also provides an illegal building early warning device.

Fig. 8 is a schematic structural diagram of a violation building early warning device provided by the embodiment of the present invention. As shown in fig. 8, the illegal building early warning device may include the following modules:

the image acquisition module 810 is configured to acquire an image to be detected, which is acquired by the monitoring device;

the image detection module 820 is used for inputting an image to be detected into a preset default detection model to obtain a detection result output by the default detection model; wherein, it obtains to break away to establish the detection model based on the sample label training of training sample and training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels for the training samples were: training the construction state of the illegal building in the sample;

the first early warning value calculation module 830 is configured to, when the detection result indicates that a violation building in a predetermined construction state exists in the image to be detected, obtain a comparison result between the image to be detected and a preset reference image, and calculate a first early warning value of the image to be detected based on the comparison result and the detection result; wherein the comparison result is: comparing the image content of the image to be detected with the image content of the reference image to determine a first position coordinate of an area with image content difference meeting a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

the first information generation module 840 is configured to generate first warning information when the first warning value is greater than a first warning threshold; wherein, the first early warning information is: and the early warning information about the existence of illegal buildings in a preset construction state in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the violation building early-warning device may further include an image comparison module;

in this specific implementation manner, the image comparison module is configured to determine the comparison result;

specifically, the image comparison module may be specifically configured to:

respectively cutting the image to be detected and the reference image to obtain a plurality of pairs of image area blocks corresponding to the positions; and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold or not aiming at the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area block corresponding to the pair of positions as a comparison result.

Optionally, in a specific implementation manner, the detection result may include a second position coordinate and a confidence of an image area where the illegal building in the predetermined construction state exists in the image to be detected;

therefore, in this specific implementation manner, the first warning value calculating module 830 may be specifically configured to: and calculating a first early warning value of the image to be detected by utilizing the area overlapping rate and the confidence coefficient of the image area corresponding to the first position coordinate and the image area corresponding to the second position coordinate.

Optionally, in a specific implementation manner, the image to be detected may be an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range can be a shooting range corresponding to an image to be detected;

therefore, in this specific implementation manner, the first warning value calculating module 830 may be specifically configured to:

Optionally, in a specific implementation manner, the calculating manner of the area overlapping ratio may include:

x(i)＝Area_i/(Area1_i+Area2_i-Area_i)

therefore, in this specific implementation manner, the violation detection model may be specifically trained based on a training sample, an auxiliary sample, a sample label of the training sample, and a sample label of the auxiliary sample, where the auxiliary sample is an image in which a violation building in an established state exists and an image area where the violation building is marked, and the sample label of the auxiliary sample is in the established state;

furthermore, the device for warning against buildings violating regulations provided by the embodiments of the present invention may further include:

Optionally, in a specific implementation manner, the detection result may include a confidence level;

therefore, in this specific implementation manner, the second warning value calculating module may be specifically configured to:

Optionally, in a specific implementation manner, the image to be detected is an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to an image to be detected;

the second warning value calculation module may be specifically configured to:

An embodiment of the present invention further provides an electronic device, as shown in fig. 9, which includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904,

a memory 903 for storing computer programs;

the processor 901 is configured to implement the method for warning against buildings violating regulations provided by the embodiment of the present invention when executing the program stored in the memory 903.

Specifically, the violation building early warning method includes:

acquiring an image to be detected acquired by monitoring equipment;

inputting an image to be detected into a preset default detection model to obtain a detection result output by the default detection model; wherein, it obtains to break away to establish the detection model based on the sample label training of training sample and training sample, and the training sample is: the method comprises the steps that illegal buildings in a preset construction state exist, and images of image areas where the illegal buildings are located are marked; the sample labels for the training samples were: training the construction state of the illegal building in the sample;

when the first early warning value is larger than a first early warning threshold value, generating first early warning information; wherein, the first early warning information is: and the early warning information about the existence of illegal buildings in a preset construction state in the target building included in the image to be detected.

It should be noted that other implementation manners of the violation building early-warning method implemented by the processor 901 executing the program stored in the memory 903 are the same as the violation building early-warning method provided in the foregoing method embodiment, and are not described here again.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

Corresponding to the violation building early warning method provided by the embodiment of the invention, the embodiment of the invention also provides a computer readable storage medium, and the computer program is executed by the processor to realize the violation building early warning method provided by the embodiment of the invention.

Specifically, the violation building early warning method includes:

acquiring an image to be detected acquired by monitoring equipment;

comparing the image contents of the image to be detected and the reference image to obtain a first position coordinate of an area with the difference of the image contents meeting a preset difference rule as a comparison result;

It should be noted that other implementation manners of the violation building early-warning method implemented when the computer program is executed by the processor are the same as the violation building early-warning method provided in the foregoing method embodiment section, and are not described again here.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, the electronic device embodiment and the computer-readable storage medium embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A violation building early warning method is characterized by comprising the following steps:

acquiring an image to be detected acquired by monitoring equipment;

2. The method according to claim 1, wherein the determining manner of the comparison result comprises:

3. The method according to claim 1, characterized in that the detection result comprises a second position coordinate and a confidence degree of an image area where the illegal building in a preset construction state exists in the image to be detected;

4. The method according to claim 3, wherein the image to be detected is an image to be detected obtained by shooting a target shooting range for the Nth time by the monitoring equipment; the target shooting range is a shooting range corresponding to the image to be detected;

5. The method of claim 4, wherein the region overlap ratio is calculated by:

x(i)＝Area_i/(Area1_i+Area2_i-Area_i)

6. The method of claim 1, wherein the predetermined construction state is: establishing a state;

the method further comprises the following steps:

7. The method of claim 6, wherein a confidence level is included in the detection result;

8. The method according to claim 6, wherein the image to be detected is an image to be detected obtained by shooting a target shooting range for the Nth time by the monitoring equipment; the target shooting range is a shooting range corresponding to the image to be detected;

9. A violation building early warning device, characterized in that the device includes:

10. The device according to claim 9, characterized in that the detection result includes a second position coordinate and a confidence degree of an image area where the illegal building in a preset construction state exists in the image to be detected;

11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1 to 8 when executing a program stored in the memory.

12. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-8.