CN111368615B

CN111368615B - Illegal building early warning method and device and electronic equipment

Info

Publication number: CN111368615B
Application number: CN201910563579.5A
Authority: CN
Inventors: 吕军; 户军; 唐槐
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2019-06-26
Filing date: 2019-06-26
Publication date: 2024-04-05
Anticipated expiration: 2039-06-26
Also published as: CN111368615A

Abstract

The embodiment of the invention provides a method and a device for early warning of a illegal building and electronic equipment. The method comprises the following steps: acquiring an image to be detected acquired by monitoring equipment; inputting the image to be detected into a preset illegal detection model to obtain a detection result output by the illegal detection model; when the detection result represents that a violation building in a preset construction state exists in the image to be detected, a comparison result of the image to be detected and a preset reference image is obtained, and a first early warning value of the image to be detected is calculated based on the comparison result and the detection result; and when the first early warning value is larger than the first early warning threshold value, generating first early warning information. Compared with the prior art, by applying the scheme provided by the embodiment of the invention, the illegal building existing on the actual building can be found in time, and the illegal building in the preset construction state can be effectively early warned.

Description

Illegal building early warning method and device and electronic equipment

Technical Field

The invention relates to the technical field of video monitoring, in particular to a method and a device for warning illegal buildings and electronic equipment.

Background

In recent years, various illegal buildings not only occupy social public resources and urban space and destroy good image of cities, but also bring certain harm to market economic order and social management, thus being a great difficult problem to be solved in the urban management process.

As once the illegal building cannot be stopped and dismantled in time in the initial stage of construction, the difficulty of subsequent dismantling work is increased, and higher dismantling cost is brought, particularly for the built illegal building, the dismantling difficulty and the dismantling cost are higher than those of the built illegal building, so that the demand of early warning on the illegal building existing on the actual building exists.

Then, how to timely warn the illegal buildings existing on the building to reduce the dismantling difficulty and the dismantling cost becomes a current urgent problem to be solved.

Disclosure of Invention

The embodiment of the invention aims to provide a method and a device for warning a illegal building and electronic equipment, so as to realize timely finding the illegal building existing on an actual building. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a method for warning a illegal building, where the method includes:

acquiring an image to be detected acquired by monitoring equipment;

inputting the image to be detected into a preset illegal building detection model to obtain a detection result output by the illegal building detection model; the illegal detection model is obtained by training based on a training sample and a sample label of the training sample, and the training sample is: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of the training samples are: the construction state of the illegal building existing in the training sample;

When the detection result represents that the illegal building in the preset construction state exists in the image to be detected, a comparison result of the image to be detected and a preset reference image is obtained, and a first early warning value of the image to be detected is calculated based on the comparison result and the detection result; wherein, the comparison result is: the first position coordinates of the region, in which the difference of the image content determined by comparing the image content of the image to be detected and the reference image meets a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

when the first early warning value is larger than a first early warning threshold value, first early warning information is generated; wherein, the first early warning information is: and early warning information about the illegal building in the preset construction state exists in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the determining manner of the comparison result includes:

cutting the image to be detected and the reference image respectively to obtain a plurality of pairs of image area blocks corresponding to the positions;

and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold value or not according to the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area blocks corresponding to each pair of positions as a comparison result.

Optionally, in a specific implementation manner, the detection result includes a second position coordinate and a confidence coefficient of an image area where the violation building in a predetermined construction state exists in the image to be detected;

the step of calculating the first early warning value of the image to be detected based on the comparison result and the detection result comprises the following steps:

and calculating a first early warning value of the image to be detected by using the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence.

Optionally, in a specific implementation manner, the image to be detected is an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to the image to be detected;

the step of calculating the first early warning value of the image to be detected by using the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence coefficient comprises the following steps:

calculating a first early warning value of the image to be detected by using a first formula; wherein, the first formula is:

Wherein W is a first early warning value of the image to be detected, x (i) is a region overlapping rate of a first position coordinate corresponding image region corresponding to the ith image and a second position coordinate corresponding image region, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is an image obtained by shooting the target shooting range for the ith time by the monitoring equipment.

Optionally, in a specific implementation manner, the calculating manner of the area overlapping ratio includes:

calculating the region overlapping rate by using a second formula; wherein the second formula is:

x(i)＝Area _i /(Area1 _i +Area2 _i -Area _i )

wherein, area _i The area of the overlapping area of the first position coordinate corresponding area and the second position coordinate corresponding area corresponding to the ith image; area1 _i The area of the area corresponding to the first position coordinate corresponding to the ith image; area2 _i And the area of the area corresponding to the second position coordinate corresponding to the ith image.

Optionally, in a specific implementation manner, the predetermined construction state is: a building state;

the violation detection model is specifically trained based on a training sample, an auxiliary sample, a sample label of the training sample and a sample label of the auxiliary sample, wherein the auxiliary sample is an image which has a violation building in an established state and is marked with an image area where the violation building is located, and the sample label of the auxiliary sample is in the established state;

The method further comprises the steps of:

when the detection result represents that the illegal building in the established state exists in the image to be detected, calculating a second early warning value of the image to be detected based on the detection result;

generating second early warning information when the second early warning value is larger than a second early warning threshold value; wherein, the second early warning information is: and the pre-warning information about the violation building in the established state exists in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the detection result includes a confidence level;

the step of calculating the second early warning value of the image to be detected based on the detection result comprises the following steps:

and calculating a second early warning value of the image to be detected by using the confidence coefficient.

the step of calculating the second early warning value of the image to be detected by using the confidence coefficient comprises the following steps:

calculating a second early warning value of the image to be detected by using a third formula; wherein the third formula is:

And E is a second early warning value of the image to be detected, y (i) is a confidence coefficient corresponding to the ith image, and the ith image is an image obtained by shooting the target shooting range for the ith time by the monitoring equipment.

In a second aspect, an embodiment of the present invention provides a device for warning a illegal building, where the device includes:

the image acquisition module is used for acquiring the image to be detected acquired by the monitoring equipment;

the image detection module is used for inputting the image to be detected into a preset illegal detection model to obtain a detection result output by the illegal detection model; the illegal detection model is obtained by training based on a training sample and a sample label of the training sample, and the training sample is: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of the training samples are: the construction state of the illegal building existing in the training sample;

the first early warning value calculation module is used for acquiring a comparison result of the image to be detected and a preset reference image when the detection result represents that the illegal building in the preset construction state exists in the image to be detected, and calculating a first early warning value of the image to be detected based on the comparison result and the detection result; wherein, the comparison result is: the first position coordinates of the region, in which the difference of the image content determined by comparing the image content of the image to be detected and the reference image meets a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

The first information generation module is used for generating first early warning information when the first early warning value is larger than a first early warning threshold value; wherein, the first early warning information is: and early warning information about the illegal building in the preset construction state exists in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the device further includes an image comparison module;

the image comparison module is used for determining the comparison result; the image comparison module is specifically used for:

cutting the image to be detected and the reference image respectively to obtain a plurality of pairs of image area blocks corresponding to the positions; and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold value or not according to the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area blocks corresponding to each pair of positions as a comparison result.

the first early warning value calculation module is specifically configured to: and calculating a first early warning value of the image to be detected by using the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence.

the first early warning value calculation module is specifically configured to:

x(i)＝Area _i /(Area1 _i +Area2 _i -Area _i )

wherein, area _i The area of the overlapping area of the first position coordinate corresponding area and the second position coordinate corresponding area corresponding to the ith image; area1 _i The area of the area corresponding to the first position coordinate corresponding to the ith image; area2 _i The ith image is corresponding to the ith imageThe two position coordinates correspond to the area of the region.

the apparatus further comprises:

the second early warning value calculation module is used for calculating a second early warning value of the image to be detected based on the detection result when the detection result represents that the violation building in the established state exists in the image to be detected;

the second information generation module is used for generating second early warning information when the second early warning value is larger than a second early warning threshold value; wherein, the second early warning information is: and the pre-warning information about the violation building in the established state exists in the target building included in the image to be detected.

the second early warning value calculation module is specifically configured to:

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

a memory for storing a computer program;

And the processor is used for realizing any one of the method steps in the violation early warning method provided in the first aspect when executing the program stored in the memory.

In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium, in which a computer program is stored, where the computer program, when executed by a processor, implements any of the method steps of a violation early warning method provided in the first aspect.

The above can be seen that by applying the scheme provided by the embodiment of the invention, aiming at the area needing to be subjected to the illegal building early warning, the area can be monitored by the monitoring equipment, so that the image to be detected acquired by the monitoring equipment is acquired. When the detection result obtained by detecting the image to be detected by using the preset illegal building detection model is used for representing that the illegal building in the preset building state exists in the image to be detected, the comparison result of the image to be detected and the preset reference image which is the same as the shooting range of the image to be detected can be obtained, the first early warning value of the image to be detected is calculated through the comparison result obtained by comparing and analyzing the image to be detected and the reference image and the detection result, and when the first early warning value is larger than the preset threshold value, early warning information about that the target building included in the image to be detected exists in the illegal building in the preset building state is generated. Therefore, after the image to be detected acquired by the monitoring equipment is obtained, the illegal building existing on the actual building can be found in time through the fusion application of the image comparison analysis method and the model detection method, and early warning information aiming at the preset construction state is generated, so that effective early warning of the illegal building is realized, and the dismantling difficulty and the dismantling cost are reduced.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a method for warning a offensive building according to an embodiment of the present invention;

FIG. 2 is a schematic view of a panoramic image corresponding to a high altitude dome camera;

FIG. 3 is a schematic diagram showing the correspondence between preset points and actual buildings;

FIG. 4 is a flow chart of a method of determining a comparison result;

FIGS. 5 (a) - (c) are each training samples for the presence of a offending building in an established state;

FIGS. 6 (a) - (c) are each training samples for the presence of a offending building in an as-built state;

FIG. 7 is a schematic flow chart of another method for warning a offending building according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a violation building early warning device according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In recent years, various illegal buildings not only occupy social public resources and urban space and destroy good image of cities, but also bring certain harm to market economic order and social management, thus being a great difficult problem to be solved in the urban management process. How to timely early warn the illegal buildings on the building to reduce the dismantling difficulty and the dismantling cost becomes a current problem to be solved urgently. In order to solve the technical problems, the embodiment of the invention provides a violation building early warning method.

In the following, first, an early warning method for illegal buildings provided by the embodiment of the invention is described.

Fig. 1 is a schematic diagram of a method for warning a offensive structure according to an embodiment of the present invention, where the method may be applied to any electronic device that needs to perform offensive structure warning, for example, a notebook computer, a desktop computer, a tablet computer, a mobile phone, a server, etc., and the embodiment of the present invention is not limited specifically, and is hereinafter referred to as an electronic device. The electronic equipment is in communication connection with the monitoring equipment, so that an image to be detected acquired by the monitoring equipment can be acquired, and further, the follow-up related steps are executed, and the early warning of the illegal building is realized.

As shown in fig. 1, the method for warning a illegal building provided by the embodiment of the invention may include the following steps:

s101: acquiring an image to be detected acquired by monitoring equipment;

s102: inputting the image to be detected into a preset illegal detection model to obtain a detection result output by the illegal detection model;

the illegal detection model is obtained by training based on training samples and sample labels of the training samples, and the training samples are as follows: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of training samples are: training the construction state of the illegal building existing in the sample;

s103: when the detection result represents that a violation building in a preset construction state exists in the image to be detected, a comparison result of the image to be detected and a preset reference image is obtained, and a first early warning value of the image to be detected is calculated based on the comparison result and the detection result;

wherein, the comparison result is: the method comprises the steps that image content differences determined by comparing image content of an image to be detected and image content of a reference image meet first position coordinates of an area of a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

S104: when the first early warning value is larger than a first early warning threshold value, generating first early warning information;

the first early warning information is as follows: and the pre-warning information about the illegal building in a preset construction state exists in the target building included in the image to be detected.

The above can be seen that by applying the scheme provided by the embodiment of the invention, aiming at the area needing to be subjected to the illegal building early warning, the area can be monitored by the monitoring equipment, so that the image to be detected acquired by the monitoring equipment is acquired. When the detection result obtained by detecting the image to be detected by using the preset illegal building detection model is used for representing that the illegal building in the preset building state exists in the image to be detected, the comparison result of the image to be detected and the preset reference image which is the same as the shooting range of the image to be detected can be obtained, the first early warning value of the image to be detected is calculated through the comparison result obtained by comparing and analyzing the image to be detected and the reference image and the detection result, and when the first early warning value is larger than the preset threshold value, early warning information about that the target building included in the image to be detected exists in the illegal building in the preset building state is generated. Therefore, after the image to be detected acquired by the monitoring equipment is obtained, the illegal building existing on the actual building can be found in time through the fusion application of the image comparison analysis method and the model detection method, so that the dismantling difficulty and the dismantling cost are reduced.

In the step S101, after the monitoring device collects the image to be detected, the collected image to be detected may be sent to the electronic device. Thus, the electronic device can acquire the image to be detected acquired by the monitoring device.

The monitoring device may be a high-altitude dome camera or a monitoring device mounted on an unmanned aerial vehicle, for example, a camera mounted on the unmanned aerial vehicle. Of course, the monitoring device may be other types of image acquisition devices. This is reasonable.

Next, taking the high-altitude balloon machine as an example, the monitoring device in the above step S101 will be described. Specific:

when the monitoring device is a high-altitude dome camera, the shooting range of the high-altitude dome camera may be determined first before executing the method for warning the illegal building provided by the embodiment of the invention.

For example, first, the horizontal viewing angle of the high altitude balloon machine is maintained at 0 degrees, and then the high altitude balloon machine is rotated 180 degrees in the vertical direction, and at every 30 degrees of rotation, an image photographed by the high altitude balloon machine is acquired; then, rotating the high-altitude dome camera in the horizontal direction, keeping the horizontal visual angle of the high-altitude dome camera at 30 degrees, further rotating the high-altitude dome camera in the vertical direction again by 180 degrees, and acquiring an image shot by the high-altitude dome camera when rotating every 30 degrees; then, continuously rotating the high-altitude ball machine in the horizontal direction, keeping the horizontal visual angle of the high-altitude ball machine at 60 degrees, further rotating the high-altitude ball machine in the vertical direction again by 180 degrees, and acquiring an image shot by the high-altitude ball machine when rotating every 30 degrees; and so on until the horizontal viewing angle of the high-altitude dome camera is kept at 90 degrees, and then the high-altitude dome camera is rotated again by 180 degrees in the vertical direction, and at every 30 degrees of rotation, an image shot by the high-altitude dome camera is acquired.

Therefore, the acquired images can be spliced by using an image splicing algorithm, so that a panoramic image corresponding to the high-altitude dome camera is obtained, and the content included in the panoramic image is the shooting range of the high-altitude dome camera. Each building in the panoramic image may have address or number information forming a specific location sample tag. The address or the number information can be directly obtained from the GIS or manually marked by people, which is reasonable.

For example, as shown in fig. 2, a schematic view of a panoramic image corresponding to a high-altitude dome camera is shown, where 210 is the high-altitude dome camera, a position sample label of each building is number information, and through the panoramic image, it can be determined that a shooting range of the high-altitude dome camera is the residential building 1-15. Further, the address information of the shooting range of the high altitude dome camera can be further determined according to a predetermined correspondence between the number of each residential building and the real address.

Of course, for different high-altitude ball machines and different positions where the high-altitude ball machines are installed, other ways of determining the shooting range of the high-altitude ball machines may be adopted, for example, changing the viewing angle change rule in the above way. This is reasonable.

The electronic device may directly determine the shooting range of the high altitude dome camera, or may determine the shooting range of the high altitude dome camera by using other electronic devices connected in communication, and the electronic device may further obtain the determined shooting range of the high altitude dome camera from the other electronic devices.

Similarly, when the monitoring device is a monitoring device installed on an unmanned aerial vehicle, before executing the method for warning the illegal building provided by the embodiment of the invention, the shooting range of the monitoring device installed on the unmanned aerial vehicle may be determined first.

For example, according to a preset cruising route of the unmanned aerial vehicle, after the unmanned aerial vehicle flies for a preset time or a preset flight distance, an image acquired by monitoring equipment erected on the unmanned aerial vehicle is acquired. Therefore, after the unmanned aerial vehicle finishes one-time cruising route flight, the images acquired in the flight process can be spliced, so that a panoramic image corresponding to the monitoring equipment erected on the unmanned aerial vehicle is obtained, and the content included in the panoramic image is the shooting range of the monitoring equipment erected on the unmanned aerial vehicle. The preset cruising route, the preset time and the preset distance can be defined according to the requirements in practical application.

Further, it may be understood that the image to be detected collected by the monitoring device may include a plurality of actual buildings, and the monitoring device may not be able to capture details of each actual building based on the physical parameters of the monitoring device itself.

Based on the method, in order to quickly determine the building with the illegal building when the early warning information is generated, the problem that the address information of the building with the illegal building cannot be distinguished from the image to be detected is avoided.

Optionally, in a specific implementation manner, a plurality of preset points may be preset, and a corresponding relationship is established between each preset point and address information of an actual building in a shooting range corresponding to the preset point when the monitoring device is located. In the corresponding relation, the actual building in the shooting range corresponding to the monitoring equipment corresponding to each preset point is the actual building in which the monitoring equipment can shoot the detail information at the preset point. When a plurality of actual buildings exist in the image to be detected, which is shot at a certain preset point, a specific actual building area corresponding to the preset point can be further configured.

As shown in fig. 3, the actual building area included in the image frame 310 is a specific actual building area corresponding to a preset point where the monitoring device is located when the image to be detected shown in fig. 3 is taken. In addition, the area of the image to be detected, which does not need to be analyzed or possibly involves privacy, needs to be configured to protect the privacy of the person associated with the actual building area that is photographed. Specifically, the region of the image to be detected, which does not need to be analyzed or possibly involves privacy, may be blocked. Such as the actual building area included in image box 320 in fig. 3.

Based on this, it can be understood that when an area that does not need to be analyzed or that may involve privacy in the image to be detected is configured, then, for a reference image that is the same as the shooting range of the image to be detected, an area that does not need to be analyzed or that may involve privacy in the reference image is also configured.

Thus, in this embodiment, in step S101, the image to be detected acquired by the monitoring device and acquired by the electronic device is the image to be detected captured by the monitoring device at a certain preset point. When the electronic device generates early warning information for the illegal building with the image to be detected, the electronic device can determine the preset points where the monitoring device is located when shooting the image to be detected, and further determine the address information of the actual building with the illegal building in the image to be detected through the corresponding relation between each preset point of the monitoring device and the position sample label of the actual building.

When the monitoring device sends the acquired image to be detected to the monitoring device, the monitoring device can send preset point information of the monitoring device when the detection image is shot. For example, the number of the preset point where the detection image itself is located when the detection image is taken, and the like are simultaneously transmitted to the monitoring device. This is reasonable.

Further, in this embodiment, in step S101, the reference image obtained by the electronic device and the image capturing range to be detected may be: before a preset period, the monitoring equipment shoots the image at the preset point when the image to be detected is shot. Obviously, the reference image has a correspondence relationship with the address information of the actual building in the shooting range corresponding to the preset point.

For example, the image to be detected is the image shot by the monitoring device at the preset point 1, and the reference image may be the image shot by the monitoring device at the preset point 1 30 days before.

Based on the above description, when the monitoring device is the high-altitude dome camera, the preset points are a plurality of shooting visual angles selected in a shooting time range which can be reached by the high-altitude dome camera. When the shooting visual angle of the high-altitude dome camera rotates to the shooting visual angle corresponding to a certain preset point, the pixels of the shot image to be detected can be adjusted through focal length adjustment, so that the pixels of the shot image of the actual building corresponding to the preset point can meet the pixel requirements of subsequent image comparison and image detection.

Further, the residence time and the training interval of the high altitude balloon machine at each preset point can be preset.

For example, assuming that 10 preset points are preset, the residence time of the high altitude balloon machine at each preset point is 20 minutes, the training period of the 10 preset points is 200 minutes, and the training interval is set to 100 minutes.

In order to avoid image blurring caused by rotation of the high-altitude dome camera in an image to be detected shot by each preset point, when the high-altitude dome camera rotates to a shooting view angle corresponding to each preset point, the high-altitude dome camera can send the acquired image to the electronic equipment as the image to be detected when the duration of the shooting view angle is up to a certain length and the stay time is over. Therefore, image blurring caused by rotation of the high-altitude dome camera is avoided when the high-altitude dome camera rotates from other shooting visual angles to the shooting visual angle corresponding to each preset point or from the shooting visual angle corresponding to each preset point to other shooting visual angles.

For example, when the residence time of the high-altitude dome camera at each preset point is 20 minutes, the high-altitude dome camera may send the acquired image as an image to be detected to the electronic device when the duration of the high-altitude dome camera at the shooting viewing angle reaches 10 minutes. Of course, the acquired image may be sent to the electronic device as the image to be detected when the duration of the shooting angle reaches 8 minutes, 9 minutes, 30 seconds, or 12 minutes. The high-altitude ball machine is ensured to be in a shooting visual angle for a certain length, and the stay time is finished for a certain length.

When the monitoring device is a monitoring device erected on the unmanned aerial vehicle, a plurality of cruising positions on the cruising route can be set as preset points according to a preset cruising route and a flight cycle. Of course, the cruising interval of the unmanned aerial vehicle may also be set.

Therefore, the monitoring equipment erected on the unmanned aerial vehicle can take the image shot at the cruising position as the image to be detected when the unmanned aerial vehicle flies to the cruising position corresponding to each preset point, and sends the image to the electronic equipment.

After the step S101 is performed and the image to be detected is obtained, the electronic device may continue to perform the step S102, and input the image to be detected into a preset detection model for detecting the failure in construction, so as to obtain a detection result output by the detection model for detecting the failure in construction.

The illegal detection model is obtained by training based on training samples and sample labels of the training samples, and the training samples are as follows: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of training samples are: training the construction state of the illegal building existing in the sample.

Alternatively, in a specific implementation manner, the predetermined building state may be a building state and an established state, where the established state refers to that the offending building is already built, as shown in fig. 5 (a) - (c), and each is a training sample of the offending building in the established state. The building state refers to a training sample of a building in which a violation is present in the building state, as shown in fig. 6 (a) to (c), respectively, when the violation is not completed in the building process.

Based on the above, a large number of training samples can be obtained, the training samples are images including the offending buildings in the building state or the established state, the image areas where the offending buildings exist are marked in the training samples, and the building states of the offending buildings included in the training samples are determined as sample labels of the training samples.

Wherein, for the image comprising the violation building in the established state, the image area where the violation building exists can be determined by the characteristics of various types of established violation buildings. However, for an image including a offending building in a built state, since the offending building is not yet formed, an image area where the offending building exists cannot be determined by characteristics of various types of built offending buildings, but it is necessary to determine an image area where the offending building exists by characteristics related to construction processes, for example, there are more than 3 persons continuously staying, there are construction building materials (e.g., semi-formed steel bars, reinforced concrete, bricks, etc.), there are construction covers, etc.

For example, for fig. 5 (a), 510 is a label of an image area where the built color board house violation building is located, and the sample label is in a built state; for fig. 5 (b), 520 is a label of an image area where the built glass house violation building is located, and the sample label is in a built state; for fig. 5 (c), 530 is the annotation of the image area where the established layer-added offending building is located, and the sample tag is the established state.

For another example, for fig. 6 (a), 610 is a label of an image area where the tarpaulin for construction masking is located, and the sample label is in a construction state; for fig. 6 (b), 620 is the label of the image area where the constructor is located, and the sample label is in the construction state; for fig. 6 (c), 630 is the annotation of the image area where the construction scene is located, and the sample tag is in the build state.

Thus, in this specific implementation manner, after obtaining a plurality of training samples and sample labels of each training sample, the obtained plurality of training samples and sample labels of each training sample may be used to train a preset initial model, so that the preset initial model learns the characteristics of the offending building and the coordinates of the image area where the offending building is located in the training sample, and learns the sample labels of the training sample, and gradually establishes the correspondence between the characteristics of the offending building and the coordinates of the image area where the offending building is located in the training sample, and the sample labels of the training sample, through learning a large number of training samples. And stopping training when the preset convergence condition is met, and obtaining the training-completed illegal building detection model.

Furthermore, in this specific implementation manner, when executing the step S103, the electronic device may input the image to be detected into the training-completed illegal building detection model, and further, detect whether the illegal building exists in the image to be detected by using the illegal building detection model, and detect the building type of the illegal building existing and the image area where the illegal building exists in the image to be detected when detecting that the illegal building exists in the image to be detected, so as to obtain the detection result output by the illegal building detection model.

The detection result output by the illegal building detection model may include a construction state of the illegal building, where the construction state of the illegal building is any one of the following three states: the condition of the building with violations does not exist, the condition of the built building with violations and the condition of the building with violations is built. The condition of the non-illegal building indicates that the non-illegal building exists in the image to be detected; the built illegal building state characterizes that the illegal building in the built state exists in the image to be detected; and representing that the illegal building in the building state exists in the image to be detected.

Obviously, in this embodiment, since the predetermined construction state is a construction state and a built state, when the detection result output by the illegal construction detection model includes the built illegal construction state or is in the built illegal construction state, the detection result may indicate that the illegal construction in the predetermined construction state exists in the image to be detected, and then the electronic device may continue to execute the subsequent step S103.

In addition, as the feature of the illegal building existing in the training sample, the coordinates of the image area where the illegal building is located and the corresponding relation of the sample label of the training sample can be established in the training process of the illegal building detection model, when the detection result output by the illegal building detection model comprises the established illegal building state or the illegal building state, the detection result can also comprise the second position coordinates of the image area where the illegal building in the established illegal building state or the illegal building state existing in the image to be detected.

In the specific implementation manner, the predetermined construction state may be a construction state corresponding to the established state and the established state, and in another specific implementation manner, the predetermined construction state may be a construction state; in still another specific implementation, the predetermined construction state may be an established state. Obviously, in the two specific implementations, the manner in which the electronic device performs the above step S102 is similar to that in the specific implementations, and the training manner of the detection model is also similar to that in the specific implementations, so that the details are not repeated here.

The training process of the violation detection model can be completed in the electronic device, or can be completed in other electronic devices in communication connection with the electronic device, and the training process is sent to the electronic device by the other electronic devices, which is reasonable. In addition, the embodiment of the invention also does not limit the sources of the training samples for training the illegal detection model.

In addition, in one embodiment, when the above-mentioned violation detection model is trained, the type of the violation building in the built state, for example, a color board room, a glass room, a tile-shaped house, etc. may also be marked in the training sample. Thus, when the output result of the violation detection model characterizes that the violation building in the built state exists in the image to be detected, the detection result can also comprise the type of the existing violation building in the built state.

Similarly, when the above-mentioned violation detection model is trained, the construction information of the violation building in the building state, for example, constructors, construction scenes, construction covers, construction building materials, and the like, may be marked in the training sample. Thus, when the output result of the violation detection model characterizes that the violation building in the building state exists in the image to be detected, the detection result can also comprise the existing building information of the violation building in the building state.

After the step S102 is performed to obtain the detection result output by the illegal building detection model, the electronic device may determine whether the illegal building in the predetermined construction state exists in the image to be detected according to the construction state of the illegal building included in the detection result. And when the electronic device determines that the detection result output by the illegal building detection model represents that the illegal building in the preset building state exists in the image to be detected, the electronic device can continue to execute the step S103, namely, a comparison result of the image to be detected and the preset reference image is obtained, and a first early warning value of the image to be detected is calculated based on the obtained comparison result and the detection result obtained in the step S102.

Wherein, the obtained comparison result is as follows: comparing the image content of the image to be detected with the image content of the reference image, and obtaining a first position coordinate of an area with the difference of the image content meeting a preset difference rule. Therefore, the electronic device performs the above step S103 specifically as follows: and calculating a first early warning value of the image to be detected based on the obtained first position coordinates of the region where the difference of the obtained image content meets the preset difference rule and the detection result obtained in S103 by comparing the image content of the image to be detected with the image content of the reference image.

Preferably, in the step S103, the comparison result obtained by the electronic device may be determined when it is determined that the detection result represents that the illegal building in the predetermined construction state exists in the image to be detected. That is, when it is determined that the detection result indicates that a violation building in a predetermined construction state exists in the image to be detected, the electronic device acquires a preset reference image with the same shooting range as the image to be detected, and further, compares the image content of the image to be detected with the image content of the preset reference image, and obtains a first position coordinate of an area where the difference of the obtained image content meets a preset difference rule, thereby obtaining the comparison result.

Preferably, in the step S103, the comparison result obtained by the electronic device may be predetermined. That is, after the electronic device obtains the image to be detected, the electronic device may obtain a preset reference image with the same shooting range as the image to be detected, and compare the image content of the image to be detected with the image content of the preset reference image, and obtain the first position coordinates of the region where the difference of the obtained image content meets the preset difference rule, so as to obtain the comparison result.

The electronic device may acquire the reference image in various manners, for example, the electronic device may acquire the reference image in a plurality of locally stored images, or the electronic device may acquire the reference image at other electronic devices connected in communication, which is not limited in particular.

In addition, the electronic device may determine the comparison result in a plurality of ways, that is, the electronic device may compare the image content of the image to be detected and the image content of the reference image in a plurality of ways, so as to obtain the first position coordinate of the region where the difference of the image content meets the preset difference rule.

Optionally, in a specific implementation manner, as shown in fig. 4, the manner in which the electronic device determines the comparison result may include the following steps:

s401: cutting the image to be detected and the reference image respectively to obtain image area blocks corresponding to a plurality of pairs of positions;

for the obtained image to be detected and the reference image, the electronic device wants to determine the region with smaller similarity in the image to be detected and the reference image, so that the electronic device can cut the image to be detected and the reference image respectively, and cut the image to be detected and the reference image into a plurality of image region blocks respectively, and obtain image region blocks corresponding to a plurality of pairs of positions.

Specific: the shooting range corresponding to each image area block corresponding to the image to be detected is a sub-range of the shooting range of the image to be detected. Similarly, since the shooting range of the reference image is the same as the shooting range of the image to be detected, the shooting range corresponding to each image block corresponding to the reference image is also a sub-range of the shooting range of the image to be detected.

In order to determine the region with smaller similarity between the image to be detected and the reference image, the shooting range corresponding to the image region block should be the same as the shooting range corresponding to a certain image block among the plurality of image blocks corresponding to the reference image for each image region block corresponding to the image to be detected. That is, for each image area block corresponding to the image to be detected, the position of the image area block in the image to be detected should correspond to the position of a certain image block in the reference image among the plurality of image blocks corresponding to the reference image.

Based on the method, the electronic equipment can cut the image to be detected and the reference image respectively to obtain image area blocks corresponding to a plurality of pairs of positions. Therefore, the number of the image area blocks obtained by cutting the image to be detected by the electronic device is the same as the number of the image area blocks obtained by cutting the reference image, and for each image area block corresponding to the image to be detected, a pair of image area blocks corresponding to the position can be obtained by only one image area block corresponding to the position and a plurality of image area blocks corresponding to the reference image exist.

The electronic device may cut the image to be detected and the reference image in various manners, such as a gray threshold segmentation method, an edge-based segmentation method, a clustering analysis-based segmentation method, and the like, which is not particularly limited in the embodiment of the present invention.

In addition, in order to reduce shooting range errors caused by image sizes, position deviations and the like of the image to be detected and the reference image, the shooting ranges corresponding to the obtained image area blocks corresponding to the multiple pairs of positions are more similar, the accuracy of the comparison result is improved, and the accuracy of the early warning result is finally improved.

Alternatively, before performing the above step S401, the electronic device may perform image alignment of the image to be detected and the reference image using an image analysis algorithm, and correct positional deviation of the two images. Furthermore, the electronic device may cut the image to be detected and the reference image obtained after the image alignment and the positional deviation correction, respectively, to obtain image area blocks corresponding to the plurality of pairs of positions.

The electronic device may perform image alignment on the image to be detected and the reference image through various image analysis algorithms, and correct the positional deviation of the two images, which is not specifically limited in the embodiment of the present invention. For example Harris corner detection and feature matching algorithms, etc.

S402: and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold value or not according to the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area blocks corresponding to each pair of positions as a comparison result.

After the step S402 is performed to obtain image area blocks corresponding to multiple pairs of positions, the electronic device may determine, for each pair of image area blocks corresponding to the positions, whether the similarity of the image area blocks corresponding to the positions is smaller than a preset similarity threshold, and if the determination result is yes, obtain a first position coordinate of the image area block corresponding to the positions, as a comparison result.

The electronic equipment can compare the image area blocks corresponding to each pair of positions through various image comparison algorithms, and the similarity of the image area blocks corresponding to the positions is obtained. The embodiment of the present invention is not particularly limited. For example, a similarity discrimination algorithm based on a 2-channel network (two-channel network structure).

In addition, the specific value of the preset similarity threshold may be defined according to the requirement in practical application, and the embodiment of the present invention does not define the specific value of the preset similarity threshold.

Optionally, when the similarity of the image area blocks corresponding to the pair of positions is smaller than the preset similarity threshold, the electronic device may use the first position coordinates of the image area blocks corresponding to the pair of positions as the comparison result.

Optionally, when the similarity of the image area blocks corresponding to the multiple pairs of positions is smaller than a preset similarity threshold, the electronic device may use the first position coordinates of the image area blocks corresponding to the multiple pairs of positions as a comparison result.

Optionally, when the similarity of the image area blocks corresponding to the plurality of pairs of positions is smaller than a preset similarity threshold, the electronic device may use the first position coordinates of the image area block corresponding to the pair of positions with the smallest similarity among the image area blocks corresponding to the plurality of pairs of positions as the comparison result.

Optionally, when the similarity of the image area blocks corresponding to the plurality of pairs of positions is smaller than a preset similarity threshold, the electronic device determines first position coordinates in the image area blocks corresponding to the plurality of pairs of positions, further, according to the determined first position coordinates, in the image to be detected, a minimum image area range capable of containing the image area block belonging to the image area block to be detected in the image area blocks corresponding to the plurality of pairs of positions is determined, and the determined coordinates of the minimum image area range are used as a comparison result. Of course, the electronic device also determines, in the reference image, a minimum image area range capable of containing an image area block belonging to the reference image among the image area blocks corresponding to the plurality of pairs of positions, based on the determined plurality of first position coordinates, and uses the coordinates of the determined minimum image area range as a comparison result. Wherein the coordinates of the minimum image area range determined in the image to be detected are the same as the coordinates of the minimum image area range determined in the reference image.

Thus, after the step S103 is performed and the first warning value of the image to be detected is obtained, the electronic device may continue to perform the step S104, that is, determine whether the calculated first warning value is greater than the first warning threshold, and generate warning information about the presence of a violation building in a predetermined construction state in the target building included in the image to be detected when the determination result is that the first warning value is greater than the first warning threshold, that is, generate the first warning information.

Wherein the target building is a building which is included in the image to be detected and has the illegal building in the preset construction state detected in the step S102; the specific value of the first early warning threshold may be set according to the actual application requirement, and the embodiment of the present invention does not limit the specific value of the first early warning threshold.

When the image to be detected is an image captured by the monitoring device when the image is at a preset point, the first early warning information generated in the step S104 is early warning information about that the actual building corresponding to the preset point has a violation building in a preset construction state.

The electronic device may execute the step S103 in a plurality of manners, that is, calculate the first early warning value of the image to be detected based on the comparison result and the detection result in a plurality of manners. The embodiment of the present invention is not particularly limited in this regard.

Optionally, in a specific implementation manner, the detection result output by the violation detection model may include a second position coordinate and a confidence coefficient of an image area where the violation building in a predetermined construction state exists in the image to be detected; therefore, in this specific implementation manner, the manner in which the electronic device performs the above step S103 may include the following steps:

The electronic device may calculate the first early warning value of the image to be detected according to the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence, which are not limited in detail.

Optionally, in an embodiment, the image to be detected may be an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to the image to be detected; therefore, in this embodiment, the manner of calculating the first early warning value of the image to be detected by the electronic device using the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence level may include the following steps:

calculating a first early warning value of an image to be detected by using a first formula; wherein, the first formula is:

wherein W is a first early warning value of an image to be detected, x (i) is the region overlapping rate of a first position coordinate corresponding image region corresponding to an ith image and a second position coordinate corresponding image region, y (i) is the confidence coefficient corresponding to the ith image, and the ith image is the image to be detected obtained by shooting the target shooting range for the ith time by the monitoring equipment.

Further, optionally, the calculating manner of the region overlapping ratio in the first formula includes:

calculating the region overlapping rate by using a second formula; wherein, the second formula is:

x(i)＝Area _i /(Area1 _i +Area2 _i -Area _i )

wherein, area _i The area of the overlapping area of the first position coordinate corresponding area and the second position coordinate corresponding area corresponding to the ith image; area1 _i The area of the area corresponding to the first position coordinate corresponding to the ith image; area2 _i The area of the area corresponding to the second position coordinate corresponding to the ith image.

Obviously, in this embodiment, when the image to be detected is an image captured by the monitoring device when the monitoring device is at a certain preset point, the image to be detected may be an image captured by the monitoring device when the monitoring device is at the preset point for the nth time, and the ith image is an image to be detected captured by the monitoring device when the monitoring device is at the preset point for the ith time.

In addition, since there may be a plurality of illegal buildings in a predetermined construction state in a certain image or a plurality of images to be detected, which are obtained by photographing the target photographing range by the monitoring device, such images to be detected may correspond to a plurality of second position coordinates and a plurality of confidence degrees, and each second position coordinate corresponds to one confidence degree.

Therefore, for such an image to be detected, the highest confidence corresponding to the image to be detected may be determined as the confidence utilized in the first formula, and the region overlapping rate in the first formula is the region overlapping rate of the first position coordinate corresponding image region corresponding to the image to be detected and the second position coordinate corresponding image region corresponding to the highest confidence corresponding to the image to be detected. Furthermore, the area of the second location coordinate corresponding region corresponding to the to-be-detected image utilized in the second formula is the area of the second location coordinate corresponding to the highest confidence corresponding to the to-be-detected image.

Further, for the second formula:

preferably, when aiming at the ith image, the electronic equipmentWhen the comparison result determined in the step S102 includes a first position coordinate, the Area1 in the second formula _i The area of the area corresponding to the first position coordinate corresponding to the determined ith image.

Preferably, when the electronic device includes a plurality of first position coordinates in the comparison result determined in the above step S102 for the ith image, the electronic device may regard the Area of each first position coordinate corresponding region as Area1 in the above second formula _i Thus, a plurality of x (i) are calculated, and further, the calculated maximum x (i) can be determined as the region overlapping rate employed in the above-described first formula.

Preferably, when the electronic device includes a plurality of first position coordinates in the comparison result determined in the step S102 for the ith image, the electronic device may further obtain a similarity corresponding to each first position coordinate, thereby determining Area1 in the second formula _i And determining the area of the first position coordinate corresponding area corresponding to the minimum similarity.

In addition, in the first formula, the specific value of N may be defined according to the actual application requirement, and the embodiment of the present invention does not limit the specific value of N.

Obviously, when n=1, the first formula described above can be modified as:

W＝(x+y)/2

wherein W is a first early warning value of the image to be detected, x is the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate corresponding to the image to be detected, and y is the confidence corresponding to the image to be detected.

Further, the second formula may be modified as follows:

x＝Area/(Area1+Area2-Area)

the Area is the Area of the overlapping Area of the first position coordinate corresponding Area and the second position coordinate corresponding Area corresponding to the image to be detected, and the Area1 is the Area of the first position coordinate corresponding Area corresponding to the image to be detected; area2 is the Area of the region corresponding to the second position coordinate corresponding to the image to be detected.

Further, in order to better distinguish whether the to-be-detected image comprises the violation building in the built state or the violation building in the built state, different violation building early warning methods can be adopted for the violation buildings in different built states.

Optionally, in a specific implementation manner, the predetermined construction state may be: a building state;

thus, the violation detection model applied in the above step S102: the violation detection model is specifically trained based on a training sample, an auxiliary sample, a sample label of the training sample and a sample label of the auxiliary sample, wherein the auxiliary sample is an image which is in an established state and marked with an image area where the violation building is located, and the sample label of the auxiliary sample is in the established state.

In this specific implementation manner, since the predetermined construction state may be a construction state, the training violation detection model is an image in which the training sample is used as an image in which a violation building in the construction state exists and an image area in which the violation building is located is marked; sample labels of training samples are: and (5) establishing a state. Therefore, in order to enable the violation detection model to detect the violation building in the built state existing in the image to be detected, the violation detection model can be obtained through training by using the training sample, the auxiliary sample, the sample label of the training sample and the sample label of the auxiliary sample. Obviously, the auxiliary sample is an image of the illegal building in the established state and marked with the image area of the illegal building, and the sample label of the auxiliary sample is the established state.

Thus, in this specific implementation manner, as shown in fig. 7, another method for warning a illegal building provided in the above embodiment of the present invention may further include the following steps:

s105: when the detection result represents that the illegal building in the established state exists in the image to be detected, calculating a second early warning value of the image to be detected based on the detection result;

s106: generating second early warning information when the second early warning value is larger than a second early warning threshold value;

the second early warning information is as follows: and the pre-warning information about the violation building in the established state exists in the target building included in the image to be detected.

That is, when the detection result output by the illegal building detection model indicates that there is an illegal building in the built state in the image to be detected, the electronic device cannot execute the above steps S103 and S104 because the built state does not belong to the above predetermined building state in the present implementation. Based on this, the electronic device may continue to execute the step S105, that is, calculate the second early warning value of the image to be detected based on the detection result obtained in the step S102.

Thus, after the step S105 is performed and the second warning value of the image to be detected is obtained, the electronic device may continue to perform the step S106, that is, determine whether the calculated second warning value is greater than the second warning threshold, and generate warning information about the presence of the violation building in the built state of the target building included in the image to be detected when the second warning value is greater than the second warning threshold as a result of the determination, that is, generate the second warning information.

The specific value of the second early warning threshold may be set according to the actual application requirement, and the embodiment of the present invention does not limit the specific value of the second early warning threshold.

When the image to be detected is an image captured by the monitoring device at a preset point, the second early warning information generated in the step S106 is early warning information about that the actual building corresponding to the preset point has a violation building in an established state.

In this embodiment, the steps S101 to S104 are the same as the embodiments of steps S101 to S104 shown in the embodiment of fig. 1, and are not described herein.

The electronic device may perform the step S105 in a plurality of manners, that is, calculate the second early warning value of the image to be detected based on the detection result in a plurality of manners. The embodiment of the present invention is not particularly limited in this regard.

Optionally, in one embodiment, the detection result output by the violation detection model may include a confidence level; thus, the manner in which the electronic device performs the above step S105 may include the steps of:

Preferably, the image to be detected may be an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to the image to be detected, so the method for calculating the second early warning value of the image to be detected by the electronic device through the confidence coefficient can comprise the following steps:

Calculating a second early warning value of the image to be detected by using a third formula; wherein, the third formula is:

and E is a second early warning value of the image to be detected, y (i) is the confidence corresponding to the ith image, and the ith image is the image to be detected obtained by shooting the target shooting range for the ith time by the monitoring equipment.

In addition, since there may be a plurality of illegal buildings in a predetermined construction state in a certain image or a plurality of images to be detected, which are obtained by photographing the target photographing range by the monitoring device, such images to be detected may correspond to a plurality of second position coordinates and a plurality of confidence degrees, and each second position coordinate corresponds to one confidence degree. Thus, for such an image to be detected, the highest confidence corresponding to the image to be detected may be determined as the confidence utilized in the above-described third formula.

In addition, in the third formula, the specific value of N may be defined according to the actual application requirement, and the embodiment of the present invention does not limit the specific value of N.

Obviously, when n=1, the first formula described above can be modified as:

E＝y(i)

the second early warning value of the image to be detected is the confidence corresponding to the image to be detected.

In addition, based on the description of the embodiments provided in the embodiments of the present invention, in a specific implementation manner, the predetermined construction state may be a construction state and a built state, and the detection result output by the non-construction detection model includes the second position coordinates and the confidence coefficient of the image area where the construction state or the built state is located, which exist in the image to be detected; thus, in this particular implementation:

when the detection result output by the illegal building detection model represents that the illegal building in a building state exists in the image to be detected, the electronic equipment can calculate a first early warning value of the image to be detected by utilizing the region overlapping rate of the image region corresponding to the first position coordinate in the comparison result and the image region corresponding to the second position coordinate in the detection result and the confidence coefficient;

the electronic device may further calculate the first early warning value by using the first formula and the second formula.

When the detection result output by the illegal building detection model represents that the illegal building in the built state exists in the image to be detected, the electronic equipment can calculate a second early warning value of the image to be detected by utilizing the image area corresponding to the first position coordinate in the comparison result and the confidence in the detection result;

the electronic device may further calculate the second early warning value according to the third formula.

In this embodiment, the specific implementation manner of each step is the same as the specific implementation manner of the same step provided in each embodiment in the embodiment of the present invention, and is not described herein again.

Obviously, in this specific implementation manner, although the predetermined construction state is a construction state and an established state, it is still possible to distinguish whether the to-be-detected image includes a violation building in the construction state or a violation building in the established state, and different violation building early warning methods are adopted for the violation buildings in different construction states.

In addition, after the first early warning information or the second early warning information is obtained, as the monitoring device can be a video acquisition device, the monitoring device can shoot a video of a shooting range, so that a user can call videos in a period before and after the moment when the monitoring device shoots the image to be detected in the video shot by the monitoring device. Further, the user can further determine the specific situation of the offending building in the building state or the established state included in the image to be detected and the specific situation of the actual building in which the offending building exists by observing the image information of the actual building in the video. Meanwhile, the user can acquire more specific information of the actual building from the recorded information about the actual building. Such as the owner of the actual building, the user of the actual building, the time of construction of the actual building, whether or not a offending building has occurred before the actual building, etc.

Further, after the information is obtained, the user can archive and package the related information, so that the information can be pushed to related government staff to facilitate subsequent processing, such as dismantling illegal buildings and the like.

Compared with the method for warning the illegal building provided by the embodiment of the invention, the embodiment of the invention also provides a device for warning the illegal building.

Fig. 8 is a schematic structural diagram of a violation building early warning device according to an embodiment of the present invention. As shown in fig. 8, the violation building early warning device may include the following modules:

the image acquisition module 810 is configured to acquire an image to be detected acquired by the monitoring device;

the image detection module 820 is configured to input an image to be detected into a preset illegal detection model, so as to obtain a detection result output by the illegal detection model; the illegal detection model is obtained by training based on training samples and sample labels of the training samples, and the training samples are as follows: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of training samples are: training the construction state of the illegal building existing in the sample;

The first early warning value calculation module 830 is configured to obtain a comparison result of the to-be-detected image and a preset reference image when the detection result represents that a violation building in a predetermined construction state exists in the to-be-detected image, and calculate a first early warning value of the to-be-detected image based on the comparison result and the detection result; wherein, the comparison result is: the first position coordinates of the region, in which the difference of the image content determined by comparing the image content of the image to be detected and the reference image meets a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

the first information generating module 840 is configured to generate first early warning information when the first early warning value is greater than the first early warning threshold; the first early warning information is as follows: and the pre-warning information about the illegal building in a preset construction state exists in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the above-mentioned illegal building early warning device may further include an image comparison module;

in this specific implementation manner, the image comparison module is configured to determine the comparison result;

specifically, the image comparison module may be specifically configured to:

cutting the image to be detected and the reference image respectively to obtain image area blocks corresponding to a plurality of pairs of positions; and judging whether the similarity of the image area blocks corresponding to each pair of positions is smaller than a preset similarity threshold value or not according to the image area blocks corresponding to each pair of positions, and if so, obtaining a first position coordinate of the image area blocks corresponding to each pair of positions as a comparison result.

Optionally, in a specific implementation manner, the detection result may include a second position coordinate and a confidence coefficient of an image area where the violation building in a predetermined construction state exists in the image to be detected;

thus, in this specific implementation, the first early warning value calculation module 830 may be specifically configured to: and calculating a first early warning value of the image to be detected by using the region overlapping rate of the image region corresponding to the first position coordinate and the image region corresponding to the second position coordinate and the confidence.

Optionally, in a specific implementation manner, the image to be detected may be an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range may be a shooting range corresponding to the image to be detected;

thus, in this specific implementation, the first early warning value calculation module 830 may be specifically configured to:

Optionally, in a specific implementation manner, the calculating manner of the above area overlapping ratio may include:

x(i)＝Area _i /(Area1 _i +Area2 _i -Area _i )

therefore, in this specific implementation manner, the above-mentioned violation detection model may be specifically trained based on a training sample, an auxiliary sample, a sample label of the training sample, and a sample label of the auxiliary sample, where the auxiliary sample is an image that has a violation building in an established state and is marked with an image area where the violation building is located, and the sample label of the auxiliary sample is in the established state;

furthermore, the device for warning the illegal building provided by the embodiment of the invention may further include:

the second information generation module is used for generating second early warning information when the second early warning value is larger than a second early warning threshold value; the second early warning information is as follows: and the pre-warning information about the violation building in the established state exists in the target building included in the image to be detected.

Optionally, in a specific implementation manner, the detection result may include a confidence level;

therefore, in this specific implementation manner, the second early warning value calculating module may be specifically configured to:

the second early warning value calculating module may be specifically configured to:

The embodiment of the present invention also provides an electronic device, as shown in fig. 9, including a processor 901, a communication interface 902, a memory 903, and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 perform communication with each other through the communication bus 904,

a memory 903 for storing a computer program;

The processor 901 is configured to implement the method for warning a illegal building provided in the embodiment of the present invention when executing the program stored in the memory 903.

Specifically, the method for warning the illegal building comprises the following steps:

acquiring an image to be detected acquired by monitoring equipment;

inputting the image to be detected into a preset illegal detection model to obtain a detection result output by the illegal detection model; the illegal detection model is obtained by training based on training samples and sample labels of the training samples, and the training samples are as follows: an image of the illegal building in a preset construction state and marked with an image area where the illegal building is located exists; sample labels of training samples are: training the construction state of the illegal building existing in the sample;

when the detection result represents that a violation building in a preset construction state exists in an image to be detected, a comparison result of the image to be detected and a preset reference image is obtained, and a first early warning value of the image to be detected is calculated based on the comparison result and the detection result; wherein, the comparison result is: the first position coordinates of the region, in which the difference of the image content determined by comparing the image content of the image to be detected and the reference image meets a preset difference rule; the shooting range of the reference image is the same as that of the image to be detected;

When the first early warning value is larger than a first early warning threshold value, generating first early warning information; the first early warning information is as follows: and the pre-warning information about the illegal building in a preset construction state exists in the target building included in the image to be detected.

It should be noted that, other implementation manners of the method for warning the illegal building, which are implemented by the processor 901 executing the program stored in the memory 903, are the same as the embodiments of the method for warning the illegal building provided in the foregoing method embodiment, and are not described herein again.

The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Corresponding to the method for warning the illegal building provided by the embodiment of the invention, the embodiment of the invention also provides a computer readable storage medium, and the computer program realizes the method for warning the illegal building provided by the embodiment of the invention when being executed by a processor.

acquiring an image to be detected acquired by monitoring equipment;

comparing the image content of the image to be detected with the image content of the reference image to obtain a first position coordinate of an area with the difference of the image content meeting a preset difference rule, and taking the first position coordinate as a comparison result;

It should be noted that, other implementation manners of a violation building early warning method implemented when the computer program is executed by the processor are the same as those provided in the foregoing method embodiment, and are not repeated here.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiments, the electronic device embodiments, the computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the section of the method embodiments for relevance.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for warning a offending building, the method comprising:

acquiring an image to be detected acquired by monitoring equipment;

2. The method according to claim 1, wherein the determining means of the comparison result comprises:

3. The method according to claim 1, wherein the detection result comprises second position coordinates and confidence of an image area where the violation building in a preset construction state exists in the image to be detected;

4. A method according to claim 3, wherein the image to be detected is an image to be detected obtained by shooting the target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to the image to be detected;

5. The method of claim 4, wherein the calculating the region overlap ratio comprises:

x(i)＝Area _i /(Area1 _i +Area2 _i -Area _i )

6. The method of claim 1, wherein the predetermined construction state is: a building state;

the illegal building detection model is specifically trained based on a training sample, an auxiliary sample, a sample label of the training sample and a sample label of the auxiliary sample, wherein the auxiliary sample is an image which is in a built state and marked with an image area where the illegal building is located, and the sample label of the auxiliary sample is in the built state;

the method further comprises the steps of:

7. The method of claim 6, wherein the test result includes a confidence level;

8. The method according to claim 7, wherein the image to be detected is an image to be detected obtained by shooting a target shooting range for the nth time by the monitoring device; the target shooting range is a shooting range corresponding to the image to be detected;

and E is a second early warning value of the image to be detected, y (i) is a confidence coefficient corresponding to an ith image, and the ith image is an image obtained by shooting the target shooting range for the ith time by the monitoring equipment.

9. A violation building early warning device, the device comprising:

10. The device according to claim 9, wherein the detection result comprises second position coordinates and confidence of an image area where the violation building in a preset construction state exists in the image to be detected;

11. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for carrying out the method steps of any one of claims 1-8 when executing a program stored on a memory.

12. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-8.