CN112241659B - Detection device and detection method for illegal building and terminal equipment - Google Patents

Detection device and detection method for illegal building and terminal equipment Download PDF

Info

Publication number
CN112241659B
CN112241659B CN201910646202.6A CN201910646202A CN112241659B CN 112241659 B CN112241659 B CN 112241659B CN 201910646202 A CN201910646202 A CN 201910646202A CN 112241659 B CN112241659 B CN 112241659B
Authority
CN
China
Prior art keywords
building
violation
neural network
remote sensing
suspected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910646202.6A
Other languages
Chinese (zh)
Other versions
CN112241659A (en
Inventor
朱明凌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201910646202.6A priority Critical patent/CN112241659B/en
Publication of CN112241659A publication Critical patent/CN112241659A/en
Application granted granted Critical
Publication of CN112241659B publication Critical patent/CN112241659B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/176Urban or other man-made structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/26Government or public services

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Tourism & Hospitality (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Image Analysis (AREA)
  • Alarm Systems (AREA)

Abstract

The invention provides a device and a method for detecting a violation building and terminal equipment. Based on the invention, remote sensing data acquired by cruising can be used for detecting the illegal building in different areas of the aircraft so as to ensure the detection effectiveness; moreover, the detection of the remote sensing image can adopt a cascade mode of the first neural network and the second neural network to sequentially remove the non-building image and judge the violation of the building graph. Therefore, automatic detection of the illegal building can be realized, so that the detection efficiency is improved. In addition, based on the invention, besides the violation judgment result indicating whether the building graph belongs to the suspected violation building, as an alternative scheme for further optimization, the violation grade and/or visual or coordinated positioning information of the suspected violation building can be obtained, and the alarm information with rich information can be generated.

Description

Detection device and detection method for illegal building and terminal equipment
Technical Field
The invention relates to the field of monitoring automation, in particular to a detection device for a illegal building, a detection method for the illegal building, a terminal device and a detection system for the illegal building.
Background
The offending building is an important concern in urban management, because the offending building can seriously affect urban appearance and social sustainable development of the city.
At present, management of illegal buildings mainly depends on manual inspection or manual supervision by calling monitoring data.
However, the method is limited by insufficient staff compiling and excessive workload of manual supervision of massive videos, and the detection efficiency and the effectiveness of the illegal buildings are not high.
Disclosure of Invention
In view of this, the embodiments of the present invention respectively provide a device for detecting a offending building, a method for detecting a offending building, a terminal device, and a system for detecting a offending building, which can implement automatic detection of a offending building.
In one embodiment, there is provided a detection apparatus for a offending building, comprising:
the remote sensing acquisition module is used for acquiring a first remote sensing image transmitted wirelessly from the aircraft;
the image screening module is used for detecting the content of the acquired first remote sensing image by using a first neural network based on deep learning, and screening a second remote sensing image containing the building graph from the acquired first remote sensing image by using a content detection result which is generated by the first neural network and indicates whether the building graph is contained, wherein first sample characteristics in a first image sample used for training the first neural network are used for representing at least one of the texture, the color and the outline of the building;
The violation judging module is used for:
Inputting the screened second remote sensing image into a second neural network based on deep learning, wherein the second sample features in the second image sample for training the second neural network comprise at least one of contour difference features of the offending building compared with the adjacent building, position relation features of the offending building compared with the adjacent building, texture and/or color difference features of the offending building compared with the adjacent building, and position relation features of the offending building and a road boundary;
obtaining a violation judgment result which is generated by the second neural network and used for indicating whether the building graph in the second remote sensing image belongs to a suspected violation building or not;
Obtaining positioning information and violation grades of suspected violation buildings generated by a second neural network, wherein the positioning information of the suspected violation buildings generated by the second neural network is used for dividing the suspected violation buildings from surrounding environments, the violation grades of the suspected violation buildings generated by the second neural network represent the violation severity of the suspected violation buildings determined in the second remote sensing image, and the second neural network is configured to determine the violation grades by the following steps: the severity of the violation is set to be proportional to the estimated floor area of the suspected violation building determined in the second remote sensing image, and the regional attribute of the suspected violation building is utilized to generate weighted compensation for the proportional influence of the estimated floor area and the severity of the violation, wherein the regional attribute represents the severity level of the violation of the suspected violation building in the region.
Optionally, the positioning information of the suspected offending building generated by the second neural network includes visual positioning information, and/or coordinated positioning information.
Optionally, the violation determination module is further configured to add an area segmentation mark as the visual positioning information in the image area of the suspected violation building in the second remote sensing image according to the image coordinates of the suspected violation building in the second remote sensing image generated by the second neural network.
Optionally, the remote sensing acquisition module is further configured to acquire an area coordinate of a shooting area that is synchronously and wirelessly transmitted from the aircraft and the first remote sensing image; the violation judging module is further used for determining the position coordinates of the suspected violation building as the coordinated positioning information according to the region coordinates of the shooting region of the second remote sensing image and the image coordinates of the suspected violation building in the second remote sensing image generated by the second neural network.
Optionally, the method further comprises: and the back-end interaction module is used for generating a violation alarm with the violation grade and/or positioning information of the suspected illegal building according to the violation judgment result indicating that the building graph belongs to the suspected illegal building.
Optionally, the method further comprises: the data storage module is used for storing the violation judgment result output by the violation judgment module and the violation grade and/or positioning information of the suspected violation building so as to be acquired by the back-end interaction module.
Optionally, the method further comprises: and the image preprocessing module is used for carrying out graying and/or denoising processing on the received first remote sensing image before the content detection by the image screening module.
In another embodiment, a method for detecting a offending building is provided, comprising:
Acquiring a first remote sensing image transmitted wirelessly from an aircraft;
Performing content detection on the acquired first remote sensing image by using a first neural network based on deep learning, wherein first sample characteristics in a first image sample for training the first neural network are used for representing at least one of texture, color and outline of a building;
screening a second remote sensing image containing the building pattern from the acquired first remote sensing image by using a content detection result which is generated by the first neural network and indicates whether the building pattern is contained or not;
Inputting the screened second remote sensing image into a second neural network based on deep learning, wherein the second sample features in the second image sample for training the second neural network comprise at least one of contour difference features of the offending building compared with the adjacent building, position relation features of the offending building compared with the adjacent building, texture and/or color difference features of the offending building compared with the adjacent building, and position relation features of the offending building and a road boundary;
obtaining a violation judgment result which is generated by the second neural network and used for indicating whether the building graph in the second remote sensing image belongs to a suspected violation building or not;
Obtaining positioning information and violation grades of suspected violation buildings generated by a second neural network, wherein the positioning information of the suspected violation buildings generated by the second neural network is used for dividing the suspected violation buildings from surrounding environments, the violation grades of the suspected violation buildings generated by the second neural network represent the violation severity of the suspected violation buildings determined in the second remote sensing image, and the second neural network is configured to determine the violation grades by the following steps: the severity of the violation is set to be proportional to the estimated floor area of the suspected violation building determined in the second remote sensing image, and the regional attribute of the suspected violation building is utilized to generate weighted compensation for the proportional influence of the estimated floor area and the severity of the violation, wherein the regional attribute represents the severity level of the violation of the suspected violation building in the region.
Optionally, the positioning information of the suspected offending building generated by the second neural network includes visual positioning information, and/or coordinated positioning information.
Optionally, obtaining the positioning information of the suspected offending building generated by the second neural network includes: acquiring image coordinates of suspected illegal buildings generated by a second neural network in a second remote sensing image; and adding an area segmentation mark as visual positioning information to the image area containing the suspected illegal building in the second remote sensing image according to the image coordinates of the suspected illegal building in the second remote sensing image.
Optionally, obtaining the positioning information of the suspected offending building generated by the second neural network includes: acquiring region coordinates of a shooting region synchronously and wirelessly transmitted by an aircraft and a first remote sensing image; acquiring image coordinates of suspected illegal buildings generated by a second neural network in a second remote sensing image; and determining the position coordinates of the suspected illegal building as the coordinated positioning information according to the region coordinates of the shooting region of the second remote sensing image and the image coordinates of the suspected illegal building in the second remote sensing image.
In another embodiment, there is provided a terminal device including a processor, a wireless transceiver, and a smart chip, wherein:
the processor is used for executing the steps in the detection method;
The wireless transceiver is used for receiving remote sensing images transmitted wirelessly from the aircraft through a communication connection with the aircraft;
The smart chip is configured to operate the first neural network and the second neural network.
In another embodiment, a system for detecting a offending building is provided, comprising a terminal device as described above, an aircraft establishing a wireless communication connection with the terminal device, a remote control device controlling the aircraft, and a back-end device establishing a wired or wireless communication connection with the terminal device.
In another embodiment, a non-transitory computer readable storage medium is provided that stores instructions that, when executed by a processor, cause the processor to perform the steps in the detection method as described above.
Based on the embodiment, remote sensing data acquired by cruising the aircraft in different areas can be utilized to detect the illegal building so as to ensure the detection effectiveness; moreover, the detection of the remote sensing image can adopt a cascade mode of the first neural network and the second neural network to sequentially remove the non-building image and judge the violation of the building graph. Therefore, automatic detection of the illegal building can be realized, so that the detection efficiency is improved.
In addition, compared with the mode of comparing the characteristics of the remote sensing image with the characteristics of the historical image, the cascade mode of adopting the first neural network and the second neural network can save time and processing resource consumption required by comparing the characteristics of the remote sensing image and save storage resource consumption required by storing the historical image for supporting the characteristics comparison.
In addition, based on the above embodiment, in addition to the rule-breaking determination result indicating whether the building pattern belongs to the suspected rule-breaking building, as an alternative scheme of further optimization, the rule-breaking level and/or visual or coordinated positioning information of the suspected rule-breaking building can be obtained, and thus the alarm information with rich information can be generated.
Drawings
The following drawings are only illustrative of the invention and do not limit the scope of the invention:
FIG. 1 is an exemplary structural schematic diagram of a detection device for a offending building in one embodiment;
FIG. 2 is a schematic diagram of the detecting device shown in FIG. 1;
FIG. 3 is a schematic diagram of an expansion principle of the detecting device shown in FIG. 1 based on diversification of detection results;
FIG. 4 is a schematic diagram of a training process of a neural network used by the detection device shown in FIG. 1;
FIG. 5 is a schematic diagram of a neural network used in the detection apparatus shown in FIG. 1;
FIG. 6 is a schematic diagram of an extended structure of the detection device shown in FIG. 1 based on detection efficiency optimization;
FIG. 7 is a schematic diagram of a frame structure of a offending building detection system including the offending building detection device shown in FIG. 1;
FIG. 8 is a schematic diagram of an expansion structure of the detection device shown in FIG. 1 based on a system docking expansion;
FIG. 9 is an exemplary flow chart of a method of detecting a offending building in another embodiment;
FIG. 10 is a schematic diagram of an optimization flow of the detection method shown in FIG. 9 based on diversification of detection results;
FIG. 11 is a schematic diagram of an optimization flow of the detection method shown in FIG. 10 based on detection efficiency optimization;
FIG. 12 is a schematic diagram of an optimization flow of the detection method based on detection application extension as shown in FIG. 10;
fig. 13 is a schematic structural diagram of a terminal device in another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below by referring to the accompanying drawings and examples.
FIG. 1 is an exemplary structural schematic diagram of a detection device for a offending building in one embodiment. Fig. 2 is a schematic diagram of the detection device shown in fig. 1.
Referring to fig. 1 in combination with fig. 2, in one embodiment, a detection apparatus 100 for a offending building may include a remote sensing acquisition module 110, an image screening module 120, and a offending determination module 130.
The telemetry acquisition module 110 is configured to acquire a telemetry image 210 wirelessly transmitted from an aircraft 200 (e.g., a drone).
The image filtering module 120 is configured to perform content detection on the acquired remote sensing image 210 by using a first neural network 310 based on deep learning, and filter the remote sensing image 220 including the building pattern by using a content detection result generated by the first neural network 310 and representing whether the building pattern is included. That is, the first neural network 310 may be input with the remote sensing image 210 and output with the content detection result indicating whether or not the building pattern is included in the image, which is an uncertainty theorem.
The violation determination module 130 is configured to perform violation determination on the screened remote sensing image 220 by using the second neural network 320 based on deep learning, and obtain a violation determination result 230 generated by the second neural network 320 and indicating whether the building graphic belongs to a suspected violation building. That is, the second neural network 320 may be a violation determination result 230 that takes the remote sensing image 220 as an input and whose output contains an uncertainty theorem indicating whether the building graphic belongs to a suspected offending building.
Based on the above embodiment, the remote sensing data collected by the aircraft 200 during cruising can be used to detect the illegal building in different areas, so as to ensure the validity of detection; further, the detection of the remote sensing image 210 may sequentially perform the elimination of the non-building image and the selective violation determination of the remote sensing image 220 including the building pattern by using the cascade method of the first neural network 310 and the second neural network 320. Therefore, automatic detection of the illegal building can be realized, so that the detection efficiency is improved.
In addition, compared with the feature comparison mode of the remote sensing image and the historical image, the cascade mode of the first neural network 310 and the second neural network 320 can save time and processing resource consumption required by feature comparison and save storage resource consumption required by storing the historical image for supporting feature comparison.
Fig. 3 is a schematic diagram of an expansion principle of the detection device shown in fig. 1 based on diversification of detection results.
Referring to fig. 3, to support diversification of detection results, the violation determination module 130 may be further configured to obtain the violation level 231 of the suspected offending structure and/or the positioning information 232 of the suspected offending structure generated by the second neural network 320.
The violation level 231 may represent a severity of a violation of the suspected offending building, which may be determined based on factors such as an estimated footprint of the suspected offending building, and an area attribute of the suspected offending building.
For example, the size of the estimated footprint of the suspected offending building is proportional to the severity of the violation represented by the violation level 231, i.e., the greater the estimated footprint of the suspected offending building, the more heavily weighted the violation level 231 tends to be, whereas the smaller the estimated footprint of the suspected offending building, the less lightly weighted the violation level 231 tends to be.
For another example, the regional attribute of the suspected offending building may be weighted with the proportional impact of the estimated floor area on the severity of the offending, i.e., the higher the severity level of the offending specification represented by the regional attribute (such as when the regional attribute is set to a accent unit, demonstration park, etc.), the greater the proportional impact of the estimated floor area on the severity of the offending specification, whereas the lower the severity level of the offending specification represented by the regional attribute (such as when the severity level of the offending specification is set to a plain park, etc.), the less the proportional impact of the estimated floor area on the severity of the offending specification.
The positioning information 232 may be information that facilitates the segmentation of the suspected offending building from the surrounding environment, which may be presented as visual positioning information or as coordinated positioning information.
For example, the violation determination module 130 may be further configured to include, as the visual positioning information, an area division mark with an increased image area of the suspected offending building in the remote sensing image according to the image coordinates of the suspected offending building in the remote sensing image generated by the second neural network 320. The region segmentation markers may be closed polygons surrounding the image region of the suspected offending building. The region segmentation markers as visual positioning information can be used for subsequent screenshot extraction of the image region of the suspected offending building.
For another example, the remote sensing acquisition module 110 may be further configured to acquire the region coordinates of the shooting region that is wirelessly transmitted from the aircraft 200 in synchronization with the remote sensing image 210, and the violation determination module 130 may be further configured to determine, as the coordinated positioning information, the position coordinates of the suspected violation building based on the region coordinates of the shooting region of the remote sensing image 220 and the image coordinates of the suspected violation building in the remote sensing image 220 generated by the second neural network 320.
Based on the above-described expansion principle, in addition to the violation determination result 230 indicating whether the building pattern belongs to the suspected offending building, the offending grade 231 and/or the visualized or coordinated positioning information 232 of the suspected offending building may be obtained. Therefore, integrated detection of conclusion detection, grade identification and positioning segmentation can be realized, the accuracy of detection can be facilitated, and the manual processing at the rear end can be further reduced, so that the detection efficiency and timeliness of the illegal building can be further improved.
Fig. 4 is a schematic diagram of a training process of a neural network used by the detection device shown in fig. 1.
Referring to fig. 4, the first neural network 310 and the second neural network 320 used in the above embodiment may be obtained through sample training, where the sample training process includes:
S410: and (5) inputting a sample.
The image sample input during training of the first neural network 310 may include sample features that characterize the texture, color, contour, etc. of the building, so that the first neural network 310 may generate, for the input image, an output of a content detection result that indicates whether the image includes a building graphic or not, which is an uncertainty theorem. The image samples input during the training of the second neural network 320 may further emphasize the regularity feature for identifying whether the violation is performed, for example, the contour difference feature of the violation building compared to the neighboring legal building, the position relationship feature of the violation building compared to the neighboring legal building, the texture and/or color difference feature of the violation building compared to the legal building, the position relationship feature of the violation building and the road boundary, and so on.
S420: the samples are enhanced.
The sample enhancement can be realized through reinforcement learning (Reinforcement Learning) so as to improve the sample generalization capability and support the recognition capability of high-importance features and noise features in the training process under the condition of non-massive samples.
S430: deep network construction. For example, deep network construction may be a process of constructing a multi-layer convolutional neural network (Convolutional Neural Networks, CNN) through parameter training based on deep learning (DEEP LEARNING).
Fig. 5 is a schematic diagram of a neural network used in the detection apparatus shown in fig. 1.
Please refer to fig. 5, taking the neural network based on deep learning as CNN as an example:
s510: the remote sensing image is input through an input layer of CNN:
s520: the multilayer convolution of CNN realizes feature extraction;
S530: the extracted features are perceived by a sensor and then output.
That is, the first neural network 310 may generate the output of the content detection result representing whether the image includes the building graphic or not through the above-described process after inputting the remote sensing image 210; after inputting the remote sensing image 220, the second neural network 320 may generate the violation determination result 230 indicating whether the building graphic belongs to the suspected offending building, and the violation level 231 and/or the visualized or coordinated positioning information 232 of the suspected offending building through the above process.
In practical applications, the remote sensing image is usually an RGB true color image, which is divided into RGB three component images at the input layer of the CNN, and a large number of parameters for normalizing the input image are required at the input layer of the CNN. Therefore, in order to reduce the complexity of the input layer of CNN, it is considered to use a grayed remote sensing image as an input to improve the processing efficiency.
In addition, because the remote sensing image is affected by factors such as ambient illumination change and imaging quality of a camera in the acquisition process, the remote sensing image usually contains noise, blurring and the like. Therefore, denoising processing can be considered to be performed on the remote sensing image in a mode of median filtering, gaussian filtering, wiener filtering and the like, so that the image quality of the remote sensing image is enhanced.
Fig. 6 is a schematic diagram of an extended structure of the detection device shown in fig. 1 based on detection efficiency optimization.
As shown in fig. 6, the detection apparatus 100 for a offending building may further include an image preprocessing module 140 for graying and/or denoising the received remote sensing image 210 before the content detection by the image screening module 120. Accordingly, the image screening module 120 may perform content detection on the remote sensing image 210' after the graying and/or denoising process.
It will be appreciated that in the extended configuration shown in fig. 6, the violation determination module 130 may obtain a violation determination result 230 indicating whether the building pattern belongs to a suspected offending building, and may further obtain a violation level 231 and/or visual or coordinated positioning information 232 of the suspected offending building.
Fig. 7 is a schematic diagram of a frame structure of a offending building detection system including the offending building detection device shown in fig. 1.
Referring to fig. 7, the system for detecting a offending building may include a terminal device 710, and the apparatus 100 for detecting a offending building may be included in the terminal device 710. The terminal device 710 may establish a wireless communication connection with the aircraft 200 to obtain the telemetry image. Also, the offending building detection system as shown in fig. 7 may further include a remote control 720 for controlling the flight of the aircraft 200, and a back-end device 730 for establishing a wired or wireless communication connection with the terminal device 710. The back-end device 730 may include a display device and a monitoring center disposed at the back-end, and the violation determination result 230 acquired by the terminal device 710 and the violation level 231 and/or the visualized or coordinated positioning information 232 of the suspected violation building may be displayed at the back-end device 730.
Based on the violation building detection system shown in fig. 7, the violation judgment basis can be provided for the supervisory personnel through the back-end equipment 730, and the violation condition can be intuitively displayed on the back-end equipment 730, so that the early warning capability and the user interaction experience can be effectively improved.
Fig. 8 is a schematic diagram of an expansion structure of the detection device shown in fig. 1 based on a system docking expansion.
Referring to fig. 8, in order to interface with the back-end device 730 when the detecting apparatus 100 for a offending building is applied to the terminal device 710 shown in fig. 7, the detecting apparatus 100 may further include a back-end interaction module 150 for generating the offending alarm 240 with the offending grade 231 and/or the positioning information 232 of the suspected offending building according to the offending determination result 230 indicating that the building graphic belongs to the suspected offending building.
Also, referring still to fig. 8, in order for the back-end interaction module 150 to obtain the violation determination result 230 used to generate the alarm information 240 and the violation level 231 and/or the positioning information 232 of the suspected violation building, the detection device 100 may further include a data storage module 160 configured to store the violation determination result 230 output by the violation determination module 130 and the violation level 231 and/or the positioning information 232 of the suspected violation building, for the back-end interaction module 150 to obtain.
It will be appreciated that the extended architecture shown in fig. 8 also supports the case where the detection apparatus 100 as shown in fig. 6 may further include an image preprocessing module 140.
Fig. 9 is an exemplary flow chart of a method of detecting a offending building in another embodiment. Referring to fig. 9, in this embodiment, a method for detecting a illegal building includes:
s910: a remote sensing image transmitted wirelessly from the aircraft is acquired.
S920: and detecting the content of the acquired remote sensing image by using a first neural network based on deep learning.
S930: and screening out the remote sensing image containing the building pattern by using the content detection result which is generated by the first neural network and indicates whether the building pattern is contained.
The first neural network used in S920 to S930 may be obtained by training according to the principle shown in fig. 4, and input a remote sensing image according to the principle shown in fig. 5, and output a content detection result indicating whether the image includes a building graphic or not, which is an uncertainty theory.
S940: and performing violation judgment on the screened remote sensing image by using a second neural network based on deep learning.
S950: and obtaining a violation judgment result which is generated by the second neural network and indicates whether the building graph belongs to a suspected violation building.
The second neural network used in S940 to S950 may be obtained by training according to the principle shown in fig. 4, and the filtered remote sensing image is taken as input according to the principle shown in fig. 5, and the output thereof includes a rule-breaking determination result of an uncertainty theorem indicating whether the building pattern belongs to a suspected rule-breaking building.
Fig. 10 is a schematic diagram of an optimization flow of the detection method shown in fig. 9 based on diversification of detection results. Referring to fig. 10, in order to diversify the detection results, the flow shown in fig. 9 may be optimized as follows:
s1010: a remote sensing image transmitted wirelessly from the aircraft is acquired.
S1020: and detecting the content of the acquired remote sensing image by using a first neural network based on deep learning.
The first neural network used in S1020 to S1030 may be the same as the first neural network used in S920 to S930 in fig. 9, and will not be described here again.
S1030: and screening out the remote sensing image containing the building pattern by using the content detection result which is generated by the first neural network and indicates whether the building pattern is contained.
S1040: and performing violation judgment on the screened remote sensing image by using a second neural network based on deep learning.
S1050: and obtaining a violation judgment result which is generated by the second neural network and indicates whether the building graph belongs to the suspected violation building, and obtaining the violation grade and/or positioning information of the suspected violation building generated by the second neural network.
The second neural network used in S1040 to S1050 may be obtained by training according to the principle shown in fig. 4, and uses the filtered remote sensing image as input according to the principle shown in fig. 5, and the generated output includes a violation determination result indicating whether the building graphic belongs to a non-qualitative theory of a suspected violation building, and an additional determination result such as a violation level and/or positioning information of the suspected violation building.
Moreover, for obtaining the positioning information of the suspected offending building generated by the second neural network, two modes, namely, visual positioning information and coordinated positioning information, can be included.
The visual positioning information obtaining method may specifically include: and obtaining the image coordinates of the suspected illegal building in the remote sensing image generated by the second neural network, and according to the image coordinates of the suspected illegal building in the remote sensing image, taking the region segmentation mark added to the image region of the suspected illegal building in the remote sensing image as the visual positioning information.
The method for acquiring the coordinated positioning information may specifically include: the method comprises the steps of acquiring the region coordinates of a shooting region which is synchronously and wirelessly transmitted from an aircraft and a remote sensing image (which can be executed simultaneously with S1010), acquiring the image coordinates of suspected illegal buildings in the remote sensing image generated by a second neural network, and determining the position coordinates of the suspected illegal buildings as coordinated positioning information according to the region coordinates of the shooting region of the remote sensing image and the image coordinates of the suspected illegal buildings in the remote sensing image.
Fig. 11 is a schematic diagram of an optimization flow of the detection method based on detection efficiency optimization as shown in fig. 10. Referring to fig. 11, when the first neural network and the second neural network are CNNs, in order to simplify the input layer complexity of the CNNs, the flow shown in fig. 10 may be further optimized as:
s1110: a remote sensing image transmitted wirelessly from the aircraft is acquired.
S1120: and carrying out graying treatment on the received remote sensing image.
In this step, the received remote sensing image may be further subjected to denoising processing.
S1130: and detecting the content of the grey remote sensing image by using a first neural network based on deep learning.
S1140: and screening out the remote sensing image containing the building pattern by using the content detection result which is generated by the first neural network and indicates whether the building pattern is contained.
The first neural network used in S1130 to S1140 may be CNN trained by the principle shown in fig. 4, and input a remote sensing image according to the principle shown in fig. 5, and output a content detection result indicating whether the image includes a building graphic or not, which is an uncertainty theorem.
S1150: performing violation judgment on the screened remote sensing image by using a second neural network based on deep learning;
s1160: and obtaining a violation judgment result which is generated by the second neural network and indicates whether the building graph belongs to the suspected violation building, and obtaining the violation grade and/or positioning information of the suspected violation building generated by the second neural network.
The second neural network used in S1150-S1160 may be CNN trained according to the principle shown in fig. 4, and the filtered remote sensing image is used as input according to the principle shown in fig. 5, and the generated output may include an uncertainty rule breaking determination result indicating whether the building graphic belongs to a suspected rule breaking building or not, and an additional determination result such as rule breaking level and/or positioning information of the suspected rule breaking building.
Moreover, for obtaining the positioning information of the suspected offending building generated by the second neural network, two modes, namely, visual positioning information and coordinated positioning information, can be included. For a specific procedure in both ways, reference may be made to the additional description of S1050 above, and this is not repeated here.
FIG. 12 is a schematic diagram of an optimization flow of the detection method based on detection application extension as shown in FIG. 10. Referring to fig. 12, to support interactive interfacing with the backend, the flow as shown in fig. 10 may be further optimized to:
S1210: a remote sensing image transmitted wirelessly from the aircraft is acquired.
S1220: and detecting the content of the acquired remote sensing image by using a first neural network based on deep learning.
Before S1220, the received remote sensing image may be subjected to graying processing, that is, the remote sensing image subjected to content detection in this step may be a grayed image.
S1230: and screening out the remote sensing image containing the building pattern by using the content detection result which is generated by the first neural network and indicates whether the building pattern is contained.
The first neural network used in S1220 to S1230 may be the same as the first neural network used in S1020 to S1030 in fig. 10, and will not be described here again.
S1240: performing violation judgment on the screened remote sensing image by using a second neural network based on deep learning;
S1250: and obtaining a violation judgment result which is generated by the second neural network and indicates whether the building graph belongs to the suspected violation building, and obtaining the violation grade and/or positioning information of the suspected violation building generated by the second neural network.
The second neural network used in S1240 to S1250 may be the same as the second neural network used in S1040 to S1050 in fig. 10, and will not be described here. Moreover, for obtaining the positioning information of the suspected offending building generated by the second neural network, two modes, namely, visual positioning information and coordinated positioning information, can be included. For a specific procedure in both ways, reference may be made to the additional description of S1050 above, and this is not repeated here.
S1260: and generating a violation alarm with violation grade and/or positioning information of the suspected violation building according to a violation judgment result indicating that the building graph belongs to the suspected violation building.
Fig. 13 is a schematic structural diagram of a terminal device in another embodiment. Referring to fig. 13, as an improvement to the de-modularity of the terminal device 710 as shown in fig. 7, the terminal device in this embodiment may include a processor 1310, a wireless transceiver 1320, and a smart chip 1330, wherein:
the processor 1310 is configured to perform steps in the detection method as shown in any one of fig. 9 to 12;
the wireless transceiver 1320 is configured to receive a telemetry image wirelessly transmitted from an aircraft over a communication connection with the aircraft;
the smart chip 1330 is used to operate the first neural network and the second neural network.
Also, as can be seen in fig. 13, the terminal device may further comprise a non-transitory computer readable storage medium 1300 storing instructions, some of which may, when executed by the processor 1310, cause the processor 1310 to perform the steps of the detection method as shown in any of fig. 9-12.
In addition, the instructions stored by the non-transitory computer-readable storage medium 1300 may include another portion that, when executed by the smart chip 1330, may cause the smart chip 1330 to operate the first neural network and the second neural network.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the invention.

Claims (13)

1. A detection device for a offending building, comprising:
the remote sensing acquisition module is used for acquiring a first remote sensing image transmitted wirelessly from the aircraft;
the image screening module is used for detecting the content of the acquired first remote sensing image by using a first neural network based on deep learning, and screening a second remote sensing image containing the building graph from the acquired first remote sensing image by using a content detection result which is generated by the first neural network and indicates whether the building graph is contained, wherein first sample characteristics in a first image sample used for training the first neural network are used for representing at least one of the texture, the color and the outline of the building;
The violation judging module is used for:
Inputting the screened second remote sensing image into a second neural network based on deep learning, wherein the second sample features in the second image sample for training the second neural network comprise at least one of contour difference features of the offending building compared with the adjacent building, position relation features of the offending building compared with the adjacent building, texture and/or color difference features of the offending building compared with the adjacent building, and position relation features of the offending building and a road boundary;
obtaining a violation judgment result which is generated by the second neural network and used for indicating whether the building graph in the second remote sensing image belongs to a suspected violation building or not;
Obtaining positioning information and violation grades of suspected violation buildings generated by a second neural network, wherein the positioning information of the suspected violation buildings generated by the second neural network is used for dividing the suspected violation buildings from surrounding environments, the violation grades of the suspected violation buildings generated by the second neural network represent the violation severity of the suspected violation buildings determined in the second remote sensing image, and the second neural network is configured to determine the violation grades by the following steps: the severity of the violation is set to be proportional to the estimated floor area of the suspected violation building determined in the second remote sensing image, and the regional attribute of the suspected violation building is utilized to generate weighted compensation for the proportional influence of the estimated floor area and the severity of the violation, wherein the regional attribute represents the severity level of the violation of the suspected violation building in the region.
2. The detecting device according to claim 1, wherein,
The positioning information of the suspected offending building generated by the second neural network comprises visual positioning information and/or coordinated positioning information.
3. The detecting device according to claim 2, wherein,
The violation judging module is further used for adding an area segmentation mark to the image area containing the suspected violation building in the second remote sensing image as visual positioning information according to the image coordinates of the suspected violation building in the second remote sensing image generated by the second neural network.
4. The detecting device according to claim 2, wherein,
The remote sensing acquisition module is further used for acquiring the region coordinates of a shooting region which is synchronously and wirelessly transmitted from the aircraft and the first remote sensing image;
The violation judging module is further used for determining the position coordinates of the suspected violation building as the coordinated positioning information according to the region coordinates of the shooting region of the second remote sensing image and the image coordinates of the suspected violation building in the second remote sensing image generated by the second neural network.
5. The detection apparatus according to claim 2, characterized by further comprising:
The back-end interaction module is used for generating a violation alarm with violation grade and/or positioning information of the suspected violation building according to a violation judgment result indicating that the building graph in the second remote sensing image belongs to the suspected violation building;
the data storage module is used for storing the violation judgment result output by the violation judgment module and the violation grade and/or positioning information of the suspected violation building so as to be acquired by the back-end interaction module.
6. The detection apparatus according to claim 1, characterized by further comprising:
and the image preprocessing module is used for carrying out graying and/or denoising processing on the received first remote sensing image before the content detection by the image screening module.
7. A method of detecting a offending structure, comprising:
Acquiring a first remote sensing image transmitted wirelessly from an aircraft;
Performing content detection on the acquired first remote sensing image by using a first neural network based on deep learning, wherein first sample characteristics in a first image sample for training the first neural network are used for representing at least one of texture, color and outline of a building;
screening a second remote sensing image containing the building pattern from the acquired first remote sensing image by using a content detection result which is generated by the first neural network and indicates whether the building pattern is contained or not;
Inputting the screened second remote sensing image into a second neural network based on deep learning, wherein the second sample features in the second image sample for training the second neural network comprise at least one of contour difference features of the offending building compared with the adjacent building, position relation features of the offending building compared with the adjacent building, texture and/or color difference features of the offending building compared with the adjacent building, and position relation features of the offending building and a road boundary;
obtaining a violation judgment result which is generated by the second neural network and used for indicating whether the building graph in the second remote sensing image belongs to a suspected violation building or not;
Obtaining positioning information and violation grades of suspected violation buildings generated by a second neural network, wherein the positioning information of the suspected violation buildings generated by the second neural network is used for dividing the suspected violation buildings from surrounding environments, the violation grades of the suspected violation buildings generated by the second neural network represent the violation severity of the suspected violation buildings determined in the second remote sensing image, and the second neural network is configured to determine the violation grades by the following steps: the severity of the violation is set to be proportional to the estimated floor area of the suspected violation building determined in the second remote sensing image, and the regional attribute of the suspected violation building is utilized to generate weighted compensation for the proportional influence of the estimated floor area and the severity of the violation, wherein the regional attribute represents the severity level of the violation of the suspected violation building in the region.
8. The method of detecting according to claim 7, further comprising:
The positioning information of the suspected offending building generated by the second neural network comprises visual positioning information and/or coordinated positioning information.
9. The method of detecting of claim 8, wherein obtaining location information of a suspected offending structure generated by the second neural network comprises:
acquiring image coordinates of suspected illegal buildings generated by a second neural network in a second remote sensing image;
And adding an area segmentation mark as visual positioning information to the image area containing the suspected illegal building in the second remote sensing image according to the image coordinates of the suspected illegal building in the second remote sensing image.
10. The method of detecting of claim 8, wherein obtaining location information of a suspected offending structure generated by the second neural network comprises:
acquiring region coordinates of a shooting region synchronously and wirelessly transmitted by an aircraft and a first remote sensing image;
acquiring image coordinates of suspected illegal buildings generated by a second neural network in a second remote sensing image;
And determining the position coordinates of the suspected illegal building as the coordinated positioning information according to the region coordinates of the shooting region of the second remote sensing image and the image coordinates of the suspected illegal building in the second remote sensing image.
11. A terminal device comprising a processor, a wireless transceiver, and a smart chip, wherein:
The processor is configured to perform the steps in the detection method according to any one of claims 7 to 10;
The wireless transceiver is configured to receive a first telemetry image wirelessly transmitted from an aircraft over a communication connection with the aircraft;
The smart chip is configured to operate the first neural network and the second neural network.
12. A system for detecting a offending building comprising the terminal device of claim 11, an aircraft establishing a wireless communication connection with the terminal device, a remote control device controlling the aircraft, and a back-end device establishing a wired or wireless communication connection with the terminal device.
13. A non-transitory computer readable storage medium storing instructions which, when executed by a processor, cause the processor to perform the steps in the detection method of any of claims 7 to 10.
CN201910646202.6A 2019-07-17 2019-07-17 Detection device and detection method for illegal building and terminal equipment Active CN112241659B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910646202.6A CN112241659B (en) 2019-07-17 2019-07-17 Detection device and detection method for illegal building and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910646202.6A CN112241659B (en) 2019-07-17 2019-07-17 Detection device and detection method for illegal building and terminal equipment

Publications (2)

Publication Number Publication Date
CN112241659A CN112241659A (en) 2021-01-19
CN112241659B true CN112241659B (en) 2024-05-17

Family

ID=74167032

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910646202.6A Active CN112241659B (en) 2019-07-17 2019-07-17 Detection device and detection method for illegal building and terminal equipment

Country Status (1)

Country Link
CN (1) CN112241659B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112990168B (en) * 2021-05-20 2021-08-03 江苏瞭望神州大数据科技有限公司 Illegal land monitoring method and system
CN113920425A (en) * 2021-09-03 2022-01-11 佛山中科云图智能科技有限公司 Target violation point acquisition method and system based on neural network model
CN113822247B (en) * 2021-11-22 2022-02-18 广东泰一高新技术发展有限公司 Method and system for identifying illegal building based on aerial image

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013101428A (en) * 2011-11-07 2013-05-23 Pasuko:Kk Building contour extraction device, building contour extraction method, and building contour extraction program
CN105893972A (en) * 2016-04-08 2016-08-24 深圳市智绘科技有限公司 Automatic illegal building monitoring method based on image and realization system thereof
CN107194396A (en) * 2017-05-08 2017-09-22 武汉大学 Method for early warning is recognized based on the specific architecture against regulations in land resources video monitoring system
CN107871125A (en) * 2017-11-14 2018-04-03 深圳码隆科技有限公司 Architecture against regulations recognition methods, device and electronic equipment
CN107909260A (en) * 2017-11-10 2018-04-13 浙江省地理信息中心 A kind of natural resource assets are left one's post audit commenting method
CN108776772A (en) * 2018-05-02 2018-11-09 北京佳格天地科技有限公司 Across the time building variation detection modeling method of one kind and detection device, method and storage medium
CN109035251A (en) * 2018-06-06 2018-12-18 杭州电子科技大学 One kind being based on the decoded image outline detection method of Analysis On Multi-scale Features
JP2019028657A (en) * 2017-07-28 2019-02-21 株式会社パスコ Learned model for building region extraction
KR20190025162A (en) * 2017-08-31 2019-03-11 서울시립대학교 산학협력단 Methods and system for real-time supervised learning using geo-spatial information
CN109711295A (en) * 2018-12-14 2019-05-03 北京航空航天大学 A kind of remote sensing image offshore Ship Detection
CN109711348A (en) * 2018-12-28 2019-05-03 湖南航天远望科技有限公司 Intelligent monitoring method and system based on the long-term real-time architecture against regulations in hollow panel
CN109753928A (en) * 2019-01-03 2019-05-14 北京百度网讯科技有限公司 The recognition methods of architecture against regulations object and device
CN109871730A (en) * 2017-12-05 2019-06-11 杭州海康威视数字技术股份有限公司 A kind of target identification method, device and monitoring device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9418319B2 (en) * 2014-11-21 2016-08-16 Adobe Systems Incorporated Object detection using cascaded convolutional neural networks

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013101428A (en) * 2011-11-07 2013-05-23 Pasuko:Kk Building contour extraction device, building contour extraction method, and building contour extraction program
CN105893972A (en) * 2016-04-08 2016-08-24 深圳市智绘科技有限公司 Automatic illegal building monitoring method based on image and realization system thereof
CN107194396A (en) * 2017-05-08 2017-09-22 武汉大学 Method for early warning is recognized based on the specific architecture against regulations in land resources video monitoring system
JP2019028657A (en) * 2017-07-28 2019-02-21 株式会社パスコ Learned model for building region extraction
KR20190025162A (en) * 2017-08-31 2019-03-11 서울시립대학교 산학협력단 Methods and system for real-time supervised learning using geo-spatial information
CN107909260A (en) * 2017-11-10 2018-04-13 浙江省地理信息中心 A kind of natural resource assets are left one's post audit commenting method
CN107871125A (en) * 2017-11-14 2018-04-03 深圳码隆科技有限公司 Architecture against regulations recognition methods, device and electronic equipment
CN109871730A (en) * 2017-12-05 2019-06-11 杭州海康威视数字技术股份有限公司 A kind of target identification method, device and monitoring device
CN108776772A (en) * 2018-05-02 2018-11-09 北京佳格天地科技有限公司 Across the time building variation detection modeling method of one kind and detection device, method and storage medium
CN109035251A (en) * 2018-06-06 2018-12-18 杭州电子科技大学 One kind being based on the decoded image outline detection method of Analysis On Multi-scale Features
CN109711295A (en) * 2018-12-14 2019-05-03 北京航空航天大学 A kind of remote sensing image offshore Ship Detection
CN109711348A (en) * 2018-12-28 2019-05-03 湖南航天远望科技有限公司 Intelligent monitoring method and system based on the long-term real-time architecture against regulations in hollow panel
CN109753928A (en) * 2019-01-03 2019-05-14 北京百度网讯科技有限公司 The recognition methods of architecture against regulations object and device

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
An Algorithm to Detect Illegal Buildings Using Color Transferring and Texture Difference;Xiao-Feng Chen et al.;《2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS)》;20190310;第545-550页 *
判定土地违法性质是依据现状还是依据规划;魏姗等;《法制天地》;20170908;第48页 *
土地三级动态巡查对土地违法行为遏制效果评价——以常州市为例;张彩萍等;《南京师大学报(自然科学版)》;20170930;第40卷(第3期);第144-150页 *
城市更新背景下非保护性胡同居住形态研究;王欢等;《城市更新》;20190228;第26卷(第2期);第57-65页 *
基于ArcGIS_Engine的非法土地上报系统;房旭等;《地理空间信息》;20180630;第16卷(第6期);第101-103页 *
基于无人机视觉的道路违法搭建检测;张晨等;《计算机技术与发展》;20180731;第28卷(第7期);第140-143页 *
结合像元级和目标级的高分辨率遥感影像建筑物变化检测;张志强等;《测绘学报》;20180131;第47卷(第1期);第102-112页 *

Also Published As

Publication number Publication date
CN112241659A (en) 2021-01-19

Similar Documents

Publication Publication Date Title
CN112241659B (en) Detection device and detection method for illegal building and terminal equipment
CN111178183B (en) Face detection method and related device
CN110751678A (en) Moving object detection method and device and electronic equipment
CN111898538B (en) Certificate authentication method and device, electronic equipment and storage medium
CN104077776B (en) A kind of visual background extracting method based on color space adaptive updates
CN115861400B (en) Target object detection method, training device and electronic equipment
CN113989683A (en) Ship detection method for synthesizing synchronous orbit sequence optical image space-time information
CN112206541A (en) Game plug-in identification method and device, storage medium and computer equipment
CN112560584A (en) Face detection method and device, storage medium and terminal
CN117576632B (en) Multi-mode AI large model-based power grid monitoring fire early warning system and method
CN112052730A (en) 3D dynamic portrait recognition monitoring device and method
US11836981B2 (en) Method for assisting real-time monitoring of at least one person on sequences of images
CN111862040A (en) Portrait picture quality evaluation method, device, equipment and storage medium
CN111126112B (en) Candidate region determination method and device
CN110909674A (en) Traffic sign identification method, device, equipment and storage medium
CN108563997A (en) It is a kind of establish Face datection model, recognition of face method and apparatus
CN112037255A (en) Target tracking method and device
US9538146B2 (en) Apparatus and method for automatically detecting an event in sensor data
CN109800678A (en) The attribute determining method and device of object in a kind of video
CN113222843B (en) Image restoration method and related equipment thereof
CN108647600A (en) Face identification method, equipment and computer readable storage medium
KR20200005853A (en) Method and System for People Count based on Deep Learning
CN113361444B (en) Image processing method and device, electronic equipment and storage medium
CN114549866A (en) Smoke and fire detection method, device, equipment and medium
CN114445711A (en) Image detection method, image detection device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant