CN111709346A - Historical building identification and detection method based on deep learning and high-resolution images - Google Patents
Historical building identification and detection method based on deep learning and high-resolution images Download PDFInfo
- Publication number
- CN111709346A CN111709346A CN202010523895.2A CN202010523895A CN111709346A CN 111709346 A CN111709346 A CN 111709346A CN 202010523895 A CN202010523895 A CN 202010523895A CN 111709346 A CN111709346 A CN 111709346A
- Authority
- CN
- China
- Prior art keywords
- building
- picture
- boundary frames
- characteristic value
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a historical building identification and detection method, a device, equipment and a storage medium based on deep learning and high-resolution images, belonging to the technical field of object identification, wherein the method comprises the steps of acquiring a remotely acquired building picture, and dividing the building picture into a plurality of unit grids; judging the four boundary frames of the unit grid, and judging whether the four boundary frames are building boundary frames or not; building units in the unit grid are obtained, the boundary frame is removed, a building body is obtained, and an estimated characteristic value of the building body is obtained; judging whether the estimated characteristic value of the building in the picture meets the estimated characteristic value threshold of the building classification in the training set or not based on a pre-trained building classification training set; and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification. The method and the device for identifying the pictures are beneficial to improving the accuracy and robustness in picture identification, and provide convenience for the investigation work of historical buildings.
Description
The application relates to the technical field of object identification, in particular to a method, a device, equipment and a storage medium for identifying and detecting a historical building based on deep learning and high-resolution images.
Background
As one of the representatives of five traditional dwellings in China, the Hakka dragon surrounding house is a famous historical dwelling with more than one hundred years and is also an important symbol of world Hakka culture. In order to protect the historical cultural heritage, the distribution and quantity investigation of the guest house is needed. Traditional field survey not only wastes time and energy, and the cost is higher moreover.
The existing identification method mainly uses the modern high-resolution remote sensing technology to provide important help for the large-area investigation of the dragon surrounding house, but no matter visual interpretation or object-oriented information extraction and analysis, the rapid, automatic and intelligent detection of the dragon surrounding house from a remote sensing image is difficult to realize. Therefore, when the prior art identifies and detects historical buildings, the identification precision and robustness are low, and the problem of difficulty in investigation is solved.
Disclosure of Invention
The embodiment of the application aims to provide a historical building identification and detection method, a historical building identification and detection device, historical building identification and detection equipment and a storage medium based on deep learning and high-resolution images, so as to solve the problem that in the prior art, when historical buildings are identified and detected, the identification accuracy and robustness are low, and the troubleshooting is difficult.
In order to solve the above technical problem, an embodiment of the present application provides a historical building identification and detection method based on deep learning and high-resolution images, which adopts the following technical solutions:
a historical building identification and detection method based on deep learning and high-resolution images comprises the following steps:
obtaining a remotely acquired building picture, and segmenting the building picture into
A grid of cells, N being a positive integer;
judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, and if the sum is not 0, acquiring the unit grid corresponding to the non-0 value;
acquiring all adjacent unit grids with the sum of four boundary frames of the unit grids being not 0, removing the boundary frames to acquire a building body, acquiring a characteristic value of the building body based on a preset characteristic constant, and taking the characteristic value as an estimated characteristic value of the building body;
acquiring a pre-estimated characteristic value, and judging whether the pre-estimated characteristic value of the building in the picture meets a pre-estimated characteristic value threshold value of building classification in a training set based on a pre-trained building class training set;
and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification.
Further, the historical building identification and detection method based on deep learning and high-resolution images includes the steps of:
can acquire picture information through the mode that cell-phone, unmanned aerial vehicle's mode was shot, send the picture to the remote processing end, cut apart the processing by the remote processing end.
Further, the method for identifying and detecting historical buildings based on deep learning and high-resolution images, wherein the step of judging the four bounding boxes of the unit grid and the step of judging whether the four bounding boxes are building bounding boxes comprises the following steps:
based on the Faster-RCNN picture convolution layer and the color RGB format, the color RGB of each object in the picture is obtained, the color RGB of the ground is set as the reference RGB, and if the color RGB of a plurality of boundary frames in the color RGB of the four boundary frames of the unit grid is different from the reference RGB, the picture unit of the building exists in the unit grid.
Further, the historical building identification and detection method based on deep learning and high-resolution images obtains the characteristic value of the building body based on a preset characteristic constant, and the method using the characteristic value as the estimated characteristic value of the building body comprises the following steps:
obtaining the prediction category of each unit grid based on a Yolov2 target detection model
And obtaining the confidence of the cell grids around each cell grid
And based on a predetermined algorithm formula
Calculating the class confidence of the whole building
Namely the characteristic value of the building body, and taking the characteristic value as an estimated characteristic value of the building body.
Further, according to the historical building identification and detection method based on deep learning and high-score images, the building category training set based on pre-training comprises:
acquiring a plurality of different building pictures in advance, and taking the plurality of different building pictures as a training set;
for the training set, based on the YOLOv2 and ResNet-50 models, the category confidence of each building picture is obtained
Building pictures meeting a preset category confidence coefficient threshold are added into the same set, and the set is named based on the academic name information of the building.
Further, the historical building identification and detection method based on deep learning and high-resolution images, wherein the step of judging whether the estimated eigenvalue of the building in the picture meets the estimated eigenvalue threshold of the building classification in the training set comprises the following steps:
and obtaining the estimated characteristic value of the building in the picture, comparing the category confidence degrees of different building classifications in the training set in a polling mode, and judging whether the estimated characteristic value of the building in the picture meets the category confidence degree threshold of the building classification in the training set.
Further, in the historical building identification and detection method based on deep learning and high-resolution images, if yes, category information corresponding to the building classification is obtained, the category information is determined as the category of the building in the picture, and the identification is completed by:
and if so, acquiring a set name corresponding to the building classification, and naming the category of the building in the picture by using the set name.
In order to solve the above technical problem, an embodiment of the present application further provides a historical building identification and detection device based on deep learning and high-resolution images, which adopts the following technical solutions:
a historical building identification and detection device based on deep learning and high-resolution images comprises:
a picture acquisition and preprocessing module for acquiring remotely acquired building pictures and dividing the building pictures into
A grid of cells, N being a positive integer;
the unit grid judging module is used for judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, and if the sum is not 0, acquiring the unit grid corresponding to the non-0 value;
the building body characteristic value estimation module is used for acquiring all adjacent unit grids of which the sum values of the four boundary frames of the unit grids are not 0, removing the boundary frames, acquiring a building body, acquiring the characteristic value of the building body based on a preset characteristic constant, and taking the characteristic value as an estimated characteristic value of the building body;
the building type judging module is used for acquiring the estimated characteristic value and judging whether the estimated characteristic value of the building in the picture meets the estimated characteristic value threshold of the building classification in the training set or not based on a pre-trained building type training set;
and the building type determining module is used for acquiring the type information corresponding to the building classification if the building type information meets the requirement, determining the building type information as the building type in the picture, and finishing the identification.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:
a computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to realize the steps of the historical building identification and detection method based on deep learning and high score images, which is provided by the embodiment of the application.
In order to solve the above technical problem, an embodiment of the present application further provides a nonvolatile computer-readable storage medium, which adopts the following technical solutions:
a non-transitory computer-readable storage medium, on which a computer program is stored, and when executed by a processor, the computer program implements the steps of a deep learning and high score image-based historical building identification and detection method proposed in an embodiment of the present application.
Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:
the embodiment of the application discloses a method, a device, equipment and a storage medium for identifying and detecting historical buildings based on deep learning and high-resolution images, wherein the method comprises the steps of obtaining a remotely acquired building picture, and dividing the building picture into the parts
The unit grids, N is a positive integer, the preprocessing mode of picture segmentation facilitates the post-processing of the picture and also enables the picture processing to be more refined; judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, if the sum of the four boundary frames is not 0, acquiring the unit grid corresponding to the value of the non-0, and judging whether the unit grid has building units or not in a mode of picture color RGB below the frame, wherein the judgment mode avoids unnecessary reduction of detection precision when the whole picture is judged; acquiring all adjacent unit grids with the sum of four boundary frames of the unit grids being not 0, removing the boundary frames to acquire a building body, acquiring the characteristic value of the building body based on a preset characteristic constant,the characteristic value is used as an estimated characteristic value of the building body, the building body is determined by directly obtaining grid units in the picture, and then the estimated characteristic value of the building body is obtained by using a YOLOv2 model, so that the robustness during picture identification is improved; acquiring a pre-estimated characteristic value, and judging whether the pre-estimated characteristic value of the building in the picture meets a pre-estimated characteristic value threshold value of building classification in a training set based on a pre-trained building class training set; and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification. The method and the device for identifying the historical building are beneficial to improving the accuracy and robustness in picture identification, and provide convenience for the troubleshooting work of the historical building.
Drawings
In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a diagram of an exemplary system architecture to which embodiments of the present application may be applied;
FIG. 2 is a flowchart of an embodiment of a method for identifying and detecting historical buildings based on deep learning and high-score images according to the embodiment of the present application;
FIG. 3 is a schematic diagram of building determination in a unit grid according to an embodiment of the present application;
FIG. 4 is a flowchart of a process of the Yolov2 target detection model in the embodiment of the present application;
FIG. 5 is a schematic structural diagram of an embodiment of the deep learning and high-score image-based historical building identification and detection apparatus according to the present disclosure;
FIG. 6 is a schematic structural diagram of a picture acquisition and preprocessing module in an embodiment of the present application;
FIG. 7 is a schematic structural diagram of a building type determination module in an embodiment of the present application;
FIG. 8 is a schematic block diagram of an embodiment of a computer device in an embodiment of the present application;
fig. 9 is a diagram of four detection results of the historical building identification and detection method based on deep learning and high-resolution images in this embodiment.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that the historical building identification and detection method based on deep learning and high-resolution images provided by the embodiment of the present application is generally executed by a server/terminal device, and accordingly, the historical building identification and detection device based on deep learning and high-resolution images is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to fig. 2, a flowchart of an embodiment of the deep learning and high score image based historical building identification and detection method according to the present application is shown, and the deep learning and high score image based historical building identification and detection method includes the following steps:
In this embodiment, the acquiring of the remotely acquired building picture information includes: can acquire picture information through the mode that cell-phone, unmanned aerial vehicle's mode was shot, send the picture to the remote processing end, cut apart the processing by the remote processing end.
In some embodiments of the present application, the determining four bounding boxes of the unit grid to determine whether the four bounding boxes are building bounding boxes includes: based on the Faster-RCNN picture convolution layer and the color RGB format, the color RGB of each object in the picture is obtained, the color RGB of the ground is set as the reference RGB, and if the color RGB of a plurality of boundary frames in the color RGB of the four boundary frames of the unit grid is different from the reference RGB, the picture unit of the building exists in the unit grid.
Wherein, the Faster-RCNN picture convolution layer comprises: performing convolution layer processing on the picture, and acquiring accurate position information of each unit grid in the picture from the picture after convolution processing for multiple times;
the method for acquiring the color RGB of each object in the picture based on the color RGB format comprises the following specific implementation mode: after the fast-RCNN picture convolution layer processing, the positions of the object and the ground in the picture unit grid are determined, the color RGB of the ground is set as reference RGB, for example, the RGB of the ground is (10, 10, 10), the RGB of the building is obviously different from the RGB of the ground, at this time, whether the RGB at the frame of the unit grid is (10, 10, 10) or not is judged, if not, it is determined that part or all of the building exists in the whole unit grid, if the RGB at the four frames is not (10, 10, 10), it is determined that the whole unit grid is the building unit grid, and if the RGB at the four frames is (10, 10, 10), it is determined that the unit grid is the ground unit grid.
Then, the flag indicating RGB at the frame as (10, 10, 10) is set to 0, the flag indicating RGB at the frame as not (10, 10, 10) is set to 1, and since there are 4 frames per cell grid, the frame sum value ranges from 0 to 4.
Referring specifically to fig. 3, fig. 3 is a schematic diagram of building determination in a unit grid in the embodiment of the present application, in which 301 shows a building unit grid, 301a shows a full building unit grid, 301b shows a half building unit grid, 302 shows a non-building unit grid, 303 shows a frame and a value calculation manner, 303a is a frame in a setting format of a building, and 303b is a frame in a setting format of a non-building.
In some embodiments of the present application, the obtaining of all adjacent unit grids with a sum of four bounding boxes of the unit grid being not 0 and removing the bounding boxes includes: firstly, judging whether the sum of the set values of the four frames of each unit grid is 0, if not, judging whether the unit grids exist buildings, and then judging whether the unit grids which are not 0 are adjacent, if so, indicating that the unit grids are an integral building, otherwise, indicating that the unit grids are a plurality of or non-adjacent buildings. And acquiring all adjacent unit grids, removing the frame and acquiring an integral building body.
In some embodiments of the present application, the obtaining a characteristic value of the building body based on a preset characteristic constant, and using the characteristic value as an estimated characteristic value of the building body includes: based on a Yolov2 target detection model, obtaining the prediction category of each unit grid, and obtaining the confidence of the unit grids around each unit grid
And based on a predetermined algorithm formula
Calculating the class confidence of the whole building
Namely the characteristic value of the building body, and taking the characteristic value as an estimated characteristic value of the building body.
Wherein, the following steps: obtaining the prediction category of each unit grid based on a Yolov2 target detection model
The method comprises the following specific steps: firstly, dividing an obtained building body into a plurality of unit modules based on a preset dividing method of a YOLOv2 target detection model, judging whether each unit module is fully occupied by the building body, if so, obtaining surrounding unit modules of the unit modules, judging whether the unit modules are fully occupied by the building body, judging which category the building body belongs to based on the whole shape until the outermost unit module which is not fully occupied is obtained, and giving a category predicted value
。
Obtaining the confidence of the surrounding unit grids of each unit grid
And based on a predetermined algorithm formula
Calculating the class confidence of the whole building
The specific mode is as follows: if all the surrounding modules of each unit module are fully occupied by the building body, the confidence coefficient is set to be 1, if the surrounding modules of the unit module are not fully occupied by the building body, the non-building body area in the unit module which is not fully occupied by the building body is calculated, the percentage is calculated, the confidence coefficient is obtained, namely the value range of the confidence coefficient is 0To 1.
Finally, using the formula
Obtaining class confidence of whole building
。
Specifically referring to fig. 4, fig. 4 is a flowchart of a processing flow of the YOLOv2 target detection model in the embodiment of the present application, and the specific steps are as follows: dividing the obtained building body into a plurality of unit modules based on a preset division method of a YOLOv2 target detection model; judging whether each unit module is fully occupied by the building body, if so, acquiring peripheral unit modules of the unit modules, and judging whether the unit modules are fully occupied by the building body until the outermost unit module is acquired; based on the whole shape, judging which category the building body belongs to, and giving a category predicted value
(ii) a If all the surrounding modules of each unit module are occupied by the building body, setting the confidence coefficient to be 1; if the peripheral modules of the unit modules are not occupied by the building body, calculating the area of the non-building body in the unit modules not occupied by the building body, calculating the percentage and obtaining the confidence coefficient of each module; using the formula
Obtaining class confidence of whole building
。
And 204, acquiring the estimated characteristic value, and judging whether the estimated characteristic value of the building in the picture meets the estimated characteristic value threshold of the building classification in the training set based on the pre-trained building classification training set.
In some embodiments of the present application, the pre-training based building category training set comprises: pre-acquiring a plurality of different sheetsThe plurality of different building pictures are used as a training set; for the training set, based on the YOLOv2 and ResNet-50 models, the category confidence of each building picture is obtained
Building pictures meeting a preset category confidence coefficient threshold are added into the same set, and the set is named based on the academic name information of the building.
Wherein, the training set is partially based on a YOLOv2 model and a ResNet-50 model, and the category confidence of each building picture is obtained
The ResNet-50 model is used in the training set to obtain the class confidence of the picture, and the ResNet-50 model is used in a 3 x 3 convolution layer mode to perform residual module training, so that the accuracy of the class confidence of the picture is effectively improved.
The building pictures meeting the preset category confidence coefficient threshold are added into the same set, and the set is named based on the academic name information of the building, and the specific implementation mode is as follows: if the buildings in the pictures do not have the same kind of buildings, the class confidence coefficients are often the same or similar, for example, the picture A and the picture B belong to the Hakka Bingchang, after the class confidence coefficient calculation, the class confidence coefficient of the picture A is 0.98, and the class confidence coefficient of the picture B is 0.97; at this time, whether the category confidence of the picture A and the picture B exceeds 0.90 is judged based on a preset category confidence threshold value of 0.90, if so, the picture A and the picture B are added into the same set, and the name of the set is 'guest house dragon surrounding house' or 'kejiaweilingwu'.
In some embodiments of the present application, the determining whether the estimated eigenvalue of the building in the picture meets the estimated eigenvalue threshold of the building classification in the training set includes: and obtaining the estimated characteristic value of the building in the picture, comparing the category confidence degrees of different building classifications in the training set in a polling mode, and judging whether the estimated characteristic value of the building in the picture meets the category confidence degree threshold of the building classification in the training set.
The method comprises the following steps of obtaining estimated characteristic values of buildings in pictures, and comparing class confidence coefficients of different building classifications in a training set in a polling mode, wherein the specific steps are as follows: firstly, acquiring an estimated characteristic value of a building in a picture, and comparing class confidence thresholds of different sets in a training set, wherein for example, the estimated characteristic value of the building in the picture, namely the Hakka round dragon house, is 0.80, and the class confidence threshold of the Hakka round dragon house in the training set is 0.90, the building in the picture is not the Hakka round dragon house; at this time, if the estimated characteristic value of the quadrangle in the picture is 0.96 and the category confidence threshold of the quadrangle in the training set is 0.90, the building in the picture is the quadrangle.
And step 205, if yes, obtaining category information corresponding to the building categories, determining the category information as the category of the building in the picture, and completing identification.
In some embodiments of the application, if the category information corresponding to the building classification is satisfied, the category information is obtained and determined as a category of a building in a picture, and completing the identification includes: and if so, acquiring a set name corresponding to the building classification, and naming the category of the building in the picture by using the set name.
Explanation: and acquiring a set name of a set to which the picture belongs, namely the name of the building in the picture.
According to the historical building identification and detection method based on deep learning and high-resolution images, the building picture can be segmented into the building pictures through obtaining the remotely collected building pictures
The unit grids, N is a positive integer, the preprocessing mode of picture segmentation facilitates the post-processing of the picture and also enables the picture processing to be more refined; judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0 or not, if the sum is not 0, acquiring a list corresponding to the value not 0The element grid judges whether the unit grid has a building unit or not in a mode of picture color RGB below the frame, and the judgment mode avoids unnecessary reduction of detection precision when the whole picture is judged; acquiring all adjacent unit grids with the sum of four boundary frames of the unit grids being not 0, removing the boundary frames to acquire a building body, acquiring a characteristic value of the building body based on a preset characteristic constant, taking the characteristic value as an estimated characteristic value of the building body, determining the building body by directly acquiring grid units in a picture, and acquiring the estimated characteristic value of the building body by using a YOLOv2 model, so that the robustness during picture identification is improved; acquiring a pre-estimated characteristic value, and judging whether the pre-estimated characteristic value of the building in the picture meets a pre-estimated characteristic value threshold value of building classification in a training set based on a pre-trained building class training set; and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification. The method and the device for identifying the historical building are beneficial to improving the accuracy and robustness in picture identification, and provide convenience for the troubleshooting work of the historical building.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
With further reference to fig. 5, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a historical building identification and detection apparatus based on deep learning and high-resolution images, which corresponds to the embodiment of the method shown in fig. 2, and which can be applied to various electronic devices.
As shown in fig. 5, the historical building identification and detection device 5 based on deep learning and high-score images according to the present embodiment includes: the system comprises a picture acquisition and preprocessing module 501, a unit grid judgment module 502, a building body characteristic value estimation module 503, a building type judgment module 504 and a building type determination module 505. Wherein:
a picture acquisition and preprocessing module 501 for acquiring a remotely acquired building picture and dividing the building picture into
A grid of cells, N being a positive integer;
a unit grid judging module 502, configured to judge four bounding boxes of a unit grid, and judge whether the four bounding boxes are building bounding boxes, if the four bounding boxes are non-building bounding boxes, set to 0, otherwise, if the four bounding boxes are building bounding boxes, set to 1, judge whether a sum value of the four bounding boxes of the unit grid is not 0, and if the sum value is not 0, obtain a unit grid corresponding to the non-0 value;
the building body characteristic value estimation module 503 is configured to obtain all adjacent unit grids of which the sum of four boundary frames of the unit grid is not 0, remove the boundary frames, obtain a building body, obtain a characteristic value of the building body based on a preset characteristic constant, and use the characteristic value as an estimated characteristic value of the building body;
a building category judgment module 504, configured to obtain the estimated feature value, and judge whether the estimated feature value of the building in the picture satisfies an estimated feature value threshold of the building category in the training set based on a pre-trained building category training set;
and the building type determining module 505 is configured to, if the building type information is satisfied, obtain the type information corresponding to the building classification, determine that the building type is the type of the building in the picture, and complete the identification.
In some embodiments of the present application, as shown in fig. 6, fig. 6 is a schematic structural diagram of a picture collecting and preprocessing module in an embodiment of the present application, where the picture collecting and preprocessing module 501 includes a picture collecting unit 501a and a picture preprocessing unit 501 b.
In some embodiments of the present application, the picture collecting unit 501a is configured to obtain picture information in a mode of shooting by using a mobile phone or an unmanned aerial vehicle, and send the picture to a remote processing terminal.
In some embodiments of the present application, the synonym expanding and writing unit 501b is configured to perform a segmentation process by a remote processing end, and segment the synonym expanding and writing unit into segments
A grid of cells, N being a positive integer.
In some embodiments of the present application, the unit grid determining module 502 is configured to obtain color RGB of each object in the picture based on the fast-RCNN picture convolution layer and the color RGB format, set the color RGB of the ground as reference RGB, and determine that there is a picture unit of the building in the unit grid if the color RGB of the four bounding boxes of the unit grid has a plurality of color RGB of the bounding boxes different from the reference RGB.
In some embodiments of the present application, the building body feature value estimation module 503 is configured to obtain a prediction category of each unit grid based on the YOLOv2 target detection model
And obtaining the confidence of the cell grids around each cell grid
And based on a predetermined algorithm formula
Calculating the class confidence of the whole building
Namely the characteristic value of the building body, and taking the characteristic value as an estimated characteristic value of the building body.
In some embodiments of the present application, as shown in fig. 7, fig. 7 is a schematic structural diagram of a building type determination module in the embodiments of the present application, where the building type determination module 504 includes a model training unit 504a and an input picture identification unit 504 b.
In some embodiments of the present application, the model training unit 504a is configured to obtain a plurality of different building pictures in advance, and use the plurality of different building pictures as a training set; for the training set, based on the YOLOv2 and ResNet-50 models, the category confidence of each building picture is obtained
Building pictures meeting a preset category confidence coefficient threshold are added into the same set, and the set is named based on the academic name information of the building.
In some embodiments of the present application, the input picture identifying unit 504b is configured to obtain an estimated feature value of a building in a picture, perform category confidence comparison on different building classifications in a training set in a polling manner, and determine whether the estimated feature value of the building in the picture meets a category confidence threshold of the building classifications in the training set.
In some embodiments of the present application, the building category determining module 505 acquires a set name corresponding to the building classification when the estimated feature value of the building in the picture meets a category confidence threshold of the building classification in the training set, names the category of the building in the picture with the set name, and completes the identification.
The historical building identification and detection device based on deep learning and high-resolution images, which is provided by the embodiment of the application, is constructed by acquiring remotely acquired building picturesDividing the architectural picture into
The unit grids, N is a positive integer, the preprocessing mode of picture segmentation facilitates the post-processing of the picture and also enables the picture processing to be more refined; judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, if the sum of the four boundary frames is not 0, acquiring the unit grid corresponding to the value of the non-0, and judging whether the unit grid has building units or not in a mode of picture color RGB below the frame, wherein the judgment mode avoids unnecessary reduction of detection precision when the whole picture is judged; acquiring all adjacent unit grids with the sum of four boundary frames of the unit grids being not 0, removing the boundary frames to acquire a building body, acquiring a characteristic value of the building body based on a preset characteristic constant, taking the characteristic value as an estimated characteristic value of the building body, determining the building body by directly acquiring grid units in a picture, and acquiring the estimated characteristic value of the building body by using a YOLOv2 model, so that the robustness during picture identification is improved; acquiring a pre-estimated characteristic value, and judging whether the pre-estimated characteristic value of the building in the picture meets a pre-estimated characteristic value threshold value of building classification in a training set based on a pre-trained building class training set; and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification. The method and the device for identifying the historical building are beneficial to improving the accuracy and robustness in picture identification, and provide convenience for the troubleshooting work of the historical building.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 8, fig. 8 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 8 comprises a memory 8a, a processor 8b, a network interface 8c communicatively connected to each other via a system bus. It is noted that only a computer device 8 having components 8a-8c is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 8a includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 8a may be an internal storage unit of the computer device 8, such as a hard disk or a memory of the computer device 8. In other embodiments, the memory 8a may also be an external storage device of the computer device 8, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a flash Card (FlashCard), and the like, which are provided on the computer device 8. Of course, the memory 8a may also comprise both an internal storage unit of the computer device 8 and an external storage device thereof. In this embodiment, the memory 8a is generally used for storing an operating system and various application software installed on the computer device 8, such as program codes of historical building identification and detection methods based on deep learning and high-resolution images. In addition, the memory 8a may also be used to temporarily store various types of data that have been output or are to be output.
The processor 8b may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data Processing chip in some embodiments. The processor 8b is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is configured to run a program code stored in the memory 8a or process data, for example, a program code of the historical building identification and detection method based on deep learning and high-resolution images.
The network interface 8c may comprise a wireless network interface or a wired network interface, and the network interface 8c is typically used to establish a communication connection between the computer device 8 and other electronic devices.
The present application further provides another embodiment, which is to provide a non-volatile computer readable storage medium storing a deep learning and high score image based historical building identification and detection method program, which is executable by at least one processor to cause the at least one processor to perform the steps of the deep learning and high score image based historical building identification and detection method as described above.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.
In addition, researches show that when the historical building identification and detection method based on deep learning and high-resolution images is used for detection, the minimum batch Loss rate Loss is reduced to 0.0250, RMSE is reduced to 0.1580, the detection average precision AP reaches 0.9599 +/-0.0150, the average detection time of each slice image is 0.0383 +/-0.0150 seconds, and real-time automatic intelligent detection and accuracy providing can be achieved. The embodiment of the present application further provides four detection result graphs, referring to fig. 9 specifically, fig. 9 is four detection result graphs of the historical building identification and detection method based on deep learning and high-resolution images in this embodiment.
Claims (10)
1. A historical building identification and detection method based on deep learning and high-resolution images is characterized by comprising the following steps:
obtaining a remotely acquired building picture, and segmenting the building picture into
A grid of cells, N being a positive integer;
judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, and if the sum is not 0, acquiring the unit grid corresponding to the non-0 value;
acquiring all adjacent unit grids with the sum of four boundary frames of the unit grids being not 0, removing the boundary frames to acquire a building body, acquiring a characteristic value of the building body based on a preset characteristic constant, and taking the characteristic value as an estimated characteristic value of the building body;
acquiring a pre-estimated characteristic value, and judging whether the pre-estimated characteristic value of the building in the picture meets a pre-estimated characteristic value threshold value of building classification in a training set based on a pre-trained building class training set;
and if so, acquiring category information corresponding to the building categories, determining the category information as the category of the building in the picture, and finishing identification.
2. The method for identifying and detecting historical buildings based on deep learning and high-resolution images as claimed in claim 1, wherein the obtaining of the remotely acquired building picture information comprises:
can acquire picture information through the mode that cell-phone, unmanned aerial vehicle's mode was shot, send the picture to the remote processing end, cut apart the processing by the remote processing end.
3. The method as claimed in claim 2, wherein the determining four bounding boxes of the unit grid, and the determining whether the four bounding boxes are building bounding boxes includes:
based on the YOLOv2 and ResNet-50 picture convolution layer and color RGB format, acquiring the color RGB of each object in the picture, setting the color RGB of the ground as reference RGB, and if the color RGB of a plurality of bounding boxes in the color RGB of the four bounding boxes of the unit grid is different from the reference RGB, judging that the picture unit of the building exists in the unit grid.
4. The method for identifying and detecting historical buildings based on deep learning and high-resolution images as claimed in claim 3, wherein the step of obtaining the eigenvalue of the building body based on a preset characteristic constant and using the eigenvalue as the estimated eigenvalue of the building body comprises the steps of:
obtaining the prediction category of each unit grid based on a Yolov2 target detection model
And obtaining the confidence of the cell grids around each cell grid
And based on a predetermined algorithm formula
Calculating the class confidence of the whole building
Namely the characteristic value of the building body, and taking the characteristic value as an estimated characteristic value of the building body.
5. The deep learning and high-score image based historical building identification and detection method according to claim 4, wherein the pre-training based building category training set comprises:
acquiring a plurality of different building pictures in advance, and taking the plurality of different building pictures as a training set;
and acquiring the class confidence of each building picture for the training set based on a YOLOv2 and ResNet-50 model, adding the building pictures meeting a preset class confidence threshold into the same set, and naming the set based on the academic name information of the building.
6. The method for identifying and detecting historical buildings based on deep learning and high-resolution images as claimed in claim 5, wherein the step of judging whether the estimated eigenvalue of the buildings in the pictures meets the estimated eigenvalue threshold of the building classification in the training set comprises the steps of:
and obtaining the estimated characteristic value of the building in the picture, comparing the category confidence degrees of different building classifications in the training set in a polling mode, and judging whether the estimated characteristic value of the building in the picture meets the category confidence degree threshold of the building classification in the training set.
7. The method for identifying and detecting historical buildings based on deep learning and high-resolution images as claimed in claim 6, wherein if the historical buildings are satisfied, the category information corresponding to the building classification is obtained and determined as the category of the buildings in the picture, and the identification is completed by:
and if so, acquiring a set name corresponding to the building classification, and naming the category of the building in the picture by using the set name.
8. A historical building discernment and detection device based on degree of depth study and high score image, its characterized in that includes:
a picture acquisition and preprocessing module for acquiring remotely acquired building pictures and dividing the building pictures into
A grid of cells, N being a positive integer;
the unit grid judging module is used for judging the four boundary frames of the unit grid, judging whether the four boundary frames are building boundary frames or not, if the four boundary frames are non-building boundary frames, setting the four boundary frames to be 0, otherwise, if the four boundary frames are building boundary frames, setting the four boundary frames to be 1, judging whether the sum of the four boundary frames of the unit grid is not 0, and if the sum is not 0, acquiring the unit grid corresponding to the non-0 value;
the building body characteristic value estimation module is used for acquiring all adjacent unit grids of which the sum values of the four boundary frames of the unit grids are not 0, removing the boundary frames, acquiring a building body, acquiring the characteristic value of the building body based on a preset characteristic constant, and taking the characteristic value as an estimated characteristic value of the building body;
the building type judging module is used for acquiring the estimated characteristic value and judging whether the estimated characteristic value of the building in the picture meets the estimated characteristic value threshold of the building classification in the training set or not based on a pre-trained building type training set;
and the building type determining module is used for acquiring the type information corresponding to the building classification if the building type information meets the requirement, determining the building type information as the building type in the picture, and finishing the identification.
9. A computer device comprising a memory having stored therein a computer program and a processor which when executed implements the steps of the deep learning and high score imagery based historical building identification and detection method of any one of claims 1 to 7.
10. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the deep learning and high score image based historical building identification and detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010523895.2A CN111709346A (en) | 2020-06-10 | 2020-06-10 | Historical building identification and detection method based on deep learning and high-resolution images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010523895.2A CN111709346A (en) | 2020-06-10 | 2020-06-10 | Historical building identification and detection method based on deep learning and high-resolution images |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111709346A true CN111709346A (en) | 2020-09-25 |
Family
ID=72539925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010523895.2A Pending CN111709346A (en) | 2020-06-10 | 2020-06-10 | Historical building identification and detection method based on deep learning and high-resolution images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111709346A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114331031A (en) * | 2021-12-08 | 2022-04-12 | 北京华清安地建筑设计有限公司 | Building traditional feature recognition and evaluation method and system |
| WO2024125141A1 (en) * | 2022-12-12 | 2024-06-20 | 华南理工大学 | Deep neural network-based hakka walled village building geographic space positioning method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109446992A (en) * | 2018-10-30 | 2019-03-08 | 苏州中科天启遥感科技有限公司 | Remote sensing image building extracting method and system, storage medium, electronic equipment based on deep learning |
| US20190258895A1 (en) * | 2018-02-20 | 2019-08-22 | Microsoft Technology Licensing, Llc | Object detection from image content |
| CN111178206A (en) * | 2019-12-20 | 2020-05-19 | 山东大学 | Building embedded part detection method and system based on improved YOLO |
-
2020
- 2020-06-10 CN CN202010523895.2A patent/CN111709346A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190258895A1 (en) * | 2018-02-20 | 2019-08-22 | Microsoft Technology Licensing, Llc | Object detection from image content |
| CN109446992A (en) * | 2018-10-30 | 2019-03-08 | 苏州中科天启遥感科技有限公司 | Remote sensing image building extracting method and system, storage medium, electronic equipment based on deep learning |
| CN111178206A (en) * | 2019-12-20 | 2020-05-19 | 山东大学 | Building embedded part detection method and system based on improved YOLO |
Non-Patent Citations (3)
| Title |
|---|
| 周立旺;潘天翔;杨泽曦;王斌;: "多阶段优化的小目标聚焦检测" * |
| 周自强;陈强;马必焕;齐冬莲;: "一种改进的YOLO目标检测方法在电缆设备异常状态识别中的应用" * |
| 赵永强;饶元;董世鹏;张君毅;: "深度学习目标检测方法综述" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114331031A (en) * | 2021-12-08 | 2022-04-12 | 北京华清安地建筑设计有限公司 | Building traditional feature recognition and evaluation method and system |
| CN114331031B (en) * | 2021-12-08 | 2022-12-09 | 北京华清安地建筑设计有限公司 | Building traditional feature recognition and evaluation method and system |
| WO2024125141A1 (en) * | 2022-12-12 | 2024-06-20 | 华南理工大学 | Deep neural network-based hakka walled village building geographic space positioning method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020147410A1 (en) | Pedestrian detection method and system, computer device, and computer readable storage medium | |
| CN113177133B (en) | Image retrieval method, device, equipment and storage medium | |
| CN113537070B (en) | Detection method, detection device, electronic equipment and storage medium | |
| CN112528029A (en) | Text classification model processing method and device, computer equipment and storage medium | |
| CN110503643B (en) | Target detection method and device based on multi-scale rapid scene retrieval | |
| CN112989995B (en) | Text detection method and device and electronic equipment | |
| CN111709346A (en) | Historical building identification and detection method based on deep learning and high-resolution images | |
| CN115941322A (en) | Attack detection method, device, equipment and storage medium based on artificial intelligence | |
| CN115359308A (en) | Model training method, apparatus, device, storage medium, and program for identifying difficult cases | |
| CN114581732A (en) | Image processing and model training method, device, equipment and storage medium | |
| CN111709851B (en) | Hotel safety check-in method, device and equipment based on RFID and facial recognition | |
| CN113899367A (en) | Positioning method and device for unmanned aerial vehicle landing, computer equipment and storage medium | |
| CN110766938B (en) | Road network topological structure construction method and device, computer equipment and storage medium | |
| Zhang et al. | Automatic identification of building structure types using unmanned aerial vehicle oblique images and deep learning considering facade prior knowledge | |
| CN115482436B (en) | Training method and device for image screening model and image screening method | |
| CN114241411B (en) | Counting model processing method and device based on target detection and computer equipment | |
| CN110704650A (en) | OTA picture tag identification method, electronic device and medium | |
| CN115700845A (en) | Face recognition model training method, face recognition device and related equipment | |
| AU2021101857A4 (en) | Historical building identification and detection method based on deep learning and high-resolution images | |
| CN114936395A (en) | Household type graph recognition method and device, computer equipment and storage medium | |
| CN112395450A (en) | Picture character detection method and device, computer equipment and storage medium | |
| CN113032071A (en) | Page element positioning method, page testing method, device, equipment and medium | |
| CN115294536B (en) | Violation detection method, device, equipment and storage medium based on artificial intelligence | |
| CN117746069B (en) | Graph searching model training method and graph searching method | |
| CN112052773B (en) | Unmanned aerial vehicle traffic dispersion intelligent broadcasting method and device based on image sensing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |