CN116229299A - Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium - Google Patents
Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium Download PDFInfo
- Publication number
- CN116229299A CN116229299A CN202310246162.2A CN202310246162A CN116229299A CN 116229299 A CN116229299 A CN 116229299A CN 202310246162 A CN202310246162 A CN 202310246162A CN 116229299 A CN116229299 A CN 116229299A
- Authority
- CN
- China
- Prior art keywords
- building
- unmanned aerial
- aerial vehicle
- appearance
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention relates to the technical field of unmanned aerial vehicle inspection, in particular to a building or structure damage assessment method, a control terminal and a storage medium based on an unmanned aerial vehicle, wherein the method comprises the following steps: according to multi-mode inspection and monitoring data of different modes obtained by collecting a target building or structure by an unmanned aerial vehicle, determining an apparent damage result of the target building or structure; estimating physical performance indexes of the target building or structure based on the appearance damage result; and determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index. The method realizes the safety assessment of the building or the structure from the outside to the inside, improves the automation degree and the inspection efficiency of the unmanned aerial vehicle applied to urban inspection, and solves the problems that the hidden danger assessment of the building or the structure by the unmanned aerial vehicle flows on the surface and the reliability is poor.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicles, in particular to a building or structure damage assessment method, a terminal and a medium based on unmanned aerial vehicles.
Background
In the related technical scheme for identifying hidden danger of urban building or structure facilities through unmanned aerial vehicle, unmanned aerial vehicle identifies whether the building or structure and infrastructure have potential safety hazard or not through collecting image data of urban building or structure and urban infrastructure, marks the position and reports the position to the background.
At present, for urban construction projects such as urban buildings or structures and infrastructure, an unmanned aerial vehicle is adopted to conduct safety inspection and monitoring, so that a gradually rising supervision mode is adopted, the urban project is supervised through the unmanned aerial vehicle, the comprehensiveness and effectiveness of supervision can be improved, and the labor cost can be reduced.
However, since the images of the urban buildings or structures or infrastructure are various, the judgment of the existing recognition results is only stopped at the damage (such as the damage of curtain walls and roofs) of the visual surfaces of the buildings or structures or facilities, the judgment and research of the safety performance indexes inside the buildings or structures cannot be performed, the defect of single mode of collecting information exists, the requirement of the urban unmanned aerial vehicle supervision project on the safety detection of the buildings or structures from the outside to the inside is difficult to be met, and the problem of low reliability of hidden danger recognition of the buildings or structures exists.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a building or structure damage assessment method based on an unmanned aerial vehicle, and aims to solve the problem of improving the reliability of identifying hidden danger of a building or structure by the unmanned aerial vehicle.
In order to achieve the above purpose, the present invention provides a building or structure damage assessment method based on an unmanned aerial vehicle, the method comprising:
according to inspection and monitoring data of different modes obtained by collecting a target building or structure by an unmanned aerial vehicle, determining an appearance damage result of the target building or structure;
estimating physical performance indexes of the target building or structure based on the appearance damage result;
and determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index.
Optionally, before the step of estimating the physical performance index of the target building or structure based on the appearance damage result, the method further includes:
acquiring historical inspection and monitoring data obtained by acquiring different buildings or structures at historical moments by the unmanned aerial vehicle;
establishing a multi-modal knowledge graph between the historical inspection and monitoring data and a preset physical performance index template to form a mapping relation between the inspection and monitoring data and the physical performance index;
the step of estimating the physical performance index of the target building or structure based on the appearance damage result comprises the following steps:
and matching the physical performance index corresponding to the appearance damage result based on the multi-modeling knowledge graph.
Optionally, the step of establishing a knowledge graph based on the historical inspection and monitoring data and the historical physical performance index to form a mapping relationship between the inspection and monitoring data and the physical performance index includes:
marking a region to be marked in the historical inspection and monitoring data according to the physical performance index template so as to construct the multi-modal knowledge graph according to the marked historical inspection and monitoring data; or alternatively, the first and second heat exchangers may be,
and marking multi-modal data items associated with the physical performance index templates in the historical inspection and monitoring data so as to construct the multi-modal knowledge graph according to the multi-modal data items.
Optionally, the safety evaluation result includes a safety rate, and the step of determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index includes:
determining a first classification accuracy of the appearance damage result in a preset building or structure appearance damage classification set, and determining a second classification accuracy of the physical performance index in a preset building or structure internal performance classification set;
and determining the safety rate of the target building or structure according to the first classification accuracy rate and the second classification accuracy rate.
Optionally, the safety evaluation result includes a safety evaluation value, and the step of determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index includes:
determining a first predicted value of the appearance damage result in a preset building or structure appearance damage prediction set, and determining a second predicted value of the physical performance index in a preset building or structure internal performance test set;
determining a first prediction difference between the first prediction value and a preset first threshold value, and determining a second prediction difference between the second prediction value and a preset second threshold value;
and determining a safety evaluation value of the target building or structure according to the first prediction difference value and the second prediction difference value.
Optionally, the unmanned aerial vehicle includes vision sensor and infrared sensor, the step of determining the appearance damage result of target building or structure according to the inspection and monitoring data of different modes that unmanned aerial vehicle gathers target building or structure includes:
the infrared sensor acquires the obtained infrared inspection and monitoring data of the target building or structure in the same period, and the vision sensor acquires the image inspection and monitoring data of the target building or structure to perform data fusion to obtain fusion data;
and inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm, and determining the appearance damage result.
Optionally, after the step of inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm and determining the appearance damage result, the method further comprises:
and taking the fusion data as a training sample of the building or structure appearance diagnosis model to realize self-updating of the building or structure appearance diagnosis model.
Optionally, after the step of determining the security assessment result of the target building or structure according to the appearance damage result and the physical performance index, the method further includes:
determining whether the safety evaluation result meets preset building or structure safety evaluation conditions;
if not, outputting the risk prompt information of the building or the structure.
In addition, to achieve the above object, the present invention also provides a control terminal including: the system comprises a memory, a processor and an unmanned aerial vehicle-based building or structure damage assessment program which is stored in the memory and can run on the processor, wherein the unmanned aerial vehicle-based building or structure damage assessment program realizes the steps of the unmanned aerial vehicle-based building or structure damage assessment method when being executed by the processor.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon an unmanned aerial vehicle-based building or structure damage evaluation program which, when executed by a processor, implements the steps of the unmanned aerial vehicle-based building or structure damage evaluation method described above.
The embodiment of the invention provides a building or structure damage assessment method, a control terminal and a storage medium based on an unmanned aerial vehicle. Therefore, the safety assessment of the building or the structure from the outside to the inside is realized, and the reliability of the unmanned aerial vehicle in urban inspection is improved.
Drawings
FIG. 1 is a schematic architecture diagram of a hardware operating environment of a control terminal according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of a method for evaluating damage to a building or structure based on an unmanned aerial vehicle according to the present invention;
FIG. 3 is a flow chart of a second embodiment of the unmanned aerial vehicle-based building or structure damage assessment method of the present invention;
fig. 4 is a flow chart of a third embodiment of the unmanned aerial vehicle-based building or structure damage assessment method according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As an implementation scheme, fig. 1 is a schematic architecture diagram of a hardware running environment of a control terminal according to an embodiment of the present invention.
As shown in fig. 1, the control terminal may include: a processor 1001, such as a CPU, memory 1005, user interface 1003, network interface 1004, communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the architecture of the control terminal shown in fig. 1 is not limiting of the control terminal and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a building or structure damage assessment program based on the unmanned aerial vehicle may be included in the memory 1005 as one type of storage medium. The operating system is a program for managing and controlling hardware and software resources of the control terminal, and is based on the operation of a building or structure damage assessment program and other software or programs of the unmanned aerial vehicle.
In the control terminal shown in fig. 1, the user interface 1003 is mainly used for connecting a terminal, and performs data communication with the terminal; the network interface 1004 is mainly used for a background server and is in data communication with the background server; the processor 1001 may be used to invoke the unmanned aerial vehicle-based building or structure damage-assessment program stored in the memory 1005.
In this embodiment, the control terminal includes: a memory 1005, a processor 1001, and a drone-based building or structure damage assessment program stored on the memory and executable on the processor, wherein:
when the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
acquiring historical inspection and monitoring data obtained by acquiring different buildings or structures at historical moments by the unmanned aerial vehicle;
establishing a multi-modal knowledge graph between the historical inspection and monitoring data and a preset physical performance index template to form a mapping relation between the inspection and monitoring data and the physical performance index;
and matching the physical performance index corresponding to the appearance damage result based on the multi-modeling knowledge graph.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
marking a region to be marked in the historical inspection and monitoring data according to the physical performance index template so as to construct the multi-modal knowledge graph according to the marked historical inspection and monitoring data; or alternatively, the first and second heat exchangers may be,
and marking multi-modal data items associated with the physical performance index templates in the historical inspection and monitoring data so as to construct the multi-modal knowledge graph according to the multi-modal data items.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
determining a first classification accuracy of the appearance damage result in a preset building or structure appearance damage classification set, and determining a second classification accuracy of the physical performance index in a preset building or structure internal performance classification set;
and determining the safety rate of the target building or structure according to the first classification accuracy rate and the second classification accuracy rate.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
determining a first predicted value of the appearance damage result in a preset building or structure appearance damage prediction set, and determining a second predicted value of the physical performance index in a preset building or structure internal performance measurement set;
determining a first prediction difference between the first prediction value and a preset first threshold value, and determining a second prediction difference between the second prediction value and a preset second threshold value;
and determining a safety evaluation value of the target building or structure according to the first prediction difference value and the second prediction difference value.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
the infrared sensor acquires the obtained infrared inspection and monitoring data of the target building or structure in the same period, and the vision sensor acquires the image inspection and monitoring data of the target building or structure to perform data fusion to obtain fusion data;
and inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm, and determining the appearance damage result.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
and taking the fusion data as a training sample of the building or structure appearance diagnosis model to realize self-updating of the building or structure appearance diagnosis model.
When the processor 1001 invokes the unmanned aerial vehicle-based building or structure damage assessment program stored in the memory 1005, the following operations are performed:
determining whether the safety evaluation result meets preset building or structure safety evaluation conditions;
if not, outputting the risk prompt information of the building or the structure.
Based on the hardware architecture of the control terminal based on the unmanned aerial vehicle technology, the embodiment of the unmanned aerial vehicle-based building or structure damage assessment method is provided.
Referring to fig. 2, in a first embodiment, the unmanned aerial vehicle-based building or structure damage assessment method includes the steps of:
step S10, determining an appearance damage result of a target building or structure according to inspection and monitoring data of different modes obtained by collecting the target building or structure by an unmanned aerial vehicle;
in the implementation, firstly, the unmanned aerial vehicle is controlled to carry out inspection and monitoring data acquisition on a target building or structure needing to be subjected to building or structure damage evaluation, and the appearance damage result of the target building or structure is determined according to the inspection and monitoring data.
Optionally, the data modes of the inspection and monitoring data include, but are not limited to, visible light images, video, infrared images, and other data information.
Appearance impairment results are characterized by the degree of difference between the appearance of a building or structure as compared to the appearance data of the building or structure in a healthy state. Apparent damage includes, but is not limited to: curtain wall glass cracking, wall seam cracking, wall paint falling off and the like. The damage diagnosis threshold is constructed by adopting inspection and monitoring data of the building or structure in a healthy state, and whether the building or structure is damaged is judged by comparing the accumulated damage judging factors under the state to be evaluated at different moments with the damage diagnosis threshold.
Step S20, estimating physical performance indexes of the target building or structure based on the appearance damage result;
in this embodiment, the unmanned aerial vehicle obtains the appearance damage result of the target building or structure, and then estimates the physical performance index of the target building or structure based on the appearance damage result. The physical property index is characterized as a safety index of the internal structural hierarchy of the building or structure. Physical performance metrics include, but are not limited to: the safety degree of the building or the structure, the earthquake resistance of the building or the structure, the lightning protection capability of the building or the structure, the water leakage prevention capability of the roof of the building or the structure, the circuit connection condition of the building or the structure and the like.
Illustratively, after the unmanned aerial vehicle collects inspection and monitoring data of the target building or structure, it is determined that the appearance damage result of the target building or structure is: the window is not provided with glass, the wall surface has serious weathering degree, the roof wall seam has serious cracking degree, and the roof is not provided with a lightning rod, so that the physical performance indexes of the building or the structure are estimated as follows: the safety of the building or the structure is low, the lightning protection capability of the building or the structure is poor or no lightning protection capability is provided, and the water leakage prevention capability of the building or the structure is poor.
It should be noted that, in this embodiment, a small sample enhanced deep learning algorithm is adopted, and in combination with the data enhanced CutMix and Copy-Paste method, the model can be trained with enhanced data in a state based on a small sample data source. And data is accumulated in the intelligent inspection process in the use process, so that the model has self-evolution capability, and the self-evolution and the precision improvement of the model are realized by continuously accumulating the self-evolution trained by the model.
And step S30, determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index.
In this embodiment, after the physical performance index of the target building or structure is estimated, the security evaluation result of the target building or structure is determined together according to the obtained appearance damage result and the physical performance index.
Alternatively, the security assessment result may be the security rate of a building or structure, i.e., the security probability of a person moving within the building or structure. And the safety rate is that when the safety rate is larger than a probability threshold value, the building or structure is judged to be qualified in evaluation. In some embodiments, two classification sets are preset, one is a classification set of appearance damage of a building or a structure, and the other is a classification set of internal performance of the building or the structure, wherein the classification set of appearance damage of the building or the structure comprises a plurality of characteristic data representing appearance damage of the building or the structure, and the classification set of internal performance of the building or the structure comprises a plurality of hierarchical data representing internal structure of the building or the structure. And placing the appearance damage result into a building or structure appearance damage classification set, determining the matching similarity of the appearance damage result and each appearance damage characteristic data, determining the characteristics with the matching similarity larger than a similarity threshold value as the characteristics hit with the outward turning damage result, wherein the more the hit characteristics are, the higher the classification accuracy is, which means that the higher the reliability of the result is, and taking the classification accuracy of the appearance damage result in the building or structure appearance damage classification set as a first classification accuracy. And similarly, taking the classification accuracy of the performance classification set of the physical performance indexes of the building or the structure in the building or the structure as a second classification accuracy. Further, according to the first classification accuracy and the second classification accuracy, determining the safety rate of the target building or structure, wherein the greater the first classification accuracy and the second classification accuracy are, the greater the obtained safety rate is, and when the safety rate is greater than a safety rate threshold, judging that the safety evaluation of the target building or structure is qualified.
Alternatively, the safety evaluation result may be a safety evaluation value, where the safety evaluation value is a continuous value, and in some embodiments, two classification sets are preset, one is a prediction set of appearance damage of a building or a structure, and the other is a prediction set of internal performance of the building or the structure, and the prediction set obtains the prediction value of the appearance or the interior of the building or the structure through a function in a linear regression manner. Since the predicted value usually has an error, the predicted value needs to be obtained and then needs to be different from a preset threshold value (also called a true value), and the obtained result is a predicted difference value. The prediction difference value corresponding to the appearance damage result is called a first prediction difference value, and the prediction difference value corresponding to the physical performance index is called a second prediction difference value. And finally, determining the safety evaluation value of the target building or structure according to the first prediction difference value and the second prediction difference value, wherein the smaller the first prediction difference value and the second prediction difference value (namely, the closer the prediction value is to the true value), the larger the safety evaluation value is, which means that the more accurate the safety evaluation result is. And when the safety evaluation value is larger than a preset safety evaluation threshold value, judging that the safety evaluation of the target building or structure is qualified.
Optionally, when the safety evaluation result does not meet the preset building or structure safety evaluation condition, the unmanned aerial vehicle outputs building or structure risk prompt information of the target building or structure with potential risk to the control terminal so as to inform related personnel to carry out corresponding repair treatment on the target building or structure.
In the technical scheme provided by the embodiment, after the appearance damage result is determined according to the inspection and monitoring data of the target building or structure collected by the unmanned aerial vehicle, the physical performance index of the building or structure is estimated based on the appearance damage result, and finally, the safety evaluation result of the target building or structure is determined together according to the appearance damage result and the physical performance index. Therefore, the safety assessment of the building or the structure from the outside to the inside is realized, and the reliability of the unmanned aerial vehicle in urban inspection is improved.
Referring to fig. 3, in the second embodiment, before the step S20, based on the first embodiment, the method further includes:
step S40, acquiring historical inspection and monitoring data obtained by collecting different buildings or structures at historical moment by the unmanned aerial vehicle
Step S50, establishing a multi-modal knowledge graph between the historical inspection and monitoring data and a preset physical performance index template to form a mapping relation between the inspection and monitoring data and the physical performance index;
the step S20 includes:
and step S21, matching the physical performance index corresponding to the appearance damage result based on the knowledge graph.
Optionally, in this embodiment, a mapping relationship between an appearance damage result and a physical performance index of the unmanned aerial vehicle is established through a knowledge graph. In this embodiment, before the unmanned aerial vehicle performs the step of estimating the physical performance index, the unmanned aerial vehicle is controlled in advance to collect different buildings or structures as the historical inspection and monitoring data collected at the historical moment, and then a Multi-modal Knowledge Graph (Multi-Modal Knowledge Graph, MMKG) between the historical inspection and monitoring data and the preset physical performance index template is established, and the Knowledge Graph (knowledgegraph Graph, KG) describes Knowledge resources and carriers thereof through a visualization technology, and mines, analyzes, builds, draws and displays Knowledge and interrelations between the Knowledge resources and the structures. Existing knowledge maps are mostly represented by pure symbols and are represented by text, which weakens the description and understanding ability of a machine to the real world, and multi-modal knowledge maps can give the machine the ability to identify specific entities in an image, so that the machine can generate a more informative entity instead of a fuzzy conceptual description. In the embodiment, the multi-modal knowledge graph is applied to unmanned aerial vehicle building or structure safety evaluation, semantic relation is established between urban building or structure and infrastructure apparent damage image modal information and text and index concepts of building or structure damage evaluation, knowledge graph is established between visual-based urban building or structure and infrastructure apparent damage intelligent algorithm identification result and structural performance index evaluation, and a mapping relation between apparent damage and performance indexes is formed, so that apparent-to-performance relation matching is realized.
For example, in some embodiments, semantic association is established between the apparent damage image mode information of urban buildings and infrastructure and the text and index concepts of building damage evaluation in the existing knowledge and specifications, so that the establishment of a multi-mode knowledge graph is realized, the effect of enriching the dimension of recognition results is achieved, and a reliable knowledge base is provided for potential safety hazard assessment.
Wherein, the step S21 includes:
step S211, marking a region to be marked in the historical inspection and monitoring data according to the physical performance index template, so as to construct the multi-modal knowledge graph according to the marked historical inspection and monitoring data;
alternatively, as a construction method, a multi-modal knowledge graph may be constructed by means of marking an image. In this way, the data mode of the historical inspection and monitoring data is usually image data, and a specific area to be marked in the historical inspection and monitoring image data is marked by taking a physical performance index template as a knowledge symbol, wherein the area to be marked needs to be drawn and marked by a worker. And then constructing a multi-modal knowledge graph through the marked images.
Step S212, marking multi-modal data items associated with the physical performance index template in the historical inspection and monitoring data to construct the multi-modal knowledge graph according to the multi-modal data items;
alternatively, as another construction mode, the multi-modal knowledge graph can be constructed by a symbol positioning mode. The symbol positioning is to take the data from the historical inspection and monitoring data through a physical performance index template, and take the taken data as a multi-mode data item, so as to construct a multi-mode knowledge graph according to the multi-mode data item.
In the technical scheme provided by the embodiment, the multi-modal knowledge graph is established between the appearance of the building or the structure and the interior of the building or the structure, and when the unmanned aerial vehicle collects the inspection and monitoring data of the target building or the structure, the physical performance indexes corresponding to the inspection and monitoring data are matched based on the multi-modal knowledge graph, so that the safety evaluation from the outside to the inside of the building or the structure in the unmanned aerial vehicle inspection process is realized.
Referring to fig. 4, in a third embodiment, based on any one of the embodiments, the step S10 includes:
step S11, carrying out data fusion on the obtained infrared inspection and monitoring data of the target building or structure acquired by the infrared sensor and the image inspection and monitoring data of the target building or structure acquired by the visual sensor in the same period to obtain fusion data;
and step S12, inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm, and determining the appearance damage result.
Optionally, in order to improve accuracy of evaluating the appearance damage of the unmanned aerial vehicle to the building or the structure, fusion data obtained after fusion between the infrared data and the visual data is adopted to determine an appearance damage result of the appearance of the building or the structure. In this embodiment, be equipped with infrared sensor and vision sensor on the unmanned aerial vehicle, unmanned aerial vehicle is every predetermine periodic interval, and infrared inspection and monitoring data and image inspection and monitoring data with same cycle fuses, and the image definition of the image data that the unmanned aerial vehicle gathered is optimized through infrared data, obtains the higher fusion data of precision. And inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm, and determining an appearance damage result of the target building or structure.
The method has the advantages that the sample imbalance proportion can be reduced by adopting a small sample enhanced deep learning algorithm, high-efficiency feature extraction and high-accuracy target identification of the deep learning method under the condition of the small sample are realized, and generalization of the model is improved.
Wherein, after S12, further includes:
and step S60, taking the fusion data as a training sample of the building or structure appearance diagnosis model to realize self-updating of the building or structure appearance diagnosis model.
Optionally, in order to enable the unmanned aerial vehicle to continuously perform self-updating of the deep learning model according to data acquired by the unmanned aerial vehicle during the inspection process, in this embodiment, the fused data fused with the infrared and the image is used as a training sample of the building or structure appearance diagnosis model, so that the building or structure appearance diagnosis model is subjected to model training, self-updating of the building or structure appearance diagnosis model is achieved, the unmanned aerial vehicle continuously accumulates data during the inspection process, self-evolution of the model is achieved, and evaluation accuracy of appearance damage results of the unmanned aerial vehicle during the flight process is continuously improved.
In the technical scheme provided by the embodiment, the diagnosis of the appearance damage result of the unmanned aerial vehicle with higher precision is realized by fusing the infrared data and the image data, and the fused data obtained after fusion is input into the building or structure appearance diagnosis model as a training sample, so that the self-evolution of the unmanned aerial vehicle diagnosis model is realized, the evaluation precision of the appearance damage result of the unmanned aerial vehicle in the flight process is improved, and the reliability of the safety evaluation of the building or structure from the outside to the inside in the unmanned aerial vehicle inspection process is improved.
Furthermore, it will be appreciated by those of ordinary skill in the art that implementing all or part of the processes in the methods of the above embodiments may be accomplished by computer programs to instruct related hardware. The computer program comprises program instructions, and the computer program may be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the control terminal to carry out the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a computer-readable storage medium storing an unmanned aerial vehicle-based building or structure damage evaluation program, which when executed by a processor, implements the steps of the unmanned aerial vehicle-based building or structure damage evaluation method described in the above embodiments.
The computer readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, etc. which may store the program code.
It should be noted that, because the storage medium provided in the embodiments of the present application is a storage medium used to implement the method in the embodiments of the present application, based on the method described in the embodiments of the present application, a person skilled in the art can understand the specific structure and the modification of the storage medium, and therefore, the description thereof is omitted herein. All storage media used in the methods of the embodiments of the present application are within the scope of protection intended in the present application.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A method for evaluating damage to a building or structure based on an unmanned aerial vehicle, the method comprising:
according to inspection and monitoring data of different modes obtained by collecting a target building or structure by an unmanned aerial vehicle, determining an appearance damage result of the target building or structure;
estimating physical performance indexes of the target building or structure based on the appearance damage result;
and determining the safety evaluation result of the target building or structure according to the appearance damage result and the physical performance index.
2. The method of claim 1, wherein prior to the step of estimating the physical performance index of the target building or structure based on the appearance impairment results, further comprising:
acquiring historical inspection and monitoring data obtained by acquiring different buildings or structures at historical moments by the unmanned aerial vehicle;
based on the historical inspection and monitoring data and physical performance indexes of the building or structure, establishing a multi-modal knowledge graph to form a mapping relation between the inspection and monitoring data and the physical performance indexes;
the step of estimating the physical performance index of the target building or structure based on the appearance damage result comprises the following steps:
and matching the physical performance index corresponding to the appearance damage result based on the multi-modeling knowledge graph.
3. The method of claim 2, wherein the step of establishing a multi-modal knowledge-graph based on the historical inspection and monitoring data and physical performance metrics of the building or structure to form a mapping between the inspection and monitoring data and the physical performance metrics comprises:
marking a region to be marked in the historical inspection and monitoring data according to the physical performance index so as to construct the multi-modal knowledge graph according to the marked historical inspection and monitoring data; or alternatively, the first and second heat exchangers may be,
and marking multi-modal data items associated with the physical performance index templates in the historical inspection and monitoring data so as to construct the multi-modal knowledge graph according to the multi-modal data items.
4. The method of claim 1, wherein the security assessment result comprises a security rate, and determining the security assessment result of the target building or structure based on the appearance impairment result and the physical property indicator comprises:
the first classification accuracy of the appearance damage result in a preset building or structure appearance damage classification set and the second classification accuracy of the physical performance index in a preset building or structure internal performance classification set are determined;
and determining the safety rate of the target building or structure according to the first classification accuracy rate and the second classification accuracy rate.
5. The method of claim 1, wherein the security assessment result comprises a security assessment value, and the step of determining the security assessment result of the target building or structure based on the appearance impairment result and the physical performance index comprises:
determining a first predicted value of the appearance damage result in a preset building or structure appearance damage prediction set, and determining a second predicted value of the physical performance index in a preset building or structure internal performance test set;
determining a first prediction difference between the first prediction value and a preset first threshold value, and determining a second prediction difference between the second prediction value and a preset second threshold value;
and determining a safety evaluation value of the target building or structure according to the first prediction difference value and the second prediction difference value.
6. The method of claim 1, wherein the unmanned aerial vehicle comprises a vision sensor and an infrared sensor, and wherein the step of determining the appearance damage result of the target building or structure based on inspection and monitoring data of different modalities obtained by the unmanned aerial vehicle collecting the target building or structure comprises:
the infrared sensor acquires the obtained infrared inspection data of the target building or structure in the same period, and the vision sensor acquires the image inspection data of the target building or structure to perform data fusion to obtain fusion data;
and inputting the fusion data into a building or structure appearance diagnosis model trained based on a small sample enhanced deep learning algorithm, and determining the appearance damage result.
7. The method of claim 6, wherein the step of inputting the fused data into a building or structure visual diagnostic model trained based on a small sample enhanced deep learning algorithm, and determining the visual impairment results further comprises, after the step of:
and taking the fusion data as a training sample of the building or structure appearance diagnosis model to realize self-updating of the building or structure appearance diagnosis model.
8. The method of claim 1, wherein after the step of determining the security assessment result of the target building or structure based on the appearance impairment result and the physical property indicator, further comprising:
determining whether the safety evaluation result meets preset building or structure safety evaluation conditions;
if not, outputting the risk prompt information of the building or the structure.
9. A control terminal, characterized in that the control terminal comprises: a memory, a processor and an unmanned aerial vehicle-based building or structure damage assessment program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the unmanned aerial vehicle-based building or structure damage assessment method according to any one of claims 1 to 8.
10. A computer-readable storage medium, wherein a building or structure damage assessment program based on an unmanned aerial vehicle is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the building or structure damage assessment method based on an unmanned aerial vehicle as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310246162.2A CN116229299A (en) | 2023-03-03 | 2023-03-03 | Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310246162.2A CN116229299A (en) | 2023-03-03 | 2023-03-03 | Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116229299A true CN116229299A (en) | 2023-06-06 |
Family
ID=86589025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310246162.2A Pending CN116229299A (en) | 2023-03-03 | 2023-03-03 | Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116229299A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116541942A (en) * | 2023-07-06 | 2023-08-04 | 河北世元工程建设咨询有限公司 | Judgment method for building design optimization scheme |
| CN116739357A (en) * | 2023-08-16 | 2023-09-12 | 北京科技大学 | Multi-mode fusion perception city existing building wide area monitoring and early warning method and device |
-
2023
- 2023-03-03 CN CN202310246162.2A patent/CN116229299A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116541942A (en) * | 2023-07-06 | 2023-08-04 | 河北世元工程建设咨询有限公司 | Judgment method for building design optimization scheme |
| CN116541942B (en) * | 2023-07-06 | 2023-09-12 | 河北世元工程建设咨询有限公司 | Judgment method for building design optimization scheme |
| CN116739357A (en) * | 2023-08-16 | 2023-09-12 | 北京科技大学 | Multi-mode fusion perception city existing building wide area monitoring and early warning method and device |
| CN116739357B (en) * | 2023-08-16 | 2023-11-17 | 北京科技大学 | Multi-mode fusion perception city existing building wide area monitoring and early warning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fang et al. | A deep learning-based method for detecting non-certified work on construction sites | |
| CN116229299A (en) | Unmanned aerial vehicle-based building or structure damage assessment method, terminal and medium | |
| CN110706509B (en) | Parking space and direction angle detection method, device, equipment and medium thereof | |
| CN110866515B (en) | Method and device for identifying behaviors of objects in factory building and electronic equipment | |
| CN112581463A (en) | Image defect detection method and device, electronic equipment, storage medium and product | |
| CN108596221B (en) | Image recognition method and device for scale reading | |
| CN109739904B (en) | Time sequence marking method, device, equipment and storage medium | |
| CN111126153B (en) | Safety monitoring method, system, server and storage medium based on deep learning | |
| CN113688957A (en) | Target detection method, device, equipment and medium based on multi-model fusion | |
| CN113537070B (en) | Detection method, detection device, electronic equipment and storage medium | |
| EP3913527A1 (en) | Method and device for performing behavior prediction by using explainable self-focused attention | |
| CN109446926A (en) | A kind of traffic monitoring method and device, electronic equipment and storage medium | |
| CN111563530A (en) | Intelligent bridge disease inspection method and system | |
| CN116091477A (en) | Deep learning multi-mode image fusion-based building exterior wall defect detection method | |
| CN115171045A (en) | YOLO-based power grid operation field violation identification method and terminal | |
| Seo et al. | Temporary traffic control device detection for road construction projects using deep learning application | |
| CN112183532A (en) | Safety helmet identification method based on weak supervision collaborative learning algorithm and storage medium | |
| CN109686110A (en) | Parking stall sky expires condition discrimination method and apparatus | |
| CN115392407A (en) | Method, device, equipment and medium for warning danger source based on unsupervised learning | |
| CN115965625A (en) | Instrument detection device based on visual identification and detection method thereof | |
| Chang et al. | Safety risk assessment of electric power operation site based on variable precision rough set | |
| CN113312696B (en) | Bridge health condition dynamic prediction alarm method based on ARIMA algorithm | |
| CN112329601A (en) | Parking space detection method and device based on multi-task network | |
| CN113888599A (en) | Target detection system operation monitoring method based on label statistics and result post-processing | |
| CN114004138A (en) | Building monitoring method and system based on big data artificial intelligence and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |