KR102203135B1 - Method and system for detecting disaster damage information based on artificial intelligence using drone - Google Patents

Abstract

The present invention relates to a method and a system for detecting disaster damage information based on artificial intelligence using a drone. According to the present invention, the method for detecting disaster damage information based on artificial intelligence using a drone comprises the steps of: receiving video information of a disaster area from a drone; determining a disaster type and a disaster damage area; and producing an image map. According to the present invention, disaster damage information can be more accurately detected.

Description

Disaster damage information detection method and system based on artificial intelligence using a drone {METHOD AND SYSTEM FOR DETECTING DISASTER DAMAGE INFORMATION BASED ON ARTIFICIAL INTELLIGENCE USING DRONE}

The present invention relates to a method and system for detecting disaster damage information based on artificial intelligence using a drone, and in more detail, by constructing an artificial intelligence model by constructing a learning data set based on the drone's camera video information and the drone's flight log information. , It relates to a technology that can more accurately and objectively detect disaster damage information.

In general, when natural disasters such as floods occur, local governments or central agencies in the affected area conduct damage investigations on private facilities and public facilities.

However, not only the lack of investigative manpower for the actual damage investigation, but also the site manager in charge of the damage site conducted a full investigation of the road around the sub-river at the disaster site, river banks, and facilities in and out of the disaster site to determine the scale of the damage. The overall survey status, such as surveys and calculations, is insufficient.

In this way, in accurately investigating and calculating the magnitude of damage in a disaster area, there is a problem that the accuracy of information as a result of the investigation of the affected area is significantly deteriorated as the number of personnel and survey time is insufficient and the survey tool is also insufficient.

In order to solve this problem, recently, a drone-based investigation platform technology that can quickly and conveniently acquire a wide range of disaster damage information by flying a drone over a disaster area is in the spotlight.It is more effective to use drones to promptly and objectively detect disasters. There is a need for a way to detect and investigate damage information by type.

Artificial Intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing Rule-based smart systems, machines learn and judge themselves.

As artificial intelligence systems are used, the recognition rate improves and target types can be understood more accurately, and the existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

As a disaster investigation system technology based on such artificial intelligence, a disaster response system technology using a smart drone of Korean Patent Publication No. 2020-0065316 has been developed.

The disaster response system technology using the smart drone first specifies the object to be recognized as an object by labeling object parts such as people, cars, fire, and animals in the image data transmitted by the drone, and is primarily filtered. The accuracy of object recognition is improved by inserting the object recognition target into the disaster situation object recognition model prepared in advance to match the object recognition target and the object recognition model secondarily, but the labeled object that does not match properly is the object through machine learning. By extracting object information by matching the recognition model, specific object information such as disaster response-related information (e.g., fire scale, number of people or animals to be rescued, damage targets such as vehicles, etc.) is provided based on the analyzed object information. It is characterized by that.

On the other hand, in constructing artificial intelligence models such as machine learning, the image information captured by the drone's camera in the disaster area is transmitted to the analysis device and becomes the basis for generating image information for modeling. The analysis device is the drone's flight information. The video file is saved by synchronizing the log and the video meta information of the camera.

In other words, synchronization is required to map (geotag) the still image constituting the video and the location information of the drone.

At this time, the video information taken by the camera is generally stored at an average of 30 frames per second (30 Hz), and the flight log information of the drone (the position, speed and direction information of the drone) is less than that (for example, an average of 10 frames per second (10 Hz)). ), the period (Hz) of the two data does not match.

Therefore, if time is accumulated and synchronized in sequence using only the drone's camera video information and the drone's flight log information (drone's position, speed, and direction information), errors are accumulated as time passes. Occurs.

Therefore, when constructing an artificial intelligence (AI) model for detecting disaster damage information using drones, by constructing and modeling a more accurate and objective learning data set, it is possible to automatically extract area-based disaster information and provide accurate damage information. Disaster damage information detection technology based on artificial intelligence is required.

Korean Patent Application Publication No. 10-2020-0065316

The present invention has been conceived to solve the above-described problem, and an object of the present invention is to present a method and a system for automatically detecting information on the type of disaster and the area affected by the disaster using artificial intelligence and drone aerial video.

In particular, the training data set for artificial intelligence (AI) models to detect disaster damage information through the process of precisely synchronizing the individual frames of the drone's video captured using the drone and the drone's automatic flight log information. When constructing, creating an artificial intelligence model trained with these learning data sets, and photographing a random disaster area using the created artificial intelligence model, the artificial intelligence model more accurately and objectively judges the type of disaster and the area affected by the disaster. , It aims to provide a technology that can conveniently provide disaster type and disaster damage information to users by mapping it on a map using location information.

According to an aspect of the present invention, in a disaster damage information detection method of an artificial intelligence-based disaster damage information detection system using a drone, receiving video information of a disaster area from a drone, and pre-set artificial intelligence ( AI) determining the disaster type and the disaster damage area through the model, including the step of making and visualizing the determined disaster type and disaster damage area information as an image map, the artificial intelligence (AI) model, the flight of the drone It is characterized by being formed by learning through a plurality of learning data sets based on synchronization of log information and video meta information.

According to another aspect of the present invention, in an artificial intelligence-based disaster damage information detection system using a drone, a drone that acquires video meta information of a disaster area through a camera mounted on the body and obtains flight log information of the body. By wirelessly communicating with the device and the drone device, the video meta information of the disaster area and the flight log information of the drone are transmitted from the drone, and the disaster type according to the video information of the disaster area transmitted through a preset artificial intelligence (AI) model and Comprising: a workstation device that determines disaster damage area information and visualizes it to a user, wherein the artificial intelligence (AI) model includes a plurality of learning data based on synchronization of drone flight log information and video meta information It is characterized by being formed by learning through three.

The method and system for detecting disaster damage information based on artificial intelligence using a drone according to the present invention captures a disaster area with a drone, and uses the captured image information to obtain real-time information on the type of disaster and disaster damage area through a preset artificial intelligence model. It has the advantage that it can be accurately and easily grasped.

In addition, in forming a training data set for building an artificial intelligence model, it is possible to reduce the error between the viewpoints of the video frames that may occur during the creation of a disaster video through precise synchronization between the video information of the drone and the flight information of the drone. You can build an accurate and objective learning data set.

In addition, there is an effect of constructing an artificial intelligence model for disaster analysis with high accuracy by performing labeling according to specific and various types of disaster information through the thus constructed learning data set and learning a number of them.

1 is a view for explaining a basic investigation system of a disaster damage information detection system based on artificial intelligence using a drone according to an embodiment of the present invention.
2 is a system diagram showing the configuration of an artificial intelligence-based disaster damage information detection system using a drone according to an embodiment of the present invention.
3 is a system diagram showing a detailed configuration of a disaster information analysis unit of an artificial intelligence-based disaster damage information detection system using a drone according to an embodiment of the present invention.
4 is a diagram illustrating a process of synchronizing flight log information and video meta information of a drone according to an embodiment of the present invention.
5 is a flow chart illustrating a method of detecting disaster damage information based on artificial intelligence using a drone according to an embodiment of the present invention.
6 is a flowchart illustrating a method of constructing an artificial intelligence model according to an embodiment of the present invention.
7 is a view for explaining a method of constructing a learning data set according to a landslide disaster type in the present invention.
8 is a diagram for explaining a method of constructing a learning data set according to a type of heavy rain disaster in the present invention.
9 is a view for explaining a method of constructing a learning data set according to a traffic accident disaster type in the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another component.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a basic investigation system of an artificial intelligence-based disaster damage information detection system 10 using a drone according to an embodiment of the present invention, and FIG. 2 is a diagram using a drone according to an embodiment of the present invention. A system diagram showing the configuration of the artificial intelligence-based disaster damage information detection system 10, and FIG. 3 is a disaster information analysis unit of the artificial intelligence-based disaster damage information detection system 10 using a drone according to an embodiment of the present invention ( 300) is a system diagram showing a specific configuration, and FIG. 4 is a diagram illustrating a process of synchronizing flight log information and video meta information of a drone according to an embodiment of the present invention.

First, referring to FIG. 1, an artificial intelligence-based disaster damage information detection system 10 using a drone according to an embodiment of the present invention is provided through the drone device 100 while flying over the disaster area. And transmits the video information and the flight information of the drone captured through the drone device 100 to the workstation device, and the type of disaster and the disaster damage area through the artificial intelligence model preset in the workstation device 200. It has a system that detects and analyzes quickly and accurately.

Here, the workstation device 200 is a computer device equipped with a high-performance GPU, and can be formed as a field operation artificial intelligence system that can be operated at the disaster site by being provided in an investigation vehicle that explores the disaster site. It is processed by the disaster management agency's automatic detection server, and can form an artificial intelligence system for internal operation that automatically detects with high performance.

Referring to FIG. 2, an artificial intelligence-based disaster damage information detection system 10 using a drone according to an embodiment of the present invention may include a drone device 100 and a workstation device 200.

The drone device 100 acquires video meta information of a disaster area through a camera module mounted on the body, acquires flight log information of the body, and transmits it to the workstation device 200.

To this end, the drone device 100 is configured to include a body (not shown), a camera module 110, a sensor module 120, a flight module 130, a communication module 140 and a control module 150.

The camera module 110 is provided in the main body of the device and is composed of an optical camera to photograph the disaster area to generate image information of the disaster area.

The sensor module 120 is provided in the main body, and may include various sensors such as a Global Navigation Satellite System (GNSS)/Inertial Navigation System (INS), a gyro sensor, an accelerometer and a barometer.

The flight module 130 is provided in the main body and can be operated by floating the main body in the sky, and stores flight log information such as the position, speed, and camera direction of the drone during flight operation of the drone.

Here, the flight module 130 may be applied to various drone flight devices currently available, and of course, it may be variously changed according to a flight operation setting.

The communication module 140 is provided in the main body and consists of communication equipment such as frequency communication and LTE-based mobile communication, and performs real-time wireless communication with the workstation device 200.

The control module 150 is provided inside the main body, and transmits the image information generated by the camera module 110 to the workstation device 200 through the communication module 140, and from the workstation device 200 The camera module 110, the sensor module 120, and the flight module 130 are respectively controlled according to the transmitted control signal.

The workstation device 200 includes a communication unit 210, a disaster information analysis unit 300, an image processing unit 220, a damage information analysis unit 230, an information input unit 240, and an information display unit 250. .

The communication unit 210 is composed of a communication module such as frequency communication and LTE-based mobile communication, and performs real-time wireless communication with the drone device 100.

The disaster information analysis unit 300 determines disaster type and disaster damage area information through a preset artificial intelligence model based on the video information transmitted from the drone device 100.

Referring to FIG. 3, the disaster information analysis unit 300 includes a learning data set construction unit 310 for constructing a plurality of learning data sets based on synchronization of drone flight log information and video meta information, and the learning data set. It is configured to include an artificial intelligence model unit 320 formed by learning a plurality of learning through the training data set built in the construction unit 310.

The learning data set construction unit 310 includes an information recording unit 311, an information synchronization unit 312, an image extracting unit 313, an image correction unit 314, a learning information generation unit 315, and a database unit 316. Done.

The information recording unit 311 records and stores meta information of the drone video, and the information synchronization unit 312 synchronizes the individual frames of the video and flight log information of the drone through a preset synchronization algorithm.

Here, referring to FIG. 4, the preset synchronization algorithm finds a first specific time point in which the amount of pixel change between each frame is more than a certain range in the individual frame of the video, and the attitude change amount of the drone in the flight log information of the drone is within a certain range. Find a second specific point in time that is abnormal, find a third specific point in time in which the gimbal's posture change amount is more than a certain range from the flight log information of the drone, and use the first, second, and third specific points in time, that is, synchronization points. You can synchronize the individual frames of the video and the flight log information of the drone.

That is, as described above, synchronization is required in order to map (geotag) the location information of the drone and the still image constituting the video. In general, the video information taken by the camera is stored at an average of 30 frames per second (30 Hz), and Since the flight log information (the drone's position, speed, and direction information) is recorded at a smaller period (for example, an average of 10 frames per second (10Hz)), the drone's camera video information and the drone's flight log information ( When time is accumulated and synchronized sequentially using only initial time information (position, speed, and direction information), an error increases as time passes.

Accordingly, in the present invention, a specific time point with a large change in the entire video section is found using the amount of pixel change (difference from adjacent video frames) between each frame of the video through a synchronization algorithm, and the attitude of the drone and the camera gimbal posture from the flight log information. It is possible to improve the accuracy by reducing the accumulated synchronization error by finding and synchronizing the time point where the change (difference from adjacent log information) is large.

The image extracting unit 313 extracts a specific key frame (still image) based on preset key frame extraction reference information among individual frames of a video of the drone.

The reason for extracting a specific key frame is that the number of frames in the video is too many and there is little change in the content of the video in each frame, so if the attitude of a drone or camera changes rapidly, the image taken by the camera will change significantly. According to the information, a specific key frame in which the posture of the drone/camera changes rapidly is extracted for a certain period, a part with a large amount of change in the image.

At this time, the preset key frame extraction reference information includes information for each period, pixel change amount information, and attitude change amount information of drones and cameras, and can be set and changed by a user.

The image correction unit 314 records (geotags) the log information (position/position information) of the drone corresponding to the image information of the key frame extracted by the image extraction unit 313 as meta information of the image. Meta information can be corrected.

The learning information generation unit 315 generates learning data by labeling the disaster type and the disaster damage area according to the preset disaster hierarchy information in the extracted image information of the key frame. In the labeled area, each pixel coordinate can be saved in Json format for learning.

That is, the learning information generation unit 315 receives a plurality of image information for each type of disaster as input data, and learns a plurality of pieces of information on the type of disaster according to the hierarchy and the area information labeled for the disaster-damaged area as output data. It is to create the learning data according to.

Disaster hierarchy information is classified into upper, middle, and lower hierarchy information for each disaster type, and may be preset by a system manufacturer, expert, or manager.

The upper hierarchical information includes natural and social disaster information, and the median hierarchical information includes 20 kinds of nature including typhoon, heavy rain, cold wave, heavy snow, heatwave, drought, earthquake, yellow dust, tsunami, flood, flood, landslide, volcano. Disaster information, fire, collapse, explosion, forest fire, traffic accident, electric accident, water accident, infectious disease, and 20 kinds of social disaster information including fine dust can be included.

In addition, the lower hierarchical information may include specific disaster information including soil leakage, facility destruction, road destruction, landslide leakage, building fire, explosion, and vehicle damage according to natural and social disaster information.

The database unit 316 generates a learning data set by converting a plurality of learning data generated through the learning information generation unit 315 into a database (DB).

As described above, the training data set construction unit 310 generates training images of an appropriate size from drone videos in the affected area, classifies objects according to disaster types, and labels them by hierarchy according to the hierarchy to generate learning data. By becoming a database, it is possible to construct a learning data set.

The artificial intelligence model unit 320 is composed of an information learning unit 321 and a modeling unit 322, and by learning a plurality of the training data set built in the learning data set construction unit 310, the disaster image information input by itself is Processing, and constructing an artificial intelligence model to automatically determine the disaster type and disaster damage information.

The information learning unit 321 generates a plurality of determination information through automatic learning of various disaster event information a plurality of times based on a plurality of learning data sets generated through the learning data set construction unit 310. The modeling unit 322 forms a detection and analysis algorithm using a plurality of determination information generated through the information learning unit.

That is, the artificial intelligence model unit 320 searches for a disaster-damaged area from a drone video and forms an optimal artificial intelligence algorithm for extracting damage information.

In more detail, in consideration of the real-time nature of the system, when an HD image is given as an input, a neural network structure capable of inferencing can be formed by performing domain-based semantic segmentation of the presence or absence of a disaster within a short time.

In addition, the artificial intelligence model unit 320 may be configured as a neural network module to utilize not only the local characteristics of the image but also the global characteristics, taking into account the characteristics of disaster detection, and in a semantic segmentation database occurring in a general disaster situation. It may also have a function of a loss function for correcting the unbalance of the labeling of.

Meanwhile, the artificial intelligence model unit 320 may include a model evaluation unit (not shown) for evaluating the artificial intelligence model built through the modeling unit 322.

The model evaluation unit (not shown) automatically extracts area-based disaster information through artificial intelligence learning using the established training data and detection analysis algorithm, analyzes accuracy, and evaluates performance.

Such a model evaluation unit (not shown) is a learning module that learns neural network parameters using the built training data set, an extraction module that automatically extracts area-based disaster information using a learned model algorithm, and a trained model. It can be composed of an evaluation module that measures and evaluates the performance of the algorithm as mIOU by using the evaluation data set built in and.

The image processing unit 220 creates and visualizes the disaster type and last damage area information determined by the disaster information analysis unit 300 as orthogonal image map information.

At this time, the image processing unit 220 determines the absolute position of each pixel using information on the position/posture angle (X, Y, Z, Roll, Pitch, Yaw) of the drone among meta information of the video, and generates an orthophoto. After generation, the generated orthogonal image is visualized by superimposing it on a background map (OpenstreetMap) driven by a web browser. Here, the user may roughly obtain the area of the damaged area by drawing an area with a mouse on the orthogonal image visualized on the map.

In addition, the image processing unit 220 may perform 3D modeling to obtain 3D image information such as 3D point cloud data, 3D topographic information, and orthogonal image map information.

The damage information analysis unit 230 analyzes the damage length, area, and outflow amount of the feature through a preset analysis program based on the orthogonal image information acquired through the image processing unit 220 to provide damage scale information.

The information input unit 240 receives various input information of a user, and the information display unit 250 displays and provides various input/output information and image information to the user.

As described above, the artificial intelligence-based disaster damage information detection system using a drone according to the present invention precisely synchronizes individual frames of a drone's video with the drone's automatic flight log information, thereby establishing a learning data set. There are features that can provide artificial intelligence models for high disaster analysis.

Hereinafter, a method of detecting disaster damage information using an artificial intelligence-based disaster damage information detection system using a drone as described above will be described in detail with reference to FIGS. 5 to 9.

5 is a flow chart illustrating a method of detecting disaster damage information based on artificial intelligence using a drone according to an embodiment of the present invention, and FIG. 6 is a flowchart illustrating a method of constructing an artificial intelligence model according to an embodiment of the present invention. It is a flow chart, and FIG. 7 is a diagram for explaining a method of constructing a learning data set according to a type of landslide disaster in the present invention, and FIG. 8 is a diagram for explaining a method of constructing a learning data set according to a type of heavy rain disaster in the present invention. 9 is a diagram for explaining a method of constructing a learning data set according to a traffic accident disaster type in the present invention.

As shown in FIG. 5, the method for detecting disaster damage information based on artificial intelligence using a drone according to an embodiment of the present invention is, prior to photographing a damaged area where a disaster has occurred through a camera module mounted on the drone device, artificial intelligence The model presetting step S100 is performed. Thereafter, the disaster area is photographed with a drone device (S200), and the recorded disaster area video information is determined by self-determining the type information of the disaster and the disaster-damaged area information through the execution of a preset artificial intelligence model (S210). Disaster type information and disaster damage area information may be visualized and provided to a user through a display device (S220).

Referring to FIG. 6, in the step of pre-setting an artificial intelligence model (S100), first, flight log information of a drone in a disaster area and meta information of a video captured by the drone through a camera module are input (S110).

At this time, the drone's flight log information includes the drone's location (usually, GNSS latitude and longitude coordinates, flight altitude) information, the drone's speed information, and camera attitude information, where the drone's location information is the camera's location information or the video It may be location information that is photographing.

The video meta information includes basic information such as the size of the photo file, shooting date and time, aperture, ISO, camera model name, and the focal length at the time of shooting, the latitude/longitude coordinates of the GNSS mounted on the drone, the flight altitude, and the roll angle. , pitch, yaw) information may be included.

Thereafter, in order to map (geotag) the individual frames (still images) constituting the video and the location information of the drone, the individual frames of the video and the flight log information of the drone are synchronized (S120).

In this case, a detailed description of the information synchronization has been described in detail with reference to FIG. 4 for the synchronization process of the flight log information and the video meta information of the drone of the information synchronization unit of the present invention, and thus will be omitted in this embodiment.

After synchronizing the individual frames of the video with the flight log information of the drone (S120), a specific key frame in which the posture of the drone/camera changes rapidly by a certain period, a part with a large amount of change in the image according to the preset key frame extraction reference information Extract (S130).

Thereafter, in order to map (geotag) the location information of the drone to the image information (still screen) of the extracted key frame, the log information (position/position information) of the drone corresponding to the image information of the extracted key frame is imaged. Record (geo-taking) with meta information of (S140).

Thereafter, a plurality of image information for each disaster type is created according to the preset disaster hierarchy information, and the corresponding disaster image information is labeled (S150) to form labeling area information.

Thereafter, according to the preset disaster hierarchy information, a plurality of image information for each disaster type is used as input data, and the disaster type information according to the hierarchy and the labeling area information for the disaster-damaged area are used as output data to learn according to the disaster hierarchy information. Data is generated (S160).

For example, referring to FIG. 7, video information on a landslide disaster is set as input data, and the disaster hierarchy information is set to natural disasters, intermediate information is a landslide, and summer information is set to a landslide leakage. By labeling (red area) and learning it as output data, afterwards, when a disaster image similar to the disaster is input, it automatically determines that the disaster type of the image is a landslide, and the disaster damage area is also extracted. You can do it.

In addition, referring to FIG. 8, by setting the image information on the heavy rain disaster as input data, the disaster hierarchy information as the natural disaster, the middle information as the heavy rain, and the summer information as the road flood, the area where the inundation occurred is labeled. By learning with the output data, afterwards, when a disaster image similar to the disaster is input, it automatically determines that the disaster type of the image is heavy rain, and the disaster damage area can also be extracted.

In addition, referring to FIG. 9, image information on traffic accidents is set as input data, and disaster hierarchy information is set as social disasters, median information is traffic accidents, and summer information is set as vehicle damage. After labeling and learning as output data, when a disaster image similar to the disaster is input, it automatically determines that the disaster type of the image is a traffic accident, and the disaster damage area can also be extracted. .

Using the plurality of learning data generated in this way, the training data set is built into a database (S170), and various disaster event information is automatically learned a number of times through the built learning data set to determine the disaster-affected area from the drone video. An optimal disaster analysis artificial intelligence model can be constructed (S180) to search and determine and extract damage information by itself.

As described above, the method and system for detecting disaster damage information based on artificial intelligence using a drone according to the present invention is a frame-by-frame view of a video that may occur when a disaster video is generated through precise synchronization between the video information of the drone and the flight information of the drone. A more accurate and objective training data set can be reduced by reducing the intermittent error, and an artificial intelligence model for disaster analysis with high accuracy is created by performing labeling according to specific and various types of disaster information through the training data set. By constructing, there is an effect of being able to accurately and easily grasp the type of disaster and information on the disaster damage area in real time through a preset artificial intelligence model in the video taken by the drone device.

10: Disaster damage information detection system based on artificial intelligence using drones
100: drone device
110: camera module, 120: sensor module
130: flight module, 140: communication module
150: control module
200: workstation device
210: communication unit, 220: image processing unit
230: damage information analysis unit, 240: information input unit
250: information display unit
300: Disaster Information Analysis Department
310: Learning data set construction unit
311: information recording unit, 312: information synchronization unit
313: image extraction unit, 314: image correction unit
315: learning information generation unit, 316: database unit
320: artificial intelligence model unit
321: Information Learning Department, 322: Modeling Department

Claims (21)

In the disaster damage information detection method of a disaster damage information detection system based on artificial intelligence using a drone,
Receiving video information of the disaster area from the drone;
Determining a disaster type and a disaster damage area using the transmitted video information through a preset artificial intelligence (AI) model;
Including; and visualizing the determined disaster type and disaster damage area information by making an image map,
The artificial intelligence (AI) model,
It is formed by learning through a plurality of learning data sets based on synchronization of drone flight log information and video meta information,
The method of constructing the learning data set,
Receiving flight log information and video meta information about the disaster area of the drone from the drone to the workstation device, respectively;
Recording and storing the transmitted flight log information and video meta information of the drone in an information storage unit;
Synchronizing individual frames of the video and flight log information of the drone through a preset synchronization algorithm;
Extracting a specific key frame based on preset key frame extraction reference information from among individual frames of a video of the drone;
Recording log information of the drone corresponding to the image information of the extracted key frame as meta information of the image;
Generating learning data by labeling a disaster type and a disaster damage region according to preset disaster hierarchy information on the extracted image information of the key frame; And
Converting the generated learning data into a database;
Including,
The synchronization algorithm,
Finding a first specific viewpoint in which a pixel change amount between each frame is greater than or equal to a predetermined range in individual frames of the video;
Finding a second specific point in time in which the amount of change in attitude of the drone is greater than a certain range from the flight log information of the drone;
Finding a third specific point in time in which the posture change amount of the gimbal is greater than a certain range from the flight log information of the drone; And
Synchronizing individual frames of the drone video and flight log information of the drone using the first, second and third specific points of view;
Artificial intelligence-based disaster damage information detection method using a drone comprising a.
The method of claim 1,
The training data set,
Artificial using drones, characterized in that it is constructed by generating a plurality of learning data by using a plurality of image information for each disaster type as input data, and using disaster type information according to the hierarchy and labeling area information for the disaster-damaged area as output data. Intelligence-based disaster damage information detection method. delete The method of claim 1,
The video meta information includes basic information including the size of the photo file, the shooting date, aperture, ISO, and the camera model name, the focal length at the time of shooting, the latitude/longitude coordinates of the GNSS mounted on the drone, the flight altitude, and A method of detecting disaster damage information based on artificial intelligence using a drone, characterized in that it includes information about an attitude angle (roll, pitch, yaw).
delete The method of claim 1,
The key frame extraction reference information,
A method of detecting disaster damage information based on artificial intelligence using a drone, characterized in that it includes information for each period, pixel change information, and attitude change information of drones and cameras, and is set by a user.
The method of claim 1,
The above disaster hierarchy information,
Higher, medium, and lower hierarchical information is included for each type of disaster,
Higher hierarchy information includes natural and social disaster information,
Median hierarchical information includes information on natural disasters including typhoon, heavy rain, cold wave, heavy snow, heatwave, drought, earthquake, yellow dust, tsunami, flood, inundation, landslide, volcano, fire, collapse, explosion, forest fire, traffic accident, electric accident. , Water disaster, infectious disease, and social disaster information including fine dust,
Disaster based on artificial intelligence using drones, characterized in that the lower hierarchical information includes specific disaster information including soil leakage, facility destruction, road destruction, landslide leakage, building fire, explosion, and vehicle damage according to natural and social disaster information. How to detect damage information.
The method of claim 1,
In the step of making and visualizing the disaster type and disaster damage area information as an image map,
Among the meta information of the video, the absolute position of each pixel is determined using the drone's position/position (X, Y, Z/roll, pitch, and yaw) information, and after generating an orthophoto, the generated orthogonal image A method of detecting disaster damage information based on artificial intelligence using a drone, characterized in that superimposed on and visualized on a background map (OpenstreetMap) driven in a web browser.
The method of claim 1,
After the step of making and visualizing the disaster type and disaster damage area information as an image map,
An artificial intelligence-based disaster damage information detection method using a drone, characterized in that the step of providing damage scale information by analyzing the damage length, area, and outflow amount of the feature through a preset analysis program.
In a disaster damage information detection system based on artificial intelligence using a drone,
A drone device that acquires video meta information of a disaster area through a camera mounted on the body, and acquires flight log information of the body; And
Through wireless communication with the drone device, video meta information of the disaster area and flight log information of the drone are transmitted from the drone, and the disaster type and disaster damage according to the video information of the disaster area transmitted through a preset artificial intelligence (AI) model. It is configured to include; a workstation device that determines the area information and visualizes it to the user,
The artificial intelligence (AI) model,
It is formed by learning through a plurality of learning data sets based on synchronization of drone flight log information and video meta information,
The workstation device,
A communication unit for wireless communication with the drone device;
A disaster information analysis unit for determining disaster type and disaster damage area information through a preset artificial intelligence model from the video information transmitted from the drone device;
An image processing unit for producing and visualizing the disaster type and disaster damage area information determined by the disaster information analysis unit as orthogonal image map information;
An information input unit for receiving various input information of a user; And
An information display unit for displaying various input/output information and image information to a user;
It is composed including,
The disaster information analysis unit,
A learning data set construction unit for constructing a plurality of learning data sets based on synchronization of drone flight log information and video meta information; And
An artificial intelligence model unit formed by learning a plurality of learning data sets constructed by the learning data set construction unit;
Including,
The learning data set construction unit,
An information recording unit for recording and storing meta information of a video of a drone;
An information synchronization unit for synchronizing individual frames of the video and flight log information of the drone through a preset synchronization algorithm;
An image extracting unit for extracting a specific key frame from among individual frames of a video of the drone based on preset key frame extraction reference information;
An image correction unit for recording log information of the drone corresponding to image information of the extracted key frame as meta information of the image;
A learning information generator for generating learning data by labeling a disaster type and a disaster damage area according to preset disaster hierarchy information on the extracted image information of the key frame; And
A database unit for generating a learning data set by converting a plurality of learning data generated through the learning information generation unit into a DB;
Including,
The synchronization algorithm,
In individual frames of the video, a first specific point in time in which the amount of pixel change between each frame is more than a certain range is found, a second specific time point in which the attitude change amount of the drone is more than a certain range is found in the flight log information of the drone, and the flight log of the drone A third specific point in time in which the gimbal's posture change amount is more than a certain range from the information, and the individual frames of the drone's video and the drone's flight log information are synchronized using the first, second, and third specific points of view. Disaster damage information detection system based on artificial intelligence using drones.
The method of claim 10,
The drone device,
A camera module provided in the main body of the device and configured of an optical camera to photograph the disaster area to generate video information of the disaster area;
A sensor module provided in the main body and comprising various sensors;
A flight module provided on the main body, floating the main body in the air, and generating flight log information of the main body;
A communication module provided in the main body for real-time wireless communication with the workstation device; And
It is provided inside the main body, transmits video information generated by the camera module to the workstation device through the communication module, and transmits flight log information generated by the flight module to the workstation device through the communication module, A control module for controlling the camera module, the sensor module and the flight module according to the control signal transmitted from the workstation device;
Disaster damage information detection system based on artificial intelligence using a drone, characterized in that configured to include.

delete delete delete The method of claim 10,
The artificial intelligence model unit,
An information learning unit for learning various disaster-related information by using a plurality of learning data sets generated through the learning data set construction unit; And
A modeling unit for forming an analysis model using a plurality of determination information generated through the information learning unit;
Disaster damage information detection system based on artificial intelligence using a drone comprising a.
The method of claim 10,
The training data set,
Artificial using drones, characterized in that it is constructed by generating a plurality of learning data by using a plurality of image information for each disaster type as input data, and using disaster type information according to the hierarchy and labeling area information for the disaster-damaged area as output data. Intelligence-based disaster damage information detection system.

delete The method of claim 10,
The key frame extraction reference information,
Disaster damage information detection system based on artificial intelligence using a drone, characterized in that it includes information for each period, pixel change information, and attitude change information of drones and cameras, and is set by a user.
The method of claim 10,
The above disaster hierarchy information,
Higher, medium, and lower hierarchical information is included for each type of disaster,
Higher hierarchy information includes natural and social disaster information,
Median hierarchical information includes information on natural disasters including typhoon, heavy rain, cold wave, heavy snow, heatwave, drought, earthquake, yellow dust, tsunami, flood, inundation, landslide, volcano, fire, collapse, explosion, forest fire, traffic accident, electric accident. , Water disaster, infectious disease, and social disaster information including fine dust,
Disaster based on artificial intelligence using drones, characterized in that the lower hierarchical information includes specific disaster information including soil leakage, facility destruction, road destruction, landslide leakage, building fire, explosion, and vehicle damage according to natural and social disaster information. Damage information detection system.
The method of claim 10,
The image processing unit,
Among the meta information of the video, the absolute position of each pixel is determined using the drone's position/position (X, Y, Z/roll, pitch, and yaw) information, and after generating an orthophoto, the generated orthogonal image Disaster damage information detection system based on artificial intelligence using a drone, characterized in that superimposed and visualized on a background map (OpenstreetMap) driven in a web browser.
The method of claim 10,
Based on the orthogonal image map information obtained through the image processing unit, a damage information analysis unit that analyzes the damage length, area, and outflow amount of the feature through a preset analysis program and provides damage scale information is additionally included. Disaster damage information detection system based on artificial intelligence using drones.

Images