KR102559307B1 - Method, device and system for processing and utilizing images and video data generated at disaster sites - Google Patents

Abstract

According to an embodiment, the sensor transmits field data including detection data and location data to the server when it is determined that a disaster has been detected by detection and sensor values of the sensor, and when the server receives the field data from the sensor, image data is obtained through a camera installed in the region, and the acquired image data is analyzed to determine whether a disaster has occurred. Considering this, a message to check the sensor is transmitted to the terminal of the regional manager.

Description

Methods, devices and systems for processing and utilizing images and video data generated at disaster sites { METHOD, DEVICE AND SYSTEM FOR PROCESSING AND UTILIZING IMAGES AND VIDEO DATA GENERATED AT DISASTER SITES }

The following embodiments relate to methods of processing and utilizing images and video data generated at a disaster site.

Managing and utilizing disaster data and predicting disaster accidents are important because they can minimize human and property damage due to disasters, combine past data and on-site data to establish efficient response plans, provide data on the possibility of expansion and scale of support for accidents, and provide input budget data so that budgets can be used efficiently, ensuring social safety and ensuring financial soundness through preemptive measures.

In addition, predicting a disaster accident is much more important than responding after a disaster occurs, which enables preparation in advance, minimizing damage to life and property, shortening response time, and further preventing the occurrence of accidents.

Recently, as the possibility of occurrence of natural disasters increases in relation to climate change, it is becoming more important to predict and cope with them in advance.

In addition, it is expected that crime, industrial accidents, and traffic accidents can be reduced by applying image and video data generation and utilization to situations such as various crimes, traffic accidents, and industrial accidents, and the technology to predict accidents (crime) is emerging as a very important issue and is expected to be further developed.

Korean Patent Registration No. 10-2284747 (2021.08.03. Notice) Korean Patent Registration No. 10-2490062 (2023.01.18. Notice) Korean Patent Registration No. 10-2485227 (2023.01.06. Notice) Korean Patent Registration No. 10-1732819 (Announced on May 8, 2017)

Embodiments attempt to predict an accident by processing and utilizing images and video data generated at a disaster site.

In the case of a disaster, in the case of a disaster, human damage is predicted through field data and image data, and the scale and spread of the disaster are predicted to select which safety manager to support and to what extent to request the safety manager to mobilize.

Embodiments attempt to predict the occurrence of disasters by identifying patterns before and after disasters occur through artificial neural networks and comparing corresponding patterns with field data and image data.

A method of processing and utilizing images and video data generated at a disaster site, when a sensor determines that a disaster has been detected by the sensor's detection and sensor values, transmitting field data including detection data and location data to a server; acquiring image data through a camera installed in the region when the server receives field data from the sensor; determining, by the server, whether a disaster has occurred by analyzing the obtained image data; If it is confirmed that a disaster has occurred, the server transmitting a message requesting dispatch to the region to a terminal of a safety manager matched to the region; and if it is determined that no disaster has occurred, the server transmitting a message requesting confirmation of the sensor to a terminal of a regional manager in consideration of a possibility of error of the sensor.

If it is confirmed that a disaster has occurred, the server transmitting a message requesting dispatch to the area to the terminals of safety managers matched to the area, the server confirming the type of disaster through the field data, the server determining whether a person is included in the image data through the image data, if it is confirmed that a person is included in the image data, the server selecting a safety manager related to the type of disaster and the lifesaving among safety managers matched to the area as candidate safety managers, a person in the image If it is determined that is not included, the server selecting a safety manager related to the type of disaster among safety managers matched to the region as candidate safety managers, the server generating the emergency severity of the disaster through the image data and the site data, the server generating the disaster spread through the range of the sensor that transmitted the field data, the server predicting the scale of the disaster through the disaster severity and the disaster spread, and the server corresponding to the disaster scale among the candidate safety managers Selecting a final safety manager by extracting the safety managers of, and transmitting, by the server, a message requesting dispatch to the area to a terminal of the final safety manager.

The server generating the urgency of the disaster through the image data and the field data, the server increasing the sensitivity of the sensor if it is determined that a person is included in the image data, the server maintaining the sensitivity of the sensor if it is determined that a person is not included in the image data, the server confirming a period in which field data is transmitted from the sensor, the server determining whether the period is shorter than a preset reference period, if the period is determined to be shorter than the reference period, the server Generating the urgency of the disaster as high, and if it is determined that the period is not shorter than the reference period, the server generating the urgency of the disaster as low.

Generating, by the server, the spread of the disaster through the range of the sensor that transmitted the field data, the server determining whether all the sensors that transmitted the field data were included within a preset first range, if it was determined that all the sensors that transmitted the field data were included within the first range, the server generating the spread of the disaster as low, and if it was confirmed that at least one of the sensors that transmitted the field data was not included within the first range, the server transmitted all the sensors that transmitted the field data in advance. Determining whether or not it is included within a set second range; if it is determined that all of the sensors that transmitted the field data are included within the second range, the server generating a normal spread of the disaster; and if it is determined that at least one of the sensors that transmitted the field data is not included within the second range, the server generating a high spread of the disaster.

A method for processing and utilizing image and video data generated at a disaster site, the server generating sensor value change information based on field data obtained for a preset time before a disaster occurs; generating, by the server, image change information based on image data captured during the set time period before a disaster occurs; generating, by the server, an input signal based on the sensor value change information and the image change information; obtaining, by the server, an output signal by applying the input signal to an artificial neural network; Confirming, by the server, a disaster occurrence pattern based on the output signal; Comparing, by the server, field data obtained from the sensor or image data acquired through the camera with the disaster occurrence pattern to determine whether a matching pattern exists; and when it is determined that a pattern matching the disaster warning pattern exists, the server tracking a change in field data received from the sensor, and transmitting a message requesting disaster prevention response to a terminal of a user located in the corresponding region.

The server according to an embodiment may be combined with hardware and controlled by a computer program stored in a medium in order to execute any one of the methods described above.

Embodiments can predict an accident by processing and utilizing images and video data generated at a disaster site.

When a disaster occurs, the embodiments predict whether there is human damage through field data and image data, the scale and spread of the disaster, select which safety manager to support and to what extent, and request the safety manager to mobilize.

Embodiments can identify patterns before and after a disaster occurs through an artificial neural network, and predict the occurrence of a disaster by comparing the pattern with field data and image data.

On the other hand, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically showing the configuration of a system according to an embodiment.
2 is a flowchart illustrating a process of receiving image data through a camera and determining a disaster through the image data when field data is received according to an embodiment.
3 is a flowchart illustrating a process of transmitting a message requesting a dispatch to a region to a terminal of a safety manager matched to a region according to an embodiment.
4 is a diagram for explaining a process of generating an emergency degree of disaster according to an embodiment.
5 is a flowchart illustrating a process of generating a spread of a disaster according to an embodiment.
6 is a flowchart illustrating a process of predicting occurrence of a disaster according to an embodiment.
7 is a flowchart illustrating a process of providing a path to a safety manager according to an embodiment.
8 is a flowchart illustrating a process of determining whether there is a risk of fire based on a thermal imaging camera according to an exemplary embodiment.
9 is a diagram of a color spectrum according to an embodiment.
10 is an exemplary diagram of a configuration of a server according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes can be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents or substitutes to the embodiments are included within the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be modified and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Terms used in the examples are used only for descriptive purposes and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "having" are intended to specify that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but it should be understood that the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof is not excluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and are not interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

The embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

1 is a diagram schematically showing the configuration of a system according to an embodiment.

Referring to FIG. 1 , a system according to an embodiment may include a sensor 110, a camera 120, a server 200, a safety manager's terminal 310, a regional manager's terminal 320, and a user's terminal 330.

First, the sensor 110 may include at least one of a flame detection sensor, a smoke sensor, a temperature sensor, and an immersion sensor, and may be implemented as various types of sensors according to embodiments.

Specifically, the sensor 110 includes at least one of a flame detection sensor, a smoke sensor, a temperature sensor, a immersion sensor, and a vibration sensor, and may be installed in an area to determine whether a disaster has occurred in the area. When a disaster is detected by the sensor value, the sensor 110 may generate field data including detection data and location data, and may transmit the generated field data to the server 200 . That is, the sensor 110 may communicate with the server 200 by wire or wireless. Here, the disaster may include fire, flood, typhoon, earthquake, and the like, and may further include other accidents that may cause damage to human life or property.

In addition, the sensor 110 may include an embedded system in the sensor itself, the sensor 100 is normally in a sleep mode, may generate sensing data when a condition is satisfied by an input system, and may transmit field data including location data including information of the area where the sensor is located and the generated sensing data to the server 200.

Meanwhile, one or more sensors 100 may be installed in one region, and the number of sensors 110 installed in a region may vary according to embodiments. In addition, the sensor 110 may perform a task normally performed by a sensor although not described herein.

The camera 120 is a Full HD (High Definition) quality camera device for capturing video or images, and may be a recording device based on 30 to 60 frames, but is not limited thereto. The camera 120 may automatically adjust exposure, shutter speed, and sensitivity, and may be manually adjusted by a local manager.

Specifically, the camera 120 may be installed in an area to check whether a disaster has occurred in the area. The camera 120 may capture a region at the location where it is installed, and may transmit captured image data or image data to the server 200 . That is, the camera 120 may communicate with the server 200 wired or wirelessly. Also, in the following description, image data is used for convenience of explanation, but image data may include both image data and video data, and at least one piece of data may be included. In addition, the camera 120 may perform tasks normally performed by a camera, although not described in the text.

The terminal of the safety manager's terminal 310 is a terminal used by a safety manager who manages regional safety, and may be implemented as a mobile phone, desktop PC, laptop PC, tablet PC, smart phone, etc., but is not limited thereto, and may be implemented as various types of communication devices that can be connected to an external server. For example, as shown in FIG. 1 , the safety manager's terminal 310 may be a smart phone, and may be employed differently depending on the embodiment. In addition, the safety manager is a manager who performs rescue and damage restoration work for the safety of life and property, and may be a firefighter, police officer, emergency rescuer, disaster manager, etc., but is not limited thereto.

The terminal 320 of the regional manager is a terminal used by the regional manager who manages the region, and may be implemented as a mobile phone, desktop PC, laptop PC, tablet PC, smart phone, etc., but is not limited thereto, and may be implemented as various types of communication devices that can be connected to an external server. For example, as shown in FIG. 1 , the terminal 320 of the regional manager may be a smart phone or may be employed differently depending on the embodiment. In addition, the regional manager is a manager who manages the region, and may be a public official such as a mayor, a district head, or a dong head, but is not limited thereto.

The user's terminal 330 is a terminal used by a user located in a region, and may be implemented as a mobile phone, desktop PC, laptop PC, tablet PC, smart phone, etc., but is not limited thereto, and may be implemented as various types of communication devices that can be connected to an external server. For example, as shown in FIG. 1 , the user's terminal 330 may be a smart phone, and may be employed differently depending on embodiments.

The server 200 may be a server owned by a person or organization that provides services using the server 200, may be a cloud server, or may be a peer-to-peer (p2p) set of distributed nodes. The server 200 may be configured to perform all or part of an arithmetic function, a storage/reference function, an input/output function, and a control function of a normal computer. The server 200 may be configured to communicate with the sensor 110, the camera 120, the safety manager's terminal 310, the regional manager's terminal 320, and the user's terminal 330 through wired or wireless communication.

The server 200 can control the overall operation of each of the sensor 110 and the camera 120, and the server 200 obtains information through the sensor 110 and the camera 120, It is possible to check whether a disaster has occurred in an area.

When the sensor 110 determines that a disaster has been detected by the detection of the sensor 110 and the sensor value, it can transmit field data including detection data and location data to the server 200. When the server 200 receives the field data from the sensor 110, the server 200 obtains image data through the camera 120 installed in the region, analyzes the acquired image data to determine whether a disaster has occurred, and as a result of the determination, if it is confirmed that a disaster has occurred, a safety manager (3) matched to the region 10) transmits a message requesting dispatch to the region, and when it is confirmed that no disaster has occurred, a message requesting confirmation of the sensor 110 may be transmitted to the terminal 320 of the regional manager considering the possibility of a sensor error.

In the present invention, artificial intelligence (AI) means a technology that imitates human learning ability, reasoning ability, perception ability, etc., and implements it with a computer, and includes concepts such as machine learning and symbolic logic. Machine learning (ML) is an algorithm technology that classifies or learns the characteristics of input data by itself. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the result of the analysis, and can make judgments or predictions based on the result of the learning. In addition, technologies that mimic the functions of the human brain, such as recognition and judgment, using machine learning algorithms, can also be understood as artificial intelligence. For example, technical fields such as linguistic understanding, visual understanding, inference/prediction, knowledge expression, and motion control may be included.

Machine learning may refer to processing that trains a neural network model using experience of processing data. Through machine learning, computer software could mean improving its own data processing capabilities. A neural network model is constructed by modeling a correlation between data, and the correlation may be expressed by a plurality of parameters. The neural network model derives a correlation between data by extracting and analyzing features from given data, and optimizing the parameters of the neural network model by repeating this process can be referred to as machine learning. For example, a neural network model may learn a mapping (correlation) between an input and an output with respect to data given as an input/output pair. Alternatively, even when only input data is given, the neural network model may learn the relationship by deriving a regularity between given data.

An artificial intelligence learning model or a neural network model may be designed to implement a human brain structure on a computer, and may include a plurality of network nodes having weights while simulating neurons of a human neural network. A plurality of network nodes may have a connection relationship between them by simulating synaptic activities of neurons that transmit and receive signals through synapses. In the artificial intelligence learning model, a plurality of network nodes can send and receive data according to a convolutional connection relationship while being located in layers of different depths. The artificial intelligence learning model may be, for example, an artificial neural network model, a convolutional neural network model (CNN), and the like. As an embodiment, the artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include decision trees, Bayesian networks, support vector machines, artificial neural networks, Ada-boost, Perceptron, genetic programming, clustering, and the like.

Of these, CNNs are a type of multilayer perceptrons designed to use minimal preprocessing. A CNN consists of one or several convolution layers and general artificial neural network layers on top, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back-propagation. CNNs are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have greatly improved many computer vision tasks. With huge data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. To this end, a distributed, scalable implementation of deep neural networks can be used.

2 is a flowchart illustrating a process of receiving image data through a camera and determining a disaster through the image data when field data is received according to an embodiment.

Referring to FIG. 2, first, in step S201, the sensor 110 may determine that a disaster has been detected by the detection of the sensor and the sensor value, and if the sensor 110 determines that a disaster has been detected, the server 200 may transmit field data including detection data and location data.

Specifically, the sensor 110 may determine that a disaster has been detected when a sensor value equal to or greater than a preset set value is measured through the embedded system, and if it is determined that a disaster has been detected, the sensor 110 may generate field data including detection data including the sensor value and location data including information of a region where the sensor is located, and transmit the generated field data to the server 200.

In step S202 , when the server 200 receives field data from the sensor 110 , image data may be acquired through the camera 120 installed in the area. Here, the image data is written as image data for convenience, but may include image data.

Specifically, when the sensor 110 determines that a disaster has been detected through sensor detection by sensor values, the sensor 110 may transmit field data including detection data and location data to the server 200. When the server 200 receives the field data from the sensor 110, it may check the area where the sensor 110 is installed through the location data included in the field data. In addition, the server 200 may acquire image data through the camera 120 installed in the area where the sensor 110 identified through the location data is installed. That is, when the server 200 receives field data from the sensor 110, the server 200 can identify the region where the sensor 110 is installed through the location data, and transmit a request for image data to the camera 120 installed in the identified region.

In step S203, the server 200 may determine whether a disaster has occurred by analyzing the acquired image data.

Specifically, the server 200 may analyze image data obtained through the camera 120 to determine whether a disaster has occurred, and may include a database including existing image data, which is image data previously captured through a corresponding camera, and compare existing image data, which is image data previously captured through a camera, with image data currently captured through a camera to determine a difference. At this time, although a process of comparing image data and existing image data and extracting a difference is not specifically prepared, a difference can be extracted by comparing image data and existing image data by a commonly used method.

In addition, the server 200 may check whether the extracted difference is registered in the disaster occurrence database, and if the difference is registered in the disaster occurrence database, it may be determined that a disaster has occurred. If there is no difference or the difference is not registered in the disaster occurrence database, it may be determined that no disaster has occurred. Here, the server 200 may have a disaster occurrence database, and the disaster occurrence database may store information on differences including at least one of a difference image or a type of difference corresponding to differences included in image data taken when a disaster occurs. In this case, differences registered in the disaster occurrence database may include smoke, fire, water level rise, ground cracks, building damage, and the like.

In addition, the process of determining whether a disaster has occurred by analyzing the image data obtained by the server 200 through the camera 120 is not limited thereto, and the process of determining whether a disaster has occurred by analyzing the image data can be determined through various commonly used methods.

As a result of determining whether a disaster has occurred by analyzing the image data acquired in step S203, if it is confirmed that a disaster has occurred, in step S204, the server 200 may transmit a message requesting dispatch to the region to the terminal 310 of the safety manager matched to the region.

Specifically, when it is confirmed that a disaster has occurred by analyzing the acquired image data, the server 200 may transmit a message requesting dispatch to the region to the terminal 310 of the safety manager matched to the region. Here, the database provided in the server 200 may match and store information on a region and a safety manager who manages safety in the region, and the safety manager may include firefighters, police officers, and first responders in the region, and may be other people or facilities that manage safety in the region. In addition, the server 200 may communicate with the terminal 310 of the safety manager by wire or wireless.

As a result of determining whether a disaster has occurred by analyzing the image data acquired in step S203, if it is confirmed that no disaster has occurred, in step S205, the server 200 may transmit a message to the terminal 320 of the regional manager to check the sensor in consideration of the possibility of a sensor error.

Specifically, when it is determined that no disaster has occurred by analyzing the obtained image data, the server 200 may transmit a message requesting confirmation of the sensor to the terminal 310 of the region manager matched to the region in consideration of the possibility of error in the sensor. Here, in the database provided in the server 200, information on a region and a region manager managing the region may be matched and stored, and the region manager may include a governor, mayor, mayor, etc. of the region, and may be other people or facilities that manage the region. In addition, the server 200 may communicate with the terminal 320 of the regional manager by wire or wireless.

For this reason, when a disaster is detected primarily through a sensor and field data is received after a disaster is detected through a sensor, the server 200 secondarily detects a disaster through a camera, thereby more accurately confirming whether a disaster has occurred.

3 is a flowchart illustrating a process of transmitting a message requesting a dispatch to a region to a terminal of a safety manager matched to a region according to an embodiment.

Referring to FIG. 3 , first in step S301, the server 200 may check the type of disaster through on-site data.

Specifically, the server 200 may check the type of disaster through field data received from the sensor 110 . That is, the type of disaster may be included in the field data. When the sensor 110 is detected by the sensor value, the sensor 110 can check the type of disaster matched to the corresponding sensor, and transmit the identified disaster type to the server 200 by including it in field data.

For example, a flame detection sensor is matched with a fire among types of disasters, and field data generated from the flame detection sensor may include fire as a type of disaster. A smoke sensor may be matched with a fire among types of disasters, and field data generated from the smoke sensor may include fire as a type of disaster. Floods and typhoons may be included, and the vibration sensor is matched with an earthquake among types of disasters, and field data generated from the vibration sensor may include an earthquake as a type of disaster.

In step S302, the server 200 may determine whether there is a person on site.

Specifically, the server 200 may determine whether a person is included in the image data through image data obtained from the camera 120, and determine whether a person is present at the site based on whether a person is included in the image data. That is, if it is confirmed that a person is included in the image data, the server 200 may determine that there is a person in the scene, and if it is confirmed that a person is not included in the image data, the server 200 may determine that there is no person in the scene. Here, a method for determining whether a person is included in the image data is not specifically prepared, but it is possible to determine whether a person is included in the image data through a commonly used method.

If it is confirmed that there is a person at the site in step S302, in step S303, the server 200 may select a safety manager related to the type of disaster and lifesaving among safety managers matched to the region as candidate safety managers. At this time, in the process of storing the safety manager matched with the region stored in the database, the server 200 may match and store the safety manager and the type of disaster based on the information of the corresponding safety manager. For example, in the database, police officers, doctors, paramedics, etc. may be matched and stored with all disasters, firefighters may be matched with all disasters, but may be stored matched with fires, and water rescue teams may be matched with floods and typhoons, and mountain rescue teams may be matched with forest fires. In addition, in the process of storing the safety manager matched with the region stored in the database, based on the information of the corresponding safety manager, a safety manager performing lifesaving among the safety managers may be matched with the lifesaving and stored. For example, safety managers who rescue people, such as doctors, paramedics, and lifeguards, may be matched with and stored in the database.

If it is confirmed that there is no person at the site in step S302, in step S304, the server 200 may select a safety manager related to the type of disaster from safety managers matched to the region as candidate safety managers.

In step S305, the server 200 may generate a disaster severity level through image data and field data.

Specifically, the server 200 may increase the sensitivity of the sensor or maintain the sensitivity of the sensor through whether or not there is a person included in the image data, and may generate an emergency degree of disaster by checking a cycle in which field data is transmitted from the sensor. A detailed description of the process of generating the urgency of a disaster through image data and field data will be referred to FIG. 4 .

In step S306, the server 200 may generate the spread of the disaster through the range of the sensor that has transmitted the field data.

Specifically, the server 200 can generate a low spread of disaster when sensors that transmit field data are gathered, and can generate a high spread of disaster when sensors that transmit field data are spread out in several places. A detailed description of a process of generating a spread of a disaster through a range of a sensor that transmits field data will be described with reference to FIG. 5 .

In step S307, the server 200 may predict the scale of the disaster through the urgency of the disaster and the degree of spread of the disaster.

Specifically, the server 200 can predict the scale of the disaster through the urgency of the disaster and the spread of the disaster. At this time, the server 200 may assign a first value differently depending on whether the severity of the disaster is high or low, and may assign a second value differently depending on whether the spread of the disaster is high, normal, or low. Here, the first value given according to the emergency level of the disaster may be a preset value and may vary depending on the embodiment, and when the emergency level of the disaster is high, a higher value may be assigned as the first value than when the emergency level of the disaster is low. The second value assigned according to the degree of emergency of the disaster may be a preset value and may vary depending on the embodiment, and when the degree of spread of the disaster is high, a higher value may be assigned as the second value than when the degree of spread of the disaster is normal. For example, the server 200 may assign 5 as the first value when the severity of the disaster is high, and may assign 2 as the first value when the severity of the disaster is low. In addition, when the spread of the disaster is high, the server 200 may assign 5 as the second value, and when the spread of the disaster is normal, 3 may be assigned as the second value, and when the spread of the disaster is low, 1 may be assigned as the second value.

In addition, the server 200 may predict the scale of a disaster by adding a first value assigned according to the severity of the disaster and a second value assigned according to the spread of the disaster.

For example, if the first value corresponding to the high degree of emergency of the disaster is 5, the first value corresponding to the low severity of the disaster is 2, the second value corresponding to the high spread of the disaster is 5, the second value corresponding to the normal spread of the disaster is 3, the second value corresponding to the low spread of the disaster is 1, the urgency of the disaster obtained from image data and field data is high, and the spread of the disaster is high , The server 200 may generate a disaster scale of 10 as a value obtained by adding 5, which is a first value corresponding to a high degree of emergency, and 5, which is a second value corresponding to a high degree of spread of a disaster.

For example, if the first value corresponding to the high degree of emergency of the disaster is 5, the first value corresponding to the low severity of the disaster is 2, the second value corresponding to the high spread of the disaster is 5, the second value corresponding to the normal spread of the disaster is 3, the second value corresponding to the low spread of the disaster is 1, the urgency of the disaster obtained from image data and field data is low, and the spread of the disaster is low , The server 200 may generate a disaster scale of 3 as a value obtained by adding 2, which is a first value corresponding to a low severity of a disaster, and 1, which is a second value corresponding to a low spread of a disaster.

That is, the server 200 may generate the largest scale of disaster when the urgency of the disaster is high and the spread of the disaster is high, and when the urgency of the disaster is low and the spread of the disaster is low, it can generate the smallest scale of the disaster.

In step S308, the server 200 may select a final safety manager by extracting the number of safety managers corresponding to the scale of the disaster from among the candidate safety managers.

In detail, the server 200 may extract candidate safety managers from safety managers matched to the region through the type of disaster, and select a final safety manager by extracting the number of safety managers corresponding to the size of the disaster from among the extracted candidate safety managers.

In step S309, the server 200 may transmit a message requesting a local dispatch to the terminal of the final safety manager.

For this reason, the server 200 selects an appropriate safety manager through the type and scale of the disaster, selects the number of safety managers, and requests dispatch to the area, thereby requesting dispatch to the safety manager and reducing the time required to arrive to the area, thereby minimizing damage caused by the disaster.

4 is a diagram for explaining a process of generating an emergency degree of disaster according to an embodiment.

Referring to FIG. 4 , first, if it is confirmed in step S302 that there is a person in the field, in step S401, the server 200 may increase the sensitivity of the sensor if it is determined that a person is included in the image data.

Specifically, the server 200 may determine whether a person is included in the image data through image data, and if it is determined through the image data that a person is included, the server 200 may increase the sensitivity of the sensor. At this time, the sensor 110 may transmit field data based on a narrower unit through increased sensitivity.

For example, in the case of a gas sensor, when the sensor enters a meaningful detection section through PPM data or section data, field data including measured and measured sensor values can be measured in a certain unit from the point of entry to the server 200. If the server 200 determines that a person is included in the image data, the sensitivity of the sensor can be increased. It is possible to measure what was being transmitted in units of 10PPM and to transmit the generated field data.

When it is confirmed in step S302 that there is no person in the scene, in step S402, the server 200 may maintain the sensitivity of the sensor when it is determined that no person is included in the image data.

Specifically, the server 200 may determine whether a person is included in the image data through image data, and if it is determined through the image data that a person is not included, the server 200 may maintain sensitivity of the sensor.

For example, in the case of a gas sensor, when the sensor enters a meaningful detection range through PPM data or interval data, field data including measured and measured sensor values can be measured and transmitted to the server 200 in a certain unit from the point of entry. You can keep what you do.

In step S403, the server 200 may check a period in which field data is transmitted from the sensor.

Specifically, the server 200 may check a period in which field data is transmitted from the sensor through a time point at which field data is received from the sensor.

In step S404, the server 200 may determine whether a period in which field data is transmitted from the sensor is shorter than a reference period. Here, the reference period is a preset period and may vary according to embodiments.

In step S404, if it is determined that the period in which field data is transmitted from the sensor is shorter than the reference period, in step S405, the server 200 may generate a high level of emergency.

Specifically, the server 200 may check a period in which field data is transmitted from a sensor through a time point at which field data is received from a sensor, and determine whether a period in which field data is transmitted from a sensor is shorter than a reference period. In addition, when the server 200 determines that the period in which field data is transmitted from the sensor is shorter than the reference period, the server 200 determines that the value measured by the sensor, that is, the sensor value increases rapidly, and generates a high level of emergency.

For example, if the field data transmission period from the sensor is 1 minute and the reference period is 2 minutes, the server 200 can confirm that the field data transmission period from the sensor, 1 minute, is shorter than the reference period of 2 minutes.

In step S404, if it is determined that the period in which field data is transmitted from the sensor is not shorter than the reference period, in step S406, the server 200 may generate a disaster emergency as low.

Specifically, the server 200 may check a period in which field data is transmitted from a sensor through a time point at which field data is received from a sensor, and determine whether a period in which field data is transmitted from a sensor is shorter than a reference period. In addition, if the server 200 determines that the period at which field data is transmitted from the sensor is not shorter than the reference period, the value measured by the sensor, that is, the sensor value, increases slowly or does not increase, and determines the severity of the disaster as low. Can be generated.

For example, if the field data transmission period from the sensor is 5 minutes and the reference period is 2 minutes, the server 200 can confirm that the field data transmission period from the sensor, 5 minutes, is longer than the reference period of 2 minutes.

For this reason, the server 200 may increase the sensitivity of the sensor when it is determined that a person is included in the site, and may generate a disaster emergency based on a cycle in which field data is transmitted from the sensor. When the disaster grows rapidly, the disaster emergency can be created as high, and when the disaster grows slowly, the disaster emergency can be created as low.

5 is a flowchart illustrating a process of generating a spread of a disaster according to an embodiment.

Referring to FIG. 5 , first, in step S501 , the server 200 may determine whether all sensors that transmit field data are included within a first range. Here, the first range is a preset range and may vary according to embodiments.

Specifically, when it is confirmed that a disaster has occurred in an area, the server 200 checks location data of sensors that have transmitted field data among sensors included in the area, and determines whether all sensors that have transmitted field data are included within the first range.

When it is confirmed in step S501 that all sensors that have transmitted field data are included within the first range, in step S502, the server 200 may generate a low spread of the disaster.

Specifically, when it is confirmed that a disaster has occurred in an area, the server 200 checks location data of sensors included in the area that have transmitted field data to determine whether all sensors that have transmitted field data are included within the first range. If it is confirmed that all sensors that have transmitted field data are included within the first range, the server 200 may generate a low spread of the disaster.

For example, if it is confirmed that a disaster has occurred in the first area, and it is confirmed that the first, second, third, fourth, fifth, sixth, and seventh sensors among sensors included in the first area transmit field data, and the first sensor, second sensor, third sensor, fourth sensor, fifth sensor, sixth sensor, and seventh sensor are all included within the first range, the server 200 determines that all of the sensors included in the first area that have transmitted the field data fall within the first range. By confirming that it is within the range, the spread of the disaster can be created as low.

If it is determined that at least one of the sensors that have transmitted the field data is not included within the first range in step S501, the server 200 may determine whether all of the sensors that have transmitted the field data are included within the second range in step S503. Here, the second range includes the first range and may vary according to embodiments as a preset range wider than the first range.

Specifically, when it is confirmed that a disaster has occurred in an area, the server 200 may determine whether all sensors that transmit field data are included within a first range by checking location data of sensors included in the area that transmit field data, and if it is determined that at least one of the sensors that transmit field data is not included within the first range, it may determine whether all sensors that transmit field data are included within a second range wider than the first range.

For example, if it is confirmed that a disaster has occurred in the first area, and it is confirmed that the first sensor, the second sensor, the third sensor, the fourth sensor, the fifth sensor, the sixth sensor, and the seventh sensor among the sensors included in the first area transmit field data, the first sensor, the second sensor, and the third sensor are included within the first range, and the fourth sensor, the fifth sensor, the sixth sensor, and the seventh sensor are not included within the first range, the server 200 transmits field data to four sensors. By confirming that is not included within the first range, it is possible to determine whether all sensors that have transmitted field data are included in the second range wider than the first range.

When it is determined in step S503 that all sensors that have transmitted field data are included within the second range, in step S504, the server 200 may generate a normal spread of the disaster.

Specifically, when it is confirmed that a disaster has occurred in an area, the server 200 may determine whether all sensors that transmit field data are included within the first range by checking location data of sensors included in the area that transmit field data, and if it is determined that at least one sensor that transmits field data is not included within the first range, it may determine whether all sensors that transmit field data are included within a second range wider than the first range, and if it is determined that all sensors that transmit field data are included within the second range, , can generate normal spread of disaster.

For example, when it is confirmed that a disaster has occurred in the first region, and among the sensors included in the first region, the first sensor, the second sensor, the third sensor, the fourth sensor, the fifth sensor, the sixth sensor, and the seventh sensor transmit field data, the first sensor, the second sensor, and the third sensor are included within the first range, and the fourth sensor, fifth sensor, sixth sensor, and seventh sensor are not included within the first range but are included in the second range. Although four of the sensors are not included within the first range, it is confirmed that the four sensors are included in the second range, so that the spread of the disaster may be normal.

If it is determined in step S503 that at least one of the sensors that have transmitted field data is not included within the second range, in step S505, the server 200 may generate a high degree of spread of the disaster.

Specifically, if it is confirmed that a disaster has occurred in the area, the server 200 may determine whether all sensors that have transmitted the field data are included within the first range by checking location data of sensors included in the area that have transmitted field data, and if it is determined that at least one or more of the sensors that have transmitted the field data are not included within the first range, it may determine whether all of the sensors that have transmitted the field data are included within the second range wider than the first range, and at least one of the sensors that have transmitted the field data is not included within the second range. If it is confirmed that it is not, the spread of the disaster can be created as high.

For example, if it is confirmed that a disaster has occurred in the first region, and among the sensors included in the first region, the first sensor, the second sensor, the third sensor, the fourth sensor, the fifth sensor, the sixth sensor, and the seventh sensor transmit field data, the first sensor, the second sensor, and the third sensor are included within the first range, the fourth sensor and the fifth sensor are not included within the first range but are included within the second range, and the sixth sensor and the seventh sensor are not included within the second range. 0) confirms that 4 of the sensors that transmitted field data are not included within the first range, and 2 of the 4 sensors are not included within the second range, so that the spread of the disaster is high. Can be generated.

For this reason, the server 200 may check location data of a sensor that has transmitted field data among sensors included in the region to determine whether a disaster has spread through the range of the sensor that has transmitted field data.

6 is a flowchart illustrating a process of predicting occurrence of a disaster according to an embodiment.

Referring to FIG. 6 , first, in step S601 , the server 200 may generate sensor value change information based on field data obtained for a preset time before a disaster occurs. Here, the preset time may be from the time when a meaningful reference value capable of causing a disaster is measured to the time when it is confirmed that a disaster has occurred. In this case, a meaningful reference value may be different according to an embodiment, and the reference value may be changed according to a disaster occurrence pattern generated through an artificial neural network.

Specifically, the server 200 may generate sensor value change information based on field data obtained through the sensor 110 for a preset time from the time when a meaningful reference value at which a disaster may occur is measured to the time when it is confirmed that a disaster has occurred.

For example, when a meaningful reference value that can cause a disaster is measured at a first time point and it is confirmed that a disaster has occurred at a second time point, the server 200 may check field data acquired during the time from the first time point to the second time point, and generate sensor value change information based on the field data obtained during the time from the first time point to the second time point.

In step S602, the server 200 may generate image change information based on image data captured for a preset time before a disaster occurs.

Specifically, the server 200 may generate sensor value change information based on field data acquired through the sensor 110 for a preset time, from the time when a meaningful reference value at which a disaster may occur is measured to the time when a disaster is confirmed to have occurred, and may generate image change information based on image data captured through the camera 120 for a preset time from the time when a meaningful reference value at which a disaster may occur is measured to the time when it is confirmed that a disaster has occurred. In this case, the camera 120 may continuously capture image data at preset intervals, and may store the captured images in a camera database. In addition, the server 200 may obtain stored image data from the camera database by communicating with the camera and the camera database through wired/wireless communication.

For example, when a meaningful reference value at which a disaster may occur is measured at a first time point and it is confirmed that a disaster occurs at a second time point, the server 200 may obtain image data taken during the time from the first time point to the second time point from the camera database, and generate image change information based on the image data taken during the time from the first time point to the second time point.

In step S603, the server 200 may generate an input signal based on the sensor value change information and the image change information.

Specifically, the server 200 may perform a process of pre-processing the generated sensor value change information and image change information. The pre-processed sensor value change information and image change information may be used as inputs to an artificial neural network, or may be generated through normal processing to remove unnecessary information and noise.

In step S604, the server 200 may obtain an output signal by applying the input signal to the artificial neural network.

Here, the artificial neural network may be an algorithm that receives sensor value change information and image change information as inputs and outputs a disaster occurrence pattern.

That is, the artificial neural network may output a disaster occurrence pattern, which pattern is before a disaster occurs, in consideration of sensor value change information and image change information.

The learning device in which the learning of the artificial neural network is performed may be the same device as the server 200 that generates the disaster occurrence pattern using the learned artificial neural network, or may be a separate device.

The learning device may apply the input to the artificial neural network. The artificial neural network may be an artificial neural network that is learned according to reinforcement learning. The first artificial neural network may be a Q-Network, a Depp Q-Network (DQN), or a relational network (RL) structure suitable for outputting abstract inference through reinforcement learning.

The first artificial neural network learned through reinforcement learning may be updated and optimized by reflecting evaluations on various rewards. For example, in the first compensation, the compensation value may increase as a suitable disaster occurrence pattern is output according to the sensor value change information, in the second compensation, the compensation value may increase as a suitable disaster occurrence pattern is output according to the image change information, in the third compensation, the compensation value may increase as an unsuitable disaster occurrence pattern is not output according to the sensor value change information, and in the fourth compensation, the compensation value may increase as an inappropriate disaster occurrence pattern is not output according to the image change information.

The learning device may obtain an output from the artificial neural network. An output of the artificial neural network may be a disaster occurrence pattern according to sensor value change information and image change information. At this time, the artificial neural network may analyze the disaster occurrence pattern of what pattern it is before a disaster occurs in consideration of the sensor value change information and the image change information, and output the disaster occurrence pattern.

The learning device may evaluate the output of the artificial neural network and provide a reward. At this time, the evaluation of the output may be divided into first compensation, second compensation, third compensation, and fourth compensation.

Specifically, the learning device may award many first rewards if an appropriate disaster occurrence pattern is output according to the sensor value change information, award a second reward if an appropriate disaster occurrence pattern is output according to the image change information, award many third rewards if an unsuitable disaster occurrence pattern is not output according to the sensor value change information, and award a fourth reward if an inappropriate disaster occurrence pattern is not output according to the image change information.

The learning device may update the artificial neural network based on the evaluation.

Specifically, the learning device may update the artificial neural network through a process of optimizing a policy for determining actions to be taken in specific states so that the expected value of the sum of reward values is maximized in an environment in which the artificial neural network analyzes information on a disaster occurrence pattern in consideration of sensor value change information and image change information.

Meanwhile, the process of optimizing the policy may be performed through a process of estimating the maximum value of the expected value of the sum of rewards or the maximum value of the Q-function, or estimating the minimum value of the loss function of the Q-function. Estimation of the minimum value of the loss function may be performed through stochastic gradient descent (SGD) or the like. The process of optimizing the policy is not limited thereto, and various optimization algorithms used in reinforcement learning may be used.

The learning device may gradually update the artificial neural network by repeating the above-described learning process of the artificial neural network. Through this, the learning device may learn an artificial neural network that outputs a disaster occurrence pattern in consideration of sensor value change information and image change information.

That is, when the learning device outputs a disaster occurrence pattern of what pattern it is before a disaster occurs, in consideration of sensor value change information and image change information, the artificial neural network can be learned by reflecting reinforcement learning through the first compensation, the second compensation, the third compensation, and the fourth compensation and adjusting the analysis criteria.

In step S605, the server 200 may check a disaster occurrence pattern based on the output signal.

In step S606, the server 200 may compare field data obtained from the sensor 110 or image data acquired through the camera 120 with a disaster occurrence pattern to determine whether a matching pattern exists.

Specifically, the server 200 may compare the field data received from the sensor 110 with the disaster occurrence pattern to determine whether a matching pattern exists, and the server 200 may receive image data from the camera 120 at predetermined intervals to check whether a pattern matching the disaster occurrence pattern exists, and compare the received image data with the disaster occurrence pattern to determine whether a matching pattern exists.

In step S607, if it is confirmed that there is a pattern matching the disaster occurrence pattern, the server 200 tracks the change in the field data received from the sensor 110, and sends a message requesting response to disaster prevention to the terminal 330 of the user located in the corresponding area.

Specifically, the server 200 based on signals obtained from sensors or images obtained from cameras. If it is confirmed that there is a pattern matching the disaster occurrence pattern, it is possible to track changes in field data received from the sensor 110 in anticipation of a disaster, and to recognize the user's terminal 330 located in the corresponding area. From the user's terminal 330, a message requesting disaster prevention response can be transmitted. Here, the message requesting response to disaster prevention may be a message to evacuate, a message to cut off electricity including a gas valve, or other messages.

Due to this, the server 200 has an effect of predicting a disaster by grasping a disaster occurrence pattern.

7 is a flowchart illustrating a process of providing a path to a safety manager according to an embodiment.

Referring to FIG. 7 , first, in step S701, the server 200 may obtain the current location of the safety manager from the terminal 310 of the safety manager.

In step S702, the server 200 may obtain a location of the region based on the information of the region.

In detail, the region information included in the database may include the location of the region, and the server 200 may check the location of the region through the database.

In step S703, the server 200 may generate first route information through a real-time navigation database using the current location of the safety manager and the location of the area. Here, the real-time navigation database may be a real-time navigation database including a map database and real-time traffic conditions, or a real-time navigation database such as currently commercialized T-map or Kakao Navi. The server 200 may communicate with the real-time navigation database by wire or wireless.

Specifically, the server 200 may generate first route information through a real-time navigation database using the current location of the safety manager and the location of the region.

In step S704, the server 200 may generate a distribution degree of sensors located in the area where the sensor values are higher than the target value. Here, the target value is a preset value and may vary according to embodiments.

In detail, the server 200 may check the sensor values of sensors located in the area, determine the location of the sensor having a higher sensor value than the target value, and generate the degree of distribution.

In step S705, the server 200 may modify the first route information into second route information using the generated distribution level.

Specifically, if the server 200 determines that the first route information includes more than a threshold number of sensors having a sensor value greater than the target value in the first route information using the generated distribution, the server 200 may generate second route information avoiding the corresponding location.

In step S706, the server 200 may transmit second path information to the terminal 310 of the safety manager.

Specifically, if the server 200 determines that the first route information includes more than a threshold number of locations where the sensor value is greater than the target value, the server 200 may generate second route information for avoiding the corresponding location, and transmit the generated second route information to the terminal 310 of the safety manager.

Meanwhile, the server 200 may generate map data indicating locations in which sensors having a sensor value greater than a target value are greater than a threshold number, and may share the generated map data with the user's terminal 330 . Here, the user's terminal may be a terminal recognized by the server 200 included within a preset distance from a sensor that transmits field data.

For this reason, the server 200 provides the safety manager's terminal with second route information in which a sensor whose sensor value is greater than the target value avoids more than a threshold number of locations, thereby enabling the safety manager to more quickly and safely arrive at an area where a disaster has occurred and perform damage recovery.

8 is a flowchart illustrating a process of determining whether there is a risk of fire based on a thermal imaging camera according to an exemplary embodiment.

Specifically, the server 200 may communicate with a thermal imaging camera installed in a region by wire or wirelessly, control the thermal imaging camera, and receive a thermal image from the thermal imaging camera. At this time, the thermal imaging camera may be a camera that displays the temperature of a space to be captured in a predetermined color (red/yellow for a high-temperature region, purple for a low-temperature region, and blue-green for a central region) based on infrared rays. In this case, the infrared measurement/photography range may be 780 nm to 14 μm.

Next, detailed steps of determining whether there is a risk of fire based on the thermal image will be described.

Referring to FIG. 8 , first, in step S801, the server 200 may extract a first image from a first viewpoint of a first thermal image.

Specifically, the server 200 may control a first thermal image of the region to be captured through a thermal imaging camera installed in the region, receive the first thermal image in real time, capture the first image, which is an image at any one time point (first time point) of the first thermal image (video) measured according to time, and extract the image as a still image.

In step S802, the server 200 may extract a second image at a second viewpoint after a predetermined time has elapsed from the first viewpoint of the first thermal image.

Specifically, the server 200 may extract the second image at a second point of view that is later than the first point of view in the same manner as the method of extracting the first image.

In this case, the shooting range of the first thermal imaging camera may be fixed so as not to be adjusted between the first viewpoint and the second viewpoint. In other words, the first image and the second image are identically displayed in all shapes other than the color in which the temperature appears.

In step S803, the server 200 may generate a color spectrum from the second image from the lowest temperature to the highest temperature appearing in the second image.

Specifically, the server 200 may generate a color spectrum bar in which the color changes continuously from purple, which is the lowest temperature color, to red, which is the highest temperature color, based on the temperature color appearing in the second image.

The color spectrum generated as described above is shown in FIG. 9(a).

In step S804, the server 200 may classify colors from the lowest temperature to the highest temperature appearing in the second image into at least five grades based on the color spectrum.

Specifically, the server 200 may convert color spectrum bars of 'continuous' color changes into as many colors as the number of levels.

FIG. 9 (b) shows spectrum bars in the form of dividing the FIG. 9 (a) into five levels.

In step S805, the server 200 may convert the first color C1 appearing in the first image and the second image into a second color C2 according to each grade based on the rank.

Specifically, the server 200 may convert various colors such as red-week-yellow-second-blue-south-violet appearing in the first image and the second image into colors for each grade. Therefore, only a total of 5 colors (as many as the number of the ratings) may be displayed in the converted final first image and second image.

In step S806, the server 200 may count the number of first pixels in which the second colors C2 appear in the first image for each second color C2.

In step S807, the server 200 may count the number of second pixels in which the second colors C2 appear in the second image for each second color C2.

In step S808, the server 200 may count the number of third pixels that is the total number of pixels of the first image.

Specifically, the server 200 may count the number of pixels constituting the first image and the second image.

For example, if the sizes of the first image and the second image are '1080*760', the server 200 may count '1080*760 = 820,800' as the third pixel number.

Here, the first pixel number and the second pixel number may be counted as shown in Table 1 below.

Secondary Color (Rating) One 2 3 4 5 Total (number of third pixels) first number of pixels 205167 246130 246129 81932 41442 820800 Second number of pixels 123481 123013 205121 205025 164160 820800

In Table 1, a higher grade (from 1 to 5) may mean a color (red) representing a higher temperature. For example, in the example of FIG. 9 (b), purple may indicate a first grade, cyan a second grade, green a third grade, yellow a fourth grade, and red a fifth grade. In step S809, the server 200 may calculate [the number of first pixels / the number of third pixels = a first ratio] and [the number of second pixels / the number of third pixels = a second ratio] for each second color.

That is, the server 200 may calculate how much each second color occupies in each of the first image and the second image.

An example of calculating the first ratio and the second ratio according to the example of Table 1 is shown in Table 2.

Grade (Second Color) One 2 3 4 5 total first rate 0.24996 0.299866 0.299865 0.09982 0.05049 One second rate 0.15044 0.14987 0.249904 0.249787 0.2 One

As shown in Table 2, the sum of the first ratios and the sum of the second ratios each become 1. In step S810, the server 200 may extract at least one check group that is a combination of second colors C2 that differ by at least four degrees or more from among the second colors C2.

That is, the server 200 may extract a combination of second colors C2 whose grades differ by four or more grades.

In the example of Table 2, since the grades are 5 steps from 1 to 5, only one pair of grades 1 and 5 can be extracted from the inspection group.

As another example, if the ratings are 6 steps from 1 to 6, three combinations [(1, 5), (1, 6), (2, 6)], which are combinations whose grades differ by more than four grades among the combinations of 6C2 that extract two out of six, can be extracted as a check group.

In step S811, the server 200 may calculate [(first ratio - second ratio) / first ratio = first change rate] of [the third color that is the second color (C2) of the relatively low grade in the check group] for each inspection group.

That is, the server 200 may calculate the first increase/decrease rate for each combination of check groups.

For example, if [(1, 5), (1, 6), (2, 6)] is the check group, the server 200 may designate 1, 1, 2, which is the second color C2 of a relatively low grade in each of the three check groups, as the third color, and calculate the first increase/decrease rate for 1, 1, 2.

The first change rate for grade 1 may be calculated as '(0.24996 - 0.15044) / 0.24996 = 0.3981' based on Table 2.

In step S812, the server 200 may calculate [(second ratio - first ratio) / first ratio = second change rate] of [the fourth color that is the second color (C2) of a relatively high grade in the check group] for each inspection group.

Specifically, the server 200 calculates the second change rate in the same manner as the first change rate calculation process, but calculates the second change rate based on the relatively high grade of the second color C2 in each of the three inspection groups.

For example, if [(1, 5), (1, 6), (2, 6)] is the check group, the server 200 may designate 5, 6, and 6, which are the second colors (C2) of a relatively high rank in each of the three check groups, as the fourth color, and calculate the second change rate for 5, 6, and 6.

The second increase/decrease rate for grade 5 may be calculated as '(0.20000 - 0.05049) / 0.20000 = 2.961' based on Table 2.

In step S813, the server 200 may calculate [(second increase/decrease rate - first increase/decrease rate) * grade difference between the second colors C2 = first increase index] for each inspection group.

That is, the server 200 may calculate the first increase factor by reflecting the grade difference between the second colors C2 based on the calculated first increase and decrease rates and the second increase and decrease rates.

Based on Table 2, since the first change rate is 0.3981, the second change rate is 2.961, and the grade difference between the second colors C2 (the grade difference between the third color and the fourth color) is 4, the server 200 can calculate (2.961 - 0.3981) * 4 = 10.2516 as the first increase factor.

In addition, when there are several combinations of check groups, the server 200 may calculate several first increase indices.

In step S814, the server 200 may return the first incremental factor as a second incremental factor when there is one inspection group, and may return the average value of the first incremental factors of the inspection groups as the second incremental factor when there are two or more inspection groups.

That is, the server 200 may calculate the second increment factor as the final value based on the first increment factor(s).

For example, when the first incremental index is 10.2516, the server 200 calculates (returns) 10.2516 as the second incremental index, and the server 200 calculates (returns) 10.2516 and 12.4518 as the second incremental index.

In addition, the server 200 may determine that there is a risk of fire when the second increase factor is greater than or equal to a predetermined value, and accordingly, the server 200 may determine whether there is a risk of fire based on the first thermal image.

For example, the server 200 may determine that there is a risk of fire when the temperature change is rapid by designating a case where the second increase factor is 15 or more as a risk of fire.

10 is an exemplary diagram of a configuration of a server according to an embodiment.

Server 200 according to an embodiment includes a processor 210 and a memory 220. The processor 210 may include at least one server described above with reference to FIGS. 1 to 9 or may perform at least one method described above with reference to FIGS. 1 to 9 . A person or organization using the server 200 may provide services related to some or all of the methods described above with reference to FIGS. 1 to 9 .

The memory 220 may store information related to the methods described above or may store a program in which the methods described below are implemented. Memory 220 may be volatile memory or non-volatile memory.

The processor 210 may execute a program and control the server 200 . Program codes executed by the processor 210 may be stored in the memory 220 . The server 200 may be connected to an external device (eg, a personal computer or network) through an input/output device (not shown) and exchange data through wired/wireless communication.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general purpose or special purpose computers, such as, for example, a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will recognize that the processing device may include a plurality of processing elements and/or multiple types of processing elements. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired, or may independently or collectively direct a processing device. Software and/or data may be permanently or temporarily embodied in any tangible machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, to be interpreted by, or to provide instructions or data to, a processing device. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, even if the described techniques are performed in an order different from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (3)

In the image and video data processing and utilization method generated at the disaster site,
Transmitting field data including detection data and location data to a server when the sensor determines that a disaster has been detected by the detection of the sensor and the sensor value;
when the server receives field data from the sensor, identifying an area where the sensor is installed through the location data, and acquiring image data through a camera installed in the area;
determining, by the server, whether a disaster has occurred by analyzing the acquired image data;
If it is confirmed that a disaster has occurred, the server transmitting a message requesting dispatch to the region to a terminal of a safety manager matched to the region; and
When it is confirmed that no disaster has occurred, the server sending a message requesting confirmation of the sensor to a terminal of a regional manager in consideration of a possibility of error of the sensor,
If it is confirmed that a disaster has occurred, the step of transmitting a message requesting dispatch to the area to the terminal of the safety manager matched to the area,
Confirming the type of disaster through the field data;
Determining whether a person is included in the image data through the image data;
If it is confirmed that a person is included in the image data, selecting a safety manager related to the type of disaster and lifesaving among safety managers matched to the region as a candidate safety manager;
If it is confirmed that no person is included in the image, selecting a safety manager related to the type of disaster among safety managers matched to the region as a candidate safety manager;
Generating the urgency of the disaster through the image data and the field data;
Generating the spread of the disaster through the range of the sensor that transmitted the field data;
Predicting the scale of the disaster through the urgency of the disaster and the spread of the disaster;
Selecting a final safety manager by extracting the number of safety managers corresponding to the scale of the disaster from among the candidate safety managers; and
Transmitting a message requesting dispatch to the area to a terminal of the final safety manager,
Generating the urgency of the disaster through the image data and the field data,
If it is determined that a person is included in the image data, increasing the sensitivity of the sensor;
If it is determined that a person is not included in the image data, maintaining the sensitivity of the sensor;
Checking a cycle in which field data is transmitted from the sensor;
Determining whether the period is shorter than a preset reference period;
If it is determined that the period is shorter than the reference period, generating the emergency of the disaster as high;
If it is determined that the period is not shorter than the reference period, generating the emergency of the disaster as low,
Generating the spread of the disaster through the range of the sensor that transmitted the field data,
Determining whether all sensors transmitting the field data are included within a preset first range;
When it is confirmed that all the sensors transmitting the field data are included within the first range, generating a low spread of the disaster;
When it is determined that at least one of the sensors transmitting the field data is not included within the first range, determining whether all of the sensors transmitting the field data are included within a preset second range;
If it is confirmed that all sensors that have transmitted the field data are included within the second range, generating a normal spread of the disaster; and
When it is determined that at least one of the sensors transmitting the field data is not included within the second range, generating a high spread of the disaster,
How to process and utilize images and video data generated at disaster sites. delete According to claim 1,
Generating sensor value change information based on field data obtained for a preset time before a disaster occurs;
generating image change information based on image data captured during the set time period before a disaster occurs;
generating an input signal based on the sensor value change information and the image change information;
obtaining an output signal by applying the input signal to an artificial neural network;
Confirming a disaster occurrence pattern based on the output signal;
Comparing field data obtained from the sensor or image data acquired through the camera with the disaster occurrence pattern to determine whether a matching pattern exists; and
When it is confirmed that a pattern matching the disaster occurrence pattern exists, tracking a change in field data received from the sensor and transmitting a message requesting a disaster prevention response to a terminal of a user located in the corresponding area,
How to process and utilize images and video data generated at disaster sites.