KR102626476B1 - Disaster response method and system - Google Patents

Abstract

The disaster response method of the present invention includes capturing images and acquiring fire detection sensor data with a camera; Checking the connection status with the server through the network; If the network connection is good, transmitting the obtained data to the server and determining whether there is a fire in the large AI model of the server; If the network connection condition is not good, determining whether there is a fire from the acquired data in the lightweight AI model of the AI camera; When the large AI model of the server determines that a fire has occurred, taking action by a firefighting robot; And when it is determined that a fire has occurred in the lightweight AI model of the AI camera, it may include performing fire response measures according to the reliability of the lightweight AI model.

Description

Disaster response method and system {DISASTER RESPONSE METHOD AND SYSTEM}

The present invention relates to a disaster response method and system that can determine a disaster state by applying electronic detection sensors and image analysis technology and take response measures for the determined fire.

As a way to detect disasters/disasters by applying electronic communication technology, a system that detects fires and earthquakes using images of stations to be monitored and seismic waves using electronic devices has been proposed.

However, the systems proposed to date only provide specific measures for disaster/disaster detection and warning (alarm), and do not present effective measures for automatic measures against impending disasters/disasters.

Republic of Korea Registered Publication No. 10-2242421

The present invention seeks to provide a disaster response method and system that can provide effective measures for automatic measures against impending disasters/disasters.

A disaster response method according to an aspect of the present invention includes capturing images and acquiring fire detection sensor data with a camera; Checking the connection status with the server through the network; If the network connection is good, transmitting the obtained data to the server and determining whether there is a fire in the large AI model of the server; If the network connection condition is not good, determining whether there is a fire from the acquired data in the lightweight AI model of the AI camera; When the large AI model of the server determines that a fire has occurred, taking action by a firefighting robot; And when it is determined that a fire has occurred in the lightweight AI model of the AI camera, it may include performing fire response measures according to the reliability of the lightweight AI model.

Here, the step of performing fire response measures according to the reliability of the lightweight AI model includes checking the number of days elapsed after updating the lightweight AI model, and performing measures by a firefighting robot if the number of elapsed days is less than a predetermined standard number of days. ; And if the number of elapsed days is more than the predetermined standard number of days, it may include notifying the manager of the occurrence of a fire and waiting for a firefighting robot action instruction from the manager.

Here, in the step of notifying the manager of the occurrence of a fire and waiting for action instructions, the steps include performing a visual or auditory fire alarm outside the AI camera; Confirming a fire or receiving an input indicating that it is not a fire through a manual control unit of the AI camera; And when fire confirmation is input to the manual operation unit, it may include the step of taking action by the fire-fighting robot.

Here, the step of providing information about the fire point to the fire-fighting robot may be further included.

Here, the step of training a large AI model of the server with images captured by the camera for a predetermined learning time after installing the camera at the site to be monitored may be further included.

Here, in the step of learning the large AI model of the server, even if a fire is determined through the image of the camera during the learning period, if the fire detection sensor does not determine a fire, the image can be re-trained as not a fire. there is.

An AI server according to another aspect of the present invention includes a camera module that photographs a scene subject to surveillance; an image buffer that collects images captured by the camera module; A sensor value collection unit that collects measurement values from sensors that measure disaster-related parameters of the monitored site; A lightweight AI model that determines whether a fire has occurred from the captured images and measured values of the sensors; a server communication unit that transmits the captured images and measurement values of the sensors to an external AI server and downloads the lightweight AI model; And when it is determined that a fire has occurred in the lightweight AI model in a situation where communication with the AI server is poor, it may include a terminal control unit that performs fire response measures according to the reliability of the lightweight AI model.

Here, a manual operation unit that receives information according to the manual operation of the field user; And it may further include an alarm unit that outputs a determination result of a fire occurrence to an on-site user.

Here, it stores the lightweight AI model reflection settings of the field manager through the manual operation unit or the lightweight AI model reflection settings of the upper manager through the AI server, and may further include a storage unit for storing a SW image of the lightweight AI model. .

Here, the terminal control unit combines conditions allowing active fire suppression, including the time difference between the last update time of the lightweight AI model and the current time, and a time period during which active fire suppression by the lightweight AI model alone is permitted. Thus, the above fire response measures can be carried out.

Here, when the terminal control unit determines that a fire has occurred in the lightweight AI model in a situation where communication with the AI server is poor, it checks the number of days that have elapsed since the update of the lightweight AI model, and if the number of days that have elapsed is less than a predetermined standard number of days, the terminal control unit determines that a fire has occurred in the lightweight AI model. If action is taken by the robot and the elapsed number of days is more than the predetermined standard number of days, the occurrence of a fire is notified to the alarm unit, and when a fire robot action instruction is input from the site manager through the manual operation unit, firefighting at the site to be monitored is performed. The robot can be instructed to extinguish a fire.

The disaster response system according to another aspect of the present invention collects camera image data of the site to be monitored and measurement values of disaster monitoring sensors and transmits it to the following AI server, and when communication with the AI server is cut off, its own AI terminal that performs response actions; A firefighting robot capable of extinguishing a fire at the monitored site; and

An AI terminal communication unit that receives the camera images and measurement values of the sensors and downloads the following lightweight AI model, and determines whether a fire has occurred at the site to be monitored from the camera images and the measurement values of the sensors. A large AI model that operates, a lightweight AI model that uses the learned large AI model as a lightweight model template for the AI terminal, and a lightweight AI model that operates the large AI model for the AI terminal and provides a lightweight AI for the AI terminal. An AI server that has a server control unit that manages the model, a database that records information about the installation site of the AI terminal and the downloaded lightweight AI model, and performs fire suppression instructions for the fire robot and SMS alarm to the manager. may include.

Here, when communication with the AI server is cut off and the lightweight AI model downloaded from the AI server determines that a fire has occurred, the AI terminal determines that a fire has occurred, according to the reliability of the lightweight AI model or the preset of the site manager. Robots can be instructed to extinguish fires.

Here, the AI terminal includes a camera module that photographs the scene to be monitored; an image buffer that collects images captured by the camera module; A sensor value collection unit that collects measurement values from sensors that measure disaster-related parameters of the monitored site; A lightweight AI model that determines whether a fire has occurred from the captured images and measured values of the sensors; a server communication unit that transmits the captured images and measurement values of the sensors to the AI server and downloads the lightweight AI model; And it may be an AI camera including a terminal control unit that performs fire response measures according to the reliability of the lightweight AI model when it is determined that a fire has occurred in the lightweight AI model in a situation where communication with the AI server is poor.

Here, the large AI model includes an image determination unit that determines whether a fire has occurred from images captured by the camera module by applying an image classification technique; a sensor value determination unit that determines whether a fire has occurred from the measured values of the sensors; It may include an integrated determination unit that ultimately determines whether a fire has occurred by reflecting weights on each determination result of the image determination unit and the sensor value determination unit.

Here, the large AI model may further include a fall image determination unit that determines whether a fall accident has occurred from images captured by the camera module by applying an image classification technique.

Here, the sensor value determination unit performs an earthquake determination, and the integrated determination unit may make a final determination that an earthquake has occurred when the camera-captured images include a noise image component due to vibration.

Here, the server control unit may update and train the large AI model, generate the lightweight AI model as a lightweight version of the large AI model for which update learning has been completed, and transmit it to the AI terminal.

When the disaster response method and/or system according to the spirit of the present invention of the above-described configuration is implemented, CO, CO2, VOC, PM, temperature, and humidity sensor values in addition to images are additionally used to detect fires with high accuracy using machine learning. There is an advantage in being able to determine. Additionally, there is the advantage of being able to provide additional functions such as detecting not only fires but also falls and earthquakes.

The disaster response method and/or system of the present invention has the advantage of storing all records in a database and providing an SMS notification function to managers when an emergency situation occurs.

In particular, the disaster response method and/or system of the present invention has the advantage of enabling early response in the event of a fire using an autonomous robot, etc., thereby enabling early extinguishment.

1 is a flowchart showing an embodiment of a disaster response method according to the spirit of the present invention.
Figure 2 is a conceptual diagram showing an example of building the above-described disaster response system.
Figure 3 is a conceptual diagram showing another example of building the disaster response system described above.
Figure 4 is a block diagram showing an embodiment of an AI camera that supports the implementation of a disaster response method according to the spirit of the present invention.
Figure 5 is a block diagram showing an embodiment of an AI server supporting the performance of a disaster response method according to the spirit of the present invention.
FIG. 6 is a top view showing an embodiment of a camera/sensor module that may be provided in the AI camera of FIG. 4.
Figure 7 is a captured image showing how to determine the occurrence of a fire.
Figure 8 is a captured image showing the determination of the occurrence of a fall.
Figure 9 is a conceptual diagram showing an example of a convolution technique that can be applied to analyze camera captured images.
Figure 10 is a conceptual diagram showing an example of an integrated fire detection algorithm that can be applied for final fire determination according to the spirit of the present invention.
11 is a flow chart illustrating a process for performing fire response, reflecting an example of the learning process of a large AI model and/or a lightweight AI model.
Figure 12 is a conceptual diagram showing the operation process of an autonomous fire-fighting robot.

In describing the present invention, terms such as first and second may be used to describe various components, but the components may not be limited by the terms. Terms are intended only to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present invention.

When a component is mentioned as being connected or connected to another component, it can be understood that it may be directly connected to or connected to the other component, but other components may exist in between. .

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions may include plural expressions, unless the context clearly indicates otherwise.

In this specification, terms such as include or have are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, including one or more other features or numbers, It can be understood that the existence or addition possibility of steps, operations, components, parts, or combinations thereof is not excluded in advance.

Additionally, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

1 is a flowchart showing an embodiment of a disaster response method according to the spirit of the present invention.

The disaster response method applying the illustrated electronic detection sensor and image analysis technology includes the steps of capturing images and acquiring fire detection sensor data with a camera (S110); Checking the connection status with the server through the network (S120); If the network connection is good, transmitting the obtained data to the server and determining whether there is a fire in the large AI model of the server (S130); If the network connection condition is not good, determining whether there is a fire from the acquired data in the lightweight AI model of the AI camera (S140); When the large AI model of the server determines that a fire has occurred, taking action by a firefighting robot (S160); And when it is determined that a fire has occurred in the lightweight AI model of the AI camera, it may include performing fire response measures according to the reliability of the lightweight AI model (S180).

In addition, as a preemptive measure in step S160 or step S180, steps (S151, S152) of performing an alarm on site using the alarm device of the AI camera (AI terminal) may be performed.

More specifically, the step of performing fire response measures according to the reliability of the lightweight AI model (S180) is to check the number of days elapsed after updating the lightweight AI model, and if the number of days elapsed is less than a predetermined standard number of days, performing actions; And if the number of elapsed days is more than the predetermined standard number of days, it may include notifying the manager of the occurrence of a fire and waiting for a firefighting robot action instruction from the manager.

If the firefighting robot is capable of moving, the step of providing information about the fire point to the firefighting robot may be further included after step S151 and before step S160.

The large AI model of the server and the lightweight AI model of the AI camera may be models capable of machine learning using learning data suitable for the field to be monitored. In this case, before step S110, the camera is monitored. It may further include training a large AI model of the server and/or a lightweight AI model of the AI camera using images captured by the camera for a predetermined learning time after installation at the target site. However, in general, a method of training only the large AI model of the server and creating (updating) a lightweight AI model based on the large AI model that has been trained will be applied.

Meanwhile, in the step of learning the large AI model of the server, if a fire is not determined by the fire detection sensor even if a fire is determined through the image (video) of the camera during a predetermined learning period, the image is considered not a fire. It can be trained by applying learning data to make a decision.

In order to perform the disaster response method according to the idea of the present invention shown in FIG. 1, the sensing values and images that serve as the basis for fire (disaster) determination for the site to be monitored are collected, and the site that runs the lightweight AI model is used. AI terminal; AI server that manages and operates the large AI model; It is required to build a disaster response system consisting of a firefighting robot that performs firefighting measures for fires that occur at a monitored site.

Figure 2 is a conceptual diagram showing an example of building the above-described disaster response system.

The disaster response system shown is an integrated field AI terminal in the form of a camera & sensor terminal, including an AI camera (100); Self-driving firefighting robot (400) capable of moving to the fire site; and an AI server 200 that provides dispatch instructions for the self-driving firefighting robot 400 and sends an SMS alarm to the manager.

The illustrated AI camera 100 collects camera captured images (videos) and sensor data (sensing values) and transmits them to the AI server 200, and when communication with the AI server 200 is cut off, carry out appropriate countermeasures.

The AI camera 100 may instruct the self-driving firefighting robot 400 to dispatch or operate fire extinguishing equipment at the point where a fire occurs, depending on the reliability of the internal lightweight AI model and/or the preset settings of the site manager.

The illustrated AI server 200 can collect and manage data from remote client facilities (AI cameras, firefighting robots) and perform monitoring for disaster detection/response.

For example, the AI server 200 can perform fire extinguishing commands and administrator SMS notifications through fire object detection.

To this end, the AI server 200 can configure a DB for managing image data, sensor data, etc. In addition, it is possible to configure a DB for managing the placement/specification information of the AI camera and firefighting robot, and a DB for managing lightweight AI models for each site.

For example, the self-driving firefighting robot 400 shown receives fire location information from the AI server 200, performs autonomous driving while avoiding obstacles to the fire location according to the location information, and tilts the fire extinguisher at the arrived fire location. Aiming at the target point and spraying a fire extinguisher can be performed through control.

Figure 3 is a conceptual diagram showing another example of building the disaster response system described above.

The AI camera 100 according to the illustrated system construction example includes an AI control unit 140 as a set of HW modules and SW modules for executing a lightweight AI model; and a camera/sensor module 110 in which a camera and fire detection sensors are integrated and mounted in a form that can be embedded inside the case of the AI camera device. In addition, as a firefighting robot, a remotely controllable fire extinguisher (401) installed at key points in the monitored site was applied.

In the drawing, the logo of AI SW is expressed as the SW module, and a photo of the main controller and an acceleration sensor that can be installed adjacent to the main controller are expressed as the HW module. For example, the main controller and acceleration sensor on the main board in the AI camera and sensor terminal of the invention may be configured as shown in the photo attached to FIG. 3.

The controller has a CPU and GPU capable of AI inference in a local environment, and GPIOs, I2C, SPI, UART, and I2S functions may be required to install additional components.

The main board requires MIPI and USB slots capable of mounting a camera, and it is advantageous to enable communication with a server. The illustrated acceleration sensor can be connected through the GPIO of the main controller.

Figure 4 is a block diagram showing an embodiment of an AI camera that supports the implementation of a disaster response method according to the spirit of the present invention.

A camera module 110 that photographs the scene to be monitored; an image buffer 120 that collects images captured by the camera module 110; A sensor value collection unit 130 that collects measurement values from sensors that measure disaster-related parameters of a monitored site; A lightweight AI model 150 that determines whether a fire has occurred from the captured images and measured values of the sensors; A server communication unit 160 that transmits the captured images and measurement values of the sensors to an external AI server and downloads the lightweight AI model; A manual operation unit 170 that receives information according to manual operation by an on-site user; An alarm unit 180 that outputs a determination result of a fire occurrence to field users; When it is determined that a fire has occurred in the lightweight AI model, it may include a terminal control unit 140 that performs fire response measures according to the reliability of the lightweight AI model.

The AI camera 100 includes the camera module 110, an image buffer 120, a sensor value collection unit 130, a lightweight AI model 150, a server communication unit 160, and a manual operation unit 170 inside a predetermined case. ), the alarm unit 180, and the terminal control unit 140 may be implemented in a built-in form.

The manual operation unit 170 may be implemented as a button, a touch screen, etc. on the outer surface of the case through which an external person can input an input operation. In addition, a means for verifying the identity of the on-site manager (e.g., fingerprint reader, keypad for password entry) may be further provided.

Most of the sensors that generate the sensing values collected by the sensor value collection unit 130 are installed separately outside the AI camera 100, but depending on the implementation, among the sensors, an acceleration sensor that is relatively inexpensive and has a small volume, The temperature sensor may be integrated with the AI camera 100 and installed inside the case.

The sensor value collection unit 130 includes connection terminals for connecting signal lines of sensors separately installed externally; and sensing value acquisition circuits that obtain sensing values from signals input to each connection terminal.

Depending on implementation, it may further include a storage unit for temporarily storing the captured images and measurement values of the sensors before transmitting them to the external AI server.

In this case, the SW image of the lightweight AI model may also be stored in the storage unit.

Alternatively, the on-site manager's lightweight AI model reflection settings through the manual operation unit 170 or the upper manager's lightweight AI model reflection settings through the AI server may be stored in the storage.

The lightweight AI model reflection setting is for cases where communication with the AI server is poor and it is difficult for an on-site manager to intervene through the manual operation unit 170. That is, as a setting reflecting the lightweight AI model, conditions that allow active fire suppression can be specified only through the self-determination of the lightweight AI model 150,

For example, conditions allowing active fire suppression include the time difference (date) between the last update time of the lightweight AI model 150 and the current time, and the time zone in which active fire suppression by the lightweight AI model 150 alone is allowed. (e.g. time zone when the manager is not working), etc., and each condition can be applied in combination with and/or with each other.

For example, when applying only the time difference between the last update time of the lightweight AI model 150 and the current time, the terminal control unit 140 performs fire response measures according to the reliability of the lightweight AI model in FIG. 1. In the step (S180), the number of elapsed days after the update of the lightweight AI model 150 is checked, and if the number of elapsed days is less than the predetermined standard number of days, action is taken by the firefighting robot. Conversely, the elapsed number of days is less than the predetermined standard number of days. If this is the case, the occurrence of a fire is notified to the alarm unit 180, and upon receiving firefighting robot action instructions from the site manager through the manual operation unit 170, fire suppression is instructed to the firefighting robot at the site to be monitored. It can be done with

When defining the operation of the AI camera 100 as an integrated AI terminal for disaster response that collects image data and sensor value data, data is acquired through the camera and various sensor modules, and after the acquisition process, whether the network is connected or not. Confirm, and after the network confirmation process, determine whether there is a disaster (e.g., a fire) from the obtained data (receiving disaster status from the AI server when connected to the network, determine the terminal itself when not connected to the network), and determine the disaster. After the process, it can be divided into finally sounding an alarm in case of a disaster and warning the site.

When a fire (disaster) is determined by the large AI model or the lightweight AI model 150, the site manager or other users may be notified by visual and/or auditory means through the alarm unit 180.

A nearby site manager or other user who has been notified of this can operate the manual control unit 170 and input his/her judgment result regarding whether a fire has occurred at the site. During active fire suppression using a fire robot, if it is input that it is not a fire through the manual control unit 170, active fire suppression can be stopped after a predetermined confirmation procedure. On the other hand, when a fire robot is not yet deployed due to a fire determination by the lightweight AI model 150, and confirmation of a fire occurrence is input to the manual operation unit 170, the fire robot can be deployed to actively suppress the fire. there is.

Figure 5 is a block diagram showing an embodiment of an AI server that supports the implementation of a disaster response method according to the spirit of the present invention.

An AI camera communication unit 260 that receives the captured images and measurement values of the sensors from the AI camera and downloads the lightweight AI model; A large AI model 250 that determines whether a fire has occurred from the captured images and measured values of the sensors; A lightweight AI model (290) as a lightweight model template made by lightweighting the learned large AI model (250) for the AI camera; A server control unit 240 that operates the large AI model 250 for the AI camera and manages the lightweight AI model 290 for the AI camera; And it may include a database 270 that records information about the installation site of the AI camera and the downloaded lightweight AI model.

The AI camera communication unit 260, together with the server communication unit 160 of the AI camera, may be implemented as a wireless communication module for forming a data communication channel according to technologies such as LTE communication, 5G communication, and wireless Internet communication.

Depending on implementation, training data for the AI model, learning history data of the large AI model 250, hardware specification information of the AI camera, etc. may be further stored in the database 370.

Depending on the implementation, the server control unit 240 uses images captured by the camera for a predetermined learning time (e.g., pre-learning time, update learning time) after installing the camera at the site to be monitored. You can train a model.

For example, the server control unit 240 may update and train the large AI model 250 periodically or according to instructions from a higher level manager. At this time, the lightweight AI model 290 can be created as a lightweight version of the large AI model 250 for which each update learning has been completed.

The server control unit 240 reflects the lightweight AI model 290 as a lightweight model template according to the installation site of the AI camera and/or the computational hardware specifications of the AI camera, and creates a lightweight AI model for each AI camera. You can create and transmit it to the corresponding AI camera.

The large AI model 250 shown includes an image determination unit 252 that determines whether a fire has occurred from images captured by a camera module by applying an image classification technique; a sensor value determination unit 254 that determines whether a fire has occurred from the measured values of the sensors; And it may include an integrated determination unit 256 that ultimately determines whether a fire has occurred by reflecting weights on each determination result of the image determination unit 252 and the sensor value determination unit 254.

In the case of an implementation that detects a person's fall as an additional disaster, the large AI model 250 further includes a fall image determination unit that determines whether a fall accident has occurred from the images captured by the camera module by applying an image classification technique. can do.

Specific examples of fall accident determination will be described later.

In the case of an implementation that detects the occurrence of an earthquake as an additional disaster, the sensor value determination unit 254 performs earthquake determination from measurement values of related sensors such as an acceleration sensor or a seismic wave sensor, and the integrated determination unit 256, If the camera images taken at the time of the earthquake determined by the sensor value determination unit 254 include noise image components due to vibration, a final determination can be made that an earthquake has occurred.

We will illustrate the process of determining an earthquake using the values of the 3-axis acceleration sensor mounted on the main controller.

As shown in Equation 1 below, an earthquake detection standard is set by measuring the additional acceleration based on the acceleration value measured in a stationary state.

As shown in Equation 2 below, a simple moving average value is applied to the sensor value to filter the noisy signal.

As shown in Equation 3 below, if the change in acceleration is greater than the reference value for a certain period of time, it is detected as an earthquake.

It is a central AI server that analyzes image data and sensor value data to determine the occurrence of a disaster and takes response measures. When defining the operation of the AI server 200, AI camera and sensor module data are received through a network, After the data reception process, the presence of a disaster is determined through the large AI model 250, the sensor measurements and image data are converted into binary form and stored in the database 270, and AI camera alarms and alarms are generated according to the disaster determination results. A sensor terminal alarm is transmitted, and after the alarm instruction transmission process, fire location coordinates and extinguishing (fire suppression) commands are transmitted to the firefighting robot according to the disaster judgment result. Afterwards, an alarm instruction can be transmitted to alert again that “a fire has occurred and the firefighting robot is extinguishing the fire.”

Next, an example of an operation plan for the database 270 will be provided.

For example, when detecting a fire or earthquake, images and sensor data can be stored in the database 270. The database type can use NoSQL to store unstructured data.

Additionally, image data can be stored by storing time, detection location, detection temperature, and image and encoded in binary form to reduce image size. Sensor data can store CO2, VOC, relative humidity, outside temperature, and PM.

FIG. 6 is a top view illustrating an embodiment of the camera/sensor module 110 that may be provided in the AI camera of FIG. 4.

For example, in the AI camera and sensor terminal of the present invention, an integrated module in the form of a camera & sensor may be composed of a real camera, a thermal camera, and a sensor module arranged as shown in FIG. 6.

Images acquired with a real-world camera can be used to determine fire by using it as input data for a deep learning model for fire detection. In addition, data acquired with a thermal camera can be used to determine fire by calculating the temperature value for each pixel of the image acquired with a real camera.

Sensor values of CO2, VOC, relative humidity, temperature, and PM measured by the sensor module can be used as input data for large/light AI models to determine fire.

Figure 7 is a captured image showing how to determine the occurrence of a fire.

Figure 8 is a captured image showing the determination of the occurrence of a fall accident.

Figures 7 and 8 show how to distinguish between a fire and a fall accident using a deep learning object detection model that uses a real camera image as input data. For example, a fire/fall accident can be determined through the following processes.

First, the size of the input image is fixed, the object location is found in each feature map extracted through four or more stages of convolution operation, and the object class is determined.

Define a loss function for object location and object class discrimination and create a fire discrimination model and/or a fall accident discrimination model through iterative calculations using gradient descent to minimize the loss function.

Considering the specifications of the main controller, the convolution operation is calculated using the Depthwise Separable Convolution technique.

In particular, the large AI model, which is run on a server with high computing hardware specifications, can be equipped with various additional functions in addition to the fire and fall accident determination based on image analysis, and is located on the server, so it can simply run the large AI model. It is easy to add additional functions during operation simply by updating/replacing.

Figure 9 is a conceptual diagram illustrating an example of a convolution technique that can be applied to analyze camera captured images.

The structure shown is a calculation method that combines Depthwise Convolution and Pointwise Convolution to reduce the amount of calculation of general convolution, and represents the Depthwise Saperable Convolution technique.

Figure 10 is a conceptual diagram showing an example of an integrated fire detection algorithm that can be applied for final fire determination according to the spirit of the present invention. In other words, it illustrates an integrated fire detection algorithm using a fire detection model and a sensor detection model that can be operated on the large AI model in Figure 5.

As shown, the results of the sensor model (which can be implemented as a support vector machine (SVM)) and the fire model can be redefined as input to the integrated model. The final fire determination function is performed by applying weights to the feature values of the sensor model (e.g., the sensor value determination unit 254 of FIG. 5) and the fire model (e.g., the image determination unit 252 of FIG. 5) by the integrated model. can do. Rather than a simple ensemble model that votes on the results of the SVM model and the fire detection model, accuracy can be improved by passing the two results through an integrated model (e.g., the integrated decision unit 256 in FIG. 5) once more.

Figure 11 is a flowchart showing a process for performing fire response, reflecting an example of a learning process of a large AI model and/or a lightweight AI model. The flowchart shown is intended to explain the additional learning algorithm of the fire model for misdetection of fire images.

This is an algorithm to prevent misdetection of a fire model using images for a given environment. In particular, it seeks to prevent misdetection by judging the background as a fire rather than a fire. After initial installation, fire objects detected in the fire model are verified using sensors, which reflects that the reliability of sensor values is higher in the early stages of installation.

After initial installation, the fire model is strengthened using the algorithm below using the data set collected during a designated random time (t), and then adjustments are made to increase the reliability of the fire model using images. The reliability may be reflected in the integrated decision unit 256 of FIG. 5.

Figure 12 is a conceptual diagram showing the operation process of an autonomous fire-fighting robot.

The operation of the self-driving firefighting robot 400 of the present invention includes the steps of acquiring fire status data from an AI server; After the acquisition step, a control step of autonomously driving and moving to the fire detection location; After the moving step, detecting the location of the fire and aiming a fire extinguisher; And after the aiming step, it may be comprised of a control step of finally spraying the extinguishing agent.

Those skilled in the art to which the present invention pertains should understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features, and that the embodiments described above are illustrative in all respects and not restrictive. Just do it. The scope of the present invention is indicated by the claims described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

Those skilled in the art to which the present invention pertains should understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features, and that the embodiments described above are illustrative in all respects and not restrictive. Just do it. The scope of the present invention is indicated by the claims described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

100: AI camera 110: Camera module
120: image buffer 130: sensor value collection unit
140: terminal control unit 150: lightweight AI model
160: Server communication unit 170: Manual control unit
180: Alarm unit 200: AI server
240: Server control unit 250: Large AI model
252: image determination unit 254: sensor value determination unit
256: Integrated decision unit 260: AI camera communication unit
270: Database 290: Lightweight AI model
400: Firefighting robot

Claims (18)

Taking images and acquiring fire detection sensor data with a camera;
Checking the connection status with the server through the network;
If the network connection is good, transmitting the obtained data to the server and determining whether there is a fire in the large AI model of the server;
If the network connection condition is not good, determining whether there is a fire from the acquired data in the lightweight AI model of the AI camera;
When the large AI model of the server determines that a fire has occurred, taking action by a firefighting robot; and
When the lightweight AI model of the AI camera determines that a fire has occurred, performing fire response measures according to the reliability of the lightweight AI model.
Including,
The step of performing fire response measures according to the reliability of the lightweight AI model is,
Checking the number of days elapsed after updating the lightweight AI model, and taking action by a firefighting robot if the number of days elapsed is less than a predetermined standard number of days; and
If the number of elapsed days is more than the predetermined standard number of days, notifying the manager of the occurrence of a fire and waiting for instructions for firefighting robot action from the manager.
Disaster response methods including.
delete According to paragraph 1,
In the step of notifying the manager of a fire and waiting for action instructions,
Performing a visual or auditory fire alarm external to the AI camera;
Confirming a fire or receiving an input indicating that it is not a fire through a manual control unit of the AI camera; and
When fire confirmation is input to the manual operation unit, taking action by the fire robot
Disaster response methods including.
According to paragraph 1,
Step of providing information about the fire point with the fire robot
Disaster response method further comprising:
According to paragraph 1,
A step of training a large AI model of the server with images captured by the camera for a predetermined learning time after installing the camera at the surveillance target site.
Disaster response method further comprising:
According to clause 5,
In the step of learning the large AI model of the server,
A disaster response method for re-learning the image as not a fire if the fire detection sensor does not determine a fire even if a fire is determined through the image of the camera during the learning period.
Camera module that takes pictures of the scene to be monitored;
an image buffer that collects images captured by the camera module;
A sensor value collection unit that collects measurement values from sensors that measure disaster-related parameters of the monitored site;
A lightweight AI model that determines whether a fire has occurred from the captured images and measured values of the sensors;
a server communication unit that transmits the captured images and measurement values of the sensors to an external AI server and downloads the lightweight AI model; and
If it is determined that a fire has occurred in the lightweight AI model in a situation where communication with the AI server is poor, a terminal control unit that performs fire response measures according to the reliability of the lightweight AI model.
Including,
The terminal control unit,
The fire response measures are performed by combining conditions allowing active fire suppression, including the time difference between the last update of the lightweight AI model and the current time, and the time period during which active fire suppression by the lightweight AI model alone is permitted. AI camera that works.
In clause 7,
A manual control unit that receives information based on manual operation by field users; and
Alarm unit that outputs the judgment result of fire occurrence to field users
An AI camera further comprising:
According to clause 8,
Saves the on-site manager's lightweight AI model reflection settings through the manual operation unit or the upper manager's lightweight AI model reflection settings through the AI server,
A storage unit that stores the SW image of the lightweight AI model
An AI camera further comprising:
delete According to clause 8,
The terminal control unit,
If the lightweight AI model determines that a fire has occurred in a situation where communication with the AI server is poor,
Check the number of days that have passed since the update of the lightweight AI model,
If the number of elapsed days is less than the predetermined standard number of days, action is taken by the firefighting robot,
If the number of days elapsed is more than the predetermined standard number of days, the AI notifies the alarm unit of the occurrence of a fire, and when a fire robot action instruction is input from the site manager through the manual operation unit, it instructs the fire robot at the site to be monitored to suppress the fire. camera.
An AI terminal that collects camera image data and measurements from disaster monitoring sensors for the site to be monitored and transmits them to the following AI server, and performs its own response measures when communication with the AI server is cut off;
A firefighting robot capable of extinguishing a fire at the monitored site; and
An AI terminal communication unit that receives the camera images and measurement values of the sensors and downloads the following lightweight AI model;
A large AI model that determines whether a fire has occurred at the site to be monitored based on the camera images and measured values of the sensors;
A lightweight AI model as a lightweight model template in which the learned large AI model is lightened for the AI terminal;
a server control unit that operates the large AI model for the AI terminal and manages a lightweight AI model for the AI terminal;
Equipped with a database that records information about the installation site of the AI terminal and the downloaded lightweight AI model,
AI server that performs fire suppression instructions for the fire robot and SMS alarm to the manager
Disaster response system including.
According to clause 12,
The AI terminal is,
When communication with the AI server is interrupted and the lightweight AI model downloaded from the AI server determines that a fire has occurred,
A disaster response system that instructs the firefighting robot to extinguish a fire, depending on the reliability of the lightweight AI model or the preset settings of the site manager.
According to clause 12,
The AI terminal is,
A camera module that photographs the surveillance target site;
an image buffer that collects images captured by the camera module;
A sensor value collection unit that collects measurement values from sensors that measure disaster-related parameters of the monitored site;
A lightweight AI model that determines whether a fire has occurred from the captured images and measured values of the sensors;
a server communication unit that transmits the captured images and measurement values of the sensors to the AI server and downloads the lightweight AI model; and
A disaster response system characterized in that it is an AI camera including a terminal control unit that performs fire response measures according to the reliability of the lightweight AI model when the lightweight AI model determines that a fire has occurred in a situation where communication with the AI server is poor.
According to clause 12,
The large AI model is,
an image determination unit that determines whether a fire has occurred from images captured by the camera by applying an image classification technique;
a sensor value determination unit that determines whether a fire has occurred from the measured values of the sensors;
An integrated decision unit that ultimately determines whether a fire has occurred by reflecting weights on each decision result of the image determination unit and the sensor value determination unit.
Disaster response system including.
According to clause 15,
The large AI model is,
A fall image determination unit that determines whether a fall accident has occurred from the images captured by the camera by applying an image classification technique.
A disaster response system further comprising:
According to clause 15,
The sensor value determination unit,
Perform seismic judgment,
The integrated decision department,
If the camera captured images contain noise image components due to vibration,
A disaster response system that makes the final determination that an earthquake has occurred.
According to clause 12,
The server control unit,
A disaster response system that updates and trains the large AI model, creates the lightweight AI model as a lightweight version of the large AI model for which update learning has been completed, and transmits it to the AI terminal.

Images