KR102265291B1 - Real time fire detection system and fire detection method using the same - Google Patents

Abstract

The present invention relates to a fire detection system and a fire detection method capable of detecting a fire by analyzing a visible light image and a thermal image image by tying one subject rather than an outline, and relates to a first image captured by a visible light camera and a heat A photographing apparatus comprising: a photographing apparatus for acquiring a second image photographed by an image camera; and a control unit for receiving the first and second images acquired by the photographing apparatus through communication with the photographing apparatus and controlling the photographing apparatus; And a real-time fire detection system and a fire detection method using the same, comprising an image analysis unit that analyzes the first image and the second image through a fusion method that binds one subject, not the outline, to check whether a fire has occurred, and a flying drone And by using a real-time fire detection device and fire detection method that applies a fusion method that binds one subject rather than an outline using the visible light image and thermal image captured by CCTV, the effect of reducing damage from fire can indicate

Description

Real time fire detection system and fire detection method using the same}

The present invention relates to a real-time fire detection system and a fire detection method, and more particularly, a fire detection system and fire capable of detecting a fire by analyzing a visible light image and a thermal image by tying a single subject instead of an outline. It is about the detection method.

In order to detect whether a fire has occurred, a visible light camera and a thermal infrared camera are widely used.

The fire detection method through the visible light camera is mainly used as a software method. That is, when a fire occurs, the spread of smoke or the characteristic change of the shape of the flame are algorithmized and compared with the captured images to determine whether a fire has occurred. However, in this case, there is a risk of errors due to water vapor, clouds, dust, etc. similar to the form of smoke generated during a fire.

In the fire detection method through a thermal infrared camera, it is determined whether a fire has occurred by analyzing a temperature change from an image taken by the thermal infrared camera. However, since the image composition based on the temperature of the heat source has a very low resolution and is a black and white image, it is difficult to grasp the current situation through the surrounding image as an auxiliary method to finally confirm when a high temperature that is expected to cause a fire is detected. That is, as a method to eliminate the error of recognizing a fire when the temperature is unconditionally higher than the set temperature, there is a problem in that it is difficult to analyze and process the image through software or to determine it by visually checking the operator directly.

As one of the techniques for solving these problems, "a device for optically fusion of a visible ray image and a far-infrared image" according to Republic of Korea Patent No. 10-1441589 is equipped with a visible ray camera and a far-infrared camera in one optical module. A technique for acquiring images has been presented. However, this technology may have RAW values of different subjects at the same location in the image depending on the sensor characteristics such as spatial resolution, even if they are simultaneously photographed optically. As another technique, "a fusion image acquisition method based on a stereo visible light camera and a thermal imaging camera" according to Korean Patent Registration No. 10-1932411 extracts outlines from the left and right images and thermal infrared images of a stereo camera, and matches each outline It relates to an apparatus and method for fusing a thermal image to a stereo camera image. However, this technique has a problem that many matching errors may occur due to many contours on complex backgrounds (leafy trees, striped clothes, etc.). As another technology, "fire prediction and preservation system using visible light and infrared thermal image" according to Korean Patent Registration No. 10-1863530 is a system that sends visible light and thermal image to a control server and controls each image to predict and preserve fire is about Although this technology uses visible light and infrared thermal imagery, the image is not directly used for fire recognition but is used to help control.

An object of the present invention is to provide a fire detection system and a detection method capable of detecting a fire in real time by analyzing a visible light image and a thermal image image by tying one subject instead of an outline.

Another object of the present invention is to provide a real-time fire detection system and detection method capable of aligning two images even in a complex background.

Another object of the present invention is to provide a real-time fire detection system and detection method with improved performance by fusing two images in RAW-level.

A real-time fire detection system according to the present invention for achieving this object includes: a photographing device for acquiring a first image photographed by a visible light camera and a second image photographed by a thermal imaging camera; a control unit configured to receive the first image and the second image acquired by the photographing device through communication with the photographing device and to control the photographing apparatus; and an image analysis unit that analyzes the first image and the second image transmitted through the controller through a fusion method that binds one subject into one, rather than an outline, to determine whether a fire has occurred.

In the real-time fire detection system according to the present invention, the photographing device may be at least one CCTV installed in a fire detection target building.

In the real-time fire detection system according to the present invention, the photographing device and the control unit communicate through wired communication or wireless communication method.

In the real-time fire detection system according to a preferred embodiment of the present invention, the photographing device may be a flying drone that moves along a flight path and shoots.

In the real-time fire detection system according to the present invention, the flight path of the flying drone may be controlled by a signal provided in real time from a control unit mounted on the vehicle.

In the real-time fire detection system according to the present invention, the flight path of the flying drone is preset and stored inside the flying drone, and the flying drone is equipped with a global positioning system (GPS).

In the real-time fire detection system and detection method according to the present invention, the first image and the second image are transmitted as a RAW file in the form of an unprocessed image file format.

In the real-time fire detection system and detection method according to the present invention, the fusion method divides the first image and the second image using image segmentation technology, matches corner points of the segmented image, and matches It is possible to align the images and detect whether there is a fire using the geometric transformation matrix extracted from the image.

In the real-time fire detection system and detection method according to the present invention, the image segmentation technique uses a MASK R-CNN object detection algorithm.

The real-time fire detection apparatus and detection method according to the present invention can exhibit the effect of reducing damage caused by fire by improving the fire detection performance in drone reconnaissance and CCTV.

The real-time fire detection apparatus and detection method according to the present invention can improve fire detection performance by fusing two images in RAW-level even in a complex background.

1 is an exemplary diagram showing the configuration of a real-time fire detection device according to a first embodiment of the present invention.
2 is an exemplary diagram showing the configuration of a real-time fire detection device according to a second embodiment of the present invention.
3 is an exemplary diagram illustrating an example of monitoring a plurality of areas through a plurality of flying drones using a control unit mounted on a vehicle in the real-time fire detection device according to FIG. 2 .
4 is a perspective view showing a schematic configuration of a flying drone according to the present invention.
5 is a flowchart showing the progress of the real-time fire detection method according to the present invention.
6 is an exemplary diagram illustrating an embodiment of image segmentation in a real-time fire detection method according to the present invention.
7 is an exemplary diagram illustrating an example of corner point matching in a real-time fire detection method according to the present invention.
8 is an exemplary diagram illustrating an example of aligning images using the geometric transformation matrix extracted in the real-time fire detection method according to the present invention.
9 is an exemplary diagram illustrating determining whether a fire exists by using the aligned images in the real-time fire detection method according to the present invention.

With respect to the embodiments of the present invention disclosed in the text, specific structural or functional descriptions are only exemplified for the purpose of describing the embodiments of the present invention, the embodiments of the present invention may be implemented in various forms and It should not be construed as being limited to the described embodiments.

Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that there is no other element in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", should be interpreted similarly.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this application, terms such as “comprises” or “having” are intended to designate that the disclosed feature, number, step, action, component, part, or combination thereof exists, but includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be construed as indicating meanings consistent with the meanings in the context of the related art, and should not be construed in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

On the other hand, when an embodiment can be implemented differently, functions or operations specified in a specific block may occur differently from the order specified in the flowchart. For example, two consecutive blocks may be performed substantially simultaneously, or the blocks may be performed in reverse according to a related function or operation.

Hereinafter, a real-time fire detection apparatus and a fire detection method using the same according to the present invention will be described with reference to the accompanying drawings.

1 is an exemplary diagram showing the configuration of a real-time fire detection device according to a first embodiment of the present invention. A photographing apparatus 100 for acquiring a first image photographed by a visible light camera and a second image photographed by a thermal imaging camera, and a first acquired by the photographing apparatus 10 through communication with the photographing apparatus 10 a control unit 200 for receiving an image and a second image and controlling the photographing device; and an image analysis unit 300 that analyzes the first image and the second image transmitted through the control unit 200 through a fusion method that binds one subject into one, rather than an outline, to determine whether a fire has occurred.

The photographing apparatus 100 in the first embodiment is illustrated as an example of a plurality of CCTVs (110-1, 110-2, ..., 110-n) installed in a fire detection target building. In the example diagram, some of the plurality of CCTVs 110-1, 110-2, ..., 110-n perform wired communication with the control unit 200, and some of the CCTVs 110-1, 110-2, ..., 110-n perform wireless communication. Of course, all the plurality of CCTVs (110-1, 110-2, ..., 110-n) transmit and receive data with the control unit 200 in a wired communication method, or all the plurality of CCTVs (110-1, 110-2, ..., 110-n) ) may transmit/receive data to and from the controller 200 in a wireless communication method. In this case, the photographing apparatus 100 may transmit a captured image to the control unit 200 , and the control unit 200 may transmit a control signal to the photographing apparatus 100 .

2 is an exemplary view showing the configuration of a real-time fire detection device according to a second embodiment according to the present invention. The photographing device 100 is an example made up of a plurality of flying drones 120-1, 120-2, ..., 120-n that take images while moving along a flight path.

As shown in FIG. 3 , each flying drone 120-1, 120-2, ..., 120-n may be divided into a plurality of regions (first region, second region, …, n-th region) and photographed. In this case, the flying drones 120-1, 120-2, ..., 120-n may be controlled in real time by a signal provided from the controller 200 mounted in the vehicle.

On the other hand, although a plurality of flying drones are shown in this exemplary diagram, a real-time fire detection device according to the present invention may be implemented with a single flying drone. In addition, the flight path of the flying drone 120 may be a preset flight path.

As shown in FIG. 4 , the flying drone 120 includes a visible light camera 111 that captures a visible light image of a subject based on a wavelength of the visible light band, and an infrared image of the subject based on a wavelength of the infrared range. A thermal imaging camera 112 and a positioning system (Global 　Positioning 　System: GPS) 113 may be provided. The flying drone 120 takes an image while flying a predetermined flight path using the location information received through the positioning system (GPS: 113). In addition, although not shown, the flying drone 120 includes a communication unit such as a long-distance communication unit/medium-range communication unit/pairing unit/Wi-Fi communication unit, and a sensor unit including a three-axis acceleration sensor/temperature sensor/smoke sensor, and the like. A power supply unit for supplying necessary power and power required for operation of each component may be provided. The flying drone 120 has a fire extinguishing unit that sprays a fire extinguishing agent for immediately extinguishing a fire or fires a fire bomb depending on the fire situation, and when many people are trapped by a fire in a high-rise building or a large place, preferentially oxygen It may include a disaster prevention article supply unit provided with articles for saving lives by first supplying rescue articles such as a respirator and a gas mask.

The first image and the second image captured by the photographing device 100 , which may be a CCTV or a flying drone, are transmitted to the control unit 200 as a RAW file in the form of an unprocessed image file format.

RAW files are one of the image file formats and contain the meaning of the English word 'RAW', and have the meaning of 'unprocessed image data'. Compared to JPEG files, which are widely used image file formats, RAW files have a higher quality and use lossless compression technology to maintain the original quality. Therefore, it has the advantage of being excellent in color and suitable for subsequent corrections such as exposure and white balance.

5 is a flowchart schematically illustrating a process of a real-time fire detection method according to the present invention. As shown, the first image captured by the visible light camera mounted on the photographing device and the second image photographed by the thermal imaging camera are acquired, and the first image acquired by the photographing device through communication with the photographing device and An image collection process of receiving the second image is performed. In this case, the first image and the second image are transmitted as RAW files, which are unprocessed image file formats (S510).

The image analysis unit 300 analyzes the first image and the second image received through the control unit 200 through a fusion method that binds one subject into one, rather than an outline, so that an image analysis process is performed to determine whether a fire has occurred. (S520).

Meanwhile, the image analysis process 520 is performed as follows.

First, the first image and the second image are segmented using an image segmentation technique, as shown in FIG. 6 . There are three main methods for implementing image segmentation. A first method is a pixel-based segmentation method. After changing an image to a specific domain, it is an image binarization method that selects and divides a threshold value that can be segmented well. As a next method, there is an edge based segmentation method that extracts an edge (edge) from image data and then divides it based on the edge. As a last method, there is a region-based segmentation method in which pixels are segmented while expanding from a pixel at a specific reference point in an image to an image range having the same meaning.

In the present invention, it is exemplified that an image is segmented through image recognition technology and a pixel segmentation method of assigning a label to each pixel is used as an example, but it cannot be overemphasized that the present invention is not limited to this segmentation method. will be. The first image captured by the visible light camera 111 and the second image photographed by the thermal imaging camera 112 are divided, respectively, and a label is assigned to each pixel to generate a first image and a second image. . The image segmentation technique in this embodiment applies a MASK R-CNN object detection algorithm that extends convolutional neural networks (CNN) to pixel-level segmentation (S521).

Corner points are detected in the 'first image' and the second image divided at the pixel level (S522).

As shown in FIG. 7 , the corner points are matched so that the distance between each corner point of the 'first image' and the second image' has a minimum value (S523), and a geometric transformation matrix is extracted from the matched image (S524).

Then, the images are aligned using the extracted geometric transformation matrix. For example, as shown in FIG. 8 , a third image is generated by geometrically transforming the 'second image, which is a thermal image, so that values from the object are mapped to the same position as the 'first image, which is a visible light image' (S525). .

As shown in FIG. 9 , by using the value of the aligned image as an input value of the fire detection network, the image is recognized to determine whether or not there is a fire ( S526 ).

Aligning two sensors with different characteristics is a very difficult problem. Unlike other existing methods, the present invention solves this problem with image segmentation technology, which has recently been spotlighted by two separate single cameras. For example, since it uses an image segmentation algorithm such as MASK R-CNN to tie a single subject (object) into one rather than an outline, it is possible to align two sensor images robustly even in a complex background. In addition, by fusion in RAW-LEVEL, the performance can be improved compared to the fusion method at the crystal level.

As described above, using a real-time fire detection device and fire detection method that applies a fusion method that binds one subject rather than an outline using visible light images and thermal images captured by flying drones and CCTVs, It can have an effect that can reduce damage.

Although the above has been described with reference to the preferred embodiment of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: recording device 110: CCTV
120: flying drone 200: control unit
300: image analysis unit

Claims (18)

a photographing device for acquiring a first image photographed by a visible light camera and a second image photographed by a thermal imaging camera;
a control unit configured to receive the first image and the second image acquired by the photographing device through communication with the photographing device and to control the photographing apparatus; and
It comprises an image analysis unit that analyzes the first image and the second image transmitted through the control unit through a fusion method that binds one subject, not an outline, to check whether a fire has occurred,
The fusion method divides the first image and the second image using image segmentation technology, matches the corner points so that the distance between each corner point of the segmented image has a minimum value, and matches the matched image A real-time fire detection system that extracts a geometry transformation matrix from The real-time fire detection system according to claim 1, wherein the photographing device is at least one CCTV installed in a building subject to fire detection. The real-time fire detection system according to claim 1, wherein the photographing device and the control unit communicate using a wireless communication method. The real-time fire detection system according to claim 1, wherein the photographing device is a flying drone that takes pictures while moving along a flight path. The real-time fire detection system according to claim 4, wherein the flight path of the flying drone is controlled by a signal provided in real time from the control unit. The real-time fire detection system according to claim 5, wherein the control unit is mounted on a vehicle. The real-time fire detection system according to claim 4, wherein the flight path of the flying drone is a preset flight path, and the flying drone is equipped with a global positioning system (GPS). The real-time fire detection system according to claim 1, wherein the first image and the second image are transmitted as RAW files in an unprocessed image file format. delete The real-time fire detection system according to claim 1, wherein the image segmentation technique is a MASK R-CNN object detection algorithm. A process of acquiring a first image photographed by a visible light camera mounted on a photographing apparatus and a second image photographed by a thermal imaging camera;
receiving the first image and the second image acquired by the photographing device through communication with the photographing device; and
It includes a process of checking whether a fire has occurred by analyzing the received first image and the second image through a fusion method that binds one subject, not the outline,
The fusion method is
segmenting the first image and the second image using an image segmentation technique;
detecting each corner point in the segmented image;
matching the corner points so that the distance between each corner point of the first image and the second image has a minimum value;
extracting a geometric transformation matrix from the matched image;
aligning the image using the extracted geometric transformation matrix;
A real-time fire detection method comprising the step of determining whether a fire exists by recognizing an image by using the value of the aligned image as an input value. The real-time fire detection method according to claim 11, wherein the photographing device is at least one CCTV installed in a building subject to fire detection. The method of claim 11, wherein the photographing device is a flying drone that moves along a flight path and takes pictures. The method of claim 13, wherein the flight path of the flying drone is provided in real time. The method of claim 14, wherein the flight path of the flying drone is a preset flight path, and the flying drone is provided with a global positioning system (GPS). The real-time fire detection method according to claim 11, wherein the first image and the second image are transmitted as RAW files in the form of an unprocessed image file format. delete The method of claim 11, wherein the image segmentation technique is a MASK R-CNN object detection algorithm.

Images