KR20210132380A - The system and the method for detecting fire based on thermal and real images - Google Patents

Abstract

The present invention relates to an image-based fire detection system and, more specifically, to an image-based fire detection system capable of supplementing a false alarm for a fire-like image, detecting an internal fire which is not visible, and delivering temperature data of a specific area in an image to a user in real time, and a method thereof. According to one embodiment of the present invention, the image-based fire detection system includes: an region-of-interest (ROI) detection part detecting a specific ROI from an actual image received from a camera, using an artificial intelligence algorithm; a heat detection part outputting a thermal image from a thermal image camera and predicting a surface temperature of each spot; an image matching part matching the actual image about the specific ROI detected by the ROI detection part with the thermal image outputted by the heat detection part; and a fire determination part detecting a fire based on the matched image created by the image matching part and the specific ROI detected by the ROI detection part.

Description

The system and the method for detecting fire based on thermal and real images

The present invention relates to a thermal image and real image-based fire detection system, and more particularly, to a system and method capable of more accurately detecting a fire by matching a thermal image and a real image obtained from a general camera.

Fire accidents cause numerous lives and damage to property. In 2019 alone, there were 40,103 fires, about 110 fires per day. Carelessness accounts for more than half of the causes of fires as described above, so if you take precautions and pay attention to fires in advance, the loss and risk can be sufficiently reduced. Accordingly, methods for detecting a fire are required.

Most of the fire detection and alarm systems currently used are sensor-based detectors such as heat and smoke, and these detectors can only be detected when heat or smoke spreads over a certain level after a fire and reaches the sensor. As the smoke monitoring method, a photoelectric spot type using a sensor, a photoelectric separation type, an ionization type, and an air suction type have been used.

However, such a fire detection sensor can detect the occurrence of a fire only when heat or smoke is directly introduced or sucked into the sensor. For this reason, the possibility of false detection of fire is high depending on the surrounding environment, and an image-based fire detection method is required to prevent such false detection.

As a prior art related to an image-based fire detection method, Korean Patent Registration No. 10-1620989 (hereinafter abbreviated as 'prior art') discloses a method for automatically detecting a fire from a video including a night image and a system for performing the same do. However, in the prior art, the area of interest where the fire occurred was designated based on the brightness of the frame in a series of processes for determining the fire area. This is to detect the presence or absence of a fire by flashing lights or flames that can be identified in the image. It is difficult to detect small embers in the early stages of a fire, or if the screen contains only smoke without flames. There is a problem that the fire detection rate is low because the fire site cannot be detected. In addition, it is not possible to distinguish a condition in which a change in brightness is repeated, such as a bicycle blinker, a car blinker, a neon sign, etc.

In addition, the conventional image-based fire detection method has a high false detection rate when it is similar to a flame even though it is not a flame, and there is a problem in that the internal fire is not detected because it does not appear as a flame in the image.

(KR) Registered Patent No. 10-1620989 (KR) Registered Patent No. 10-1998639 (KR) Patent Publication No. 10-2016-0010387

The present invention has been devised to solve the above problem, supplements false alarms for fire-like images, detects invisible internal fires, and delivers temperature data in a specific area in the image to the user in real time. An object of the present invention is to provide an image-based fire detection system and a method therefor.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

An image-based fire detection system according to an embodiment of the present invention includes an ROI detector for detecting a specific region of interest (ROI) from a real image received from a camera using an artificial intelligence algorithm; a thermal sensor that outputs a thermal image from the thermal imaging camera and predicts the surface temperature of each point; an image matching unit for matching the real image of a specific region of interest detected by the ROI detection unit with the thermal image output from the thermal sensing unit; and a fire determination unit configured to detect a fire based on the matched image generated by the image matching unit and a specific region of interest (ROI) detected through the ROI detection unit.

A specific region of interest according to an embodiment of the present invention may be an object having a high internal fire probability, and may be a preset one.

The ROI detector according to an embodiment of the present invention may further use a region extraction algorithm.

The heat sensor according to an embodiment of the present invention may predict the room temperature by applying a pre-stored temperature DB to the temperature information collected through the temperature sensor.

The thermal sensing unit according to an embodiment of the present invention may predict the surface temperature according to the type of object detected by the ROI detection unit.

The thermal sensor according to an embodiment of the present invention may predict the temperature based on the distance between the object and the temperature sensor detected by the ROI detector.

The image matching unit according to an embodiment of the present invention may match the real image and the thermal image by resizing an image scale of the thermal image according to at least one of a lens specification and a resolution.

The image matching unit according to an embodiment of the present invention uses the DB obtained according to the distance between the object and the temperature sensor to match the coordinates of the thermal image of the object on the real image coordinate system, so that the real image and the thermal image can be matched.

An image-based fire detection system according to an embodiment of the present invention is performed by a device including a processor, and the processor uses an artificial intelligence algorithm to obtain a specific region of interest (ROI) from a real image received from a camera. ROI detection step of detecting interest); The processor outputs a thermal image from the thermal imaging camera and a thermal sensing step of predicting the surface temperature of each point; The processor may include: an image matching step of matching the real image of the specific region of interest detected in the ROI detection step with the thermal image output in the thermal detection step; and a fire determination step in which the processor detects a fire based on the matched image generated in the image matching step and a specific region of interest (ROI) detected through the ROI detection step.

The image-based fire detection system and method according to an embodiment of the present invention can supplement false alarms for fire-like images compared to existing systems.

In addition, the present invention can detect a fire that occurs inside an object that is not visible in the real image.

Furthermore, the present invention can provide the user with the actual temperature of the detected specific region of interest more accurately than the conventional system.

Additionally, the present invention can increase the accuracy of the matched image for the real image and the temperature image.

Effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1A shows a result detected by a conventional image-based fire detection method.
Figure 1b shows a result detected by the image-based fire detection method according to an embodiment of the present invention.
2 is a block diagram of an image-based fire detection system according to an embodiment of the present invention.
3 is a flowchart of an image-based fire detection method according to an embodiment of the present invention.
4A shows a technique for detecting a region of interest (ROI) according to an embodiment of the present invention, and FIG. 4B shows a bounding box used in a YOLOv3 structure applied to detect a region of interest (ROI).
5 illustrates a process of detecting a region of interest (ROI) according to an embodiment of the present invention.
6 illustrates an experiment for confirming a temperature pattern according to a target object detected according to an embodiment of the present invention.
7 is a graph illustrating a surface temperature pattern according to a target object (switchboard case) detected according to an embodiment of the present invention.
8 is a graph illustrating a surface temperature pattern according to a target object (vehicle engine bonnet) detected according to another embodiment of the present invention.
9A is a graph showing a temperature measurement result according to a distance according to another embodiment of the present invention, and FIG. 9B shows a method for measuring a distance according to an embodiment of the present invention.
10 is a diagram illustrating implementation by adding a thermal imaging camera to a system having only an existing real image camera according to an embodiment of the present invention.
11A and 11B illustrate setting a multi-region of interest (ROI) according to an embodiment of the present invention.
12 shows setting the temperature according to an embodiment of the present invention.
13 shows specifications of two lenses for matching a real image and a temperature image according to an embodiment of the present invention.
14 illustrates a test for matching coordinate data of an object obtained according to a distance in order to match a real image and a temperature image according to another embodiment of the present invention.
15 shows a registered image obtained at a distance of 1 m according to the present invention.
16 shows a registered image obtained at a distance of 2 m according to the present invention.
17 shows a registered image obtained at a distance of 3 m according to the present invention.
18 shows a registered image obtained at a distance of 4 m according to the present invention.
19 shows a registered image obtained at a distance of 5 m according to the present invention.
20 shows a registered image obtained at a distance of 6 m according to the present invention.
21 shows a registered image obtained at a distance of 7 m according to the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. and/or includes a combination of a plurality of related description items or any of a plurality of related description items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it should be understood that other components may exist in between. something to do. On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

It should also be understood that the terms "first" and "second" are used herein for distinguishing purposes only, and are not meant to indicate or anticipate sequence or priority in any way.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Throughout the specification and claims, when a part includes a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1A shows a result detected by a conventional image-based fire detection method.

As shown in FIG. 1A, the conventional fire detection method/system distinguishes a condition (flicking in which a change in motion is repeated) in which a change in brightness is repeated, such as a car flasher 10, a bicycle blinker, a neon sign, etc. Since it cannot, there is a problem that the false alarm rate of fire detection is high.

In addition, there is a problem in that the fire generated inside the object is not detected by the conventional fire detection method because it does not appear as a flame in the image.

Figure 1b shows a result detected by the image-based fire detection method according to an embodiment of the present invention.

As shown in FIG. 1B , the fire detection method/system according to an embodiment of the present invention can detect a fire 20 generated inside a switchboard and a fire 30 caused by an explosion such as an electronic device included in a box. It is effective in detecting internal fires.

In addition, the fire detection method according to an embodiment of the present invention can prevent erroneous detection due to a similar flame by setting an object with a high probability of occurrence of a fire as a region of interest (ROI).

2 is a block diagram of an image-based fire detection system according to an embodiment of the present invention.

The image-based fire detection system 1000 according to an embodiment of the present invention may include an ROI detection unit 100 , a heat detection unit 200 , an image matching unit 300 , and a fire determination unit 400 .

The ROI detector 100 according to an embodiment of the present invention may detect a specific region of interest (ROI) by applying an artificial intelligence algorithm to the real image received from the camera. The real image photographed by the camera is an image of the fire monitoring area, for example, may be photographed by CCTV, etc., and the real image means a color image having RGB values.

The ROI detector 100 may detect a specific region of interest (ROI) by using a region extraction algorithm, and the specific region of interest is an object having a high internal fire potential, and may be preset in the algorithm.

A specific region of interest (ROI) according to an embodiment of the present invention may be, for example, a switchboard or a bonnet of a parked vehicle. The switchboard is an object that fires due to the internal short circuit of electric wires and is the main cause of large fires, so it is preset as a specific area of interest (ROI). It may be preset as a region of interest (ROI). However, the specific region of interest (ROI) is not limited to the above-described switchboard or the bonnet of the vehicle, and an object having a high possibility of internal fire may be further included.

Meanwhile, a method of extracting a specific region of interest (ROI) is performed based on deep learning, which will be described later with reference to FIGS. 4 and 5 .

The thermal sensor 200 according to an embodiment of the present invention may output a thermal image from a thermal imaging camera and predict the surface temperature of each point. A thermal imaging camera is capable of outputting a thermal image by sensing infrared rays and imaging the temperature as a color, and a conventional thermal imaging camera may be used.

The image matching unit 300 according to an embodiment of the present invention may match the real image of the specific region of interest detected by the ROI detection unit 100 with the thermal image output from the thermal sensing unit 200 . Image registration will be described later with reference to FIGS. 13 to 21 .

The fire determination unit 400 according to an embodiment of the present invention detects a fire based on the matched image generated by the image matching unit 300 and a specific region of interest (ROI) detected by the ROI detection unit 100 . can

The fire determination unit 400 according to an embodiment of the present invention may actively select a specific region of interest (ROI) requiring fire detection from the matched image in which the real image and the thermal image are matched. In addition, the fire determination unit 400 may predict and provide an abnormal symptom temperature value using a temperature DB secured in advance according to the object type of the selected ROI and the distance between the object and the temperature sensor.

The fire determination unit 400 according to an embodiment of the present invention detects a fire based on temperature in a specific region of interest (ROI), and detects fire based on an actual image in a region other than the ROI, enabling more accurate fire detection let it do

Meanwhile, a specific region of interest (ROI) according to an embodiment of the present invention may be directly set by a user on a program.

The fire detection system according to an embodiment of the present invention may also include a communication unit (not shown) for receiving an image captured by a camera in a wired/wireless manner. Furthermore, when a fire is detected through the fire determination unit, the communication unit may transmit fire occurrence information including an ignition point to a remote central control center. A central control center located remotely can be implemented as a management server that integrates and operates a plurality of ignition point monitoring devices.

3 is a flowchart of an image-based fire detection method according to an embodiment of the present invention.

The image-based fire detection method according to an embodiment of the present invention may be performed by a device including a processor, and the image-based fire detection method includes:

an ROI detection step (S310) in which the processor detects a specific region of interest (ROI) from the real image received from the camera by using an artificial intelligence algorithm;

A thermal sensing step (S320) of outputting a thermal image from the thermal imaging camera and predicting the surface temperature of each point;

an image matching step (S330) of matching the real image of the specific region of interest detected in the ROI detection step and the thermal image output in the thermal detection step; and

and a fire determination step (S340) of detecting a fire based on the matched image generated in the image matching step and a specific region of interest (ROI) detected through the ROI detection step.

Hereinafter, a specific implementation method of each step will be described in detail.

4A shows a technique for detecting a region of interest (ROI) according to an embodiment of the present invention, and FIG. 4B shows a bounding box used in a YOLOv3 structure applied to detect a region of interest (ROI).

The method for detecting a specific region of interest (ROI) according to an embodiment of the present invention may be a method of extracting a specific object based on a deep learning algorithm.

The target to learn in the deep learning algorithm can be set to the switchboard 40 or the parking vehicle 41 with a high probability of an internal fire, and based on the object features of the switchboard 40 or the parking vehicle 41, the deep learning algorithm can be learned

According to an embodiment of the present invention, a YOLOv3 network structure may be applied to detection of a specific region of interest (ROI). The YOLOv3 architecture uses a bounding box with dimension priority and position prediction. The width and height of the bounding box are predicted as an offset from the cluster center. Use the sigmoid function to predict the center coordinates of the bounding box based on the location of the filter application.

Five elements of the bounding box, b _x , b _y , b _w , b _h , and confidence are predicted through Equation 1.

[Equation 1]

Then, a class-specific confidence score for each box is calculated, and a plurality of overlapping b-boxes are later deleted using an NMS algorithm.

[Equation 2]

Classification loss is as in Equation 3 below, where S ² is a grid cell.

[Equation 3]

는 물체가 i번째 셀에서 나타나면 1이고, 아니면 0이다.

는 i번째 셀에서 클래스 c에 대한 조건부 클래스 확률을 나타낸다.here,

is 1 if the object appears in the i-th cell, otherwise 0.

denotes the conditional class probability for class c in the i-th cell.

The localization loss is as in Equation 4 below, where B = Predicted Bounding Box is satisfied.

[Equation 4]

는 i번째 셀의 j번째 경계 박스가 물체를 감지하는 경우 1, 그렇지 않으면 0이다.

는 경계 박스 좌표에서 손실에 대한 가중치를 증가시킨다.here,

is 1 if the j-th bounding box of the i-th cell detects an object, otherwise 0.

increases the weight for the loss in bounding box coordinates.

는 셀 i에서 상자 j의 상자 신뢰도 점수이고,

i번째 셀의 j번째 경계 박스가 물체를 감지하는 경우 1, 그렇지 않으면 0이다.Confidence loss is the following Equation 5, where

is the box confidence score for box j in cell i,

1 if the j-th bounding box of the i-th cell detects an object, otherwise 0.

[Equation 5]

Therefore, the overall loss function is equal to the sum of the localization loss, the confidence loss, and the classification loss, as shown in Equation 6.

[Equation 6]

The last three terms up to Equation 6 are the squared error, and YOLOv3 is replaced with the cross-entropy error term. And object reliability and class prediction in YOLOv3 are now predicted through logistic regression.

5 illustrates a process of detecting a region of interest (ROI) according to an embodiment of the present invention. As shown in FIG. 5 , the method for detecting a region of interest (ROI) according to an embodiment of the present invention applies a polygon automatic setting technique (U-Net) to effectively extract the boundary of the target object 50 to be detected. ROI 51 may be detected.

On the other hand, the conventional U-Net technique is applied to the U-Net structure, and learning is performed in the following order.

1. First energy function, Equation 7 is calculated as the soft max in pixels in the final map combined with the cross entropy loss function.

[Equation 7]

의 편차를 불이익을 준다.2. Cross entropy is a function from 1 using Equation 8 below at each position

penalize the deviation of

[Equation 8]

3. Precompute a weight map for each ground truth.

It compensates for pixels of different frequencies of a certain class in the training data set and forces the network to learn small separation boundaries introduced between touch cells.

[Equation 9]

4. Set the correct initialization of the weights. In this case, the Gaussian distribution has a standard deviation, and N below represents the number of inputs to one neural node.

6 illustrates an experiment for confirming a temperature pattern according to a target object detected according to an embodiment of the present invention.

The fire detection method/system according to an embodiment of the present invention may predict a room temperature by applying a surface temperature according to the type of a detected object and a pre-stored temperature DB. The experimental environment for measuring temperature data to build the temperature DB is as follows.

- When heat is generated at a distance of 5cm from the switchboard (61) consisting of a stainless steel plate with dimensions of 650mm x 450mm x 1mm

- Room temperature about 10.4℃

- Temperature measuring equipment: EX-POWER EIT-600 Infrared Thermometer

The present invention team conducted a test by removing the door and generating heat at a distance of 5 cm to reproduce the occurrence of a fire in the internal parts because the distance between the internal parts of the switchboard and the door is about 5 cm. 62) was used for measuring the temperature.

7 is a graph illustrating a surface temperature pattern according to a target object (switchboard case) detected according to an embodiment of the present invention.

7 shows the surface temperature according to the internal temperature of the stainless steel case, such as a switchboard, and is recorded in detail in Table 1 below.

internal temperature
(℃) surface temperature
(℃) 20 12 40 17 60 22 80 30.2 100 36 120 43.3 140 51.5 160 59.9 180 72.2 200 84 220 93 240 104 260 122 280 138 300 150 320 165

8 is a graph illustrating a surface temperature pattern according to a target object (vehicle engine bonnet) detected according to another embodiment of the present invention.

The experimental environment for measuring the surface temperature of the car engine bonnet is as follows.

- Test vehicle: Ssangyong Tivoli

- Room temperature about 10.4℃

- Temperature measuring equipment: EX-POWER EIT-600 Infrared Thermometer

- Engine temperature and bonnet surface temperature measurement on the instrument panel

- Test up to 120℃ for engine protection

The surface temperature according to the internal temperature of the vehicle engine is recorded in detail in Table 2 below.

internal temperature
(℃) surface temperature
(℃) 20 11 40 15 60 20 80 28 100 36 120 39

9A is a graph illustrating a temperature measurement result according to a distance according to another embodiment of the present invention.

According to another embodiment of the present invention, the fire detection method/system may provide a corrected value for a temperature value actually measured by the temperature sensor according to the actual value, in consideration of the characteristics of the temperature sensor.

As the distance of the temperature sensor increases, the measured thermal energy value is smaller than the actual temperature value, and the laser thermometer has the same characteristics.

If the temperature to be detected through the temperature sensor is set based on the experimental results below according to the camera installation distance, more accurate temperature detection is possible.

The temperature record DB of the temperature sensor according to the distance can be constructed as shown in Table 3 below.

target temperature
(℃) Thermal measurement temperature (℃) 1m 3m 5m 20 18 17.5 17 40 31.9 31 29 60 49 42.6 40 80 59 55.6 51 100 75 69.5 65.6 120 90 82 73.5 140 108 97 88.5 160 120 113.6 101.5 180 143 126 116 200 162 145 130 220 182 156 145 240 182 170 165

Meanwhile, as a method of measuring the distance, a method of estimating the distance of an object based on a conventional image may be applied. In an embodiment of the present invention, the object may be a flame, and the distance between the camera and the center of the flame may be estimated.

More specifically, if the ground coordinate corresponding to the image coordinate p is P, the camera origin and p, P may exist on a straight line as shown in FIG. 9B .

Assume that the camera internal parameters (focal length fx, fy, main points cx, cy), the height from the ground (h), and the camera tilt (θ _tilt ) are given (θ _tilt is 0 when the camera direction is horizontal with the ground, + when looking up, -) when looking down. And it is assumed that the pixel coordinates of the input image are p(x,y), and the corresponding ground coordinates are P(X,Y). Based on the above assumption, the image-based object distance estimation process is as follows.

First, the pixel coordinates are converted into regular coordinates in order to eliminate the influence on the internal parameters of the camera.

Second, with reference to FIG. 9c, using the obtained regular coordinates (u, v), camera height h, and tilt θ _tilt , it is calculated as shown in the following equation. However, note that in the normal image plane, the coordinates of the main point become c(0, 0) and the focal length between the image plane and the camera origin becomes 1.

In the above calculation result, d is the straight line distance between the camera origin and the object on the ground, θ is the direction angle of the object with respect to the camera optical axis, and the left side is + and the right side is - based on the camera optical axis.

When the position shown in FIG. 9c is viewed in 　top-view, it is the same as in FIG. 9d, and at this point, it can be calculated in which direction (θ) and how far (d) the object is from the camera.

Finally, when the position of the object is expressed in terms of 　 ground coordinates, it is as follows. However, these are coordinates assuming that the camera position is the origin of the ground coordinate system and the optical axis direction is the X axis.

P(X, Y) finally calculated by the method described above with reference to FIGS. 9B to 9D is a value that can vary depending on how the ground coordinate system is defined. Coordinates when the axis direction is taken as the X-axis.

10 is a diagram illustrating implementation by adding a thermal imaging camera to a system having only an existing real image camera according to an embodiment of the present invention.

The image-based fire method/system according to an embodiment of the present invention can be applied to a system having only an existing real image camera by adding a thermal imaging camera. Through the program to which the present invention is applied as shown in Fig. 10, it can be set as follows.

- Main camera (real video camera) connection: Enter the camera information in “Add by IP address” or “Add by RTSP address”

- Thermal image camera connection: Enter thermal image RTSP stream information in “Thermal”

In the present invention, when only the main camera is connected, only the image-based fire detection algorithm can operate in the same way as in the existing system, and when the main camera and the thermal imager are connected, the ROI area of the thermal imager detects fire based on temperature, Other areas may operate to detect a fire based on an actual image.

In addition, if the program to which the method according to an embodiment of the present invention is applied is used, the ROI region may be directly added. The detailed operation in the program is as follows.

The program according to an embodiment of the present invention may include a function of adding/deleting individual/deleting all ROIs by right-clicking the mouse on the enlarged view of the thermal image.

- Add ROI: Set the area you want to add and the minimum, maximum temperature and satisfaction rate that will trigger an event in that area

- Individual ROI Deletion: Click the desired ROI among the previously created ROIs to delete it.

- Deleting all ROIs: Removes all previously created ROI information.

In addition, the following additional functions may be further included.

- Set up to 3 thermal image ROIs per channel

- Set temperature can range from -10 to 150 °C

- Settable temperature satisfaction zone ratio can range from 1 to 100%

-

11A and 11B illustrate setting a multi-region of interest (ROI) according to an embodiment of the present invention.

According to an embodiment of the present invention, a plurality of specific ROIs may be designated. When the temperature value in at least one ROI among the plurality of ROIs 111, 112, and 113 exists more than a set ratio (%), it is determined that a fire is detected.

According to the present invention, it is determined that the area 120 other than the set ROI has detected a fire based on an actual image. A method of detecting a fire based on an actual image may be determined as detecting a flame or smoke displayed in the image.

12 illustrates temperature setting according to an embodiment of the present invention, which can be set by a user on a program to which the present invention is applied.

The program according to an embodiment of the present invention includes a thermal image display function 121 configurable for each channel, a temperature comprehensive information display function 122 configurable for each channel, a thermal image position setting function 123, and a color mode setting function 124 ) may be included.

13 shows specifications of two lenses for matching a real image and a temperature image according to an embodiment of the present invention.

The specification 131 of a real video camera (CCTV) used in a fire detection system according to an embodiment of the present invention uses a 1/3'' 2Mega CMOS image sensor, has a focal length of 3.6mm, and has a focal length of 75.7°. 1/3'' CCD, 1/4'' CCD of 62.2°, monitoring distance is 0m to 3m.

The specification 132 of the thermal imaging camera used in the fire detection system according to an embodiment of the present invention is Lepton 2.0:500-0659-01, 80x60 array format, 50° horizontal field of view, shutter, 8% With the following distortion (barrel), thermal measurement range of -10° to 140°, pixel pitch of 17um, 1/3'' CCD of 50°, 1/4'' CCD of 39.8°, monitoring distance from 5m to It is 10 m.

When the lens of the real image camera and the lens of the thermal image camera are used, there is an error that may occur due to a difference in the angle of view between the two lenses. Therefore, it is necessary to match the real image and the thermal image for accurate ROI display.

14 illustrates a test for matching coordinate data of an object obtained according to a distance in order to match a real image and a temperature image according to another embodiment of the present invention.

The environment of the test according to FIG. 14 is as follows.

- Test live resolution: 640 x 360

- Test thermal image resolution: 80 x 60, but is resized to the same scale as the real image according to the lens specification, and is resized to 426 x 320 in the test according to an embodiment of the present invention.

And, in the test according to the embodiment of the present invention, in order to build a DB of coordinate data according to the distance between the object and the temperature sensor, the coordinates between the two images are aligned and the position data of the target object is measured according to the distance of the object. do.

According to an embodiment of the present invention, when the object has a distance from the temperature sensor in units of 1 m to 1 m and up to 7 m, data as shown in Table 4 is acquired. In addition, data is acquired by changing the position of the object to the left 141 , the right 142 , and the center 143 for each distance.

target distance Column image start coordinates in real image coordinate system (X, Y) average coordinates
(X, Y) left right middle 1m 120, 7 120, 8 123, 10 121, 8 2m 123, 10 123, 10 123, 10 123, 10 3m 122, 6 119, 5 119, 7 120, 6 4m 123, 2 121, 4 119, 2 121, 3 5m 123, 4 118, 3 120, 3 120, 3 6m 123, 2 121, 1 121, 1 122, 1 7m 121, 4 119, 1 120, 2 120, 2

15 to 21 show registered images obtained when the distance of the object is changed from 1 m to 7 m in 1 m increments, according to an embodiment of the present invention.

The above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

1000: image-based fire detection system
100: ROI detection unit
200: heat sensing unit
300: image matching unit
400: fire judgment unit

Claims (16)

In the image-based fire detection system,
an ROI detector for detecting a specific region of interest (ROI) from the real image received from the camera by using an artificial intelligence algorithm;
a thermal sensor that outputs a thermal image from the thermal imaging camera and predicts the surface temperature of each point;
an image matching unit for matching the real image of a specific region of interest detected by the ROI detection unit with the thermal image output from the thermal sensing unit; and
An image-based fire detection system comprising a; fire determination unit that detects a fire based on the matched image generated by the image matching unit and a specific region of interest (ROI) detected through the ROI detection unit. According to claim 1,
The specific region of interest is an object with a high possibility of internally generating a fire, and the image-based fire detection system is preset. According to claim 1,
The ROI detection unit is an image-based fire detection system that further uses an area extraction algorithm. According to claim 1,
The heat sensor predicts the room temperature by applying the pre-stored temperature DB to the temperature information collected through the temperature sensor,
Image-based fire detection system. According to claim 1,
The heat detector predicts the surface temperature according to the type of object detected by the ROI detector,
Image-based fire detection system. According to claim 1,
The heat detector predicts the temperature based on the distance between the object and the temperature sensor detected by the ROI detector,
Image-based fire detection system. According to claim 1,
The image matching unit matches the real image and the thermal image by resizing the image scale for the thermal image according to at least one of a lens specification and a resolution,
Image-based fire detection system. 8. The method of claim 7,
The image matching unit
Using the DB obtained according to the distance between the object and the temperature sensor,
By matching the coordinates of the thermal image of the object on the real image coordinate system,
To register the real image and the thermal image,
Image-based fire detection system. An image-based fire detection method performed by a device including a processor, the method comprising:
The processor includes an ROI detection step of detecting a specific region of interest (ROI) from the real image received from the camera using an artificial intelligence algorithm;
The processor outputs a thermal image from the thermal imaging camera and a thermal sensing step of predicting the surface temperature of each point;
The processor may include: an image matching step of matching the real image of the specific region of interest detected in the ROI detection step with the thermal image output in the thermal detection step; and
The image-based fire detection method including a; wherein the processor detects a fire based on the matched image generated in the image matching step and a specific region of interest (ROI) detected through the ROI detection step. 10. The method of claim 9,
The specific region of interest is an object with a high possibility of internally generating a fire, and the image-based fire detection method is preset. 10. The method of claim 9,
The ROI detection step is an image-based fire detection method that further uses an area extraction algorithm. 10. The method of claim 9,
The heat sensing step is to predict the room temperature by applying a pre-stored temperature DB to the temperature information collected through the temperature sensor,
An image-based fire detection method. 10. The method of claim 9,
The thermal sensing step is to predict the surface temperature according to the type of object detected in the ROI detection step,
An image-based fire detection method. 10. The method of claim 9,
wherein the thermal sensing step predicts a temperature based on the distance between the object and the temperature sensor detected in the ROI detection step,
An image-based fire detection method. 10. The method of claim 9,
In the image matching step, the real image and the thermal image are matched by resizing the image scale for the thermal image according to at least one of a specification and a resolution of a lens,
An image-based fire detection method. 16. The method of claim 15,
The image matching step is
Using the DB obtained according to the distance between the object and the temperature sensor,
By matching the coordinates of the thermal image of the object on the real image coordinate system,
To register the real image and the thermal image,
An image-based fire detection method.

Images