KR20130101873A - Monitoring apparatus for forest fire using thermal camera and method thereof - Google Patents

Abstract

PURPOSE: A forest fire monitoring apparatus using a thermal imaging camera and a method thereof are provided to detect accurately smoke with the unique pattern analysis of the smoke even when the smoke is hidden by fog or cloud, thereby enabling the forest fire to be under control at an early stage. CONSTITUTION: A forest fire monitoring apparatus using a thermal imaging camera comprises a thermal imaging camera (10), a foreground/background division unit (110), an object extraction unit (120), an object tracking unit (130), a pattern storage unit (140), and an output unit (150). The foreground/background division unit receives consecutive thermal imaging data from the thermal imaging camera and divides each thermal imaging data into a foreground image and a background image according to a long-term background modeling mode. The pattern storage unit includes pattern information considering the temperature range, movement direction, or spread of smoke generated by a forest fire. The object extraction unit extracts objects within the temperature range included in the pattern information based on brightness information according to the temperature of objects which is included in the foreground image divided by the foreground/background division unit. The object tracking unit monitors the objects extracted from the consecutive thermal imaging data by the object extraction unit, and outputs forest fire information when an object of which a change of a pixel number or the movement direction satisfies the pattern information is detected. [Reference numerals] (10) Thermal imaging camera; (11) Calibration unit; (110) Foreground/background division unit; (120) Object extraction unit; (130) Object tracking unit; (140) Pattern storage unit; (150) Output unit

Description

Monitoring Apparatus for forest fire using thermal camera and method

The present invention relates to a forest fire monitoring apparatus and a method thereof, and more particularly, to a forest fire monitoring apparatus and method for effectively detecting the occurrence of wildfire at a long distance.

With the development of various video equipments, the field of video surveillance such as tracking an object or setting specific criteria in the video to monitor a specific target through analysis of the captured video has been highlighted. In addition, the development of various algorithms applied to the video equipment for video analysis provides a more sophisticated surveillance system, the field of video surveillance system is expected to continue to grow.

One example of the use of the imaging equipment is a fire monitoring system that detects the occurrence of a fire and notifies a danger.

Conventionally, a device used to monitor a fire is mostly used to monitor a fire occurring indoors, and a method of detecting a fire by analyzing a photographed image using a visible light camera is generally used. That is, the space where the fire occurred through the visible light camera, and by analyzing the captured image to determine whether the fire occurs by the change of color detected.

However, when the image analysis apparatus using the visible light camera is applied to forest fire monitoring, it is inefficient to install a camera on a mountain, so it is inevitable to determine whether a wildfire occurs in a wide range of images captured from a long distance.

However, in the case of monitoring a wildfire by installing a visible light camera at a long distance, it is difficult to detect a flame due to a large distance, unlike in the case of an indoor fire, and in particular, when a fire occurs over a mountain, the flame cannot be detected. There is no choice but to determine whether a fire has occurred.

However, if there is fog or cloud at the fire occurrence point, it is difficult to distinguish it from the smoke. In the video analysis process, the fog or cloud is recognized as smoke, and an error or smoke is not recognized. There is a problem.

In addition, if the flame is detected through the visible light camera, the extent of the flame is already detected by the visible light camera, so the scale of the fire is already serious. Therefore, it is not suitable for the purpose of the device to determine the signs of wildfire and to extinguish it early. Do not.

Therefore, as a conventional forest fire detection device to compensate for this, a method of detecting a forest fire through a repeater which distributes a signal detected from the sensor by installing a plurality of sensors distributed at a forest fire occurrence point has been considered. Although it is possible to increase the accuracy of the sensor, there is a concern that damage to the sensor due to forest fire and the cost of maintaining and maintaining the device is significant, resulting in inefficient problems.

Korean Patent Registration No. 100716306

The present invention detects the smoke generated during the combustion process when the forest fire occurs using a thermal imaging camera through image analysis to determine whether the forest fire occurs at a long distance early, so that a forest fire monitoring device that can effectively cope with the occurrence of forest fires The purpose is to provide.

Forest fire monitoring apparatus using a thermal imaging camera according to an embodiment of the present invention for achieving the above object receives a continuous thermal image data from the thermal imager, the foreground image according to the long-term background modeling method for each thermal image data And a foreground / background separator which separates into a background image, a pattern storage unit including a temperature range of smoke generated by a forest fire, pattern information considering a moving direction or diffusion, and a separate from the foreground / background separator. An object extracting unit for extracting an object included in a temperature range preset in the pattern information based on brightness information according to the temperature of the object included in the foreground image, and an object extracted by the object extracting unit from continuous thermal image data If an object whose pixel number change or movement direction satisfies the pattern information is detected It includes an object tracking unit for outputting information about the occurrence of forest fires.

In this case, the object extractor may label the object by giving an ID to each object included in a preset temperature range.

The object tracking unit may detect an object moving in an upward direction or a diagonal upward direction according to the pattern information by analyzing a moving direction of the extracted object.

In addition, the object tracking unit may detect an object in which the spreading speed of calculating a change amount of the number of pixels for a predetermined time is greater than a value predetermined in the pattern information among the extracted objects.

In addition, the object tracking unit may analyze the brightness information of the selected object and output the information on the occurrence of forest fire only for the object where the high frequency component is detected.

The apparatus may further include an image matching unit configured to generate matching information by mutually matching surveillance spaces between cameras using calibration information of the thermal imaging camera and the visible light camera, and the object tracking unit may analyze analysis information of the detected object. Matching may be performed based on matching information of the matching unit, and may be output to an image of the visible light camera at a position corresponding to the position of the detected object in the image of the thermal imaging camera.

In addition, the object extracting unit and the object tracking unit transmits the analysis information for the remaining objects except at least one or more objects extracted or detected based on the pattern information to the foreground / background separator, the foreground / background separator is the analysis information Based on this, the remaining objects can be processed in the background.

The forest fire monitoring method of the forest fire monitoring apparatus using the thermal imaging camera according to an embodiment of the present invention for achieving the above object, receives continuous thermal image data from the thermal imager, and the long-term background modeling for each thermal image data The first step of separating the foreground image and the background image according to the method, and the temperature range of the smoke generated by the forest fire by analyzing the brightness information according to the temperature of the object included in the separated foreground image through the first step And a second step of extracting an object included in a preset temperature range in the pattern information considering a moving direction or diffusion, and monitoring an object extracted from continuous thermal image data through the second step, and monitoring the pixel of the object. 3-stage outputting information on wildfire occurrence when an object whose number change or movement direction satisfies the pattern information is detected It includes.

In this case, the second step may label by giving an ID to each object included in a preset temperature range, and the third step may track each labeled object.

In addition, the third step may be characterized by detecting the object moving in the upward direction or diagonal upward direction according to the pattern information by analyzing the movement direction of the extracted object.

In addition, the third step may be characterized in that for detecting the object of the extracted object, the diffusion speed of calculating the amount of change of the number of pixels for a predetermined time is greater than or equal to the predetermined value in the pattern information.

In addition, the third step may further include detecting a high frequency component by analyzing brightness information of the selected object, and outputting information on a fire occurrence only to the object where the high frequency component is detected.

According to the present invention, it is possible to easily determine whether a fire occurs by distinguishing the smoke caused by a forest fire from other objects through a thermal imaging camera at a long distance, and to analyze the unique pattern of the smoke even if the smoke is covered by fog or clouds. Through the accurate detection of smoke through the forest fire suppression is made early and at the same time has the effect of increasing the reliability of the forest fire determination.

In addition, the present invention can effectively monitor forest fire generation only through a thermal imaging camera at a long distance, it is possible to significantly reduce the device required to configure a forest fire monitoring device, including the existing sensor can significantly reduce the cost.

1 is a view showing the operating environment of the forest fire monitoring apparatus according to an embodiment of the present invention.
2 is a block diagram of a forest fire monitoring apparatus according to an embodiment of the present invention.
3 is a diagram illustrating an embodiment of thermal image data obtained through a thermal imaging camera according to an embodiment of the present invention.
4 is a view showing a thermal image data analysis process of the forest fire monitoring apparatus according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating smoke detection using pattern information of a forest fire monitoring apparatus according to an exemplary embodiment of the present invention. FIG.
6 is a block diagram of a forest fire monitoring device further connected to a visible light camera according to an embodiment of the present invention.
FIG. 7 illustrates a switching output for detected smoke of a forest fire monitoring device additionally connected to a visible light camera according to an embodiment of the present invention. FIG.
8 is a flow chart for a forest fire monitoring method of the forest fire monitoring apparatus according to an embodiment of the present invention.

Hereinafter, an embodiment of a forest fire monitoring apparatus and a method using a thermal imaging camera according to an embodiment of the present invention with reference to the drawings in detail.

1 is a view showing an operating environment of a forest fire monitoring apparatus using a thermal imaging camera according to an embodiment of the present invention, as shown in the image is taken from the remote monitoring area (1) including a mountain installed at a distance It may include a thermal imaging camera 10 and a forest fire monitoring device 100 for receiving thermal image data from the thermal imaging camera 10 and analyzing the thermal image data to determine whether a fire occurs.

In this case, the thermal imaging camera 10 and the forest fire monitoring device 100 may be connected to each other through a wired or wireless network.

2 is a view showing a detailed configuration of a forest fire monitoring device 100 using a thermal imaging camera 10 according to an embodiment of the present invention, the thermal imaging camera unit 10, the calibration unit 11 as shown The foreground / background separator 110, the object extractor 120, the object tracking unit 130, the pattern storage unit 140, and the output unit 150 may be included.

In detail, the thermal imaging camera unit 10 generates thermal image data by capturing a surveillance region to be a forest fire monitoring target, and the calibration unit 11 is configured for the thermal imaging camera unit 10. Calibration information may be generated by matching the two-dimensional image of the thermal imaging camera unit 10 generated through the calibration with the surveillance space. In this case, the calibration unit 11 may be included in the thermal camera unit 10.

In this case, the thermal imaging camera unit 10 may include a thermal sensing camera such as an infrared camera.

Meanwhile, the foreground / background separator 130 analyzes the thermal image data received from the thermal camera unit 10 with reference to the calibration information received through the calibration unit 11, and the long term background modeling method. Accordingly, the foreground image including the object to be monitored and the background image except the foreground image may be separated.

In addition, the object extractor 120 may extract an object associated with a forest fire among objects included in the foreground image from the thermal image data divided into a foreground image and a background image through the foreground / background separator.

In this case, the object extractor 120 may extract an object satisfying the pattern information from the foreground image by referring to the pattern information stored in the pattern storage 140 to extract an object associated with a forest fire. .

In more detail, as shown in FIG. 3, in the foreground image separated from the thermal image data, various objects such as clouds (2), fog (3), automobile (5), etc., in addition to smoke (4) associated with wildfire generation, are included in the foreground image. May be included.

In this case, since the smoke 4 generated by the fire is relatively high in temperature, information on the temperature range of the smoke 4 may be set in the pattern information, and the temperature of each object is included in the thermal image data. It may be determined based on brightness information according to temperature.

Accordingly, the object extractor 120 may first have brightness information corresponding to a temperature range set in the pattern information in order to distinguish the object for smoke 4 from the plurality of objects included in the foreground image from other objects. You can extract the object. That is, it is possible to easily exclude an object such as a cloud 2 or a fog 3 having a low brightness due to its relatively low temperature.

In addition, the object extractor 120 may transmit analysis information about an object that does not belong to the temperature range included in the pattern information to the foreground / background separator 110, and the foreground / background separator 110. May perform background processing on all of the objects based on the analysis information. Accordingly, it is possible to reduce the amount of computation generated by subsequent analysis of the thermal image data.

Meanwhile, when extracting an object for smoke considering only the temperature range, when the temperature of the cloud 2 or the fog 3 temporarily increases or the temperature of the vehicle 5 satisfies the temperature range set in the pattern information. May occur.

In order to solve this problem, the pattern information may include information on a movement direction or diffusion, which is a unique property of smoke.

That is, since the smoke 4 generally moves upward or diagonally upward and has a characteristic of being diffused, the smoke 4 may be stored as pattern information to more clearly distinguish it from other objects included in the foreground image. In this case, the movement direction may be determined based on the movement vector of the pixels constituting the object and the diffusion based on the number of pixels.

Accordingly, in order to exclude other objects except for the smoke 4, the object tracking unit 130 tracks an object extracted from the object extraction unit 120 based on the pattern information, and pixels for each extracted object. An object that satisfies the condition set in the pattern information may be detected by calculating a number change or a moving direction. Accordingly, the object tracking unit 130 may determine the detected object as the smoke 4. Detailed description thereof will be described with reference to FIG. 4.

On the other hand, when the object tracking unit 130 detects an object that satisfies the conditions set in the pattern information, it is determined as the smoke (4), and outputs the output of the forest fire through the output unit 150. have. In this case, the output unit 150 may include various output devices including a speaker, a display, and the like.

In addition, when there is no object that satisfies the pattern information in the foreground image, the object extractor 120 may not transmit data to the object tracking unit 130, thereby preventing unnecessary operations from occurring.

4 is a diagram illustrating a processing procedure of a forest fire monitoring apparatus using a thermal imaging camera according to an embodiment of the present invention. Referring to FIG. 4 with reference to the configuration of FIG. 3, the thermal imaging camera unit is illustrated. The foreground / background separator 110 receiving the thermal image data ① provided from (10) separates the foreground image ③ and the background image ② from each thermal image data according to the long-term background modeling method.

Thereafter, the object extractor 120 may extract an object satisfying the temperature range set for the smoke 4 from the foreground image with reference to the pattern information. In this case, the object extractor 120 may extract and label each object satisfying the temperature range set in the pattern information by ID, thereby clarifying object classification.

Meanwhile, in the process of extracting an object from the foreground image, the temperature of the cloud 2 or the fog 3 temporarily increases due to the external heat, or the temperature of the vehicle 5 temporarily rises by the engine heat, so that the temperature range of the pattern information is increased. If belonging, objects other than the deferral 4 may be extracted as an object such as the deferral 4.

Therefore, in order to detect the object for smoke 4 among the objects extracted by the object extraction unit 120, the object tracking unit 130 takes into account the information on the movement direction or diffusion set in the pattern information. Can detect objects for smoke (④).

In detail, as described above, since the object for the smoke moves in the upward direction or the diagonal upward direction unlike other objects, the object tracking unit 130 is an object extracted by being labeled in a continuous foreground image. It analyzes the motion vectors of the pixels constituting the extracted object by tracking each time to determine the moving direction.If the moving direction of the object satisfies the upward direction or diagonal diagonal direction according to the pattern information, the object is judged as smoke and the object is determined. It can be detected (⑤).

That is, as shown, since the moving direction of the object such as the cloud 2, the fog 3, or the vehicle 5 is left and right, it can be clearly distinguished from the smoke 4, which is an object moving upward.

On the other hand, in addition to the consideration of the moving direction, the object for the smoke (4) has a property that is spread unlike other objects. Accordingly, the object tracking unit 130 detects a change in the number of pixels by tracking an object that is labeled and extracted through the object extraction unit 120 in a continuous foreground image, and determines that the object whose number of pixels increases is smoke. can do. That is, the increase in the number of pixels may be determined as diffusion, which is an attribute of smoke, to distinguish it from other objects.

In this case, as shown in FIG. 5A, the object tracking unit 130 performs a difference (pixel number change) between the number of pixels of the previous foreground image and the number of pixels of the current foreground image during a preset time interval Ts-T1. The diffusion rate can be obtained by calculating (Ps-P1), and it can be determined as the smoke (4) when it is equal to or greater than the predetermined value (Y) by comparing it with the value (Y) included in the pattern information.

In other words, in order to distinguish clouds that develop in the upward direction, such as cumulus clouds, from the smoke 4, the cloud diffusion speed is significantly lower than that of the smoke, so that the cloud of rapid diffusion 4 is diffused upward. It can be easily distinguished from objects such as

In this case, the object tracking unit 130 may generate a bounding box 200 for each of the objects labeled from the object extractor 120 in the thermal image data of the thermal camera unit 10, and the bounding box 200. The number of pixels included in the bounding box 200 may be calculated based on the pixel position of 200. Through this, the object tracking unit 130 calculates a change in the number of pixels included in the bounding box 200 of each object for a predetermined time for each object, and compares it with the predetermined value as described above. Delay 4 of the labeled objects can be determined.

After the process as described above, the object tracking unit 130 may finally output the information on the forest fire generation through the output unit 150 when the object for the smoke is detected.

On the other hand, the object tracking unit 130, the object tracking unit 130 as shown in Figure 5 (b) in order to further increase the accuracy of whether the forest fire is determined only by the smoke (4), the brightness information of the selected object By analyzing the information on the occurrence of forest fire may be output only for the object for which the high frequency component by the flame (A) is detected.

That is, the presence of the flame (A) accompanying the smoke (4) at the time of fire can be confirmed through the detection of the high-frequency component, it is possible to further increase the accuracy of whether or not a fire occurs. At this time, since the present invention detects the flame by using a thermal imaging camera, it is possible to detect whether the flame occurs earlier than when using a conventional visible light camera.

In addition, when the object tracking unit 130 is not an object that satisfies the condition included in the pattern information, the object tracking unit 130 transmits the analysis information about the object to the foreground / background separator 110 and the foreground / background separator 110. ) Can background all the objects.

Accordingly, the amount of computation can be greatly reduced by preventing tracking of the same object later.

Meanwhile, the present invention adds a visible light camera unit to obtain an image of the same surveillance area in conjunction with the thermal imaging camera unit 10, the object for the smoke obtained through the image analysis of the thermal imaging camera unit 10 It may be configured to display on the image of the visible light camera.

That is, the object detected in the image of the thermal imaging camera unit 10 may be displayed on the visible light image of the visible light camera unit to identify it, and thus, it may be possible to more clearly identify a point where a fire occurs during the daytime.

To this end, a configuration for outputting the object detected by the thermal imaging camera unit 10 to the visible light camera unit 20 as shown in Figure 6 is shown.

Referring to the configuration illustrated in FIG. 6, the forest fire monitoring apparatus according to the exemplary embodiment of the present invention includes calibration information of the thermal imaging camera unit 10 and the visible light camera unit 20 in the configuration of FIG. 2. The image matching unit 160 may further include an image matching unit 160 that receives the data from the unit 11 and generates matching information by mutually matching the surveillance spaces between the cameras using the calibration information.

Accordingly, the object tracking unit 130 is based on the pattern information among the objects extracted from the thermal image data of the thermal camera unit 10 through the foreground / background separation unit 110 and the object extraction unit 120. The object for smoke is detected, and the analysis information of the object detected through the thermal image data of the thermal imaging camera unit 10 is matched based on the matching information of the image matching unit 160 and the thermal imaging camera unit ( The image may be output to the image of the visible light camera unit 20 at a position corresponding to the position of the detected object in the image of 10).

As an example of this, as shown in FIG. 7, the object tracking unit 130 is based on the pattern information among the objects labeled from the object extraction unit 120 in the thermal image data of the thermal camera unit 10. The bounding box 200 may be generated with respect to the detected object, and a pixel position constituting an edge of the bounding box 200 may be obtained.

Thereafter, the object tracking unit 130 generates a bounding box 220 at a position corresponding to the pixel position in the image of the visible light camera unit 20 based on the matching information of the image matching unit 160, The same object as the object obtained from the thermal image data may be displayed on the image of the visible light camera unit 20.

8 is a flowchart illustrating a forest fire monitoring method of a forest fire monitoring apparatus using a thermal imaging camera using a thermal imaging camera according to an embodiment of the present invention.

Referring to FIG. 8, the foreground / background separation unit includes an object to be monitored according to a long-term background modeling method of the thermal image data received from the thermal imaging camera based on the calibration information received through the calibration unit. A foreground image and a background image except the foreground image may be separated.

Thereafter, the object extracting unit analyzes brightness information according to the temperature of the object included in the separated foreground image and extracts an object belonging to the temperature range included in the pattern information among the objects included in the foreground image.

In this case, the object extracting unit transmits analysis information about an object that does not belong to the temperature range included in the pattern information to the foreground / background separating unit, and the foreground / background separating unit performs background processing on the object for subsequent thermal imaging. You can reduce the amount of computation that results from analyzing your data.

Next, the object tracking unit monitors an object extracted through the object extracting unit from consecutive thermal image data, and detects a forest fire when an object whose pixel number change or moving direction satisfies the pattern information is detected. Information may be output through the output unit.

In this case, the object tracking unit determines whether the moving direction is upward when detecting the object based on the moving direction, and calculates the change in the number of pixels for a preset time when detecting the object based on the change in the number of pixels. Is determined to be equal to or greater than a preset reference value, and if the corresponding condition is satisfied, the fire alarm notification information may be output by judging it as a postponement due to a fire.

On the other hand, if the object does not satisfy the above conditions, as described above, all of the background processing to prevent the tracking of the same object in the future, it is possible to significantly reduce the amount of computation.

1: surveillance zone 2: clouds
3: fog 4: smoke
5: automobile 10: thermal imaging camera unit
11: calibration part 20: visible light camera part
100: forest fire monitoring device 110: foreground / background separator
120: object extraction unit 130: object tracking unit
140: pattern storage unit 150: output unit
160: image matching unit 200: bounding box

Claims (12)

A foreground / background separation unit which receives continuous thermal image data from a thermal imager and separates each thermal image into a foreground image and a background image according to a long term background modeling method;
A pattern storage unit including a temperature range of smoke generated by a forest fire and pattern information considering a moving direction or diffusion;
An object extracting unit extracting an object included in a temperature range included in the pattern information based on brightness information according to a temperature of an object included in the foreground image separated from the foreground / background separating unit; And
Object tracking that monitors an object extracted by the object extraction unit from consecutive thermal image data and outputs information on wildfire occurrence when an object whose pixel number change or movement direction of the object satisfies the pattern information is detected. part
Forest fire monitoring apparatus using a thermal imaging camera comprising a.
The method according to claim 1,
The object extracting unit is a forest fire monitoring apparatus using a thermal imaging camera for labeling by giving an ID for each object included in a predetermined temperature range.
The method according to claim 1,
And the object tracking unit analyzes a moving direction of the extracted object and detects an object moving in an upward direction or a diagonal upward direction according to the pattern information.
The method according to claim 1,
And the object tracking unit detects an object having a spreading speed of calculating a change amount of the number of pixels for a predetermined time among the extracted objects more than a value included in the pattern information.
The method according to claim 1,
And the object tracking unit analyzes brightness information of the selected object and outputs information on wildfire generation only to an object for which a high frequency component is detected.
The method according to claim 1,
And an image matching unit for generating matching information by mutually matching surveillance spaces between cameras using calibration information of the thermal imaging camera and the visible light camera.
The object tracking unit matches analysis information of the detected object based on matching information of the image matching unit and outputs the image to the visible light camera at a position corresponding to the position of the detected object in the image of the thermal imaging camera. Forest fire monitoring device using a thermal imaging camera characterized in that.
The method according to claim 1,
The object extracting unit and the object tracking unit transmit analysis information about the remaining objects except at least one or more objects extracted or detected based on the pattern information to the foreground / background separation unit, and the foreground / background separation unit based on the analysis information. Forest fire monitoring apparatus using a thermal imaging camera, characterized in that for processing the remaining objects in the background.
In the forest fire monitoring method of a forest fire monitoring device using a thermal imaging camera,
A first step of receiving continuous thermal image data from a thermal imager and separating each thermal image into a foreground image and a background image according to a long term background modeling method;
Analyzing brightness information according to the temperature of the object included in the foreground image separated through the first step, the temperature range for the smoke generated by the forest fire and the temperature range included in the pattern information considering the moving direction or diffusion. Extracting an included object; And
A third step of monitoring an object extracted from continuous thermal image data through the second step, and outputting information on a wildfire occurrence when an object whose pixel number change or movement direction of the object satisfies the pattern information is detected;
Forest fire monitoring method using a thermal imaging camera comprising a.
The method according to claim 8,
The second step is to label by giving the ID to each object included in the preset temperature range,
The third step is a forest fire monitoring method using a thermal imaging camera, characterized in that for tracking each labeled object.
The method according to claim 8,
The third step is a forest fire monitoring method using a thermal imaging camera, characterized in that for analyzing the moving direction of the extracted object, detecting the object moving in the upward or diagonal upward direction according to the pattern information.
The method according to claim 8,
The third step is a forest fire monitoring method using a thermal imaging camera, characterized in that for detecting the object whose diffusion rate calculated by the amount of change of the number of pixels for a predetermined time of the extracted object is greater than the value included in the pattern information.
The method according to claim 8,
The third step may further include detecting high frequency components by analyzing brightness information of the selected object.
Forest fire monitoring method using a thermal imaging camera, characterized in that for outputting information about the occurrence of forest fires only for the object where the high frequency component is detected.

Images