CN116153016B - Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof - Google Patents

Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof Download PDF

Info

Publication number
CN116153016B
CN116153016B CN202310436213.8A CN202310436213A CN116153016B CN 116153016 B CN116153016 B CN 116153016B CN 202310436213 A CN202310436213 A CN 202310436213A CN 116153016 B CN116153016 B CN 116153016B
Authority
CN
China
Prior art keywords
image
assembly
real
target area
thermal imaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310436213.8A
Other languages
Chinese (zh)
Other versions
CN116153016A (en
Inventor
郝纯
蒋先勇
李志刚
魏长江
李财
胡晓晨
税强
曹尔成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Sanside Technology Co ltd
Original Assignee
Sichuan Sanside Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Sanside Technology Co ltd filed Critical Sichuan Sanside Technology Co ltd
Priority to CN202310436213.8A priority Critical patent/CN116153016B/en
Publication of CN116153016A publication Critical patent/CN116153016A/en
Application granted granted Critical
Publication of CN116153016B publication Critical patent/CN116153016B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00Fire alarms; Alarms responsive to explosion
    • G08B17/12Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00Fire alarms; Alarms responsive to explosion
    • G08B17/10Actuation by presence of smoke or gases, e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B17/00Fire alarms; Alarms responsive to explosion
    • G08B17/12Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
    • G08B17/125Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
    • HELECTRICITY
    • H02GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02JCIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J7/00Circuit arrangements for charging or depolarising batteries or for supplying loads from batteries
    • H02J7/34Parallel operation in networks using both storage and other dc sources, e.g. providing buffering
    • H02J7/35Parallel operation in networks using both storage and other dc sources, e.g. providing buffering with light sensitive cells
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
    • Y02A40/28Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture specially adapted for farming

Landscapes

  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Power Engineering (AREA)
  • Radiation Pyrometers (AREA)
  • Fire-Detection Mechanisms (AREA)

Abstract

The invention discloses a multi-sensor fusion forest fire real-time monitoring and early warning device and a method thereof, wherein the device comprises the following components: the thermal imaging assembly, the image acquisition assembly and the air detection assembly are deployed in a forest area to be monitored; a processing module deployed at the monitoring station; the method comprises the following steps: s1, judging whether a forest area is abnormal or not through a thermal imaging assembly and an image acquisition assembly; s2, judging whether the forest area is abnormal or not through an air detection assembly; s3, outputting a low risk alarm if the abnormality exists in the S1 or the abnormality exists in the S2; if S1 is abnormal and S2 is abnormal, outputting a high risk alarm; the invention adopts three different sensors of the thermal imaging assembly, the image acquisition assembly and the air detection assembly to monitor and fuse multiple sensors, so that mutual verification and mutual cooperation can be realized, and the detection accuracy and reliability can be improved.

Description

Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof
Technical Field
The invention relates to the field of monitoring of the Internet of things, in particular to a multi-sensor fusion forest fire real-time monitoring and early warning device and a method thereof.
Background
Forest fires are a common natural disaster, and easily cause huge property loss and casualties to human society, so that the fire must be extinguished in time, and the best time for extinguishing the fire is in the initial stage of the fire, so that the fire monitoring for forests is very important.
The existing forest fire detection generally adopts manual inspection or unmanned aerial vehicle inspection, has the problems of low efficiency, limited range, incapability of covering comprehensively and the like, cannot meet the large-scale monitoring requirement, and has poor monitoring effect especially at night and in complex environments.
The existing thermal imaging technology is adopted more: the thermal imaging technology is a technology for detecting and imaging the temperature distribution of the surface of an object through an infrared imager, and can be used for real-time monitoring and early warning of forest fires. However, thermal imaging techniques may have defects in complex contexts (such as occlusion of branches, leaves, etc.), and it is difficult to accurately determine whether the target is a fire.
Disclosure of Invention
The invention aims to provide a multi-sensor fusion forest fire real-time monitoring and early warning device and a method thereof, which can timely find forest fires and send early warning at the initial stage of the fires.
The invention is realized by the following technical scheme:
in a first aspect, a multi-sensor fusion forest fire real-time monitoring and early warning device includes: the thermal imaging assembly, the image acquisition assembly and the air detection assembly are deployed in a forest area to be monitored; a processing module deployed at the monitoring station;
the thermal imaging assembly, the image acquisition assembly and the air detection assembly are in communication connection with the processing module through a wireless communication technology;
the thermal imaging assembly, the image acquisition assembly and the air detection assembly are all connected with a power supply assembly, and the power supply assembly comprises: the solar energy assembly is electrically connected with the storage battery, the wind energy assembly is electrically connected with the storage battery, the storage battery is electrically connected with the thermal imaging assembly, the image acquisition assembly and the air detection assembly, and the storage battery supplies power for the thermal imaging assembly, the image acquisition assembly and the air detection assembly.
Specifically, the thermal imaging assembly and the image acquisition assembly are both arranged in a camera, the camera is deployed in a forest area, and the camera is configured with a visible light channel and an infrared channel;
the air detection assembly comprises an oxygen detection assembly, a carbon monoxide detection assembly and a carbon dioxide detection assembly, and is deployed in a forest area.
In a second aspect, a multi-sensor fusion forest fire real-time monitoring and early warning method is based on the multi-sensor fusion forest fire real-time monitoring and early warning device, and the method comprises the following steps:
s1, judging whether a forest area is abnormal or not through a thermal imaging assembly and an image acquisition assembly;
s2, judging whether the forest area is abnormal or not through an air detection assembly;
s3, outputting a low risk alarm if the abnormality exists in the S1 or the abnormality exists in the S2; if there is an abnormality in S1 and an abnormality in S2, a high risk alarm is output.
Specifically, step S1 specifically includes:
s11, monitoring a forest area in real time through a thermal imaging assembly, and acquiring a target area;
s12, calculating the heat radiation flux of a target area, and simultaneously setting a first threshold and a second threshold, wherein the first threshold is a normal heat radiation flux threshold, and the second threshold is a fire heat radiation flux threshold;
s13, if the heat radiation flux of the target area is larger than a second threshold value, judging that the target area is abnormal; if the heat radiation flux of the target area is between the first threshold value and the second threshold value, judging that the target area is in a pending state; if the heat radiation flux of the target area is smaller than a first threshold value, judging that the target area is normal;
s14, if the target area is in a pending state, starting an image acquisition assembly, and acquiring a real-time image of the target area through the image acquisition assembly; acquiring an infrared image of the target area through a thermal imaging assembly;
s15, fusing the real-time image and the infrared image, outputting the fused image to be judged, and judging whether the image to be judged is abnormal or not by manpower.
Further, after step S15, outputting an early warning signal that needs to be manually judged;
s16, if the manual judgment is carried out, determining whether an abnormality exists according to the manual judgment result; if the manual judgment is not performed within the set time, determining that the abnormality exists.
Specifically, the specific method of step S11 includes:
acquiring an infrared image through a thermal imaging assembly, and extracting an interested region in the infrared image, wherein the interested region is a temperature abnormal region;
graying the region of interest to obtain a gray image, wherein i=0.299g+0.587r+0.114 b, I is a gray value, and R, G, B is pixel values of three channels of RGB respectively;
setting a binarization threshold, binarizing the gray level image to obtain a binary image, and carrying out morphological processing on the binary image to obtain a processed image;
acquiring all communication areas, wherein the communication areas are communicated foreground pixel areas;
calculating geometrical properties and gray scale properties corresponding to the connected region, and setting screening conditions according to priori knowledge;
and marking the markers meeting the screening conditions as target areas and marking the markers not meeting the screening conditions as non-target areas.
Specifically, in step S12, the method for calculating the heat radiation flux of the target area includes:
acquiring a gray image of a target area;
calculating average gray value of target region:/>Wherein N is the number of pixel points in the target area, < >>The gray value of the ith pixel point;
determining magnification factor of thermal imaging assemblyAnd offset coefficient->And calculates the kelvin temperature T of the target area: />Wherein->For the radiation intensity->For the surface emissivity of the target area, < >>Is Planck's radiation constant, +.>Is Planck's displacement constant;
the heat radiation flux Q of the target area is acquired,wherein->For the Stoffal-Boltzmann constant, S is the total area of the target region.
Specifically, the method for acquiring the fusion image in step S15 includes:
s151, acquiring an infrared image and a real-time image of a target area, and graying the infrared image and the real-time image; determining an average gray value of an infrared imageAnd average gray value of real-time image +.>
S152, respectively converting the infrared image and the real-time image:
performing non-subsampled wavelet transform on the image to decompose the image into a low frequency subband and a high frequency subband;
performing a non-subsampling directional filter bank on the high-frequency sub-bands to obtain a plurality of high-frequency sub-bands with different directional characteristics;
s153, carrying out low-frequency fusion on a low-frequency sub-band of the infrared image and a low-frequency sub-band of the real-time image:wherein->For the fused low frequency subband +.>Is a low frequency subband of the infrared image, < >>Low frequency subband for real time image +.>Is a Laplace convolution kernel;
s154, high-frequency fusion is carried out on a high-frequency sub-band of the infrared image and a high-frequency sub-band of the real-time image:,/>,/>
wherein,,for the fused high frequency subband +.>Gray value for the i-th pixel in the real-time image,>gray value of ith pixel point in infrared image, < >>For the pixel sharpness of the i-th pixel point in the real-time image,the pixel definition of the ith pixel point in the infrared image;
s155, performing the inverse transformation of the step S152And->And obtaining an image to be judged after fusion.
Optionally, the air detection assembly used in step S2 includes: the laser device comprises a laser generator, a laser receiver and an air cavity, wherein the laser generator and the laser receiver are respectively arranged at two sides of the air cavity, and emitted laser of the laser generator passes through the air cavity and is received by the laser receiver;
the laser generator and the laser receiver are electrically connected with the processing module, and the processing module is used for adjusting the laser wavelength and the emission intensity of the laser generator.
Specifically, the specific steps of step S2 include:
determining the incident intensity of a laser by means of a laser generatorThe method comprises the steps of carrying out a first treatment on the surface of the Determination of the transmitted intensity of a laser light by a laser receiver
Obtaining absorbanceObtaining the light absorption coefficient K of the gas to be detected;
the concentration of the measured gas is calculated and obtained,wherein L is the light absorption optical path in the air cavity;
and setting a concentration threshold value, and judging whether the gas is normal or not according to the relation between the concentration threshold value and the concentration of the detected gas.
Compared with the prior art, the invention has the following advantages and beneficial effects:
the invention adopts three different sensors of the thermal imaging assembly, the image acquisition assembly and the air detection assembly to monitor and fuse multiple sensors, so that mutual verification and mutual cooperation can be realized, and the detection accuracy and reliability can be improved.
The wireless communication technology is adopted, wireless data transmission between the sensor and the processing module is realized, remote monitoring and control are realized, and convenience and rapidness are realized.
The scheme adopts green energy sources such as a storage battery, a solar energy component and/or a wind energy component to supply power, has the advantages of environmental protection and energy saving, and can ensure long-time stable operation of the sensor.
The forest fire monitoring and early warning system can realize real-time monitoring and early warning of forest fires, and timely find abnormal conditions and respond correspondingly, so that occurrence and diffusion of the fires are effectively avoided and reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the principles of the invention.
Fig. 1 is a schematic flow chart of a multi-sensor fusion forest fire real-time monitoring and early warning method according to the invention.
Fig. 2 is a schematic flow chart of a third embodiment according to the present invention.
Description of the embodiments
The present invention will be described in further detail with reference to the drawings and embodiments, for the purpose of making the objects, technical solutions and advantages of the present invention more apparent. It is to be understood that the specific embodiments described herein are merely illustrative of the substances, and not restrictive of the invention.
It should be further noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
Embodiments of the present invention and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Example 1
A multi-sensor fusion forest fire real-time monitoring and early warning device comprises: the thermal imaging assembly, the image acquisition assembly and the air detection assembly are deployed in a forest area to be monitored; a processing module deployed at the monitoring station;
the thermal imaging assembly and the image acquisition assembly are both arranged in the camera, and the camera is deployed in a forest area, namely the camera is provided with a visible light channel and an infrared channel; the camera and the air detection assembly are in communication connection with the processing module through a wireless communication technology;
the thermal imaging assembly is responsible for capturing infrared thermal images and can detect temperature anomalies in the event of a fire. The image acquisition component is responsible for capturing real-time images and providing actual pictures of the scene, and can manually judge whether the scene is abnormal or not. The air detection assembly evaluates the likelihood of a fire occurring by detecting certain gas concentrations (e.g., carbon monoxide and carbon dioxide) in the air.
The thermal imaging subassembly, image acquisition subassembly and air detection subassembly all are connected with power supply unit, and power supply unit includes: the storage battery, solar module and/or wind energy component, solar module and battery electricity are connected, and wind energy component is connected with the battery electricity, and the battery is connected with thermal imaging subassembly, image acquisition subassembly and air detection subassembly electricity, and the battery is thermal imaging subassembly, image acquisition subassembly and air detection subassembly power supply.
To power these sensor assemblies, they are all connected with a power supply assembly. The power supply assembly includes a battery, a solar assembly, and/or a wind energy assembly. The solar energy component and the wind energy component charge the storage battery through solar energy and wind energy respectively, and then the storage battery provides power for the thermal imaging component, the image acquisition component and the air detection component. In this way, the monitoring system can continue to operate steadily even in remote forest areas where no grid access is available.
The thermal imaging assembly can more widely monitor the temperature conditions of the forest, detect the temperature by capturing infrared radiation, and quickly discover and report to the processing module when the temperature of an area in the forest abnormally increases.
The image acquisition component is responsible for capturing real-time images, which can acquire live-action images of forests in real time. When the processing module receives an abnormality report of the thermal imaging assembly, the camera can provide a scene picture to assist in judging whether a fire disaster occurs.
The air detection assembly comprises an oxygen detection assembly, a carbon monoxide detection assembly and a carbon dioxide detection assembly, and is deployed in a forest area. These components evaluate the likelihood of a fire occurrence by measuring the concentration of gas in the forest. During a fire, oxygen is consumed and carbon monoxide and carbon dioxide concentrations increase significantly. By monitoring these changes in gas concentration, the air detection assembly can detect a fire and send an alarm to the process module.
By combining the data of these three components, the processing module can more accurately determine the probability of a fire occurring. In practical application, the deployment and the number of the components can be adjusted according to the characteristics of different forest areas so as to achieve a better monitoring effect. Meanwhile, other sensors (such as a humidity sensor and a wind speed sensor) can be considered to be incorporated into the monitoring system, so that the accuracy and timeliness of fire early warning are further improved.
Example two
The multi-sensor fusion forest fire real-time monitoring and early warning method is based on the multi-sensor fusion forest fire real-time monitoring and early warning device, as shown in fig. 1, and comprises the following steps:
s1, judging whether a forest area is abnormal or not through a thermal imaging assembly and an image acquisition assembly; the thermal imaging assembly is responsible for capturing infrared radiation and monitoring the temperature change of the forest area. The image acquisition component is responsible for acquiring real-time images and displaying forest scenes. The two work cooperatively to judge whether the forest area has abnormality, such as the sign of fire occurrence.
S2, judging whether the forest area is abnormal or not through an air detection assembly; the air detection assembly measures the concentration of oxygen, carbon monoxide and carbon dioxide in the forest area. When a fire occurs, oxygen consumption increases and carbon monoxide and carbon dioxide concentrations rise. By monitoring these changes in gas concentration, it can be determined whether or not an abnormality exists in the forest area.
S3, outputting a low-risk alarm if the abnormality exists in the S1 or the abnormality exists in the S2, which indicates that a certain fire risk exists; if the abnormality exists in S1 and S2, which means that the risk of fire is high, a high risk alarm is output.
By the multi-sensor fusion method, the probability of forest fire occurrence can be judged more accurately, and corresponding alarms are output according to the risk degree, so that the forest fire can be found and extinguished in time.
Example III
In this embodiment, step S1 in the second embodiment is specifically described, as shown in fig. 2, and the specific steps include:
s11, monitoring a forest area in real time through a thermal imaging assembly, and acquiring a target area; the thermal imaging assembly is responsible for capturing infrared radiation, monitoring a forest area in real time, and acquiring a target area which may have an abnormality according to the infrared radiation. The thermal imaging assembly captures infrared radiation of the target area using a thermal infrared imager. This enables the system to monitor forest areas in real time, such as night or heavy fog weather, without being affected by lighting conditions. The purpose of acquiring the target area is to determine areas where there may be a risk of fire, thereby focusing limited resources on critical areas.
S12, calculating the heat radiation flux of a target area, wherein the heat radiation flux is the transmission rate of heat energy in the target area, and simultaneously setting a first threshold and a second threshold, the first threshold is a normal heat radiation flux threshold, and the second threshold is a fire heat radiation flux threshold; by calculating the heat radiation flux, the temperature change of the target area and the risk of fire can be evaluated. Setting the first threshold and the second threshold helps to determine the state of the target area, thereby avoiding false positives and false negatives.
S13, if the heat radiation flux of the target area is larger than a second threshold value, judging that the target area is abnormal; if the heat radiation flux of the target area is between the first threshold value and the second threshold value, judging that the target area is in a pending state; if the heat radiation flux of the target area is smaller than a first threshold value, judging that the target area is normal;
s14, if the target area is in a pending state, starting an image acquisition assembly, and acquiring a real-time image of the target area through the image acquisition assembly; acquiring an infrared image of the target area through a thermal imaging assembly; in the event that the status of the target area is uncertain, the image acquisition assembly is activated to acquire real-time images, which helps to provide more information to more accurately determine the status of the target area. The combination of the infrared image and the real-time image can effectively reduce the false alarm rate and unnecessary alarms.
S15, fusing the real-time image and the infrared image, outputting the fused image to be judged, and judging whether the image to be judged is abnormal or not by manpower. In the event that the status of the target area is uncertain, the image acquisition assembly is activated to acquire real-time images, which helps to provide more information to more accurately determine the status of the target area. The combination of the infrared image and the real-time image can effectively reduce the false alarm rate and unnecessary alarms.
After the step S15 is carried out, outputting an early warning signal which needs to be judged manually; that means, if the system is still unable to determine whether a fire is present in the target area after the multisensor fusion, then manual intervention is required to make a more accurate determination.
S16, if the manual judgment is carried out, determining whether an abnormality exists according to the manual judgment result; if the manual judgment is not performed within the set time, determining that the abnormality exists.
After receiving the early warning signal, the human judgment personnel analyzes the fused image to be judged according to the actual situation. The human judgment staff can be staff of the monitoring station or professionals of other related departments, and the staff has rich practical experience and fire disaster recognition capability. After the manual determination is made, they make a determination and return the result to the processing module. The processing module determines whether an abnormality exists, i.e. whether a fire occurs, according to the result of the manual judgment.
In practice, the manual judgment may be limited by time and human resources. To ensure that the response is possible as soon as possible in an emergency situation, the system sets a time threshold. If the manual judgment is not carried out within the set time, the processing module is used for judging that the default target area is abnormal. The purpose of this is to prevent fire from spreading due to delays in manual judgment, and to take countermeasures as soon as possible. Of course, in practical application, the specific value of the set time threshold needs to be adjusted according to the actual situation and the requirement.
The following describes a specific method of step S11, which includes:
s111, acquiring an infrared image through a thermal imaging assembly, and extracting an area of interest in the infrared image, wherein the area of interest is a temperature abnormal area;
s112, graying the region of interest to obtain a gray image, wherein I=0.299G+0.587R+0.114B, wherein I is a gray value, and R, G, B is pixel values of three RGB channels respectively;
s113, setting a binarization threshold, binarizing the gray level image to obtain a binary image, and carrying out morphological processing on the binary image to obtain a processed image; by setting an appropriate threshold, we can convert the gray image into a binary image containing only two pixel values, black and white. Then, morphological processing such as erosion, dilation, open and close operations, etc. are performed on the binary image to remove noise and smooth the target edge.
S114, acquiring all the connected areas, wherein the connected areas are connected foreground pixel areas; a connected region refers to a set of pixels in a binary image that have the same pixel value (e.g., foreground pixels, typically 1 or 255) and are spatially connected to each other. Here, the foreground pixels generally represent regions of higher temperature, possibly a source of fire.
a. Each pixel of the binary image is traversed.
b. When a foreground pixel is found (e.g., having a value of 1 or 255), a connected region search is performed starting from the pixel. Depth-first search (DFS) or breadth-first search (BFS) algorithms may be used to find all foreground pixels adjacent to the current pixel. The definition of adjacent may be a 4 neighborhood (up, down, left, right) or an 8 neighborhood (including diagonal directions).
c. The found neighboring foreground pixels are classified as a connected region and removed or marked as accessed from the original image.
d. Continuing to traverse the binary image, repeating steps b and c until all pixels have been accessed.
S115, calculating geometrical properties and gray scale properties corresponding to the connected region, and setting screening conditions according to priori knowledge; for each connected region, we calculate its geometric properties (e.g., area, perimeter, etc.) and gray scale properties (e.g., gray scale mean, gray scale variance, etc.). Based on known fire characteristics and prior knowledge, we set a series of screening conditions to further distinguish between target and non-target areas.
S116, marking the mark meeting the screening condition as a target area, and marking the mark not meeting the screening condition as a non-target area. The connected areas satisfying the screening conditions are considered as potential sources of fire, and are marked as target areas. These target areas require further analysis and monitoring. The connected region that does not satisfy the screening condition is considered normal, and is marked as a non-target region without special attention.
For the step S12, the method for calculating the heat radiation flux of the target area includes:
s121, acquiring a gray image of the target area.
S122, calculating the average gray value of the target area:/>Wherein N is the number of pixel points in the target area, < >>The gray value of the ith pixel point; in the infrared image acquired by the thermal imaging assembly, the gray value of each pixel represents the relative magnitude of the thermal radiation intensity. The higher the gray value, the greater the heat radiation intensity of the pixel point, and the temperature is relatively higher.
S123, determining the amplification factor of the thermal imaging assemblyAnd offset coefficient->And calculates the kelvin temperature T of the target area: />Wherein->For the radiation intensity->For the surface emissivity of the target area, < >>Is Planck's radiation constant, +.>Is Planck's displacement constant;
the magnification factor and the offset factor of the thermal imager are two parameters obtained from the characteristics of the thermal imager and the calibration data. They are used to convert the gray values into the actual radiation intensity.
The radiation intensity represents the thermal radiation energy emitted by the surface of the object. The gray value can be converted into the radiation intensity by the amplification factor and the offset factor.
S124, acquiring the heat radiation flux Q of the target area,wherein->For the Stoffal-Boltzmann constant, S is the total area of the target region.
The acquisition method for the fusion image in step S15 includes:
s151, acquiring an infrared image and a real-time image of a target area, and graying the infrared image and the real-time image; determining an average gray value of an infrared imageAnd average gray value of real-time image +.>
S152, respectively converting the infrared image and the real-time image:
performing non-subsampled wavelet transform on the image, and decomposing the image into a low-frequency subband (containing rough information of the image) and a high-frequency subband (containing detailed information of the image); the low frequency sub-band contains most of the energy of the image, representing the overall structural information of the image; the high frequency sub-bands contain detailed information of the image such as edges and textures.
Performing a non-subsampling directional filter bank on the high-frequency sub-bands to obtain a plurality of high-frequency sub-bands with different directional characteristics; detail information of different directions in the image can be analyzed and retained.
S153, carrying out low-frequency fusion on a low-frequency sub-band of the infrared image and a low-frequency sub-band of the real-time image:wherein->For the fused low frequency subband +.>Is a low frequency subband of the infrared image, < >>Low frequency subband for real time image +.>Is a Laplace convolution kernel; and fusing the integral structure information of the two images so as to simultaneously embody the basic forms of the infrared image and the real-time image in the fused image. The fused image has the heat radiation information of the infrared image and the visible light information of the real-time image.
S154, high-frequency fusion is carried out on a high-frequency sub-band of the infrared image and a high-frequency sub-band of the real-time image:,/>,/>
wherein,,for the fused high frequency subband +.>Gray value for the i-th pixel in the real-time image,>gray value of ith pixel point in infrared image, < >>For the pixel sharpness of the i-th pixel point in the real-time image,the pixel definition of the ith pixel point in the infrared image;
and fusing the detail information of the two images so as to simultaneously reflect the characteristics of the infrared image, the edge, the texture and the like of the real-time image in the fused image.
S155, performing the inverse transformation of the step S152And->And obtaining an image to be judged after fusion. And after the low-frequency sub-band and the high-frequency sub-band are fused, the spatial resolution of the original image is restored through inverse transformation. Thus, the image to be judged can be obtained, wherein the image to be judged contains fusion information of the infrared image and the real-time image.
Example IV
The air detection assembly used in step S2 includes: the laser generator, the laser receiver and the air cavity are respectively arranged at two sides of the air cavity, and emitted laser of the laser generator passes through the air cavity and is received by the laser receiver;
the laser generator and the laser receiver are electrically connected with the processing module, and the processing module is used for adjusting the laser wavelength and the emission intensity of the laser generator.
The laser generator is responsible for emitting laser light that is capable of penetrating the gas within the air cavity at a specific wavelength and emission intensity. As the laser passes through the air chamber, the gas therein absorbs the laser light, thereby changing the intensity of the laser light. The laser receiver is positioned on the other side of the air chamber for receiving laser light passing through the air chamber.
The processing module is electrically connected with the laser generator and the laser receiver. It is responsible for adjusting the laser wavelength and emission intensity of the laser generator in order to obtain an optimal detection effect under different measurement conditions. The processing module can calculate the concentration of various gases in the air cavity by measuring the intensity change of the laser after passing through the air cavity, so as to judge whether abnormal gas components exist in the forest area.
The specific steps of the step S2 include:
determining the incident intensity of a laser by means of a laser generatorThe method comprises the steps of carrying out a first treatment on the surface of the The laser generator emits a beam of laser light whose incident intensity is indicative of the initial intensity of the laser light before it enters the air cavity.
Determination of the transmitted intensity of a laser light by a laser receiverThe method comprises the steps of carrying out a first treatment on the surface of the After passing through the gas in the air chamber, the intensity of the laser changes due to the absorption of the gas. The laser receiver measures the laser intensity after absorption by the gas, i.e. the transmitted intensity.
Obtaining absorbanceObtaining the light absorption coefficient K of the gas to be detected; the absorbance coefficient (K) is a physical parameter of a particular gas at a particular wavelength, which is indicative of the absorption capacity of the gas for light, and can be obtained from literature or experimental measurements.
The concentration of the measured gas is calculated and obtained,where L is the light absorption path within the air cavity.
After the measured gas concentration is calculated, it is compared with a preset concentration threshold. This threshold is typically set according to safety standards for the gas or environmental requirements. By comparing the concentration threshold value with the measured gas concentration, the method can judge whether the gas state in the forest area is normal or not, so as to evaluate the fire risk.
Carbon monoxide (CO): carbon monoxide is a colorless and odorless toxic gas, and a large amount of carbon monoxide is generated in the fire process. Typically, if the carbon monoxide concentration in the air exceeds 50ppm (i.e., 50 mg/cubic meter), there may be a fire hazard.
Carbon dioxide (CO 2): carbon dioxide is a colorless and odorless gas and normally has a carbon dioxide concentration in air of about 400ppm. During a fire, the concentration of carbon dioxide may rise significantly. In general, if the carbon dioxide concentration in the air exceeds 1000ppm, there may be a fire hazard.
Oxygen (O2): oxygen is one of the main components in air, and the oxygen concentration in air is normally about 20.9%. During a fire, the concentration of oxygen may decrease. Typically, if the oxygen concentration in the air is below 19.5%, fire hazards may exist.
Example five
A multi-sensor fusion forest fire real-time monitoring and early warning terminal comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the steps of the antenna interface unit testing method when executing the computer program.
The memory may be used to store software programs and modules, and the processor executes various functional applications of the terminal and data processing by running the software programs and modules stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an execution program required for at least one function, and the like.
The storage data area may store data created according to the use of the terminal, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
A computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the steps of the multi-sensor fusion forest fire real-time monitoring and early warning method when being executed by a processor.
Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instruction data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The above-described system memory and mass storage devices may be collectively referred to as memory.
In the description of the present specification, reference to the terms "one embodiment/manner," "some embodiments/manner," "example," "specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment/manner or example is included in at least one embodiment/manner or example of the present application. In this specification, the schematic representations of the above terms are not necessarily for the same embodiment/manner or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments/modes or examples. Furthermore, the various embodiments/modes or examples described in this specification and the features of the various embodiments/modes or examples can be combined and combined by persons skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless explicitly defined otherwise.
It will be appreciated by persons skilled in the art that the above embodiments are provided for clarity of illustration only and are not intended to limit the scope of the invention. Other variations or modifications of the above-described invention will be apparent to those of skill in the art, and are still within the scope of the invention.

Claims (8)

1. The multi-sensor fusion forest fire real-time monitoring and early warning method is characterized by comprising the following steps of: the thermal imaging assembly, the image acquisition assembly and the air detection assembly are deployed in a forest area to be monitored; a processing module deployed at the monitoring station;
the thermal imaging assembly, the image acquisition assembly and the air detection assembly are in communication connection with the processing module through a wireless communication technology;
the thermal imaging assembly, the image acquisition assembly and the air detection assembly are all connected with a power supply assembly, and the power supply assembly comprises: the solar energy assembly is electrically connected with the storage battery, the wind energy assembly is electrically connected with the storage battery, the storage battery is electrically connected with the thermal imaging assembly, the image acquisition assembly and the air detection assembly, and the storage battery supplies power for the thermal imaging assembly, the image acquisition assembly and the air detection assembly;
the method comprises the following steps:
s1, judging whether a forest area is abnormal or not through a thermal imaging assembly and an image acquisition assembly;
s2, judging whether the forest area is abnormal or not through an air detection assembly;
s3, outputting a low risk alarm if the abnormality exists in the S1 or the abnormality exists in the S2; if S1 is abnormal and S2 is abnormal, outputting a high risk alarm;
the step S1 specifically includes:
s11, monitoring a forest area in real time through a thermal imaging assembly, and acquiring a target area;
s12, calculating the heat radiation flux of a target area, and simultaneously setting a first threshold and a second threshold, wherein the first threshold is a normal heat radiation flux threshold, and the second threshold is a fire heat radiation flux threshold;
s13, if the heat radiation flux of the target area is larger than a second threshold value, judging that the target area is abnormal; if the heat radiation flux of the target area is between the first threshold value and the second threshold value, judging that the target area is in a pending state; if the heat radiation flux of the target area is smaller than a first threshold value, judging that the target area is normal;
s14, if the target area is in a pending state, starting an image acquisition assembly, and acquiring a real-time image of the target area through the image acquisition assembly; acquiring an infrared image of the target area through a thermal imaging assembly;
s15, fusing the real-time image and the infrared image, outputting the fused image to be judged, and judging whether the image to be judged is abnormal or not by manpower.
2. The multi-sensor fusion forest fire real-time monitoring and early warning method according to claim 1, wherein after step S15, an early warning signal which needs to be manually judged is output;
s16, if the manual judgment is carried out, determining whether an abnormality exists according to the manual judgment result; if the manual judgment is not performed within the set time, determining that the abnormality exists.
3. The multi-sensor fusion forest fire real-time monitoring and early warning method according to claim 1, wherein the specific method of step S11 comprises:
acquiring an infrared image through a thermal imaging assembly, and extracting an interested region in the infrared image, wherein the interested region is a temperature abnormal region;
graying the region of interest to obtain a gray image, wherein i=0.299g+0.587r+0.114 b, I is a gray value, and R, G, B is pixel values of three channels of RGB respectively;
setting a binarization threshold, binarizing the gray level image to obtain a binary image, and carrying out morphological processing on the binary image to obtain a processed image;
acquiring all communication areas, wherein the communication areas are communicated foreground pixel areas;
calculating geometrical properties and gray scale properties corresponding to the connected region, and setting screening conditions according to priori knowledge;
and marking the markers meeting the screening conditions as target areas and marking the markers not meeting the screening conditions as non-target areas.
4. The method for monitoring and early warning a multi-sensor fused forest fire in real time according to claim 3, wherein in step S12, the method for calculating the heat radiation flux of the target area comprises:
acquiring a gray image of a target area;
calculating average gray value of target region:/>Wherein N is the number of pixel points in the target area, < >>The gray value of the ith pixel point;
determining magnification factor of thermal imaging assemblyAnd offset coefficient->And calculates the kelvin temperature T of the target area:wherein->For the radiation intensity->For the surface emissivity of the target area, < >>Is Planck's radiation constant, +.>Is Planck's displacement constant;
the heat radiation flux Q of the target area is acquired,wherein->For the Stoffal-Boltzmann constant, S is the total area of the target region.
5. The method for monitoring and early warning a multi-sensor fused forest fire in real time according to claim 1, wherein the method for acquiring the fused image in step S15 comprises the following steps:
s151, acquiring an infrared image and a real-time image of a target area, and graying the infrared image and the real-time image; determining an average gray value of an infrared imageAnd average gray value of real-time image +.>
S152, respectively converting the infrared image and the real-time image:
performing non-subsampled wavelet transform on the image to decompose the image into a low frequency subband and a high frequency subband;
performing a non-subsampling directional filter bank on the high-frequency sub-bands to obtain a plurality of high-frequency sub-bands with different directional characteristics;
s153, carrying out low-frequency fusion on a low-frequency sub-band of the infrared image and a low-frequency sub-band of the real-time image:wherein->For the fused low frequency subband +.>Is a low frequency subband of the infrared image, < >>Low frequency subband for real time image +.>Is a Laplace convolution kernel;
s154, high-frequency fusion is carried out on a high-frequency sub-band of the infrared image and a high-frequency sub-band of the real-time image:,/>,/>
wherein,,for the fused high frequency subband +.>Gray value for the i-th pixel in the real-time image,>gray value of ith pixel point in infrared image, < >>For the pixel definition of the ith pixel point in the real-time image, < >>Is an infrared imagePixel definition of the ith pixel point in the image;
s155, performing the inverse transformation of the step S152And->And obtaining an image to be judged after fusion.
6. The multi-sensor fusion forest fire real-time monitoring and early warning method according to claim 1, wherein the air detection assembly used in step S2 comprises: the laser device comprises a laser generator, a laser receiver and an air cavity, wherein the laser generator and the laser receiver are respectively arranged at two sides of the air cavity, and emitted laser of the laser generator passes through the air cavity and is received by the laser receiver;
the laser generator and the laser receiver are electrically connected with the processing module, and the processing module is used for adjusting the laser wavelength and the emission intensity of the laser generator.
7. The multi-sensor fusion forest fire real-time monitoring and early warning method according to claim 6, wherein the specific steps of the step S2 include:
determining the incident intensity of a laser by means of a laser generatorThe method comprises the steps of carrying out a first treatment on the surface of the Determination of the transmission intensity of a laser light by means of a laser receiver>
Obtaining absorbanceObtaining the light absorption coefficient K of the gas to be detected;
the concentration of the measured gas is calculated and obtained,wherein L is the light absorption optical path in the air cavity;
and setting a concentration threshold value, and judging whether the gas is normal or not according to the relation between the concentration threshold value and the concentration of the detected gas.
8. The multi-sensor fusion forest fire real-time monitoring and early warning method according to claim 1, wherein the thermal imaging assembly and the image acquisition assembly are both arranged in a camera, the camera is deployed in a forest area, and the camera is provided with a visible light channel and an infrared channel;
the air detection assembly comprises an oxygen detection assembly, a carbon monoxide detection assembly and a carbon dioxide detection assembly, and is deployed in a forest area.
CN202310436213.8A 2023-04-23 2023-04-23 Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof Active CN116153016B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310436213.8A CN116153016B (en) 2023-04-23 2023-04-23 Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310436213.8A CN116153016B (en) 2023-04-23 2023-04-23 Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof

Publications (2)

Publication Number Publication Date
CN116153016A CN116153016A (en) 2023-05-23
CN116153016B true CN116153016B (en) 2023-07-28

Family

ID=86374027

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310436213.8A Active CN116153016B (en) 2023-04-23 2023-04-23 Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof

Country Status (1)

Country Link
CN (1) CN116153016B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116433035B (en) * 2023-06-13 2023-09-15 中科数创(临沂)数字科技有限公司 Building electrical fire risk assessment prediction method based on artificial intelligence
CN117250319B (en) * 2023-11-14 2024-03-01 北京中科航星科技有限公司 Multi-gas environment unmanned aerial vehicle monitoring method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108378450A (en) * 2018-03-08 2018-08-10 公安部天津消防研究所 A kind of perception of blast accident and risk profile early warning Intelligent fire-fighting helmet implementation method
CN113808378A (en) * 2021-10-25 2021-12-17 应急管理部沈阳消防研究所 Thermal interference resistance testing device and method for image type temperature-sensing fire detector
GB202117938D0 (en) * 2021-08-07 2022-01-26 Kelly Andrew Leslie An intelligent, policy-based computational modelling method & system for building fires

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4023201A (en) * 1976-04-26 1977-05-10 Infrared Surveys, Incorporated Infrared thermography for determining equipment temperatures in oil well fires
RU2276808C2 (en) * 2004-04-29 2006-05-20 Закрытое акционерное общество "НПО Космического Приборостроения" Device for twenty-four-hour detection and monitoring of spread of fire centers in region
US8493212B2 (en) * 2007-06-15 2013-07-23 Icore and Associates, LLC Passive microwave system and method for protecting a structure from fire threats
CN102855726B (en) * 2012-08-25 2017-09-05 镇江市金舟船舶设备有限公司 Visualize phase battle array fire alarm system
US11298575B2 (en) * 2018-04-10 2022-04-12 Flashpoint Fire Equipment, Inc. Systems and methods for training firefighters
CN110898351A (en) * 2018-09-18 2020-03-24 株洲中车时代电气股份有限公司 Electrical cabinet fireproof system and method
FR3087985B1 (en) * 2018-10-31 2023-12-15 Univ De Corse P Paoli DEVICE FOR CHARACTERIZING A FIRE AND ASSOCIATED METHOD FOR DETERMINING RADIATIVE FLOWS
US11521479B2 (en) * 2020-05-08 2022-12-06 Qualcomm Incorporated Fire warning system and devices
CN113283276A (en) * 2020-12-30 2021-08-20 四川弘和通讯有限公司 Linkage thermal imaging self-learning fire point detection method and system
CN113866112A (en) * 2021-09-27 2021-12-31 唐山市智明电子科技有限公司 Portable gas sensing system
CN117079424A (en) * 2022-03-31 2023-11-17 旭宇光电(深圳)股份有限公司 Remote forest fire monitoring and early warning system and method based on photovoltaic cell power supply
CN115762033B (en) * 2022-11-17 2024-04-26 湘潭大学 Forest fire monitoring and responding system based on 5G communication technology

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108378450A (en) * 2018-03-08 2018-08-10 公安部天津消防研究所 A kind of perception of blast accident and risk profile early warning Intelligent fire-fighting helmet implementation method
GB202117938D0 (en) * 2021-08-07 2022-01-26 Kelly Andrew Leslie An intelligent, policy-based computational modelling method & system for building fires
CN113808378A (en) * 2021-10-25 2021-12-17 应急管理部沈阳消防研究所 Thermal interference resistance testing device and method for image type temperature-sensing fire detector

Also Published As

Publication number Publication date
CN116153016A (en) 2023-05-23

Similar Documents

Publication Publication Date Title
CN116153016B (en) Multi-sensor fusion forest fire real-time monitoring and early warning device and method thereof
US11835676B2 (en) Early-warning fire detection system based on a multivariable approach
US11218628B2 (en) Method and system for identifying light source and application thereof
US10914653B2 (en) Infrared imaging systems and methods for oil leak detection
EP2686667B1 (en) Mwir sensor for flame detection
Krstinić et al. Histogram-based smoke segmentation in forest fire detection system
US20160260306A1 (en) Method and device for automated early detection of forest fires by means of optical detection of smoke clouds
Burnett et al. A low-cost near-infrared digital camera for fire detection and monitoring
CN108648400A (en) One kind is based on multispectral transmission line forest fire exploration prior-warning device and method for early warning
KR102585066B1 (en) Combined fire alarm system using stand-alone fire alarm and visible light camera
CN112001327A (en) Valve hall equipment fault identification method and system
CN114664048B (en) Fire monitoring and fire early warning method based on satellite remote sensing monitoring
CN108682105A (en) One kind is based on multispectral transmission line forest fire exploration prior-warning device and method for early warning
US8165340B2 (en) Methods for gas detection using stationary hyperspectral imaging sensors
JP4440010B2 (en) Electrical equipment inspection method and apparatus
CN116307740B (en) Fire point analysis method, system, equipment and medium based on digital twin city
CN117079082A (en) Intelligent visual image target object detection method and device and DMC (digital media control) equipment
CN117037065A (en) Flame smoke concentration detection method, device, computer equipment and storage medium
Rajan et al. Forest Fire Detection Using Machine Learning
KR102174606B1 (en) Sense integrated IP network camera, and monitoring system for the same
Komar et al. Performance of UV and IR Sensors for Inspections of Power Equipment
CN111297336B (en) Body temperature measuring method and device based on infrared and terahertz and security check equipment
KR20210036173A (en) Real time leak gas detection device and method using infrared spectral imaging camera analysis technique
TWI812896B (en) Thermal camera health monitoring
CN116246222A (en) Forest fire detection method based on extremum detection and self-adaptive segmentation algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant