CN113205562B - Mine thermodynamic disaster identification and positioning method based on binocular vision - Google Patents
Mine thermodynamic disaster identification and positioning method based on binocular vision Download PDFInfo
- Publication number
- CN113205562B CN113205562B CN202110597107.9A CN202110597107A CN113205562B CN 113205562 B CN113205562 B CN 113205562B CN 202110597107 A CN202110597107 A CN 202110597107A CN 113205562 B CN113205562 B CN 113205562B
- Authority
- CN
- China
- Prior art keywords
- disaster
- thermodynamic
- model
- mine
- suspected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012544 monitoring process Methods 0.000 claims abstract description 44
- 230000000007 visual effect Effects 0.000 claims abstract description 22
- 239000002131 composite material Substances 0.000 claims description 22
- 239000003245 coal Substances 0.000 claims description 21
- 238000009529 body temperature measurement Methods 0.000 claims description 17
- 230000004927 fusion Effects 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 13
- 230000003068 static effect Effects 0.000 claims description 9
- 230000005641 tunneling Effects 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 6
- 230000005855 radiation Effects 0.000 claims description 6
- 238000009423 ventilation Methods 0.000 claims description 4
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000004891 communication Methods 0.000 description 17
- 238000002834 transmittance Methods 0.000 description 12
- 238000005065 mining Methods 0.000 description 6
- 230000010365 information processing Effects 0.000 description 5
- 238000010521 absorption reaction Methods 0.000 description 4
- 238000004880 explosion Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000000443 aerosol Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 239000007789 gas Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 1
- 208000001408 Carbon monoxide poisoning Diseases 0.000 description 1
- 244000027321 Lychnis chalcedonica Species 0.000 description 1
- 235000017899 Spathodea campanulata Nutrition 0.000 description 1
- 206010052428 Wound Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 229910002091 carbon monoxide Inorganic materials 0.000 description 1
- 239000002817 coal dust Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000035699 permeability Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 231100000331 toxic Toxicity 0.000 description 1
- 230000002588 toxic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Radiation Pyrometers (AREA)
- Geophysics And Detection Of Objects (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a mine thermodynamic disaster identification and positioning method based on binocular vision, which mainly comprises the following steps: calibrating the binocular vision camera; registering and fusing the images; extracting visual features of various images and establishing an identification model; monitoring whether a suspected thermodynamic disaster exists through an identification model; when the existence is identified, starting binocular ranging to locate and compensate temperature; inputting the acquired multi-source information into a judging model to judge whether the multi-source information is a thermodynamic disaster or not; when the disaster is judged to be a thermodynamic disaster, the disaster type of the disaster is judged by the judging model according to different visual characteristics and temperature differences. The method for judging and positioning the thermodynamic disasters has the advantages of wide monitoring range, good real-time performance, high accuracy, accurate positioning and the like; the method can solve the problems of false alarm and missing alarm existing in the existing monitoring and alarming technology.
Description
Technical Field
The invention relates to a mine thermodynamic disaster judging and positioning method based on binocular vision, in particular to an image-based target detection technology, a binocular vision ranging technology, a multispectral image fusion technology and an infrared radiation temperature measurement technology.
Background
In recent years, statistics of serious accidents of coal mines in China show that thermodynamic disasters (fires and explosions) are disasters with highest proportion and most serious disaster caused in the serious accidents of the coal mines. At present, related researchers have insufficient knowledge on disaster causing mechanisms, disaster relief risks and the like of the thermodynamic disasters, so that an effective technical method is still lacking in the aspects of sensing and judging the thermodynamic disasters, serious accidents are caused, huge economic losses are caused, and casualties under mine are caused under serious conditions.
When a coal mine thermodynamic disaster occurs, flames (fire) or fireballs (explosion), smoke, toxic and harmful gases are usually generated, and the wind flow and direction are changed. Meanwhile, the investigation of the thermodynamic disaster accident shows that: in accidents such as mine fire, gas (coal dust) explosion and the like, the death rate of personnel caused by wounds and burns is less than 20%, and the death rate of personnel caused by carbon monoxide poisoning and choking is higher than 80%. The higher the carbon monoxide concentration, the longer the duration, the greater the damage to the human body, until death. Therefore, in underground coal mines, the method can find out the thermal power disasters in time, quickly position the disaster source, accurately judge and identify the disaster types, and pertinently start emergency plans and emergency rescue, which is very important for coal safety production.
At present, researchers at home and abroad carry out a great deal of theoretical research, experimental analysis and field test on the coal mine thermodynamic disasters for realizing the monitoring and early warning of the coal mine thermodynamic disasters, and various sensing and early warning methods are provided, so that a great deal of research results are obtained. However, the prior art relates to the geological occurrence condition of coal, and the influence of a plurality of factors such as mining method and process, sensor technology and the like, so that the failure report rate and the false report rate of the existing coal mine thermodynamic disaster early warning system are high. Meanwhile, the existing monitoring and early warning technology cannot judge the type of the thermal power disaster, cannot locate and track disaster sources, and cannot meet the safety production requirement of the coal mine. In addition, in the underground severe environment, sensors on which disaster sensing devices depend are easily interfered by environmental factors, and the sensors can only monitor the installation position or the surrounding small range, so that large-area monitoring of the underground disaster condition is difficult to realize.
In the coal mine thermodynamic disaster monitoring, the existing infrared radiation temperature measuring equipment can only measure the temperature of a fire source, and technical equipment aiming at explosion perception is freshly reported. Meanwhile, the resolution of the video image collected by the visible light or short wave infrared camera is high, the contained scene information is quite rich, and the video image is easily influenced by external interference objects and artificial light sources; the real-time temperature field under the mine can be measured by the long-wave infrared camera, but the temperature measurement precision of the existing temperature measurement equipment is greatly influenced by environmental factors, so that the characteristic information contained in the video image is limited, and along with the increase of the temperature measurement distance, the larger the difference of the measured value of the target temperature relative to the true value is, the more difficult is to accurately judge the surface temperature of the thermal power disaster.
Aiming at the problems existing in the monitoring of the coal mine thermodynamic disasters, the invention realizes the method for judging and positioning the coal mine thermodynamic disasters through binocular vision by using external equipment such as an explosion-proof camera and the like as a video image acquisition device, can acquire video images of a monitoring area in real time, and can realize large-area monitoring of the coal mine underground only by deploying a small number of cameras in a key area in the underground; meanwhile, the video images acquired by the multi-source video monitoring equipment are analyzed by computer vision and image processing technology, the video images are identified based on a deep learning model or a feature matching model by utilizing the visual features of the thermal power disasters, and the disaster sources are positioned and judged by multi-spectrum image fusion, binocular ranging and infrared temperature measurement technologies, so that the thermal power disasters, disaster types and disaster positions in a monitored area can be identified rapidly, accurately and reliably. Compared with the existing monitoring and early warning method, the method for judging and identifying the thermal power disasters is quicker, more accurate and more reliable, is lower in interference degree by external environment, and can accurately position the disaster source. In addition, in the mine emergency rescue process, rescue workers can be assisted to pre-judge the scene disaster situation through the video image, and corresponding emergency plans can be made.
Disclosure of Invention
The invention aims to solve the technical problems of false alarm and missing alarm when the existing thermal power disaster monitoring and early warning technology is used for identifying the mine thermal power disaster, and rapidly, accurately and reliably judging the type of the thermal power disaster on the basis of realizing underground large-area monitoring, positioning and tracking the disaster source, and meeting the actual requirements of coal mine safety production.
The technical scheme adopted by the invention specifically solves the technical problems as follows:
a mine thermodynamic disaster identification and positioning method based on binocular vision is characterized in that: the mine thermodynamic disaster judging and positioning method comprises the following steps:
step 1: mounting binocular vision cameras in a key monitoring area under a mine, and calibrating the distance of the binocular vision cameras; performing visual registration and multispectral image fusion on the acquired two-way video image to generate a composite video image;
step 2: extracting visual features of a video image and a composite video image of the mine thermodynamic disaster, and establishing an identification model of the mine thermodynamic disaster according to the visual features to obtain a model structure and model parameters; according to the evolution of the mine thermodynamic disaster, the identification model is continuously perfected, and the model structure and model parameters are optimized;
step 3: the binocular vision camera collects video images in a monitoring area in real time, and monitors whether suspected thermodynamic disasters exist in the video images or the composite video images through the identification model;
step 4: the step 3 is circulated, when a suspected thermodynamic disaster exists in the monitoring area, binocular ranging is started to position the suspected thermodynamic disaster source, temperature compensation is conducted on the infrared temperature measurement model according to the target position, and the real temperature of the suspected thermodynamic disaster source after the temperature compensation is obtained;
step 5: inputting the suspected thermodynamic disaster confidence coefficient in the video image, the suspected thermodynamic disaster confidence coefficient in the composite video image and the true temperature confidence coefficient of the suspected thermodynamic disaster source into a thermodynamic disaster discrimination model, wherein the discrimination model discriminates whether the thermodynamic disaster is according to the input confidence coefficient; when the type of the thermal power disaster is judged to be the thermal power disaster, the judging model judges the type of the thermal power disaster according to the visual characteristics and the temperature difference of different thermal power disasters;
step 6: and according to the acquired type of the thermal power disaster, starting binocular ranging to position the thermal power disaster source in real time, and starting different emergency plans.
Further, the key monitoring area comprises a transportation roadway, a ventilation roadway, a connecting roadway, a tunneling roadway, a cutting hole, a coal face, a tunneling face, an electromechanical chamber and an underground transformer substation.
Further, the binocular vision camera adopts a base line adjustable base, and the binocular vision camera adopts a combination of a long-wave infrared camera and any one of visible light, near infrared and short-wave infrared cameras.
Further, the visual features comprise dynamic features, static features and depth features, wherein the dynamic features comprise a highlight target moving direction, a highlight target moving speed, a highlight area change rate, a highlight area perimeter change rate, a highlight area temperature change rate and a centroid moving area; static features include highlight area, highlight perimeter, highlight circularity, highlight sharp corner count, highlight color and texture; the depth features include static feature numbers and feature number change rates for different depth layers.
Further, the image fusion process includes: carrying out strict registration on the two-way video images acquired by the binocular vision camera, wherein the rows and columns of the images are consistent; acquiring a brightness image of a visible light or short wave infrared image and a gray level image of a long wave infrared image; at least one method of data level fusion, feature level fusion and decision level fusion is adopted to fuse the brightness image and the gray level image.
Further, the identification model at least adopts one of a neural network model and a feature matching model, the model structure of the neural network model adopts feature graphs with multiple scales, and the number of the feature graphs is not less than 2; the recognition model dynamically adjusts the model structure and model parameters according to the characteristics of the mine thermodynamic disasters in different environments and the evolution of the mine thermodynamic disasters.
Further, the basis for the existence of the suspected thermodynamic disaster is any one or two paths in the two paths of video images monitored by the identification model, or the suspected thermodynamic disaster exists in the composite video image; the infrared temperature measurement model adopts an infrared radiation temperature measurement principle and a measuring light path attenuation principle to measure the surface temperature of a target.
Further, the input data of the thermodynamic disaster discrimination model includes: output value x of two paths of video images monitored by recognition model 1 And x 2 Output value x of composite video image monitored by recognition model 3 And a logic value x corresponding to the actual temperature of the suspected thermodynamic disaster source 4 The method comprises the steps of carrying out a first treatment on the surface of the When suspected thermodynamic disasters exist in the monitoring video image, the corresponding output value is 1, otherwise, the corresponding output value is 0; when the real temperature of the suspected thermodynamic disaster source is larger than the set threshold, the corresponding output value is 1, otherwise, the corresponding output value is 0; the confidence coefficient of suspected thermodynamic disasters in the two paths of video images is respectively set to be alpha 1 And alpha 2 The confidence of suspected thermodynamic disaster in the composite video image is set to be alpha 3 The confidence of the true temperature of the suspected thermodynamic disaster source is set to be alpha 4 The method comprises the steps of carrying out a first treatment on the surface of the Output value y=α of thermodynamic disaster discrimination model 1 x 1 +α 2 x 2 +α 3 x 3 +α 4 x 4 。
Further, the confidence coefficient range of the thermodynamic disaster discrimination model is alpha 1 ∈[0,1]、α 2 ∈[0,1]、α 3 ∈[0,1]The method comprises the steps of carrying out a first treatment on the surface of the When the output value y of the thermodynamic disaster discrimination model is more than or equal to T, discriminating the thermodynamic disaster; when the output value y of the thermodynamic disaster discrimination model<And at the time of T, judging that the disaster is a non-thermal power disaster.
Drawings
FIG. 1 is a schematic diagram of a mine thermodynamic disaster identification and location system of the present invention;
FIG. 2 is a flow chart of a method for identifying and locating a mine thermodynamic disaster.
Detailed Description
For the purpose of promoting an understanding of the principles and advantages of the invention, reference should be made to the following detailed description, taken in conjunction with the accompanying drawings and specific embodiments, and the examples should not be taken to limit the scope of the invention.
As shown in fig. 1, the system for identifying and positioning a mine thermodynamic disaster is divided into an uphole part and a downhole part, and is used for identifying and positioning the thermodynamic disaster, and the main components of the system comprise:
1. information processing server (101): the method comprises the steps of storing video images collected by a binocular vision camera (105), calibrating the binocular vision camera at different distances according to the collected video images, carrying out image registration and image fusion on two paths of video images, obtaining a multispectral fused composite video image, judging whether coal mine thermodynamic disaster hidden danger exists in a monitoring area according to the video images, when judging that the thermodynamic disaster video image exists in the monitoring area, obtaining an estimated distance of a disaster hidden danger source through a binocular vision algorithm according to the same target image adopted by the binocular vision camera at the same moment, enabling an alarm module (107) to send alarm signals such as sound, light and vibration through a communication network, and enabling a monitoring server (102) to carry out alarm prompt and man-machine interaction on a monitoring screen while enabling the alarm module to send alarm information.
2. Monitoring server (102): the monitoring server (102) is in communication connection with the information processing server (101) and is responsible for displaying monitoring data of a coal mine monitoring area, real-time images of the monitoring area are displayed, alarm prompt and man-machine interaction are carried out after an alarm module (107) sends alarm information, production management staff can call and inquire historical data stored by the information processing server (101) through the monitoring server (102), and the monitoring server (102) is connected with a core switch (103) through a communication line to access a mining communication network.
3. Core switch (103): the core management and exchange equipment of the mining communication network is responsible for management and data exchange of all equipment accessed to the mining communication network, has a routing function, and is connected with the external Internet through a firewall.
4. Ring network switch (104): underground exchange equipment of a mining communication network is arranged underground, and a plurality of looped network exchanges are connected in a looped network mode.
5. Binocular vision camera (105): the image acquisition equipment is arranged in a key monitoring area under the mine and is used for acquiring video images of the monitoring areas which are easy to generate disaster hidden danger, such as a transportation roadway, a ventilation roadway, a communication roadway, a tunneling roadway, a cutting hole, a coal face, a tunneling face, an electromechanical chamber, an underground transformer substation and the like under the mine, wherein the video images can be color images, gray images or pseudo-color images; the binocular vision camera (105) adopts a base line adjustable base, and the camera adopts a combination of a long-wave infrared camera and any one of visible light, near infrared and short-wave infrared cameras.
6. Communication substation (106): one end is in communication connection with the binocular vision camera (105), the other end is in communication connection with the ring network switch (104), and can be connected with the two-end equipment through a wireless communication network or a wired communication network, and the description is adopted in the wired communication mode in the example.
7. Alarm module (107): the system adopts alarm modes such as sound, light, vibration and the like, is directly connected with a monitoring server (102) in a communication way through an existing communication network, and is connected with an information processing server (101) in a communication way through a core switch (103); when the alarm module (107) receives the alarm signal sent by the information processing server (101), one or more alarm modes of sound, light and vibration alarm are carried out to give an alarm, and the staff is prompted to treat the scene and start an emergency plan.
The mine thermodynamic disaster identification and positioning method flow shown in fig. 2 comprises the following steps:
1. initialization (201): mounting binocular vision cameras in a key monitoring area under a mine, and calibrating the distance of the binocular vision cameras; visual matching is carried out on the binocular vision camera by adopting a three-dimensional matching method; the key monitoring areas comprise areas which are easy to generate disaster hidden trouble, such as underground transportation lanes, ventilation lanes, connecting lanes, tunneling lanes, open and cut eyes, coal mining working faces, tunneling working faces, electromechanical chambers, underground substations and the like; the distance calibration process comprises the following steps:
step A1: printing chessboard calibration paper, respectively flatly attaching the chessboard calibration paper to positions with different distances from a binocular vision camera in sequence, and sequentially recording the distance values of the chessboard calibration paper and the camera;
step A2: shooting the chessboard calibration paper simultaneously at the same time by using a binocular vision camera, changing the angle and the distance of the chessboard calibration board, and repeatedly shooting for a plurality of times;
step A3: obtaining chessboard calibration images at the same position, sequentially extracting corner features for calibration, and obtaining internal and external parameters and distortion matrixes of the camera when the average calibration error is smaller than 0.5 pixel;
step A4: correcting the image by using the internal and external parameters and the distortion matrix of the camera to ensure that the matching points on the left view and the right view are positioned on the same straight line;
step A5: searching matching points on the left view and the right view by adopting a three-dimensional matching method to finish image matching;
step A6: repeating the steps A3-A5, and completing the image calibration at all distances.
2. Obtaining a functional relation (202) of the estimated distance and the measured distance: setting a plurality of salient targets at different positions in sequence from near to far in a camera monitoring area, and recording the actual measurement distance between each target position and a binocular vision camera; meanwhile, the binocular range algorithm is adopted to measure the distance of the target, and the estimated distances of the targets at different positions are calculated.
The average value of the estimated distance and the measured distance of the target at the same position is obtained by adopting a method of measuring and averaging for a plurality of times, and the average value of the estimated distance and the measured distance of the target at different positions is sequentially obtained by adopting the same method; when the mean square error obtained at a certain position meets the allowable error, adopting the measured distance, and when the mean square error does not meet the allowable error, eliminating the measured distance at the position; fitting all the optimized measurement data to obtain a functional relation between the estimated distance and the measured distance;
3. extracting visual features of the thermodynamic disaster (203): performing visual registration and multispectral image fusion on video images in a camera monitoring area to generate a composite video image; extracting visual features of the thermodynamic disasters according to the video images and the composite video images of the thermodynamic disasters of the mines; the visual features comprise dynamic features, static features and depth features, wherein the dynamic features comprise a highlight target moving direction, a highlight target moving speed, a highlight area change rate, a highlight area perimeter change rate, a temperature change rate and a centroid moving area; static features include highlight area, highlight perimeter, highlight circularity, highlight sharp corner count, highlight color and texture; the depth features comprise static feature quantity and feature quantity change rate of different depth layers;
the image fusion process comprises the following steps: carrying out strict visual registration on two paths of video images obtained by a binocular visual camera, wherein the number of lines and rows of the images is kept consistent; acquiring a brightness image of a visible light or short wave infrared image and a gray level image of a long wave infrared image; at least one method of data level fusion, feature level fusion and decision level fusion is adopted to fuse the brightness image and the gray level image.
4. Establishing an identification model (204): according to the visual characteristics, a model structure and model parameters are obtained, and an identification model of the mine thermodynamic disaster is established; the recognition model is continuously perfected according to the characteristics of the mine thermodynamic disasters under different environments and the evolution dynamic adjustment model structure and model parameters of the mine thermodynamic disasters; the recognition model at least adopts one of a neural network model and a feature matching model, the model structure of the neural network model adopts feature graphs with multiple scales, and the number of the feature graphs is not less than 2.
5. Identifying a thermodynamic hazard (205): and the binocular vision camera acquires video images in the monitoring area in real time, and recognizes whether a suspected thermodynamic disaster exists in the video images or the composite video images through the recognition model. And the judging basis of the suspected thermodynamic disasters is that any one or two paths of video images or composite video images monitored by the identification model exist suspected thermodynamic disasters.
6. Whether a suspected thermodynamic hazard exists (206): when the mine thermodynamic disaster identification and positioning system monitors that a suspected thermodynamic disaster exists in the video image, the method is sequentially executed (207), otherwise, the method returns to the execution (205);
7. binocular range positioning and temperature compensation (207): when the suspected thermodynamic disaster is identified, the binocular vision camera starts binocular ranging, and the estimated distance between the suspected thermodynamic disaster source and the camera is calculated; according to the calculated estimated distance, carrying out distance measurement error compensation on the suspected thermodynamic disaster source by taking a functional relation between the estimated distance and the measured distance, and further realizing real-time positioning of the suspected thermodynamic disaster source; performing temperature compensation on the infrared temperature measurement model according to the real-time positioning data to obtain the real temperature of the suspected thermodynamic disaster source after the temperature compensation; the infrared temperature measurement model adopts an infrared radiation temperature measurement principle and a measuring light path attenuation principle to measure the surface temperature of a target.
Further, the air transmittance of the infrared temperature measurement model is calculated by adopting the following method:
firstly, calculating the transmittance and CO of the water vapor after absorption and attenuation in a temperature measuring path 2 The transmittance after absorption and attenuation, and the transmittance after dust and aerosol scattering and attenuation; then, according to the single wave transmittance formulaCalculating the air single wave transmittance, wherein: />For the single wave transmittance after the water vapor absorption attenuation,>is CO 2 Absorption of the attenuated single wave transmittance, τ s (lambda) is the single wave transmittance after scattering attenuation; secondly, correcting the air single wave transmittance in real time according to the atmospheric pressure in the measuring environment of the infrared temperature measuring module, the molecular number of the air, the density change of aerosol and dust; finally, according to the transmittance formula->Calculating to obtain air transmittance tau a (d) Wherein: lambda (lambda) 1 Is the integral lower limit, lambda 2 The upper limit of the integration is determined by the detection response wave band of the infrared temperature measuring module.
Further, the temperature measurement formula of the infrared temperature measurement model is calculated by adopting the following method:
according toCalculating to obtain the true temperature T of the measured object 0 And a real-time temperature image, wherein: τ a (d) The air permeability corresponds to the distance d between the infrared temperature measuring module and the belt to be measured; epsilon (T) 0 ) For a belt surface temperature T 0 When the belt surface has average normal emissivity; t is the radiation temperature; t (T) a The unit is ambient temperature: K.
8. thermodynamic disaster type discrimination (208): inputting the suspected thermodynamic disaster confidence coefficient in the video image, the suspected thermodynamic disaster confidence coefficient in the composite video image and the true temperature confidence coefficient of the suspected thermodynamic disaster source into a thermodynamic disaster discrimination model, wherein the discrimination model discriminates whether the thermodynamic disaster is according to the input confidence coefficient; when the type of the thermal power disaster is judged to be the thermal power disaster, the judging model judges the type of the thermal power disaster according to the visual characteristics and the temperature difference of different thermal power disasters;
the input data of the thermodynamic disaster discrimination model comprises: output value x of two paths of video images monitored by recognition model 1 And x 2 Output value x of composite video image monitored by recognition model 3 And a logic value x corresponding to the actual temperature of the suspected thermodynamic disaster source 4 The method comprises the steps of carrying out a first treatment on the surface of the When suspected thermodynamic disasters exist in the monitoring video image, the corresponding output value is 1, otherwise, the corresponding output value is 0; when the real temperature of the suspected thermodynamic disaster source is larger than the set threshold, the corresponding output value is 1, otherwise, the corresponding output value is 0; the confidence coefficient of suspected thermodynamic disasters in the two paths of video images is respectively set to be alpha 1 And alpha 2 The confidence of suspected thermodynamic disaster in the composite video image is set to be alpha 3 The confidence of the true temperature of the suspected thermodynamic disaster source is set to be alpha 4 The method comprises the steps of carrying out a first treatment on the surface of the Output value y=α of thermodynamic disaster discrimination model 1 x 1 +α 2 x 2 +α 3 x 3 +α 4 x 4 。
The confidence range is alpha 1 ∈[0,1]、α 2 ∈[0,1]、α 3 ∈[0,1]The method comprises the steps of carrying out a first treatment on the surface of the When the output value y of the thermodynamic disaster discrimination model is more than or equal to T, discriminatingIs a thermal power disaster; when the output value y of the thermodynamic disaster discrimination model<And at the time of T, judging that the disaster is a non-thermal power disaster.
9. Real-time positioning and alerting (209): and according to the acquired type of the thermal power disaster, starting binocular ranging to position the thermal power disaster source in real time, and starting different emergency plans.
Claims (7)
1. A mine thermodynamic disaster identification and positioning method based on binocular vision is characterized in that: the mine thermodynamic disaster judging and positioning method comprises the following steps:
step 1: mounting binocular vision cameras in a key monitoring area under a mine, and calibrating the distance of the binocular vision cameras; performing visual registration and multispectral image fusion on the acquired two-way video image to generate a composite video image;
step 2: extracting visual features of a video image and a composite video image of the mine thermodynamic disaster, and establishing an identification model of the mine thermodynamic disaster according to the visual features to obtain a model structure and model parameters; according to the evolution of the mine thermodynamic disaster, the identification model is continuously perfected, and model structures and model parameters are optimized;
step 3: the binocular vision camera collects video images in a monitoring area in real time, and monitors whether suspected thermodynamic disasters exist in the video images or the composite video images through the identification model;
step 4: the step 3 is circulated, when a suspected thermodynamic disaster exists in the monitoring area, binocular ranging is started to position the suspected thermodynamic disaster source, temperature compensation is conducted on the infrared temperature measurement model according to positioning data, and the real temperature of the suspected thermodynamic disaster source after the temperature compensation is obtained;
step 5: inputting the suspected thermodynamic disaster confidence coefficient in the video image, the suspected thermodynamic disaster confidence coefficient in the composite video image and the true temperature confidence coefficient of the suspected thermodynamic disaster source into a thermodynamic disaster discrimination model, wherein the discrimination model discriminates whether the thermodynamic disaster is according to the input confidence coefficient; when the type of the thermal power disaster is judged to be the thermal power disaster, the judging model judges the type of the thermal power disaster according to the visual characteristics and the temperature difference of different thermal power disasters;
the input data of the thermodynamic disaster discrimination model comprises: output value x of two paths of video images monitored by recognition model 1 And x 2 Output value x of composite video image monitored by recognition model 3 And a logic value x corresponding to the actual temperature of the suspected thermodynamic disaster source 4 The method comprises the steps of carrying out a first treatment on the surface of the When suspected thermodynamic disasters exist in the monitoring video image, the corresponding output value is 1, otherwise, the corresponding output value is 0; when the real temperature of the suspected thermodynamic disaster source is larger than the set threshold, the corresponding output value is 1, otherwise, the corresponding output value is 0; the confidence coefficient of suspected thermodynamic disasters in the two paths of video images is respectively set to be alpha 1 And alpha 2 The confidence of suspected thermodynamic disaster in the composite video image is set to be alpha 3 The confidence of the true temperature of the suspected thermodynamic disaster source is set to be alpha 4 The method comprises the steps of carrying out a first treatment on the surface of the Output value y=α of thermodynamic disaster discrimination model 1 x 1 +α 2 x 2 +α 3 x 3 +α 4 x 4 The method comprises the steps of carrying out a first treatment on the surface of the Confidence range alpha 1 ∈[0,1]、α 2 ∈[0,1]、α 3 ∈[0,1]The method comprises the steps of carrying out a first treatment on the surface of the When the output value y of the thermodynamic disaster discrimination model is more than or equal to T, discriminating the thermodynamic disaster; when the output value y of the thermodynamic disaster discrimination model<When T is determined to be a non-thermal power disaster, T is a thermal power disaster determination threshold;
step 6: and according to the acquired type of the thermal power disaster, starting binocular ranging to position the thermal power disaster source in real time, and starting different emergency plans.
2. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the key monitoring areas comprise a transportation roadway, a ventilation roadway, a connecting roadway, a tunneling roadway, a cutting hole, a coal face, a tunneling face, an electromechanical chamber and an underground transformer substation.
3. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the binocular vision camera adopts a base line adjustable base, and the binocular vision camera adopts a combination of a long-wave infrared camera and any one of visible light, near infrared and short-wave infrared cameras.
4. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the visual features comprise dynamic features, static features and depth features, wherein the dynamic features comprise a highlight target moving direction, a highlight target moving speed, a highlight area change rate, a highlight area perimeter change rate, a temperature change rate and a centroid moving area; static features include highlight area, highlight perimeter, highlight circularity, highlight sharp corner count, highlight color and texture; the depth features include static feature numbers and feature number change rates for different depth layers.
5. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the image fusion process comprises the following steps: carrying out strict registration on the two-way video images acquired by the binocular vision camera, wherein the rows and columns of the images are consistent; acquiring a brightness image of a visible light or short wave infrared image and a gray level image of a long wave infrared image; at least one method of data level fusion, feature level fusion and decision level fusion is adopted to fuse the brightness image and the gray level image.
6. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the identification model at least adopts one of a neural network model and a feature matching model, the model structure of the neural network model adopts feature graphs with multiple scales, and the number of the feature graphs is not less than 2; the recognition model dynamically adjusts the model structure and model parameters according to the characteristics of the mine thermodynamic disasters in different environments and the evolution of the mine thermodynamic disasters.
7. The mine thermodynamic disaster identification and location method as claimed in claim 1, wherein: the basis of the existence of the suspected thermodynamic disaster is any one or two paths in the two paths of video images monitored by the identification model, or the suspected thermodynamic disaster exists in the composite video image; the infrared temperature measurement model adopts an infrared radiation temperature measurement principle and a measuring light path attenuation principle to measure the surface temperature of a target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110597107.9A CN113205562B (en) | 2021-05-31 | 2021-05-31 | Mine thermodynamic disaster identification and positioning method based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110597107.9A CN113205562B (en) | 2021-05-31 | 2021-05-31 | Mine thermodynamic disaster identification and positioning method based on binocular vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113205562A CN113205562A (en) | 2021-08-03 |
| CN113205562B true CN113205562B (en) | 2023-09-15 |
Family
ID=77023602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110597107.9A Active CN113205562B (en) | 2021-05-31 | 2021-05-31 | Mine thermodynamic disaster identification and positioning method based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113205562B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413395A (en) * | 2013-08-15 | 2013-11-27 | 北京声迅电子股份有限公司 | Intelligent smoke detecting and early warning method and device |
| CN107886670A (en) * | 2017-10-17 | 2018-04-06 | 湖北林青测控科技有限公司 | Forest zone initial fire disaster quickly identifies and localization method, storage medium, electronic equipment |
| CN108447219A (en) * | 2018-05-21 | 2018-08-24 | 中国计量大学 | System and method for detecting fire hazard based on video image |
| CN110276284A (en) * | 2019-06-11 | 2019-09-24 | 暨南大学 | Flame identification method, device, equipment and storage medium based on video quality assessment |
| CN110619293A (en) * | 2019-09-06 | 2019-12-27 | 沈阳天眼智云信息科技有限公司 | Flame detection method based on binocular vision |
| CN111047568A (en) * | 2019-12-06 | 2020-04-21 | 汕头大学 | Steam leakage defect detection and identification method and system |
| CN111260597A (en) * | 2020-01-10 | 2020-06-09 | 大连理工大学 | Parallax image fusion method of multiband stereo camera |
| CN112525352A (en) * | 2020-11-24 | 2021-03-19 | 深圳市高巨创新科技开发有限公司 | Infrared temperature measurement compensation method based on face recognition and terminal |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102444165B1 (en) * | 2017-01-20 | 2022-09-16 | 삼성전자주식회사 | Apparatus and method for providing a meeting adaptively |
-
2021
- 2021-05-31 CN CN202110597107.9A patent/CN113205562B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413395A (en) * | 2013-08-15 | 2013-11-27 | 北京声迅电子股份有限公司 | Intelligent smoke detecting and early warning method and device |
| CN107886670A (en) * | 2017-10-17 | 2018-04-06 | 湖北林青测控科技有限公司 | Forest zone initial fire disaster quickly identifies and localization method, storage medium, electronic equipment |
| CN108447219A (en) * | 2018-05-21 | 2018-08-24 | 中国计量大学 | System and method for detecting fire hazard based on video image |
| CN110276284A (en) * | 2019-06-11 | 2019-09-24 | 暨南大学 | Flame identification method, device, equipment and storage medium based on video quality assessment |
| CN110619293A (en) * | 2019-09-06 | 2019-12-27 | 沈阳天眼智云信息科技有限公司 | Flame detection method based on binocular vision |
| CN111047568A (en) * | 2019-12-06 | 2020-04-21 | 汕头大学 | Steam leakage defect detection and identification method and system |
| CN111260597A (en) * | 2020-01-10 | 2020-06-09 | 大连理工大学 | Parallax image fusion method of multiband stereo camera |
| CN112525352A (en) * | 2020-11-24 | 2021-03-19 | 深圳市高巨创新科技开发有限公司 | Infrared temperature measurement compensation method based on face recognition and terminal |
Non-Patent Citations (5)
| Title |
|---|
| A Video-Based Fire Detection Using Deep Learning Models;Kim, Byoungjun等;《Applied Sciences》;第9卷;1-19 * |
| Image fusion of fault detection in power system based on deep learning;Li, Yu等;《Cluster Computing》;第22卷(第4期);9435-9443 * |
| 基于双目视觉的火焰目标检测;王中林;《中国优秀》(第1期);I138-2524 * |
| 基于可见光和红外图像的矿井外因火灾识别方法;孙继平等;《工矿自动化》;第45卷(第5期);1-5，21 * |
| 基于红外可见光图像融合的目标跟踪研究;鲁婷婷;《中国优秀硕士学位论文全文数据库：信息科技辑》(第10期);I135-28 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113205562A (en) | 2021-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111739250B (en) | Fire detection method and system combining image processing technology and infrared sensor | |
| US7991187B2 (en) | Intelligent image smoke/flame sensor and detection system | |
| CN101650866A (en) | Fire detecting system applied to unmanned helicopter and fire detecting method thereof | |
| CN111595453A (en) | Infrared temperature measurement system and method based on face recognition | |
| CN105336085A (en) | Remote large-space fire monitoring alarm method based on image processing technology | |
| CN117037406B (en) | Intelligent monitoring and early warning system for forest fire | |
| KR20140127574A (en) | Fire detecting system using unmanned aerial vehicle for reducing of fire misinformation | |
| CN102881109B (en) | Forest fire early-warning system based on non-refrigerant infrared temperature detector | |
| CN103400463B (en) | A kind of forest fires localization method based on two dimensional image and device | |
| CN108037545A (en) | A kind of directional type optical interference system for unmanned plane | |
| CN109903505A (en) | A kind of fire monitoring system, method and medium | |
| CN112906674A (en) | Mine fire identification and fire source positioning method based on binocular vision | |
| CN108744341A (en) | A kind of high-precision forest fires saving method | |
| CN112906447A (en) | Video monitoring high-risk area-based abnormal event detection system | |
| CN113917458A (en) | Automatic monitoring method for building structure displacement in fire | |
| CN110070692A (en) | A kind of intelligent vision fire-fighting monitoring system and method | |
| CN115512307A (en) | Wide-area space infrared multi-point real-time fire detection method and system and positioning method | |
| CN113205562B (en) | Mine thermodynamic disaster identification and positioning method based on binocular vision | |
| CN108280895A (en) | A kind of explosion-proof crusing robot of petrochemical industry and explosion protection system | |
| CN110500138B (en) | Colliery belt conflagration early warning system in pit | |
| CN206096497U (en) | Danger source detection system | |
| CN116778192A (en) | Fire safety early warning system based on air-ground equipment cooperation | |
| CN106382857A (en) | Unmanned aerial vehicle intercepting method and system | |
| CN115294718A (en) | Fire early warning system based on multisource data fusion | |
| CN114112972B (en) | Closed space SF 6 Gas leakage infrared telemetry device and imaging positioning method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |