CN111428695B - Straw combustion detection method based on deep learning and regional characteristics - Google Patents
Straw combustion detection method based on deep learning and regional characteristics Download PDFInfo
- Publication number
- CN111428695B CN111428695B CN202010394290.8A CN202010394290A CN111428695B CN 111428695 B CN111428695 B CN 111428695B CN 202010394290 A CN202010394290 A CN 202010394290A CN 111428695 B CN111428695 B CN 111428695B
- Authority
- CN
- China
- Prior art keywords
- smoke
- area
- image
- suspected
- smokeless
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Fire-Detection Mechanisms (AREA)
Abstract
The invention belongs to the field of environmental protection, and discloses a straw combustion detection method based on deep learning and regional characteristics, which comprises the following steps: collecting images of scenes with different visual angles at an observation position, and calibrating a smokeless area; acquiring images from a captured real-time image sequence; identifying and positioning a suspected smoke area in the suspected smoke image by using a smoke detection neural network model; and identifying the visual angle of the suspected smoke image, calculating the position coincidence rate relative to the smoke-free area with the same visual angle, and filtering the suspected smoke candidate area. The smoke identification method firstly utilizes the model to detect the image, and then utilizes the visual angle identification and the smokeless area in the image of the scene with the same visual angle to filter, thereby improving the smoke detection efficiency and reducing the false alarm rate.
Description
Technical Field
The invention belongs to the field of environmental protection, and particularly relates to a straw combustion detection method based on deep learning and regional characteristics.
Background
The open-air burning of straw belongs to low temperature burning, and the burning is incomplete, contains a large amount of carbon monoxide, carbon dioxide, nitrogen oxide, photochemical oxidant and suspended particles etc. in its flue gas and causes atmospheric pollution, and can aggravate the emergence of haze to a certain extent.
With the rapid development of computer vision technology, more and more scenes can be identified by a computer, and therefore, detection technology based on video analysis is used in more and more occasions. In recent years, smoke detection methods based on video analysis have emerged. Chinese patent No. CN109389185A, "video smoke recognition method using three-dimensional convolutional neural network" is to obtain a result frame of suspected smoke region and smoke score by processing at fast R-CNN, collect a certain number of images before and after a target frame as continuous video frames, perform three-dimensional feature extraction on the video frames by using three-dimensional convolutional neural network, combine the extracted feature vectors and the smoke scores of the result frame into a new one, input the new one into an SVM classifier, and classify whether the video smoke is smoke or not. The method has high complexity, needs larger operation and storage cost, and is not necessarily applicable to the field of straw combustion detection. Chinese patent CN109490930A, "a straw burning positioning system and method" adopts monitoring center, location detection node, route detection node, mobile detection node and matched unmanned aerial vehicle to carry out straw burning detection. The method cannot detect the burning condition of the straw in real time and also needs higher cost.
Disclosure of Invention
The invention has the technical problems that the existing smoke identification method adopting a neural network has high complexity, large calculated amount and high false alarm rate, and the existing straw combustion detection method adopting an unmanned aerial vehicle has high cost and poor real-time property.
The invention aims to solve the problems and provides a straw burning detection method based on deep learning and regional characteristics, which is used for judging the coincidence rate of a suspected smoke candidate area identified by a neural network model and a smokeless area with the same visual angle and filtering the suspected smoke candidate area, so that the smoke detection efficiency is improved, the false alarm rate is reduced, and the straw burning detection method is used for real-time detection of straw burning.
The technical scheme of the invention is a straw burning detection method based on deep learning and regional characteristics, which utilizes a smoke detection neural network model to judge whether an image shot in real time contains a suspected smoke area, and utilizes a smokeless area to filter the smoke image containing the suspected smoke area detected by the model, thereby reducing the false alarm rate of the model,
step 1: collecting images of scenes at different viewing angles at an observation position, calibrating an area where smoke is unlikely to appear in the image of the scene at each viewing angle, and setting the area as a smoke-free area;
step 2: acquiring images from a captured real-time image sequence;
and step 3: carrying out smoke identification on the image in the step 2 by using a smoke detection neural network model and positioning a suspected smoke area in the image;
and 4, step 4: and identifying the visual angle of the image containing the suspected smoke area, and calculating the position coincidence rate by using the smokeless areas with the same visual angle after the visual angle is determined, thereby filtering the suspected smoke candidate area.
And judging the position coincidence rate, and if the coincidence rate of the suspected smoke candidate area and the smokeless area exceeds a threshold value R, judging the suspected smoke candidate area as an invalid candidate area.
The smokeless area is the position of the ground object which can not generate smoke in the image, and the ground object which can not generate smoke comprises houses, lakes and rivers.
Preferably, the smoke detection neural network model comprises a convolutional neural network, an RPN network and a full connection layer which are connected in sequence, the convolutional neural network is used for extracting features of an input picture, the RPN network is used for recommending a candidate region, and an output is converted into a fixed size by using a RoI Pooling layer. The convolutional neural network includes 10 conv layers, 10 rule layers, and 4 Pooling layers.
Preferably, the visual angle identification is to match the image containing the suspected smoke region with the image of the scene at each visual angle in step 1, perform similarity calculation by using the contrast and brightness of the pixel points, find the image of the most similar scene, and determine the visual angle of the image containing the suspected smoke region. Methods utilizing pixel points such as peak signal-to-noise ratio, structural similarity, histogram methods, matrix decomposition methods.
Preferably, the overlapping rate is calculated by calculating the coordinate value of the suspected smoke area and the coordinate value of the smokeless area at the same viewing angle to obtain the coordinate value of the overlapping area, and the overlapping rate is calculated according to the ratio of the area of the overlapping area to the area of the smokeless area, i.e. the ratio of the overlapping area.
Preferably, the threshold value R is 0.3.
Compared with the prior art, the invention has the beneficial effects that:
1) The smoke identification method filters the suspected smoke candidate area identified by the neural network, improves the smoke detection efficiency and reduces the false alarm rate;
2) The method extracts the characteristic diagram by using the convolutional neural network, recommends the candidate region by using the RPN network, optimizes the structure of the smoke detection neural network model, and enables the smoke detection of the smoke detection neural network model to be more efficient and reliable;
3) According to the method, a small amount of samples are adopted to train the smoke detection neural network model, so that the accuracy of smoke identification of straw combustion can be ensured, and the problem of insufficient samples is solved;
4) The method provided by the invention can be used for carrying out real-time straw combustion smoke detection on the image in the collected video of the smoke detection area, and the real-time performance is good.
Drawings
The invention is further illustrated by the following examples in conjunction with the drawings.
FIG. 1 is a flow diagram of a straw burning detection method based on deep learning and regional characteristics.
Fig. 2 is a schematic diagram of a house in which the scene image includes a smokeless area.
Fig. 3 is a schematic diagram of a unit building including a smoke-free area for a scene image.
Fig. 4 is a schematic diagram of an image of smoke including a suspected smoke region detected in an embodiment.
FIG. 5 is a schematic illustration of the suspected smoke region detected in the example coinciding with the smokeless region.
Fig. 6 is a schematic structural diagram of a smoke detection neural network model according to an embodiment.
Detailed Description
The embodiment selects a communication tower top with a height of 30 meters as an observation place, a camera is installed on the top of the communication tower, a lens of the camera can horizontally rotate around a longitudinal axis of a camera holder to change a visual angle shot by the camera, the lens of the camera rotates by 60 degrees once, and returns to an initial visual angle after 6 times of rotation, images of scenes with 6 different visual angles are collected by rotating the camera lens during shooting, and a fixed focal length is adopted during shooting by the camera.
As shown in fig. 1, a straw burning detection method based on deep learning and regional characteristics, which adopts a pixel coordinate system to determine the coincidence rate of a suspected smoke candidate area identified by a smoke detection neural network model and a smokeless area with the same visual angle, wherein the smokeless area is the position of a ground object which does not generate smoke in an image, and filters the suspected smoke candidate area, comprises the following steps,
step 1: shooting images of outdoor scenes for straw burning detection from different visual angles, setting the serial numbers of the visual angles, calibrating the areas where smoke is unlikely to appear in the scene images of each visual angle, and setting the areas as smoke-free areas, as shown in fig. 2 and 3;
and 2, step: acquiring images from a captured real-time image sequence;
and 3, step 3: performing smoke recognition on the image in the step 2 by using a smoke detection neural network model, and positioning a suspected smoke area in the image, as shown in fig. 4;
and 4, step 4: and identifying the visual angle of the image containing the suspected smoke area, and calculating the position coincidence rate by using the smokeless areas with the same visual angle after the visual angle is determined, thereby filtering the suspected smoke candidate area.
And 4, judging the position coincidence rate, and if the coincidence rate of the suspected smoke candidate area and the target frame of the smokeless object exceeds 0.3, judging that the suspected smoke candidate area is an invalid candidate area.
Objects in the smokeless zone include houses, lakes, rivers.
In step 4, the visual angle identification is to match the image containing the suspected smoke area with the scene image of each visual angle in step 1, perform similarity calculation by using the contrast and brightness of the pixel points, find the most similar scene image, and determine the visual angle of the image containing the suspected smoke area. Methods utilizing pixel points such as peak signal-to-noise ratio, structural similarity, histogram methods, matrix factorization methods.
And 4, calculating the coincidence rate, namely calculating by using the coordinate values of the suspected smoke area and the coordinate values of the smokeless area with the same visual angle to obtain the coordinate values of the coincident area, and calculating the coincidence rate according to the ratio of the area of the coincident area to the area of the smokeless area. As shown in fig. 5, in the embodiment, the identified suspected smoke area is overlapped with the smokeless area in a large area, and the identified suspected smoke area is filtered.
As shown in fig. 6, the smoke detection neural network model of the embodiment includes a convolutional neural network, an RPN network, and a fully-connected layer, which are connected in sequence, where the convolutional neural network is used to extract features of an input picture, and the RPN network is used to recommend a candidate region, and convert an output into a fixed size using a RoI firing layer. The convolutional neural network includes 10 conv layers, 10 rule layers, and 4 Pooling layers. The smoke detection neural network model of the embodiment is more simplified than the Faster R-CNN neural network model, and the complexity of the model is reduced.
In this embodiment, compared with a smoke detection method that does not employ a suspected smoke candidate area for filtering, the smoke detection method that employs the suspected smoke candidate area for filtering of the present invention reduces the false alarm rate from 77.2% to 26.1%.
Claims (6)
1. A straw burning detection method based on deep learning and regional characteristics is characterized in that images of scenes with different visual angles are collected at an outdoor observation position, a smoke detection neural network model is used for judging a suspected smoke area in the real-time scene image, a smoke image which is detected by the model and contains the suspected smoke area is filtered by a smokeless area, and the false alarm rate of the model is reduced, the straw burning detection method comprises the following steps,
step 1: collecting images of scenes at different viewing angles at an observation position, calibrating an area where smoke is unlikely to appear in the image of the scene at each viewing angle, and setting the area as a smoke-free area;
the method comprises the following steps that a camera is installed on the top of a communication tower, a lens of the camera can horizontally rotate around a longitudinal axis of a camera holder to change the shooting angle of the camera, the lens of the camera rotates 60 degrees once, returns to the initial angle of view 6 times of rotation, collects images of scenes with 6 different angles of view by rotating the lens of the camera during shooting, and adopts a fixed focal length during shooting of the camera;
step 2: acquiring images from a captured real-time image sequence;
and step 3: carrying out smoke identification on the image in the step (2) by using a smoke detection neural network model and positioning a suspected smoke area in the image;
and 4, step 4: carrying out visual angle identification on the image containing the suspected smoke area, and after the visual angle is determined, calculating the position coincidence rate relative to the smokeless areas with the same visual angle so as to filter the suspected smoke candidate area;
the smoke detection neural network model comprises a convolutional neural network, an RPN network and a full connection layer which are sequentially connected, wherein the convolutional neural network is used for extracting the characteristics of an input picture, the RPN network is used for recommending a candidate region, and the output is converted into a fixed size by utilizing a RoI Pooling layer; the convolutional neural network comprises 10 conv layers, 10 relu layers and 4 Pooling layers.
2. The straw burning detection method based on deep learning and regional characteristics as claimed in claim 1, wherein the position coincidence rate determination determines that the suspected smoke candidate area is an invalid candidate area if the coincidence rate of the suspected smoke candidate area and the smokeless area exceeds a threshold value R.
3. The straw burning detection method based on deep learning and regional characteristics as claimed in claim 1, wherein the smokeless area is a position of a ground object which does not generate smoke in the image, and the ground object which does not generate smoke includes a house, a lake, and a river.
4. The straw burning detection method based on deep learning and regional characteristics as claimed in claim 1, wherein the perspective identification is performed by matching the image containing the suspected smoke region with the image of the scene at each perspective in step 1, performing similarity calculation by using the contrast and brightness of the pixel points to find the most similar image of the scene, and determining the perspective of the image containing the suspected smoke region.
5. The straw burning detection method based on deep learning and regional characteristics as claimed in claim 1, wherein the coincidence rate calculation is performed by using the coordinate values of the suspected smoke region and the coordinate values of the smokeless region at the same viewing angle to obtain the coordinate values of the coincidence region, and the coincidence rate is calculated according to the ratio of the area of the coincidence region to the area of the smokeless region, that is, the proportion of the coincidence region.
6. The straw combustion detection method based on deep learning and regional characteristics as claimed in claim 2, wherein the threshold value R is 0.3.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2019110578701 | 2019-11-01 | ||
| CN201911057870 | 2019-11-01 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111428695A CN111428695A (en) | 2020-07-17 |
| CN111428695B true CN111428695B (en) | 2023-04-07 |
Family
ID=71552767
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010394290.8A Active CN111428695B (en) | 2019-11-01 | 2020-05-11 | Straw combustion detection method based on deep learning and regional characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111428695B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283332A (en) * | 2021-05-20 | 2021-08-20 | 国电南瑞科技股份有限公司 | Fire-fighting robot flame identification method, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106997461B (en) * | 2017-03-28 | 2019-09-17 | 浙江大华技术股份有限公司 | A kind of firework detecting method and device |
| CN109147254B (en) * | 2018-07-18 | 2021-05-18 | 武汉大学 | Video field fire smoke real-time detection method based on convolutional neural network |
-
2020
- 2020-05-11 CN CN202010394290.8A patent/CN111428695B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111428695A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210073573A1 (en) | Ship identity recognition method based on fusion of ais data and video data | |
| CN106802113B (en) | Intelligent hit telling system and method based on many shell hole algorithm for pattern recognitions | |
| CN108062349B (en) | Video monitoring method and system based on video structured data and deep learning | |
| CN107609470B (en) | Method for detecting early smoke of field fire by video | |
| CN109145708B (en) | Pedestrian flow statistical method based on RGB and D information fusion | |
| CN111462451B (en) | Straw burning detection alarm system based on video information | |
| CN103824070B (en) | A kind of rapid pedestrian detection method based on computer vision | |
| CN110543867A (en) | crowd density estimation system and method under condition of multiple cameras | |
| CN111985365A (en) | Straw burning monitoring method and system based on target detection technology | |
| CN110490043A (en) | A kind of forest rocket detection method based on region division and feature extraction | |
| CN110570461A (en) | method for realizing automatic person-searching tracking by combining community lighting network and cloud platform community population information base | |
| CN111652035B (en) | Pedestrian re-identification method and system based on ST-SSCA-Net | |
| CN109165602B (en) | Black smoke vehicle detection method based on video analysis | |
| CN113269091A (en) | Personnel trajectory analysis method, equipment and medium for intelligent park | |
| CN111145222A (en) | Fire detection method combining smoke movement trend and textural features | |
| CN114202646A (en) | Infrared image smoking detection method and system based on deep learning | |
| CN113362374A (en) | High-altitude parabolic detection method and system based on target tracking network | |
| CN112949439A (en) | Method and system for monitoring invasion of personnel in key area of oil tank truck | |
| CN112257554A (en) | Forest fire recognition method, system, program and storage medium | |
| CN111428695B (en) | Straw combustion detection method based on deep learning and regional characteristics | |
| CN115690564A (en) | Outdoor fire smoke image detection method based on Recursive BIFPN network | |
| CN116977937A (en) | Pedestrian re-identification method and system | |
| CN115171336A (en) | Drowned protection system of beach control | |
| CN113269099B (en) | Vehicle re-identification method under heterogeneous unmanned system based on graph matching | |
| CN113657250A (en) | Flame detection method and system based on monitoring video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |