CN111461076A - Smoke detection method and smoke detection system combining frame difference method and neural network - Google Patents

Smoke detection method and smoke detection system combining frame difference method and neural network Download PDF

Info

Publication number
CN111461076A
CN111461076A CN202010393465.3A CN202010393465A CN111461076A CN 111461076 A CN111461076 A CN 111461076A CN 202010393465 A CN202010393465 A CN 202010393465A CN 111461076 A CN111461076 A CN 111461076A
Authority
CN
China
Prior art keywords
smoke
smoke detection
neural network
pixel
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010393465.3A
Other languages
Chinese (zh)
Inventor
姜明华
王时威
余锋
周昌龙
宋坤芳
莫德辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Textile University
Original Assignee
Wuhan Textile University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Textile University filed Critical Wuhan Textile University
Publication of CN111461076A publication Critical patent/CN111461076A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the field of environmental protection, and discloses a smoke detection method combining a frame difference method and a neural network, which comprises the following steps: processing continuous image frames in the collected real-time video by using a frame difference method to obtain a difference image; performing feature enhancement on the difference image to obtain an enhanced feature image; according to the characteristic diagram, a suspected smoke candidate area is obtained; and taking the suspected smoke candidate area as the input of a neural network model, and carrying out smoke detection by using the model. The invention also discloses a corresponding smoke detection system which comprises a candidate area generating unit, a smoke detection unit and a smoke alarm unit. The method screens the real-time image, determines the suspected smoke candidate area, and then performs smoke detection by using the neural network model, so that the calculated amount is reduced, and the real-time performance of straw combustion detection is effectively improved. The method and the system of the invention realize the intelligent detection of the straw combustion, can alarm the straw combustion in time and prompt the relevant units to intervene in time, and have important significance for reducing the environmental pollution.

Description

Smoke detection method and smoke detection system combining frame difference method and neural network
Technical Field
The invention belongs to the field of environmental protection, and particularly relates to a smoke detection method and a smoke detection system combining a frame difference method and a neural network.
Background
The open-air burning of straw belongs to low temperature burning, and the burning is incomplete, contains a large amount of carbon monoxide, carbon dioxide, nitrogen oxide, photochemical oxidant and suspended particles etc. in its flue gas and causes atmospheric pollution, and can aggravate the emergence of haze to a certain extent.
With the rapid development of computer vision technology, more and more scenes can be identified by a computer, and therefore, detection technology based on video analysis is used in more and more occasions. In recent years, smoke detection methods based on video analysis have emerged. Chinese patent publication No. CN110046625A, "convolutional neural network for identifying smoke", discloses a smoke identification network that generates spatial features such as texture and shape from color channels by first generating color channels. The method has the advantages that the smoke detection accuracy of straw combustion is not very high, and the method has certain limitations. The technical scheme disclosed in chinese patent publication CN109490930A, "a straw combustion positioning system and method" adopts a monitoring center, positioning detection nodes, routing detection nodes, mobile detection nodes and a matched unmanned aerial vehicle to perform straw combustion detection, which cannot detect the straw combustion condition in real time and also requires a large cost.
Disclosure of Invention
The invention aims to solve the problems and provides a smoke detection method and a smoke detection system combining a frame difference method and a neural network.
The technical scheme of the invention is a smoke detection method combining a frame difference method and a neural network, which comprises the following steps,
step 1: processing front and back continuous image frames in a real-time video acquired in a smoke detection area by using a frame difference method to obtain a difference image;
step 2: performing feature enhancement on the difference image to obtain an enhanced feature image;
and step 3: according to the characteristic diagram, a suspected smoke candidate area is obtained;
and 4, step 4: and taking the suspected smoke candidate area as the input of a smoke detection neural network model to carry out smoke detection.
Preferably, the smoke detection neural network model comprises a convolutional neural network, an RPN network and a full connection layer which are connected in sequence, the convolutional neural network is used for extracting features of an input picture, the RPN network is used for recommending a candidate region, and output is converted into a fixed size by using a RoIPooling layer.
Preferably, step 1 employs a modified frame difference method, the modified frame difference method comprising the steps of:
1) dividing two continuous frames of images into pixel blocks with fixed sizes along the horizontal axis and the vertical axis of a pixel coordinate system respectively, calculating the mean value of pixel values of all channels of all pixel points of the pixel blocks, and updating the pixel values of all channels of the pixel points of the pixel blocks one by one into the mean value of the pixel values of all the pixel points:
Figure BDA0002486781620000021
i＝1,2…,M，
where i represents the ith pixel Block(i)Pixel value, R, of a pixel point representing the ith pixel blocki、Gi、BiRespectively representing the sum of R components, G components and B components of all pixel points in the ith pixel block; n is the number of pixel points of the pixel block; m is the number of pixel blocks of the image segmentation;
2) and (2) calculating the difference of pixel values of pixel points of pixel blocks with the same coordinates of the two frames of images processed in the step 1), and taking the absolute value of the difference to obtain a difference image.
Preferably, in step 2, the feature enhancement adopts a maximum inter-class variance method.
Further, in step 3, the suspected smoke candidate area is obtained, a connected domain with an area larger than a threshold value in the feature map is obtained, a center point of the connected domain is kept unchanged, the connected domain is expanded to a fixed size, and the same area of the real-time image frame is framed as the suspected smoke candidate area according to the position coordinates of the connected domain.
The smoke detection system adopting the smoke detection method comprises a candidate area generation unit, a smoke detection unit and a smoke alarm unit which are sequentially connected. The candidate area generating unit is used for acquiring a candidate area containing a suspected smoke image from a real-time image sequence; the smoke detection unit is used for detecting a candidate area of the suspected smoke image by using a smoke detection neural network model and determining an image containing smoke; and the smoke alarm unit responds to the processing result of the suspected smoke detection unit and sends alarm information to managers when detecting the smoke generated by straw burning.
Compared with the prior art, the invention has the beneficial effects that:
1) the method of the invention utilizes a frame difference method and an image enhancement technology to determine whether a suspected smoke candidate area exists in the image to screen the real-time image, and then utilizes the neural network model to carry out smoke detection on the suspected smoke candidate area, thereby reducing the calculated amount of the neural network model and effectively improving the real-time performance of straw burning detection;
2) according to the invention, the real-time image is screened, a suspected smoke candidate area is determined, and the smoke is identified by using the neural network model, so that the accuracy of smoke detection is effectively improved, and the false alarm rate is reduced;
3) an improved frame difference method is adopted to determine a connected domain in a difference image, so that on the premise of ensuring the smoke detection efficiency, the calculation is simplified, the complexity is reduced, and the method is time-saving and efficient;
4) the smoke detection neural network model is more simplified than an Faster R-CNN neural network model, the complexity of the neural network is reduced, and the accurate detection of the straw combustion smoke is realized;
5) the system provided by the invention realizes intelligent detection of straw combustion, can alarm the straw combustion in time and prompt related units to intervene in time, and has important significance in reducing environmental pollution. And artificial intelligence is used for replacing manpower, so that manpower is saved.
Drawings
The invention is further illustrated by the following figures and examples.
Fig. 1 is a schematic flow chart of the smoke detection method of the present invention.
Fig. 2 is a schematic structural diagram of the smoke detection system of the present invention.
Fig. 3 is a schematic diagram of dividing a real-time image into pixel blocks according to an embodiment.
FIG. 3(a) is a real-time image before dividing pixel blocks;
fig. 3(b) shows an image obtained by dividing pixel blocks and averaging the pixel blocks.
FIG. 4 is a diagram illustrating an embodiment of determining suspected smoke candidate areas using a modified frame difference method.
Fig. 5 is a schematic structural diagram of a smoke detection neural network model according to an embodiment.
Detailed Description
As shown in fig. 1, the smoke detection method combining the frame difference method and the neural network includes the following steps,
step 1: processing front and back continuous image frames in a real-time video acquired in a smoke detection area by using an improved frame difference method to obtain a difference image;
step 2: performing feature enhancement on the difference image by adopting a maximum inter-class variance method to obtain an enhanced feature image;
and step 3: acquiring a connected domain with an area larger than 12 × 12 in the feature map according to the feature map, keeping a center point of the connected domain unchanged, expanding the connected domain to a square area of 80 × 80, and framing the same area of the real-time image frame as a suspected smoke candidate area according to the position coordinates of the connected domain, as shown in fig. 4;
and 4, step 4: taking the suspected smoke candidate area as the input of a smoke detection neural network model, and carrying out smoke detection by using the smoke detection neural network model;
and 5: and issuing and alarming the smoke detection result to a manager or a worker.
In step 1, the improved frame difference method comprises the following steps:
1) cutting two continuous frames of images along the horizontal axis and the vertical axis of a pixel coordinate system respectively, dividing the images into pixel blocks with the size of 12 × 12, and discarding the remaining areas less than 12 × 12 in the images; calculating the mean value of each component of RGB of all pixels in the pixel block, and updating the pixel values of the pixels of the pixel block one by one to the mean value of the pixel values of each channel of all pixels, as shown in fig. 3;
the pixel values are calculated as follows:
Figure BDA0002486781620000031
i＝1,2…,M，
where i represents the ith pixel Block(i)Pixel value, R, of a pixel point representing the ith pixel blocki，GiAnd BiRespectively representing the sum of R components, G components and B components of all pixel points in the ith pixel block; n is the number of pixel points of the pixel block; m is the number of pixel blocks of the image segmentation;
2) and (2) calculating the difference of pixel values of pixel points of pixel blocks with the same coordinates of the two frames of images processed in the step 1), and taking the absolute value of the difference to obtain a difference image.
The maximum inter-class variance method is proposed by the great amount of scholars in japan (Nobuyuki Otsu), and is an adaptive threshold segmentation method, wherein an optimal threshold is selected to achieve optimal segmentation under the condition that inter-class variance is maximum, pixels on an image are classified into two classes, and the classification is performed and then the enhancement is performed respectively.
As shown in fig. 5, the smoke detection neural network model of the embodiment includes a convolutional neural network, an RPN network, and a fully-connected layer, which are connected in sequence, where the convolutional neural network is used to extract features of an input picture, the RPN network is used to recommend a candidate region, and an output is converted into a fixed size by using a RoI firing layer. The convolutional neural network includes 10 conv layers, 10 rule layers, and 4 Pooling layers. The smoke detection neural network model of the embodiment is more simplified than the Faster R-CNN neural network model, and the complexity of the model is reduced.
The smoke detection system adopting the smoke detection method comprises a candidate area generation unit, a smoke detection unit and a smoke alarm unit which are sequentially connected, as shown in fig. 2. The candidate region generating unit is used for acquiring a candidate region containing a suspected smoke image from a real-time image sequence by utilizing an improved frame difference method and a maximum inter-class variance method; the smoke detection unit is used for detecting a candidate area of the suspected smoke image by using a smoke detection neural network model and determining an image containing smoke; and the smoke alarm unit responds to the processing result of the suspected smoke detection unit and sends alarm information to managers when detecting the smoke generated by straw burning. The smoke alarm unit sends the alarm information to a manager through a short message and an email.
The smoke detection method and the smoke detection system combining the frame difference method and the neural network can accurately detect the burning condition of the straws in real time. Compared with the prior art, the method greatly reduces the calculation amount and the calculation cost, reduces the requirements on the processing performance of the image processing unit, and can improve the real-time performance.

Claims (9)

1. The smoke detection method combining the frame difference method and the neural network is used for detecting the smoke generated by straw combustion and is characterized by comprising the following steps,
step 1: processing front and back continuous image frames in a real-time video acquired in a smoke detection area by using a frame difference method to obtain a difference image;
step 2: performing feature enhancement on the difference image to obtain an enhanced feature image;
and step 3: according to the characteristic diagram, a suspected smoke candidate area is obtained;
and 4, step 4: and taking the suspected smoke candidate area as the input of a smoke detection neural network model to carry out smoke detection.
2. The smoke detection method of claim 1, wherein the smoke detection neural network model comprises a convolutional neural network, an RPN network and a fully connected layer, which are connected in sequence, the convolutional neural network is used for extracting features of an input picture, the RPN network is used for recommending candidate regions, and the output is converted into a fixed size by using a RoI Pooling layer.
3. The smoke detection method of claim 1, wherein step 1 employs a modified frame difference method, the modified frame difference method comprising the steps of:
1) dividing two continuous frames of images into pixel blocks with fixed sizes along the horizontal axis and the longitudinal axis of a pixel coordinate system respectively, calculating the mean value of pixel values of all channels of all pixel points of the pixel blocks, and updating the pixel values of all channels of the pixel points of the pixel blocks into the mean value of the pixel values of all the pixel points one by one;
the pixel values are calculated as follows:
Figure FDA0002486781610000011
where i represents the ith pixel Block(i)Pixel value, R, of a pixel point representing the ith pixel blocki，GiAnd BiRespectively representing the sum of R component, G component and B component of all pixel points in the ith pixel block(ii) a N is the number of pixel points of the pixel block; m is the number of pixel blocks of the image segmentation;
2) and (2) calculating the difference of pixel values of pixel points of pixel blocks with the same coordinates of the two frames of images processed in the step 1), and taking the absolute value of the difference to obtain a difference image.
4. The method for smoke detection by combining frame difference method and neural network as claimed in claim 1, wherein in step 2, the feature enhancement adopts the maximum between class variance method.
5. The smoke detection method according to claim 1, wherein in step 3, the suspected smoke candidate area is obtained, a connected domain with an area larger than a threshold value in the feature map is obtained, a center point of the connected domain is kept unchanged, the connected domain is expanded to a fixed size, and the same area of the real-time image frame is framed as the suspected smoke candidate area according to the position coordinates of the connected domain.
6. The frame difference and neural network combined smoke detection method of claim 3, wherein said fixed size pixel block has a size of 12 x 12 in pixels.
7. The method for smoke detection by frame difference and neural network as claimed in claim 5, wherein in step 3, the threshold is 144.
8. A detection system using the smoke detection method according to any one of claims 1 to 7, comprising a candidate area generation unit, a smoke detection unit, and a smoke alarm unit connected in this order,
the candidate area generating unit is used for acquiring a candidate area containing a suspected smoke image from a real-time image sequence;
the smoke detection unit is used for detecting a candidate area of the suspected smoke image by using a smoke detection neural network model and determining an image containing smoke;
and the smoke alarm unit responds to the processing result of the suspected smoke detection unit and sends alarm information to managers when detecting the smoke generated by straw burning.
9. The detection system according to claim 8, wherein the smoke alarm unit sends alarm information to the manager by short message or mail.
CN202010393465.3A 2019-11-01 2020-05-11 Smoke detection method and smoke detection system combining frame difference method and neural network Pending CN111461076A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911057871 2019-11-01
CN2019110578716 2019-11-01

Publications (1)

Publication Number Publication Date
CN111461076A true CN111461076A (en) 2020-07-28

Family

ID=71681907

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010393465.3A Pending CN111461076A (en) 2019-11-01 2020-05-11 Smoke detection method and smoke detection system combining frame difference method and neural network

Country Status (1)

Country Link
CN (1) CN111461076A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689650A (en) * 2021-09-07 2021-11-23 广州邦讯信息系统有限公司 Forest fire prevention smoke detection method and system based on monitoring camera
CN114283367A (en) * 2021-12-26 2022-04-05 特斯联科技集团有限公司 Artificial intelligent open fire detection method and system for garden fire early warning
CN115187929A (en) * 2022-08-24 2022-10-14 长扬科技(北京)股份有限公司 AI visual inspection method and device of two-stage transaction strategy

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050478A (en) * 2014-07-09 2014-09-17 湖南大学 Smog detection method and system
CN107609470A (en) * 2017-07-31 2018-01-19 成都信息工程大学 The method of outdoor fire disaster early-stage smog video detection
CN107742307A (en) * 2017-10-25 2018-02-27 哈尔滨理工大学 Based on the transmission line galloping feature extraction and parameters analysis method for improving frame difference method
CN109977790A (en) * 2019-03-04 2019-07-05 浙江工业大学 A kind of video smoke detection and recognition methods based on transfer learning
US20190258866A1 (en) * 2018-02-19 2019-08-22 Krishna Khadloya Human presence detection in edge devices
CN110335435A (en) * 2019-06-26 2019-10-15 深圳市微纳集成电路与系统应用研究院 Fire disaster alarming device based on Smoke Detection

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050478A (en) * 2014-07-09 2014-09-17 湖南大学 Smog detection method and system
CN107609470A (en) * 2017-07-31 2018-01-19 成都信息工程大学 The method of outdoor fire disaster early-stage smog video detection
CN107742307A (en) * 2017-10-25 2018-02-27 哈尔滨理工大学 Based on the transmission line galloping feature extraction and parameters analysis method for improving frame difference method
US20190258866A1 (en) * 2018-02-19 2019-08-22 Krishna Khadloya Human presence detection in edge devices
CN109977790A (en) * 2019-03-04 2019-07-05 浙江工业大学 A kind of video smoke detection and recognition methods based on transfer learning
CN110335435A (en) * 2019-06-26 2019-10-15 深圳市微纳集成电路与系统应用研究院 Fire disaster alarming device based on Smoke Detection

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689650A (en) * 2021-09-07 2021-11-23 广州邦讯信息系统有限公司 Forest fire prevention smoke detection method and system based on monitoring camera
CN114283367A (en) * 2021-12-26 2022-04-05 特斯联科技集团有限公司 Artificial intelligent open fire detection method and system for garden fire early warning
CN114283367B (en) * 2021-12-26 2022-09-06 特斯联科技集团有限公司 Artificial intelligent open fire detection method and system for garden fire early warning
CN115187929A (en) * 2022-08-24 2022-10-14 长扬科技(北京)股份有限公司 AI visual inspection method and device of two-stage transaction strategy

Similar Documents

Publication Publication Date Title
CN110135269B (en) Fire image detection method based on mixed color model and neural network
CN108985169B (en) Shop cross-door operation detection method based on deep learning target detection and dynamic background modeling
CN105046196B (en) Front truck information of vehicles structuring output method based on concatenated convolutional neutral net
CN109977782B (en) Cross-store operation behavior detection method based on target position information reasoning
CN111461076A (en) Smoke detection method and smoke detection system combining frame difference method and neural network
CN103208123B (en) Image partition method and system
CN106339657B (en) Crop straw burning monitoring method based on monitor video, device
CN104978567B (en) Vehicle checking method based on scene classification
CN112149543B (en) Building dust recognition system and method based on computer vision
CN112287941B (en) License plate recognition method based on automatic character region perception
CN110334660A (en) A kind of forest fire monitoring method based on machine vision under the conditions of greasy weather
Yang et al. A vehicle license plate recognition system based on fixed color collocation
CN105139429B (en) A kind of fire detection method based on flame notable figure and spatial pyramid histogram
CN107330390A (en) A kind of demographic method based on graphical analysis and deep learning
CN110309765B (en) High-efficiency detection method for video moving target
CN105184308B (en) Remote sensing image building detection classification method based on global optimization decision
CN113221976A (en) Multi-video-frame black smoke diesel vehicle detection method and system based on space-time optical flow network
CN111145222A (en) Fire detection method combining smoke movement trend and textural features
CN112381043A (en) Flag detection method
CN110503049B (en) Satellite video vehicle number estimation method based on generation countermeasure network
CN114885119A (en) Intelligent monitoring alarm system and method based on computer vision
CN110222647B (en) Face in-vivo detection method based on convolutional neural network
CN108921147B (en) Black smoke vehicle identification method based on dynamic texture and transform domain space-time characteristics
CN109325426B (en) Black smoke vehicle detection method based on three orthogonal planes time-space characteristics
CN114821493A (en) Ship information display method and system based on computer vision, AIS and radar

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination