CN114998788A - Smog judgment method based on video analysis - Google Patents

Smog judgment method based on video analysis Download PDF

Info

Publication number
CN114998788A
CN114998788A CN202210583081.7A CN202210583081A CN114998788A CN 114998788 A CN114998788 A CN 114998788A CN 202210583081 A CN202210583081 A CN 202210583081A CN 114998788 A CN114998788 A CN 114998788A
Authority
CN
China
Prior art keywords
image
smoke
coordinate
value
pixel point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210583081.7A
Other languages
Chinese (zh)
Inventor
章耀锋
高飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202210583081.7A priority Critical patent/CN114998788A/en
Publication of CN114998788A publication Critical patent/CN114998788A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the field of smoke detection, and discloses a smoke judgment method based on video analysis, in particular to a method for filtering HSV threshold values of video images and analyzing the change rate of the video images to determine whether smoke exists.

Description

Smoke judgment method based on video analysis
Technical Field
The invention relates to the field of smoke detection, in particular to a method for carrying out HSV threshold filtering on a video image and analyzing the change rate of the video image so as to determine whether smoke exists.
Background
Video smoke detection is an important mode for detecting and finding early fire, and a target detection network such as YOLO and Faster-RCNN can obtain higher accuracy and Faster detection speed through training of a smoke data set. However, because the smoke data sets are very few and the smoke itself has no fixed form, the target detection network often has a certain undetected rate and false detection rate, which is unacceptable for fire safety.
Before the target detection network is applied to smoke detection, a smoke detection system based on manually made features is widely used, images are filtered and screened according to textures, colors and dynamic features of smoke, false detection rate can be effectively reduced, although the target detection network occupies the mainstream in the current image detection field, a single target detection network cannot achieve satisfactory effects due to the few smoke data sets and the irregular morphological features of smoke, and continuous video picture information is difficult to effectively utilize.
Therefore, a smoke judgment method based on video analysis is provided by combining a neural network and a traditional image processing method.
Disclosure of Invention
The invention aims to filter partial non-smoke areas in a video image by using an HSV color space, continuously count pixel changes of adjacent frame video images after filtering, detect the image by using a YOLOv5 neural network, find a suspected smoke area, respectively accumulate the pixel changes of the image in a period of time before and after the suspected smoke area is found, and finally compare the total amount of the pixel changes of the image in a period of time before and after the suspected smoke area to determine whether smoke exists, thereby further improving the detection precision and reducing the false alarm rate.
The technical scheme of the invention is as follows:
a smog judgment method based on video analysis comprises the following steps:
step 1: acquiring n frames of images in the monitored video, and recording as I 1 ,I 2 ,…,I n (ii) a Using YOLOv5 to image I of any ith frame i Carrying out smoke detection to obtain K i A candidate smoke region, and recording the kth candidate smoke region as
Figure BDA0003664924890000021
Wherein,
Figure BDA0003664924890000022
coordinates representing the upper left corner of the kth smoke candidate region,
Figure BDA0003664924890000023
coordinates representing the lower right corner of the kth smoke candidate region;
step 2: calculating adjacent frame changes of the n frame images to obtain an image set S ═ I 1a ,I 2a ,…,I n-1a The method comprises the following specific steps:
step 2.1: for any I i And I i+1 Performing HSV threshold filtering, wherein i is 1,2, …, n-1, converting the image from the RGB color space to the HSV color space to obtain an image G i And G i+1 Setting a filter condition Q ═ V ∈ [150,255 ]],S∈[0,25]To G i And G i+1 The HSV value of each pixel point is judged, namely I is judged according to the formulas (1), (2) and (3) i And I i+1 Filtering to obtain I i And I i+1 Image I filtered under filtering condition Q iq And I i+1q ，I i And I i+1 The RGB values of the pixels that do not satisfy the filtering condition are all set to zero, and the HSV threshold filtering formula is as follows:
Figure BDA0003664924890000024
Figure BDA0003664924890000025
Figure BDA0003664924890000026
wherein, I iR (x,y)、I iG (x, y) and I iB (x, y) are each I i RGB value, G, of pixel point in coordinate (x, y) iS (x, y) and G iV (x, y) are each I i SV value, I, of a pixel point centered at coordinate (x, y) iqR 、I iqG And I iqB Are respectively I i InThe RGB value of the pixel point at the coordinate (x, y) after threshold value filtration is stored in the filtered image I iq In respect of I i+1 The same operation is carried out to obtain a filtered image I i+1q ；
Step 2.2: calculation of I according to formula (4) iq The total RGB value C of the pixel points at any coordinate (x, y) iq (x, y) and calculating to obtain I by the same method i+1q The total value C of RGB values of pixel points at any coordinates (x, y) in the middle i+1q (x, y); calculating the neighboring frame I according to equation (5) iq And I i+1q Change image I of ia ；
C iq (x,y)＝I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4)
Figure BDA0003664924890000031
Wherein, I ia (x, y) represents I ia The value of the pixel point at coordinate (x, y);
step 2.3: for image I ia Carrying out corrosion and expansion;
and step 3: get S 1 ＝{I i-t-1a ,I i-ta ,…,I i-1a }，S 2 ＝{I ia ,I i+1a ,…,I i+t-1a }，S 1 For t frames adjacent to the ith frame image, S 2 For the adjacent frame change image of t frames following the ith frame image, i-t is more than or equal to 1 and less than or equal to n, S is calculated according to the formula (6) 1 And S 2 The accumulated change of the image pixels is obtained to obtain an image R 1 And R 2 ；
Figure BDA0003664924890000032
Wherein, | represents a bitwise or operation;
and 4, step 4: from R 1 And R 2 To select and candidate smoke regions
Figure BDA0003664924890000033
Superposed images, resulting in R 1ik And R 2ik Statistics of R 1ik And R 2ik The number of pixels with 255 three channels in the image is recorded as U 1ik And U 2ik If U is present 1ik /U 2ik <0.5, then it is considered
Figure BDA0003664924890000034
Where smoke is present.
The invention has the following beneficial effects: the method can effectively utilize the convolutional neural network model to detect the video images before and after the smoke, analyze the total amount of image pixel change between adjacent frames of the monitoring video within a period of time, reduce the influence of video noise and jitter on image pixel change statistics through HSV color space filtering and corrosion and expansion operations, and eliminate partial false detection caused by insufficient neural network precision.
Drawings
FIG. 1 is an unprocessed video image of the present invention;
FIG. 2 is an image after detection by a convolutional neural network model according to the present invention;
FIG. 3 is an HSV filtered image of the invention with filter condition Q;
FIG. 4 is a change image of adjacent frames according to the present invention;
FIG. 5 is an image of the present invention after erosion and dilation;
fig. 6 is an image of the invention accumulating 100 frames of adjacent frame pixel changes.
Detailed Description
The present invention is explained in detail below with reference to examples and images.
A smog judgment method based on video analysis comprises the following specific steps:
step 1: acquiring n frames of images in the monitored video, and recording as I 1 ,I 2 ,…,I n FIG. 1 is a block diagram of an unprocessed video frame; using convolution neural network model to carry out image I on any ith frame i Carrying out smoke detection to obtain K i A smoke candidate region, and the kth candidateThe smoke region is marked as
Figure BDA0003664924890000044
Wherein,
Figure BDA0003664924890000045
coordinates representing the upper left corner of the kth smoke candidate region,
Figure BDA0003664924890000046
representing coordinates of the lower right corner of the kth candidate smoke region, and fig. 2 is an image detected by the convolutional neural network model in fig. 1, wherein smoke represents a detected object class, 0.39 represents a confidence level, and a box is 1 candidate smoke region;
and 2, step: calculating adjacent frame changes of the n frames of images to obtain an image set S ═ { I } 1a ,I 2a ,…,I n-1a The method comprises the following specific steps:
step 2.1: for any I i And I i+1 Performing HSV threshold filtering, wherein i is 1,2, …, n-1, converting the image from the RGB color space to the HSV color space to obtain an image G i And G i+1 Setting a filter condition Q ═ V ∈ [150,255 ]],S∈[0,25]To G i And G i+1 The HSV value of each pixel point is judged, namely I is judged according to the formulas (1), (2) and (3) i And I i+1 Filtering to obtain I i And I i+1 Image I filtered under Filter Condition Q iq And I i+1q ，I i And I i+1 The RGB values of the pixels which do not satisfy the filtering condition are all set to zero, and the HSV threshold filtering formula is as follows:
Figure BDA0003664924890000041
Figure BDA0003664924890000042
Figure BDA0003664924890000043
wherein, I iR (x,y)、I iG (x, y) and I iB (x, y) are each I i RGB value, G, of pixel point centered at coordinate (x, y) iS (x, y) and G iV (x, y) are each I i SV value, I, of a pixel point centered at coordinate (x, y) iqR 、I iqG And I iqB Are respectively I i The RGB value of the pixel point at the coordinate (x, y) after threshold value filtration is stored in the filtered image I iq In respect of I i+1 The same operation is carried out to obtain a filtered image I i+1q (ii) a FIG. 3 is the HSV-filtered image of FIG. 1 with the filter condition Q, showing that a small number of pixels satisfying the filter condition remain after filtering;
step 2.2: calculation of I according to equation (4) iq The total RGB value C of the pixel points at any coordinate (x, y) iq (x, y) and calculating to obtain I by the same method i+1q The total RGB value C of the pixel points at any coordinate (x, y) i+1q (x, y); calculating the neighboring frame I according to equation (5) iq And I i+1q Change image I of ia FIG. 4 is a variation image of FIG. 3 and its neighboring frames;
C iq (x,y)＝I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4)
Figure BDA0003664924890000051
wherein, I ia (x, y) represents I ia The value of the pixel point at coordinate (x, y);
step 2.3: for image I ia Performing corrosion and expansion, wherein fig. 5 is the image of fig. 4 after corrosion and expansion, and fine interference pixel points are filtered;
and step 3: get S 1 ＝{I i-t-1a ,I i-ta ,…,I i-1a }，S 2 ＝{I ia ,I i+1a ,…,I i+t-1a }，S 1 For the phase of t frames preceding the image of the ith frameAdjacent frame change image, S 2 For the adjacent frame change image of t frames following the ith frame image, i-t is more than or equal to 1 and less than or equal to n, S is calculated according to the formula (6) 1 And S 2 The accumulated change of the image pixels is obtained to obtain an image R 1 And R 2 Fig. 6 is an image of accumulating pixel changes of 100 frames of adjacent frames, and it can be seen that the number of accumulated pixels in the candidate smoke region is more than that in other non-candidate smoke regions;
Figure BDA0003664924890000052
wherein, | represents a bitwise or operation;
and 4, step 4: from R 1 And R 2 To select and candidate smoke regions
Figure BDA0003664924890000053
Superposed images, resulting in R 1ik And R 2ik Statistics of R 1ik And R 2ik The number of pixels of which the three channels are 255 in the image is recorded as U 1ik And U 2ik If U is present 1ik /U 2ik <0.5, then it is considered
Figure BDA0003664924890000054
Where smoke is present.
The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.

Claims (1)

1. A smog judgment method based on video analysis is characterized by comprising the following steps:
step 1: acquiring n frames of images in the monitoring video, and recording as I 1 ,I 2 ,…,I n (ii) a Using YOLOv5 to image I of any ith frame i Carrying out smoke detection to obtain K i A candidate smoke region, and the kth candidate smoke regionIs marked as
Figure FDA0003664924880000011
Wherein,
Figure FDA0003664924880000012
coordinates representing the upper left corner of the kth smoke candidate region,
Figure FDA0003664924880000013
coordinates representing the lower right corner of the kth smoke candidate region;
step 2: calculating adjacent frame changes of the n frames of images to obtain an image set S ═ { I } 1a ,I 2a ,…,I n-1a }；
Step 2.1: for any I i And I i+1 Performing HSV threshold filtering, wherein i is 1,2, …, n-1, converting the image from RGB color space to HSV color space to obtain an image G i And G i+1 Setting a filter condition Q ═ V ∈ [150,255 ]],S∈[0,25]To G i And G i+1 The HSV value of each pixel point is judged, namely I is judged according to the formulas (1), (2) and (3) i And I i+1 Filtering to obtain I i And I i+1 Image I filtered under filtering condition Q iq And I i+1q ，I i And I i+1 The RGB values of the pixels which do not satisfy the filtering condition are all set to zero, and the HSV threshold filtering formula is as follows:
Figure FDA0003664924880000014
Figure FDA0003664924880000015
Figure FDA0003664924880000016
wherein, I iR (x,y)、I iG (x, y) and I iB (x, y) are each I i RGB value, G, of pixel point centered at coordinate (x, y) iS (x, y) and G iV (x, y) are each I i SV value, I, of a pixel point centered at coordinate (x, y) iqR 、I iqG And I iqB Are respectively I i The RGB value of the pixel point at the coordinate (x, y) after threshold value filtration is stored in the filtered image I iq In respect of I i+1 The same operation is carried out to obtain a filtered image I i+1q ；
Step 2.2: calculation of I according to formula (4) iq The total RGB value C of the pixel points at any coordinate (x, y) iq (x, y) and calculating to obtain I by the same method i+1q The total RGB value C of the pixel points at any coordinate (x, y) i+1q (x, y); computing neighboring frame I according to equation (5) iq And I i+1q Change image I of ia ；
C iq (x,y)＝I iqR (x,y)+I iqG (x,y)+I iqB (x,y) (4)
Figure FDA0003664924880000017
Wherein, I ia (x, y) represents I ia The value of the pixel point at coordinate (x, y);
step 2.3: for image I ia Carrying out corrosion and expansion;
and step 3: get S 1 ＝{I i-t-1a ,I i-ta ,…,I i-1a }，S 2 ＝{I ia ,I i+1a ,…,I i+t-1a }，S 1 For t frames adjacent to the ith frame image, S 2 For the adjacent frame change image of t frames following the ith frame image, i-t is more than or equal to 1 and less than or equal to n, S is calculated according to the formula (6) 1 And S 2 The accumulated change of the image pixels is obtained to obtain an image R 1 And R 2 ；
Figure FDA0003664924880000021
Wherein, | represents a bitwise or operation;
and 4, step 4: from R 1 And R 2 To select and candidate smoke regions
Figure FDA0003664924880000022
Superposed images, resulting in R 1ik And R 2ik Statistics of R 1ik And R 2ik The number of pixels with 255 three channels in the image is recorded as U 1ik And U 2ik If U is 1ik /U 2ik If less than 0.5, it is considered that
Figure FDA0003664924880000023
Where smoke is present.
CN202210583081.7A 2022-05-26 2022-05-26 Smog judgment method based on video analysis Pending CN114998788A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210583081.7A CN114998788A (en) 2022-05-26 2022-05-26 Smog judgment method based on video analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210583081.7A CN114998788A (en) 2022-05-26 2022-05-26 Smog judgment method based on video analysis

Publications (1)

Publication Number Publication Date
CN114998788A true CN114998788A (en) 2022-09-02

Family

ID=83029057

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210583081.7A Pending CN114998788A (en) 2022-05-26 2022-05-26 Smog judgment method based on video analysis

Country Status (1)

Country Link
CN (1) CN114998788A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091935A (en) * 2023-04-07 2023-05-09 四川三思德科技有限公司 Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116091935A (en) * 2023-04-07 2023-05-09 四川三思德科技有限公司 Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium
CN116091935B (en) * 2023-04-07 2023-08-01 四川三思德科技有限公司 Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium

Similar Documents

Publication Publication Date Title
CN110135269B (en) Fire image detection method based on mixed color model and neural network
CN101236606B (en) Shadow cancelling method and system in vision frequency monitoring
CN107085714B (en) Forest fire detection method based on video
CN112036254B (en) Moving vehicle foreground detection method based on video image
CN101738394B (en) Method and system for detecting indoor smog
CN113139521B (en) Pedestrian boundary crossing monitoring method for electric power monitoring
CN105915840B (en) A method of the factory smoke discharge based on vision signal monitors automatically
JP2019505866A (en) Passerby head identification method and system
CN111723644A (en) Method and system for detecting occlusion of surveillance video
CN109087363B (en) HSV color space-based sewage discharge detection method
JP2008171392A (en) Image edge detection method, device therefor, and computer-readable recording medium embodying same
CN105069778B (en) Based on the industrial products detection method of surface flaw that target signature notable figure builds
CN110853077B (en) Self-adaptive infrared dynamic frame feature extraction method based on morphological change estimation
CN113096103A (en) Intelligent smoke image sensing method for emptying torch
CN114998788A (en) Smog judgment method based on video analysis
CN106651923A (en) Method and system for video image target detection and segmentation
CN112257523A (en) Smoke identification method and system of image type fire detector
CN111709964A (en) PCBA target edge detection method
CN112258403A (en) Method for extracting suspected smoke area from dynamic smoke
CN109949344B (en) Nuclear correlation filtering tracking method based on color probability target suggestion window
CN113657250A (en) Flame detection method and system based on monitoring video
CN106530292B (en) A kind of steel strip surface defect image Fast Identification Method based on line scan camera
CN112613456A (en) Small target detection method based on multi-frame differential image accumulation
CN114882401A (en) Flame detection method and system based on RGB-HSI model and flame initial growth characteristics
CN114863330A (en) Target object detection method, system and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination