CN114998788A - A smoke detection method based on video analysis - Google Patents

Abstract

The invention relates to the field of smoke detection, and discloses a smoke judgment method based on video analysis, in particular to a method for filtering HSV threshold values of video images and analyzing the change rate of the video images to determine whether smoke exists.

Description

A smoke detection method based on video analysis

technical field

The invention relates to the field of smoke detection, in particular to a method for performing HSV threshold filtering on a video image and analyzing the change rate of the video image to determine whether there is smoke.

Background technique

Video smoke detection is an important way to detect and discover early fires. Target detection networks, such as YOLO and Faster-RCNN, can obtain higher accuracy and faster detection speed after being trained on smoke datasets. However, since there are very few smoke datasets and the smoke itself has no fixed shape, the target detection network often has a certain missed detection rate and false detection rate, which is unacceptable for fire safety.

Before the target detection network was applied to smoke detection, smoke detection systems based on hand-crafted features were widely used. Filtering and filtering images according to the texture, color and dynamic characteristics of smoke can effectively reduce the false detection rate. Although the target detection network It occupies the mainstream in the current image detection field, but few smoke datasets and irregular morphological features of smoke make it impossible for a single target detection network to achieve satisfactory results, and it is also difficult to effectively utilize continuous video image information.

To this end, a method for smoke determination based on video analysis is proposed by combining neural network and traditional image processing methods.

SUMMARY OF THE INVENTION

The purpose of the present invention is to use the HSV color space to filter some non-smoky areas in the video image, continue to count the pixel changes of adjacent frames after filtering, and then use the YOLOv5 neural network to detect the image and find the suspected smoke area. The changes of image pixels before and after a period of time are accumulated respectively, and finally the total amount of changes in image pixels before and after the suspected smoke area is compared to determine whether there is smoke, which further improves the detection accuracy and reduces the false alarm rate.

The technical scheme of the present invention is as follows:

A method for determining smoke based on video analysis, comprising the following steps:

其中，

表示第k个候选烟雾区域左上角的坐标，

表示第k个候选烟雾区域右下角的坐标；Step 1: collect and obtain n frames of images in the surveillance video, denoted as I ₁ , I ₂ , ..., I _n ; use YOLOv5 to perform smoke detection on any i-th frame image I _i to obtain K _i candidate smoke regions, and use The kth candidate smoke area is recorded as

in,

represents the coordinates of the upper left corner of the kth candidate smoke area,

Represents the coordinates of the lower right corner of the kth candidate smoke area;

Step 2: Calculate and obtain the adjacent frame changes of n frames of images, and obtain the image set S={I _1a , I _2a ,...,I _n-1a }, and the specific steps are as follows:

Step 2.1: Perform HSV threshold filtering on any I _i and I _i+1 , i=1,2,...,n-1, convert the image from RGB color space to HSV color space, and obtain images G _i and G _i+1 , set the filter condition Q={V∈[150,255], S∈[0,25]}, and determine the HSV value of each pixel of G _i and G _i+1 , that is, according to formulas (1), ( 2) and (3) filter I _i and I _i+1 to obtain the filtered images I _i and I _i+1 under the filter condition Q I _iq and I _i+1q , in I _i and I _i+1 The RGB values of pixels that do not meet the filtering conditions are set to zero, and the HSV threshold filtering formula is as follows:

Among them, I _iR (x, y), I _iG (x, y) and I _iB (x, y) are the RGB values of the pixel points at coordinates (x, y) in I _i , respectively, G _iS (x, y) y) and G _iV (x, y) are respectively the SV values of the pixel points in I _i at coordinates (x, y), and I _iqR , I _iqG and I _iqB are respectively the coordinates (x, y) in I _i The pixel point at the place is filtered through the RGB value of the threshold value, and this value is stored in the filtered image I _iq , and the same operation is performed for I _i+1 to obtain the filtered image I _i+1q ;

Step 2.2: Calculate the RGB value sum C _iq (x, y) of the pixel points at any coordinates (x, y) in I _iq according to formula (4), and use the same method to calculate any coordinates (x, y) in I _i+1q . , y) the RGB value summation C _i+1q (x, y) of the pixel point at place; Calculate the change image I _ia of adjacent frame I _iq and I _i+1q according to formula (5);

C _iq (x,y)=I _iqR (x,y)+I _iqG (x,y)+I _iqB (x,y) (4)

Among them, I _ia (x, y) represents the value of the pixel point at the coordinate (x, y) in I _ia ;

Step 2.3: Erase and dilate the image _Iia ;

Step 3: Take S ₁ ={I _it-1a ,I _i-ta ,...,I _i-1a },S ₂ ={I _ia ,I _i+1a ,...,I _i+t-1a },S ₁ is the adjacent frame change image of the t frame before the ith frame image, S ₂ is the adjacent frame change image of the t frame after the ith frame image, 1≤it≤n, according to formula ( ₆ ) Calculate S1 and S The image pixels in ₂ are cumulatively changed to obtain images R ₁ and R ₂ ;

where | represents a bitwise OR operation;

重合的图像，得到R_1ik和R_2ik，统计R_1ik和R_2ik图像中三通道均为255的像素数，记为U_1ik和U_2ik，若U_1ik/U_2ik<0.5，则认为

处存在烟雾。Step _{4 :} Pick and candidate smoke regions from R1 and R2

Coincident images, get R _1ik and R _2ik , count the number of pixels in the three channels of R _1ik and R _2ik images are 255, record as U _1ik and U _2ik , if U _1ik /U _2ik <0.5, it is considered that

There is smoke.

The beneficial effects of the present invention are as follows: the video images before and after the smoke detection can be effectively used by the convolutional neural network model, the total amount of image pixel changes between adjacent frames of the monitoring video can be analyzed within a period of time, and the HSV color space filtering and corrosion are combined with The expansion operation reduces the influence of video noise and jitter on the statistics of image pixel changes, and eliminates some false detections caused by insufficient neural network accuracy.

Description of drawings

Fig. 1 is the unprocessed video image of the present invention;

Fig. 2 is the image after the present invention utilizes the convolutional neural network model to detect;

Fig. 3 is the image that the present invention is filtered through HSV and filter condition is Q;

Fig. 4 is the change image of adjacent frame of the present invention;

Fig. 5 is the image after corrosion and expansion of the present invention;

FIG. 6 is an image of accumulating pixel changes of 100 adjacent frames according to the present invention.

Detailed ways

The present invention is described in detail below with reference to examples and images.

A smoke determination method based on video analysis, the specific steps are as follows:

其中，

表示第k个候选烟雾区域左上角的坐标，

表示第k个候选烟雾区域右下角的坐标，图2为图1利用卷积神经网络模型检测后的图像，其中smoke表示检测到的对象类别，0.39表示置信度，方框为1个候选烟雾区域；Step 1: Collect and obtain n frames of images in the surveillance video, denoted as I ₁ , I ₂ ,..., I _n , and Figure 1 shows one of the unprocessed video images; The image I _i is subjected to smoke detection to obtain K _i candidate smoke regions, and the kth candidate smoke region among them is recorded as

in,

represents the coordinates of the upper left corner of the kth candidate smoke area,

Indicates the coordinates of the lower right corner of the kth candidate smoke area. Figure 2 is the image detected by the convolutional neural network model in Figure 1, where smoke represents the detected object category, 0.39 represents the confidence, and the box is a candidate smoke area. ;

Step 2: Calculate and obtain the adjacent frame changes of n frames of images, and obtain the image set S={I _1a , I _2a ,...,I _n-1a }, and the specific steps are as follows:

Step 2.1: Perform HSV threshold filtering on any I _i and I _i+1 , i=1,2,...,n-1, convert the image from RGB color space to HSV color space, and obtain images G _i and G _i+1 , set the filter condition Q={V∈[150,255], S∈[0,25]}, and determine the HSV value of each pixel of G _i and G _i+1 , that is, according to formulas (1), ( 2) and (3) filter I _i and I _i+1 to obtain the filtered images I _i and I _i+1 under the filter condition Q I _iq and I _i+1q , in I _i and I _i+1 The RGB values of pixels that do not meet the filtering conditions are set to zero, and the HSV threshold filtering formula is as follows:

Among them, I _iR (x, y), I _iG (x, y) and I _iB (x, y) are the RGB values of the pixel points at coordinates (x, y) in I _i , respectively, G _iS (x, y) y) and G _iV (x, y) are respectively the SV values of the pixel points in I _i at coordinates (x, y), and I _iqR , I _iqG and I _iqB are respectively the coordinates (x, y) in I _i The pixel point at the place is filtered through the RGB value of the threshold value, and this value is stored in the filtered image I _iq , and the same operation is performed for I _i+1 to obtain the filtered image I _i+1q ; Fig. 3 is that Fig. 1 passes through The image filtered by HSV and the filter condition is Q, as shown in the figure, after filtering, a small number of pixels that meet the conditions remain;

Step 2.2: Calculate the RGB value sum C _iq (x, y) of the pixel points at any coordinates (x, y) in I _iq according to formula (4), and use the same method to calculate any coordinates (x, y) in I _i+1q . , y) the RGB value summation C _i+1q (x, y) of the pixel point at place; Calculate the change image I _ia of adjacent frame I _iq and I _i+1q according to formula (5), Fig. 4 is Fig. 3 and its Changed images of adjacent frames;

C _iq (x,y)=I _iqR (x,y)+I _iqG (x,y)+I _iqB (x,y) (4)

Among them, I _ia (x, y) represents the value of the pixel point at the coordinate (x, y) in I _ia ;

Step 2.3: Corrode and dilate the image I _ia . Figure 5 is the image of Figure 4 after the erosion and dilation, and filter out the small interfering pixels;

Step 3: Take S ₁ ={I _it-1a ,I _i-ta ,...,I _i-1a },S ₂ ={I _ia ,I _i+1a ,...,I _i+t-1a },S ₁ is the adjacent frame change image of the t frame before the ith frame image, S ₂ is the adjacent frame change image of the t frame after the ith frame image, 1≤it≤n, according to formula ( ₆ ) Calculate S1 and S Figure ₂ shows the cumulative changes of image pixels in 2 to obtain images R ₁ and R ₂ . Figure 6 is an image of cumulative pixel changes of 100 adjacent frames. It can be seen from the figure that the cumulative number of pixels in the candidate smoke area is more than that in other non-candidate smoke areas;

where | represents a bitwise OR operation;

重合的图像，得到R_1ik和R_2ik，统计R_1ik和R_2ik图像中三通道均为255的像素数，记为U_1ik和U_2ik，若U_1ik/U_2ik<0.5，则认为

处存在烟雾。Step _{4 :} Pick and candidate smoke regions from R1 and R2

Coincident images, get R _1ik and R _2ik , count the number of pixels in the three channels of R _1ik and R _2ik images are 255, record as U _1ik and U _2ik , if U _1ik /U _2ik <0.5, it is considered that

There is smoke.

The content described in the embodiments of the present specification is only an enumeration of the realization forms of the inventive concept, and the protection scope of the present invention should not be regarded as limited to the specific forms stated in the embodiments, and the protection scope of the present invention also extends to the field Equivalent technical means that can be conceived by a skilled person according to the inventive concept.

Claims (1)

1. A smog judgment method based on video analysis is characterized by comprising the following steps:

step 1: acquiring n frames of images in the monitoring video, and recording as I ₁ ,I ₂ ,…,I _n (ii) a Using YOLOv5 to image I of any ith frame _i Carrying out smoke detection to obtain K _i A candidate smoke region, and the kth candidate smoke regionIs marked as

Wherein,

coordinates representing the upper left corner of the kth smoke candidate region,

coordinates representing the lower right corner of the kth smoke candidate region;

step 2: calculating adjacent frame changes of the n frames of images to obtain an image set S ═ { I } _1a ,I _2a ,…,I _n-1a }；

Step 2.1: for any I _i And I _i+1 Performing HSV threshold filtering, wherein i is 1,2, …, n-1, converting the image from RGB color space to HSV color space to obtain an image G _i And G _i+1 Setting a filter condition Q ═ V ∈ [150,255 ]],S∈[0,25]To G _i And G _i+1 The HSV value of each pixel point is judged, namely I is judged according to the formulas (1), (2) and (3) _i And I _i+1 Filtering to obtain I _i And I _i+1 Image I filtered under filtering condition Q _iq And I _i+1q ，I _i And I _i+1 The RGB values of the pixels which do not satisfy the filtering condition are all set to zero, and the HSV threshold filtering formula is as follows:

wherein, I _iR (x,y)、I _iG (x, y) and I _iB (x, y) are each I _i RGB value, G, of pixel point centered at coordinate (x, y) _iS (x, y) and G _iV (x, y) are each I _i SV value, I, of a pixel point centered at coordinate (x, y) _iqR 、I _iqG And I _iqB Are respectively I _i The RGB value of the pixel point at the coordinate (x, y) after threshold value filtration is stored in the filtered image I _iq In respect of I _i+1 The same operation is carried out to obtain a filtered image I _i+1q ；

Step 2.2: calculation of I according to formula (4) _iq The total RGB value C of the pixel points at any coordinate (x, y) _iq (x, y) and calculating to obtain I by the same method _i+1q The total RGB value C of the pixel points at any coordinate (x, y) _i+1q (x, y); computing neighboring frame I according to equation (5) _iq And I _i+1q Change image I of _ia ；

C _iq (x,y)＝I _iqR (x,y)+I _iqG (x,y)+I _iqB (x,y) (4)

Wherein, I _ia (x, y) represents I _ia The value of the pixel point at coordinate (x, y);

step 2.3: for image I _ia Carrying out corrosion and expansion;

and step 3: get S ₁ ＝{I _i-t-1a ,I _i-ta ,…,I _i-1a }，S ₂ ＝{I _ia ,I _i+1a ,…,I _i+t-1a }，S ₁ For t frames adjacent to the ith frame image, S ₂ For the adjacent frame change image of t frames following the ith frame image, i-t is more than or equal to 1 and less than or equal to n, S is calculated according to the formula (6) ₁ And S ₂ The accumulated change of the image pixels is obtained to obtain an image R ₁ And R ₂ ；

Wherein, | represents a bitwise or operation;

and 4, step 4: from R ₁ And R ₂ To select and candidate smoke regions

Superposed images, resulting in R _1ik And R _2ik Statistics of R _1ik And R _2ik The number of pixels with 255 three channels in the image is recorded as U _1ik And U _2ik If U is _1ik /U _2ik If less than 0.5, it is considered that

Where smoke is present.

Images