CN110751089A - Flame target detection method based on digital image and convolution characteristic - Google Patents

Flame target detection method based on digital image and convolution characteristic Download PDF

Info

Publication number
CN110751089A
CN110751089A CN201910992933.6A CN201910992933A CN110751089A CN 110751089 A CN110751089 A CN 110751089A CN 201910992933 A CN201910992933 A CN 201910992933A CN 110751089 A CN110751089 A CN 110751089A
Authority
CN
China
Prior art keywords
flame
image
convolution
video
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910992933.6A
Other languages
Chinese (zh)
Inventor
赵亚琴
卢鹏
丁志鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Forestry University
Original Assignee
Nanjing Forestry University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Forestry University filed Critical Nanjing Forestry University
Priority to CN201910992933.6A priority Critical patent/CN110751089A/en
Publication of CN110751089A publication Critical patent/CN110751089A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Abstract

Because the flame detection model based on the image characteristics has low generalization and the deep neural network model has higher requirement on the number of training samples, the patent provides a flame target detection method based on digital images and convolution characteristics, and firstly, a data set containing video dynamic characteristics is made; then replacing the standard convolution of VGG16 in the classic Faster R-CNN with a depth separable convolution, and reducing the number of convolution layers; then cutting out 256 image blocks from an original image according to a candidate frame generated by RPN, and extracting LBP (local binary pattern) features for each image block; the size of an output characteristic diagram of the ROI pooling layer and the number of neurons of a full connection layer are reduced through convolution, and network parameters are further reduced; and finally, combining the extracted LBP characteristics, the dynamic characteristics in the data set and the characteristic vectors tiled after pooling, and sending the combined dynamic characteristics and the characteristic vectors into a full-connection layer for classification and regression. The flame target detection model that this patent was constructed has higher detection precision, is convenient for improve to the not enough of test result, and the flexibility is high.

Description

Flame target detection method based on digital image and convolution characteristic
Technical Field
The present invention relates to a method for detecting flame.
Background
The rapid detection of the flame has great significance for early warning and timely processing of the fire, the fire monitoring video system is an important means for preventing the fire, and the flame is an important visual sign of the fire, so that the flame identification method based on the image characteristics becomes a hotspot in the fire prevention field. Flame identification methods based on image characteristics are mainly divided into two types, namely flame static characteristics and dynamic characteristics. Flame static characteristics mainly comprise edge and color space information and the like, and the Joe construction strength and the like provide a flame detection method based on edge characteristics, and the edge characteristic information is extracted from the image so as to identify the flame; chentianyan and the like propose a YCbCr color space threshold segmentation-based method, and a flame region is extracted by utilizing the characteristics of flame in a color space. The flame dynamic characteristics comprise a Gaussian mixture background model, frame difference and the like, for example, Liqinghui and the like detect a moving target in a video sequence by using a self-adaptive Gaussian mixture model, and then a clustering algorithm is adopted to divide a suspected flame area and a non-fire area; stadler et al extracts flame candidate regions using a weighted interframe difference method, distinguishes flames from non-flames by high flicker frequency, and applies threshold filtering and a high-pass filter to intensity variations to improve the recognition rate. Since the outer flame of the flame has moving characteristics, and the moving characteristics of the flame kernel are not obvious, it is difficult to distinguish the flame from the interfering objects such as safflower or illumination similar to the flame color and texture by using the common characteristics based on static images such as color, edge and texture, and the detection method based on the dynamic characteristics can make the flame kernel area be wrongly judged as the background.
The deep convolutional neural network can learn and induce the image characteristics, has stronger discrimination capability and generalization capability, and is more and more widely applied to the field of image processing. Girshick et al propose that fast R-CNN is used for target detection, and a candidate region is firstly generated and then classified and regressed; redmon et al propose YOLO, which predicts the frame and class probabilities directly from the entire input image with only one evaluation, and has a lower accuracy but higher detection speed than Faster R-CNN; the SSD and the YOLO proposed by Liu et al belong to a single detector, and meanwhile, the anchor point frame mechanism of Faster R-CNN is used for reference, so that the rapid characteristic is kept under the condition of no precision loss.
The training of the deep convolutional neural network has higher requirements on the capacity of a data set, although a good effect can be achieved sometimes by using transfer learning on small and medium data sets, the similarity between the flame and an object in the current public data set is extremely small, and the effect of the transfer learning is extremely small.
Disclosure of Invention
The invention aims to provide a flame target detection method based on digital images and convolution characteristics, which has the advantages of few convolution layers, few network parameters and high flexibility.
In order to achieve the above object, the method for detecting a flame target based on digital images and convolution features according to the present invention comprises the following steps:
1) dividing a video to be detected into a plurality of video blocks, wherein each video block comprises 31 frames of images, reading the data of the video blocks and extracting dynamic characteristics from each video block: flame area variation characteristics, shape similarity characteristics and flicker frequency characteristics;
2) randomly extracting an image from the video block and carrying out image regularization on the image;
3) convolving the normalized image to generate a characteristic diagram;
4) generating 256 candidate areas on the feature diagram obtained by convolution by adopting an RPN (fast R-CNN) network of the Faster R-CNN, cutting out corresponding image blocks from the normalized image through candidate area coordinates, and extracting LBP (local binary pattern) features for each image block;
5) pooling the characteristic maps ROI corresponding to the candidate regions to a fixed size, and then reducing dimensions by convolution and flattening to a characteristic vector with fixed dimensions;
6) and dividing the network into two branches, wherein one branch is used for correcting the coordinates of the frame, the other branch is used for combining the dynamic characteristics and the LBP characteristics after the dimensionality reduction of the characteristic vector, and the softmax classifier is used for carrying out flame identification.
As a further improvement of the flame target detection method, in step 2), when the image is normalized, the short edge of the image is set to be not more than 300 pixels, and the long edge of the image is set to be not more than 500 pixels, and the image beyond the limit is scaled in an equal ratio.
As a further improvement to the flame target detection method, in step 3), the layer 1 of the convolutional network is a standard convolution, and then the layer 11 is a deep separable convolution; the convolution step size of the convolution network layers 3, 5, 7 and 11 is 2, so that the convolution reduces the length and width of the feature map by half each time, and finally reduces the length and width of the feature map to 1/16 of the normalized image.
As a further improvement of the flame target detection method, in step 5), the feature maps corresponding to the candidate regions are respectively pooled to a fixed size of 5 × 5 × 512, and then subjected to convolution with a step size of 2, so as to reduce the size of the feature maps and reduce the dimension of the flattened feature vectors, wherein the size of the flattened feature vectors is 256 × 2048.
As a further improvement to the flame target detection method, in step 1),
extracting a suspected flame area from a video image, and representing the change of the flame area by using the change of the number of pixel points, wherein the area change rate of the flame in a video frame is as follows:
Figure BDA0002238840740000021
in the formula, StAnd St-1Respectively representing the number of pixel points, delta A, in the suspected flame area of the t frame and the t-1 frame in the video sequencetRepresenting the change rate of the total number of the pixel points in the suspected flame area, namely the change rate of the area of the suspected flame area; each video block has 31 frames of images, and 30-dimensional flame area change feature vectors are extracted.
Flame shape similarity ξtThe calculation formula is as follows:
Figure BDA0002238840740000022
wherein, bt(x,y) And bt+3(x, y) are the image sequences of the t-th frame and the t + 3-th frame, respectively, omega is the suspected flame area, (x,y) Are the coordinates of the points of the image sequence. Each video block extracts a 10-dimensional shape similarity feature vector.
The flame flicker frequency is used as an important criterion for recognizing the flame of the forest fire. Considering that the size of a video block is fixed, the flame flicker frequency characteristic is indirectly reflected by the change of the pixels at the edge of the suspected flame area, and the formula is as follows:
ΔPt＝|Pt-Pt-3|
wherein, PtAnd Pt-3Representing the suspected flame area edge pixel points of the t frame and the t-3 frame in the video sequence; delta PtRepresenting edge pixel variation; extracting a 10-dimensional flicker frequency characteristic vector from each video block;
and combining the flame area change, the shape similarity and the flicker frequency characteristics, and extracting a 50-dimensional dynamic characteristic vector from each video block.
As a further improvement of the flame target detection method, in step 4), 59-dimensional LBP feature vectors are respectively extracted for the RGB three channels of each image block, and 177-dimensional LBP feature vectors are totally extracted.
As a further improvement to the flame target detection method, in step 6), the class prediction network first uses one fully-connected layer to reduce the feature vector to 256 × 512, then combines the feature vector with the previously extracted 256 × 177 LBP feature vector and the 256 × 50 dynamic feature vector extracted from the video data to form a new feature vector of 256 × 739, then generates feature vectors with dimensions equal to the total number of classes from two fully-connected layers, and finally obtains the classification result by the softmax classifier.
The invention has the beneficial effects that: aiming at the problems that the flame detection model based on the image characteristics is not strong in generalization, the deep neural network model has high requirements on the number of training samples and is difficult to improve aiming at the test result, an improved Faster R-CNN model combining flame space-time characteristics is constructed for flame target detection. Firstly, making a data set containing video dynamic characteristics; then replacing the standard convolution of VGG16 in the classic Faster R-CNN with a depth separable convolution, and reducing the number of convolution layers; then cutting out 256 image blocks from an original image according to a candidate frame generated by RPN, and extracting LBP (local binary pattern) features for each image block; the size of an output characteristic diagram of the ROI pooling layer and the number of neurons of a full connection layer are reduced through convolution, and network parameters are further reduced; and finally, combining the extracted LBP characteristics, the dynamic characteristics in the data set and the characteristic vectors tiled after pooling, and sending the combined dynamic characteristics and the characteristic vectors into a full-connection layer for classification and regression. The flame target detection model that this patent was established has higher detection precision to be convenient for improve to the not enough of test result, have higher flexibility.
Thus, an improved Faster R-CNN model incorporating flame spatiotemporal features was constructed herein for flame target detection. The space-time characteristics comprise time sequence dynamic characteristics (including dynamic characteristics such as flame area change, shape similarity and flicker frequency characteristics) which cannot be extracted by the Faster R-CNN deep convolution neural network and unobvious space texture characteristics, and are combined with the Faster R-CNN deep convolution neural network, so that the target characteristics can be extracted more comprehensively, and a certain generalization of the model is ensured.
Drawings
FIG. 1 is a flow chart of extracting a suspected flame region;
FIG. 2 is a flow chart of the modified Faster R-CNN algorithm;
FIG. 3 is a structural diagram of an improved Faster R-CNN model.
Detailed Description
The following examples are provided to illustrate the flame target detection method based on digital images and convolution features of the present invention.
1 data set production
1.1 training set format
The model detects flames by extracting static features and dynamic features, wherein the static features comprise features extracted by a convolutional network and LBP texture features, and the dynamic features comprise flame area change features, shape similarity features and flicker frequency features.
Static features must be extracted in real time according to candidate frames in the training process, and dynamic features are extracted through video frames and are irrelevant to network parameter changes, so that the dynamic features are extracted from a data set and taken together with images with labels as a training set before model training, and redundant calculation is simplified and simplified for facilitating model training. The model training needs a large amount of flame videos, the videos need to comprise more than ten different scenes, the videos are divided into a plurality of video blocks, each video block comprises 31 frames of images, 1 image is extracted from the videos, a box coordinate and a category label are marked on the images, and then dynamic features are extracted from each video block. Finally, each individual training sample format is shown in table 1.
TABLE 1 training set sample format
Sample numbering Image data Coordinates of the square Object classes Dynamic feature vector
2.2 dynamic feature extraction
Firstly, a method of combining color and motion information is adopted to extract a suspected flame area, and the extraction process is shown in fig. 1.
Color pixels are detected using three rules:
Figure BDA0002238840740000041
r, G, B are three channels of red, green and blue, TRIs the threshold of S channel, S refers to the saturation of the pixel, TSBeing S-channelAnd (4) a threshold value.
And detecting the motion pixels by adopting a mixed Gaussian background model method.
Next, flame area variation, shape similarity, and flicker frequency characteristics are extracted.
The area of the flame in the early fire presents a continuous and expandable growth trend, so that the change of the area of the flame area along with time can be used as a powerful basis for forest fire detection. In the video image, the change of the flame area can be represented by the change of the number of the pixel points. After color and motion pixel detection, obtaining a binary image of a suspected flame area, and calculating the total number of pixel points of a flame target in the binary image, wherein the area change rate of the flame in the video frame is as follows:
Figure BDA0002238840740000051
in the formula, StAnd St-1Respectively representing the number of pixel points, delta A, in the suspected flame area of the t frame and the t-1 frame in the video sequencetAnd (3) representing the change rate of the total number of the pixel points in the suspected flame area, namely the area change rate of the suspected flame area. Each video block has 31 frames of images, and 30-dimensional flame area change feature vectors (t is 2, 3, 4, 5, … …, 31) are extracted in total.
The difference between two adjacent frames is small, so that the similarity of images separated by 3 frames is calculated, and the shape similarity ξ is obtainedtThe calculation formula is as follows:
Figure BDA0002238840740000052
wherein, bt(x, y) and bt+3And (x, y) are the image sequences of the t frame and the t +3 frame respectively, and omega is a suspected flame area. (x, y) are coordinates of sequence points, and a 10-dimensional shape similarity feature vector (t ═ 1, 4, 7, 10, … …, 28) is extracted for each video block.
When the combustible materials are combusted, flames flicker continuously, the flickering surfaces of the flames look disordered and irregular, and each combustible material has the fixed flickering frequency, so that the flickering frequency of the flames can be selected as an important criterion for identifying the flames of forest fires. Considering that the size of a video block is fixed, the flicker frequency characteristic is indirectly reflected by the edge pixel change, and the formula is as follows:
ΔPt＝|Pt-Pt-3| (4)
wherein, PtAnd Pt-3Representing the suspected flame area edge pixel points of the t frame and the t-3 frame in the video sequence; delta PtIndicating edge pixel variation. Each video block extracts a 10-dimensional flicker frequency feature vector (t ═ 4, 7, 10, 13, … …, 31).
After merging, each video block extracts a 50-dimensional motion feature vector.
3 model training
The flow chart of the improved Faster R-CNN algorithm is shown in FIG. 2, and the steps are as follows: 1) read the data. 2) Then the read image data is subjected to image regularization. 3) Convolving the normalized image to produce a feature map. 4) Generating 256 candidate regions on the feature map obtained by convolution by using an RPN network of Faster R-CNN, and extracting an LBP feature for each candidate region. 5) Pooling the feature maps ROI corresponding to each candidate region to a fixed size, and then reducing dimensions by convolution and flattening to a feature vector of a fixed dimension. 6) And dividing the network into two branches, wherein one branch is used for correcting the coordinates of the frame, and the other branch combines the dynamic features and the LBP features after reducing the dimension of the feature vectors, and then predicts the category.
The specific structure of the model is shown in fig. 3 and table 4. The method comprises the following specific steps:
1) read training set data in the specified format.
2) And performing image normalization on the image data, setting the short edge of the image not to exceed 300 and the long edge of the image not to exceed 500, and performing equal-ratio scaling on the image which exceeds the limit.
3) Feeding the normalized image into a convolution network to generate a feature map, wherein the first layer of the convolution network is standard convolution and the later 11 layers are depth separable convolution as shown in FIG. 3; wherein the convolution step of conv3, conv5, conv7 and conv11 is 2, so that the convolution reduces the length and width of the feature map by half each time, and finally reduces the length and width of the feature map to 1/16 of the normalized image. The size of the finally obtained characteristic diagram is the same as that of the original Faster R-CNN characteristic diagram, but the parameters of the convolutional network are reduced to about 1/9, so that the network is lighter.
4) Generating 256 candidate areas on the feature map obtained by convolution by adopting an RPN network of Faster R-CNN, cutting out corresponding image blocks from the normalized image through candidate area coordinates, and respectively extracting 59-dimensional LBP feature vectors and 177-dimensional LBP feature vectors in total for RGB three channels of each image block.
5) Respectively pooling the feature maps corresponding to the candidate regions to a fixed size of 5 × 5 × 512, performing convolution with a step size of 2 (conv 13 in fig. 3), reducing the size of the feature maps, and reducing the dimension of the flattened feature vectors, wherein the size of the flattened feature vectors is 256 × 2048.
6) The network is divided into two branches, one branch uses two full connection layers to carry out frame regression, and frame coordinates are corrected; the other branch is used to predict the class. The class prediction network firstly uses a layer of full connection layer to reduce the dimension of the feature vector to 256 multiplied by 512, then combines the feature vector with the LBP feature vector of 256 multiplied by 177 extracted before and the dynamic feature vector of 256 multiplied by 50 extracted from the video data to be a new feature vector of 256 multiplied by 739, then connects two layers of full connection layer to reduce the dimension to the total number of classes, and finally obtains the classification result by a softmax classifier.
The parameters for training the convolutional network, RPN network and fully-connected layer with the gradient descent method are shown in table 2.
TABLE 2 fast R-CNN model Structure
Figure BDA0002238840740000061
Figure BDA0002238840740000071
4 model prediction
And reading the video block data by the prediction stage model so as to realize the detection of the flame target in the video. Unlike the training phase, what the model predicts is an image patch without block coordinates and class labels input to the network. The model firstly extracts the dynamic characteristics of the video block according to the above mode, and then extracts an image from the video block to perform operations such as regularization and subsequent convolution. Once the model detects a flame, the frame and class of the flame region is labeled on the extracted image.
The invention divides the video into video blocks, extracts dynamic characteristics for each video block and makes a training set according to a fixed format. The network model adopts a light-weight depth separable convolution network to extract an image characteristic diagram, adopts an RPN network of Faster Rcnn to generate candidate regions, extracts LBP characteristic vectors for each candidate region, reduces the dimension of the convolved characteristic vectors, combines the characteristic vectors with the LBP characteristic vectors and the dynamic characteristic vectors, and then obtains a classification result and frame regression respectively through a full connection layer, so that the flame detection identification precision is high.

Claims (7)

1. The flame target detection method based on the digital image and the convolution characteristic is characterized in that: it comprises the following steps:
1) dividing a video to be detected into a plurality of video blocks, wherein each video block comprises 31 frames of images, reading the data of the video blocks and extracting dynamic characteristics from each video block: flame area variation characteristics, shape similarity characteristics and flicker frequency characteristics;
2) randomly extracting an image from the video block and carrying out image regularization on the image;
3) convolving the normalized image to generate a characteristic diagram;
4) generating 256 candidate areas on the feature diagram obtained by convolution by adopting an RPN (fast R-CNN) network of the Faster R-CNN, cutting out corresponding image blocks from the normalized image through candidate area coordinates, and extracting LBP (local binary pattern) features for each image block;
5) pooling the characteristic maps ROI corresponding to the candidate regions to a fixed size, and then reducing dimensions by convolution and flattening to a characteristic vector with fixed dimensions;
6) and dividing the network into two branches, wherein one branch is used for correcting the coordinates of the frame, and the other branch is used for combining the dynamic characteristics and the LBP characteristics after the dimensionality reduction of the characteristic vector, and performing flame identification by using a softmax classifier.
2. A flame target detection method as defined in claim 1, wherein: and 2) in step 2), setting the short edge of the image not to exceed 300 pixels and the long edge of the image not to exceed 500 pixels when the image is normalized, and scaling the image exceeding the limit in an equal ratio.
3. A flame target detection method as defined in claim 2, wherein: in the step 3), the 1 st layer of the convolution network is standard convolution, and the later 11 layers are depth separable convolution; the convolution step size of the convolution network layers 3, 5, 7 and 11 is 2, so that the convolution reduces the length and width of the feature map by half each time, and finally reduces the length and width of the feature map to 1/16 of the normalized image.
4. A flame target detection method as defined in claim 1, wherein: and 5), pooling the characteristic maps corresponding to the candidate regions to a fixed size of 5 × 5 × 512 in ROI respectively, and performing convolution with a step length of 2 to reduce the size of the characteristic maps and the dimension of the flattened characteristic vectors, wherein the size of the flattened characteristic vectors is 256 × 2048.
5. A flame target detection method as defined in claim 1, wherein: in the step 1), the step (A) is carried out,
extracting a suspected flame area from a video image, and representing the change of the flame area by using the change of the number of pixel points, wherein the area change rate of the flame in a video frame is as follows:
Figure FDA0002238840730000011
in the formula, StAnd St-1Respectively representing the number of pixel points, delta A, in the suspected flame area of the t frame and the t-1 frame in the video sequencetRepresenting the change rate of the total number of the pixel points in the suspected flame area, namely the change rate of the area of the suspected flame area; each video block has 31 frames of images, and 30-dimensional flame area change characteristic vectors are extracted in total;
flame shape similarity ξtThe calculation formula is as follows:
Figure FDA0002238840730000021
wherein, bt(x, y) and bt+3(x, y) are the image sequences of the t frame and the t +3 frame respectively, omega is a suspected flame area, and each video block extracts 10-dimensional shape similarity characteristic vectors;
the method indirectly reflects the flame flicker frequency characteristics by using the edge pixel change of the suspected flame area, and the formula is as follows:
ΔPt＝|Pt-Pt-3|
wherein, PtAnd Pt-3Representing the suspected flame area edge pixel points of the t frame and the t-3 frame in the video sequence; delta PtRepresenting edge pixel variation; extracting a 10-dimensional flicker frequency characteristic vector from each video block;
and combining the flame area change, the shape similarity and the flicker frequency characteristics, and extracting a 50-dimensional dynamic characteristic vector from each video block.
6. A flame target detection method as defined in claim 5, wherein: in step 4), 59-dimensional LBP feature vectors are respectively extracted from the RGB three channels of each image block, and 177-dimensional LBP feature vectors are extracted in total.
7. A flame target detection method as defined in claim 6, wherein: in step 6), the class prediction network firstly uses one full-link layer to reduce the dimension of the feature vector to 256 × 512, then combines the feature vector with the LBP feature vector of 256 × 177 extracted previously and the dynamic feature vector of 256 × 50 extracted from the video data into a new feature vector of 256 × 739, then connects two full-link layers to reduce the dimension to the total number of classes, and finally obtains the classification result by the softmax classifier.
CN201910992933.6A 2019-10-18 2019-10-18 Flame target detection method based on digital image and convolution characteristic Pending CN110751089A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910992933.6A CN110751089A (en) 2019-10-18 2019-10-18 Flame target detection method based on digital image and convolution characteristic

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910992933.6A CN110751089A (en) 2019-10-18 2019-10-18 Flame target detection method based on digital image and convolution characteristic

Publications (1)

Publication Number Publication Date
CN110751089A true CN110751089A (en) 2020-02-04

Family

ID=69278796

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910992933.6A Pending CN110751089A (en) 2019-10-18 2019-10-18 Flame target detection method based on digital image and convolution characteristic

Country Status (1)

Country Link
CN (1) CN110751089A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111223265A (en) * 2020-04-16 2020-06-02 上海翼捷工业安全设备股份有限公司 Fire detection method, device, equipment and storage medium based on neural network
CN112541397A (en) * 2020-11-17 2021-03-23 南京林业大学 Flame detection method based on improved ViBe algorithm and lightweight convolutional network
CN112633061A (en) * 2020-11-18 2021-04-09 淮阴工学院 Lightweight FIRE-DET flame detection method and system
CN113516618A (en) * 2021-04-08 2021-10-19 河南中烟工业有限责任公司 Cigarette bead blasting defect detection method, device and equipment based on fast RCNN
CN113657250A (en) * 2021-08-16 2021-11-16 南京图菱视频科技有限公司 Flame detection method and system based on monitoring video
CN113723300A (en) * 2021-08-31 2021-11-30 平安国际智慧城市科技股份有限公司 Artificial intelligence-based fire monitoring method and device and storage medium
CN113743190A (en) * 2021-07-13 2021-12-03 淮阴工学院 Flame detection method and system based on BiHR-Net and YOLOv3-head
CN114093116A (en) * 2020-08-25 2022-02-25 中国电信股份有限公司 Method, device and system for fire detection

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110064264A1 (en) * 2008-05-08 2011-03-17 Utc Fire & Security System and method for video detection of smoke and flame
CN105788142A (en) * 2016-05-11 2016-07-20 中国计量大学 Video image processing-based fire detection system and detection method
CN109376747A (en) * 2018-12-11 2019-02-22 北京工业大学 A kind of video flame detecting method based on double-current convolutional neural networks
CN109815904A (en) * 2019-01-25 2019-05-28 吉林大学 Fire identification method based on convolutional neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110064264A1 (en) * 2008-05-08 2011-03-17 Utc Fire & Security System and method for video detection of smoke and flame
CN105788142A (en) * 2016-05-11 2016-07-20 中国计量大学 Video image processing-based fire detection system and detection method
CN109376747A (en) * 2018-12-11 2019-02-22 北京工业大学 A kind of video flame detecting method based on double-current convolutional neural networks
CN109815904A (en) * 2019-01-25 2019-05-28 吉林大学 Fire identification method based on convolutional neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
徐铭铭;周宏平等: "基于时空特征的林火视频火焰识别研究", 《林业工程学报》 *
林高华: "基于动态纹理和卷积神经网络的视频烟雾探测方法研究", 《中国优秀博硕士学位论文全文数据库(博士)工程科技Ⅱ辑》 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111223265A (en) * 2020-04-16 2020-06-02 上海翼捷工业安全设备股份有限公司 Fire detection method, device, equipment and storage medium based on neural network
CN114093116A (en) * 2020-08-25 2022-02-25 中国电信股份有限公司 Method, device and system for fire detection
CN112541397A (en) * 2020-11-17 2021-03-23 南京林业大学 Flame detection method based on improved ViBe algorithm and lightweight convolutional network
CN112541397B (en) * 2020-11-17 2022-04-01 南京林业大学 Flame detection method based on improved ViBe algorithm and lightweight convolutional network
CN112633061A (en) * 2020-11-18 2021-04-09 淮阴工学院 Lightweight FIRE-DET flame detection method and system
CN112633061B (en) * 2020-11-18 2023-03-24 淮阴工学院 Lightweight FIRE-DET flame detection method and system
CN113516618A (en) * 2021-04-08 2021-10-19 河南中烟工业有限责任公司 Cigarette bead blasting defect detection method, device and equipment based on fast RCNN
CN113743190A (en) * 2021-07-13 2021-12-03 淮阴工学院 Flame detection method and system based on BiHR-Net and YOLOv3-head
CN113743190B (en) * 2021-07-13 2023-12-22 淮阴工学院 Flame detection method and system based on BiHR-Net and YOLOv3-head
CN113657250A (en) * 2021-08-16 2021-11-16 南京图菱视频科技有限公司 Flame detection method and system based on monitoring video
CN113723300A (en) * 2021-08-31 2021-11-30 平安国际智慧城市科技股份有限公司 Artificial intelligence-based fire monitoring method and device and storage medium

Similar Documents

Publication Publication Date Title
CN110751089A (en) Flame target detection method based on digital image and convolution characteristic
CN110135269B (en) Fire image detection method based on mixed color model and neural network
Liang et al. Material based salient object detection from hyperspectral images
CN107016357B (en) Video pedestrian detection method based on time domain convolutional neural network
US20210256258A1 (en) Method, apparatus, and computer program for extracting representative characteristics of object in image
Arévalo et al. Shadow detection in colour high‐resolution satellite images
CN105184818B (en) A kind of video monitoring anomaly detection method and its detecting system
CN107025652A (en) A kind of flame detecting method based on kinetic characteristic and color space time information
CN115496760B (en) Donkey-hide gelatin quality identification method
CN104077605A (en) Pedestrian search and recognition method based on color topological structure
CN108921120B (en) Cigarette identification method suitable for wide retail scene
CN103295013A (en) Pared area based single-image shadow detection method
CN112861635A (en) Fire and smoke real-time detection method based on deep learning
CN111898627B (en) SVM cloud microparticle optimization classification recognition method based on PCA
CN111126115A (en) Violence sorting behavior identification method and device
CN108921857A (en) A kind of video image focus area dividing method towards monitoring scene
Chowdhury et al. Vegetables detection from the glossary shop for the blind
CN113221763A (en) Flame identification method based on video image brightness
Dawod et al. ResNet interpretation methods applied to the classification of foliar diseases in sunflower
Chakraborty et al. Bangladeshi road sign detection based on YCbCr color model and DtBs vector
Saha et al. Unsupervised multiple-change detection in VHR optical images using deep features
CN113762162A (en) Fire early warning method and system based on semantic segmentation and recognition
Deshmukh et al. Real-time traffic sign recognition system based on colour image segmentation
Shi et al. Video-based fire detection with spatio-temporal SURF and color features
CN107341456B (en) Weather sunny and cloudy classification method based on single outdoor color image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination