CN108765454A - A kind of smog detection method, device and device end based on video - Google Patents
A kind of smog detection method, device and device end based on video Download PDFInfo
- Publication number
- CN108765454A CN108765454A CN201810380051.XA CN201810380051A CN108765454A CN 108765454 A CN108765454 A CN 108765454A CN 201810380051 A CN201810380051 A CN 201810380051A CN 108765454 A CN108765454 A CN 108765454A
- Authority
- CN
- China
- Prior art keywords
- image block
- block
- original picture
- smog
- candidate image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention is suitable for technical field, provides a kind of smog detection method, device and terminal device based on video.This method includes：Energy measuring is carried out to the first candidate image block, obtains the second candidate image block that energy meets the second preset condition；The direction of motion of moving target in the second candidate image block is detected, the third candidate image block that the direction of motion meets third preset condition is obtained；The feature vector of the third candidate image block is obtained, and described eigenvector is inputted in preset model, if output result meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.The present invention realizes the real-time detection to monitoring area using computer vision technique and artificial intelligence technology, and find the position of smog appearance, so as to accurately a little be positioned to catching fire, the generation and sprawling of fire are avoided, there is stronger practicability and ease for use.
Description
Technical field
The invention belongs to technical field more particularly to a kind of smog detection method, device and device ends based on video.
Background technology
Traditional fire detection technology mainly uses smoke alarm, infrared detector, ultraviolet detector etc., works as detector
More than given threshold value, then detector alarm, but they are unsuitable for large-space clean factory building and open area (such as grassland, forest, tunnel
Road, airport, market, bulk storage plant etc.) and there are response speeds it is slow, rate of false alarm is high the problems such as.And the Smoke Detection based on video
Method realizes the real-time detection to monitoring area using computer vision technique, artificial intelligence technology, and finds smog appearance
Position avoids the generation and sprawling of fire so as to accurately a little be positioned to catching fire.The advantage of this technology has：It can
Smog monitoring is carried out to large-scale scene, fast response time, environmental pollution are small.
The existing smog detection method based on image uses some features of smog to identify, such as color, profile, mould
Paste, texture etc..Due to the complexity and irregularities of smog, the defects of there are anti-interference is not high and bad adaptability, to lead
Cause rate of false alarm high.
Invention content
In view of this, an embodiment of the present invention provides a kind of smog detection method, device and device end based on video,
To solve the problems, such as that there are rate of false alarms is high for the smog detection method based on video in the prior art.
The first aspect of the embodiment of the present invention provides a kind of smog detection method based on video, including：
To video flowing carry out motion detection, obtain motion target area image, and to every frame motion target area image into
Row piecemeal obtains original picture block；
Color detection is carried out to the original picture block of present frame, obtains the first candidate figure that color meets the first preset condition
As block；
Energy measuring is carried out to the first candidate image block, obtains the second candidate figure that energy meets the second preset condition
As block；
The direction of motion of moving target in the second candidate image block is detected, the direction of motion is obtained and meets third
The third candidate image block of preset condition；
The feature vector of the third candidate image block is obtained, and described eigenvector is inputted in preset model, if
It exports result and meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
The second aspect of the embodiment of the present invention provides a kind of mist detecting device based on video, including：
To video flowing carry out motion detection, obtain motion target area image, and to every frame motion target area image into
Row piecemeal obtains original picture block；
Color detection is carried out to the original picture block of present frame, obtains the first candidate figure that color meets the first preset condition
As block；
Energy measuring is carried out to the first candidate image block, obtains the second candidate figure that energy meets the second preset condition
As block；
The direction of motion of moving target in the second candidate image block is detected, the direction of motion is obtained and meets third
The third candidate image block of preset condition；
The feature vector of the third candidate image block is obtained, and described eigenvector is inputted in preset model, if
It exports result and meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
The third aspect of the embodiment of the present invention provides a kind of terminal device, including memory, processor and is stored in
In the memory and the computer program that can run on the processor, which is characterized in that described in the processor executes
Following steps are realized when computer program：
Motion detection block obtains motion target area image, and transport to every frame for carrying out motion detection to video flowing
Moving-target area image carries out piecemeal and obtains original picture block；
Color detection module carries out color detection for the original picture block to present frame, obtains color and meets first in advance
If the first candidate image block of condition；
Energy detection module carries out energy measuring to the first candidate image block, obtains energy and meets the second default item
Second candidate image block of part；
Direction of motion detection module is examined for the direction of motion to moving target in the second candidate image block
It surveys, obtains the third candidate image block that the direction of motion meets third preset condition；
Determining module obtains the feature vector of the third candidate image block, and described eigenvector is inputted preset
In model, if output result meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
The fourth aspect of the embodiment of the present invention provides a kind of computer readable storage medium, the computer-readable storage
Media storage has computer program, which is characterized in that the computer program realizes following steps when being executed by processor：
To video flowing carry out motion detection, obtain motion target area image, and to every frame motion target area image into
Row piecemeal obtains original picture block；
Color detection is carried out to the original picture block of present frame, obtains the first candidate figure that color meets the first preset condition
As block；
Energy measuring is carried out to the first candidate image block, obtains the second candidate figure that energy meets the second preset condition
As block；
The direction of motion of moving target in the second candidate image block is detected, the direction of motion is obtained and meets third
The third candidate image block of preset condition；
The feature vector of the third candidate image block is obtained, and described eigenvector is inputted in preset model, if
It exports result and meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
The embodiment of the present invention obtains motion target area image, and transport to every frame by carrying out motion detection to video flowing
Moving-target area image carries out piecemeal and obtains original picture block；Color detection is carried out to the original picture block of present frame, obtains face
First candidate image block of chroman the first preset condition of foot；Energy measuring is carried out to the first candidate image block, obtains energy
Meet the second candidate image block of the second preset condition；The direction of motion of moving target in the second candidate image block is carried out
Detection obtains the third candidate image block that the direction of motion meets third preset condition；Obtain the spy of the third candidate image block
Sign vector, and described eigenvector is inputted in preset model, if output result meets the 4th preset condition, it is determined that current
The original picture block of frame is smog image block.The embodiment of the present invention is realized using computer vision technique and artificial intelligence technology
Real-time detection to monitoring area, and the position for finding smog appearance avoids so as to accurately a little be positioned to catching fire
The generation and sprawling of fire have stronger practicability and ease for use.
Description of the drawings
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described, it should be apparent that, the accompanying drawings in the following description be only the present invention some
Embodiment for those of ordinary skill in the art without having to pay creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is the implementation process schematic diagram for the smog detection method based on video that the embodiment of the present invention one provides；
Fig. 2 is the schematic diagram of the smog image block direction of motion in the embodiment of the present invention；
Fig. 3 is the schematic diagram of LBP values provided in an embodiment of the present invention；
Fig. 4 is the structure diagram of the mist detecting device provided by Embodiment 2 of the present invention based on video；
Fig. 5 is the schematic diagram for the terminal device that the embodiment of the present invention three provides.
Specific implementation mode
In being described below, for illustration and not for limitation, it is proposed that such as tool of particular system structure, technology etc
Body details, to understand thoroughly the embodiment of the present invention.However, it will be clear to one skilled in the art that there is no these specific
The present invention can also be realized in the other embodiments of details.In other situations, it omits to well-known system, device, electricity
The detailed description of road and method, in case unnecessary details interferes description of the invention.
In order to illustrate technical solutions according to the invention, illustrated below by specific embodiment.
Embodiment one
Fig. 1 shows the implementation process schematic diagram for the smog detection method based on video that the embodiment of the present invention one provides.
As shown in Figure 1, being somebody's turn to do the smog detection method based on video specifically may include following steps.
Step S101：Motion detection is carried out to video flowing, obtains motion target area image, and to every frame moving target area
Area image carries out piecemeal and obtains original picture block.
Wherein, motion detection is carried out to the video flowing of monitor video using mixed Gauss model, detects foreground.Based on statistics
Method establishes background model, and foreground is detached with background to realize the detection to moving target in scene, and is constantly updated
Background model.Motion target area image is obtained, and piecemeal is carried out to every frame motion target area image and obtains original picture block.
Step S102：Color detection is carried out to the original picture block of present frame, color is obtained and meets the first preset condition
First candidate image block.
First preset condition includes RGB color condition and YUV color space conditions, the original to present frame
Beginning image block carries out color detection, obtains the first candidate image block that color meets the first preset condition, including：
Sub-step S201：The pixel number for meeting predetermined relationship in the original picture block of present frame is obtained, if pixel
The ratio of number and total number of pixels in original picture block is greater than or equal to preset ratio, it is determined that original image cigarette in the block
Mist meets the RGB color condition, and the preset ratio is the constant for being less than 1 more than zero；The predetermined relationship is：max
(r, g, b)-min (r, g, b)≤τ, and I=(r+g+b)/3；Wherein, r, g, b are respectively value of the pixel on R, G and channel B, τ
For color threshold, max (r, g, b) indicates that r, the maximum value in g, b, min (r, g, b) indicate r, and the minimum value in g, b, I is picture
Plain brightness；
For example, τ (15≤τ≤20) is color threshold, I (80≤I≤220) is pixel intensity.It counts in each image block
Meet the pixel number of RGB color condition, if pixel number is no less than 20% of total number of pixels in whole image block,
Determination meets first condition.
Sub-step S202：RGB color is converted into YUV color spaces, and calculates original picture block on the channel y, u and v
Mean valueWithCalculation formula is as follows：
Wherein, N is the number of pixel in original picture block, (xi,yi) it is i-th of image point coordinates, y (xi,yi)、u(xi,yi) and v
(xi,yi) it is respectively value of i-th of picture point on the channel Y, U and V；
Wherein, conversion formula can be：
Sub-step S203：It is average according to the original image of present frame background image block average value in the block and current image block
The situation of change of value judges whether to meet YUV color space conditions：
Wherein,Respectively average value of the current image block on the channel y, u and v,Respectively with institute
State average value of the corresponding background image block of current image block on the channel y, u and v, τ and τyFor fixed constant；
Wherein, under general scenario, τ=10, τy=55
Sub-step S204：The original graph of the RGB color condition and the YUV color spaces condition will be met simultaneously
As block is as the first candidate image block.
Step S103：Energy measuring is carried out to the first candidate image block, energy is obtained and meets the second preset condition
Second candidate image block.
Optionally, step S103 specifically may include：
Sub-step S301：First candidate image block is converted into gray level image block.
Sub-step S302：Two-dimensional discrete wavelet conversion is carried out to gray level image block, calculates background image in gray level image block
The first frequency energy of block and the first frequency energy of second frequency energy and foreground image block and second frequency energy.
Wherein, the gray level image block after wavelet transformation, is made of four parts, upper left corner LL, upper right corner HL, lower left corner LH and
Lower right corner HH.
Wherein, first frequency energy is high-frequency energy, and second frequency energy is low frequency energy；
Then, the expression formula of image block medium-high frequency energy be for：
The expression formula of low frequency energy is in image block：
Sub-step S303：Calculate the ratio of the first frequency energy of background image block and the first frequency energy of foreground image block
The ratio of the second frequency energy of the second frequency energy and foreground image block of value and background image block, meets following condition：
eHB/eHF>=1.05 and eLB/eLF≤ 0.9, then it will meet first frequency energy in the first smog image block and reduce and the
The increased image block of two frequency energies is determined as the second candidate image block；Wherein, eHBAnd eHFRespectively background image block and foreground
The first frequency energy of image block, eLBAnd eLFThe respectively second frequency energy of background image block and foreground image block.
Step S104：The direction of motion in second image block is belonged to default by the direction of motion for judging the second smog image block
The subimage block in direction is determined as third smog image block.
Optionally, step S104 specifically comprises the following steps：
Sub-step S401：The former frame of second candidate image block is subjected to greyscale transformation and obtains gray level image, and if being divided into
The dry identical image block of size.
Sub-step S402：N number of direction is chosen, former frame gray level image block and present frame gray on N number of direction are calculated separately
The Pearson correlation coefficient of image block chooses the maximum direction of Pearson correlation coefficient as current from N number of direction
The movement principal direction of image block, Pearson correlation coefficient computational methods are as follows：
Wherein, gnAnd gn-1Present frame and former frame gray value, (x are indicated respectivelyi,yi) indicating pixel coordinate, N is image
Pixel number in block, dx and dy indicate offset respectively.
For example, choosing N=17, then (with 5x5 image around current frame image block corresponding position on 17 directions
Block) Pearson correlation coefficient of previous frame gray level image block and present frame gray image block is calculated separately, maximum value is chosen as working as
The movement principal direction of preceding image block.Pearson correlation coefficient computational methods are as follows：
Wherein, gnAnd gn-1Present frame and former frame gray value, (x are indicated respectivelyi,yi) indicating pixel coordinate, N is image
Pixel number in block, dx (dx ∈ [- 2 2]) and dy (dy ∈ [- 2 2]) indicate offset (value respectively：- 2, -1,0,1,
2)。
Sub-step S403：If the movement principal direction is consistent with the preset direction of motion, it is determined that the described second candidate figure
As block is third candidate image block.
The direction of motion of the second candidate image block is calculated, if (being upward) consistent with preset direction, when the second smog
The direction of motion of image block is upper left, upwards or upper right (direction 2 to the direction 8 in such as Fig. 2), then this image block is candidate cigarette the
Three smog image block mist image blocks.
Step S105：The feature vector of the third candidate image block is obtained, and described eigenvector is inputted preset
In model, if output result meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
Optionally, step S105 may include：
Sub-step S501：Intermediate image respective value is multiplied with last piece image respective value, obtains corresponding LBP values,
Wherein, LBP calculation formula are：
Wherein, P is current pixel neighborhood of a point point number, and R is the radius of neighbourhood.
To each pixel in image, each pixel in 3x3 neighborhoods is compared with the gray value of current pixel, it will
Intermediate image respective value can acquire corresponding LBP values with the multiplication of last piece image respective value, as shown in figure 3, LBP=1+4+
8+16+32+128=189.
Sub-step S502：The number M of data set of the transition times less than or equal to 2 in the binary form of statistics 0 to 255,
Remaining is as another kind of；Wherein, the calculation formula of transition times is：
Sub-step S503：The occurrence number of M+1 class data is counted, the probabilistic combination that every class is occurred is as M+1 dimensional features
Vector.
For step S502 and S503, saltus step in the binary form of statistics 0 to 255 (0 to 1,1 to 0 between adjacent number)
Number is less than or equal to 2 data set, shares 58, remaining is as another kind of.The occurrence number of 59 class data is counted, it will be per class
The probabilistic combination of appearance is as feature vector (59 dimension).Wherein transition times are calculated as：
Sub-step S504：Described eigenvector is input to Adaboost smog disaggregated models, if output is just, it is determined that
The original picture block of present frame is smog image block.
Judge for example, feature vector is input to Adaboost smog disaggregated models, if differentiating, result is 1, table
Showing current image block, there are smog, are otherwise non-smog image block.
Optionally, after determining that original picture block is smog image block, further include：
If it is determined that continuous Z frame original picture blocks are smog image block, and adjacent original image in the Z frames original picture block
The lap of block is less than first threshold and/or the difference on boundary is more than second threshold, it is determined that is non-smoke module, Z is more than 1
Integer.
If it is determined that continuous Z frame original picture blocks are smog image block, and adjacent original graph in the Z frames original picture block
As the difference that the lap of block is less than first threshold and/or boundary is more than second threshold, it is determined that the cigarette of the original picture block
Mist testing result is unstable smoke region.For example, adjacent sets stablize too fast (such as weight of smoke region variation in 10 frame testing results
Folded rate is very few and/or the difference on boundary is more than threshold value), then this smoke region is rapid moving object (such as class flame color clothes
Pedestrian or vehicle etc.), it is thus determined that being non-smoke module.
The embodiment of the present invention obtains motion target area image, and transport to every frame by carrying out motion detection to video flowing
Moving-target area image carries out piecemeal and obtains original picture block；Color detection is carried out to the original picture block of present frame, obtains face
First candidate image block of chroman the first preset condition of foot；Energy measuring is carried out to the first candidate image block, obtains energy
Meet the second candidate image block of the second preset condition；The direction of motion of moving target in the second candidate image block is carried out
Detection obtains the third candidate image block that the direction of motion meets third preset condition；Obtain the spy of the third candidate image block
Sign vector, and described eigenvector is inputted in preset model, if output result meets the 4th preset condition, it is determined that current
The original picture block of frame is smog image block.The embodiment of the present invention is using computer vision technique and artificial intelligence technology realization pair
The real-time detection of monitoring area, and the position for finding smog appearance avoids fire so as to accurately a little be positioned to catching fire
The generation and sprawling of calamity have stronger practicability and ease for use.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process
Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention constitutes any limit
It is fixed.
Embodiment two
Referring to FIG. 4, it illustrates the structural frames of the mist detecting device provided by Embodiment 2 of the present invention based on video
Figure.The mist detecting device 40 based on video includes：Motion detection block 41, color detection module 42, energy measuring mould
Block 43, direction of motion detection module 44 and determining module 45.Wherein, the concrete function of each module is as follows：
Motion detection block 41 obtains motion target area image, and to every frame for carrying out motion detection to video flowing
Motion target area image carries out piecemeal and obtains original picture block；
Color detection module 42 carries out color detection for the original picture block to present frame, obtains color and meet first
First candidate image block of preset condition；
Energy detection module 43 carries out energy measuring to the first candidate image block, and it is default to obtain energy satisfaction second
Second candidate image block of condition；
Direction of motion detection module 44 is examined for the direction of motion to moving target in the second candidate image block
It surveys, obtains the third candidate image block that the direction of motion meets third preset condition；
Determining module 45, the feature vector for obtaining the third candidate image block, and described eigenvector is inputted
In preset model, if output result meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
Optionally, color detection module 42 includes：
Acquiring unit obtains the pixel number for meeting predetermined relationship in the original picture block of present frame, if pixel
Number and the ratio of total number of pixels in original picture block are greater than or equal to preset ratio, it is determined that original image smog in the block
Meet the RGB color condition, the preset ratio is the constant for being less than 1 more than zero；The predetermined relationship is：max
(r, g, b)-min (r, g, b)≤τ, and I=(r+g+b)/3；Wherein, r, g, b are respectively value of the pixel on R, G and channel B, τ
For color threshold, max (r, g, b) indicates that r, the maximum value in g, b, min (r, g, b) indicate r, and the minimum value in g, b, I is picture
Plain brightness；
Computing unit is converted, for RGB color to be converted into YUV color spaces, and it is logical in y, u and v to calculate original picture block
Mean value on roadWithCalculation formula is as follows：
Wherein, N is the number of pixel in original picture block, (xi,yi) it is i-th of image point coordinates, y (xi,yi)、u(xi,yi) and v
(xi,yi) it is respectively value of i-th of picture point on the channel Y, U and V；
First judging unit, for the original image background image block average value in the block and present image according to present frame
The situation of change of block average value judges whether to meet YUV color space conditions：
Wherein,Respectively average value of the current image block on the channel y, u and v,Respectively with institute
State average value of the corresponding background image block of current image block on the channel y, u and v, τ and τyFor fixed constant；
First determination unit, for the RGB color condition and the YUV color spaces condition will to be met simultaneously
Original picture block is as the first candidate image block.
Optionally, energy detection module 43 includes：
Converting unit, for the first candidate image block to be converted into gray level image block；
Wavelet transform unit is calculated and is carried on the back in gray level image block for carrying out two-dimensional discrete wavelet conversion to gray level image block
The first frequency energy of scape image block and the first frequency energy of second frequency energy and foreground image block and second frequency energy
Amount；
Calculate determination unit, the first frequency energy of first frequency energy and foreground image block for calculating background image block
The ratio of the ratio of amount and the second frequency energy of the second frequency energy of background image block and foreground image block, under satisfaction
Row condition：
eHB/eHF>=1.05 and eLB/eLF≤ 0.9, then it will meet first frequency energy in the first smog image block and reduce and the
The increased image block of two frequency energies is determined as the second candidate image block；Wherein, eHBAnd eHFRespectively background image block and foreground
The first frequency energy of image block, eLBAnd eLFThe respectively second frequency energy of background image block and foreground image block.
Optionally, direction of motion detection module 44 includes：
Greyscale transformation unit obtains gray level image for the former frame of the second candidate image block to be carried out greyscale transformation, and
It is divided into the identical image block of several sizes；
Related coefficient computing unit, for choosing N number of direction, calculate separately on N number of direction former frame gray level image block and
The Pearson correlation coefficient of present frame gray image block chooses the maximum side of Pearson correlation coefficient from N number of direction
To the movement principal direction as current image block, Pearson correlation coefficient computing device is as follows：
Wherein, gnAnd gn-1Present frame and former frame gray value, (x are indicated respectivelyi,yi) indicating pixel coordinate, N is image
Pixel number in block, dx and dy indicate offset respectively；
Second determination unit, for when the movement principal direction is consistent with the preset direction of motion, it is determined that described the
Two candidate image blocks are third candidate image block.
Optionally it is determined that module 45 includes：
Computing unit obtains corresponding LBP for intermediate image respective value to be multiplied with last piece image respective value
Value, wherein LBP calculation formula are：
Wherein, P is current pixel neighborhood of a point point number, and R is the radius of neighbourhood；
First statistic unit, the data set for being less than or equal to 2 for count transition times in 0 to 255 binary form
Number M, remaining is as another kind of；Wherein, the calculation formula of transition times is：
Second statistic unit counts the occurrence number of M+1 class data, and the probabilistic combination that every class is occurred is as M+1 Wei Te
Sign vector；
Second judgment unit, for the M+1 dimensional feature vectors to be inputted Adaboost smog disaggregated models, if output knot
Fruit is just, it is determined that the original picture block of present frame is smog image block.
Optionally, the mist detecting device based on video further includes：
Alarm module is used for if it is determined that continuous Z frame original picture blocks are smog image block, and the Z frames original picture block
In the lap of adjacent original picture block be less than the difference on first threshold and/or boundary and be more than second threshold, it is determined that be non-cigarette
Mist module, Z are the integer more than 1.
The embodiment of the present invention obtains motion target area image, and transport to every frame by carrying out motion detection to video flowing
Moving-target area image carries out piecemeal and obtains original picture block；Color detection is carried out to the original picture block of present frame, obtains face
First candidate image block of chroman the first preset condition of foot；Energy measuring is carried out to the first candidate image block, obtains energy
Meet the second candidate image block of the second preset condition；The direction of motion of moving target in the second candidate image block is carried out
Detection obtains the third candidate image block that the direction of motion meets third preset condition；Obtain the spy of the third candidate image block
Sign vector, and described eigenvector is inputted in preset model, if output result meets the 4th preset condition, it is determined that current
The original picture block of frame is smog image block.The embodiment of the present invention is using computer vision technique and artificial intelligence technology realization pair
The real-time detection of monitoring area, and the position for finding smog appearance avoids fire so as to accurately a little be positioned to catching fire
The generation and sprawling of calamity have stronger practicability and ease for use.
Embodiment three
Fig. 5 is the schematic diagram for the terminal device that three embodiments of the invention provide.As shown in figure 5, the terminal of the embodiment is set
Standby 5 include：Processor 50, memory 51 and it is stored in the meter that can be run in the memory 51 and on the processor 50
Calculation machine program 52, such as the smog detection method program based on video.When the processor 50 executes the computer program 52
Realize the step in above-mentioned each smog detection method embodiment based on video, such as step S101 to S105 shown in FIG. 1.
Alternatively, the processor 50 realizes the function of each unit in above-mentioned each device embodiment, example when executing the computer program 52
The function of the module 41 to 45 as shown in Fig. 4.
Illustratively, the computer program 52 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 51, and are executed by the processor 50, to complete the present invention.Described one
A or multiple module/units can be the series of computation machine program instruction section that can complete specific function, which is used for
Implementation procedure of the computer program 52 in the terminal device 5 is described.For example, the computer program 52 can be divided
It is cut into motion detection block, color detection module, energy detection module, direction of motion detection module and determining module, each module
Concrete function it is as follows：
Motion detection block obtains motion target area image, and transport to every frame for carrying out motion detection to video flowing
Moving-target area image carries out piecemeal and obtains original picture block；
Color detection module carries out color detection for the original picture block to present frame, obtains color and meets first in advance
If the first candidate image block of condition；
Energy detection module carries out energy measuring to the first candidate image block, obtains energy and meets the second default item
Second candidate image block of part；
Direction of motion detection module is examined for the direction of motion to moving target in the second candidate image block
It surveys, obtains the third candidate image block that the direction of motion meets third preset condition；
Determining module, the feature vector for obtaining the third candidate image block, and described eigenvector is inputted in advance
If model in, if output result meet the 4th preset condition, it is determined that the original picture block of present frame be smog image block.
The terminal device 5 can be desktop PC, notebook, palm PC, cloud server or other intelligence
The computing devices such as energy terminal.The terminal device may include, but be not limited only to, processor 50, memory 51.People in the art
Member is appreciated that Fig. 5 is only the example of terminal device, does not constitute the restriction to terminal device, may include than illustrating more
More or less component either combines certain components or different components, such as the terminal device can also include input
Output equipment, network access equipment, bus etc..
Alleged processor 50 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), application-specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor can also be any conventional processor
Deng.
The memory 51 can be the internal storage unit of the terminal device 5, for example, the hard disk of terminal device 5 or
Memory.The memory 51 can also be to be equipped on the External memory equipment of the terminal device 5, such as the terminal device 5
Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card,
Flash card (Flash Card) etc..Further, the memory 51 can also both include the storage inside of the terminal device 5
Unit also includes External memory equipment.The memory 51 is for storing needed for the computer program and the terminal device
Other programs and data.The memory 51 can be also used for temporarily storing the data that has exported or will export.
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each work(
Can unit, module division progress for example, in practical application, can be as needed and by above-mentioned function distribution by different
Functional unit, module are completed, i.e., the internal structure of described device are divided into different functional units or module, more than completion
The all or part of function of description.Each functional unit, module in embodiment can be integrated in a processing unit, also may be used
It, can also be above-mentioned integrated during two or more units are integrated in one unit to be that each unit physically exists alone
The form that hardware had both may be used in unit is realized, can also be realized in the form of SFU software functional unit.In addition, each function list
Member, the specific name of module are also only to facilitate mutually distinguish, the protection domain being not intended to limit this application.Above system
The specific work process of middle unit, module, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, is not described in detail or remembers in some embodiment
The part of load may refer to the associated description of other embodiments.
Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, depends on the specific application and design constraint of technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device/terminal device and method, it can be with
It realizes by another way.For example, device described above/terminal device embodiment is only schematical, for example, institute
The division of module or unit is stated, only a kind of division of logic function, formula that in actual implementation, there may be another division manner, such as
Multiple units or component can be combined or can be integrated into another system, or some features can be ignored or not executed.Separately
A bit, shown or discussed mutual coupling or direct-coupling or communication connection can be by some interfaces, device
Or INDIRECT COUPLING or the communication connection of unit, can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple
In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it can also
It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.Above-mentioned integrated list
The form that hardware had both may be used in member is realized, can also be realized in the form of SFU software functional unit.
If the integrated module/unit be realized in the form of SFU software functional unit and as independent product sale or
In use, can be stored in a computer read/write memory medium.Based on this understanding, the present invention realizes above-mentioned implementation
All or part of flow in example method, can also instruct relevant hardware to complete, the meter by computer program
Calculation machine program can be stored in a computer readable storage medium, the computer program when being executed by processor, it can be achieved that on
The step of stating each embodiment of the method.Wherein, the computer program includes computer program code, the computer program generation
Code can be source code form, object identification code form, executable file or certain intermediate forms etc..The computer-readable medium
May include：Any entity or device, recording medium, USB flash disk, mobile hard disk, magnetic of the computer program code can be carried
Dish, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory (RAM,
Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that described
The content that computer-readable medium includes can carry out increasing appropriate according to legislation in jurisdiction and the requirement of patent practice
Subtract, such as in certain jurisdictions, according to legislation and patent practice, computer-readable medium does not include electric carrier signal and electricity
Believe signal.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations；Although with reference to aforementioned reality
Applying example, invention is explained in detail, it will be understood by those of ordinary skill in the art that：It still can be to aforementioned each
Technical solution recorded in embodiment is modified or equivalent replacement of some of the technical features；And these are changed
Or replace, the spirit and scope for various embodiments of the present invention technical solution that it does not separate the essence of the corresponding technical solution should all
It is included within protection scope of the present invention.
Claims (10)
1. a kind of smog detection method based on video, which is characterized in that including：
Motion detection is carried out to video flowing, obtains motion target area image, and divide every frame motion target area image
Block obtains original picture block；
Color detection is carried out to the original picture block of present frame, obtains the first candidate image that color meets the first preset condition
Block；
Energy measuring is carried out to the first candidate image block, obtains the second candidate image that energy meets the second preset condition
Block；
The direction of motion of moving target in the second candidate image block is detected, it is default that the acquisition direction of motion meets third
The third candidate image block of condition；
The feature vector of the third candidate image block is obtained, and described eigenvector is inputted in preset model, if output
As a result meet the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
2. the smog detection method based on video as described in claim 1, which is characterized in that first preset condition includes
RGB color condition and YUV color space conditions, the original picture block to present frame carry out color detection, obtain face
First candidate image block of chroman the first preset condition of foot, including：
The pixel number for meeting predetermined relationship in the original picture block of present frame is obtained, if pixel number and original picture block
In the ratio of total number of pixels be greater than or equal to preset ratio, it is determined that original image smog in the block meets the RGB face
Colour space condition, the preset ratio are the constant for being less than 1 more than zero；The predetermined relationship is：max(r,g,b)-min(r,g,
B)≤τ, and I=(r+g+b)/3；Wherein, r, g, b are respectively value of the pixel on R, G and channel B, and τ is color threshold, max
(r, g, b) indicates that r, the maximum value in g, b, min (r, g, b) indicate r, and the minimum value in g, b, I is pixel intensity；
RGB color is converted into YUV color spaces, and calculates mean value of the original picture block on the channel y, u and v
WithCalculation formula is as follows： Its
In, N is the number of pixel in original picture block, (xi,yi) it is i-th of image point coordinates, y (xi,yi)、u(xi,yi) and v
(xi,yi) it is respectively value of i-th of picture point on the channel Y, U and V；
According to the situation of change of the original image of present frame background image block average value and current image block average value in the block, sentence
It is disconnected whether to meet YUV color space conditions：
Wherein,Respectively average value of the current image block on the channel y, u and v,Respectively work as with described
Average value of the corresponding background image block of preceding image block on the channel y, u and v, τ and τyFor fixed constant；
The original picture block for meeting the RGB color condition and the YUV color spaces condition simultaneously is waited as first
Select image block.
3. the smog detection method based on video as described in claim 1, which is characterized in that the first candidate image block
Energy measuring is carried out, the second candidate image block that energy meets the second preset condition is obtained, including：
First candidate image block is converted into gray level image block；
Two-dimensional discrete wavelet conversion is carried out to gray level image block, calculates the first frequency energy of background image block in gray level image block
With the first frequency energy of second frequency energy and foreground image block and second frequency energy；
Calculate the ratio and background image of the first frequency energy of background image block and the first frequency energy of foreground image block
The ratio of the second frequency energy of block and the second frequency energy of foreground image block, meets following condition：
eHB/eHF>=1.05 and eLB/eLF≤ 0.9, then it will meet the reduction of first frequency energy and the second frequency in the first smog image block
The increased image block of rate energy is determined as the second candidate image block；Wherein, eHBAnd eHFRespectively background image block and foreground image
The first frequency energy of block, eLBAnd eLFThe respectively second frequency energy of background image block and foreground image block.
4. the smog detection method based on video as described in claim 1, which is characterized in that the second candidate image block
The direction of motion of middle moving target is detected, and obtains the third candidate image block packet that the direction of motion meets third preset condition
It includes：
The former frame of second candidate image block is subjected to greyscale transformation and obtains gray level image, and is divided into the identical figure of several sizes
As block；
N number of direction is chosen, the Pearson of former frame gray level image block and present frame gray image block on N number of direction is calculated separately
Related coefficient chooses movement main side of the maximum direction of Pearson correlation coefficient as current image block from N number of direction
To Pearson correlation coefficient computational methods are as follows：
Wherein, gnAnd gn-1Present frame and former frame gray value, (x are indicated respectivelyi,yi) indicate that pixel coordinate, N are picture in image block
Vegetarian refreshments number, dx and dy indicate offset respectively；
If the movement principal direction is consistent with the preset direction of motion, it is determined that the second candidate image block is schemed for third candidate
As block.
5. the smog detection method based on video as described in claim 1, which is characterized in that obtain the third candidate image
The feature vector of block, and described eigenvector is inputted in preset model, if output result meets the 4th preset condition, really
The original picture block of settled previous frame is that smog image block includes：
Intermediate image respective value is multiplied with last piece image respective value, obtains corresponding LBP values, wherein LBP calculation formula
For：
Wherein, P is current pixel neighborhood of a point point number, and R is the radius of neighbourhood；
The number M of data set of the transition times less than or equal to 2 in the binary form of statistics 0 to 255, remaining is as another
Class；Wherein, the calculation formula of transition times is：
The occurrence number of M+1 class data is counted, the probabilistic combination that every class is occurred is as M+1 dimensional feature vectors；
The M+1 dimensional feature vectors are inputted into Adaboost smog disaggregated models, if output result is just, then to determine present frame
Original picture block be smog image block.
6. such as the smog detection method described in any one of claim 1 to 5 based on video, which is characterized in that current determining
The original picture block of frame be smog image block after, further include：
If it is determined that continuous Z frame original picture blocks are smog image block, and adjacent original picture block in the Z frames original picture block
Lap is less than first threshold and/or the difference on boundary is more than second threshold, it is determined that is non-smoke module, Z is whole more than 1
Number.
7. a kind of mist detecting device based on video, which is characterized in that including：
Motion detection block obtains motion target area image, and move mesh to every frame for carrying out motion detection to video flowing
Mark area image carries out piecemeal and obtains original picture block；
Color detection module carries out color detection for the original picture block to present frame, obtains color and meet the first default item
First candidate image block of part；
Energy detection module carries out energy measuring to the first candidate image block, obtains energy and meets the second preset condition
Second candidate image block；
Direction of motion detection module is detected for the direction of motion to moving target in the second candidate image block, obtains
Obtain the third candidate image block that the direction of motion meets third preset condition；
Determining module obtains the feature vector of the third candidate image block, and described eigenvector is inputted preset model
In, if output result meets the 4th preset condition, it is determined that the original picture block of present frame is smog image block.
8. the mist detecting device based on video as claimed in claim 7, which is characterized in that first preset condition includes
RGB color condition and YUV color space conditions, the color detection module include：
Acquiring unit obtains the pixel number for meeting predetermined relationship in the original picture block of present frame, if pixel number with
The ratio of total number of pixels is greater than or equal to preset ratio in original picture block, it is determined that original image smog in the block meets
The RGB color condition, the preset ratio are the constant for being less than 1 more than zero；The predetermined relationship is：max(r,g,
B)-min (r, g, b)≤τ, and I=(r+g+b)/3；Wherein, r, g, b are respectively value of the pixel on R, G and channel B, and τ is face
Chromatic threshold value, max (r, g, b) indicate that r, the maximum value in g, b, min (r, g, b) indicate that r, the minimum value in g, b, I are that pixel is bright
Degree；
Computing unit is converted, for RGB color to be converted into YUV color spaces, and calculates original picture block on the channel y, u and v
Mean valueWithCalculation formula is as follows：
Wherein, N is the number of pixel in original picture block, (xi,yi) it is i-th of image point coordinates, y (xi,yi)、u(xi,yi) and v
(xi,yi) it is respectively value of i-th of picture point on the channel Y, U and V；
First judging unit, for flat according to the original image background image block average value in the block and current image block of present frame
The situation of change of mean value judges whether to meet YUV color space conditions：
Wherein,Respectively average value of the current image block on the channel y, u and v,Respectively work as with described
Average value of the corresponding background image block of preceding image block on the channel y, u and v, τ and τyFor fixed constant；
First determination unit, for the original of the RGB color condition and the YUV color spaces condition will to be met simultaneously
Image block is as the first candidate image block.
9. a kind of device end, including memory, processor and it is stored in the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 6 when executing the computer program
The step of any one the method.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, feature to exist
In when the computer program is executed by processor the step of any one of such as claim 1 to 6 of realization the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810380051.XA CN108765454A (en) | 2018-04-25 | 2018-04-25 | A kind of smog detection method, device and device end based on video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810380051.XA CN108765454A (en) | 2018-04-25 | 2018-04-25 | A kind of smog detection method, device and device end based on video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108765454A true CN108765454A (en) | 2018-11-06 |
Family
ID=64011759
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810380051.XA Pending CN108765454A (en) | 2018-04-25 | 2018-04-25 | A kind of smog detection method, device and device end based on video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108765454A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110135374A (en) * | 2019-05-21 | 2019-08-16 | 吉林大学 | It is identified using image block characteristics and returns the fire hazard smoke detecting method classified |
| CN110263654A (en) * | 2019-05-23 | 2019-09-20 | 深圳市中电数通智慧安全科技股份有限公司 | A kind of flame detecting method, device and embedded device |
| CN112419413A (en) * | 2020-12-07 | 2021-02-26 | 萱闱(北京)生物科技有限公司 | Movement direction monitoring method, medium and device of terminal equipment and computing equipment |
| WO2022165735A1 (en) * | 2021-02-02 | 2022-08-11 | 豪威芯仑传感器(上海)有限公司 | Method and system for detecting moving object |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1894594A (en) * | 2003-12-16 | 2007-01-10 | 皇家飞利浦电子股份有限公司 | Ultrasonic diagnostic imaging system with automatic control of penetration, resolution and frame rate |
| CN103020628A (en) * | 2012-11-30 | 2013-04-03 | 北京理工大学 | Smoke detection method based on red, green and blue (RGB) contrast image and target shape |
| CN103870818A (en) * | 2014-03-31 | 2014-06-18 | 中安消技术有限公司 | Smog detection method and device |
| CN104050478A (en) * | 2014-07-09 | 2014-09-17 | 湖南大学 | Smog detection method and system |
| CN104794486A (en) * | 2015-04-10 | 2015-07-22 | 电子科技大学 | Video smoke detecting method based on multi-feature fusion |
| CN106228150A (en) * | 2016-08-05 | 2016-12-14 | 南京工程学院 | Smog detection method based on video image |
| CN106815567A (en) * | 2016-12-30 | 2017-06-09 | 北京邮电大学 | A kind of flame detecting method and device based on video |
-
2018
- 2018-04-25 CN CN201810380051.XA patent/CN108765454A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1894594A (en) * | 2003-12-16 | 2007-01-10 | 皇家飞利浦电子股份有限公司 | Ultrasonic diagnostic imaging system with automatic control of penetration, resolution and frame rate |
| CN103020628A (en) * | 2012-11-30 | 2013-04-03 | 北京理工大学 | Smoke detection method based on red, green and blue (RGB) contrast image and target shape |
| CN103870818A (en) * | 2014-03-31 | 2014-06-18 | 中安消技术有限公司 | Smog detection method and device |
| CN104050478A (en) * | 2014-07-09 | 2014-09-17 | 湖南大学 | Smog detection method and system |
| CN104794486A (en) * | 2015-04-10 | 2015-07-22 | 电子科技大学 | Video smoke detecting method based on multi-feature fusion |
| CN106228150A (en) * | 2016-08-05 | 2016-12-14 | 南京工程学院 | Smog detection method based on video image |
| CN106815567A (en) * | 2016-12-30 | 2017-06-09 | 北京邮电大学 | A kind of flame detecting method and device based on video |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110135374A (en) * | 2019-05-21 | 2019-08-16 | 吉林大学 | It is identified using image block characteristics and returns the fire hazard smoke detecting method classified |
| CN110263654A (en) * | 2019-05-23 | 2019-09-20 | 深圳市中电数通智慧安全科技股份有限公司 | A kind of flame detecting method, device and embedded device |
| CN112419413A (en) * | 2020-12-07 | 2021-02-26 | 萱闱(北京)生物科技有限公司 | Movement direction monitoring method, medium and device of terminal equipment and computing equipment |
| CN112419413B (en) * | 2020-12-07 | 2024-01-05 | 萱闱(北京)生物科技有限公司 | Method, medium, device and computing equipment for monitoring movement direction of terminal equipment |
| WO2022165735A1 (en) * | 2021-02-02 | 2022-08-11 | 豪威芯仑传感器(上海)有限公司 | Method and system for detecting moving object |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765454A (en) | A kind of smog detection method, device and device end based on video | |
| CN110544258B (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN110852162B (en) | Human body integrity data labeling method and device and terminal equipment | |
| CN109086724B (en) | Accelerated human face detection method and storage medium | |
| CN105590319A (en) | Method for detecting image saliency region for deep learning | |
| CN111401269B (en) | Commodity hot spot detection method, device and equipment based on monitoring video | |
| CN106951869A (en) | A kind of live body verification method and equipment | |
| CN112183501A (en) | Depth counterfeit image detection method and device | |
| CN108769634A (en) | A kind of image processing method, image processing apparatus and terminal device | |
| WO2022082999A1 (en) | Object recognition method and apparatus, and terminal device and storage medium | |
| EP3073443A1 (en) | 3D Saliency map | |
| CN110211110A (en) | A kind of detection method of scantling, system and terminal device | |
| CN111144337A (en) | Fire detection method and device and terminal equipment | |
| JP7429756B2 (en) | Image processing method, device, electronic device, storage medium and computer program | |
| CN109903265A (en) | A kind of image change area detecting threshold value setting method, system and its electronic device | |
| CN115131714A (en) | Intelligent detection and analysis method and system for video image | |
| JP2005032250A (en) | Method for processing face detection, and device for detecting faces in image | |
| JP3725784B2 (en) | Apparatus and method for detecting moving object in color frame image sequence | |
| CN116758529B (en) | Line identification method, device and computer equipment | |
| CN110135224B (en) | Method and system for extracting foreground target of surveillance video, storage medium and terminal | |
| CN111723614A (en) | Traffic signal lamp identification method and device | |
| CN108776972A (en) | A kind of method for tracing object and device | |
| CN108810407A (en) | A kind of image processing method, mobile terminal and computer readable storage medium | |
| CN108805013A (en) | A kind of smoke detection system based on video | |
| CN113780278A (en) | Method and device for identifying license plate content, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 1106, Hengmei Daxia, No. 3 Ganli Road, Jihua Street, Longgang District, Shenzhen City, Guangdong Province, 518000 Applicant after: Shenzhen CLP smart security Polytron Technologies Inc Address before: 518000 Workshop 401, No. 5, Juyin Science and Technology Industrial Factory Area, Buji Street, Longgang District, Shenzhen City, Guangdong Province Applicant before: Shenzhen CLP smart security Polytron Technologies Inc |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |