CN109190455B - Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model - Google Patents

Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model Download PDF

Info

Publication number
CN109190455B
CN109190455B CN201810789111.3A CN201810789111A CN109190455B CN 109190455 B CN109190455 B CN 109190455B CN 201810789111 A CN201810789111 A CN 201810789111A CN 109190455 B CN109190455 B CN 109190455B
Authority
CN
China
Prior art keywords
pixel
vehicle
value
gaussian
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810789111.3A
Other languages
Chinese (zh)
Other versions
CN109190455A (en
Inventor
路小波
陶焕杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201810789111.3A priority Critical patent/CN109190455B/en
Publication of CN109190455A publication Critical patent/CN109190455A/en
Application granted granted Critical
Publication of CN109190455B publication Critical patent/CN109190455B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G06V10/507—Summing image-intensity values; Histogram projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Abstract

The black smoke vehicle identification method based on the Gaussian mixture and autoregressive moving average model comprises the following steps: (1) detecting a vehicle moving target from a road monitoring video by using a Gaussian mixture model; (2) extracting three characteristics of a key area of the vehicle, including a Haar-like characteristic, a symbiotic matrix gradient direction histogram characteristic and a local binary pattern Fourier histogram characteristic; (3) modeling continuous multiframes of each characteristic by using an autoregressive moving average model to obtain three different models; (4) and for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from the key area of the vehicle, and whether the black smoke vehicle exists in the current video segment or not is judged by combining the classification results of different characteristics and the comprehensive analysis of continuous multiple frames. The invention can greatly save the manpower and financial resources consumed by the traditional method, is beneficial to the acquisition and the preservation of evidences, does not influence normal traffic and can effectively improve the law enforcement efficiency.

Description

Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model
Technical Field
The invention relates to the technical field of computer vision and pattern recognition, in particular to a black smoke vehicle recognition method based on a Gaussian mixture and autoregressive moving average model.
Background
In 16 days 6 months in 2018, the opinion on the comprehensive enhancement of ecological environment protection, the establishment of pollution prevention, control and attack and hardness fighting issued by the central national institute indicates that the pollution control and attack and hardness fighting of diesel trucks is required to be established, and … … is used for building a vehicle emission monitoring system integrating heaven and earth vehicles and perfecting a vehicle remote sensing monitoring network. The diesel vehicle with serious pollution is also called as a black smoke vehicle, dense black smoke is usually generated at an exhaust hole, the pollution is always the key point of motor vehicle pollution treatment, the black smoke vehicle running on a road is found in time and is further treated by an environmental protection department, the pollution of the motor vehicle is reduced, the air quality is improved, and the harm to a human body is reduced.
At present, a lot of cities still adopt ways of road patrol, mass report and manual video monitoring to detect black smoke vehicles on roads, waste time and labor, influence normal traffic operation and are not beneficial to storing of illegal evidences of related vehicles. At present, with the continuous development of a computer vision technology and a pattern recognition technology, how to automatically detect the black smoke car by adopting a video analysis mode becomes a hot spot more and more, and the method is very worthy of research.
The invention provides an intelligent black smoke vehicle detection method based on video analysis, which combines a Gaussian mixture model and an autoregressive moving average model, and simultaneously designs a Haar-like feature, a symbiotic matrix gradient direction histogram feature (CoHOG) and a local binary pattern Fourier histogram feature (LBP-HF), wherein the integration of the three features increases the identification accuracy rate of black smoke vehicles and reduces the false alarm rate.
Disclosure of Invention
In order to solve the problems, the invention provides a black smoke vehicle recognition method based on a Gaussian mixture and autoregressive moving average model, which can synthesize different types of static characteristics, introduce the autoregressive moving average model to depict the dynamic characteristics of a key area, has high robustness on camera shake and vehicle shadow, can reduce noise interference, further improve the recognition rate of the black smoke vehicle and reduce the false alarm rate. To achieve the purpose, the invention provides a black smoke vehicle identification method based on a Gaussian mixture and autoregressive moving average model, which comprises the following specific steps:
(1) detecting a vehicle moving target from a road monitoring video by using a Gaussian mixture model;
(2) extracting three characteristics of a key area of the vehicle, including a Haar-like characteristic, a symbiotic matrix gradient direction histogram characteristic and a local binary pattern Fourier histogram characteristic;
(3) modeling continuous multiframes of each characteristic by using an autoregressive moving average model to obtain three different models;
(4) and for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from the key area of the vehicle, and whether the black smoke vehicle exists in the current video segment or not is judged by combining the classification results of different characteristics and the comprehensive analysis of continuous multiple frames.
Further, the step (1) of detecting the vehicle moving object from the road monitoring video by using the Gaussian mixture model comprises the following steps:
(11) converting each frame image into a gray image, equally dividing the gray image into small blocks of 3-by-3 pixels, and establishing a Gaussian model for each pixel block so as to improve the background modeling speed and stability and reduce the noise interference;
(12) model initializationTaking a pixel block M of a frame image as an example, the initial model of the pixel block is established by using the pixel block sequence of the former N frame sequence image, and is recorded as { x }1,x2,...,xNGet the average value mu of the gray levels of the pixel blocks0Sum variance
Figure GDA0003100408800000021
Initializing the mean and variance of the 1 st Gaussian distribution, i.e.
Figure GDA0003100408800000022
Figure GDA0003100408800000023
Figure GDA0003100408800000024
Wherein I (x, y, k) represents a pixel value of the pixel block at a position (x, y) in the k-th frame image;
(13) taking the (N + 1) th frame as an example to illustrate the updating method of the background model,
if the gray value x of the pixel blockN+1Satisfy | xN+1-μi,N|≤2.5σi,NThen the pixel block matches the corresponding K gaussian distributions, the parameters are updated as follows,
Figure GDA0003100408800000025
Figure GDA0003100408800000026
wherein β and ρ are learning rates of a mean and a variance of the Gaussian distribution, respectively,
Figure GDA0003100408800000027
representing a Gaussian probability densityDegree function, theta and gamma are fixed values, and are used for adjusting the value range of the learning rate, xN+1Is the gray average value, mu, of the N +1 th frame pixel blocki,NAnd σi,N 2The mean and variance of the ith Gaussian model for the (N + 1) th frame;
if the gray value x of the pixel blockN+1Not matching with the corresponding K Gaussian distributions, the parameter mui,NAnd σi,N 2Remain unchanged, need to be xN+1Establishing a new Gaussian distribution for the mean value, the larger variance and the smaller weight to replace the smallest weight in the original K Gaussian distributions, and updating the weight omega of the K Gaussian distributionsi,N,
ωi,N+1＝(1-α)ωi,N+αMi,N+1,i＝1,2,...,K
Where α is the learning rate, for parameter Mi,N+1When x isN+1When it is not matched with the ith Gaussian distribution, Mi,N+11, otherwise, Mi,N+1＝0；
(14) Background estimation in accordance with
Figure GDA0003100408800000031
i is 1,2, the value of K is sorted from large to small, K Gaussian distributions of the pixel block M are sorted, the first B Gaussian distributions are taken as background models, the last K-B Gaussian distributions are taken as foreground models,
Figure GDA0003100408800000032
wherein T is a threshold value, determines the number of background distributions,
judging whether the pixel block M belongs to the moving target or not, and when the gray value x of MN+1When the M is matched with a certain one of the first B Gaussian distributions, the M is considered as a background pixel; otherwise M is the target pixel;
the area formed by all target pixels is the foreground target area (vehicle target), in order to reduce false alarm, the key area needs to be determined, the bottom edge of the bounding box of the foreground target is the bottom edge of the key area, the width of the area is equal to 0.8 times of the width of the vehicle target, and the height of the area is equal to the height of the vehicle targetFor 60 pixels, normalize the critical region to 80 x 120 pixels, denoted as Inorm。
Further, the calculation of the Haar-like features in the step (2) comprises the following steps:
(21) the Haar-like feature is a commonly used feature description operator in the field of computer vision, the feature value of the Haar-like feature is obtained by subtracting the sum of gray values of white pixels from the sum of gray values of black pixels in a small area, the calculation cost can be reduced by adopting a block feature-based mode instead of a pixel-based mode, and the feature selects a main edge feature, a line feature, a point feature (a central surrounding feature) and a diagonal feature;
(22) the Haar-like characteristic generally consists of 2-3 rectangular blocks, and in order to improve the calculation speed, the sum of all gray levels in the rectangle is quickly calculated by adopting an integral diagram method;
(23) in order to reduce the characteristic dimension, a PCA algorithm is used for characteristic selection and dimension reduction to obtain a Haar-like characteristic vector after dimension reduction, and the Haar-like characteristic vector is recorded as FHaar-like。
Further, the computing of the gradient direction histogram feature of the co-occurrence matrix in the step (2) comprises the following steps:
(24) respectively calculating normalized key regions InormThe magnitude mag (x, y) and direction ori (x, y) of the gradient at position (x, y), i.e.
Figure GDA0003100408800000033
Figure GDA0003100408800000034
Wherein, Inorm(x, y) denotes the normalized key region InormA pixel value at position (x, y);
(25) dividing the key area into m × n small blocks according to the aspect ratio, wherein the small blocks are non-overlapped areas;
(26) selecting an offset mode to scan each small block to generate a co-occurrence matrix, wherein the offset mode refers to the relative positions of the point pairs, and the offset mode includes 31 types, such as an up-down relationship, a left-right relationship and the like, one offset mode corresponds to one co-occurrence matrix,
the gradient directions of each pixel point are combined in pairs, the original gradient directions are divided into 8 gradient directions at the same time, the range is from 0 degree to 360 degrees, namely, each 45 degrees is one bin, and as each two pixel points form one combination, the size of a co-occurrence matrix is 8 × 64, so that a column vector of m × n × 64 is obtained in an offset mode;
(27) an offset relationship is transformed to scan over the entire image until all offset modes are scanned once. This will generate 31 x 64 x m n column vectors, i.e. the final co-occurrence matrix gradient direction histogram feature, denoted as FCoHOG。
Further, the calculation of the local binary pattern fourier histogram feature LBP-HF in step (2) comprises the following steps:
(28) a canonical local binary pattern is calculated using the following formula,
Figure GDA0003100408800000041
Figure GDA0003100408800000042
where P denotes the number of pixels in the neighborhood in the local region, R denotes the radius of the circular neighborhood, U (LBP)P,R) Normalized local binary pattern value g representing the central pixel of an image region with a field pixel number P and a radius RcCenter pixel value of region, gp(P ═ 1, 2.. times, P) denotes the pixel value of the P-th pixel point on the neighborhood;
calculating histogram characteristics of the normative local binary pattern;
(29) discrete Fourier transform of the histogram, i.e.
Figure GDA0003100408800000043
Wherein P is the number of neighboring pixel points, Up(n, r) is a certain normalization pattern, hI(Up(n, r)) is the presence of U in the image Ip(n, r) the number of normalized modes, H (n, u) representing the histogram after fourier transform;
(210) local binary pattern Fourier histogram features LBP-HF are obtained by the following formula,
Figure GDA0003100408800000044
wherein the content of the first and second substances,
Figure GDA0003100408800000045
denotes the complex conjugation of H (n, u), FLBP-HFRepresenting local binary pattern fourier histogram features.
Further, the modeling of the continuous multiframes of each feature by using the autoregressive moving average model in the step (3) comprises the following steps:
(31) the autoregressive moving average model is suitable for analyzing most time sequences in the objective world, has few unknown parameters and good approximation degree of characteristic optimization, considers that the current value of the sequence is a linear combination of the current and past errors and the previous sequence value, and the general form is,
Figure GDA0003100408800000046
ξt～WGN(0,σ2)
wherein p represents the autoregressive order, q represents the moving average order,
Figure GDA0003100408800000051
all represent autoregressive coefficients, θjAll represent a moving average coefficient, ξtRepresenting a random interference value, ξt～WGN(0,σ2) Is representative of xitIs a mean of 0 and a variance of σ2Normal white noise process of (Y)i(i ═ 1, 2.,. t.) (ii) representsCarving the sequence value of i;
(32) introducing probability description strategy, and transforming the model formula into
Figure GDA0003100408800000052
Figure GDA0003100408800000053
(33) Establishing a least square problem, estimating model parameters by adopting the most basic steepest descent method,
for the same critical region, the feature contains FHaar-like，FCoHOGAnd FLBP-HFAnd thirdly, establishing time sequence characteristics by adopting sequence analysis, thereby obtaining three different autoregressive moving average models.
Further, the identifying the video clip of the black smoke car in the step (4) comprises the following steps:
(41) for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from a vehicle key region, the recognition result of the model with the maximum probability is selected as the recognition result of the key region, and if one key region of the current frame is recognized as a black smoke region, the current frame is recognized as a black smoke frame;
(42) and (4) integrating the analysis of multiple frames, if more than delta frames in the continuous 100 frames are identified as black smoke frames, determining that the current video segment contains black smoke cars, and setting the parameter delta as a coefficient for adjusting the detection rate and the false alarm rate by the user.
The advantages of the invention are mainly reflected in that:
(1) the method integrates the advantages of three characteristics, namely Haar-like characteristics, symbiotic matrix gradient direction histogram characteristics (CoHOG) and local binary pattern Fourier histogram characteristics (LBP-HF), wherein the Haar-like characteristics can fully reflect the gray level change condition of the image, and particularly the extraction effect of horizontal edge characteristics is maximum; the CoHOG characteristic has robustness under illumination change and deformation, the characteristic dimension is high, the shape information of the image can be more finely expressed, and the calculation speed is high; the LBP-HF not only retains the strong advantages of the traditional LBP descriptor in the aspect of local texture description, but also increases the characteristic of rotation invariance, and has certain robustness to shadow. By integrating the advantages of each characteristic, the technology further improves the recognition rate, reduces the false alarm rate, and has high robustness to camera shake and vehicle shadow.
(2) The invention introduces an autoregressive moving average model for analyzing the feature sequence of continuous multiframes, fully utilizes the dynamic features of a key region, and describes the feature that black smoke in a black smoke region gradually diffuses and the feature that the tail part of a vehicle in a non-black smoke region gradually moves forwards and disappears, thereby reducing the false alarm rate of the black smoke vehicle.
(3) Compared with the traditional method, the technical scheme of the invention is the intelligent black smoke vehicle detection method, which can greatly save the manpower and financial resources consumed by the traditional method, is beneficial to acquiring and storing evidences, does not influence normal traffic, and can effectively improve the law enforcement efficiency.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
FIG. 2 is a graph of edge, line, point, and diagonal features of a Haar-like feature employed in the present invention.
Fig. 3 shows an example of a black smoke car detected by the present invention.
Detailed Description
The invention is described in further detail below with reference to the following detailed description and accompanying drawings:
the invention provides a black smoke vehicle recognition method based on a Gaussian mixture and autoregressive moving average model, which can synthesize different types of static characteristics, introduce the autoregressive moving average model to depict the dynamic characteristics of a key area, has high robustness on camera shake and vehicle shadow, can reduce noise interference, further improve the recognition rate of the black smoke vehicle and reduce the false alarm rate.
The invention provides a black smoke vehicle identification method based on a Gaussian mixture and autoregressive moving average model, which is shown in a flow chart of figure 1 and specifically comprises the following steps:
step 1: detecting a vehicle moving target from a road monitoring video by using a Gaussian mixture model;
step 2: extracting three characteristics of a key area of the vehicle, including a Haar-like characteristic, a symbiotic matrix gradient direction histogram characteristic (CoHOG) and a local binary pattern Fourier histogram characteristic (LBP-HF);
and step 3: modeling continuous multiframes of each characteristic by using an autoregressive moving average (ARMA) model to obtain three different models;
and 4, step 4: and for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from the key area of the vehicle, and whether the black smoke vehicle exists in the current video segment or not is judged by combining the classification results of different characteristics and the comprehensive analysis of continuous multiple frames.
The step 1 of detecting the vehicle moving target from the road monitoring video by using the Gaussian mixture model comprises the following steps:
step 1.1: converting each frame image into a gray image, equally dividing the gray image into small blocks of 3-by-3 pixels, and establishing a Gaussian model for each pixel block so as to improve the background modeling speed and stability and reduce the noise interference;
step 1.2: model initialization, taking a pixel block M of a frame image as an example, the initial model of the pixel block is established by using the pixel block sequence of the former N frame sequence image, and is marked as { x1,x2,...,xNGet the average value mu of the gray levels of the pixel blocks0Sum variance
Figure GDA0003100408800000061
Initializing the mean and variance of the 1 st Gaussian distribution, i.e.
Figure GDA0003100408800000062
Figure GDA0003100408800000063
Figure GDA0003100408800000064
Wherein I (x, y, k) represents a pixel value of the pixel block at a position (x, y) in the k-th frame image;
step 1.3: taking the (N + 1) th frame as an example to illustrate the updating method of the background model,
if the gray value x of the pixel blockN+1Satisfy | xN+1-μi,N|≤2.5σi,NThen the pixel block matches the corresponding K gaussian distributions, the parameters are updated as follows,
Figure GDA0003100408800000071
Figure GDA0003100408800000072
wherein β and ρ are learning rates of a mean and a variance of the Gaussian distribution, respectively,
Figure GDA0003100408800000073
representing a Gaussian probability density function, theta and gamma are fixed values and are used for adjusting the value range of the learning rate, xN+1Is the gray average value, mu, of the N +1 th frame pixel blocki,NAnd σi,N 2The mean and variance of the ith Gaussian model for the (N + 1) th frame;
if the gray value x of the pixel blockN+1Not matching with the corresponding K Gaussian distributions, the parameter mui,NAnd σi,N 2Remain unchanged, need to be xN+1Establishing a new Gaussian distribution for the mean value, the larger variance and the smaller weight to replace the smallest weight in the original K Gaussian distributions, and updating the weight omega of the K Gaussian distributionsi,N,
ωi,N+1＝(1-α)ωi,N+αMi,N+1,i＝1,2,...,K
Where α is the learning rate, for parameter Mi,N+1When x isN+1When it is not matched with the ith Gaussian distribution, Mi,N+11, otherwise, Mi,N+1＝0；
Step 1.4: background estimation in accordance with
Figure GDA0003100408800000074
The values of (d) are sorted from large to small, for K gaussian distributions of the pixel block M. Taking the first B Gaussian distributions as background models, taking the last K-B Gaussian distributions as foreground models,
Figure GDA0003100408800000075
wherein T is a threshold value, determines the number of background distributions,
judging whether the pixel block M belongs to the moving target or not, and when the gray value x of MN+1When the M is matched with a certain one of the first B Gaussian distributions, the M is considered as a background pixel; otherwise M is the target pixel;
the area composed of all target pixels is the foreground target area, in order to reduce false alarm, the key area is determined, the bottom edge of the bounding box of the foreground target is the bottom edge of the key area, the width of the area is equal to 0.8 times of the width of the vehicle target, the height is 60 pixels, the normalized key area is 80-120 pixels, and is marked as Inorm；
The calculation of the Haar-like features in the step 2 comprises the following steps:
step 2.1: the Haar-like feature is a commonly used feature description operator in the field of computer vision, the feature value of the Haar-like feature is obtained by subtracting the sum of gray values of white pixels from the sum of gray values of black pixels in a small area, the calculation cost can be reduced by adopting a block-based feature mode instead of a pixel-based mode, the feature selects a main edge feature, a line feature, a point feature (a central surrounding feature) and a diagonal feature, and as shown in fig. 2, each line represents one feature;
step 2.2: the Haar-like characteristic generally consists of 2-3 rectangular blocks, and in order to improve the calculation speed, the sum of all gray levels in the rectangle is quickly calculated by adopting an integral diagram method;
step 2.3: in order to reduce the characteristic dimension, a PCA algorithm is used for characteristic selection and dimension reduction to obtain a Haar-like characteristic vector after dimension reduction, and the Haar-like characteristic vector is recorded as FHaar-like。
The calculation of the gradient direction histogram feature of the co-occurrence matrix in the step 2 comprises the following steps:
step 2.4: respectively calculating normalized key regions InormThe magnitude mag (x, y) and direction ori (x, y) of the gradient at position (x, y), i.e.
Figure GDA0003100408800000081
Figure GDA0003100408800000082
Wherein, Inorm(x, y) denotes the normalized key region InormA pixel value at position (x, y);
step 2.5: dividing the key area into m × n small blocks according to the aspect ratio, wherein the small blocks are non-overlapped areas;
step 2.6: selecting an offset mode to scan each small block to generate a co-occurrence matrix, wherein the offset mode refers to the relative positions of the point pairs, and the offset mode includes 31 types, such as an up-down relationship, a left-right relationship and the like, one offset mode corresponds to one co-occurrence matrix,
the gradient directions of each pixel point are combined in pairs, the original gradient directions are divided into 8 gradient directions at the same time, the range is from 0 degree to 360 degrees, namely, each 45 degrees is one bin, and as each two pixel points form one combination, the size of a co-occurrence matrix is 8 × 64, so that a column vector of m × n × 64 is obtained in an offset mode;
step 2.7: an offset relationship is transformed to scan over the entire image until all offset modes are scanned once. This will generate 31 x 64 x m n column vectors, i.e. the final co-occurrence matrix gradient direction histogram feature, denoted as FCoHOG。
The calculation of the local binary pattern Fourier histogram feature in the step 2 comprises the following steps:
step 2.8: a canonical local binary pattern is calculated using the following formula,
Figure GDA0003100408800000083
Figure GDA0003100408800000084
where P denotes the number of pixels in the neighborhood in the local region, R denotes the radius of the circular neighborhood, U (LBP)P,R) Normalized local binary pattern value g representing the central pixel of an image region with a field pixel number P and a radius RcCenter pixel value of region, gp(P ═ 1, 2.. times, P) denotes the pixel value of the P-th pixel point on the neighborhood;
calculating histogram characteristics of the normative local binary pattern;
step 2.9: discrete Fourier transform of the histogram, i.e.
Figure GDA0003100408800000091
Wherein P is the number of neighboring pixel points, Up(n, r) is a certain normalization pattern, hI(Up(n, r)) is the presence of U in the image Ip(n, r) the number of normalized modes, H (n, u) representing the histogram after fourier transform;
step 2.10: local binary pattern Fourier histogram features are obtained by the following formula,
Figure GDA0003100408800000092
wherein the content of the first and second substances,
Figure GDA0003100408800000093
complex conjugation representing H (n, u)，FLBP-HFRepresenting local binary pattern fourier histogram features.
The modeling of continuous multiframes of each feature by using the autoregressive moving average model in the step 3 comprises the following steps:
step 3.1: the autoregressive moving average model is suitable for analyzing most time sequences in the objective world, has few unknown parameters and good approximation degree of characteristic optimization, considers that the current value of the sequence is a linear combination of the current and past errors and the previous sequence value, and the general form is,
Figure GDA0003100408800000094
ξt～WGN(0,σ2)
wherein p represents the autoregressive order, q represents the moving average order,
Figure GDA0003100408800000095
all represent autoregressive coefficients, θjAll represent a moving average coefficient, ξtRepresenting a random interference value, ξt～WGN(0,σ2) Is representative of xitIs a mean of 0 and a variance of σ2Normal white noise process of (Y)i(i 1, 2.., t.) represents a sequence value at time i;
step 3.2: introducing probability description strategy, and transforming the model formula into
Figure GDA0003100408800000096
Figure GDA0003100408800000097
Step 3.3: establishing a least square problem, estimating model parameters by adopting the most basic steepest descent method,
for the same critical region, the feature contains FHaar-like，FCoHOGAnd FLBP-HFAnd thirdly, establishing time sequence characteristics by adopting sequence analysis, thereby obtaining three different autoregressive moving average models.
The step 4 of identifying the video clip of the black smoke car comprises the following steps:
step 4.1: for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from a vehicle key region, the recognition result of the model with the maximum probability is selected as the recognition result of the key region, and if one key region of the current frame is recognized as a black smoke region, the current frame is recognized as a black smoke frame;
step 4.2: by integrating the analysis of multiple frames, if more than delta frames in the continuous 100 frames are identified as black smoke frames, determining that the current video segment contains black smoke cars;
the parameter delta is a coefficient for adjusting the detection rate and the report missing rate set by a user.
Fig. 3 shows an example of a black smoke vehicle detected from a vehicle surveillance video using the present invention, with the thin black rectangle representing the location area of the detected foreground object, the thick black rectangle representing the location of the key area, and the value at the bottom of the rectangle representing the probability that the area is a black smoke area.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, but any modifications or equivalent variations made according to the technical spirit of the present invention are within the scope of the present invention as claimed.

Claims (2)

1. The black smoke vehicle identification method based on the Gaussian mixture and autoregressive moving average model comprises the following specific steps and is characterized in that:
(1) detecting a vehicle moving target from a road monitoring video by using a Gaussian mixture model;
the step (1) of detecting the vehicle moving target from the road monitoring video by using the Gaussian mixture model comprises the following steps:
(11) converting each frame image into a gray image, equally dividing the gray image into small blocks of 3-by-3 pixels, and establishing a Gaussian model for each pixel block so as to improve the background modeling speed and stability and reduce the noise interference;
(12) model initialization, taking a pixel block M of a frame image as an example, the initial model of the pixel block is established by using the pixel block sequence of the former N frame sequence image, and is marked as { x1,x2,...,xNGet the average value mu of the gray levels of the pixel blocks0Sum variance
Figure FDA0003100408790000017
Initializing the mean and variance of the 1 st Gaussian distribution, i.e.
Figure FDA0003100408790000011
Figure FDA0003100408790000012
Figure FDA0003100408790000013
Wherein I (x, y, k) represents a pixel value of the pixel block at a position (x, y) in the k-th frame image;
(13) taking the (N + 1) th frame as an example to illustrate the updating method of the background model,
if the gray value x of the pixel blockN+1Satisfy | xN+1-μi,N|≤2.5σi,NThen the pixel block matches the corresponding K gaussian distributions, the parameters are updated as follows,
Figure FDA0003100408790000014
Figure FDA0003100408790000015
wherein, beta and rho are dividedOther than the learning rate of the mean and variance of the gaussian distribution,
Figure FDA0003100408790000016
representing a Gaussian probability density function, theta and gamma are fixed values and are used for adjusting the value range of the learning rate, xN+1Is the gray average value, mu, of the N +1 th frame pixel blocki,NAnd σi,N 2The mean and variance of the ith Gaussian model for the (N + 1) th frame;
if the gray value x of the pixel blockN+1Not matching with the corresponding K Gaussian distributions, the parameter mui,NAnd σi,N 2Remain unchanged, need to be xN+1Establishing a new Gaussian distribution for the mean value, the larger variance and the smaller weight to replace the smallest weight in the original K Gaussian distributions, and updating the weight omega of the K Gaussian distributionsi,N,
ωi,N+1＝(1-α)ωi,N+αMi,N+1,i＝1,2,...,K
Where α is the learning rate, for parameter Mi,N+1When x isN+1When it is not matched with the ith Gaussian distribution, Mi,N+11, otherwise, Mi,N+1＝0；
(14) Background estimation in accordance with
Figure FDA0003100408790000021
The values of the pixel blocks are sorted from large to small according to K Gaussian distributions of the pixel blocks M, the first B Gaussian distributions are taken as background models, the last K-B Gaussian distributions are taken as foreground models,
Figure FDA0003100408790000022
wherein T is a threshold value, determines the number of background distributions,
judging whether the pixel block M belongs to the moving target or not, and when the gray value x of MN+1When the M is matched with a certain one of the first B Gaussian distributions, the M is considered as a background pixel; otherwise M is the target pixel;
all object imagesThe area composed of pixels is a foreground target area, a key area needs to be determined for reducing false alarm, the bottom edge of a bounding box of the foreground target is the bottom edge of the key area, the width of the area is equal to 0.8 time of the width of a vehicle target, the height of the area is 60 pixels, the key area is normalized to be 80 x 120 pixels, and the pixel is marked as Inorm；
(2) Extracting three characteristics of a key area of the vehicle, including a Haar-like characteristic, a symbiotic matrix gradient direction histogram characteristic and a local binary pattern Fourier histogram characteristic;
the calculation of the Haar-like features in the step (2) comprises the following steps:
(21) the Haar-like feature is a commonly used feature description operator in the field of computer vision, the feature value of the Haar-like feature is obtained by subtracting the sum of gray values of white pixels from the sum of gray values of black pixels in a small area, the calculation cost can be reduced by adopting a mode based on block features instead of a mode based on pixels, the feature selects main edge features, line features, point features, namely, central surrounding features and diagonal features;
(22) the Haar-like characteristic is composed of 2-3 rectangular blocks, and in order to improve the calculation speed, the sum of all gray levels in the rectangle is quickly calculated by adopting an integral diagram method;
(23) in order to reduce the characteristic dimension, a PCA algorithm is used for characteristic selection and dimension reduction to obtain a Haar-like characteristic vector after dimension reduction, and the Haar-like characteristic vector is recorded as FHaar-like；
The calculation of the gradient direction histogram feature of the symbiotic matrix in the step (2) comprises the following steps:
(24) respectively calculating normalized key regions InormThe magnitude mag (x, y) and direction ori (x, y) of the gradient at position (x, y), i.e.
Figure FDA0003100408790000023
Figure FDA0003100408790000024
Wherein, Inorm(x, y) denotes the normalized key region InormA pixel value at position (x, y);
(25) dividing the key area into m × n small blocks according to the aspect ratio, wherein the small blocks are non-overlapped areas;
(26) selecting an offset mode to scan each small block to generate a co-occurrence matrix, wherein the offset mode refers to the relative positions of the point pairs, and the offset modes are 31 types, one offset mode corresponds to one co-occurrence matrix,
the gradient directions of each pixel point are combined in pairs, the original gradient directions are divided into 8 gradient directions at the same time, the range is from 0 degree to 360 degrees, namely, each 45 degrees is one bin, and as each two pixel points form one combination, the size of a co-occurrence matrix is 8 × 64, so that a column vector of m × n × 64 is obtained in an offset mode;
(27) transforming an offset relationship to scan across the entire image until all offset modes have been scanned once, will produce a column vector of 31 x 64 x m n, the final co-occurrence matrix gradient direction histogram feature, denoted as FCoHOG；
The calculation of the local binary pattern Fourier histogram feature in the step (2) comprises the following steps:
(28) a canonical local binary pattern is calculated using the following formula,
Figure FDA0003100408790000031
Figure FDA0003100408790000032
where P denotes the number of pixels in the neighborhood in the local region, R denotes the radius of the circular neighborhood, U (LBP)P,R) Normalized local binary pattern value g representing the central pixel of an image region with a field pixel number P and a radius RcCenter pixel value of region, gpP is 1,2, and P represents the pixel value of the P-th pixel point in the neighborhood;
calculating histogram characteristics of the normative local binary pattern;
(29) discrete Fourier transform of the histogram, i.e.
Figure FDA0003100408790000033
Wherein P is the number of neighboring pixel points, Up(n, r) is a certain normalization pattern, hI(Up(n, r)) is the presence of U in the image Ip(n, r) the number of normalized modes, H (n, u) representing the histogram after fourier transform;
(210) local binary pattern Fourier histogram features are obtained by the following formula,
Figure FDA0003100408790000034
wherein the content of the first and second substances,
Figure FDA0003100408790000035
denotes the complex conjugation of H (n, u), FLBP-HFRepresenting local binary pattern Fourier histogram features;
(3) modeling continuous multiframes of each characteristic by using an autoregressive moving average model to obtain three different models;
the modeling of the continuous multiframe of each feature by using the autoregressive moving average model in the step (3) comprises the following steps:
(31) the autoregressive moving average model is suitable for analyzing most time sequences in the objective world, has few unknown parameters and good approximation degree of characteristic optimization, considers that the current value of the sequence is a linear combination of the current and past errors and the previous sequence value, and has the form of,
Figure FDA0003100408790000041
ξt～WGN(0,σ2)
wherein p represents the autoregressive order, q represents the moving average order,
Figure FDA0003100408790000042
all represent autoregressive coefficients, θjAll represent a moving average coefficient, ξtRepresenting a random interference value, ξt～WGN(0,σ2) Is representative of xitIs a mean of 0 and a variance of σ2Normal white noise process of (Y)iI 1, 2.. and t represents a sequence value at the time i;
(32) introducing probability description strategy, and transforming the model formula into
Figure FDA0003100408790000043
Figure FDA0003100408790000044
(33) Establishing a least square problem, estimating model parameters by adopting the most basic steepest descent method,
for the same critical region, the feature contains FHaar-like，FCoHOGAnd FLBP-HFThirdly, establishing time sequence characteristics by adopting sequence analysis, thereby obtaining three different autoregressive moving average models;
(4) and for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from the key area of the vehicle, and whether the black smoke vehicle exists in the current video segment or not is judged by combining the classification results of different characteristics and the comprehensive analysis of continuous multiple frames.
2. The black smoke vehicle identification method based on the Gaussian mixture and autoregressive moving average model according to claim 1, characterized in that: the black smoke vehicle video clip identification in the step (4) comprises the following steps:
(41) for a new vehicle target, the three models are respectively used for classifying three characteristics extracted from a vehicle key region, the recognition result of the model with the maximum probability is selected as the recognition result of the key region, and if one key region of the current frame is recognized as a black smoke region, the current frame is recognized as a black smoke frame;
(42) and (4) integrating the analysis of multiple frames, if more than delta frames in the continuous 100 frames are identified as black smoke frames, determining that the current video segment contains black smoke cars, and setting the parameter delta as a coefficient for adjusting the detection rate and the false alarm rate by the user.
CN201810789111.3A 2018-07-18 2018-07-18 Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model Active CN109190455B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810789111.3A CN109190455B (en) 2018-07-18 2018-07-18 Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810789111.3A CN109190455B (en) 2018-07-18 2018-07-18 Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model

Publications (2)

Publication Number Publication Date
CN109190455A CN109190455A (en) 2019-01-11
CN109190455B true CN109190455B (en) 2021-08-13

Family

ID=64936156

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810789111.3A Active CN109190455B (en) 2018-07-18 2018-07-18 Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model

Country Status (1)

Country Link
CN (1) CN109190455B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110084220A (en) * 2019-05-08 2019-08-02 重庆邮电大学 A kind of vehicle-mounted fatigue detection method based on multiple dimensioned binary mode
CN111539343B (en) * 2020-04-26 2023-05-23 安徽皖仪科技股份有限公司 Black smoke vehicle detection method based on convolution attention network
CN111523492B (en) * 2020-04-26 2023-04-18 安徽皖仪科技股份有限公司 Detection method of black smoke vehicle
CN112598648B (en) * 2020-12-24 2022-08-26 重庆邮电大学 Image seam cutting tampering detection method based on image gradient direction

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103714554A (en) * 2013-12-12 2014-04-09 华中科技大学 Video tracking method based on spread fusion
CN104680508A (en) * 2013-11-29 2015-06-03 华为技术有限公司 Convolutional neural network and target object detection method based on convolutional neural network
CN105703954A (en) * 2016-03-17 2016-06-22 福州大学 Network data flow prediction method based on ARIMA model
CN106780548A (en) * 2016-11-16 2017-05-31 南宁市浩发科技有限公司 moving vehicle detection method based on traffic video

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101853389A (en) * 2009-04-01 2010-10-06 索尼株式会社 Detection device and method for multi-class targets

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104680508A (en) * 2013-11-29 2015-06-03 华为技术有限公司 Convolutional neural network and target object detection method based on convolutional neural network
CN103714554A (en) * 2013-12-12 2014-04-09 华中科技大学 Video tracking method based on spread fusion
CN105703954A (en) * 2016-03-17 2016-06-22 福州大学 Network data flow prediction method based on ARIMA model
CN106780548A (en) * 2016-11-16 2017-05-31 南宁市浩发科技有限公司 moving vehicle detection method based on traffic video

Also Published As

Publication number Publication date
CN109190455A (en) 2019-01-11

Similar Documents

Publication Publication Date Title
CN109190455B (en) Black smoke vehicle identification method based on Gaussian mixture and autoregressive moving average model
CN109816024B (en) Real-time vehicle logo detection method based on multi-scale feature fusion and DCNN
CN113537099B (en) Dynamic detection method for fire smoke in highway tunnel
CN109255326B (en) Traffic scene smoke intelligent detection method based on multi-dimensional information feature fusion
CN109191495B (en) Black smoke vehicle detection method based on self-organizing background difference model and multi-feature fusion
CN109086682B (en) Intelligent video black smoke vehicle detection method based on multi-feature fusion
CN108804992B (en) Crowd counting method based on deep learning
Bedruz et al. Real-time vehicle detection and tracking using a mean-shift based blob analysis and tracking approach
CN111611907A (en) Image-enhanced infrared target detection method
CN111274964B (en) Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle
CN109271904B (en) Black smoke vehicle detection method based on pixel adaptive segmentation and Bayesian model
CN111368742B (en) Reconstruction and identification method and system of double yellow traffic marking lines based on video analysis
CN111915583A (en) Vehicle and pedestrian detection method based on vehicle-mounted thermal infrared imager in complex scene
CN109325426B (en) Black smoke vehicle detection method based on three orthogonal planes time-space characteristics
CN108921147B (en) Black smoke vehicle identification method based on dynamic texture and transform domain space-time characteristics
CN115171218A (en) Material sample feeding abnormal behavior recognition system based on image recognition technology
CN111461076A (en) Smoke detection method and smoke detection system combining frame difference method and neural network
CN107977608B (en) Method for extracting road area of highway video image
CN112052768A (en) Urban illegal parking detection method and device based on unmanned aerial vehicle and storage medium
CN110516666B (en) License plate positioning method based on combination of MSER and ISODATA
CN111339823A (en) Threshing and sunning ground detection method based on machine vision and back projection algorithm
CN116152758A (en) Intelligent real-time accident detection and vehicle tracking method
CN111476233A (en) License plate number positioning method and device
CN110321973B (en) Combined vehicle detection method based on vision
CN110414386B (en) Lane line detection method based on improved SCNN (traffic channel network)

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant