CN110135254A - A kind of fatigue expression recognition method - Google Patents
A kind of fatigue expression recognition method Download PDFInfo
- Publication number
- CN110135254A CN110135254A CN201910292759.4A CN201910292759A CN110135254A CN 110135254 A CN110135254 A CN 110135254A CN 201910292759 A CN201910292759 A CN 201910292759A CN 110135254 A CN110135254 A CN 110135254A
- Authority
- CN
- China
- Prior art keywords
- feature
- value
- lbp
- gradient
- expression recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/175—Static expression
Abstract
The invention discloses a kind of tired expression recognition method merged based on local binary pattern with the histograms of oriented gradients of reconstruct, this method is the effective tired Expression Recognition carried out on the basis of Adaboost algorithm detects face and Kalman filtering tracing and positioning is facial.Firstly, the histograms of oriented gradients operator using reconstruct carries out structural information and edge extraction；Then, using local binary pattern, i.e., one of raw mode, equivalent formulations, invariable rotary mode, invariable rotary equivalent formulations these four modes carry out face texture information extraction.Then, Fusion Features extraction obtained, to constitute the new feature with texture information, structural information and marginal information.Finally, carrying out the classification based training of fatigue with non-tired expression to resulting characteristic value using support vector machines technology under self-built tired expression data library.The result shows that: this method computation complexity is low, and discrimination is high, can identify fatigue state well.
Description
Technical field
The present invention relates to intelligent Drivers to drive gesture recognition technical field, and in particular to one kind is based on partial binary mould
The tired expression recognition method that formula is merged with the histograms of oriented gradients of reconstruct.
Background technique
The research of driver fatigue identification can be divided into three classes: 1) based on the method for vehicle；2) method of Behavior-based control；3) base
In the method for physiological signal.In physiology method, driver fatigue is detected by using the physiological signal of body, example
If electroencephalogram (EEG) is for monitoring brain activity, electroculogram (EOG) is for monitoring eye movement, and electrocardiogram (ECG) is for monitoring
Heart rate.It has recently been demonstrated that being compared with other methods, fatigue detecting is carried out more using physiological signal (especially EEG signal)
Add reliable and accurate.However, the invasive measurement of physiological signal can hinder to drive, especially during long drives.Based on vehicle
Method from vehicle sensors collecting signal data (for example, steering wheel angle, lane position, speed, acceleration and braking)
To assess driving behavior.Although acquisition signals of vehicles is more convenient, these methods are for detection driver fatigue and not firm
When.The method of Behavior-based control captures the face of driver using onboard camera, and the row of driver is monitored by visual analysis
For, including eye closing, blink, yawn, head pose, gesture, facial expression etc..
Human facial expression recognition is one of the behavior analysis system of the most common view-based access control model.It is close with the psychological condition of people
Cut phase is closed, such as indignation, sad, fatigue.Facial expression recognition can efficiently and accurately detect driver fatigue.Existing face
Expression recognition method is mainly based upon the algorithm of appearance.
Method based on appearance considers facial appearance variation, such as wrinkle.Current most popular method include: part two into
Molding formula (Local Binary Patterns, LBP), histograms of oriented gradients (Histogram of Oriented
Gradients, HOG), temporal orientation energy histogram (Histogram of Spatiotemporal Orientation
Energy, HOE) and the feature extracting methods such as Gabor Wavelets.Wherein, LBP is initially by T.Ojala, M.With
D.Harwood proposed that for texture feature extraction, it can be used to describe the Local textural feature of image, have rotation in 1994
The advantages of turning invariance and gray scale invariance, but the textural characteristics that LBP (local binary patterns) descriptor extracts are limited, no
The edge and directional information of image can be described effectively；HOG is for the object detection in computer vision and image procossing
Feature descriptor, it, can be well by calculating and calculating the gradient orientation histogram of image local area come construction feature
Image structure information and marginal information are extracted, but HOG does not have rotational invariance, intrinsic dimensionality is high, data for noise sensitivity
Redundancy, image size fixation etc..
Summary of the invention
The purpose of the present invention is two technical problems for existing tired expression recognition method in fatigue expression identification:
(1) it is limited by the textural characteristics that LBP descriptor extracts, cannot effectively describe the edge and directional information of image；(2)
The structure feature that HOG descriptor extracts does not have rotational invariance, and the high computation complexity of intrinsic dimensionality is relatively high, proposes
A kind of tired expression recognition method merged based on local binary pattern with reconstruct histograms of oriented gradients.This method is not only
The texture information of image, structural information and marginal information have been merged, and by reconstruct HOG operator, has not lost image information
In the case where, intrinsic dimensionality is reduced, computation complexity is reduced.In addition, the size in this method for the image of training can not
It is fixed.
The purpose of the present invention can be reached by adopting the following technical scheme that:
A kind of tired expression recognition method merged based on local binary pattern with the histograms of oriented gradients of reconstruct, institute
The tired expression recognition method stated the following steps are included:
S1, pretreatment face face picture, wherein pretreatment includes filtering and gray processing；
S2, reconstruct histograms of oriented gradients (RHOG, Reconsteucted Histogram of Oriented is utilized
Gradients structure and edge feature, the process for) extracting face are as follows: adjusting brightness of image by Gamma correction, calculate image
Each pixel gradient, which includes size and Orientation, divides the image into several units, then calculates each unit
Histogram of gradients, the histogram of gradients with clustering by gradient direction, corresponding gradient value as longitudinal axis weight, will be several adjacent
Unit constitute a block, the statistical nature in entire image between block vector is calculated, using gained statistical nature vector as reconstructing
Histograms of oriented gradients RHOG feature；
S3, using local binary pattern (LBP, Local Binary Patterns) texture feature extraction, obtain LBP
Feature；
S4, fusion feature: series connection LBP and RHOG feature；
S5, classification based training and crosscheck are carried out to feature.
Further, the restructuring procedure of histograms of oriented gradients is as follows in the step S2:
S21, standardized gamma space and color space:
I (x, y)=I (x, y)gamma (1)
The wherein pixel in I (x, y) representative image at (x, y) point, gamma is correction parameter；
S22, image gradient is calculated:
Gx(x, y)=I (x+1, y)-I (x-1, y) (2)
Gy(x, y)=I (x, y+1)-I (x, y-1) (3)
Wherein, I (x, y) indicates the pixel in image at (x, y) point, Gx(x, y) and Gy(x, y) respectively represent in image (x,
Y) the horizontal and vertical gradient value at point, G (x, y) indicate the gradient value at point (x, y), and α is gradient direction angle；
S23, several cells are divided the image into.Gradient orientation histogram is counted in each cell.By all gradients
Direction is divided into trunnion axis of the t section (i.e. t dimensional feature vector, division mode are as shown in Figure 4) as histogram, and will be right
The vertical axis of histogram should be divided into the gradient value of angular range.
S24, block are made of 2 units of 2x.Each piece is 36 dimensional vectors.As shown in Fig. 3 and formula (6), it is with block vector work
Matrix F is constructed for basic unit, it is assumed that certain picture can be divided into n*m block, and wherein n indicates row, and m indicates to arrange, in each piece
Feature vector be corresponding unit composition feature vector series connection may be expressed as: Aij, i=1 ..., n, j=1 ..., m.Then with
Block is basic unit, and structural matrix F is as follows:
S25, reconstruction matrix will be rebuild on direction that matrix F is expert at or in the direction of the column, as shown in formula (7):
S26, five statistical natures for calculating reconstruction matrix: mean valueμ, variances sigma, the degree of biasγ, kurtosis κ, entropy H.Calculate mean valueμ,
Variances sigma, the degree of biasγ, shown in kurtosis κ method such as formula (8)-(11), shown in the calculation method of entropy H such as formula (12):
Wherein, aijRepresenting matrixOrThe i-th row jth column element, n is matrixOrRow, μjIndicate that jth column are equal
Value,Indicate jth column variance, γjIndicate the jth column degree of bias, κjIndicate jth column kurtosis, piIndicate numerical value aijAt jth column shared
Number ratio, HjFor the entropy of jth column.
Further, local binary pattern (LBP, Local Binary Patterns) is used in the step S3
Four kinds of modes: raw mode (Original Pattern), equivalent formulations (Uniform Pattern), invariable rotary mode
(Rotation Invariant Pattern), invariable rotary equivalent formulations (Rotation Invariant Uniform
Pattern), texture feature extraction, encoding operation are carried out to picture respectively.
Further, the raw mode (Original Pattern) refers to: assuming that intercepting the one of certain picture
A 3*3 window, shown in the intensity profile of the part such as Fig. 2 (a).Using center gray value as reference, binaryzation is carried out to periphery
Processing, is set as 1 for the point bigger than center pixel gray value, small point is set as 0 i.e.:
T=t (gc,g0,...,gP-1)≈t(s(g0-gc),...,s(gP-1-gc)) (13)
Wherein, T is textural characteristics, and t () is textural characteristics distribution function, and P is sampling number, gcIt is central pixel point ash
Angle value, gi(i=0,1,2 ..., P-1) indicates the gray value of P sampled point, and s is sign function:
Then the value of the LBP (Local Binary Patterns) of raw mode can as shown in Fig. 2 (b), in emulation in order to
Can not be using round local binary patterns as shown in Figure 3 by sampling number constraint, wherein R is sample radius, and the size of R is determined
The size of circle is determined；P is sampling number, reflects the resolution ratio of angular region:
Seen from the above description, binary for the LBP of raw mode (Local Binary Patterns) operator
Schema category N=2P(the wherein number that P is sampled point).
Further, the equivalent formulations (Uniform Pattern) refer to: in the basis of coding of raw mode
Coding carried out circulation move to move to right 1 bit manipulation, the circulation binary number corresponding to some partial binary (LBP) from 0 to 1 or
When person be up to jumps twice from 1 to 0, binary system corresponding to the LBP is known as an equivalent formulations.Equivalent formulations with U come
Measurement:
Wherein, P is sampling number, and R is sample radius, gcIt is central pixel point gray value, gi(i=0,1,2 ..., P-1)
It is the gray value of P sampled point.Therefore the number for obtaining equivalent formulations (Uniform Pattern) mode finally is P (P-1)+2.
Further, the invariable rotary mode (Rotation Invariant Pattern) refers to: working as image
It is rotated, gray value gPAnd g0Deng relative position will change, and g0Usually take center gcThe positive right side, coordinate (0, R), this
It will lead to different partial binaries (LBP) value.But the rotation of any angle does not influence, in circular symmetry neighborhood 0 and 1 it is opposite
Positional relationship obtains unique LBP value to remove rotation, definition:
Wherein, P is sampling number, and R is sample radius, and ri indicates invariable rotary mode, and ROR (x, i) execution adopts x-th
Sampling point is i times mobile, LBPP,RIndicate raw mode value.I.e. for image pixel, exactly Neighbourhood set is rotated according to clockwise
Many times, until the LBP value for currently rotating lower composition is minimum.The mode value that this way also significantly reduces LBP obtains quantity.
Further, the invariable rotary equivalent formulations (Rotation Invariant Uniform Pattern) refer to
: fusion invariable rotary mode and the available invariable rotary equivalent formulations of equivalent formulations may be expressed as:
Wherein, P is sampling number, and R is sample radius, and riu2 indicates invariable rotary equivalent formulations, gcIt is central pixel point
Gray value, gi(i=0,1,2 ..., P-1) is the gray value of P sampled point, and U is equivalent formulations measurement amount.
Further, it combines LBP to be formed with texture information, structure letter with the HOG of reconstruction in the step S4
The feature vector of breath and marginal information, if to press row restructuring matrixFor, shown in the feature of reconstruction such as formula (19):
Or:
Wherein, FeatureLBPFor the extracted feature of local binary pattern, FeatureRHOGFor the direction gradient of reconstruct
The feature that histogram extracts, mean are mean value, and var is variance, and skewness is the degree of bias, kuriusis kurtosis, and entropy is
Entropy.
The present invention has the following advantages and effects with respect to the prior art:
1, the present invention utilizes the index in statistics: mean value, variance, the degree of bias, kurtosis and entropy are come highly refined data characteristics.
This makes the resulting feature of histograms of oriented gradients (RHOG) method of reconstruct, and it is special not only to reduce histograms of oriented gradients (HOG)
The dimension of vector is levied, and remains the structure and feature of image.
2, the present invention has merged straight by local binary pattern (LBP) textural characteristics extracted and the direction gradient by reconstructing
Side figure (RHOG) extract structure feature and edge feature, solve local binary pattern (LBP) extraction feature it is limited and
Do not have the problem of marginal information and structural information.
3, the present invention changes in intensity of illumination, and under the non-uniform complex situations of picture illumination patterns, discrimination is high.
4, the present invention does not need fixed facial picture or monitors the size of window, all suitable for the picture of arbitrary size
With.
Detailed description of the invention
Fig. 1 is a kind of tired expression recognition method flow chart disclosed by the invention；
Fig. 2 is LBP texture blending schematic diagram, wherein
Fig. 2 (a) is that LBP rectangular texture extracts schematic diagram, wherein left side center pixel is starting point, is clockwise
Positive direction obtains corresponding LBP coding (1 100111 1)2；
Fig. 2 (b) is that LBP circular texture extracts schematic diagram, wherein surface local pixel is starting point, clockwise side
(1 100111 1) are encoded to corresponding LBP is obtained for positive direction2；
Fig. 3 is the process schematic that F matrix is generated in RHOG；
Fig. 4 is histogram of gradients partitioned mode schematic diagram.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Embodiment
As shown in Figure 1, present embodiment discloses one kind based on local binary pattern and reconstruct histograms of oriented gradients
(LBP-RHOG) the tired expression recognition method merged, includes the following steps:
S1, pretreatment picture: filtering and gray processing.
S2, reconstruct histograms of oriented gradients (RHOG, Reconsteucted Histogram of Oriented is utilized
Gradients structure and edge feature) are extracted.The restructuring procedure of RHOG is as follows:
S21, standardized gamma space and color space:
I (x, y)=I (x, y)gamma (1)
Wherein I (x, y) indicates the pixel at image (x, y), gamma=0.5.
S22, image gradient is calculated:
Gx(x, y)=I (x+1, y)-I (x-1, y) (2)
Gy(x, y)=I (x, y+1)-I (x, y-1) (3)
Wherein, I (x, y) indicates the pixel in image at (x, y) point, Gx(x, y) and Gy(x, y) respectively represent in image (x,
Y) the horizontal and vertical gradient value at point, G (x, y) indicate the gradient value at point (x, y), and α is gradient direction angle.
S23, several cells, the size 8*8 of each cell are divided the image into.Gradient side is counted in each cell
To histogram.All gradient directions are divided into 9 sections (i.e. 9 dimensional feature vectors, division mode such as Fig. 4) as histogram
Trunnion axis, and the gradient value for corresponding to angular range is divided into the vertical axis of histogram.
S24, block are made of 2 units of 2x.Each piece is 36 dimensional vectors.As shown in Fig. 3 and formula (6), it is with block vector work
Matrix F is constructed for basic unit, it is assumed that certain picture can be divided into n*m block, and wherein n indicates row, and m indicates to arrange, in each piece
Feature vector be corresponding unit composition feature vector series connection may be expressed as: Aij, i=1 ..., n, j=1 ..., m.Then with
Block is basic unit, and structural matrix F is as follows:
S25, reconstruction matrix will be rebuild, as shown in formula (7), below on direction that matrix F is expert at or in the direction of the column
Step will be withFor:
S26, five statistical natures for calculating reconstruction matrix: mean valueμ, variances sigma, the degree of biasγ, kurtosis κ, entropy H.Calculate mean valueμ,
Variances sigma, the degree of biasγ, kurtosis κ, shown in entropy H method such as formula (8)-(12):
Wherein, aijRepresenting matrixOrThe i-th row jth column element, n is matrixOrRow, μjIndicate that jth column are equal
Value,Indicate jth column variance, γjIndicate the jth column degree of bias, κjIndicate jth column kurtosis, piIndicate numerical value aijAt jth column shared
Number ratio, HjFor the entropy of jth column.
S3, local binary pattern (LBP, Local Binary Patterns) texture feature extraction is utilized.Firstly, with
The upper left corner of picture is that coordinate origin (0,0) chooses a Block as initial operation unit, can use 3*3 in this method
As initial coding unit, step-length is 1 and utilizes raw mode (Original Pattern), equivalent formulations the cell of size
(Uniform Pattern), invariable rotary mode (Rotation Invariant Pattern), invariable rotary equivalent formulations
Each cell of a pair of (Rotation Invariant Uniform Pattern) carries out feature extraction, encoding operation can obtain
The textural characteristics of face, wherein sample radius R is 1, and sampling number P is 8, and the operating process of four kinds of modes is as follows:
(1) characteristic extraction procedure under raw mode (Original Pattern): right using center gray value as reference
Periphery carries out binary conversion treatment, the point bigger than center pixel gray value is set as 1, small point is set as 0 i.e.:
T=t (gc,g0,...,gP-1)≈t(s(g0-gc),...,s(gP-1-gc)) (13)
Wherein, T is textural characteristics, and t () is textural characteristics distribution function, and P is sampling number, gcIt is central pixel point ash
Angle value, gi(i=0,1,2 ..., P-1) is the gray value of P sampled point, and s is sign function:
The then value of the LBP (Local Binary Patterns) of raw mode, will be using as schemed in this method simulation process
Circle local binary patterns shown in 2 (b), wherein R is sample radius；P is sampling number.
(2) equivalent formulations (Uniform Pattern) operating process: coding is carried out in the basis of coding of raw mode
1 bit manipulation of cyclic shift, the circulation binary number corresponding to some partial binary (LBP) from 0 to 1 or from 1 to 0 at most
Have when jumping twice, binary system corresponding to the LBP is known as an equivalent formulations.Equivalent formulations are measured with U:
Wherein, P is sampling number, and R is sample radius, gcIt is central pixel point gray value, gi(i=0,1,2 ..., P-1)
It is the gray value of P sampled point, all modes of U≤2 are referred to as equivalent formulations.Therefore equivalent formulations (Uniform is obtained finally
Pattern) number of mode is P (P-1)+2.
(3) invariable rotary mode (Rotation Invariant Pattern) operating process: when image is rotated, gray scale
Value gPAnd g0Deng relative position will change, and g0Usually take center gcThe positive right side, coordinate (0, R), this will lead to different
Partial binary (LBP) value.But the rotation of any angle does not influence in circular symmetry neighborhood 0 and 1 relative positional relationship, in order to
It removes rotation and obtains unique local binary patterns (LBP) value, be defined as follows:
Wherein, P is sampling number, and R is sample radius, and ri is invariable rotary mode, and ROR (x, i), which is executed, samples x-th
Point is i times mobile, LBPP,RIndicate raw mode value.It is exactly to rotate Neighbourhood set very according to clockwise i.e. for image pixel
Repeatedly, until the LBP value for currently rotating lower composition is minimum.The mode value that this way also significantly reduces LBP obtains quantity.
(4) invariable rotary equivalent formulations (Rotation Invariant Uniform Pattern) operating process: fusion
Invariable rotary mode and the available invariable rotary equivalent formulations of equivalent formulations, may be expressed as:
Wherein, P is sampling number, and R is sample radius, and riu2 is invariable rotary equivalent formulations, gcIt is central pixel point ash
Angle value, gi(i=0,1,2 ..., P-1) is the gray value of P sampled point, and U is equivalent formulations measurement amount.
S4, LBP and RHOG Fusion Features.It combines LBP to be formed with texture information, structural information with the HOG of reconstruction
With the feature vector of marginal information.If to press row restructuring matrixFor, shown in the feature of reconstruction such as formula (19):
It combines LBP to be formed and be believed with texture information, structural information and edge with the HOG of reconstruction in the step S4
The feature vector of breath, if to press row restructuring matrixFor, shown in the feature of reconstruction such as formula (19):
Or:
Wherein, FeatureLBPFor the extracted feature of local binary pattern, FeatureRHOGFor the direction gradient of reconstruct
The feature that histogram extracts, mean are mean value, and var is variance, and skewness is the degree of bias, kuriusis kurtosis, and entropy is
Entropy.
S5, classification based training and crosscheck are carried out to fused feature: the picture in tired expression library (is randomly selected
70% fatigue and the picture of non-fatigue state in picture library) repeat the operation of step S2- step S4 respectively.Utilize SVM
(Support Vector Machine) carries out classification based training to the feature coding that the above process generates.Finally utilize tired expression
Remaining 20% picture is tested in library, and 10% picture carries out cross validation.
Experiment shows that the discrimination of HOG algorithm is apparently higher than other algorithms.It is worth noting that, the feature dimensions of HOG method
Number about 25000.And the intrinsic dimensionality of RHOG algorithm is only 180.The reason is that we have used several statistics special in RHOG
Sign, wherein mean value is the index for reflecting general trend or being distributed central tendency, variance be variation for characterizing overall distribution or
Discrete, index degree.One important indicator of skewness and kurtosis, reflects the coefficient of the distribution shape of data set.And entropy is then retouched
The average information of the gradient of image is stated.These statistical data height refine the structural information of image, eliminate redundancy, thus
Reduce training and identification cost.
The above embodiment is a preferred embodiment of the present invention, but embodiments of the present invention are not by above-described embodiment
Limitation, other any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present invention,
It should be equivalent substitute mode, be included within the scope of the present invention.
Claims (9)
1. a kind of tired expression recognition method merged based on local binary pattern with the histograms of oriented gradients of reconstruct, special
Sign is, the tired expression recognition method the following steps are included:
S1, pretreatment face face picture, wherein pretreatment includes filtering and gray processing；
S2, the structure and edge feature that face is extracted using reconstruct histograms of oriented gradients, obtain RHOG feature, process is as follows:
Brightness of image is adjusted by Gamma correction, calculates the gradient of each pixel of image, which includes size and Orientation, will be schemed
As being divided into several units, the histogram of gradients of each unit is then calculated, the histogram of gradients is corresponding with clustering by gradient direction
Gradient value as longitudinal axis weight, several adjacent units are constituted into a blocks, calculate the statistics in entire image between block vector
Feature, using gained statistical nature vector as the RHOG feature of the histograms of oriented gradients of reconstruct；
S3, using local binary pattern texture feature extraction, obtain LBP feature；
S4, pass through series connection LBP feature and RHOG feature, carry out Fusion Features:
S5, classification based training and crosscheck are carried out to feature after fusion.
2. fatigue expression recognition method according to claim 1, which is characterized in that direction gradient is straight in the step S2
The restructuring procedure of square figure is as follows:
S21, standardized gamma space and color space:
I (x, y)=I (x, y)gamma (1)
The wherein pixel in I (x, y) representative image at (x, y) point, gamma is correction parameter；
S22, image gradient is calculated:
Gx(x, y)=I (x+1, y)-I (x-1, y) (2)
Gy(x, y)=I (x, y+1)-I (x, y-1) (3)
Wherein, I (x, y) indicates the pixel in image at (x, y) point, Gx(x, y) and Gy(x, y) respectively represents in image (x, y) point
The horizontal and vertical gradient value at place, G (x, y) indicate the gradient value at point (x, y), and α is gradient direction angle；
S23, several cells are divided the image into, counts gradient orientation histogram in each cell, by all gradient directions
It is divided into trunnion axis of the t section as histogram, and the gradient value for corresponding to angular range is divided into hanging down for histogram
D-axis；
S24, by 2x2 unit blocking, matrix F is constructed using block vector as basic unit, it is assumed that certain picture can be divided into
N*m block, wherein n indicates to go, and m indicates column, and the feature vector in each piece is the feature vector series connection table of corresponding unit composition
It is shown as: Aij, i=1 ..., n, j=1 ..., m, then using block as basic unit, structural matrix F is as follows:
S25, reconstruction matrix will be rebuild on direction that matrix F is expert at or in the direction of the column, as shown in formula (7):
Wherein,For by row reconstruct matrix,For by the restructuring matrix of column；
S26, the statistical nature for calculating reconstruction matrix, which includes mean μ, variances sigma, degree of bias γ, kurtosis κ, entropy H.
3. fatigue expression recognition method according to claim 2, which is characterized in that the mean μ, variances sigma2, the degree of bias
γ, kurtosis κ, the calculation formula difference of entropy H are as follows:
Wherein, aijRepresenting matrixOrThe i-th row jth column element, n is matrixOrRow, μjIndicate jth column mean,
Indicate jth column variance, γjIndicate the jth column degree of bias, κjIndicate jth column kurtosis, piIndicate numerical value aijShared number ratio is arranged in jth
Example, HjFor the entropy of jth column.
4. fatigue expression recognition method according to claim 1, which is characterized in that using part two in the step S3
Four kinds of modes of multilevel mode: raw mode, equivalent formulations, invariable rotary mode, invariable rotary equivalent formulations, respectively to picture
Carry out texture feature extraction and encoding operation.
5. fatigue expression recognition method according to claim 4, which is characterized in that the raw mode refers to: false
If intercepting a 3*3 window of certain picture, using center gray value as reference, binary conversion treatment is carried out to periphery, it will be than in
The big point of heart grey scale pixel value is set as 1, and small point is set as 0, it may be assumed that
T=t (gc,g0,...,gP-1)≈t(s(g0-gc),...,s(gP-1-gc)) (13)
Wherein, T is textural characteristics, and t () is textural characteristics distribution function, and P is sampling number, gcIt is central pixel point gray value,
gi(i=0,1,2 ..., P-1) is the gray value of P sampled point, and s is sign function:
Then the value calculation formula of the LBP of raw mode is as follows:
6. fatigue expression recognition method according to claim 4, which is characterized in that the equivalent formulations refer to:
It coding is carried out circulation in the basis of coding of raw mode moves to move to right 1 bit manipulation, follow corresponding to some partial binary LBP
When ring binary number be up to jumps twice from 0 to 1 or from 1 to 0, binary system corresponding to the LBP is known as an equivalence
Mode, equivalent formulations are measured with U:
Wherein, P is sampling number, and R is sample radius, gcIt is central pixel point gray value, gi(i=0,1,2 ..., P-1) it is P
All modes of the gray value of a sampled point, U≤2 are referred to as equivalent formulations.
7. fatigue expression recognition method according to claim 4, which is characterized in that the invariable rotary mode referred to
It is: when image is rotated, gray value gPAnd g0Deng relative position will change, and g0Take center gcThe positive right side, coordinate (0,
R), this will lead to different partial binary LBP values, consider that the rotation of any angle does not influence, 0 and 1 in circular symmetry neighborhood
Relative positional relationship, obtain unique LBP value to remove rotation, definition:
Wherein, P is sampling number, and R is sample radius, and ri indicates invariable rotary mode, and ROR (x, i) is executed x-th of sampled point
It is i times mobile, LBPP,RFor raw mode value.
8. fatigue expression recognition method according to claim 4, which is characterized in that the invariable rotary equivalent formulations refer to
: fusion invariable rotary mode and equivalent formulations obtain invariable rotary equivalent formulations, indicate are as follows:
Wherein, P is sampling number, and R is sample radius, and riu2 indicates invariable rotary equivalent formulations, gcIt is central pixel point gray scale
Value, gi(i=0,1,2 ..., P-1) is the gray value of P sampled point, and U is equivalent formulations measurement amount.
9. fatigue expression recognition method according to claim 1, which is characterized in that by LBP and again in the step S4
The HOG built is combined to form the feature vector with texture information, structural information and marginal information, if to press row restructuring matrix
For, shown in the feature of reconstruction such as formula (19):
Or:
Wherein, FeatureLBPFor the extracted feature of local binary pattern, FeatureRHOGFor the direction gradient histogram of reconstruct
Scheme the feature extracted, mean is mean value, and var is variance, and skewness is the degree of bias, and kuriusis kurtosis, entropy is entropy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910292759.4A CN110135254A (en) | 2019-04-12 | 2019-04-12 | A kind of fatigue expression recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910292759.4A CN110135254A (en) | 2019-04-12 | 2019-04-12 | A kind of fatigue expression recognition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110135254A true CN110135254A (en) | 2019-08-16 |
Family
ID=67569878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910292759.4A Pending CN110135254A (en) | 2019-04-12 | 2019-04-12 | A kind of fatigue expression recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110135254A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110619301A (en) * | 2019-09-13 | 2019-12-27 | 道和安邦（天津）安防科技有限公司 | Emotion automatic identification method based on bimodal signals |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400138A (en) * | 2013-07-29 | 2013-11-20 | 沈玉琴 | Video signal preprocessing method for artificial intelligent multimode behavior recognition and description |
| CN103810490A (en) * | 2014-02-14 | 2014-05-21 | 海信集团有限公司 | Method and device for confirming attribute of face image |
| CN104091147A (en) * | 2014-06-11 | 2014-10-08 | 华南理工大学 | Near infrared eye positioning and eye state identification method |
| CN106250811A (en) * | 2016-06-15 | 2016-12-21 | 南京工程学院 | Unconfinement face identification method based on HOG feature rarefaction representation |
| CN106339702A (en) * | 2016-11-03 | 2017-01-18 | 北京星宇联合投资管理有限公司 | Multi-feature fusion based face identification method |
| CN106781513A (en) * | 2016-11-28 | 2017-05-31 | 东南大学 | The recognition methods of vehicle behavior in a kind of urban transportation scene of feature based fusion |
| CN107578007A (en) * | 2017-09-01 | 2018-01-12 | 杭州电子科技大学 | A kind of deep learning face identification method based on multi-feature fusion |
| CN109389074A (en) * | 2018-09-29 | 2019-02-26 | 东北大学 | A kind of expression recognition method extracted based on human face characteristic point |
-
2019
- 2019-04-12 CN CN201910292759.4A patent/CN110135254A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400138A (en) * | 2013-07-29 | 2013-11-20 | 沈玉琴 | Video signal preprocessing method for artificial intelligent multimode behavior recognition and description |
| CN103810490A (en) * | 2014-02-14 | 2014-05-21 | 海信集团有限公司 | Method and device for confirming attribute of face image |
| CN104091147A (en) * | 2014-06-11 | 2014-10-08 | 华南理工大学 | Near infrared eye positioning and eye state identification method |
| CN106250811A (en) * | 2016-06-15 | 2016-12-21 | 南京工程学院 | Unconfinement face identification method based on HOG feature rarefaction representation |
| CN106339702A (en) * | 2016-11-03 | 2017-01-18 | 北京星宇联合投资管理有限公司 | Multi-feature fusion based face identification method |
| CN106781513A (en) * | 2016-11-28 | 2017-05-31 | 东南大学 | The recognition methods of vehicle behavior in a kind of urban transportation scene of feature based fusion |
| CN107578007A (en) * | 2017-09-01 | 2018-01-12 | 杭州电子科技大学 | A kind of deep learning face identification method based on multi-feature fusion |
| CN109389074A (en) * | 2018-09-29 | 2019-02-26 | 东北大学 | A kind of expression recognition method extracted based on human face characteristic point |
Non-Patent Citations (3)
| Title |
|---|
| 孙玉 等: "基于HOG与LBP特征的人脸识别方法", 《计算机工程》 * |
| 李玲: "基于智能终端的疲劳驾驶检测系统的研究与实现", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 * |
| 肖华军: "面向DSP实现的视频头肩检测与跟踪算法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110619301A (en) * | 2019-09-13 | 2019-12-27 | 道和安邦（天津）安防科技有限公司 | Emotion automatic identification method based on bimodal signals |
| CN110619301B (en) * | 2019-09-13 | 2023-04-18 | 道和安邦（天津）安防科技有限公司 | Emotion automatic identification method based on bimodal signals |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Fully convolutional network ensembles for white matter hyperintensities segmentation in MR images | |
| Hammad et al. | A novel two-dimensional ECG feature extraction and classification algorithm based on convolution neural network for human authentication | |
| Cichy et al. | Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks | |
| CN110876626B (en) | Depression detection system based on optimal lead selection of multi-lead electroencephalogram | |
| CN109389585A (en) | A kind of brain tissue extraction method based on full convolutional neural networks | |
| Fan et al. | The bottom-up and top-down processing of faces in the human occipitotemporal cortex | |
| Wang et al. | Eye gaze pattern analysis for fatigue detection based on GP-BCNN with ESM | |
| González-Ortega et al. | Real-time vision-based eye state detection for driver alertness monitoring | |
| Deepak et al. | Visual saliency based bright lesion detection and discrimination in retinal images | |
| Cuppens et al. | Using spatio-temporal interest points (STIP) for myoclonic jerk detection in nocturnal video | |
| CN104765959A (en) | Computer vision based evaluation method for general movement of baby | |
| Wei et al. | Image feature extraction and object recognition based on vision neural mechanism | |
| Zhang et al. | Fused group lasso: A new EEG classification model with spatial smooth constraint for motor imagery-based brain–computer interface | |
| CN110097012B (en) | Fatigue detection method for monitoring eye movement parameters based on N-range image processing algorithm | |
| CN110135254A (en) | A kind of fatigue expression recognition method | |
| CN111310798A (en) | Construction method of face bradykinesia detection model based on geometric features and textural features | |
| Izadpanahi et al. | Human age classification with optimal geometric ratios and wrinkle analysis | |
| Björnsdotter et al. | A memetic algorithm for selection of 3d clustered features with applications in neuroscience | |
| Panicker et al. | Open-eye detection using iris–sclera pattern analysis for driver drowsiness detection | |
| Hussein et al. | Improved driver drowsiness detection model using relevant eye image’s features | |
| CN116955943A (en) | Driving distraction state identification method based on eye movement sequence space-time semantic feature analysis | |
| Luo et al. | Multi-model fusion on real-time drowsiness detection for telemetric robotics tracking applications | |
| CN113989887A (en) | Equipment operator fatigue state detection method based on visual characteristic information fusion | |
| CN111861275B (en) | Household work mode identification method and device | |
| Xia et al. | Robust retinal vessel segmentation via clustering-based patch mapping functions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190816 |
|
| RJ01 | Rejection of invention patent application after publication |