CN105095879A - Eye state identification method based on feature fusion - Google Patents

Abstract

The invention discloses an eye state identification method based on feature fusion, belongs to the image processing and mode identification field, and is suitable for face expression and mental state identification, driver fatigue detection and sight line tracking. The method is characterized by, to begin with, carrying out face positioning and eye positioning on an input image to obtain an eye area; and then, judging the eye state by utilizing an eye state recognition algorithm based on pseudo-Zernike moment_Gabor feature fusion (PZ_GAB). The eye state identification method has very good robustness under the condition of head movement and complex illumination variation, can effectively solve the problems that eye state identification precision is reduced, and meanwhile, computation complexity is not high, and meets real-time requirement.

Description

The eye state identification method that feature based merges

Technical field

This method belongs to image procossing and area of pattern recognition, relates to the eye state identification method that a kind of feature based merges.

Background technology

There is important effect in a lot of field such as man-machine interaction, driver fatigue detection, eye tracking that is identified in of eye state, and its accuracy rate differentiated directly affects the performance of these systems.In actual applications, in order to ensure that eye state is sentenced method for distinguishing and can round-the-clockly be used, method the most frequently used is at present exactly the image-pickup method using active infrared light source and optical filter to combine.But when infrared illumination adds optical filter, there is larger difference in the eye image under eye image and normal illumination; In practical application, the significantly rotation of head makes eye image that comparatively large deformation occur in addition, causes the feature failure originally extracted.Therefore, suitable human eye feature and the excellent eye state sorter of design performance is selected to be the keys improving Eye states recognition.

Eye state method of discrimination according to the characteristic sum model of eye state identification method employing can be divided into eye state method of discrimination and Corpus--based Method based on macroscopic features to learn:

Based on the eye state method of discrimination of macroscopic features, some intrinsic external appearance characteristics of eyes are utilized to identify, the intensity profile etc. of the shape of such as iris, the curvature of eyelid, eyes.Some intrinsic external appearance characteristics of eyes can be subject to extraneous environmental impact and change, and under the uncontrollable condition of reality, this method often becomes and unreliable.

The eye state method of discrimination of Corpus--based Method study, this method often needs the training sample under a large amount of different changing pattern, such learning model that could allow can have enough good generalization ability, therefore it is when processing uncertain sample data, shows better stability and robustness than the recognition methods based on macroscopic features.

In the eye state method of discrimination of Corpus--based Method study, conventional feature mainly contains illumination invariant characteristic sum invariant moment features.Wherein illumination invariant feature is a kind of feature that can remain unchanged under different light environment, can overcome the impact of illumination variation in Eye states recognition process, and conventional illumination invariant feature mainly contains Gabor characteristic, HOG feature, LBP feature; Bending moment is not a kind of characteristics of image with translation, Invariant to rotation and scale, can overcome the impact of head rotation and distance in Eye states recognition process, and conventional moment characteristics mainly contains Hu square, Zernike square, wavelet moment etc.

In practical application scene, Eye states recognition is vulnerable to the impact of head movement, complex illumination, single feature cannot meet the robustness requirement of practical application, need carrying out eye image on the basis of manifold extraction and effect analysis, propose a kind of Eye states recognition algorithm of Fusion Features.

Summary of the invention

The object of the invention is to the eye state identification method proposing the fusion of a kind of feature based, the method better solves the identification problem of eye state under head movement and complex illumination scene change, this valve can export eye state in real time exactly, thus improves the robustness of corresponding intelligent system.The present invention is achieved through the following technical solutions.

The eye state identification method that feature based merges, the method comprises: (1) human face region and eyes location; (2) extraction of Zernike pseudo-matrix proper vector and Gabor characteristic vector; (3) fusion of Zernike pseudo-matrix proper vector and Gabor characteristic vector; (4) training of Eye states recognition model; (5) to the differentiation of new input eye state.

In said method, described step (2), comprising:

1.1): by the eye image that image capturing system collection is appropriate, comprise and open eyes and close one's eyes, and normalize to 64 × 48 pixel sizes, as training sample image.

1.2): the invariant moment features Zernike pseudo-matrix proper vector and the illumination invariant feature Gabor characteristic vector that extract training sample image respectively, and utilize PCA (PrincipalComponentAnalysis, principal component analysis (PCA)) method Gabor characteristic vector carries out dimensionality reduction and adopts LDA (LinearDiscriminantAnalysis, linear discriminant analysis) scatter matrix between scatter matrix and class is carried out in reconstruction class to the Gabor characteristic vector after dimensionality reduction, thus obtain dimensionality reduction and the vector of the Gabor characteristic after rebuilding.

1.3): Zernike pseudo-matrix proper vector and dimensionality reduction and the Gabor characteristic vector after rebuilding are normalized and dimension polishing, then carry out Parallel Fusion, obtain PZ_GAB Fusion Features proper vector.

In said method, described step (4), comprising:

2.1): the PZ_GAB fusion feature vector of all training sample image is input to based on Radial basis kernel function SVM (SupportVectorMachine, support vector machine) model training, obtains Eye states recognition model.

In said method, described step (3) comprising: merge obtaining the Zernike pseudo-matrix proper vector of eye sample image and dimensionality reduction and the Gabor characteristic vector after rebuilding in step (2), thus obtains the eye feature vector PZ_GAB feature after merging

In said method, described step (3), comprising:

3.1): gather to obtain a two field picture by image capturing system, face and eyes location are carried out respectively to this two field picture, if locate successfully, obtains eye areas image, perform step 3.2, otherwise skip this two field picture, continue to obtain next frame image.

3.2): eye areas size normalization step 3.1 obtained is to 64 × 48 pixel sizes, extract Zernike pseudo-matrix proper vector and Gabor characteristic vector respectively, and utilize PCA method to carry out dimensionality reduction to Gabor characteristic vector, LDA (LinearDiscriminantAnalysis, linear discriminant analysis) is adopted to carry out in reconstruction class scatter matrix between scatter matrix and class to the Gabor characteristic vector after dimensionality reduction.Zernike pseudo-matrix proper vector and dimensionality reduction and the Gabor characteristic vector after rebuilding are carried out Parallel Fusion, obtains PZ_GAB Fusion Features proper vector.

3.2): be input in step 2 train the Eye states recognition model obtained to carry out Eye states recognition by merging the proper vector obtained in step 3.2, finally this two field picture eye state is exported.

Compared with prior art, tool of the present invention has the following advantages and technique effect:

1, the present invention is directed to Eye states recognition be vulnerable to the impact of complex illumination change and head movement and cause the problem that accuracy of identification is not high, propose a kind of new Feature Fusion Algorithm in conjunction with illumination invariant feature and invariant moment features.The present invention can overcome open and close eyes, the impact of the factor such as glasses reflection, head rotation, robustly locate eye position and accurately export eye state information, thus improving the robustness of corresponding intelligent system;

2, after this method adopts Feature Dimension Reduction, computation complexity is low, can meet the requirement of real-time of system;

Accompanying drawing explanation

Fig. 1 is the overall flow figure of the eye state identification method that a kind of feature based of the present invention merges.

Fig. 2 is the Eye states recognition process flow diagram of Fusion Features.

Embodiment

Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described further.

Composition graphs 1, the eye state identification method that a kind of feature based of the present invention merges, embodiment is as follows:

Step 1: train the eyes cascade sort detecting device based on Haar feature, by the eye image that image capturing system collection is appropriate, comprise and open eyes and close one's eyes, to the image of eyebrow and eyes be comprised as positive sample, non-ocular image, as negative sample, trains the eyes cascade sort detecting device based on Haar feature by Adaboost algorithm.

Step 2: training Eye states recognition model, composition graphs 2, concrete steps are as follows:

2.1): by the eye image that the image capturing system identical with step 1 is appropriate, comprise and open eyes and close one's eyes, and normalize to 64 × 48 pixel sizes, as training sample image.

2.2): the Zernike pseudo-matrix feature extracting sample eye image, concrete steps are as follows:

A): extract eye image Zernike pseudo-matrix feature before, the present invention first adopt one fast and effectively adaptive thresholding algorithm binaryzation is carried out to eye image.Traversal eye image, to choose in image the maximum median pixel value with minimum pixel value as threshold value, thus carries out binaryzation to image.This threshold value is selected by pixel value domain space number percent, in the present invention based on experience value capture element codomain 0.4.

B): the Zernike pseudo-matrix feature calculating eye image, in the present invention, get its 0 to 9 rank moment characteristics, 55 dimensional vector plural numbers can be obtained.

Zernike pseudo-matrix is a kind of orthogonal plural square, and it can be tried to achieve according to formula (1):

$A_{nm}=\frac{n+1}{\mathrm{\π}}\underset{0}{\overset{2\mathrm{\π}}{\∫}}\underset{0}{\overset{1}{\∫}}f(\mathrm{\ρ},\mathrm{\θ})V_{nm}^{*}(\mathrm{\ρ},\mathrm{\θ})\mathrm{\ρ}d\mathrm{\ρ}d\mathrm{\θ}---\left(1\right)$

Zernike square is that image function f (ρ, θ) is at orthogonal polynomial V _nmprojection on (ρ, θ).Wherein V _nm(ρ, θ) is orthogonal in unit circle, and its expression formula is such as formula shown in (2).

V _nm(ρ,θ)＝R _nm(ρ)e ^jmθ(2)

$R_{nm}\left(\mathrm{\ρ}\right)={\mathrm{\Σ}}_{s=0}^{n-\left|m\right|}{D}_{n,\left|m\right|s}{\mathrm{\ρ}}^{n-s}---\left(3\right)$

$D_{n,\left|m\right|s}={(-1)}^s\frac{(2n+1-s)!}{s!(n-|m|-s)!(n+|m|+1-s)!}---\left(4\right)$

Wherein n is positive integer or zero, m is positive integer or negative integer, and must meet n-|m|=even, | m|≤n, ρ are the vector that initial point arrives that (x, y) puts length, and θ is the angle of vector ρ and axle x, R _nm(ρ) be radial polynomial.

2.3): the Gabor characteristic extracting sample image, Two-Dimensional Gabor Wavelets conversion is the important tool of carrying out signal analysis and processing at time-frequency domain, and its conversion coefficient has good visual characteristic and Biological background, and concrete steps are as follows:

A): by the definition of Two-Dimensional Gabor Wavelets kernel such as formula shown in (5), can be in the hope of.

$\mathrm{\ψ}(k,z)=\frac{\left|\right|k|{|}^2}{{\mathrm{\σ}}^2}{e}^{-\frac{\left|\right|k\left|{|}^2\right|\left|z\right|{|}^2}{2{\mathrm{\σ}}^2}}(e^{ikz}-e^{-\frac{{\mathrm{\σ}}^2}{2}})---\left(5\right)$

Wherein σ is the constant relevant with small echo frequency bandwidth, and z=(x, y) is locus coordinate; K determines direction and the yardstick of Gabor kernel.In algorithm of the present invention, adopt the sampling of 8 directions and 5 yardsticks, the k on certain direction and yardstick is just such as formula shown in (6).

$k_{u,v}=k_v{e}^{{\mathrm{i\φ}}_{\mathrm{\μ}}}---\left(6\right)$

Wherein k _v=k _max/ f ^v, be sampling scale, v belongs to 0,1 ..., 4} is yardstick label, φ _μ=π μ/8 are sample direction, μ belongs to 0,1 ..., 7} is direction label.K _maxbe maximum frequency, f is the kernel interval factor in frequency domain.In the present invention, parameter k _max=pi/2, σ=2 π, can obtain good small echo and characterize and identification effect.

B): the Gabor characteristic of being tried to achieve sample image by the convolution of sample image I and Gabor kernel, such as formula (7).

J _k(z)＝I(z)*ψ(k,z)(7)

If J _kz amplitude and the phase place of () are respectively A _kwith then the J in combination different scale and direction _kz (), composing images is at the Gabor characteristic vector of z position.

2.4): utilize PCA method to carry out dimensionality reduction to the Gabor characteristic vector obtained, obtain the proper subspace of low-dimensional, in the low dimension formulation feature got from PCA again, with LDA, scatter matrix between scatter matrix and class is carried out in reconstruction class to it, thus obtain dimensionality reduction and the vector of the Gabor characteristic after rebuilding.

2.5): the Gabor characteristic vector after carrying out PCA and LDA process is normalized with Zernike pseudo-matrix proper vector and carries out dimension polishing.Then Parallel Fusion, obtains fusion feature vector, merges such as formula (8).

γ＝α+i·β(8)

Wherein A, B are Zernike pseudo-matrix feature and Gabor feature space in sample space respectively, for arbitrary sample Γ ∈ Ω, Zernike pseudo-matrix character representation vector is, Gabor characteristic represents that vector is β ∈ B, and these two proper vectors are fused into complex vector γ by Parallel Fusion exactly.

2.6): the input of the PZ_GAB proper vector of all sample images is trained based in the SVM model of Radial basis kernel function, obtains Eye states recognition model.

Step 3: obtain a two field picture, and on the image obtained, human face region is positioned, if locate successfully, obtain facial image, perform step 4, otherwise skip this two field picture, continue to obtain next frame image.

Step 4: utilize the eyes cascade sort detecting device location eye areas based on Haar feature at the eyes area-of-interest obtained, if locate successfully, i.e. exportable eyes exact position, obtains eye image.

Step 5: composition graphs 2, eye areas size normalization step 4 obtained is to 64 × 48 pixel sizes, according to step 2.2)-2.4) extract Zernike pseudo-matrix proper vector and Gabor characteristic vector respectively, utilize PCA method to carry out dimensionality reduction to the Gabor characteristic vector obtained, adopt LDA to carry out in reconstruction class scatter matrix between scatter matrix and class to the Gabor characteristic vector after dimensionality reduction.Zernike pseudo-matrix proper vector and dimensionality reduction and the Gabor characteristic vector after rebuilding are carried out Parallel Fusion, obtains PZ_GAB Fusion Features proper vector.Being input to step 2 trains the Eye states recognition model obtained to carry out Eye states recognition, final this two field picture eye state of output.

Step 6: repeat step 3 ~ 5, exports Eye states recognition result in real time.

Claims (6)

1. the eye state identification method of feature based fusion, is characterized in that comprising the steps:

(1) human face region and eyes location;

(2) extraction of Zernike pseudo-matrix proper vector and Gabor characteristic vector;

(3) fusion of Zernike pseudo-matrix proper vector and Gabor characteristic vector;

(4) training of Eye states recognition model;

(5) to the differentiation of new input eye state.

2. the eye state identification method of feature based fusion according to claim 1, it is characterized in that described step (2) comprising: the eye areas image obtained in step (1) is normalized, then adaptive threshold binaryzation is carried out to normalized eye areas sample image, obtain binaryzation eye image; Its Zernike pseudo-matrix feature is asked to above-mentioned bianry image, obtains the Zernike pseudo-matrix proper vector of eye image.

3. the eye state identification method of feature based fusion according to claim 2, it is characterized in that described self-adaption binaryzation step comprises: by the eye areas sample image after traversal normalization, seek obtaining maximum and minimum pixel value in image, then get median pixel value in minimax pixel value as adaptive threshold.

4. the eye state identification method of feature based fusion according to claim 1, it is characterized in that described step (2) also comprises: first locating the eye areas image that obtains and Gabor kernel carries out convolution to carrying out eyes in step (1), trying to achieve the Gabor characteristic of eye sample image; The Gabor characteristic vector of the method for PCA dimensionality reduction to the eye sample image extracted is utilized to carry out dimensionality reduction, obtain to obtain the best Expressive Features of low dimension, then LDA reconstruction process is carried out to the feature in PCA proper subspace, obtain dimensionality reduction and the vector of the Gabor characteristic after rebuilding.

5. the eye state identification method of feature based fusion according to claim 1, it is characterized in that described step (3) comprising: merge obtaining the Zernike pseudo-matrix proper vector of eye sample image and dimensionality reduction and the Gabor characteristic vector after rebuilding in step (2), thus obtain the eye feature vector PZ_GAB feature after merging.

6. the eye state identification method of feature based fusion according to claim 5, it is characterized in that described Fusion Features step comprises: first to each eigenwert application min-max normalization scheme to calculate normalization Zernike pseudo-matrix proper vector and Gabor characteristic vector to make eigenwert compatible, then polishing is carried out to two feature vector dimension, namely be as the criterion with the intrinsic dimensionality of higher-dimension most in two proper vectors, by mending 0 to the vector of low-dimensional, to reach dimension consistent; Finally to normalization with neat to dimension after Zernike pseudo-matrix proper vector and Gabor characteristic vector carry out in parallel fusion, obtain merging plural PZ_GAB proper vector.