CN104036238B - The method of the human eye positioning based on active light - Google Patents

Abstract

In the human eye positioning method based on active light, the active light generation device is used in the video device to actively project the human face, and an image capture device is set to extract two images of bright pupil and dark pupil; the bright pupil effect caused by active light projection is The candidate area for human eye positioning is obtained through the difference of two images and the image filtering method; the human eye positioning is then completed by using the face and human eye positioning methods. Especially through knowledge-based, feature-based, template matching or appearance-based face positioning methods to locate the face area; according to the geometric characteristics of the face, through knowledge-based, feature-based, template matching or based on The representational human eye positioning method locates the human eye area.

Description

A Method of Human Eye Positioning Based on Active Light

technical field

The invention relates to a method for positioning human eyes, in particular to a method for positioning human eyes based on active light used in a video device.

Background technique

The research on human eye positioning has a long history, and the earliest research work can be traced back to the 1940s, but the real development is still in the last 20 years. There are usually three types of input images for human eye positioning: front, side, and oblique. Since IBM's work in 1997, most human eye positioning research works have focused on frontal or near-frontal human eye images.

Human eye positioning is a very challenging topic with important theoretical research value and application value. Eye positioning refers to checking whether there is a human eye in an image. If so, the position and scale of the human eye need to be further determined, and then a polygon or circular frame is used to mark the area of the human eye. Its potential applications include robot vision, line-of-sight mouse, fatigue driving warning, self-help for the disabled, human-computer interaction, artificial intelligence and many other aspects.

At present, there are endless methods for human eye positioning at home and abroad, and there are roughly four types: knowledge-based, feature-based, template matching or appearance-based human eye positioning methods.

The knowledge-based human eye positioning method is to encode human knowledge about typical eyes into some rules, and use these rules to locate human eyes. These rules mainly include: outline rules, such as the outline of the human eye can be approximated as an ellipse; organ arrangement rules, such as the distribution of human eyes in the upper half of the face in a frontal face; symmetry rules, such as human eyes Symmetry; movement rules, such as eye blinking, can be used to separate the human eye from the background.

The feature-based eye location method is to find some attributes or structural features about the human eye that do not depend on external conditions, and use these attributes or structural features to locate the human eye. First, find these attributes or structural features through a large number of sample learning methods, and then use these attributes or structural features to locate human eyes.

The human eye positioning method of template matching is a classic pattern recognition method. First, a standard human eye template is predefined or parameterized, and then the correlation between the detected image area and the standard template is calculated, and whether it is a human eye is judged by a threshold. Wherein, the human eye template can be dynamically updated.

Appearance-based eye localization methods generally use statistical analysis and machine learning to find relevant characteristics of human and non-human eye images. The learned characteristics are summarized into distribution models or discriminant functions, and then these distribution models or discriminant functions are used to locate human eyes. The theoretical basis of the appearance-based human eye positioning method is probability theory, and the knowledge of probability theory and mathematical statistics is generally used.

Adaboost is an iterative algorithm whose core idea is to train different classifiers (weak classifiers) for the same training set, and then combine these weak classifiers to form a stronger final classifier (strong classifier). The algorithm itself is realized by changing the data distribution. It determines the weight of each sample according to whether the classification of each sample in each training set is correct or not, and the accuracy of the last overall classification. The new data set with modified weights is sent to the lower classifier for training, and finally the classifiers obtained from each training are finally fused as the final decision classifier.

The existing technology is based on the positioning of human eyes by active light. The so-called active light refers to the light beam emitted by an infrared or near-infrared light source and projected onto the surface of the detected target. When shooting human face images under infrared light, when certain conditions are met, the brightness of the pupil of the human eye in the picture is significantly higher than that of the area around the pupil. This phenomenon is called "bright pupil effect". . Using the bright pupil effect under active light, through the difference between different images and image filtering, the candidate area for human eye positioning can be obtained, thereby accelerating the speed of human eye positioning.

Contents of the invention

The object of the invention is to propose a human eye positioning method based on active illumination. This method adopts the method of active light irradiation, through face positioning and human eye positioning, quickly and effectively distinguishes the human eye area from other areas in the image, and realizes real-time positioning of single or multiple human eyes in complex backgrounds. Among them, the tracking, template matching, correlation calculation and filtering algorithms used in the method effectively ensure the accuracy, stability and real-time performance of the human eye positioning algorithm.

The technical solution of the present invention is as follows: in the human eye positioning method based on active light, the active light generation device is used in the video device to actively project the human face, and an image capture device is provided to extract two images of bright pupil and dark pupil; The bright pupil effect caused by light projection means that the candidate area for human eye positioning is obtained through the difference of two images and image filtering method; the human face and human eye positioning methods are subsequently used to complete human eye positioning.

According to the candidate area of human eye positioning, the face area is located through knowledge-based, feature-based, template matching or appearance-based face positioning methods; according to the geometric features of the face, through knowledge-based, feature-based , template matching or appearance-based human eye positioning methods to locate the human eye area.

The method of optimizing human eye positioning uses a tracking algorithm to track the positioned face or human eye position; or uses template matching and calculation-related methods to improve the performance of human eye positioning; or uses filtering methods to improve the accuracy of human face or human eye positioning. performance, you can choose several of the above three types of methods to be used in combination with the basic method of human eye positioning based on active light.

In the stage of face positioning, the filtering algorithm is used to improve the accuracy of face positioning; in the stage of face or human eye positioning, the tracking algorithm is used to speed up the speed of human eye positioning in subsequent frame images; in the stage of human eye positioning, template matching and correlation calculation are used The method improves the accuracy and stability of human eye positioning.

The video device is used in a stereoscopic display device and can be placed in a certain position of the stereoscopic display device.

A band-pass filter is installed on the imaging lens of the image pickup device, and the center frequency of the band-pass filter is equal to or close to the center frequency of the active light source.

Using digital image processing means to analyze the image obtained by the image capture device, and further determine the position of the human eye. The process is shown in Figure 1, mainly including the following aspects:

Acquisition of the human eye candidate area: the human eye candidate area consists of two parts, one part is tracked by the position of the human eye located in the previous frame image; the other part is the difference between the bright pupil and dark pupil images obtained after threshold extraction. The second part usually uses a filtering algorithm to filter out false candidate regions caused by edges or motion. As shown in Figure 2, there are two different images of bright pupil and dark pupil.

Face localization: Face localization methods include knowledge-based, feature-based, template matching or representation-based methods. Taking the feature-based AdaBoost method as an example, several features with better performance are trained to form a cascaded strong classifier. According to the position of the human eye candidate area and the arrangement of face organs, the possible face areas are detected in sequence according to different scales, and the threshold comparison method is used to determine whether the area is a face area. Figure 3 shows several types of Haar features that may be used in the AdaBoost algorithm.

Human eye positioning and optimization: There are also knowledge-based, feature-based, template-matching or image-based methods for human eye positioning. Taking the representation-based support vector machine (SVM) method as an example, first select several Haar-like features as the feature space, and use grid search, cross-validation weighted balance error rate and support vector machine training to obtain support vectors and corresponding weight coefficients , so as to describe the classification hyperplane. Then use the hyperplane to detect the human eye area to be detected to determine whether the area is the human eye area. After the human eye area is located, the position of the human eye area can be optimized by using template matching and inter-frame correlation calculation. An example of a commonly used correlation calculation method is the minimum mean square error algorithm.

Human eye position tracking: According to the human eye position located in the current frame and previous frames, the possible position of the human eye in the next frame can be predicted by using the correlation tracking algorithm. These positions directly perform human eye detection instead of human face detection, thereby saving time for the entire human eye positioning process in the next frame. Examples of commonly used tracking algorithms include Kalman prediction algorithm and Mean-Shift prediction algorithm.

The improvement of the present invention is: compared with the existing human eye positioning method and device, the present invention uses active light illumination to quickly obtain the human eye candidate area, which speeds up the search speed of human eye positioning; The "face-eye" two-level positioning structure saves processing time compared to the method of directly locating human eyes in the image; the filtering algorithm is used in the face positioning stage to improve the accuracy of face positioning, In the positioning stage, the method of template matching and correlation calculation is used to improve the accuracy and stability of human eye positioning; after the positioning of the human face area or human eye area, the possible human eye area in the next frame of image is predicted by tracking algorithm, so as to The human eye positioning of subsequent frames saves time; a band-pass filter is placed on the imaging lens of the image capture device, and the center frequency of the band-pass filter is close to or equal to the center frequency of the active light source, thereby reducing the ambient light The effect of the change on the positioning effect of the human eye.

The beneficial effects of the present invention are: the active light is used to illuminate the detection target, the bright pupil effect under the active light is used to describe the characteristics of the detected target, the main features of the target are described with the least amount of data, and the speed of human eye positioning is accelerated. , It also reduces the impact of ambient light changes on the positioning effect of the human eye. In the image processing stage, various methods such as hierarchical positioning, filtering, tracking, template matching and correlation calculation optimization are used to improve the accuracy and stability of human eye positioning and ensure the real-time performance of human eye positioning.

Description of drawings

Fig. 1 is a flow chart of the inventive method;

Fig. 2 is bright pupil image and dark pupil image of the present invention;

Fig. 3 is the example of several classes of Haar features that can be used in the image processing of the present invention;

Fig. 4 is a schematic structural diagram of an embodiment of the present invention used in a stereoscopic display device.

Figure 5 is a block diagram of the Kalman one-step predictor.

detailed description

Figure 1 The flow chart of the active illumination human eye positioning algorithm is as follows: the active light-based human eye positioning method uses the active light generation device to actively project the face, and uses the image capture device to extract bright pupil and dark pupil images ; Obtain the candidate area of human eye positioning through the difference of two images and image filtering; according to the candidate area of human eye positioning, locate the person through knowledge-based, feature-based, template matching or appearance-based face positioning methods Face area; according to the geometric features of the face, the human eye area is located by knowledge-based, feature-based, template matching or appearance-based human eye positioning methods; among them, the filtering algorithm is used to improve the human face in the face positioning stage. Accuracy of positioning; use tracking algorithm in the stage of human face or human eye positioning to accelerate the speed of human eye positioning in subsequent frame images; use template matching and correlation calculation methods in the stage of human eye positioning to improve the accuracy and stability of human eye positioning .

In the process of the present invention, the processing is performed after taking or inputting the image, specifically as follows:

A human eye position detection and tracking device applied to a stereoscopic display device. The lower part of the screen adopts an active light generating device, and a camera is used as an image input device at the same time. The distance from the human eye to the screen is generally between 30 cm and 80 cm. When watching the screen, it is generally not lower than the lower edge of the screen and not higher than the upper edge of the 30-degree angle range.

Taking a 17-inch stereoscopic display as an example, the relative layout dimensions of the components are shown in FIG. 4 . The 17-inch LCD panel is 338 mm long and 268 mm wide. Plus the surrounding frame is about 420 mm long and 389 mm wide. When watching a screen, people's eyes are usually located in the front and upper position of the screen. Although they often move up and down or left and right between ABCD, they generally do not exceed a certain range, which also determines the active light. The typical size is marked on the map (it is normal to not be able to experience the effect of stereoscopic TV beyond this range).

The distance from the face to the screen is generally between 30 cm and 80 cm, such as point D, when viewing the screen, it will generally not be lower than the lower edge of the screen, and not higher than the upper edge of the range of 30 degrees, as shown in Figure 4 shown.

In the input image, the bright pupil image and the dark pupil image are firstly differentiated, and the point with a relatively large difference value may correspond to the human eye area. In order to avoid the situation where the difference value caused by motion or edge is relatively large, it is necessary to perform filtering processing according to the aggregation degree and shape of points with relatively large difference value, so as to reduce the number of possible human eye areas, thereby saving money for subsequent face and human eye positioning work. processing time. Bright pupil images and dark pupil images are shown in Figure 2.

After the human eye candidate area is obtained, face detection can be performed using knowledge-based, feature-based, template matching, or appearance-based methods. Taking the feature-based AdaBoost algorithm as an example, through the two types of face and non-face The sample library is used for machine learning, and some Haar features with better performance in distinguishing positive and negative sample libraries are found. Set the training sample set S={(x_1,y_1),(x_2,y_2),...,(x_m,y_m)}, initialize the weight assigned to each sample, and then use all weak classifiers in the weak classifier space H To classify the samples, multiply the classification results with the weights and add them together, select the weak classifier h_1 with the best effect, change the sample weights according to the classification results, and increase the weights of misclassified samples, and then repeat the above steps to start from the weak classifier space Select the weak classifier h_2 with the best prediction effect, and repeat N times to obtain N weak classifiers. Each weak classifier is also assigned a weight, the weak classifier with good classification effect is assigned a large weight, and the weak classifier with poor classification effect is assigned a small weight. The final classification result of the strong classifier is the result of N weak classifiers voting and classifying according to their respective weights.

According to the position of the human eye candidate area and the organ arrangement of the face, the obtained strong classifier is used to perform variable-scale face detection on the image to be detected, so as to locate the face area in the image. Several Haar-like features used in machine learning are shown in Figure 3.

After locating the face area, human eye detection can be performed using knowledge-based, feature-based, template matching or appearance-based methods. Taking the appearance-based support vector machine (SVM) algorithm as an example, firstly, through the human eye Two types of sample databases, non-human eyes, and non-human eyes are used for machine learning, and several Haar-like features are obtained to form the feature space of the support vector machine algorithm. The purpose of selection is to select a small number of representative features from the Haar feature space, thereby simplifying the calculation. We use the formula Select a small number of features with the best classification performance from the Haar features to construct the sample vector. In the above formula, Indicates the probability density of the i-th feature taking the value x in the positive sample set, is the corresponding weight, with are the representations in the negative sample set, respectively. The smaller F(i) is, the stronger the ability of the i-th feature to distinguish between positive and negative samples.

After obtaining a series of Haar features through the above criteria, we use the normalized eigenvalues of these features to form a vector, and project the training samples into this space as the training space of SVM. And use the LIBSVM library for training, which is widely used, so the specific training process will not be introduced here.

After the human eye area is obtained, the located human eye area is optimized to enhance the accuracy and stability of human eye positioning. The specific method is: first store the human eye detection images of the first few frames as templates, use template matching method around the detection position of this frame, and select the area with the highest matching degree as the human eye area, so as to reduce the granularity of the scale change of the window to be detected The error caused by too large can improve the accuracy of human eye positioning.

For the jump of human eye coordinates caused by external interference, a filtering method based on face displacement correlation is used to improve the jump. In a video with 25 frames per second, the difference between frames is very small. Even if the posture and expression of the face change, in the case of 1/25s, the relative coordinates will not change too much. We obtain the displacement of the human face and compare it with the displacement of the human eye. If the difference between the displacement of the human eye and the displacement of the human face is too large, it is considered that a jump has occurred and a jump correction is performed. The displacement of the face is obtained by a three-step search method based on template matching.

We obtain the offset of the face through a three-step search method. The accuracy of template matching is low, but the stability is high, no mutation occurs, and it can well reflect the trajectory of human eyes. In video frames without eye jumps, the trajectory obtained by template matching and the trajectory obtained by human eye positioning are highly coincident. When a jump occurs, the trajectories of the two become inconsistent, and we can monitor the correlation between the two to determine whether a jump has occurred. If a jump occurs, the detection coordinates of the last human eye plus the offset of motion estimation are used as the detection value.

After locating the human eye area, according to the human eye position in the current frame image and the human eye position in the previous several frame images, the tracking algorithm is used to predict the position of the human eye in the next frame image. Taking the Kalman predictive tracking algorithm as an example, the possible areas of the human eyes in the next frame of images are predicted. These areas are no longer subject to face detection, but directly to human eye detection, thus speeding up the process of human eye positioning in the next frame of images .

The specific method is as follows: select the uniform acceleration moving model, collect the position change sequence of the human eye, and select the appropriate parameters of the Kalman algorithm equation for the sample. The calculation process of Kalman filter is as follows:

From the above assumptions, the recursive process of Kalman prediction can be obtained:

1. At time t=k-1, calculate

2. Calculate the covariance matrix of the forecast error

3. Calculate the gain matrix

4. Calculate the estimated value of the state at the current moment:

5. Calculate the estimated error P(k|k)=(I-K(k)C(k))P(k|k-1);

At the next moment, repeat operations 1-5. A block diagram of this process is shown in Figure 5.

In the application, the Kalman algorithm will be based on the known position sequence, according to the new data and the estimated value of the parameters at the previous moment, with the help of the state transition equation of the system itself, and according to a set of recursive formulas, the new position can be calculated. parameter estimates. The possible areas of human eyes in the next frame of images are predicted, and these areas are directly detected by human eyes, thus speeding up the positioning process of the next frame of images.

In the present invention, a band-pass filter is arranged on the imaging lens of the image pickup device, and the center frequency of the band-pass filter is equal to or close to the center frequency of the active light source.

The embodiments of the present invention do not constitute a limitation of the present invention, and simple improvements or equivalent solutions based on the principle of the present invention do not exceed the protection scope of the present invention.

Claims (1)

1. based on active light human-eye positioning method, it is characterized in that in video-unit using active light generating device to face actively Projection, is provided with image-pickup device and bright pupil, dark pupil two field picture is extracted；The bright pupil effect triggered using active light projection The candidate region that human eye is positioned is obtained by the difference and image filtering method of two field picture；It is follow-up fixed using face, human eye Position method completes human eye positioning；According to human eye position candidate region, by Knowledge based engineering, feature based, template matches Or the method for Face detection based on presentation orient human face region；According to the geometric properties of face, by Knowledge based engineering, The method of feature based, template matches or based on presentation human eye positioning orients human eye area；

The method of optimization human eye positioning is tracked using track algorithm to the face or position of human eye oriented, or uses template The matching method related to calculating improves the performance of human eye positioning；Or the property of face or human eye positioning is improved using filtering method Can, select the several basic skills positioned with the human eye based on active light in above-mentioned three classes method to be applied in combination；

The accuracy of Face detection is improved using filtering algorithm in the Face detection stage；Face or human eye positioning stage use with Speed of the track algorithm to accelerate follow-up two field picture human eye to position；Template matches, the side of correlation computations are used in human eye positioning stage Method improves the accuracy and stability of human eye positioning；

Video-unit is used for 3 d display device, is placed in a certain position of 3 d display device；

Bandpass filter, centre frequency and the active radiant of bandpass filter are installed on the imaging lens of image-pickup device Centre frequency it is equal or close；

Using the means of Digital Image Processing, the image that image-pickup device is obtained is analyzed, further determines that human eye position Put, step is：

1）The acquisition of human eye candidate region：Human eye candidate region is made up of two parts, and a part is positioned by previous frame image What the position of human eye for arriving was tracked and come；Part II is extracted using threshold value after carrying out difference to bright pupil, two kinds of images of dark pupil Obtain；Wherein Part II is filtered due to the pseudo- candidate region that edge or motion etc. are caused using filtering algorithm；

2）Face detection and filtering：The AdaBoost methods of the method feature based of Face detection, training obtains some feature groups Into cascade of strong classifiers；The arrangement of position and human face according to human eye candidate region, successively according to different scale to possible Human face region detected that the method for using threshold value to compare determines whether the region is human face region；Orient human face region Afterwards, using the position of Kalman filter algorithm optimization human face region；

3）Human eye is positioned and optimized：The method of human eye positioning is based on the SVMs of presentation（SVM）Method, chooses some first Like-Fenton Oxidation is trained as feature space using grid search and cross validation weighted balance error rate and SVMs Method obtains supporting vector and respective weights coefficient, so as to depict Optimal Separating Hyperplane；Then using the hyperplane to be detected Human eye area detected, determine whether the region is human eye area；After orienting human eye area, using template matches and frame Between correlation computations method optimize human eye area position；Conventional Related Computational Methods are lms algorithm；

4）Position of human eye is tracked：According to the position of human eye that present frame and preceding some frame alignment are arrived, using correlation tracking algorithm, prediction Obtain the possible position of human eye in next frame；These positions directly carry out human eye detection, no longer carry out Face datection, so that under being Carrying out for the whole human eye position fixing process of one frame is time-consuming；Track algorithm has Kalman prediction algorithms, Mean-Shift to calculate in advance Method.