CN105224935A - A kind of real-time face key point localization method based on Android platform - Google Patents

Abstract

The invention discloses a real-time face key point positioning method based on an Android platform, belonging to the technical field of computer vision. The method of the present invention comprises the following steps: collecting a human face training picture set, and demarcating key points; randomly selecting n samples in the training set as the initial shape of each training sample; calculating the standardization target of each training sample; extracting each key point Shape index feature; use correlation analysis method to select appropriate amount of features; use two-layer enhanced regression structure (outer layer and inner layer); calculate the regressor of each stage; use face detection method to estimate face window, according to training The regression model predicts the location of facial key points. The current existing methods have high computational complexity and run too slowly on mobile platforms; they are also sensitive to noise and have low positioning accuracy. The invention uses the linear combination of samples to constrain the shape, and uses a method based on regression to improve the accuracy and efficiency of face key point positioning.

Description

A real-time face key point location method based on Android platform

technical field

The invention relates to the technical field of computer vision, and relates to a method for locating key points of a human face.

Background technique

As the key technology of computer vision research, face detection and key point positioning technology have been widely used in intelligent monitoring, identity recognition, expression analysis and other aspects. Face key point positioning refers to the precise positioning of specific facial organs such as eyes, mouth, and nose in the face image, and obtaining their geometric parameters, so as to provide accurate information for research such as expression analysis or face recognition.

At present, the location of face key points has been well realized on the computer, and the real-time and accuracy are very high. Representative works include Active Shape Model (ActiveShapeModel, ASM), Bayesian Tangent Shape Model (BayesianTangentShapeModel , BTSM), active appearance model (ActiveAppearanceModel, AAM), constrained local model (ConstrainedLocalModel, CLM) and so on. However, the existing algorithms have high computational complexity and are rarely used on mobile platforms with limited computing and storage capabilities.

Today, there are hundreds of millions of users of smart terminals (mainly mobile phones and tablet computers) in China, and smart terminals have gradually become people's information carriers. The performance of the mobile platform has also been greatly improved, which provides the possibility for real-time positioning and tracking of facial feature points on the mobile platform. However, the existing facial feature point positioning and tracking algorithms generally have high computational complexity, large memory consumption, and slow processing speed, so it is difficult to directly transplant them to mobile platforms.

Contents of the invention

In order to solve the above problems, the present invention discloses a high-efficiency, high-precision "explicit shape regression" face key point positioning method, which predicts the key points of the entire face by directly learning a vector regression function. Intrinsic shape constraints are naturally encoded into our cascaded learning framework and applied coarse-to-fine during testing. The present invention strives to comprehensively improve the performance of the algorithm in many aspects, which is specifically reflected in the relatively large improvement in feature selection and model training, which greatly improves the operating efficiency of the algorithm, and at the same time ensures the accuracy of positioning, and can be realized on the mobile platform. Real-time detection and positioning of facial feature points.

The present invention is divided into two stages of training and testing. The training phase mainly learns the regression model. Unlike most regression methods, this method does not use a shape model with fixed parameters. The present invention predicts the key points of the whole face by directly learning a vector regression function, and then explicitly minimizes the positioning error on the training set.

To achieve efficient regression, we use simple pixel-difference features, which are the intensity differences of two pixels in an image. The computational complexity of such a feature is very small, that is, according to the position of the key point and an offset, the pixel value of the position is obtained, and then the difference between two such pixels is calculated to obtain the shape index feature. This method uses local coordinates instead of global coordinates, which greatly enhances the robustness of features.

In order to keep the pixel difference feature geometrically invariant, the pixel points are indexed by the estimated shape. Here, the local coordinates are used to index the pixel Δ ^l = (Δx ^l , Δy ^l ), and l is a key point on the normalized face. Such an indexing method can maintain invariance to changes in scale, rotation, etc. and make the algorithm more robust. At the same time, this index can also help us enumerate more useful features around some static points (such as the point at the center of the eye is darker than the tip of the nose, but the center of the two eyes is similar). In the actual implementation process, we convert the local coordinate system back to the global coordinate system of the original image to obtain the shape index pixel, and then calculate the pixel difference feature. Such an approach makes the test phase run much faster. Let S be the estimated shape of a sample. Then the position of the m-th key point is obtained by π _l οS, where π _l gets the x, y coordinates of the m-th key point from the shape vector. Δ ^l is the local coordinate system, then the corresponding global coordinate system on the original image can be expressed as ^Δl is exactly the same for different samples, but the global coordinate system used to extract the underlying pixel points will be adjusted accordingly to ensure geometric invariance.

The present invention uses a two-level enhanced regressor, that is, the first level is 10 levels, and the second level is 500 levels. In this secondary structure, each node in the first level is a cascade of 500 weak classifiers, which is a second-level regressor. In the second layer of regressors, the features are held constant, while in the first layer, the features are varied. In the first layer, the output of each node is the input of the previous node. They are all features taken at the key points estimated at the previous level.

The present invention adopts a two-layer enhanced regression model, that is, the basic human face key point positioning framework (outer layer) and the cascade model of stage regressor R ^t (inner layer). The inner layer stage regressor is also called the original regressor. A key difference between outer-layer and inner-layer regressors is that shape-indexed features are fixed in inner-layer regression. That is, the features only correspond to the shape S ^t-1 of the previous estimate and remain unchanged until the current original regressor is learned. Keeping the shape-indexed features in inner-layer regression constant can make the training phase more time-saving and make the learned inner-layer regressors more stable. In the present invention, the structure of the regressor can be described as T=10, K=500, that is, the first layer (outer layer) has 10 levels, and the second layer (inner layer) has 500 levels. Each node in the first layer is a cascade of 500 weak classifiers, which is the second layer regressor. The input of each node in the first layer is the output of the previous node. The second layer of regressors adopts the so-called fern structure. Each fern is a combination of 5 features and a threshold, which divides the feature space into 2 ^F bins.

Each binb corresponds to a regression output y _b : Ω _b represents the sample in the bth bin. The optimal solution of y _b is In order to prevent overfitting during training, a shrinkage factor is introduced When the number of samples in the bin is large enough, beta has less influence, otherwise it will have a lower estimated value.

Original regressor: The present invention uses fern to represent the original regressor. A fern is a combination of F features and thresholds, and divides the feature space and all training samples into 2 ^F bins. Each bin is closely related to a regression output y _b . Represents the regression target of the training sample, then the output of a bin is to minimize the mean square distance from all training samples falling into this bin to the regression target: Ω _b is the sample in the bth bin. Its optimal solution is to take the mean of all regression targets in the bin: In order to avoid overfitting, a shrinkage factor β is introduced: It can be seen that when the number of training samples is large enough, the shrinkage factor β has little effect, otherwise β will reduce the magnitude of the estimate.

The complete method operation process of this patent is as follows:

Step 1. Collect the training picture set, and calibrate the key points of the face pictures in the training set;

Step 2. Calculate the normalization target for each training sample: MοS is the standardized target sample;

Step 3. Combining training sample sets Each sample contains a training image, a real shape and an initial shape; randomly select n samples in the training set as the initial shape of each training sample;

Step 4, extract shape index features according to the position of each key point;

Step 5, using a correlation analysis method to select an appropriate amount of features as the final feature;

Step 6. Set the regression structure: use the 2-level enhanced regression structure as the basic face registration framework (external layer), and set the internal stage regressor R ^t (internal layer) in each external layer;

Step 7. Calculate the regressor R ^t of each stage:

Step 8. Use the method of face detection to estimate the face window; predict the position of key points of the face according to the trained regression model:

Description of drawings

Fig. 1 is the basic flowchart diagram of the real-time face key point location method based on the Android platform of the present invention.

detailed description

The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings. The invention discloses a real-time face key point positioning method based on an Android platform.

As shown in Figure 1, the present invention can be divided into two stages of learning positioning and testing positioning, specifically, comprising the following steps:

Step 1, labeling and standardization of training set feature points: the face calibration points involved in the present invention are selected according to the MPEG-4 standard, because the 84 feature points defined by the face definition parameter FDP are too complicated, so we selected them as needed Of the 68 feature points, only eight feature points on the eye contour (that is, four points on the upper, lower, left, and right sides of the eyes) are reserved to improve computing efficiency. The training library is taken from MUCT and self-calibration face images. There are about 4,000 face images in total. These face images are processed with mirror symmetry, and finally a training set with 8,000 face images is obtained.

Step 2. Standardize the shape of the sample image: Since the pose of the face shape in the image is described by the size scale, rotation angle, and position parameters, it is necessary to appropriately change the pose parameters of the shape, including size, angle, and position. It is possible to make all images consistent. For example, scale transformation can make the distance between the calibration points fixed, rotation transformation can make the alignment of the calibration points fixed, and displacement transformation can make the center position of the shape fixed. First we find the average face of all training samples Then minimize the input sample S with L2 distance between MοS is the standardized shape

Step 3. Combining training sample sets Each sample contains a training image, a ground truth shape and an initial shape. Randomly select 10 samples in the training set as the initial shape of each training sample. This method of generating multiple initial shapes for a picture invisibly increases the number of training samples and also improves the generalization of training.

Step 4. Establish a local coordinate system Δ ^l = (Δx ^l , Δy ^l ) on a specific point on the standardized human face. For different training samples, the local coordinate system ^Δl is the same. Index pixels according to the position of the key point in the local coordinate system. The superscript l indicates the specific label point corresponding to the pixel point. Then transform the local coordinate system back to the original image global coordinate system to extract pixel difference features π _l means extracting the x, y coordinates of the mth label point from the shape vector. The features of each weak learner R ^t are based on the image I and the previous estimated shape S ^t-1 . In each stage regressor R ^t , randomly generate P local coordinate systems That is, P shape index pixel points are defined. First, each local coordinate system is generated by randomly selecting a label point (the _lαth label point), and then the X- and Y-offsets are randomly generated in a uniform distribution. A total of P pixels will

Generate P ² pixel difference features.

Step 5, using a correlation analysis method to select an appropriate amount of features as the final features.

After generating P ² features, it brings a new challenge - to quickly select effective features in a huge feature space. "Appropriate amount" means to select the number of features that is much smaller ^than P2, which is beneficial to reduce computational complexity and improve the efficiency of the algorithm. The present invention selects F/P ² features to establish a good fern regressor. We exploit the correlation between features and regression targets to select features. There are two principles for selecting features: maintain a high correlation between each feature in fern and the regression target; maintain a low correlation between features and features in a complementary relationship. Use Y to represent the regression target, which is a matrix with N (number of samples) rows and N _fp (number of labeled points) columns. X represents the pixel difference feature, which is a matrix with N rows and P ² columns. Our goal is to select out of the P ² columns of the X matrix the F columns that are highly correlated with the Y matrix. Since Y is a matrix, we project Y onto a column vector Y _prob with a column vector projection υ obtained from a unit Gaussian: Y _prob = Yυ. The most relevant (PearsonCorrelation) features to the projection target can be selected: By repeating F times with multiple projections, F suitable features can be obtained. These F features are the right amount of features to be selected. The correlation calculation corr(Y _proj ,ρ _m -ρ _n ) between the regression target and the pixel difference feature is designed as follows: $corr(Y_{proj},{\mathrm{\ρ}}_m-{\mathrm{\ρ}}_{\mathrm{no}})=\frac{\mathrm{cov}(Y_{proj},{\mathrm{\ρ}}_m)-\mathrm{cov}(Y_{proj},{\mathrm{\ρ}}_{\mathrm{no}})}{\sqrt{\mathrm{\σ}\left(Y_{proj}\right)\mathrm{\σ}({\mathrm{\ρ}}_m-{\mathrm{\ρ}}_{\mathrm{no}})}},$ in

σ(ρ _m −ρ _n )=cov(ρ _m ,ρ _m )+cov(ρ _n ,ρ _n )−2cov(ρ _m ,ρ _n ). The calculation of correlation here consists of two parts: target-pixel covariance and pixel-pixel covariance. Since the shape-indexed features are fixed in inner-layer cascaded regression, the pixel-pixel covariance can be computed in advance and reused in each inner layer. For the original regressor, we only need to calculate all the target-pixel covariances to form the correlation, where the correlation is linear with the number of pixel features. Therefore, the correlation calculation complexity is reduced from O(NP ² ) to O(NP).

Step 6. Set the regression structure: use the 2-level enhanced regression structure as the basic face registration framework (external layer), and set the internal stage regressor R ^t (internal layer) in each external layer. The weak learners in the inner layers are called primitive regressors. Here the inner and outer regressors share some similarities, but they differ in that all stages of the inner layer regress where the shape index features are kept constant.

Step 7. Learning the internal cascade regression, the internal cascade regression includes K original regressors {r ₁ ,...,r _K }, ie ferns. These primitive regressors in turn greedily fit the regression objective. Each original regressor processes the residue left over from the previous regressor. At each iteration, these residuals are used as targets for learning a new regressor. Compute the regressor R ^t for each stage:

Step 8. Use the method of face detection to estimate the face window; predict the position of key points of the face according to the trained regression model:

Claims (1)

1. a real-time face key point localization method based on Android platform, is characterized in that constructing a non-parametric shape model based on regression, predicts the key point of whole face by directly learning a vector regression function, comprising the following steps: Step 1. Collect the training picture set, and calibrate the key points of the face pictures in the training set; Step 2. Calculate the normalization target for each training sample: That is, the standardized target sample; Step 3. Combining training sample sets Each sample contains a training image, a real shape and an initial shape; randomly select n samples in the training set as the initial shape of each training sample; Step 4, extract shape index features according to the position of each key point; Step 5, using a correlation analysis method to select an appropriate amount of features as the final feature; Step 6. Set the regression structure: use the 2-level enhanced regression structure as the basic face registration framework (external layer), and set the internal stage regressor R ^t (internal layer) in each external layer; Step 7. Calculate the regressor R ^t of each stage: Step 8. Use the method of face detection to estimate the face window; predict the position of key points of the face according to the trained regression model: