CN107644203B - Feature point detection method for shape adaptive classification - Google Patents
Feature point detection method for shape adaptive classification Download PDFInfo
- Publication number
- CN107644203B CN107644203B CN201710815514.6A CN201710815514A CN107644203B CN 107644203 B CN107644203 B CN 107644203B CN 201710815514 A CN201710815514 A CN 201710815514A CN 107644203 B CN107644203 B CN 107644203B
- Authority
- CN
- China
- Prior art keywords
- face
- shape
- human face
- classification
- std
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 19
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 104
- 238000012549 training Methods 0.000 claims abstract description 50
- 230000008569 process Effects 0.000 claims abstract description 34
- 238000012360 testing method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 8
- 230000001815 facial effect Effects 0.000 claims description 6
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims description 4
- 238000000513 principal component analysis Methods 0.000 claims 5
- 230000036544 posture Effects 0.000 abstract description 23
- 238000004422 calculation algorithm Methods 0.000 abstract description 18
- 230000008859 change Effects 0.000 abstract description 3
- 238000002474 experimental method Methods 0.000 abstract description 2
- 238000007635 classification algorithm Methods 0.000 abstract 1
- 230000010354 integration Effects 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000985694 Polypodiopsida Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a feature point detection method for shape adaptive classification. Firstly, the multi-view model is used for correspondingly processing different human face postures, and a posture classification algorithm is used for classifying different human face samples in the multi-view model training and testing process based on cascade regression. Secondly, in the testing process, according to the characteristics of the cascade regression algorithm, a dynamic human face posture classification method is adopted, the classification accuracy is gradually improved, and therefore the accuracy of the human face feature point positioning algorithm is improved. Meanwhile, in order to further reduce the error of feature point positioning, the invention uses a plurality of multi-view model integration strategies, and in the test process, a plurality of multi-view models are simultaneously used for predicting the feature point positions. Experiments prove that compared with the traditional method, the method has better robustness on the apparent change of the human face in the non-limited environment.
Description
Technical Field
The invention relates to a human face feature point detection method based on shape self-adaptive classification, and belongs to the field of human face recognition.
Background
The face feature point positioning technology is a basic function of the human visual system, and plays an important role in a face algorithm. Not only because it can help us to deepen our cognition of the human visual system, but it also has great commercial potential. Over the last decade, there has been a tremendous growth in methods for locating facial feature points in digital images and videos, especially in controlled environments. Many public and commercial face feature point localization algorithms have been well developed and widely used in practical applications. Such as video surveillance, visit surveillance, information forensics, network-based social networking, human-computer interaction, animation, and 3D modeling.
In recent years, with the development of portable video cameras and video equipment, the trend of face feature point positioning algorithm is turning to an uncontrolled environment. In order to adapt the face feature point positioning algorithm to the existing image shooting environment, a feature point positioning algorithm which can be robust to the face image in the non-limited environment is urgently needed. However, the development of the face feature point algorithm faces a great challenge. Due to the rich variation of facial expressions, especially in non-controlled environments, a large number of different facial poses, expressions, shades, and partial occlusions may occur. These conditions have a great influence on the accuracy of the face feature point algorithm. Therefore, a human face feature point algorithm which can be suitable for various environments, multiple expressions and multiple postures is needed to improve the robustness and efficiency of the existing algorithm.
Assuming a face shape
Comprising NfpPersonal facial feature points. When a new face picture is given, the algorithm has the non-main objective of predicting the shape S, so that the predicted shape is as close to the real shape as possible
Equivalent to minimizing the following equation:
the algorithmic error formula is typically used to guide the training process and to evaluate the final experimental results. But we cannot directly minimize this formula during the testing phase. Because of the true shape at the time of testing
Is unknown. Most algorithms can be classified into the following two methods according to the method used for predicting the shape S: optimization-based methods and regression-based methods.
The quality of the optimization-based method mainly depends on the quality of an error equation. The method mainly comprises the following two algorithms: AAM and ASM. Both methods mainly use the approach of minimizing texture residue to predict the shape of the face feature points. Because the model trained by the method is limited by expressive force, when the human face pose in the image is greatly changed, the positioning result of the algorithm is not very accurate.
Regression-based face feature point detection methods have recently been rapidly developed. The algorithm model mainly comprises two parts: feature extraction and regressor. First, in each of the cascading steps, feature values are extracted from around the already predicted shape. And gradually updating the existing shape according to the learned regression matrix to enable the existing shape to approach the real shape step by step. After several steps, the prediction error converges to an absolutely small value. Regression-based methods can be developed because of their high efficiency and accuracy. And the reason for the success of the method is that weak regressors are cascaded to form strong regressors. Regression-based methods have much faster speed and accuracy than AAM and ASM methods.
Although in most cases regression-based methods are relatively fast and highly accurate. For example ESR method, 3000fps method, proposed by Cao. However, feature point detection still faces many challenges. Such as large facial attitude deflection, partial occlusion, and changes in darkness. Because the shooting environment of most of current photos is mostly uncontrollable, when the human face in the picture deflects left and right or changes of expression, a single model cannot well process the changes. Therefore, different models are needed to correspond to different face shape changes. Therefore, different models are used under different conditions, the algorithm effect can be improved, and the operation speed of the algorithm cannot be influenced.
The human face feature point detection method based on shape self-adaptive classification comprises the following key elements of PCA attitude classification, cascade regression structure, cascade regression classification and multi-model structure building. However, because a large number of training samples are needed in the process of training the multi-model, the original samples can be expanded in the training process, the diversity of the shapes of the samples is increased, and the multi-model structure can have better robustness to different shapes.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the defects in the prior art, the invention provides the face feature point detection method with the shape self-adaptive classification, which utilizes the characteristic that the cascade regression model gradually updates the feature point positions, so that the classification of the shapes can be gradually updated, the classification effect is improved, and the face shape classification result is more accurate.
The technical scheme is as follows: in order to solve the above technical problems, the method for detecting human face feature points by shape adaptive classification of the present invention comprises the following steps:
(a) randomly selecting pictures with a certain proportion from a training set, then carrying out mirror image processing on the pictures, and then carrying out left-right rotation of random angles on the pictures to finally obtain an expanded human face characteristic point library;
(b) the obtained training set is divided into three classes using the PCA face pose classification method, wherein the standard deviation of the shape of the training face is divided into three intervals of [ - ∞, -std ], [ -std, -std ], [ std, + ∞ ], thereby dividing the shape of the face into three classes. Then, a multi-view model structure is obtained by using a training method of a cascade regression model;
(c) repeating the steps (a) and (b) three times to obtain a multi-view integrated model structure;
(d) acquiring human face image samples of detection personnel through a camera, and forming a test set;
(e) positioning the face position in the face image sample, and marking the face position;
(f) positioning the initial position of the face characteristic point by using an initial prediction model, wherein the initial prediction model is obtained by pre-training in the training process;
(g) and classifying the initial human face characteristic point shape by using a PCA human face posture classification method, wherein the classification method comprises the following steps: standardizing the initial human face shape, and forming three intervals of (-infinity, std ], [ -std, std ], [ std, + ∞ ] by using the standardized standard deviation so as to divide the human face shape into three classes;
(h) putting the classified human face shapes into a multi-view regression model corresponding to the classified human face shapes to perform accurate human face characteristic point prediction;
(i) and the classified human face shape obtains a new human face shape update in a first strong regressor, and the new human face shape is classified again, namely, a dynamic human face posture classification method is used for carrying out new shape classification on the human face posture shape. After each strong regressor completes the prediction, a PCA face pose classification method is used to classify the new face feature point shape. The system comprises a strong regressor, a plurality of weak regressors, a plurality of face databases and a plurality of face databases, wherein the strong regressors are composed of a plurality of weak regressors, the weak regressors are obtained through random ferns, the number of the strong regressors is obtained through experimental data of different face databases, and the number of the preferred strong regressors is ten;
(j) classifying the face shape obtained in the last step for the step, and putting the classified face posture shape into a next strong regressor corresponding to the classified face posture shape for further prediction calculation so as to obtain new face shape update again;
(k) repeating the steps (i) and (j) until all the strong regressors finish the prediction, and obtaining the accurate coordinates of the human face characteristic points after all the strong regressors finish the prediction;
(l) Because the multi-view integrated model training method is used in the model training process, a multi-view integrated model testing method is required in the testing process, namely, the average value of a plurality of testing results is obtained to obtain the final prediction result. Step k is a prediction process of one of the face shapes. Classifying initial human face shapes before prediction, and selecting corresponding multi-view models from the classified human face shapes, wherein the multi-view models consist of a plurality of regression models, and each different regression model correspondingly processes different human face shapes; and (5) repeating the step (k) three times to obtain the result, and then averaging to obtain a more accurate predicted value.
Preferably, the step (g) comprises the following steps:
(g1) assuming the initial face shape obtained by step f
S contains N feature points, (x)i,yi) Representing the coordinates of the characteristic points in the image, and carrying out normalization processing on the initial face shape, wherein the normalization method comprises the following steps:
(g2) expressing the normalized human face feature point shape as
Carrying out zero-averaging on the normalized result to enable the shape distribution to be closer to normal distribution;
(g3) after the above steps, a new matrix a ═ Δ S is formed1,ΔS2,...,ΔSN-1,ΔSN]TWherein A is the composition of all the feature points of the image normalization;
(g4) and (3) calculating the standard deviation std of the matrix A, wherein the standard deviation constitutes three intervals of [ infinity, -std ], [ -std, std ], [ std, + ∞ ], thereby dividing the human face shapes into three classes, and experiments show that the effect of dividing into three classes is the best, because the human face postures are classified accurately under the condition of dividing into three classes. Since the PCA classification method we use retains the most important face shape information that controls the left and right angular deflection of the face, we use three classifications in the following calculations.
Preferably, the step (i) includes the steps of:
(i1) firstly, acquiring a primary face feature point position in the step b, and performing primary classification by using PCA face posture classification;
(i2) in the next step, a two-layer cascade regression structure is used, the structure is composed of a plurality of strong regressors, and the prediction process of the strong regressors is as the formula:
wherein is at
As a result of the prediction in the t-1 stage,
for a normalized shape matrix, RtIs a strong regressor of the t stage, IiIs the ith human face image. The regressor is referred to as a trained regressor, because the structure of the regressor is cascaded and sequential, and should be used sequentially, it cannot be any regressor. After each strong regressor completes the calculation, the pairs
The classification is performed again, the classification method still uses the PCA face pose classification method mentioned in g.
Preferably, the step (l) includes the following steps:
(l1) since the training process obtains a cascade structure of three sets of multiple models, a prediction method of multiple models will be used in the test process. Firstly, different models are loaded into equipment, a plurality of different models are circularly used for predicting a face image, and a plurality of feature point results h are obtainedi；
(l2) averaging the positioning results of the plurality of feature points to obtain the final detection result:
H＝(h1+h2+h3)/3 (0.4)
preferably, training of a multi-model structure requires a large number of face image samples with different poses, but the number of training samples in the existing data set is small, and the included face poses are not rich enough. The method comprises the steps of randomly selecting a certain proportion of original samples from training samples, carrying out mirror image and random angle rotation on the samples, and expanding the number of the samples to 4 times. Meanwhile, the diversity of sample deflection is increased through rotation, and the robustness of the model is increased.
Preferably, in the step (b), a simple shape index feature is used, and the feature only needs to calculate the pixel difference between different pixel points. The method is simple in calculation, and useful features are selected from the obtained multiple pixel differences to serve as finally calculated features. Meanwhile, local coordinates are used in the calculation process, the invariance of the face shape is kept, and the robustness of the selected features is improved.
In the invention, a large number of human face image samples with different postures are required for training of a multi-model structure, but the training samples of the existing data set are fewer, and the contained human face postures are not rich enough. The method comprises the steps that an original sample with a certain proportion is randomly selected from training samples, the original sample can be an existing sample, and after a training set is obtained, pictures with a certain proportion are randomly selected from the training set. And then, the pictures are subjected to mirror image processing, so that a matrix formed by all the human face characteristic points basically meets normal distribution, and convenience is provided for subsequent steps. And the face pictures are randomly rotated left and right, so that the diversity of the samples is increased to a certain extent. When a training set with relatively few human face gestures is used, the random rotation increases the change of the human face gestures, and the processing capability of multiple models on different human face shapes is enhanced.
In the present invention, a simple shape index feature is used. Firstly, randomly selecting a plurality of points around the feature points, pairing the random points one by one, calculating the pixel difference between each pair of points, and taking the difference as a candidate feature. The method is simple in calculation, and useful features are selected from the obtained multiple pixel differences to serve as finally calculated features. Meanwhile, local coordinates are used in the calculation process, and a coordinate system is established by taking the characteristic points as standards. Based on the coordinate system, selecting pixel points around the feature points. Since most useful pixel features are near the feature points, using local coordinates will effectively improve the robustness of selecting the selected features.
A face characteristic point detection method for shape adaptive classification comprises the following steps:
the camera is used for acquiring a face image sample of a detector, and sending the acquired face image sample to a sample storage library to form a test set. The resolution of the camera is similar to that of a common video chat camera, and the resolution of 320 multiplied by 240 can meet the image acquisition requirement;
the human face image detection module acquires a relatively clear human face image through a camera, identifies and positions the human face of the human face image, and stores the position of the positioned human face for use in the next step; according to the obtained face position information, using a pre-trained feature point initialization model to perform feature point initialization positioning on the detected face; normalizing the initialized human face characteristic points, and normalizing the human face shape into delta S ═ delta x1,Δy1,...,ΔxN,ΔyN]TUnifying coordinate points to the same coordinate system;
classifying the normalized human face shape delta S into corresponding classes by using a PCA human face posture classification method, and performing the next step of calculation by using a corresponding regressor; when the corresponding cascade regressors are used for calculation, after each strong regressor finishes one operation, a PCA attitude classification method is sequentially used for carrying out new classification on newly obtained human face characteristic points; because the initial human face shape is not particularly accurate, errors can occur in classification, the positions of the characteristic points can be gradually updated after the cascade regression is operated, and the classification accuracy is improved; because a multi-model cascade structure is used, the steps need to be repeated for multiple times, multiple regression results are obtained, and therefore three results h need to be obtainediThe tie value is calculated by the following method: h ═ H (H)1+h2+h3) And/3, obtaining a final result.
Has the advantages that: the human face characteristic point detection method based on shape self-adaptive classification has the following advantages:
firstly, in the training process, because the existing face feature point training set contains relatively few face gestures, the deflection angle of the face is not large enough. When a human face posture with a larger angle is deflected, the robustness of the cascade regression model trained by the training set is not enough, and the position of the predicted human face feature point is not accurate enough. Therefore, the method correspondingly expands the existing training samples when the cascade regression model is trained. Firstly, randomly selecting a certain proportion of image samples from the existing training set, and carrying out mirror image and random deflection within a certain angle range on the samples. Therefore, the total amount of the selected samples is expanded by four times, and simultaneously, the random deflection is added, so that the change of the human face posture is increased, and the robustness of the trained cascade regression model is improved.
In the process of predicting the human face pose, the original human face feature point positioning method only uses a single cascade regression model to predict all human face shapes. This is not conducive to distinguishing different face poses, shading changes, and local occlusions well. And a single cascade regression model is used for processing various human face shapes, so that the positioning precision of the human face feature points is reduced to a great extent. According to shape information contained in different human face feature point positions, the invention uses PCA human face posture classification to classify different human face shapes. Firstly, the invention classifies the PCA face pose in the training process. In the training process, the human face shapes contained in the existing training set are classified. And using the classified images and coordinate point information for training a cascade regression model. In the characteristic point detection process, the invention uses an initial prediction model to roughly position the positions of the characteristic points of the human face. Then using PCA face pose classification, the face shape is classified. And selecting a corresponding cascade regression model to predict the next step according to the classification result. Using PCA face pose classification, there may be better targeting for different face shapes than traditional methods.
The cascade regression model is composed of a plurality of strong regressors, and each strong regressor updates the shape of the face in the prediction process. Based on this property, the present invention uses the PCA face pose method after each strong regressor. Because the positions of the feature points calculated by the initial prediction model have large errors with the real positions, the face pose classification performed under the condition has high possibility of errors. Therefore, in the prediction process, the newly obtained feature point shape type needs to be updated. The cascade regression model is to update the feature point positions step by step, so the classification of the shapes is also updated step by step. Therefore, the classification effect can be improved, and the face shape classification result is more accurate.
The invention uses a multi-model structure in the whole structure, and the final detection results are multiple. To reduce the error of different models, the invention averages multiple results, i.e., H ═ H1+h2+,...+hn) And/n, obtaining a final feature point positioning result. Compared with the existing similar method, the method has higher accuracy and higher robustness to different human face postures.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of the present invention;
FIG. 2 is a block diagram of a cascade classification architecture of the present invention;
FIG. 3 is a diagram of a multi-model structure training architecture of the present invention;
FIG. 4 is a graph comparing the results of the present invention with those of other prior art methods, comparing the experimental results with those of Fan, Matinez, Deng et al, respectively, on 300-w indoor, outwood and indoor + outwood test set;
FIG. 5 is a graph showing the results of the 300-W test set of the present invention.
Detailed Description
As shown in fig. 1, a method for detecting human face feature points by shape adaptive classification includes the following steps:
a. acquiring a human face image sample of a detected person through a camera, and storing the human face image sample in a png mode;
b. positioning and detecting the face image, marking the position of the face, and storing the face position information
b＝[x,y,w,h]T(detecting the vertex coordinates of the face frame and the length and width thereof);
c. according to the obtained face position, an initial model is used for carrying out initial positioning on the face characteristic points, wherein the initial prediction model is obtained by using a plurality of training samples for training in advance;
d. carrying out primary classification on the obtained primary human face characteristic point shape by using a PCA posture;
the method comprises the following steps:
d1, assuming the initial face shape obtained by step f
(xi,yi) Representing the coordinates of the characteristic points in the image, and carrying out normalization processing on the initial face shape, wherein the normalization method comprises the following steps:
d2, representing normalized human face feature point shape as
Carrying out zero-averaging on the normalized result to enable the shape distribution to be closer to normal distribution;
d3, after the above steps, forming a new matrix a ═ Δ S1,ΔS2,...,ΔSN-1,ΔSN]TWherein A is the composition of all the feature points of the image normalization;
d4, calculating the standard deviation std of the matrix A, wherein the standard deviation constitutes three intervals of [ - ∞, -std ], [ -std, std ], [ std, + ∞ ], thereby dividing the human face shape into three categories. Tests show that the effect of the classification into three categories is the best, so that the three categories are used in the subsequent calculation;
e. putting the classified human faces into corresponding cascade models for further prediction;
f. in the further prediction process, a cascade regression classification method is used for further shape classification of the human face posture shape, so that the classification is more accurate;
the step i comprises the following steps:
f1, firstly, acquiring a primary face feature point position in the step f, and carrying out primary classification by using PCA face posture classification;
f2, using a two-layer cascade regression structure in the next step, wherein the structure is composed of a plurality of strong regressors, and the prediction process of the strong regressors is as the formula:
wherein is at
As a result of the prediction in the t-1 stage,
for a normalized shape matrix, RtIs a strong regressor of the t stage, IiIs the ith human face image.
After each strong regressor completes the calculation, the pairs
And (d) performing classification again, wherein the classification method still uses the PCA face pose classification method mentioned in the step d.
g. Putting the classified human face posture shapes into a next strong regressor to perform next prediction calculation, so as to obtain new prediction shapes;
h. repeating the step (f) and the step (g) until a face feature point position prediction result is obtained;
i. because the multi-model cascade training method is used in the model training process, a multi-model cascade prediction method is needed in the test process, and the average value of a plurality of prediction results is obtained to obtain the final prediction result;
the step i comprises the following steps:
i1, obtaining a cascading structure of three sets of multi-models in the training process, so that a multi-model prediction method is used in the testing process. First of all will notLoading the same model into the equipment, predicting the face image by circularly using various different models to obtain a plurality of feature point results hi；
i2, averaging the positioning results of the plurality of feature points to obtain the final detection result:
H＝(h1+h2+h3)/3 (0.8)
as shown in fig. 3, a method for training human face feature points by shape adaptive classification includes the following steps:
a. randomly selecting a certain proportion of pictures from the training set. Then, carrying out mirror image processing on the pictures, and then carrying out left-right rotation of random angles on the pictures to finally obtain an expanded human face characteristic point library;
b. dividing the obtained training set into three classes by using a PCA face posture classification method for training a multi-model structure;
c. and (b) repeating the steps (a) and (b) three times to obtain the multi-view integrated model.
According to the invention, a plurality of test samples are respectively selected in 300-W indoor outdoor test sets for result comparison, wherein the comparison result is shown in FIG. 5 compared with the corresponding comparison performed by an ESR method.
A large number of test data for different face feature point methods are published on the ibug website. We compared the test results of the present invention with the data of the existing published methods on the ibug website, and a graph was drawn for comparison on a plurality of test data in the present invention, and the comparison results are shown in fig. 4.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (4)
1. A face characteristic point detection method with shape self-adaptive classification is characterized by comprising the following steps:
(a) randomly selecting pictures with a certain proportion from the training set, carrying out mirror image processing on the pictures, and then carrying out left-right rotation of the images at a random angle to finally obtain an expanded human face characteristic point library;
(b) dividing the obtained training set into three classes by using a PCA (principal component analysis) face posture classification method, wherein the standard deviation of the shape of the training face is divided into three intervals of [ - ∞, -std ], [ -std, std ], [ std, + ∞ ], so that the shape of the face is divided into three classes, and then, a training method of a cascade regression model is used to obtain a multi-view model structure;
(c) repeating the steps (a) and (b) three times to obtain a multi-view integrated model structure;
(d) acquiring human face image samples of detection personnel through a camera, and forming a test set;
(e) positioning the face position in the face image sample, and marking the face position;
(f) positioning the initial position of the face characteristic point by using an initial prediction model, wherein the initial prediction model is obtained by pre-training in the training process;
(g) and classifying the initial human face characteristic point shape by using a PCA human face posture classification method, wherein the classification method comprises the following steps: standardizing the initial face shape, and forming three intervals of (-infinity, -std ], [ -std, std ], [ std, + ∞ ] by using the standardized standard deviation std, thereby dividing the face shape into three classes;
(h) putting the classified human face shapes into a multi-view regression model corresponding to the classified human face shapes to perform accurate human face characteristic point prediction;
(i) updating the new face shape obtained from the classified face shape in the first strong regressor, and classifying the new face shape again, namely, performing new shape classification on the face posture shape by using a dynamic face posture classification method, and after each strong regressor finishes prediction, classifying the new face characteristic point shape by using a PCA face posture classification method, wherein the strong regressor consists of a plurality of weak regressors, and the weak regressors are obtained by random fern;
(j) classifying the face shape obtained in the last step for the step, and putting the classified face posture shape into a next strong regressor corresponding to the classified face posture shape for further prediction calculation so as to obtain new face shape update again;
(k) repeating the steps (i) and (j) until all the strong regressors finish the prediction, and obtaining the accurate coordinates of the human face characteristic points after all the strong regressors finish the prediction;
(l) Averaging a plurality of test results to obtain a final prediction result, which specifically comprises:
(l1) because the training process obtains the cascade structure of three sets of multi-models, in the test process, the prediction method using the multi-models is to load different models into the equipment at first, to predict the face image by using various different models circularly, to obtain a plurality of characteristic point results hi；
(l2) averaging the positioning results of the plurality of feature points to obtain the final detection result:
H＝(h1+h2+h3)/3。
2. the method for detecting facial feature points by shape adaptive classification as claimed in claim 1, wherein the step (g) comprises the steps of:
(g1) assuming the initial face shape obtained by step f
(xi,yi) Representing the coordinates of the characteristic points in the image, and carrying out normalization processing on the initial face shape, wherein the normalization method comprises the following steps:
(g2) expressing the normalized human face feature point shape as
Carrying out zero-averaging on the normalized result to enable the shape distribution to be closer to normal distribution;
(g3) after the above steps, a new matrix a ═ Δ S is formed1,ΔS2,...,ΔSN-1,ΔSN]TWherein A is the composition of all the feature points of the image normalization;
(g4) the standard deviation std of the matrix A is found, which constitutes three intervals [ - ∞, -std ], [ -std, std ], [ std, + ∞ ], thus dividing the face shape into three classes.
3. The method for detecting facial feature points by shape adaptive classification as claimed in claim 2, wherein the step (i) comprises the steps of:
(i1) firstly, acquiring a primary face feature point position in the step f, and performing primary classification by using PCA face posture classification;
(i2) in the next step, a cascade regression structure is used, which is composed of a plurality of strong regressors, and the prediction process of the strong regressors is as the formula:
wherein is at
Is the predicted result of the t-1 stage, RtIs a strong regressor of the t stage, IiIs the ith human face image.
4. The method for detecting human face feature points by shape adaptive classification as claimed in claim 1, wherein the initial prediction model in the step (f) is obtained by: randomly extracting a part of pictures in the step (a), and obtaining an initial prediction model by a training method of cascade regression.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710815514.6A CN107644203B (en) | 2017-09-12 | 2017-09-12 | Feature point detection method for shape adaptive classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710815514.6A CN107644203B (en) | 2017-09-12 | 2017-09-12 | Feature point detection method for shape adaptive classification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107644203A CN107644203A (en) | 2018-01-30 |
| CN107644203B true CN107644203B (en) | 2020-08-28 |
Family
ID=61111083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710815514.6A Expired - Fee Related CN107644203B (en) | 2017-09-12 | 2017-09-12 | Feature point detection method for shape adaptive classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107644203B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110838179B (en) * | 2019-09-27 | 2024-01-19 | 深圳市三维人工智能科技有限公司 | Human body modeling method and device based on body measurement data and electronic equipment |
| CN112270308B (en) * | 2020-11-20 | 2021-07-16 | 江南大学 | Face feature point positioning method based on double-layer cascade regression model |
| CN112487993A (en) * | 2020-12-02 | 2021-03-12 | 重庆邮电大学 | Improved cascade regression human face feature point positioning algorithm |
| CN112836904A (en) * | 2021-04-07 | 2021-05-25 | 复旦大学附属中山医院 | Body quality index prediction method based on face characteristic points |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824089A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascade regression-based face 3D pose recognition method |
| CN103824050A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascade regression-based face key point positioning method |
| CN105787448A (en) * | 2016-02-28 | 2016-07-20 | 南京信息工程大学 | Facial shape tracking method based on space-time cascade shape regression |
| CN106529397A (en) * | 2016-09-21 | 2017-03-22 | 中国地质大学（武汉） | Facial feature point positioning method and system in unconstrained environment |
| CN106682598A (en) * | 2016-12-14 | 2017-05-17 | 华南理工大学 | Multi-pose facial feature point detection method based on cascade regression |
-
2017
- 2017-09-12 CN CN201710815514.6A patent/CN107644203B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824089A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascade regression-based face 3D pose recognition method |
| CN103824050A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascade regression-based face key point positioning method |
| CN105787448A (en) * | 2016-02-28 | 2016-07-20 | 南京信息工程大学 | Facial shape tracking method based on space-time cascade shape regression |
| CN106529397A (en) * | 2016-09-21 | 2017-03-22 | 中国地质大学（武汉） | Facial feature point positioning method and system in unconstrained environment |
| CN106682598A (en) * | 2016-12-14 | 2017-05-17 | 华南理工大学 | Multi-pose facial feature point detection method based on cascade regression |
Non-Patent Citations (4)
| Title |
|---|
| Localizing Facial Keypoints with Global Descriptor Search, Neighbour Alignment and Locally Linear Models;Md. Kamrul Hasan et al.;《2013 IEEE International Conference on Computer Vision Workshops》;20131231;第362-369页 * |
| 基于级联回归的多姿态人脸特征点检测算法研究;黄宇驹;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第2期);第8-36页、图2-4、图4-5 * |
| 基于自适应三维人脸模型的实时头部姿态估计;程科文;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第2期);第13-20页 * |
| 级联回归的多姿态人脸配准;伍凯等;《中国图象图形学报》;20170214;第22卷(第2期);第257-264页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107644203A (en) | 2018-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537136B (en) | Pedestrian re-identification method based on attitude normalization image generation | |
| WO2020228446A1 (en) | Model training method and apparatus, and terminal and storage medium | |
| CN108647583B (en) | Face recognition algorithm training method based on multi-target learning | |
| CN109583340B (en) | Video target detection method based on deep learning | |
| CN107644203B (en) | Feature point detection method for shape adaptive classification | |
| CN109977757B (en) | Multi-modal head posture estimation method based on mixed depth regression network | |
| CN109145717B (en) | Face recognition method for online learning | |
| CN105069424B (en) | Quick face recognition system and method | |
| CN111783532B (en) | Cross-age face recognition method based on online learning | |
| CN103810473B (en) | A kind of target identification method of human object based on HMM | |
| CN109145766A (en) | Model training method, device, recognition methods, electronic equipment and storage medium | |
| US20070053590A1 (en) | Image recognition apparatus and its method | |
| CN110598638A (en) | Model training method, face gender prediction method, device and storage medium | |
| CN108805151B (en) | Image classification method based on depth similarity network | |
| CN108595558B (en) | Image annotation method based on data equalization strategy and multi-feature fusion | |
| CN109598220A (en) | A kind of demographic method based on the polynary multiple dimensioned convolution of input | |
| CN107292272B (en) | Method and system for recognizing human face in real-time transmission video | |
| CN110516707B (en) | Image labeling method and device and storage medium thereof | |
| CN112329662B (en) | Multi-view saliency estimation method based on unsupervised learning | |
| CN108401565B (en) | Remote sensing image registration method based on improved KAZE features and Pseudo-RANSAC algorithms | |
| CN112164115A (en) | Object pose identification method and device and computer storage medium | |
| CN111382791B (en) | Deep learning task processing method, image recognition task processing method and device | |
| CN114724218A (en) | Video detection method, device, equipment and medium | |
| CN114119460A (en) | Semiconductor image defect identification method, semiconductor image defect identification device, computer equipment and storage medium | |
| CN115131580A (en) | Space target small sample identification method based on attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200828 |