WO2016184276A1 - 一种人脸关键点位定位结果的评估方法，及评估装置 - Google Patents

一种人脸关键点位定位结果的评估方法，及评估装置 Download PDF

Info

Publication number
WO2016184276A1
WO2016184276A1 PCT/CN2016/079118 CN2016079118W WO2016184276A1 WO 2016184276 A1 WO2016184276 A1 WO 2016184276A1 CN 2016079118 W CN2016079118 W CN 2016079118W WO 2016184276 A1 WO2016184276 A1 WO 2016184276A1
Authority
WO
WIPO (PCT)
Prior art keywords
evaluation
face
coordinates
feature value
result
Prior art date
Application number
PCT/CN2016/079118
Other languages
English (en)
French (fr)
Inventor
汪铖杰
李季檩
黄飞跃
盛柯恺
董未名
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2016184276A1 publication Critical patent/WO2016184276A1/zh
Priority to US15/671,041 priority Critical patent/US10331940B2/en
Priority to US16/409,341 priority patent/US10706263B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships

Definitions

  • the present invention relates to the field of computer technology, and in particular, to an evaluation method for a face key point location result, and an evaluation device.
  • the key point of the face is the facial features of the face, and the key position of the face is the positioning of the feature points of the facial features. Face key point location is a very important technique in face image analysis.
  • the result of facial features of facial features is directly related to many back-end techniques such as face beautification and face recognition. Therefore, it is a crucial issue to accurately evaluate the results of facial features.
  • Option 1 Manually pre-calibrate the point coordinates.
  • the five-point key points of a face image are manually labeled and stored as real values.
  • the facial five-point key point localization algorithm gives the result coordinates of the key points, the average distance between the measurement result coordinates and the artificial calibration coordinates is used to evaluate the quality of the positioning results.
  • Option 2 Artificial subjective assessment. Using artificial subjective assessment to determine the location of key points The pros and cons of the results. Using a variety of facial features, or using the randomness of the localization algorithm, multiple facial features are outputted to the same face image, and the most accurate one is selected by manual subjective evaluation.
  • the above scheme 2 adopts the method of artificial subjective comparison, and judges the superiority and inferior relationship between the two positioning results through the coordinates of the person positioning the key points.
  • This type of subjective assessment is less demanding on the face, and it takes only a few seconds to complete in the case where the number of results is not large.
  • this type of evaluation method has practical application in some products, it also has obvious defects.
  • this type of assessment is subjective and cannot be quantified.
  • Embodiments of the present invention provide an evaluation method for a face key point location result, and an evaluation apparatus for providing an efficient and quantifiable implementation solution.
  • a method for evaluating the location of a key point location of a face includes:
  • An apparatus for evaluating a result of a key point location of a face comprising:
  • a coordinate positioning unit configured to acquire a face image, and acquire a positioning result coordinate of a key point of the face image
  • a normalization calculation unit configured to normalize the positioning result coordinates and the average face model to obtain a normalized face image
  • a feature extraction unit configured to extract a face feature value of the normalized face image
  • an evaluation unit configured to calculate an evaluation result according to the facial feature value and the weight vector.
  • the normalization process is implemented by the average face model, and then the evaluation result is obtained according to the face feature value and the weight vector of the normalized face image.
  • the entire evaluation process does not require manual intervention, the evaluation speed is fast, and the parameters used to calculate the evaluation results are quantifiable, so the evaluation results can be quantified.
  • FIG. 1 is a schematic flowchart of a method according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of a method according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of a method according to an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a method according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an apparatus for evaluating an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an evaluation apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of an evaluation apparatus according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of an evaluation apparatus according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of an evaluation apparatus according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of an evaluation apparatus according to an embodiment of the present invention.
  • the embodiment of the invention provides a method for evaluating the location of a key point location of a face, as shown in FIG. 1 , including steps 101-103.
  • the key point algorithm used to obtain the positioning result coordinates of the key points of the above-mentioned face image can be arbitrarily selected, and different algorithms can obtain different positioning result coordinates, which can be evaluated in this embodiment.
  • multiple random sets of positioning result coordinates can be obtained by using the randomness of the ESR (Explicit Shape Regressor) facial facial feature positioning algorithm.
  • An embodiment of the present invention further provides an implementation manner of a specific normalization process, which is specifically as follows: the foregoing normalizing the positioning result coordinates and the average face model includes:
  • the coordinates of the above positioning result are reduced to the average face model to obtain the face image region, and the face image region is triangulated, and the obtained triangle is transformed as a partial portion one by one to obtain a normalized face. image.
  • the specific implementation manner of calculating the evaluation result according to the facial feature value and the weight vector may be: calculating an inner product of the facial feature value and the weight vector to obtain an evaluation result.
  • the result of the calculation of the inner product can be used to better quantify the evaluation result.
  • the embodiment of the present invention implements the normalization process by the average face model, and then calculates the evaluation result according to the face feature value and the weight vector of the normalized face image.
  • the entire evaluation process does not require manual intervention, the evaluation speed is fast, and the parameters used to calculate the evaluation results are quantifiable, so the evaluation results can be quantified.
  • the embodiment of the present invention can fully automatically evaluate the positioning result of the facial features of the facial features, and the evaluation process does not require manual intervention throughout the process. To some extent, the embodiment of the present invention fills in the automatic positioning result of the facial features of the facial features. Assessment of technology gaps. The invention can quantitatively evaluate the positioning result of the facial five sense points, and the speed is fast (each automatic evaluation takes about 10 milliseconds) and the reliability is high.
  • the facial feature value is a parameter for describing a facial feature, which is also called a feature descriptor.
  • the embodiment of the present invention can select a facial feature value correspondingly according to different requirements and focuses.
  • the face feature values may be combined, as follows: the above face feature values include: HOG (Histogram of Oriented Gradient) feature values, LBP (Local Binary Patterns) feature values. , Gabor (windowed Fourier transform) at least one of the feature values.
  • the embodiment of the invention can be used to select one or more high-precision results from the positioning results of the plurality of facial features, thereby improving the accuracy of the facial features of the facial features, and effectively avoiding the positioning with serious errors.
  • the result is as follows: the positioning result coordinates include at least two groups; after the evaluation result is obtained, the method further includes:
  • the embodiment of the invention further provides an implementation scheme for acquiring an average face model. It should be noted that the calculation of the average face model may not be performed on the evaluation device end of the face key point location result, and the evaluation end device that is sent by other devices to the face key point location result is also possible.
  • the method for obtaining the average face model may be specifically as follows: before the normalizing the positioning result coordinates and the average face model, the method further includes:
  • An embodiment of the present invention further provides an implementation scheme for acquiring a weight vector. It should be noted that the calculation of the weight vector may not be performed on the evaluation device end of the face key position positioning result, and the evaluation end device that is obtained by other devices and sent to the face key point positioning result is also possible.
  • the method for obtaining the weight vector may be specifically as follows: before the foregoing calculating the evaluation result according to the facial feature value and the weight vector, the method further includes:
  • the accuracy score of the K key point coordinates is determined according to the calculated RMSE, and the larger the RMSE is, the smaller the precision score is;
  • the K key point coordinates are reduced to the average face model to obtain a reference face image, and the face feature value of the reference face image is extracted;
  • the weight vector is obtained by using the above-described face feature value and the above-described precision score.
  • the truncation type may be, for example, the RMSE is greater than a certain value. After the value, the precision score is 0, which is less than the RMSE of a certain value. The larger the above precision score, the smaller.
  • the embodiment further provides a more specific implementation scheme for calculating the weight vector by using the above-mentioned facial feature value and the above-mentioned precision score.
  • This embodiment can obtain better effects after being executed multiple times in a loop, as follows:
  • the face feature value and the above-mentioned precision score are calculated to obtain the above weight vector including:
  • the embodiment of the invention provides a method for evaluating the positioning result of the facial features of the facial features without reference, which can automatically sort the positioning results of the key points of the facial features of the human face without manual intervention. Through this sorting, one or more more accurate results can be selected from a plurality of results, and thereby the overall precision of the facial features of the facial features can be improved, and serious false results can be avoided.
  • the embodiment of the present invention is also based on the idea of sequence comparison sorting, and considers a plurality of key point positioning data on the same person's face, by means of HOG (Histogram of Oriented Gradient) on the normalized face image.
  • HOG Heistogram of Oriented Gradient
  • the result of the feature and the weight of the training can be given corresponding to the evaluation score, and the score is used to sort the N results so that the positioning result is better than the direct median.
  • the innovation of the embodiment of the present invention includes using the ListNet (Sequence Network for Training Serialized Data) sorting algorithm framework and the HOG feature of the face image to evaluate the positioning result of the facial features of the facial features.
  • the embodiment of the invention can complete the evaluation task of the positioning result fully automatically, and at the same time obtain the effect of improving the accuracy of the facial features of the facial features and avoiding the result of serious misplacement.
  • the HOG feature value also referred to as the HOG data feature.
  • the HOG feature principle is used: the core idea of the HOG is that the detected local object shape can be described by the light intensity gradient or the edge direction distribution. By dividing the entire image into small connected areas (called cells), each cell generates a ladder The histogram or the edge direction of the pixel in the cell. The combination of these histograms can represent the descriptor (the target of the detected target). To improve accuracy, the local histogram can be normalized by comparing the light intensity of a larger area (called a block) in the image as a measure, and then normalize all the cells in the block with this measure. This normalization process completes better illumination/shadow invariance.
  • HOG-derived descriptors Compared to other descriptors, HOG-derived descriptors maintain geometric and optical transformation invariance (unless the orientation of the object changes). Therefore, the HOG descriptor is particularly suitable for human detection.
  • the HOG feature extraction method is to put an image:
  • Grayscale see the image as a three-dimensional image of x, y, z (grayscale)
  • the training data preparation process includes steps 201-209.
  • the RMSE Root of Mean Squared Error
  • the score is scored according to the RMSE to obtain an accuracy score; the calibration method is as follows:
  • RMSE The larger the value of RMSE, the smaller the calibration score. In any case, the smaller the value of RMSE, the larger the calibration score.
  • the average face model is calculated according to the coordinates of the key points of the facial features of the M face maps, and the above-mentioned tie face model is triangulated.
  • the uniform average face is stipulated, thereby obtaining different face images.
  • the integration training gets the weight W, and the evaluation score, as follows:
  • a weight W By inputting the obtained HOG data features and precision scores into the training framework of ListNet for training, a weight W can be obtained.
  • the inner product of the weight W and the HOG data feature is calculated to obtain an evaluation score of the face key point.
  • the ListNet training process includes steps 301-307.
  • the inner product of the HOG data feature and the current weight in this step is used as the evaluation score.
  • the accuracy score is the RMSE calculated from the evaluation score corresponding to the key point coordinate value of the manual mark. [0091] This step utilizes cross entropy to measure the probability value of the current sequence.
  • step 402 Perform normalization processing according to the positioning result coordinate point and the average face model (the average face model obtaining process is referred to step 206 in the foregoing embodiment).
  • the normalization scheme is as follows:
  • the face region is triangulated, and each triangle is used as a partial part, and a Piece-wise Affine Transform is performed one by one to obtain a normalized face image.
  • the process of obtaining the weight vector in this step refers to the weight obtained in the ListNet training process in the foregoing embodiment.
  • the positioning results of a plurality of face key points for example: using Multiple face key point localization algorithms, or using the randomness of the face key point localization algorithm such as ESR to output multiple positioning results for the same personal face image, and using the evaluation algorithm of the present invention, automatically perform accuracy evaluation for each positioning result. Sort according to the level of accuracy evaluation score. Further, by selecting the most accurate positioning result, or selecting several of the most accurate results for recombination, a more stable and accurate facial five-point positioning result can be obtained.
  • the embodiment of the invention can fully evaluate the accuracy of the positioning result of the facial features of the facial features, and can effectively improve the positioning accuracy of the key points of the facial features and avoid the obvious deviation of the key points.
  • Accurate face facial features can be directly used for facial makeup, face makeup and other applications, greatly reducing manual interaction and improving user experience.
  • the specific application after the evaluation of the key point location result of the face can be determined according to the needs, and the embodiment of the present invention is not limited.
  • An embodiment of the present invention further provides an apparatus for evaluating a result of a key point location of a face, as shown in FIG. 5, including:
  • a coordinate positioning unit 501 configured to acquire a face image, and acquire a positioning result coordinate of a key point of the face image
  • the normalization calculation unit 502 is configured to perform normalization processing on the positioning result coordinates and the average face model to obtain a normalized face image
  • a feature extraction unit 503, configured to extract a face feature value of the normalized face image
  • the evaluation unit 504 is configured to calculate an evaluation result according to the face feature value and the weight vector described above.
  • the key point algorithm can be arbitrarily selected, and different algorithms can obtain different positioning result coordinates, which can be evaluated in this embodiment.
  • multiple random sets of positioning result coordinates can be obtained by using the randomness of the ESR (Explicit Shape Regressor) facial facial feature positioning algorithm.
  • the embodiment of the present invention implements the homing through the average face model.
  • the processing is performed, and then the evaluation result is obtained according to the face feature value of the normalized face image and the weight vector.
  • the entire evaluation process does not require manual intervention, the evaluation speed is fast, and the parameters used to calculate the evaluation results are quantifiable, so the evaluation results can be quantified.
  • the embodiment of the present invention can fully automatically evaluate the positioning result of the facial features of the facial features, and the evaluation process does not require manual intervention throughout the process. To some extent, the embodiment of the present invention fills in the automatic positioning result of the facial features of the facial features. Assessment of technology gaps. The invention can quantitatively evaluate the positioning result of the facial five sense points, and the speed is fast (each automatic evaluation takes about 10 milliseconds) and the reliability is high.
  • the face feature value is a parameter for describing a face feature, which is also called a feature descriptor.
  • the embodiments of the present invention may be selected correspondingly according to different requirements and emphasis, and may be used in combination for stability. Specifically as follows: the above facial feature values include:
  • At least one of a direction gradient histogram HOG feature value, a local binary mode LBP feature value, and a windowed Fourier transform Gabor feature value is included in At least one of a direction gradient histogram HOG feature value, a local binary mode LBP feature value, and a windowed Fourier transform Gabor feature value.
  • the embodiment of the present invention further provides an implementation manner of a specific normalization process, which is specifically as follows: the normalization calculation unit 502 is configured to reduce the coordinates of the positioning result to an average face model to obtain a face image region, and obtain The face image area is triangulated, and the obtained triangle is reflected and transformed as a partial part one by one to obtain a normalized face image.
  • the embodiment of the invention can be used to select one or more high-precision results from the positioning results of the plurality of facial features, thereby improving the accuracy of the facial features of the facial features, and effectively avoiding the positioning with serious errors.
  • the result is as follows: further, as shown in FIG. 6, the positioning result coordinates include at least two groups; the evaluation device further includes:
  • the recombining unit 601 is configured to, after the evaluation unit 504 obtains the evaluation result, select a predetermined number of positioning result coordinates with the highest evaluation accuracy, and perform recombination to obtain the target positioning result coordinates.
  • the embodiment of the invention further provides an implementation scheme for obtaining an average face model, which needs to be explained. It is the calculation of the average face model that can be performed on the evaluation device side of the face key point location result. It is also possible to obtain the evaluation end device that is sent to the face key point location result by other devices, and the average face model is obtained.
  • the method may be specifically as follows: Further, as shown in FIG. 7, the foregoing evaluation apparatus further includes:
  • the face calculation unit 701 is configured to acquire the M face images and obtain the key point coordinates of the manual mark before the normalization calculation unit 502 normalizes the positioning result coordinates and the average face model. >1; using the least squares method, the average face model is calculated according to the key point coordinates of the above manual marking.
  • the embodiment of the present invention further provides an implementation scheme for acquiring a weight vector. It should be noted that the calculation of the weight vector may not be performed on the evaluation device end of the face key location result, and is sent to the face key point location by other devices.
  • the evaluation device of the result is also possible, and the manner of obtaining the weight vector may be specifically as follows: Further, as shown in FIG. 8, the evaluation device further includes:
  • the weight calculation unit 801 is configured to perform positioning calculation on the M facial images by using the display shape regression ESR algorithm before the evaluation unit 504 calculates the evaluation result according to the facial feature value and the weight vector, and obtain K key point coordinates. , K>1; calculating an average squared error root RMSE between the coordinates of the K key points and the key point coordinates of the manual mark; determining the precision score of the K key point coordinates according to the calculated RMSE, The larger the RMSE is, the smaller the precision score is; the K key point coordinates are reduced to the average face model to obtain a reference face image, and the face feature value of the reference face image is extracted; and the face feature is used. The value and the above-mentioned precision score are calculated to obtain the above weight vector.
  • the truncation type may be, for example, the RMSE is greater than a certain value. After the value, the precision score is 0. The larger the RMSE is less than the above value, the smaller the precision score is.
  • the embodiment further provides a more specific implementation scheme for calculating the weight vector by using the above-mentioned facial feature value and the above-mentioned precision score.
  • This embodiment can obtain better effects after being executed multiple times, as follows:
  • the weight calculation unit 801 configured to calculate the weight vector by using the face feature value and the precision score, includes: using the inner product of the face feature value and the current weight vector as an evaluation score, and sorting the evaluation score And calculating a weight deviation amount between the sorting result and the precision score, and updating the current weight vector according to the deviation amount to obtain the weight vector.
  • the embodiment of the present invention further provides another apparatus for evaluating the location of a key point location, as shown in FIG. 9, comprising: a memory 904 and a processor 903.
  • the memory 904 is used to store programs.
  • the processor 903 is configured to acquire a face image, acquire a positioning result coordinate of a key point of the face image, and perform normalization processing on the positioning result coordinate and the average face model when the program stored in the memory 904 is executed.
  • the normalized face image; the face feature value of the normalized face image is extracted, and then the evaluation result is calculated according to the face feature value and the weight vector.
  • the evaluation device can also include a receiver 901 and a transmitter 902 for receiving and transmitting data.
  • the key point algorithm can be arbitrarily selected, and different algorithms can obtain different positioning result coordinates, which can be evaluated in this embodiment.
  • multiple random sets of positioning result coordinates can be obtained by using the randomness of the ESR (Explicit Shape Regressor) facial facial feature positioning algorithm.
  • the embodiment of the present invention implements the normalization process by the average face model, and then calculates the evaluation result according to the face feature value and the weight vector of the normalized face image.
  • the entire evaluation process does not require manual intervention, the evaluation speed is fast, and the parameters used to calculate the evaluation results are quantifiable, so the evaluation results can be quantified.
  • the embodiment of the present invention can fully automatically evaluate the positioning result of the facial features of the facial features, and the evaluation process does not require manual intervention throughout the process. To some extent, the embodiment of the present invention fills in the automatic positioning result of the facial features of the facial features. Assessment of technology gaps. The invention can quantitatively evaluate the positioning result of the facial five sense points, and the speed is fast (each automatic evaluation takes about 10 milliseconds) and the reliability is high.
  • the face feature value is a parameter for describing a face feature, which is also called a feature descriptor.
  • the embodiments of the present invention may be selected correspondingly according to different requirements and emphasis, and may be used in combination for stability.
  • the details are as follows: the above facial feature values include: HOG (Histogram of Oriented Gradient) feature values, LBP (Local Binary Patterns) feature values, and Gabor (Windowed Fourier Transform) feature values. At least one.
  • the embodiment of the present invention can be used to select one or more high-precision results from the positioning results of multiple facial features, thereby improving the accuracy of the facial features of the facial features and effectively avoiding serious errors.
  • the positioning result is as follows: the positioning result coordinates include at least two groups; the processor 903, when executing the program stored in the memory 904, is further configured to obtain a predetermined number of positioning results with the highest evaluation accuracy after the evaluation result is obtained. The coordinates are reorganized to obtain the coordinates of the target positioning result.
  • the embodiment of the present invention further provides a specific implementation of the specific normalization process, as follows:
  • the processor 903 is configured to perform normalization processing on the positioning result coordinates and the average face model, including:
  • the face image area is obtained by the average face model, and the face image area is triangulated, and the obtained triangle is transformed as a partial part one by one to obtain a normalized face image.
  • the embodiment of the present invention further provides an implementation scheme for obtaining an average face model.
  • the calculation of the average face model may not be performed on the evaluation device end of the face key point location result, and is sent to the face by other devices.
  • the evaluation end device of the key point location result is also ok, and the average face model can be obtained in the following manner: the above processor 903 is executing
  • the program stored in the memory 904 is further configured to acquire the M face images and obtain the key point coordinates of the manual mark before the normalization process of the positioning result coordinates and the average face model, M>1;
  • the two-square method calculates the average face model based on the coordinates of the key points of the above-mentioned manual marking.
  • the embodiment of the present invention further provides an implementation scheme for acquiring a weight vector.
  • the calculation of the weight vector may not be performed on the evaluation device end of the face key location result, and is sent to the face key point location by other devices.
  • the result of the evaluation end device is also possible, and the weight vector can be obtained as follows: when the program stored in the memory 904 is executed, the processor 903 is further configured to use the face feature value and the weight vector before calculating the evaluation result.
  • Display shape regression ESR algorithm performs positioning calculation on the above M face images, and obtains K key point coordinates, K>1;
  • the accuracy score of the K key point coordinates is determined according to the calculated RMSE, and the larger the RMSE is, the smaller the precision score is;
  • the K key point coordinates are reduced to the average face model to obtain a reference face image, and the face feature value of the reference face image is extracted;
  • the weight vector is obtained by using the above-described face feature value and the above-described precision score.
  • the truncation type may be, for example, the RMSE is greater than a certain value. After the value, the precision score is 0. The larger the RMSE is less than the above value, the smaller the precision score is.
  • the embodiment further provides a more specific implementation scheme for calculating the weight vector by using the above-mentioned facial feature value and the above-mentioned precision score.
  • This embodiment can obtain better effects after being executed multiple times, as follows: 903, used to use the above facial feature values and on Calculating the accuracy scores to obtain the weight vector includes: using the inner product of the face feature value and the current weight vector as the evaluation score, sorting the evaluation scores, and calculating a weight deviation amount between the sort result and the precision score, and The current weight vector is updated according to the above deviation amount to obtain the above weight vector.
  • the embodiment of the present invention further provides another device for evaluating the location of the key point location of the face.
  • the terminal may be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), an in-vehicle computer, and the terminal is a mobile phone as an example:
  • FIG. 10 is a block diagram showing a partial structure of a mobile phone related to a terminal provided by an embodiment of the present invention.
  • the mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, and a processor 1080. And power supply 1090 and other components.
  • RF radio frequency
  • the RF circuit 1010 can be used for receiving and transmitting signals during the transmission or reception of information or during a call. In particular, after receiving the downlink information of the base station, it is processed by the processor 1080. In addition, the uplink data is designed to be sent to the base station. Generally, RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, RF circuit 1010 can also communicate with the network and other devices via wireless communication. The above wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division). Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail, Short Messaging Service (SMS), and the like.
  • GSM Global System of Mobile communication
  • GPRS General Packet Radio Service
  • the memory 1020 can be used to store software programs and modules, and the processor 1080 executes various functional applications and data processing of the mobile phone by running software programs and modules stored in the memory 1020.
  • the memory 1020 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to Data created by the use of the mobile phone (such as audio data, phone book, etc.).
  • memory 1020 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • the input unit 1030 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function controls of the handset.
  • the input unit 1030 may include a touch panel 1031 and other input devices 1032.
  • the touch panel 1031 also referred to as a touch screen, can collect touch operations on or near the user (such as the user using a finger, a stylus, or the like on the touch panel 1031 or near the touch panel 1031. Operation), and drive the corresponding connecting device according to a preset program.
  • the touch panel 1031 may include two parts: a touch detection device and a touch controller.
  • the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information.
  • the processor 1080 is provided and can receive commands from the processor 1080 and execute them.
  • the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.
  • the input unit 1030 may also include other input devices 1032.
  • other input devices 1032 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.
  • the display unit 1040 can be used to display information input by the user or information provided to the user as well as various menus of the mobile phone.
  • the display unit 1040 may include a display panel 1041.
  • the display panel 1041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
  • the touch panel 1031 may cover the display panel 1041, and when the touch panel 1031 detects a touch operation thereon or nearby, the touch panel 1031 transmits to the processor 1080 to determine the type of the touch event, and then the processor 1080 according to the touch event. The type provides a corresponding visual output on display panel 1041.
  • touch panel 1031 and the display panel 1041 are used as two independent components to implement the input and input functions of the mobile phone in FIG. 10, in some embodiments, the touch panel 1031 may be integrated with the display panel 1041. Realize the input and output functions of the phone.
  • the handset can also include at least one type of sensor 1050, such as a light sensor, motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may close the display panel 1041 and/or when the mobile phone moves to the ear. Or backlight.
  • the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity.
  • the mobile phone can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the mobile phone can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.
  • the gesture of the mobile phone such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration
  • vibration recognition related functions such as pedometer, tapping
  • the mobile phone can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.
  • An audio circuit 1060, a speaker 1061, and a microphone 1062 can provide an audio interface between the user and the handset.
  • the audio circuit 1060 can transmit the converted electrical data of the received audio data to the speaker 1061, and convert it into a sound signal output by the speaker 1061; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, by the audio circuit 1060. After receiving, it is converted into audio data, and then processed by the audio data output processor 1080, sent to the other mobile phone via the RF circuit 1010, or outputted to the memory 1020 for further processing.
  • WiFi is a short-range wireless transmission technology.
  • the mobile phone through the WiFi module 1070 can help users to send and receive e-mail, browse the web and access streaming media, etc. It provides users with wireless broadband Internet access.
  • FIG. 10 shows the WiFi module 1070, it can be understood that it does not belong to the essential configuration of the mobile phone, and may be omitted as needed within the scope of not changing the essence of the invention.
  • the processor 1080 is the control center of the handset, which connects various portions of the entire handset using various interfaces and lines, by executing or executing software programs and/or modules stored in the memory 1020, and invoking data stored in the memory 1020, The phone's various functions and processing data, so that the overall monitoring of the phone.
  • the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like.
  • the modem processor primarily handles wireless communications. It will be appreciated that the above described modem processor may also not be integrated into the processor 1080.
  • the mobile phone also includes a power source 1090 (such as a battery) that supplies power to various components.
  • a power source 1090 such as a battery
  • the power source can be logically coupled to the processor 1080 through a power management system to manage functions such as charging, discharging, and power management through the power management system.
  • the mobile phone may further include a camera, a Bluetooth module, and the like, and details are not described herein again.
  • the processor 1080 included in the terminal further has a function of executing the above method flow.
  • each unit included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, specific of each functional unit
  • the names are also for convenience of distinction from each other and are not intended to limit the scope of protection of the present invention.
  • the storage medium mentioned above may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

一种人脸关键点位定位结果的评估方法及评估装置，其中方法的实现包括：获取人脸图像，获取所述人脸图像的关键点的定位结果坐标（101）；对所述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像（102）；提取所述归一化后的人脸图像的人脸特征值，根据所述人脸特征值以及权重向量计算评估结果（103）。在得到定位结果坐标以后，通过平均人脸模型实现归一化处理，然后根据归一化后的人脸图像的人脸特征值以及权重向量计算获得评估结果。整个评估过程不需要人工干预，评估速度快，而且计算评估结果所使用的参数是可以量化的，因此评估结果可以量化。

Description

一种人脸关键点位定位结果的评估方法，及评估装置
本申请要求于2015年5月20日提交中国专利局、申请号为201510259823.0、发明名称为“一种人脸关键点位定位结果的评估方法，及评估装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及计算机技术领域，特别涉及一种人脸关键点位定位结果的评估方法，及评估装置。
背景技术
人脸关键点位是人脸的五官特征点位，人脸关键点位定位是对五官特征点位的定位。人脸关键点定位是人脸图像分析中的很重要的技术。人脸五官特征点定位结果的好坏直接关系人脸美化、人脸识别等多项后端技术，因此准确评估人脸五官特征点定位结果的好坏也成为一个至关重要的问题。
目前的人脸关键点定位结果评估需要依赖于人工介入，有如下两种方案：
方案一：人工预先标定点坐标。采用人工方式来对某一人脸图像的五官关键点进行坐标标注，并作为真实值存储起来。当人脸五官关键点定位算法给出关键点的结果坐标时，通过度量结果坐标与人工标定坐标之间的平均距离来评估定位结果的好坏。
方案二：人工主观评估。利用人工主观评估来判定人脸关键点定位 结果的优劣。使用多种人脸五官关键点定位算法或利用定位算法本身的随机性，对同一张人脸图片输出多个五官定位结果，再通过人工主观评估的方式从中选出最为精准的一个结果。
以上方案一采用对人脸图片中的五官关键点进行手动标定。通常采用这种形式进行坐标的标定需要耗时几分钟，人工成本很高且耗时长。这类评估方法一般用多个五官关键点定位算法的比较。
以上方案二采用人工主观比较的方式，通过人对关键点定位结果坐标，主观的判断两个定位结果的优劣关系。这类主观评估的方式对于人脸需求较小，在结果数量不是很多的情况下只需要数秒时间就能完成。这类评估方法虽然在部分产品中已具有实际应用性，但也存在明显的缺陷。第一、这类评估方式主观性较强，且无法量化。第二、当需要比较的结果的数量较多时，比较难度增大，人工评估的耗时会大幅增长，评估的可靠性也会相应降低。
以上两种方案都需要人工的介入，不但耗时长效率低，而且定位结果无法量化。
发明内容
本发明实施例提供了一种人脸关键点位定位结果的评估方法，及评估装置，用于提供高效并且能够量化的实现方案。
一种人脸关键点位定位结果的评估方法，包括：
获取人脸图像，获取所述人脸图像的关键点的定位结果坐标；
对所述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；
提取所述归一化后的人脸图像的人脸特征值，根据所述人脸特征值以及权重向量计算评估结果。
一种人脸关键点位定位结果的评估装置，包括：
坐标定位单元，用于获取人脸图像，获取所述人脸图像的关键点的定位结果坐标；
归一计算单元，用于对所述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；
特征提取单元，用于提取所述归一化后的人脸图像的人脸特征值；
评估单元，用于根据所述人脸特征值以及权重向量计算评估结果。
从以上技术方案可以看出，在得到定位结果坐标以后，通过平均人脸模型实现归一化处理，然后根据归一化后的人脸图像的人脸特征值以及权重向量计算获得评估结果。整个评估过程不需要人工干预，评估速度快，而且计算评估结果所使用的参数是可以量化的，因此评估结果可以量化。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例方法流程示意图；
图2为本发明实施例方法流程示意图；
图3为本发明实施例方法流程示意图；
图4为本发明实施例方法流程示意图；
图5为本发明实施例评估装置结构示意图；
图6为本发明实施例评估装置结构示意图；
图7为本发明实施例评估装置结构示意图；
图8为本发明实施例评估装置结构示意图；
图9为本发明实施例评估装置结构示意图；
图10为本发明实施例评估装置结构示意图。
具体实施方式
为了使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明作进一步地详细描述，显然，所描述的实施例仅仅是本发明一部份实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。
本发明实施例提供了一种人脸关键点位定位结果的评估方法，如图1所示，包括步骤101-103。
101：获取人脸图像，获取上述人脸图像的关键点的定位结果坐标。
在本发明实施例中，获取上述人脸图像的关键点的定位结果坐标所使用的关键点算法可以任意选择，不同的算法可以获得不同的定位结果坐标，本实施例可以对其进行评估。在本实施例中，可以选择使用ESR(Explicit Shape Regressor，显示形状回归)人脸五官定位算法的随机性获得多组不同的定位结果坐标。
102：对上述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像。
本发明实施例还提供了具体归一化处理的一种实现方式，具体如下：上述对上述定位结果坐标以及平均人脸模型进行归一化处理包括：
将上述定位结果坐标归约到平均人脸模型得到人脸图像区域，对得到人脸图像区域进行三角剖分，将得到的三角形作为一个局部部分逐个进行反射变换，得到归一化后的人脸图像。
103：提取上述归一化后的人脸图像的人脸特征值，根据上述人脸特 征值以及权重向量计算评估结果。
在一种实现方式中，根据上述人脸特征值以及权重向量计算评估结果的具体实现方式可以是：计算上述人脸特征值与权重向量的内积得到评估结果。采用计算内积的方式可以更好的量化评估结果，作为一个优选的实现方式不应理解为对本发明实施例的唯一性限定。
本发明实施例在得到定位结果坐标以后，通过平均人脸模型实现归一化处理，然后根据归一化后的人脸图像的人脸特征值以及权重向量计算获得评估结果。整个评估过程不需要人工干预，评估速度快，而且计算评估结果所使用的参数是可以量化的，因此评估结果可以量化。
另外，本发明实施例能够全自动地对人脸五官特征点的定位结果进行评估，评估过程全程无需人工干预，从某种程度上，本发明实施例方案填补了人脸五官关键点定位结果自动评估技术的空白。本发明能对人脸五官点的定位结果给出量化评估，速度快(每次自动评估耗时约10毫秒)、可靠性高。
在本发明实施例中人脸特征值是用于描述人脸特征的参数，也称为特征描述子；基于不同的需求和侧重，本发明实施例可以相应选取人脸特征值。为了提高稳定性可以组合使用人脸特征值，具体如下：上述人脸特征值包括：HOG(Histogram of Oriented Gradient，方向梯度直方图)特征值，LBP(Local Binary Patterns，局部二值模式)特征值，Gabor(加窗傅立叶变换)特征值中的至少一项。
本发明实施例可以用于从多个人脸五官定位算法的定位结果中挑选出精度较高的一个或多个结果，从而提升人脸五官关键点定位结果的精度，并有效规避存在严重错误的定位结果，具体如下：上述定位结果坐标包含至少两组；得到评估结果之后，上述方法还包括：
选取得到评估精度最高的预定个数的定位结果坐标进行重组，得到 目标定位结果坐标。
本发明实施例还提供了获取平均人脸模型的实现方案。需要说明的是平均人脸模型的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的。平均人脸模型的获得方式可以具体如下：上述对上述定位结果坐标以及平均人脸模型进行归一化处理之前，上述方法还包括：
获取M张人脸图像，并获取手工标记的关键点位坐标，M＞1；采用最小二乘法，依据上述手工标记的关键点位坐标计算得到平均人脸模型。
本发明实施例还提供了获取权重向量的实现方案。需要说明的是权重向量的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的。权重向量的获得方式可以具体如下：上述根据上述人脸特征值以及权重向量计算评估结果之前，上述方法还包括：
利用显示形状回归ESR算法对上述M张人脸图像进行定位计算，得到K个关键点位坐标，K＞1；
计算上述K个关键点位坐标与上述手工标记的关键点位坐标之间的平均平方误差根RMSE；
依据计算得到的RMSE确定上述K个关键点位坐标的精度分值，上述RMSE越大上述精度分值越小；
将上述K个关键点位坐标归约到上述平均人脸模型，得到参考人脸图像，提取上述参考人脸图像的人脸特征值；
使用上述人脸特征值以及上述精度分值计算得到上述权重向量。
另需说明的是以上“上述RMSE越大上述精度分值越小”仅表示一个趋势，可以是严格的RMSE越大上述精度分值越小，也可以是截断型的，例如：RMSE大于某一数值以后精度分值为0，小于以上某一数值的RMSE 越大上述精度分值越小。
本实施例还提供了使用上述人脸特征值以及上述精度分值计算得到上述权重向量的更为具体实现方案，本实施例可以循环执行多次以后获得较好的效果，具体如下：上述使用上述人脸特征值以及上述精度分值计算得到上述权重向量包括：
使用上述人脸特征值和当前权重向量的内积作为评估分数，对上述评估分数进行排序，计算上述排序结果与上述精度分值的权重偏差量，并依据上述偏差量更新当前权重向量，得到上述权重向量。
本发明实施例提出了一种无参考的人脸五官关键点定位结果的评估方法，可以在无人工干预的情况下，全自动的对多个人脸五官关键点的定位结果进行排序。通过这种排序，可以从多个结果中选取一个或多个较为精确的结果，并以此达到提升人脸五官特征点定位的整体精度，规避严重错误结果。本发明实施例还基于序列比较排序的思想，同时考虑多个在同一张人脸上的关键点定位数据，借助在归一化人脸图像上的HOG(Histogram of Oriented Gradient，方向梯度直方图)特征以及训练出的权重的结果，能够对应给出评估分数，以评估分数来对N个结果进行排序，以便能够组合得到比直接取中位数要好的定位结果。本发明实施例的创新点包含利用ListNet(序列网络，用于训练序列化的数据)排序算法框架和人脸图像的HOG特征，来对人脸五官关键点定位结果的评估。本发明实施例可以全自动的完成对定位结果的评估任务，同时取得提升人脸五官点定位结果精度，规避严重错误定位结果的效果。
以下实施例将以人脸特征值为HOG特征值(也称为HOG数据特征)进行举例。在本发明实施例中，使用到的HOG特征原理：HOG的核心思想是所检测的局部物体外形能够被光强梯度或边缘方向的分布所描述。通过将整幅图像分割成小的连接区域(称为cells)，每个cell生成一个方向梯 度直方图或者cell中pixel的边缘方向，这些直方图的组合可表示出(所检测目标的目标)描述子。为改善准确率，局部直方图可以通过计算图像中一个较大区域(称为block)的光强作为measure被对比标准化，然后用这个值(measure)归一化这个block中的所有cells。这个归一化过程完成了更好的照射/阴影不变性。
与其他描述子相比，HOG得到的描述子保持了几何和光学转化不变性(除非物体方向改变)。因此HOG描述子尤其适合人的检测。
通俗的讲：
HOG特征提取方法就是将一个image：
1、灰度化(将图像看做一个x,y,z(灰度)的三维图像)；
2、划分成小cells(2*2)；
3、计算每个cell中每个pixel的gradient(即orientation)；
4、统计每个cell的梯度直方图(不同梯度的个数)，即可形成每个cell的descriptor。
以下举例主要分为三个方面：
一、训练数据准备过程，如图2所示，包括步骤201-209。
201：收集M张不同的人脸图像。
202：手工为每张人脸标注N个五官关键点坐标，记录为：
Figure PCTCN2016079118-appb-000001
203：利用ESR(Explicit Shape Regressor，显示形状回归)人脸五官定位算法的随机性，计算对于每张人脸的K个关键点定位结果(关键点位的定位结果坐标)，记录为：
Figure PCTCN2016079118-appb-000002
204：计算RMSE(Root of Mean Squared Error,平均平方误差根)误差，具体如下：
根据手工标记的关键点坐标值，计算得到与步骤203中定位得到的关键点位结果对应的RMSE(Root of Mean Squared Error,平均平方误差根)。
205：根据RMSE来标定分值，得到精度分值；标定方式如下：
RMSE的值越大，标定分值越小，反正，RMSE的值越小，标定分值越大。
标定精度分值的具体过程如下：
将RMSE做截断化处理，0.0～10.0范围内的值保持不变，大于10.0记做10.0；将RMSE值映射到精度分值。(0.0～10.0)->(100～0)，例如RMSE＝0.0,精度分值为100，RMSE＝2.0，精度分值为80，RMSE＝10.0，精度分值为0，以此类推。
206：利用最小二乘法，依据M张人脸图的手标五官关键点坐标计算得到平均人脸模型，并对上述平局人脸模型做三角剖分(Triangulation)。
207：人脸变形归约，具体如下：
对于每张人脸图像，根据ESR算法输出的K个的关键点定位结果，规约到统一的平均人脸上，由此可以得到不同的人脸图像。
208：在规约得到的人脸图像上提取HOG数据特征。
209：整合训练得到权重W，以及评估分数，具体如下：
将得到的HOG数据特征和精度分值输入到ListNet的训练框架中进行训练，就可以得到一个权重W。计算权重W与HOG数据特征的内积得到人脸关键点位的评估分数。
二、ListNet训练过程，如图3所示，包括步骤301-307。
301：权重W初始化。
302：确定是否达到预设循环次数，若是，输出当前权重值，否则进入303。
303：根据当前权重和HOG数据特征计算评估分数。
本步骤中HOG数据特征和当前权重的内积作为评估分数。
304：对评估分数进行排序。
305：根据排序结果以及精度分值计算概率值。
精度分值为手工标记的关键点坐标值对应的评估分数计算得到的RMSE。[0091]本步骤利用交叉熵(cross entropy)来度量当前序列的概率值。
306：计算权重偏差量；本步骤利用了梯度下降法计算权重偏差量。
307：依据上述权重偏差量更新当前权重。
三、在模型评估阶段，人脸关键点位的评估的流程如图4所示，包括步骤401-405。
401：输入人脸图像，利用人脸五官的关键点算法自动定位人脸五官关键点，获得定位结果坐标。
402：根据定位结果坐标点和平均人脸模型(平均人脸模型获得过程参阅前述实施例中步骤206)做归一化处理。
归一化方案具体如下：
根据定位点对人脸区域进行三角剖分，将每个三角形作为一个局部部分，逐个进行反射变换(Piece-wise Affine Transform)，得到归一化后的人脸图像。
403：计算归一化后的人脸图像的HOG特征。
404：计算HOG特征向量和权重向量的内积作为评估分值。
本步骤中权重向量的获得过程参阅前述实施例中ListNet训练过程中获得的权重。
405：输出评估数值。
本发明实施例在获得多个人脸关键点位定位结果以后，例如：使用 多个人脸关键点定位算法，或利用ESR等人脸关键点定位算法的随机性对同一个人脸图像输出多个定位结果，利用本发明的评估算法，自动为每个定位结果进行精准度评估。根据精准度评估分数的高低进行排序。进一步，通过选择精准度最高的定位结果，或选择几个精准度最高的几个结果进行重组，可获得更为稳定，精准的人脸五官关键点定位结果。
本发明实施例可以全自动评估人脸五官关键点定位结果的精度，能有效提升五官关键点定位精度，并避免关键点定位明显偏移的情况。精准的人脸五官关键点定位结果可以直接用于人脸美妆、人脸变妆等应用，大幅度减少人工交互，提升用户体验。在对人脸关键点位定位结果评估以后的具体应用，可以依据需要进行确定，本发明实施例不作唯一性限定。
本发明实施例还提供了一种人脸关键点位定位结果的评估装置，如图5所示，包括：
坐标定位单元501，用于获取人脸图像，获取上述人脸图像的关键点的定位结果坐标；
归一计算单元502，用于对上述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；
特征提取单元503，用于提取上述归一化后的人脸图像的人脸特征值；
评估单元504，用于根据上述人脸特征值以及权重向量计算评估结果。
在本发明实施例中，关键点算法可以任意选择，不同的算法可以获得不同的定位结果坐标，本实施例可以对其进行评估。在本实施例中，可以选择使用ESR(Explicit Shape Regressor，显示形状回归)人脸五官定位算法的随机性获得多组不同的定位结果坐标。
本发明实施例在得到定位结果坐标以后，通过平均人脸模型实现归 一化处理，然后根据归一化后的人脸图像的人脸特征值以及权重向量计算获得评估结果。整个评估过程不需要人工干预，评估速度快，而且计算评估结果所使用的参数是可以量化的，因此评估结果可以量化。
另外，本发明实施例能够全自动地对人脸五官特征点的定位结果进行评估，评估过程全程无需人工干预，从某种程度上，本发明实施例方案填补了人脸五官关键点定位结果自动评估技术的空白。本发明能对人脸五官点的定位结果给出量化评估，速度快(每次自动评估耗时约10毫秒)、可靠性高。
在本发明实施例中人脸特征值是用于描述人脸特征的参数，也称为特征描述子；基于不同的需求和侧重，本发明实施例可以相应选取，为了提稳定性可以组合使用，具体如下：上述人脸特征值包括：
方向梯度直方图HOG特征值，局部二值模式LBP特征值，加窗傅立叶变换Gabor特征值中的至少一项。
本发明实施例还提供了具体归一化处理的一种实现方式，具体如下：上述归一计算单元502，用于将上述定位结果坐标归约到平均人脸模型得到人脸图像区域，对得到人脸图像区域进行三角剖分，将得到的三角形作为一个局部部分逐个进行反射变换，得到归一化后的人脸图像。
本发明实施例可以用于从多个人脸五官定位算法的定位结果中挑选出精度较高的一个或多个结果，从而提升人脸五官关键点定位结果的精度，并有效规避存在严重错误的定位结果，具体如下：进一步地，如图6所示，上述定位结果坐标包含至少两组；上述评估装置还包括：
重组单元601，用于在上述评估单元504得到评估结果之后，选取得到评估精度最高的预定个数的定位结果坐标进行重组，得到目标定位结果坐标。
本发明实施例还提供了获取平均人脸模型的实现方案，需要说明的 是平均人脸模型的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的，平均人脸模型的获得方式可以具体如下：进一步地，如图7所示，上述评估装置还包括：
人脸计算单元701，用于在上述归一计算单元502对上述定位结果坐标以及平均人脸模型进行归一化处理之前，获取M张人脸图像，并获取手工标记的关键点位坐标，M＞1；采用最小二乘法，依据上述手工标记的关键点位坐标计算得到平均人脸模型。
本发明实施例还提供了获取权重向量的实现方案，需要说明的是权重向量的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的，权重向量的获得方式可以具体如下：进一步地，如图8所示，上述评估装置还包括：
权重计算单元801，用于在上述评估单元504根据上述人脸特征值以及权重向量计算评估结果之前，利用显示形状回归ESR算法对上述M张人脸图像进行定位计算，得到K个关键点位坐标，K＞1；计算上述K个关键点位坐标与上述手工标记的关键点位坐标之间的平均平方误差根RMSE；依据计算得到的RMSE确定上述K个关键点位坐标的精度分值，上述RMSE越大上述精度分值越小；将上述K个关键点位坐标归约到上述平均人脸模型，得到参考人脸图像，提取上述参考人脸图像的人脸特征值；使用上述人脸特征值以及上述精度分值计算得到上述权重向量。
另需说明的是以上“上述RMSE越大上述精度分值越小”仅表示一个趋势，可以是严格的RMSE越大上述精度分值越小，也可以是截断型的，例如：RMSE大于某一数值以后精度分值为0，小于以上某一数值的RMSE越大上述精度分值越小。
本实施例还提供了使用上述人脸特征值以及上述精度分值计算得到上述权重向量的更为具体实现方案，本实施例可以循环执行多次以后获得较好的效果，具体如下：可选地，上述权重计算单元801，用于使用上述人脸特征值以及上述精度分值计算得到上述权重向量包括：使用上述人脸特征值和当前权重向量的内积作为评估分数，对上述评估分数进行排序，计算上述排序结果与上述精度分值的权重偏差量，并依据上述偏差量更新当前权重向量，得到上述权重向量。
本发明实施例还提供了另一种人脸关键点位定位结果的评估装置，如图9所示，包括：存储器904以及处理器903。
存储器904用于存储程序。
处理器903，在执行存储器904存储的程序时，用于获取人脸图像，获取上述人脸图像的关键点的定位结果坐标；对上述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；提取上述归一化后的人脸图像的人脸特征值，然后根据上述人脸特征值以及权重向量计算评估结果。
另外，所述评估装置还可以包括接收器901和发射器902，用于接收和发送数据。
在本发明实施例中，关键点算法可以任意选择，不同的算法可以获得不同的定位结果坐标，本实施例可以对其进行评估。在本实施例中，可以选择使用ESR(Explicit Shape Regressor，显示形状回归)人脸五官定位算法的随机性获得多组不同的定位结果坐标。
本发明实施例在得到定位结果坐标以后，通过平均人脸模型实现归一化处理，然后根据归一化后的人脸图像的人脸特征值以及权重向量计算获得评估结果。整个评估过程不需要人工干预，评估速度快，而且计算评估结果所使用的参数是可以量化的，因此评估结果可以量化。
另外，本发明实施例能够全自动地对人脸五官特征点的定位结果进行评估，评估过程全程无需人工干预，从某种程度上，本发明实施例方案填补了人脸五官关键点定位结果自动评估技术的空白。本发明能对人脸五官点的定位结果给出量化评估，速度快(每次自动评估耗时约10毫秒)、可靠性高。
在本发明实施例中人脸特征值是用于描述人脸特征的参数，也称为特征描述子；基于不同的需求和侧重，本发明实施例可以相应选取，为了提稳定性可以组合使用，具体如下：上述人脸特征值包括：HOG(Histogram of Oriented Gradient，方向梯度直方图)特征值，LBP(Local Binary Patterns，局部二值模式)特征值，Gabor(加窗傅立叶变换)特征值中的至少一项。
本发明实施例可以用于从多个人脸五官定位算法的定位结果中挑选出精度较高的一个或多个结果，从而起到提升人脸五官关键点定位结果的精度，并有效规避存在严重错误的定位结果，具体如下：上述定位结果坐标包含至少两组；上述处理器903，在执行存储器904存储的程序时，还用于得到评估结果之后，选取得到评估精度最高的预定个数的定位结果坐标进行重组，得到目标定位结果坐标。
本发明实施例还提供了具体归一化处理的优选实现方案，具体如下：上述处理器903，用于对上述定位结果坐标以及平均人脸模型进行归一化处理包括：将上述定位结果坐标归约到平均人脸模型得到人脸图像区域，对得到人脸图像区域进行三角剖分，将得到的三角形作为一个局部部分逐个进行反射变换，得到归一化后的人脸图像。
本发明实施例还提供了获取平均人脸模型的实现方案，需要说明的是平均人脸模型的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的，平均人脸模型的获得方式可以具体如下：上述处理器903，在执行 存储器904存储的程序时，还用于对上述定位结果坐标以及平均人脸模型进行归一化处理之前，获取M张人脸图像，并获取手工标记的关键点位坐标，M＞1；采用最小二乘法，依据上述手工标记的关键点位坐标计算得到平均人脸模型。
本发明实施例还提供了获取权重向量的实现方案，需要说明的是权重向量的计算可以不在人脸关键点位定位结果的评估设备端执行，由其他设备获得以后发送给人脸关键点位定位结果的评估端设备也是可以的，权重向量的获得方式可以具体如下：上述处理器903，在执行存储器904存储的程序时，还用于根据上述人脸特征值以及权重向量计算评估结果之前，利用显示形状回归ESR算法对上述M张人脸图像进行定位计算，得到K个关键点位坐标，K＞1；
计算上述K个关键点位坐标与上述手工标记的关键点位坐标之间的平均平方误差根RMSE；
依据计算得到的RMSE确定上述K个关键点位坐标的精度分值，上述RMSE越大上述精度分值越小；
将上述K个关键点位坐标归约到上述平均人脸模型，得到参考人脸图像，提取上述参考人脸图像的人脸特征值；
使用上述人脸特征值以及上述精度分值计算得到上述权重向量。
另需说明的是以上“上述RMSE越大上述精度分值越小”仅表示一个趋势，可以是严格的RMSE越大上述精度分值越小，也可以是截断型的，例如：RMSE大于某一数值以后精度分值为0，小于以上某一数值的RMSE越大上述精度分值越小。
本实施例还提供了使用上述人脸特征值以及上述精度分值计算得到上述权重向量的更为具体实现方案，本实施例可以循环执行多次以后获得较好的效果，具体如下：上述处理器903，用于使用上述人脸特征值以及上 述精度分值计算得到上述权重向量包括：使用上述人脸特征值和当前权重向量的内积作为评估分数，对上述评估分数进行排序，计算上述排序结果与上述精度分值的权重偏差量，并依据上述偏差量更新当前权重向量，得到上述权重向量。
本发明实施例还提供了另一种人脸关键点位定位结果的评估装置，如图10所示，为了便于说明，仅示出了与本发明实施例相关的部分，具体技术细节未揭示的，请参照本发明实施例方法部分。该终端可以为包括手机、平板电脑、PDA(Personal Digital Assistant，个人数字助理)、POS(Point of Sales，销售终端)、车载电脑等任意终端设备，以终端为手机为例：
图10示出的是与本发明实施例提供的终端相关的手机的部分结构的框图。参考图10，手机包括：射频(Radio Frequency，RF)电路1010、存储器1020、输入单元1030、显示单元1040、传感器1050、音频电路1060、无线保真(wireless fidelity，WiFi)模块1070、处理器1080、以及电源1090等部件。本领域技术人员可以理解，图10中示出的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图10对手机的各个构成部件进行具体的介绍：
RF电路1010可用于收发信息或通话过程中，信号的接收和发送，特别地，将基站的下行信息接收后，给处理器1080处理；另外，将设计上行的数据发送给基站。通常，RF电路1010包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器(Low Noise Amplifier，LNA)、双工器等。此外，RF电路1010还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议，包括但不限于全球移动通讯系统(Global System of Mobile communication，GSM)、通用分组无线服务(General Packet Radio Service，GPRS)、码分多址(Code Division  Multiple Access，CDMA)、宽带码分多址(Wideband Code Division Multiple Access,WCDMA)、长期演进(Long Term Evolution，LTE)、电子邮件、短消息服务(Short Messaging Service，SMS)等。
存储器1020可用于存储软件程序以及模块，处理器1080通过运行存储在存储器1020的软件程序以及模块，从而执行手机的各种功能应用以及数据处理。存储器1020可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器1020可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
输入单元1030可用于接收输入的数字或字符信息，以及产生与手机的用户设置以及功能控制有关的键信号输入。具体地，输入单元1030可包括触控面板1031以及其他输入设备1032。触控面板1031，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板1031上或在触控面板1031附近的操作)，并根据预先设定的程式驱动相应的连接装置。可选的，触控面板1031可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器1080，并能接收处理器1080发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板1031。除了触控面板1031，输入单元1030还可以包括其他输入设备1032。具体地，其他输入设备1032可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
显示单元1040可用于显示由用户输入的信息或提供给用户的信息以及手机的各种菜单。显示单元1040可包括显示面板1041，可选的，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板1041。进一步的，触控面板1031可覆盖显示面板1041，当触控面板1031检测到在其上或附近的触摸操作后，传送给处理器1080以确定触摸事件的类型，随后处理器1080根据触摸事件的类型在显示面板1041上提供相应的视觉输出。虽然在图10中，触控面板1031与显示面板1041是作为两个独立的部件来实现手机的输入和输入功能，但是在某些实施例中，可以将触控面板1031与显示面板1041集成而实现手机的输入和输出功能。
手机还可包括至少一种传感器1050，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板1041的亮度，接近传感器可在手机移动到耳边时，关闭显示面板1041和/或背光。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；至于手机还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
音频电路1060、扬声器1061，传声器1062可提供用户与手机之间的音频接口。音频电路1060可将接收到的音频数据转换后的电信号，传输到扬声器1061，由扬声器1061转换为声音信号输出；另一方面，传声器1062将收集的声音信号转换为电信号，由音频电路1060接收后转换为音频数据，再将音频数据输出处理器1080处理后，经RF电路1010以发送给比如另一手机，或者将音频数据输出至存储器1020以便进一步处理。
WiFi属于短距离无线传输技术，手机通过WiFi模块1070可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。虽然图10示出了WiFi模块1070，但是可以理解的是，其并不属于手机的必须构成，完全可以根据需要在不改变发明的本质的范围内而省略。
处理器1080是手机的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器1020内的软件程序和/或模块，以及调用存储在存储器1020内的数据，执行手机的各种功能和处理数据，从而对手机进行整体监控。可选的，处理器1080可包括一个或多个处理单元；优选的，处理器1080可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器1080中。
手机还包括给各个部件供电的电源1090(比如电池)，优选的，电源可以通过电源管理系统与处理器1080逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。
尽管未示出，手机还可以包括摄像头、蓝牙模块等，在此不再赘述。
在本发明实施例中，该终端所包括的处理器1080还具有执行以上方法流程的功能。
值得注意的是，上述评估装置实施例中，所包括的各个单元只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本发明的保护范围。
另外，本领域普通技术人员可以理解实现上述各方法实施例中的全部或部分步骤是可以通过程序来指令相关的硬件完成，相应的程序可以存 储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上仅为本发明较佳的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明实施例揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应该以权利要求的保护范围为准。

Claims (14)

  1. 一种人脸关键点位定位结果的评估方法，其特征在于，包括：
    获取人脸图像，获取所述人脸图像的关键点的定位结果坐标；
    对所述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；
    提取所述归一化后的人脸图像的人脸特征值，根据所述人脸特征值以及权重向量计算评估结果。
  2. 根据权利要求1所述方法，其特征在于，所述人脸特征值包括：
    方向梯度直方图HOG特征值，局部二值模式LBP特征值，加窗傅立叶变换Gabor特征值中的至少一项。
  3. 根据权利要求1所述方法，其特征在于，所述对所述定位结果坐标以及平均人脸模型进行归一化处理包括：
    将所述定位结果坐标归约到平均人脸模型得到人脸图像区域，对得到人脸图像区域进行三角剖分，将得到的三角形作为一个局部部分逐个进行反射变换，得到归一化后的人脸图像。
  4. 根据权利要求1至3任意一项所述方法，其特征在于，所述定位结果坐标包含至少两组；得到评估结果之后，所述方法还包括：
    选取得到评估精度最高的预定个数的定位结果坐标进行重组，得到目标定位结果坐标。
  5. 根据权利要求1至3任意一项所述方法，其特征在于，所述对所述定位结果坐标以及平均人脸模型进行归一化处理之前，所述方法还包括：
    获取M张人脸图像，并获取手工标记的关键点位坐标，M＞1；采用最小二乘法，依据所述手工标记的关键点位坐标计算得到平均人脸模型。
  6. 根据权利要求5所述方法，其特征在于，所述根据所述人脸特征值以及权重向量计算评估结果之前，所述方法还包括：
    利用显示形状回归ESR算法对所述M张人脸图像进行定位计算，得到K个关键点位坐标，K＞1；
    计算所述K个关键点位坐标与所述手工标记的关键点位坐标之间的平 均平方误差根RMSE；
    依据计算得到的RMSE确定所述K个关键点位坐标的精度分值，所述RMSE越大所述精度分值越小；
    将所述K个关键点位坐标归约到所述平均人脸模型，得到参考人脸图像，提取所述参考人脸图像的人脸特征值；
    使用所述人脸特征值以及所述精度分值计算得到所述权重向量。
  7. 根据权利要求6所述方法，其特征在于，所述使用所述人脸特征值以及所述精度分值计算得到所述权重向量包括：
    使用所述人脸特征值和当前权重向量的内积作为评估分数，对所述评估分数进行排序，计算所述排序结果与所述精度分值的权重偏差量，并依据所述偏差量更新当前权重向量，得到所述权重向量。
  8. 一种人脸关键点位定位结果的评估装置，其特征在于，包括：
    坐标定位单元，用于获取人脸图像，获取所述人脸图像的关键点的定位结果坐标；
    归一计算单元，用于对所述定位结果坐标以及平均人脸模型进行归一化处理，得到归一化后的人脸图像；
    特征提取单元，用于提取所述归一化后的人脸图像的人脸特征值；
    评估单元，用于根据所述人脸特征值以及权重向量计算评估结果。
  9. 根据权利要求8所述评估装置，其特征在于，所述人脸特征值包括：
    方向梯度直方图HOG特征值，局部二值模式LBP特征值，加窗傅立叶变换Gabor特征值中的至少一项。
  10. 根据权利要求8所述评估装置，其特征在于，
    所述归一计算单元，用于将所述定位结果坐标归约到平均人脸模型得到人脸图像区域，对得到人脸图像区域进行三角剖分，将得到的三角形作为一个局部部分逐个进行反射变换，得到归一化后的人脸图像。
  11. 根据权利要求8至10任意一项所述评估装置，其特征在于，所述定位结果坐标包含至少两组；所述评估装置还包括：
    重组单元，用于在所述评估单元得到评估结果之后，选取得到评估精度最高的预定个数的定位结果坐标进行重组，得到目标定位结果坐标。
  12. 根据权利要求8至10任意一项所述评估装置，其特征在于，所述评估装置还包括：
    人脸计算单元，用于在所述归一计算单元对所述定位结果坐标以及平均人脸模型进行归一化处理之前，获取M张人脸图像，并获取手工标记的关键点位坐标，M＞1；采用最小二乘法，依据所述手工标记的关键点位坐标计算得到平均人脸模型。
  13. 根据权利要求12所述评估装置，其特征在于，所述评估装置还包括：
    权重计算单元，用于在所述评估单元根据所述人脸特征值以及权重向量计算评估结果之前，利用显示形状回归ESR算法对所述M张人脸图像进行定位计算，得到K个关键点位坐标，K＞1；计算所述K个关键点位坐标与所述手工标记的关键点位坐标之间的平均平方误差根RMSE；依据计算得到的RMSE确定所述K个关键点位坐标的精度分值，所述RMSE越大所述精度分值越小；将所述K个关键点位坐标归约到所述平均人脸模型，得到参考人脸图像，提取所述参考人脸图像的人脸特征值；使用所述人脸特征值以及所述精度分值计算得到所述权重向量。
  14. 根据权利要求13所述评估装置，其特征在于，
    所述权重计算单元，用于使用所述人脸特征值以及所述精度分值计算得到所述权重向量包括：使用所述人脸特征值和当前权重向量的内积作为评估分数，对所述评估分数进行排序，计算所述排序结果与所述精度分值的权重偏差量，并依据所述偏差量更新当前权重向量，得到所述权重向量。
PCT/CN2016/079118 2015-05-20 2016-04-13 一种人脸关键点位定位结果的评估方法，及评估装置 WO2016184276A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/671,041 US10331940B2 (en) 2015-05-20 2017-08-07 Evaluation method and evaluation device for facial key point positioning result
US16/409,341 US10706263B2 (en) 2015-05-20 2019-05-10 Evaluation method and evaluation device for facial key point positioning result

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510259823.0A CN104966046B (zh) 2015-05-20 2015-05-20 一种人脸关键点位定位结果的评估方法，及评估装置
CN201510259823.0 2015-05-20

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/671,041 Continuation US10331940B2 (en) 2015-05-20 2017-08-07 Evaluation method and evaluation device for facial key point positioning result

Publications (1)

Publication Number Publication Date
WO2016184276A1 true WO2016184276A1 (zh) 2016-11-24

Family

ID=54220084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079118 WO2016184276A1 (zh) 2015-05-20 2016-04-13 一种人脸关键点位定位结果的评估方法，及评估装置

Country Status (3)

Country Link
US (2) US10331940B2 (zh)
CN (1) CN104966046B (zh)
WO (1) WO2016184276A1 (zh)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104966046B (zh) 2015-05-20 2017-07-21 腾讯科技（深圳）有限公司 一种人脸关键点位定位结果的评估方法，及评估装置
CN105426870B (zh) * 2015-12-15 2019-09-24 北京文安智能技术股份有限公司 一种人脸关键点定位方法及装置
US10902244B2 (en) * 2017-03-27 2021-01-26 Samsung Electronics Co., Ltd. Apparatus and method for image processing
TWI628624B (zh) * 2017-11-30 2018-07-01 國家中山科學研究院 Improved thermal image feature extraction method
CN109472269A (zh) * 2018-10-17 2019-03-15 深圳壹账通智能科技有限公司 图像特征配置及校验方法、装置、计算机设备及介质
CN109461153B (zh) * 2018-11-15 2022-04-22 联想(北京)有限公司 数据处理方法和装置
CN109598749B (zh) * 2018-11-30 2023-03-10 腾讯科技（深圳）有限公司 一种三维人脸模型的参数配置方法、装置、设备及介质
CN110929588A (zh) * 2019-10-30 2020-03-27 维沃移动通信有限公司 脸部特征点定位方法及电子设备
CN110956099B (zh) * 2019-11-14 2022-06-17 哈尔滨工程大学 一种动态手势指令识别方法
CN111601181B (zh) * 2020-04-27 2022-04-29 北京首版科技有限公司 生成视频指纹数据的方法及装置
CN113222996A (zh) * 2021-03-03 2021-08-06 中南民族大学 心脏分割质量评估方法、装置、设备及存储介质
CN113822171A (zh) * 2021-08-31 2021-12-21 苏州中科先进技术研究院有限公司 一种宠物颜值评分方法、装置、存储介质及设备
CN114648503A (zh) * 2022-03-21 2022-06-21 Oppo广东移动通信有限公司 视频关键点质量评价方法及装置、存储介质和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090208112A1 (en) * 2008-02-20 2009-08-20 Kabushiki Kaisha Toshiba Pattern recognition method, and storage medium which stores pattern recognition program
CN101706872A (zh) * 2009-11-26 2010-05-12 上海交通大学 通用的开放式人脸识别系统
CN102004899A (zh) * 2010-11-03 2011-04-06 无锡中星微电子有限公司 一种人脸认证系统及方法
CN102486868A (zh) * 2010-12-06 2012-06-06 华南理工大学 一种基于平均脸的美丽人脸合成方法
CN104966046A (zh) * 2015-05-20 2015-10-07 腾讯科技（深圳）有限公司 一种人脸关键点位定位结果的评估方法,及评估装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4745207B2 (ja) * 2006-12-08 2011-08-10 株式会社東芝 顔特徴点検出装置及びその方法
CN101877055A (zh) * 2009-12-07 2010-11-03 北京中星微电子有限公司 一种关键特征点定位的方法和装置
US10248848B2 (en) * 2012-03-13 2019-04-02 Nokia Technologies Oy Method and apparatus for improved facial recognition
CN103632129A (zh) * 2012-08-28 2014-03-12 腾讯科技（深圳）有限公司 一种人脸特征点定位方法及装置
US9443132B2 (en) * 2013-02-05 2016-09-13 Children's National Medical Center Device and method for classifying a condition based on image analysis
JP6013241B2 (ja) * 2013-03-18 2016-10-25 株式会社東芝 人物認識装置、及び方法
GB2518589B (en) * 2013-07-30 2019-12-11 Holition Ltd Image processing
CN103632136B (zh) * 2013-11-11 2017-03-29 北京天诚盛业科技有限公司 人眼定位方法和装置
CN104598936B (zh) * 2015-02-28 2018-07-27 北京畅景立达软件技术有限公司 人脸图像面部关键点的定位方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090208112A1 (en) * 2008-02-20 2009-08-20 Kabushiki Kaisha Toshiba Pattern recognition method, and storage medium which stores pattern recognition program
CN101706872A (zh) * 2009-11-26 2010-05-12 上海交通大学 通用的开放式人脸识别系统
CN102004899A (zh) * 2010-11-03 2011-04-06 无锡中星微电子有限公司 一种人脸认证系统及方法
CN102486868A (zh) * 2010-12-06 2012-06-06 华南理工大学 一种基于平均脸的美丽人脸合成方法
CN104966046A (zh) * 2015-05-20 2015-10-07 腾讯科技（深圳）有限公司 一种人脸关键点位定位结果的评估方法,及评估装置

Also Published As

Publication number Publication date
CN104966046A (zh) 2015-10-07
US10331940B2 (en) 2019-06-25
US20190266385A1 (en) 2019-08-29
US20170337420A1 (en) 2017-11-23
CN104966046B (zh) 2017-07-21
US10706263B2 (en) 2020-07-07

Similar Documents

Publication Publication Date Title
WO2016184276A1 (zh) 一种人脸关键点位定位结果的评估方法，及评估装置
WO2020187153A1 (zh) 目标检测方法、模型训练方法、装置、设备及存储介质
US10943091B2 (en) Facial feature point tracking method, apparatus, storage medium, and device
EP3252640B1 (en) Method for launching application and terminal
WO2019096008A1 (zh) 身份识别方法、计算机设备及存储介质
EP3620952B1 (en) Optical fingerprint identification method, and related product
CN108985220B (zh) 一种人脸图像处理方法、装置及存储介质
CN109670174B (zh) 一种事件识别模型的训练方法和装置
CN111476780A (zh) 一种图像检测方法、装置、电子设备以及存储介质
US11164022B2 (en) Method for fingerprint enrollment, terminal, and non-transitory computer readable storage medium
WO2018018698A1 (zh) 一种增强现实ar的信息处理方法、装置及系统
JP2020509447A (ja) 虹彩認識方法、電子装置、およびコンピュータ可読記憶媒体
CN109063558A (zh) 一种图像分类处理方法、移动终端及计算机可读存储介质
CN110083742B (zh) 一种视频查询方法和装置
CN113706446B (zh) 一种镜头检测方法和相关装置
CN115841575A (zh) 关键点检测方法、装置、电子设备、存储介质和程序产品
CN115081643B (zh) 对抗样本生成方法、相关装置及存储介质
CN113780291A (zh) 一种图像处理方法、装置、电子设备及存储介质
CN111986251A (zh) 物体体积测量方法、装置、测量设备及存储介质
CN112016345A (zh) 一种图像识别方法和装置
CN110969085B (zh) 脸部特征点定位方法及电子设备
CN112733573B (zh) 表格检测方法、装置、移动终端及存储介质
CN110929588A (zh) 脸部特征点定位方法及电子设备
CN118262381A (zh) 姿态估计方法、装置、电子设备及计算机可读存储介质
CN114332521A (zh) 图像分类方法、装置、移动终端及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16795763

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 10/04/2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16795763

Country of ref document: EP

Kind code of ref document: A1