CN110738110A - Human face key point detection method, device, system and storage medium based on anchor point - Google Patents
Human face key point detection method, device, system and storage medium based on anchor point Download PDFInfo
- Publication number
- CN110738110A CN110738110A CN201910859973.3A CN201910859973A CN110738110A CN 110738110 A CN110738110 A CN 110738110A CN 201910859973 A CN201910859973 A CN 201910859973A CN 110738110 A CN110738110 A CN 110738110A
- Authority
- CN
- China
- Prior art keywords
- face
- anchor
- points
- degree
- face image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/231—Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Geometry (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides human face key point detection methods, devices, systems and storage media based on anchor points, wherein the method comprises the steps of obtaining a human face image to be detected, extracting features of the human face image, classifying the human face image based on the extracted features to determine the types of the side face degrees of a human face in the human face image, wherein each side face degree corresponds to anchor points which are a set of average human face key points of the side face degrees obtained in advance, determining the offset of the key points of the human face in the human face image relative to the anchor points based on the extracted features and the classification results, and calculating the human face key points in the human face image based on the anchor points and the offset.
Description
Technical Field
The invention relates to the technical field of human face key points, in particular to human face key point detection methods, devices, systems and storage media based on anchor points.
Background
A face key point detection (Facial Landmark) technology has been widely applied to security, medical and other scenes by , and by integrating a face key point detection algorithm into a mobile phone or other terminal devices, a user can conveniently perform activities such as mobile phone unlocking, face brushing payment and medical cosmetology.
In the system in the current market, most human face key point detection methods treat all front face and side face data equally, and obtain spatial coordinate information of each key point by using full connection, but because the front face and the side face have angle change of degrees, partial information of the side face data is lost and the like, the characteristics extracted by the existing model lack pertinence and discriminability, and high-precision human face key point detection tasks under the side face condition cannot be completed.
Disclosure of Invention
The invention provides human face key point detection schemes based on anchor points, which extract different characteristics aiming at different degrees of side faces and greatly improve the key point detection precision under the condition of keeping the front face precision unchanged.
According to aspect of the invention, anchor point-based human face key point detection methods are provided, and the method comprises the steps of obtaining a human face image to be detected, extracting features of the human face image, classifying the human face image based on the extracted features to determine the type of the side face degree of a human face in the human face image, wherein each side face degree corresponds to anchor points which are a set of average human face key points of the side face degree obtained in advance, determining the offset of the key points of the human face in the human face image relative to the anchor points based on the extracted features and the classification result, and calculating the human face key points in the human face image based on the anchor points and the offset.
In embodiments of the present invention, the pre-determining of the anchor point includes obtaining a plurality of face sample images containing different side-face degrees, and averaging face key points in the face sample images of each side-face degree to obtain a set of average face key points corresponding to each side-face degree, which is used as the anchor point corresponding to the side-face degree.
In embodiments of the present invention, the averaging of the face key points in the face sample images of each side face degree includes clustering the plurality of face sample images by using a clustering method, and using the obtained cluster center point as the anchor point, or calculating an arithmetic mean value of each face key point in the plurality of face sample images of any side face degree, and using a set of the arithmetic mean values of each face key point in the plurality of face sample images of any side face degree as the anchor point corresponding to the any side face degree.
In embodiments of the present invention, the method is implemented by trained neural networks, where the neural networks include a th sub-network, a second sub-network, and a third sub-network, where the th sub-network is configured to obtain a face image to be detected and perform feature extraction on the face image, the second sub-network is configured to classify the face image to be detected based on features extracted by the th sub-network to determine a category of a side face degree of a face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face key points of the side face degree, the set of average face key points is obtained in advance, and the third sub-network is configured to determine an offset of a key point of the face in the face image to be detected relative to the anchor points based on features extracted by the th sub-network and a classification result of the second sub-network, and calculate a face key point in the face image to be detected based on the anchor points and the offset.
In embodiments of the invention, the third subnetwork further includes a plurality of branch networks, each branch network corresponding to side-face degree determinations of the offset and face keypoints calculations.
In embodiments of the present invention, the method is implemented by a plurality of trained neural networks, where the plurality of neural networks include a neural network, a second neural network and a third neural network, where the neural network is configured to obtain a face image to be detected and perform feature extraction on the face image, the second neural network is configured to classify the face image to be detected based on features extracted by the neural network to determine a type of a side face degree of a face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face key points of the side face degree obtained in advance, and the third neural network is configured to determine an offset of a key point of the face in the face image to be detected relative to the anchor points based on features extracted by the neural network and a classification result of the second neural network, and calculate a key point of the face in the face image to be detected based on the anchor points and the offset.
In embodiments of the invention, the third neural network further includes a plurality of sub-networks, each sub-network corresponding to the determination of the offset of side-face degrees and the calculation of face keypoints.
In embodiments of the invention, the categories of the side-face degree include left side face, front face, and right side face.
According to another aspect of the invention, anchor-based human face key point detection devices are provided, and the device comprises a feature extraction module, a classification module and a key point calculation module, wherein the feature extraction module is used for acquiring a human face image to be detected and extracting features of the human face image, the classification module is used for classifying the human face image based on the extracted features so as to determine the type of the side face degree of a human face in the human face image, each side face degree corresponds to kinds of anchor points, and the anchor points are a set of average human face key points of the side face degree obtained in advance, and the key point calculation module is used for determining the offset of the key points of the human face in the human face image relative to the anchor points based on the extracted features and the classification result and calculating the human face key points in the human face image based on the anchor points and the offset.
According to yet another aspect of the present invention, there is provided anchor point-based face keypoint detection systems, the systems comprising a storage device and a processor, the storage device having stored thereon a computer program for execution by the processor, the computer program, when executed by the processor, performing any of the anchor point-based face keypoint detection methods described above.
According to a further aspect of the present invention, there are storage media having stored thereon a computer program that, when executed, performs the anchor point-based face keypoint detection method of any item described above.
According to yet another aspect of the present invention, there are provided computer programs which, when being executed by a computer or a processor, are adapted to perform the anchor point-based face keypoint detection method of any above, and further adapted to implement the modules of the anchor point-based face keypoint detection apparatus of any above.
According to the method, the device and the system for detecting the face key points based on the anchor points, disclosed by the embodiment of the invention, the face images with different side face degrees are classified, and the face key points of the face images with different side face degrees are obtained by calculating according to the average face key points (namely the anchor points) with different side face degrees and the position offset between the real face key points and the anchor points under the side face degrees, so that the detection precision of the front face key points can be maintained, and the detection precision of the side face key points can be greatly improved.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent upon consideration of the following detailed description of the embodiments of the present invention taken in conjunction with the accompanying drawings, which are included to provide a further understanding of the embodiments of the present invention and form part of the specification, which together with the embodiment serve to explain the invention and are not to be construed as limiting the invention.
FIG. 1 shows a schematic block diagram of an example electronic device for implementing an anchor point-based face keypoint detection method, apparatus, and system in accordance with embodiments of the present invention;
FIG. 2 shows a schematic flow diagram of an anchor-based face keypoint detection method according to an embodiment of the invention;
FIG. 3 is a schematic block diagram of an anchor-based face keypoint detection apparatus according to an embodiment of the present invention; and
fig. 4 shows a schematic block diagram of an anchor-based face keypoint detection system according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings, it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments of the present invention, and it should be understood that the present invention is not limited by the exemplary embodiments described herein.
First, an example electronic device 100 for implementing the anchor-based face keypoint detection method, apparatus and system according to the embodiments of the present invention is described with reference to fig. 1.
As shown in FIG. 1, electronic device 100 includes or more processors 102, or more memory devices 104, input devices 106, and output devices 108, which are interconnected by a bus system 110 and/or other form of connection mechanism (not shown). it should be noted that the components and configuration of electronic device 100 shown in FIG. 1 are exemplary and not limiting, and that other components and configurations of electronic device may be used as desired.
The processor 102 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
The storage 104 may include or more computer program products that may include various forms of computer-readable storage media, such as volatile and/or non-volatile memory, such as Random Access Memory (RAM) and/or cache memory, etc. non-volatile memory, such as Read Only Memory (ROM), hard disk, flash memory, etc. may be stored or more computer program instructions that may be executed by the processor 102 to implement the anchor point-based face keypoint detection functionality of the embodiments of the invention described below (implemented by the processor) and/or other desired functionality.
The input device 106 may be a device used by a user to input instructions and may include or more of a keyboard, mouse, microphone, touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to an outside (e.g., a user), and may include or more of a display, speakers, etc.
Exemplary electronic devices for implementing the anchor point-based face key point detection method, apparatus and system according to embodiments of the present invention may be implemented as terminals such as smart phones, tablet computers, and the like.
In the following, a face key point detection method 200 based on anchor points according to an embodiment of the present invention will be described with reference to fig. 2. As shown in fig. 2, the method 200 for detecting a face key point based on an anchor point may include the following steps:
in step S210, a face image to be detected is acquired, and feature extraction is performed on the face image.
In the embodiment of the present invention, the face image acquired in step S210 may be a face image to be subjected to face key point detection, and the face image may include a video and/or a picture. Illustratively, the facial image may be acquired in real time or from any other source. In an embodiment of the present invention, the extraction of the features of the face image may be implemented by using a trained neural network.
In step S220, the face images are classified based on the extracted features to determine the types of the side face degrees of the faces in the face images, each side face degree corresponds to kinds of anchor points, and the anchor points are a set of average face key points of the side face degrees obtained in advance.
In the embodiment of the present invention, based on the features of the face image extracted in step S210, the face image may be classified to determine what side face degree the face image includes, that is, to determine the type of the side face degree of the face in the face image. Illustratively, a trained neural network may be employed to classify the kind of the side face degree of the face in the face image. The neural network may be trained using face sample images of various different side-face degrees.
For simplicity, in embodiments of the present invention, the categories of the side-face degree are divided into three categories: left, front and right. Based on this, a neural network may be trained using face sample images including left, front, and right faces to classify the face image to be detected. Specifically, all face sample images can be divided into three subsets of left face, front face and right face (for example, corresponding to label values of 1, 2 and 3 respectively) to train classification. Illustratively, the classification loss function may use SoftmaxWithLoss:
wherein yi is a label value, pi is a neuron corresponding to the label of the input face sample image, and n is the number of the face sample images. Of course, this is merely exemplary, and any other suitable classification loss function may be employed.
In addition, may also include the case of a face, rather than just a side face, in order for a trained neural network to classify face images that include various face situations.
For example, given three (in real applications generally comprises a greater number, for example only for simplicity herein) facial images A, B and C that are all left-sided faces, the key facial points in image A are A1 through A68 (assuming that there are 68 face key points per facial image), the key facial points in image B are B1 through B68, the key facial points in image C are C1 through C68, the anchor points corresponding to the left-sided faces (i.e., the average face key points) are E1 through E2, where E1 has the coordinates of A1, B1, and C1. where the coordinates of the three key points are C638, the average key points are the coordinates of the key points, and the anchor points are the coordinates of the key points of the face of the same number, similar to the coordinates of the key points of the face of the anchor points of the same number, similar to the key points of the face of the anchor points of the face of the same .
In other words, each side face degree corresponds to anchor points, which are sets of average face key points of the side face degree obtained in advance.
In another examples, the averaging of face keypoints in the face sample images of each side face degree may include clustering the face sample images using a clustering method and using the resulting cluster center point as the anchor point.
Based on the anchor points corresponding to different side face degrees obtained in advance and the classification result of step S220 (i.e., determining the type of the side face degree of the face in the face image to be detected), anchor points corresponding to the side face degree of the face in the face image to be detected may be obtained, and then, based on the anchor points, face keypoint detection may be performed on the face image to be detected, as will be described in the following steps.
In step S230, based on the extracted features and the classification result, determining an offset of a key point of a face in the face image with respect to the anchor point, and calculating a key point of the face in the face image based on the anchor point and the offset.
In the embodiment of the invention, the offset of the real face key point (namely, the labeled face key point) corresponding to each side face degree relative to the anchor point can be learned in advance, the offset of the face key point to be detected in the face image to be detected relative to the anchor point can be predicted through the learning of the offset calculation, and the face key point in the face image to be detected is calculated according to the side face degree (namely, the anchor point) of the face image to be detected and the offset relative to the anchor point. Specifically, the pre-learning process may include: on the basis of previously finding the anchor point corresponding to each side face degree, feature extraction is performed on each human face sample image, and each human face sample image is classified based on the extracted features to determine the type of the side face degree of the human face in each human face sample image, so as to determine the anchor point corresponding to the type, and the process can be called a classification branch. Then, based on the features extracted from each face sample image and the classification result, the offset of the position between the face key point labeled in each face sample image and the average face key point in the anchor point corresponding to the face in the face sample image is trained and learned, and this process may be referred to as a regression branch. Illustratively, the loss function of the regression branch may use winloss:
wherein w, epsilon and c are hyperparameters, and x is the offset between the labeled face key point and the average face key point. Of course, this is merely exemplary, and any other suitable regression loss function may also be employed.
Based on the above pre-learning of offset calculation, the offset of the face key point in the face image to be detected and the offset of the anchor point corresponding to the face image to be detected (the type of the side face degree of the face in the face image) can be predicted, and based on the offset and the anchor point, the face key point in the face image to be detected can be calculated.
In embodiments of the present invention, the foregoing feature extraction, classification, and learning of offsets/calculation of face keypoints may be implemented by different parts of neural networks (e.g., convolutional neural networks). for example, neural networks include a th sub-network, a second sub-network, and a third sub-network, where feature extraction is implemented by, for example, the th sub-network, classification is implemented by the second sub-network, and learning of offsets and/or calculation of face keypoints is implemented by the third sub-network.
In another embodiments of the invention, the aforementioned feature extraction, classification, and learning of offset/calculation of face keypoints can also be implemented by different neural networks, e.g., feature extraction is implemented by a neural network (e.g., a convolutional neural network), classification is implemented by a second neural network (e.g., a second convolutional neural network), learning of offset and/or calculation of face keypoints is implemented by a third neural network (e.g., a third convolutional neural network), wherein the output of the neural network is shared by the second and third neural networks, the output of the second neural network is further stepped into the third neural network, further , the third neural network can include a plurality of sub-networks, each of which can correspond to either the learning of side degrees of the offset or calculation of face keypoints of the face.
Based on the above description, the human face key point detection method based on anchor points according to the embodiment of the present invention classifies human face images with different side face degrees, and calculates human face key points of human face images with different side face degrees according to average human face key points (i.e. anchor points) with different side face degrees and position offset between real human face key points and anchor points under the side face degrees, so as to narrow the detection range of human face key points to smaller ranges, simplify the problem, and simultaneously fully utilize high and low layer spatial semantic information extracted by a neural network and spatial information provided by anchor points, so as to respectively optimize the situations with different side face degrees.
The above exemplarily describes the face key point detection method based on anchor points according to the embodiment of the present invention. Illustratively, the method for detecting human face key points based on anchor points according to the embodiments of the present invention can be implemented in a device, apparatus or system having a memory and a processor.
In addition, the anchor point-based face key point detection method can be conveniently deployed on mobile equipment such as smart phones, tablet computers and personal computers. Alternatively, the anchor-based face key point detection method according to the embodiment of the present invention may also be deployed at a server (or cloud). Alternatively, the anchor-based face key point detection method according to the embodiment of the present invention may also be distributively deployed at a server side (or a cloud side) and a personal terminal.
In the following, a description will be given, in conjunction with fig. 3, of an anchor-based face keypoint detection apparatus provided by another aspect of the present invention, fig. 3 shows a schematic block diagram of an anchor-based face keypoint detection apparatus 300 according to an embodiment of the present invention.
As shown in fig. 3, the anchor-based face keypoint detection apparatus 300 according to the embodiment of the present invention includes a feature extraction module 310, a classification module 320, and a keypoint calculation module 330, where the feature extraction module 310 is configured to obtain a face image to be detected and perform feature extraction on the face image, the classification module 320 is configured to classify the face image based on the extracted features to determine a type of a crossface degree of a face in the face image, each crossface degree corresponding to kinds of anchor points, the anchor points being a set of average face keypoints of the crossface degrees obtained in advance, the keypoint calculation module 330 is configured to determine an offset of the keypoints of the face in the face image with respect to the anchor points based on the extracted features and a result of the classification, and calculate face keypoints in the face image based on the anchor points and the offset, and the respective modules may respectively perform the respective steps/functions of the anchor-based face keypoint detection method described above with reference to fig. 2.
In an embodiment of the present invention, the face image obtained by the feature extraction module 310 may be a face image to be subjected to face keypoint detection, and the face image may include a video and/or a picture. Illustratively, the facial image may be acquired in real time or from any other source. In an embodiment of the present invention, the extraction of the features of the face image may be implemented by using a trained neural network.
In an embodiment of the present invention, based on the features of the face image extracted by the feature extraction module 310, the classification module 320 may classify the face image to determine what side degree the face image includes, that is, to determine the type of the side degree of the face in the face image. For example, the classification module 320 may employ a trained neural network to classify the type of the side face degree of the human face in the human face image. The neural network can be obtained by training face sample images with different side face degrees.
For simplicity, in embodiments of the present invention, the categories of the side-face degree are divided into three categories: left, front and right. Based on this, a neural network may be trained using face sample images including left, front, and right faces to classify the face image to be detected. Specifically, all face sample images can be divided into three subsets of left face, front face and right face (for example, corresponding to label values of 1, 2 and 3 respectively) to train classification. Illustratively, the classification loss function may use SoftmaxWithLoss:
wherein yi is a label value, pi is a neuron corresponding to the label of the input face sample image, and n is the number of the face sample images. Of course, this is merely exemplary, and any other suitable classification loss function may be employed.
In addition, may also include the case of a face, rather than just a side face, in order for a trained neural network to classify face images that include various face situations.
For example, given three (in real applications generally comprises a greater number, for example only for simplicity herein) facial images A, B and C that are all left-sided faces, the key facial points in image A are A1 through A68 (assuming that there are 68 face key points per facial image), the key facial points in image B are B1 through B68, the key facial points in image C are C1 through C68, the anchor points corresponding to the left-sided faces (i.e., the average face key points) are E1 through E2, where E1 has the coordinates of A1, B1, and C1. where the coordinates of the three key points are C638, the average key points are the coordinates of the key points, and the anchor points are the coordinates of the key points of the face of the same number, similar to the coordinates of the key points of the face of the anchor points of the same number, similar to the key points of the face of the anchor points of the face of the same .
In other words, each side face degree corresponds to anchor points, which are sets of average face key points of the side face degree obtained in advance.
In another examples, the averaging of face keypoints in the face sample images of each side face degree may include clustering the face sample images using a clustering method and using the resulting cluster center point as the anchor point.
Based on anchor points corresponding to different side face degrees obtained in advance and the classification result (i.e., determining the type of the side face degree of the face in the face image to be detected), the classification module 320 may obtain anchor points corresponding to the side face degree of the face in the face image to be detected, and then the keypoint calculation module 330 may perform face keypoint detection on the face image to be detected based on the anchor points.
In the embodiment of the invention, the offset of the real face key point (namely, the labeled face key point) corresponding to each side face degree relative to the anchor point can be learned in advance, the offset of the face key point to be detected in the face image to be detected relative to the anchor point can be predicted through the learning of the offset calculation, and the face key point in the face image to be detected is calculated according to the side face degree (namely, the anchor point) of the face image to be detected and the offset relative to the anchor point. Specifically, the pre-learning process may include: on the basis of previously finding the anchor point corresponding to each side face degree, feature extraction is performed on each human face sample image, and each human face sample image is classified based on the extracted features to determine the type of the side face degree of the human face in each human face sample image, so as to determine the anchor point corresponding to the type, and the process can be called a classification branch. Then, based on the features extracted from each face sample image and the classification result, the offset of the position between the face key point labeled in each face sample image and the average face key point in the anchor point corresponding to the face in the face sample image is trained and learned, and this process may be referred to as a regression branch. Illustratively, the loss function of the regression branch may use winloss:
wherein w, epsilon and c are hyperparameters, and x is the offset between the labeled face key point and the average face key point. Of course, this is merely exemplary, and any other suitable regression loss function may also be employed.
Based on the above pre-learning of the offset calculation, the key point calculation module 330 may calculate the face key points in the face image to be detected and anchor points corresponding to the face image to be detected (the type of the side face degree of the face in the face image), and based on the offset and the anchor points, the key point calculation module 330 may calculate the face key points in the face image to be detected.
In embodiments of the present invention, feature extraction module 310, classification module 320, and keypoint computation module 330 may be implemented by different portions of neural networks (e.g., convolutional neural networks). The feature extraction module 310 is implemented, for example, by the th subnetwork of neural networks, the classification module 320 is implemented by the second subnetwork of the neural networks, and the keypoint computation module 330 is implemented by the third subnetwork of the neural networks, where the output of the th subnetwork is shared by the second and third subnetworks.the output of the second subnetwork is further -step input to the third subnetwork -step, the third subnetwork may include a plurality of branch networks, each of which may correspond to the computation of the keypoints of human faces of side-face degrees.
In another embodiments of the invention, feature extraction module 310, classification module 320, and keypoint computation module 330 may also be implemented by different neural networks, e.g., feature extraction module 310 is implemented by a th neural network (e.g., a th convolutional neural network), classification module 320 is implemented by a second neural network (e.g., a second convolutional neural network), and keypoint computation module 330 is implemented by a third neural network (e.g., a third convolutional neural network), wherein the output of the th neural network is shared by the second and third neural networks, the output of the second neural network is further -step input to the third neural network, -step, the third neural network may include a plurality of sub-networks, each of which may correspond to the computation of the keypoints of human faces of side degrees.
Based on the above description, the anchor-based face key point detection device according to the embodiment of the present invention classifies face images of different side face degrees, and calculates face key points of the face images of different side face degrees according to average face key points (i.e., anchors) of different side face degrees and a position offset between a real face key point and an anchor point under the side face degrees to obtain face key points of the face images of different side face degrees, so that a detection range of the face key points is narrowed to smaller ranges, thereby simplifying the problem, and simultaneously, the high and low level spatial semantic information extracted by the neural network and the spatial information provided by the anchor point are fully utilized, so that the situation of different side face degrees can be optimized respectively, the neural network model extracts different facial features for side faces of different degrees, the extracted features have higher specificity and discriminability, on the premise of maintaining the detection precision of the face key points, the positioning precision of the side face key points is greatly improved, and meanwhile, the learning complexity of the neural network model is simplified by proposing the basic idea of anchor point.
Fig. 4 shows a schematic block diagram of an anchor-based face keypoint detection system 400, according to an embodiment of the invention. The anchor-based face keypoint detection system 400 comprises a storage 410 and a processor 420.
The storage device 410 stores a program for implementing corresponding steps in the anchor point-based face key point detection method according to the embodiment of the present invention. The processor 420 is configured to run a program stored in the storage device 410 to perform corresponding steps of the anchor-based face keypoint detection method according to the embodiment of the present invention, and is configured to implement corresponding modules in the anchor-based face keypoint detection device according to the embodiment of the present invention. Furthermore, the anchor-based face keypoint detection system 400 may further include an image acquisition device (not shown in fig. 4), which may be used to acquire the face image to be detected. Of course, the image capturing device is not necessary, and the anchor-based face keypoint detection system 400 may capture the face image to be detected from other external image capturing devices.
In embodiments of the present invention, when the program is run by processor 420, the anchor-based face keypoint detection system 400 performs the steps of obtaining a face image to be detected and extracting features of the face image, classifying the face image based on the extracted features to determine the type of the side face degrees of the face in the face image, each side face degree corresponding to types of anchors, the anchors being a set of average face keypoints of the side face degrees found in advance, determining the offset of the key points of the face in the face image relative to the anchors based on the extracted features and the classification result, and calculating the face keypoints in the face image based on the anchors and the offset.
In embodiments of the present invention, the pre-determining of the anchor point includes obtaining a plurality of face sample images containing different side-face degrees, and averaging face key points in the face sample images of each side-face degree to obtain a set of average face key points corresponding to each side-face degree, which is used as the anchor point corresponding to the side-face degree.
In embodiments of the present invention, the averaging of the face key points in the face sample images of each side face degree includes clustering the plurality of face sample images by using a clustering method, and using the obtained cluster center point as the anchor point, or calculating an arithmetic mean value of each face key point in the plurality of face sample images of any side face degree, and using a set of the arithmetic mean values of each face key point in the plurality of face sample images of any side face degree as the anchor point corresponding to the any side face degree.
In embodiments of the present invention, when the program is executed by the processor 420, the steps performed by the anchor-based face keypoint detection system 400 are implemented by trained neural networks, including a th sub-network, a second sub-network, and a third sub-network, wherein the th sub-network is configured to obtain a face image to be detected and perform feature extraction on the face image, the second sub-network is configured to classify the face image to be detected based on features extracted by the th sub-network to determine a category of a side face degree of a face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face keypoints of the side face degree obtained in advance, and the third sub-network is configured to determine an offset of a face keypoint in the face image to be detected relative to the anchor point based on the features extracted by the th sub-network and the classification result of the second sub-network, and calculate an offset of the face keypoint in the anchor point to be detected based on the face image to be detected and the anchor point.
In embodiments of the invention, the third subnetwork further includes a plurality of branch networks, each branch network corresponding to side-face degree determinations of the offset and face keypoints calculations.
In embodiments of the present invention, when the program is executed by the processor 420, the steps performed by the anchor point-based face keypoint detection system 400 are implemented by a plurality of trained neural networks, including a -th neural network, a second neural network and a third neural network, wherein the -th neural network is configured to obtain a face image to be detected and perform feature extraction on the face image, the second neural network is configured to classify the face image to be detected based on the features extracted by the -th neural network so as to determine the type of the side face degrees of the face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average keypoints of the side face degrees obtained in advance, and the third neural network is configured to determine the offset of the keypoints of the face in the face image to be detected relative to the face image to be detected based on the features extracted by the -th neural network and the classification result of the second neural network, and calculate the keypoint of the face in the face image to be detected based on the anchor points and the anchor points.
In embodiments of the invention, the third neural network further includes a plurality of sub-networks, each sub-network corresponding to the determination of the offset of side-face degrees and the calculation of face keypoints.
In embodiments of the invention, the categories of the side-face degree include left side face, front face, and right side face.
Furthermore, storage media are provided according to embodiments of the present invention, on which program instructions are stored, which when executed by a computer or a processor, are used to perform the corresponding steps of the anchor point-based human face keypoint detection method according to embodiments of the present invention, and to implement the corresponding modules in the anchor point-based human face keypoint detection apparatus according to embodiments of the present invention.
In embodiments, the computer program instructions, when executed by a computer, may implement the functional modules of the anchor-based face keypoint detection apparatus according to the embodiments of the present invention, and/or may execute the anchor-based face keypoint detection method according to the embodiments of the present invention.
In embodiments of the present invention, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the steps of obtaining a face image to be detected and extracting features of the face image, classifying the face image based on the extracted features to determine a type of a side face degree of a face in the face image, each side face degree corresponding to anchor points, the anchor points being a set of average face key points of the side face degree obtained in advance, determining an offset of key points of the face in the face image relative to the anchor points based on the extracted features and the classification result, and calculating the face key points in the face image based on the anchor points and the offset.
In embodiments of the present invention, the pre-determining of the anchor point includes obtaining a plurality of face sample images containing different side-face degrees, and averaging face key points in the face sample images of each side-face degree to obtain a set of average face key points corresponding to each side-face degree, which is used as the anchor point corresponding to the side-face degree.
In embodiments of the present invention, the averaging of the face key points in the face sample images of each side face degree includes clustering the plurality of face sample images by using a clustering method, and using the obtained cluster center point as the anchor point, or calculating an arithmetic mean value of each face key point in the plurality of face sample images of any side face degree, and using a set of the arithmetic mean values of each face key point in the plurality of face sample images of any side face degree as the anchor point corresponding to the any side face degree.
In embodiments of the present invention, the computer program instructions when executed by a computer or a processor cause the computer or the processor to perform the steps implemented by trained neural networks including a th sub-network, a second sub-network and a third sub-network, wherein the th sub-network is configured to acquire a face image to be detected and perform feature extraction on the face image, the second sub-network is configured to classify the face image to be detected based on the features extracted by the th sub-network to determine a category of a side face degree of a face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face key points of the side face degree, the set of average face key points is obtained in advance, and the third sub-network is configured to determine an offset of a key point of the face in the face image to be detected relative to the anchor point based on the features extracted by the th sub-network and the classification result of the second sub-network, and calculate an offset of the face in the face image to be detected based on the anchor point and the anchor point.
In embodiments of the invention, the third subnetwork further includes a plurality of branch networks, each branch network corresponding to side-face degree determinations of the offset and face keypoints calculations.
In embodiments of the present invention, the computer program instructions when executed by a computer or a processor cause the computer or the processor to perform the steps implemented by a plurality of trained neural networks including a neural network, a second neural network and a third neural network, wherein the neural network is configured to obtain a face image to be detected and perform feature extraction on the face image, the second neural network is configured to classify the face image to be detected based on the extracted features of the neural network to determine a category of a side face degree of a face in the face image to be detected, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face key points of the side face degree obtained in advance, and the third neural network is configured to determine an offset of key points of the face in the face image to be detected relative to the anchor points based on the features extracted by the neural network and the classification result of the second neural network, and calculate an offset of the face key points in the face image to be detected based on the anchor points and the classification result of the anchor points and the second neural network.
In embodiments of the invention, the third neural network further includes a plurality of sub-networks, each sub-network corresponding to the determination of the offset of side-face degrees and the calculation of face keypoints.
In embodiments of the invention, the categories of the side-face degree include left side face, front face, and right side face.
Modules in the anchor point-based face keypoint detection apparatus according to the embodiment of the present invention may be implemented by a processor of the anchor point-based face keypoint detection electronic device according to the embodiment of the present invention running computer program instructions stored in a memory, or may be implemented when computer instructions stored in a computer-readable storage medium of a computer program product according to the embodiment of the present invention are run by a computer.
computer programs, which may be stored on a storage medium in a cloud or a local area, are provided according to embodiments of the present invention, and when the computer programs are executed by a computer or a processor, the computer programs are configured to perform the corresponding steps of the anchor point-based face keypoint detection method according to embodiments of the present invention, and to implement the corresponding modules in the anchor point-based face keypoint detection apparatus according to embodiments of the present invention.
Based on the above description, the method, the device and the system for detecting face key points based on anchor points of the embodiment of the invention classify the face images with different side face degrees, and calculate the face key points of the face images with different side face degrees according to the average face key points (i.e. anchor points) with different side face degrees and the position offset between the real face key points and the anchor points under the side face degrees, so that the detection range of the face key points is narrowed to smaller ranges, the problem is simplified, meanwhile, the high-low layer spatial semantic information extracted by the neural network and the spatial information provided by the anchor points are fully utilized, the situations with different side face degrees can be respectively optimized, the neural network model extracts different facial features aiming at the side faces with different degrees, the extracted features are more targeted and discriminative, the positioning accuracy of the side face key points is greatly improved on the premise of keeping the detection accuracy of the front face key points, and the learning complexity of the neural network model is simplified by proposing the basic idea of anchor point division.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the foregoing illustrative embodiments are merely exemplary and are not intended to limit the scope of the invention thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
For example, the above-described embodiments of the apparatus are merely illustrative, e.g., the division of the units into logical functional divisions, and other divisions may be possible in practice, e.g., multiple units or components may be combined or integrated into another devices, or features may be omitted, or not performed.
However, it is understood that embodiments of the invention may be practiced without these specific details, and that examples well-known methods, structures, and techniques have not been shown in detail in order not to obscure the understanding of this description.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together by in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of the various inventive aspects, however, the method of the present invention is not to be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include some features included in other embodiments, not others, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.
It should be understood by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement or all of the functions of modules according to embodiments of the present invention may also be implemented as an apparatus program (e.g., a computer program or computer program product) for performing part or all of the methods described herein such a program embodying the present invention may be stored on a computer readable medium or may be in the form of 2 or more signals 539 such signals may be downloaded from an internet website or provided on a carrier signal or provided in any other form.
The invention may be embodied by means of hardware comprising several distinct elements, and by means of a suitably programmed computer, in a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware, the use of the words , second, third, etc. may indicate any sequence.
The above description is only for the specific embodiment of the present invention or the description thereof, and the protection scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1, human face key point detection method based on anchor point, characterized in that, the method includes:
acquiring a face image to be detected, and extracting the features of the face image;
classifying the face image based on the extracted features to determine the type of the side face degree of the face in the face image, each side face degree corresponding to kinds of anchor points, the anchor points being a set of average face key points of the side face degree obtained in advance, and
and determining the offset of the key points of the face in the face image relative to the anchor point based on the extracted features and the classification result, and calculating the key points of the face in the face image based on the anchor point and the offset.
2. The method of claim 1, wherein the pre-determining of the anchor point comprises:
and acquiring a plurality of face sample images containing different side face degrees, and averaging face key points in the face sample images of each side face degree to obtain a set of average face key points corresponding to each side face degree, wherein the set of average face key points is used as an anchor point corresponding to the side face degree.
3. The method of claim 2, wherein averaging the face keypoints in the face sample images for each side-face degree comprises:
clustering the multiple face sample images by using a clustering method, and taking the obtained clustering center point as the anchor point; or
And calculating an arithmetic mean value of each face key point in a plurality of face sample images with any side face degree, and taking a set of the arithmetic mean values of each face key point in the plurality of face sample images with any side face degree as an anchor point corresponding to any side face degree.
4. The method of any of claims 1-3, wherein the method is implemented by trained neural networks, the neural networks including a th sub-network, a second sub-network, and a third sub-network, wherein:
the th sub-network is used for acquiring a face image to be detected and extracting the features of the face image;
the second sub-network is used for classifying the face image to be detected based on the characteristics extracted by the th sub-network so as to determine the type of the side face degree of the face in the face image to be detected, wherein each side face degree corresponds to anchor points, and the anchor points are a set of average face key points of the side face degree obtained in advance;
the third sub-network is configured to determine, based on the features extracted by the th sub-network and the classification result of the second sub-network, an offset of a key point of a face in the face image to be detected with respect to the anchor point, and calculate, based on the anchor point and the offset, a key point of the face in the face image to be detected.
5. The method of claim 4, wherein the third subnetwork further comprises a plurality of branch networks, each branch network corresponding to the determination of the offset of side-face degrees and the calculation of face keypoints.
6. The method of any of claims 1-3, wherein the method is implemented by a trained plurality of neural networks, the plurality of neural networks including a th neural network, a second neural network, and a third neural network, wherein:
the neural network is used for obtaining the face image to be detected and extracting the features of the face image;
the second neural network is used for classifying the face image to be detected based on the characteristics extracted by the th neural network so as to determine the type of the side face degree of the face in the face image to be detected, wherein each side face degree corresponds to anchor points, and the anchor points are a set of average face key points of the side face degree obtained in advance;
the third neural network is used for determining the offset of the key points of the face in the face image to be detected relative to the anchor points based on the features extracted by the th neural network and the classification result of the second neural network, and calculating the key points of the face in the face image to be detected based on the anchor points and the offset.
7. The method of claim 6, wherein the third neural network further includes a plurality of sub-networks, each sub-network corresponding to the determination of the offset of side-face degrees and the calculation of face keypoints.
8. The method of any of claims , wherein the categories of side-face degrees include left-side face, front face, and right-side face.
An anchor point-based human face key point detection device of 9, kinds, characterized in that, the device includes:
the characteristic extraction module is used for acquiring a face image to be detected and extracting the characteristics of the face image;
a classification module for classifying the face image based on the extracted features to determine the type of the side face degree of the face in the face image, each side face degree corresponds to kinds of anchor points, the anchor points are a set of average face key points of the side face degree obtained in advance, and
and the key point calculating module is used for determining the offset of the key points of the face in the face image relative to the anchor point based on the extracted features and the classification result, and calculating the key points of the face in the face image based on the anchor point and the offset.
10, Anchor-based face keypoint detection system, comprising a storage means and a processor, said storage means having stored thereon a computer program for execution by said processor, said computer program, when executed by said processor, performing the anchor-based face keypoint detection method according to any of claims 1-8.
Storage medium of 11, , characterized in that, the storage medium has stored thereon a computer program which, when run, executes the anchor point based face keypoint detection method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910859973.3A CN110738110A (en) | 2019-09-11 | 2019-09-11 | Human face key point detection method, device, system and storage medium based on anchor point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910859973.3A CN110738110A (en) | 2019-09-11 | 2019-09-11 | Human face key point detection method, device, system and storage medium based on anchor point |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110738110A true CN110738110A (en) | 2020-01-31 |
Family
ID=69267625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910859973.3A Pending CN110738110A (en) | 2019-09-11 | 2019-09-11 | Human face key point detection method, device, system and storage medium based on anchor point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110738110A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111695519A (en) * | 2020-06-12 | 2020-09-22 | 北京百度网讯科技有限公司 | Key point positioning method, device, equipment and storage medium |
| CN112733671A (en) * | 2020-12-31 | 2021-04-30 | 新大陆数字技术股份有限公司 | Pedestrian detection method, device and readable storage medium |
| CN112733700A (en) * | 2021-01-05 | 2021-04-30 | 风变科技（深圳）有限公司 | Face key point detection method and device, computer equipment and storage medium |
| CN113378852A (en) * | 2021-05-24 | 2021-09-10 | 北京迈格威科技有限公司 | Key point detection method and device, electronic equipment and storage medium |
| CN113674139A (en) * | 2021-08-17 | 2021-11-19 | 北京京东尚科信息技术有限公司 | Face image processing method and device, electronic equipment and storage medium |
| CN113762168A (en) * | 2021-09-09 | 2021-12-07 | 南京领行科技股份有限公司 | Safety alarm method, device, electronic equipment and storage medium |
| CN115426505A (en) * | 2022-11-03 | 2022-12-02 | 北京蔚领时代科技有限公司 | Preset expression special effect triggering method based on face capture and related equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107358172A (en) * | 2017-06-22 | 2017-11-17 | 华南理工大学 | A kind of human face characteristic point initial method based on facial orientation classification |
| WO2018107979A1 (en) * | 2016-12-14 | 2018-06-21 | 华南理工大学 | Multi-pose human face feature point detection method based on cascade regression |
| CN108960136A (en) * | 2018-06-29 | 2018-12-07 | 杭州西纬软件科技有限公司 | The determination method and apparatus of Initial Face shape in face alignment algorithm |
| CN109558837A (en) * | 2018-11-28 | 2019-04-02 | 北京达佳互联信息技术有限公司 | Face critical point detection method, apparatus and storage medium |
| CN109858466A (en) * | 2019-03-01 | 2019-06-07 | 北京视甄智能科技有限公司 | A kind of face critical point detection method and device based on convolutional neural networks |
| CN109919097A (en) * | 2019-03-08 | 2019-06-21 | 中国科学院自动化研究所 | Face and key point combined detection system, method based on multi-task learning |
-
2019
- 2019-09-11 CN CN201910859973.3A patent/CN110738110A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018107979A1 (en) * | 2016-12-14 | 2018-06-21 | 华南理工大学 | Multi-pose human face feature point detection method based on cascade regression |
| CN107358172A (en) * | 2017-06-22 | 2017-11-17 | 华南理工大学 | A kind of human face characteristic point initial method based on facial orientation classification |
| CN108960136A (en) * | 2018-06-29 | 2018-12-07 | 杭州西纬软件科技有限公司 | The determination method and apparatus of Initial Face shape in face alignment algorithm |
| CN109558837A (en) * | 2018-11-28 | 2019-04-02 | 北京达佳互联信息技术有限公司 | Face critical point detection method, apparatus and storage medium |
| CN109858466A (en) * | 2019-03-01 | 2019-06-07 | 北京视甄智能科技有限公司 | A kind of face critical point detection method and device based on convolutional neural networks |
| CN109919097A (en) * | 2019-03-08 | 2019-06-21 | 中国科学院自动化研究所 | Face and key point combined detection system, method based on multi-task learning |
Non-Patent Citations (1)
| Title |
|---|
| 徐彬彬: "《一种基于卷积神经网络人脸朝向预判的人脸关键点检测方法》", 《中国优秀博硕士学位论文全文数据库（硕士），信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111695519A (en) * | 2020-06-12 | 2020-09-22 | 北京百度网讯科技有限公司 | Key point positioning method, device, equipment and storage medium |
| CN111695519B (en) * | 2020-06-12 | 2023-08-08 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for positioning key point |
| CN112733671A (en) * | 2020-12-31 | 2021-04-30 | 新大陆数字技术股份有限公司 | Pedestrian detection method, device and readable storage medium |
| CN112733700A (en) * | 2021-01-05 | 2021-04-30 | 风变科技（深圳）有限公司 | Face key point detection method and device, computer equipment and storage medium |
| CN112733700B (en) * | 2021-01-05 | 2024-07-09 | 风变科技（深圳）有限公司 | Face key point detection method, device, computer equipment and storage medium |
| CN113378852A (en) * | 2021-05-24 | 2021-09-10 | 北京迈格威科技有限公司 | Key point detection method and device, electronic equipment and storage medium |
| WO2022247403A1 (en) * | 2021-05-24 | 2022-12-01 | 北京迈格威科技有限公司 | Keypoint detection method, electronic device, program, and storage medium |
| CN113674139A (en) * | 2021-08-17 | 2021-11-19 | 北京京东尚科信息技术有限公司 | Face image processing method and device, electronic equipment and storage medium |
| CN113762168A (en) * | 2021-09-09 | 2021-12-07 | 南京领行科技股份有限公司 | Safety alarm method, device, electronic equipment and storage medium |
| CN115426505A (en) * | 2022-11-03 | 2022-12-02 | 北京蔚领时代科技有限公司 | Preset expression special effect triggering method based on face capture and related equipment |
| CN115426505B (en) * | 2022-11-03 | 2023-03-24 | 北京蔚领时代科技有限公司 | Preset expression special effect triggering method based on face capture and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110738110A (en) | Human face key point detection method, device, system and storage medium based on anchor point | |
| US11308334B2 (en) | Method and apparatus for integration of detected object identifiers and semantic scene graph networks for captured visual scene behavior estimation | |
| CN113056743B (en) | Training neural networks for vehicle re-identification | |
| Elharrouss et al. | Gait recognition for person re-identification | |
| CN107704877B (en) | Image privacy perception method based on deep learning | |
| CN109145766B (en) | Model training method and device, recognition method, electronic device and storage medium | |
| CN108875511B (en) | Image generation method, device, system and computer storage medium | |
| CN109241888B (en) | Neural network training and object recognition method, device and system and storage medium | |
| Zhou et al. | Semi-supervised salient object detection using a linear feedback control system model | |
| CN110222572B (en) | Tracking method, tracking device, electronic equipment and storage medium | |
| Pazho et al. | Ancilia: Scalable intelligent video surveillance for the artificial intelligence of things | |
| EP4085369A1 (en) | Forgery detection of face image | |
| CN110245714B (en) | Image recognition method and device and electronic equipment | |
| Ashok Kumar et al. | Enhanced facial emotion recognition by optimal descriptor selection with neural network | |
| US10659680B2 (en) | Method of processing object in image and apparatus for same | |
| US20180279914A1 (en) | Gait pathology detection and monitoring system, and method | |
| CN111738225B (en) | Crowd gathering detection method, device, equipment and storage medium | |
| US20160063345A1 (en) | Pattern recognition apparatus, pattern recognition method, and storage medium | |
| Xu et al. | Bottom-up saliency detection with sparse representation of learnt texture atoms | |
| Lecca et al. | Comprehensive evaluation of image enhancement for unsupervised image description and matching | |
| Yu et al. | Multibranch feature difference learning network for cross-spectral image patch matching | |
| Zhong et al. | Markov chain based computational visual attention model that learns from eye tracking data | |
| Liang et al. | Improvements and experiments of a compact statistical background model | |
| Guraya et al. | Neural networks based visual attention model for surveillance videos | |
| CN109214271B (en) | Method and device for determining loss function for re-identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |