CN110826421A - Method and device for filtering faces with difficult postures - Google Patents

Method and device for filtering faces with difficult postures Download PDF

Info

Publication number
CN110826421A
CN110826421A CN201910991099.9A CN201910991099A CN110826421A CN 110826421 A CN110826421 A CN 110826421A CN 201910991099 A CN201910991099 A CN 201910991099A CN 110826421 A CN110826421 A CN 110826421A
Authority
CN
China
Prior art keywords
face
key point
point position
position set
distance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910991099.9A
Other languages
Chinese (zh)
Other versions
CN110826421B (en
Inventor
邓卉
田泽康
危明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yi Teng Teng Polytron Technologies Inc
Original Assignee
Yi Teng Teng Polytron Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yi Teng Teng Polytron Technologies Inc filed Critical Yi Teng Teng Polytron Technologies Inc
Priority to CN201910991099.9A priority Critical patent/CN110826421B/en
Publication of CN110826421A publication Critical patent/CN110826421A/en
Application granted granted Critical
Publication of CN110826421B publication Critical patent/CN110826421B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method for filtering a face with a difficult posture, which comprises the following steps: detecting a face to obtain a face rectangular position; using a first and a second face key positioning technologies to detect face key points at the rectangular positions of the face, and acquiring a first and a second key point position sets; acquiring a similarity transformation matrix for converting the first key point position set into an average face key point position set; according to the similarity transformation matrix, transforming the first and second key point position sets to the face average face to obtain first and second transformation key point position sets; and calculating the distance between every two of the first and second transformation key point position sets and the average face key point position set, and when judging that the distances are smaller than a threshold value, the image containing the face is a candidate face easy to pose, or else, the image is a difficult pose face. The face with the difficult posture is screened out, so that the accuracy of face recognition is guaranteed. Corresponding apparatus, devices and media are also provided.

Description

Method and device for filtering faces with difficult postures
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a device for filtering a face with a difficult posture, a computer readable medium and electronic equipment.
Background
The human face is a very important biological feature, has the characteristics of complex structure, various detail changes and the like, and also contains a large amount of information. At present, face recognition and analysis are widely applied, and technologies such as automatic face recognition, facial expression analysis, three-dimensional face reconstruction and the like are vigorously developed.
However, when the human face has a difficult pose such as an extreme side face, the accuracy of the human face recognition and analysis may be reduced. Especially on a platform (such as a mobile terminal) which has high real-time requirement and weak computing capability, a very complex network cannot be used for face recognition and analysis, and the recognition accuracy rate is greatly reduced when a difficult face gesture occurs.
The difficult pose face sample (for example, as shown in fig. 2) is a face sample that is difficult to recognize and analyze because the face information presented in the camera is very small when the face rotation angle is very large.
The current method for filtering the face with difficult pose is to position key points of the face, then calculate the pose of the face according to the key points, and judge the pose of the face according to the pose. However, in the case of an extreme side face or a face pose with too large an angle, the face keypoint location method locates the wrong position, so the determined pose is also wrong.
Disclosure of Invention
Aiming at solving the defects of the prior art, the invention provides a method for cross-verifying the human face with the difficult gesture by using two methods to eliminate the difficult gesture, ensure the accuracy of key point positioning and further ensure the accuracy of human face recognition and analysis.
Specifically, the invention provides a method for filtering a face with a difficult posture, which comprises the following steps:
s110, acquiring an image containing a human face;
s120, carrying out face detection on the image containing the face to obtain a face rectangular position;
s130, performing face key point detection on the rectangular position of the face by using a first face key point positioning technology to obtain a first key point position set;
s140, performing face key point detection on the rectangular position of the face by using a second face key positioning technology to obtain a second key point position set;
s150, obtaining a human face average face and a corresponding average face key point position set, and obtaining a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set;
s160, transforming the first key point position set and the second key point position set to the face average face according to the similarity transformation matrix to obtain a first transformation key point position set and a second transformation key point position set;
s170, calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, obtaining a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-pose face when judging that the first distance, the second distance and the third distance are all smaller than a threshold value, otherwise, the image containing the face is a difficult-pose face.
Further, still include:
s180, when the image containing the face is a candidate face easy to pose, judging whether the corresponding first key point position set and the corresponding second key point position set are projected to a two-dimensional plane in the range of the image containing the face, and when the first key point position set and the second key point position set are not projected to the two-dimensional plane, judging that the candidate face easy to pose is a face difficult to pose.
Further, the step of calculating the distance between each two of the first transformation keypoint location set, the second transformation keypoint location set, and the average face keypoint location set includes:
s171, obtaining a distance vector and a distance of the minimum point pair according to the minimum point pair with the minimum European distance between the first group of key points and the second group of key points;
s172, removing the minimum point pair from the first group of key points and the second group of key points;
s173, according to the distance vector, performing translation processing on the first group of key points and the second group of key points after the minimum point pairs are removed to obtain a first group of key points and a second group of key points after transformation;
s174, taking the transformed first group of key points and second group of key points as the first group of key points and second group of key points in step S171, repeating steps S171 to S173 until the key points in the first group of key points and second group of key points are removed, and adding the distances of the minimum point pairs obtained in each iteration as the distances between the first group of key points and second group of key points.
Further, the distance in step S170 includes one or more of a hamming distance, a euclidean distance, and a mahalanobis distance.
Further, the step S180 specifically includes determining whether the nose point in the first set of key point positions and the nose point in the second set of key point positions are projected onto the two-dimensional plane within the range of the image including the face, and if not, determining that the candidate face easy to pose is a face difficult to pose.
Further, the step of judging whether the projection of the nose tip point to the two-dimensional plane is in the image range containing the human face comprises the following steps:
and judging whether the nose tip point is in a rectangle consisting of a left eye center, a right eye center, a left mouth corner and a right mouth corner, if so, projecting the nose tip point to a two-dimensional plane in the image range containing the face, otherwise, not in the image range containing the face.
Further, the first face key point positioning technique and the second face key point positioning technique include: the first face key point positioning technology is different from the second face key point positioning technology based on one or more of a Convolutional Neural Network (CNN) algorithm, a Supervised Descent (SDM) algorithm, a subjective shape model (ASM) algorithm and a cascade row propagation regression-based method.
In another aspect of the present invention, a device for filtering a difficult pose face is provided, which includes:
the receiving module is used for acquiring an image containing a human face;
the face rectangle detection module is used for carrying out face detection on the image containing the face to acquire a face rectangle position;
the first key point extraction module is used for detecting the key points of the face on the rectangular position of the face by using a first face key point positioning technology to obtain a first key point position set;
the second key point extraction module is used for detecting the key points of the face at the rectangular position of the face by using a second face key positioning technology to obtain a second key point position set;
the similarity transformation matrix calculation module is used for acquiring a human face average face and a corresponding average face key point position set, and acquiring a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set;
the key point transformation module is used for transforming the first key point position set and the second key point position set to the face average face according to the similarity transformation matrix to obtain a first transformation key point position set and a second transformation key point position set;
and the screening module is used for calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, acquiring a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-posture face when judging that the first distance, the second distance and the third distance are smaller than threshold values, otherwise, the face is a difficult-posture face.
In a third aspect of the present invention, there is provided an electronic device comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement any of the methods described above.
In a fourth aspect of the invention, a computer-readable medium is provided, on which a computer program is stored, wherein the program, when executed by a processor, implements any of the methods described above.
The method for filtering the face sample with the difficult gesture provided by the embodiment of the invention carries out cross validation by two recognition methods, and has the following beneficial effects:
1. the faces with difficult postures are screened out, and the error rate of face recognition and analysis is reduced.
2. The system can be applied to various mobile terminals, such as smart phones and tablet computers.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the invention in any way, and in which:
FIG. 1 is a schematic diagram of a system architecture for a method and apparatus for filtering faces with difficult poses according to some embodiments of the present invention;
FIG. 2 is a schematic diagram of a difficult pose face in some examples of the invention;
FIG. 3 is a flow diagram of a method for filtering difficult pose faces in some embodiments of the invention;
FIG. 4 is a flow chart illustrating a method for filtering faces with difficult poses according to other embodiments of the present invention;
FIG. 5 is a schematic diagram illustrating a process flow of distance calculation in a method for filtering a difficult-posed face according to another embodiment of the present invention;
FIG. 6 is a system diagram of a face device for filtering difficult poses implemented based on the face filtering method of the above figures according to some embodiments of the present invention;
FIG. 7 is a block diagram of a computer system on which a method or apparatus for filtering faces with difficult poses according to some embodiments of the present invention may operate;
FIG. 8 is a schematic diagram of a result of extracting face rectangles in a method for filtering a difficult-pose face according to some embodiments of the present invention;
FIG. 9 is a diagram illustrating key point extraction results of a method for filtering difficult pose faces in some embodiments of the invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the method of filtering difficult-posed faces or the apparatus for filtering difficult-posed faces of embodiments of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 101, 102, 103 to interact with a server 105 over a network 104 to receive or transmit data (e.g., video), etc. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as video playing software, video processing applications, web browser applications, shopping applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting data transmission, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, such as a background server providing support for videos displayed on the terminal devices 101, 102, 103. The background server may analyze and otherwise process data such as the received image processing request, and feed back a processing result (e.g., a video clip obtained by segmenting a video or other data) to an electronic device (e.g., a terminal device) communicatively connected to the background server.
It should be noted that the method for filtering the difficult-posed face provided by the embodiment of the present application may be executed by the server 105, and accordingly, a filtering difficult-posed face device may be disposed in the server 105. In addition, the method for filtering the faces with difficult postures provided by the embodiment of the application can also be executed by the terminal equipment 101, 102 and 103, and accordingly, the device for filtering the faces with difficult postures can also be arranged in the terminal equipment 101, 102 and 103.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. When the electronic device on which the method for filtering the difficult-posed face does not need to perform data transmission with other electronic devices, the system architecture may only include the electronic device (e.g., the terminal device 101, 102, 103 or the server 105) on which the method for filtering the difficult-posed face is performed.
FIG. 3 shows a general flow of an algorithm for filtering difficult pose faces according to an embodiment of the present invention, which includes the following specific steps:
firstly, reading an image containing a human face.
And secondly, carrying out face detection on the image by using a face detection technology, and outputting the detected face rectangular position. As a result, as shown in fig. 8, each face frame is subjected to subsequent operations individually to determine whether the face is difficult to pose. And if the human face cannot be detected, the subsequent processing is not carried out.
And thirdly, performing face key point detection on the detected face by using a first face key point positioning technology to obtain a shape of the positions of a plurality of key points on the face, wherein the shape is a coordinate position set of a series of predefined face key points in the face image.
And fourthly, performing face key point detection on the detected face by using a second face key point positioning technology to obtain the positions of a plurality of key points on the face.
The result of the keypoint detection is shown in fig. 9, which indicates each keypoint.
And fifthly, calculating a similarity transformation matrix Trans of the key points of the human face obtained by the first human face key point positioning technology to the key points of the human face average face according to the known human face average face and the human face key point avgShape.
And sixthly, obtaining a similarity transformation matrix Trans for transforming the positioned face to the average face according to the first positioning method, and respectively transforming the face key points obtained by the first and second face key point positioning technologies to the average face to obtain shape1 and shape 2.
And seventhly, calculating the distance between every two of shape1, shape2 and the face key point avgShape of the face average face as described in the fifth and sixth steps. Defining the distance between shape1 and shape2 as dist12Distance between shape1 and avgShape is dist1aDistance between shape2 and avgShape is dist2a. If the distances between the three are small (specifically denoted as dist)12Less than threshold th1, and dist1aLess than threshold th2, and dist2aLess than threshold th3), then this face is a candidate easy-pose face, otherwise it is a difficult-pose face.
Wherein, calculating the distance between two groups of facial key points uses a custom shape edit distance (shape edit distance). The method comprises the following specific steps:
we define two groups of facial key points as
Figure BDA0002238316120000061
Wherein pt isiAre the pixel coordinates of key points predefined by the face.
1. Finding out a pair of points with the minimum Euclidean distance between the first set of key points shape1 and the second set of key points shape2
Figure BDA0002238316120000062
And calculate that
Figure BDA0002238316120000063
Move to
Figure BDA0002238316120000064
And the Euclidean distance dist of the pair of pointsi。
2. Will be provided with
Figure BDA0002238316120000065
Removed from shape1 and shape2 so that they do not participate in subsequent distance calculations.
3. Each point in the first set of keypoints shape1 is moved by a distance (dx, dy), resulting in a moved keypoint set shape 1'.
4. Returning to the step 1, calculating the Euclidean distance of a point with the minimum distance for the two groups of new key points shape1 'and shape2', and completing the steps 2 and 3 to form two groups of new key points.
So as to iterate until shape1 in step 4’And shape2' is empty, i.e., the distance of all pairs of keypoints in both sets is calculated. Will be at a timeIteratively derived distance dist of a pair of pointsiAnd adding the two groups of face key points to obtain the shape editing distance between the two groups of face key points.
And eighthly, calculating whether the nose cusp is projected to the two-dimensional plane in the face of the candidate easy-pose face in the step seven. Setting flag to 1 indicates that the nose tip point is projected into the face plane after the two-dimensional plane, and setting flag to 0 indicates absence. Whether the calculated nose cusp projected to the two-dimensional plane by the key points obtained by the first and second face key point positioning technologies is in the face is respectively represented as flag1 and flag 2. This candidate face is determined to be a difficult pose face when flag1 is 0 and flag2 is 0.
The method for calculating whether the nose tip point is projected into the two-dimensional plane or not in the human face plane comprises the following steps: firstly, obtaining a positioned human face key point, and then judging whether the nose tip is in a rectangle formed by four points of a left eye center, a right eye center, a left mouth corner and a right mouth corner. If the nose tip is in the rectangle, the nose tip is in the face plane, otherwise, the nose tip is not in the face plane.
Referring to fig. 4, another embodiment of the present application is related to a face algorithm for filtering difficult poses, which includes the following specific steps:
s110, acquiring an image containing a human face;
s120, carrying out face detection on the image containing the face to obtain a face rectangular position;
s130, performing face key point detection on the rectangular position of the face by using a first face key point positioning technology to obtain a first key point position set;
s140, performing face key point detection on the rectangular position of the face by using a second face key positioning technology to obtain a second key point position set;
s150, obtaining a human face average face and a corresponding average face key point position set, and obtaining a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set;
s160, transforming the first key point position set and the second key point position set to the face average face according to the similarity transformation matrix to obtain a first transformation key point position set and a second transformation key point position set;
s170, calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, obtaining a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-pose face when judging that the first distance, the second distance and the third distance are all smaller than a threshold value, otherwise, the image containing the face is a difficult-pose face.
Further, still include:
s180, when the image containing the face is a candidate face easy to pose, judging whether the corresponding first key point position set and the corresponding second key point position set are projected to a two-dimensional plane in the range of the image containing the face, and when the first key point position set and the second key point position set are not projected to the two-dimensional plane, judging that the candidate face easy to pose is a face difficult to pose. The face with the difficult posture can be screened out more finely through the step.
Further, as shown in fig. 5, the step of calculating the distance between each two of the first transformation keypoint location set, the second transformation keypoint location set, and the average face keypoint location set includes:
s171, obtaining a distance vector and a distance of the minimum point pair according to the minimum point pair with the minimum European distance between the first group of key points and the second group of key points;
s172, removing the minimum point pair from the first group of key points and the second group of key points;
s173, according to the distance vector, performing translation processing on the first group of key points and the second group of key points after the minimum point pairs are removed to obtain a first group of key points and a second group of key points after transformation;
s174, taking the transformed first group of key points and second group of key points as the first group of key points and second group of key points in step S171, repeating steps S171 to S173 until the key points in the first group of key points and second group of key points are removed, and adding the distances of the minimum point pairs obtained in each iteration as the distances between the first group of key points and second group of key points.
The steps S171 to S174 are processes of calculating the shape editing distance in the embodiment of the present invention, and the present invention represents the distance between the sets of the keypoint locations by the shape editing distance.
Further, the distance in step S170 may further include one or more of a hamming distance, a euclidean distance, and a mahalanobis distance.
Further, the step S180 specifically includes determining whether the nose point in the first set of key point positions and the nose point in the second set of key point positions are projected onto the two-dimensional plane within the range of the image including the face, and if not, determining that the candidate face easy to pose is a face difficult to pose.
Further, the step of judging whether the projection of the nose tip point to the two-dimensional plane is in the image range containing the human face comprises the following steps:
and judging whether the nose tip point is in a rectangle consisting of a left eye center, a right eye center, a left mouth corner and a right mouth corner, if so, projecting the nose tip point to a two-dimensional plane in the image range containing the face, otherwise, not in the image range containing the face.
Further, the first face key point positioning technique and the second face key point positioning technique include: the first face key point positioning technology is different from the second face key point positioning technology based on one or more of a Convolutional Neural Network (CNN) algorithm, a Supervised Descent (SDM) algorithm, a subjective shape model (ASM) algorithm and a cascade row propagation regression-based method.
The method for filtering the face with the difficult posture acquires key points through two positioning technologies, then maps two groups of key points to an average face for distance calculation, judges whether the face with the difficult posture is the face with the difficult posture according to the distance between every two key points, further performs projection verification in the process of easily identifying the face, and refines and screens the face with the difficult posture. The method adopts a cross validation mode to identify the face with the difficult posture, and enhances the accuracy of face identification.
Based on the above method for filtering a difficult-to-pose face, another embodiment of the present invention is shown in fig. 6, and provides a device 100 for filtering a difficult-to-pose face, including:
a receiving module 110, configured to obtain an image including a human face;
a face rectangle detection module 120, configured to perform face detection on the image including the face to obtain a face rectangle position;
a first keypoint extraction module 130, configured to perform facial keypoint detection on the rectangular position of the face by using a first face keypoint location technique, so as to obtain a first keypoint position set;
a second keypoint extraction module 140, configured to perform facial keypoint detection on the rectangular position of the face by using a second face key positioning technology, so as to obtain a second keypoint position set;
a similarity transformation matrix calculation module 150, configured to obtain a face average face and a corresponding average face keypoint location set, and obtain a similarity transformation matrix from the first keypoint location set to the average face keypoint location set according to the average face keypoint location set and the first keypoint location set;
a keypoint conversion module 160, configured to convert the first keypoint location set and the second keypoint location set to the face average face according to the similarity conversion matrix, and obtain a first conversion keypoint location set and a second conversion keypoint location set;
the screening module 170 is configured to calculate distances between every two of the first transformation key point position set, the second transformation key point position set, and the average face key point position set, obtain a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and determine that the image including the face is a candidate easy-pose face when the first distance, the second distance, and the third distance are all smaller than a threshold, otherwise, the image is a difficult-pose face.
The specific execution steps of the modules are described in detail in the corresponding steps of the method for filtering the faces with difficult postures, and are not described in detail herein.
Referring now to FIG. 7, a block diagram of a computer system 800 suitable for use in implementing the control device of an embodiment of the present application is shown. The control device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program performs the above-described functions defined in the method of the present application when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Python, Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes an acquisition unit, a segmentation unit, a determination unit, and a selection unit. The names of the units do not in some cases constitute a limitation on the units themselves, and for example, the acquisition unit may also be described as a "unit that acquires a to-be-processed picture of the picture".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring an image containing a human face; carrying out face detection on the image containing the face to obtain a face rectangular position; using a first face key point positioning technology to detect face key points at the rectangular positions of the face to obtain a first key point position set; using a second face key positioning technology to detect face key points at the rectangular positions of the face to obtain a second key point position set; acquiring a human face average face and a corresponding average face key point position set, and acquiring a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set; according to the similarity transformation matrix, transforming the first key point position set and the second key point position set to the face average face to obtain a first transformation key point position set and a second transformation key point position set; calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, acquiring a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-posture face when judging that the first distance, the second distance and the third distance are smaller than a threshold value, otherwise, the image containing the face is a difficult-posture face.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims (10)

1. A method for filtering a face with a difficult pose is characterized by comprising the following steps:
s110, acquiring an image containing a human face;
s120, carrying out face detection on the image containing the face to obtain a face rectangular position;
s130, performing face key point detection on the rectangular position of the face by using a first face key point positioning technology to obtain a first key point position set;
s140, performing face key point detection on the rectangular position of the face by using a second face key positioning technology to obtain a second key point position set;
s150, obtaining a human face average face and a corresponding average face key point position set, and obtaining a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set;
s160, transforming the first key point position set and the second key point position set to the face average face according to the similarity transformation matrix to obtain a first transformation key point position set and a second transformation key point position set;
s170, calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, obtaining a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-pose face when judging that the first distance, the second distance and the third distance are all smaller than a threshold value, otherwise, the image containing the face is a difficult-pose face.
2. The method of filtering difficult posed faces according to claim 1, further comprising:
s180, when the image containing the face is a candidate face easy to pose, judging whether the corresponding first key point position set and the corresponding second key point position set are projected to a two-dimensional plane in the range of the image containing the face, and when the first key point position set and the second key point position set are not projected to the two-dimensional plane, judging that the candidate face easy to pose is a face difficult to pose.
3. The method of filtering difficult posed faces according to claim 1 or 2, wherein the step of calculating the distance between two of the first set of transformed keypoint locations, the second set of transformed keypoint locations and the set of average face keypoint locations comprises:
s171, obtaining a distance vector and a distance of the minimum point pair according to the minimum point pair with the minimum European distance between the first group of key points and the second group of key points;
s172, removing the minimum point pair from the first group of key points and the second group of key points;
s173, according to the distance vector, performing translation processing on the first group of key points and the second group of key points after the minimum point pairs are removed to obtain a first group of key points and a second group of key points after transformation;
s174, taking the transformed first group of key points and second group of key points as the first group of key points and second group of key points in step S171, repeating steps S171 to S173 until the key points in the first group of key points and second group of key points are removed, and adding the distances of the minimum point pairs obtained in each iteration as the distances between the first group of key points and second group of key points.
4. The method of filtering a difficult-posed face according to claim 1 or 2, wherein the distance in the step S170 comprises one or more of a hamming distance, a euclidean distance and a mahalanobis distance.
5. The method of claim 2, wherein the step S180 specifically includes determining whether the nose point in the first set of keypoint locations and the nose point in the second set of keypoint locations are projected onto the two-dimensional plane within the range of the image containing the face, and if not, determining that the candidate easy-to-pose face is a difficult-to-pose face.
6. The method of claim 5, wherein the step of determining whether the projection of the nose tip point onto the two-dimensional plane is within the image containing the face comprises:
and judging whether the nose tip point is in a rectangle consisting of a left eye center, a right eye center, a left mouth corner and a right mouth corner, if so, projecting the nose tip point to a two-dimensional plane in the image range containing the face, otherwise, not in the image range containing the face.
7. The method of filtering difficult-posed faces according to claim 1 or 2, wherein the first face keypoint localization technique and the second face keypoint localization technique comprise: the first face key point positioning technology is different from the second face key point positioning technology based on one or more of a Convolutional Neural Network (CNN) algorithm, a Supervised Descent (SDM) algorithm, a subjective shape model (ASM) algorithm and a cascade row propagation regression-based method.
8. A difficult to filter pose face device, comprising:
the receiving module is used for acquiring an image containing a human face;
the face rectangle detection module is used for carrying out face detection on the image containing the face to acquire a face rectangle position;
the first key point extraction module is used for detecting the key points of the face on the rectangular position of the face by using a first face key point positioning technology to obtain a first key point position set;
the second key point extraction module is used for detecting the key points of the face at the rectangular position of the face by using a second face key positioning technology to obtain a second key point position set;
the similarity transformation matrix calculation module is used for acquiring a human face average face and a corresponding average face key point position set, and acquiring a similarity transformation matrix for converting the first key point position set into the average face key point position set according to the average face key point position set and the first key point position set;
the key point transformation module is used for transforming the first key point position set and the second key point position set to the face average face according to the similarity transformation matrix to obtain a first transformation key point position set and a second transformation key point position set;
and the screening module is used for calculating the distance between every two of the first transformation key point position set, the second transformation key point position set and the average face key point position set, acquiring a first distance between the first transformation key point position set and the second transformation key point position set, a second distance between the first transformation key point position set and the average face key point position set, and a third distance between the second transformation key point position set and the average face key point position set, and judging that the image containing the face is a candidate easy-posture face when judging that the first distance, the second distance and the third distance are smaller than threshold values, otherwise, the face is a difficult-posture face.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.
CN201910991099.9A 2019-10-18 2019-10-18 Method and device for filtering faces with difficult gestures Active CN110826421B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910991099.9A CN110826421B (en) 2019-10-18 2019-10-18 Method and device for filtering faces with difficult gestures

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910991099.9A CN110826421B (en) 2019-10-18 2019-10-18 Method and device for filtering faces with difficult gestures

Publications (2)

Publication Number Publication Date
CN110826421A true CN110826421A (en) 2020-02-21
CN110826421B CN110826421B (en) 2023-09-05

Family

ID=69549464

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910991099.9A Active CN110826421B (en) 2019-10-18 2019-10-18 Method and device for filtering faces with difficult gestures

Country Status (1)

Country Link
CN (1) CN110826421B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106295476A (en) * 2015-05-29 2017-01-04 腾讯科技（深圳）有限公司 Face key point localization method and device
CN107766851A (en) * 2017-12-06 2018-03-06 北京搜狐新媒体信息技术有限公司 A kind of face key independent positioning method and positioner
CN108764048A (en) * 2018-04-28 2018-11-06 中国科学院自动化研究所 Face critical point detection method and device
CN109214343A (en) * 2018-09-14 2019-01-15 北京字节跳动网络技术有限公司 Method and apparatus for generating face critical point detection model
CN110309815A (en) * 2019-07-11 2019-10-08 广州华多网络科技有限公司 A kind of processing method and system of facial recognition data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106295476A (en) * 2015-05-29 2017-01-04 腾讯科技（深圳）有限公司 Face key point localization method and device
CN107766851A (en) * 2017-12-06 2018-03-06 北京搜狐新媒体信息技术有限公司 A kind of face key independent positioning method and positioner
CN108764048A (en) * 2018-04-28 2018-11-06 中国科学院自动化研究所 Face critical point detection method and device
CN109214343A (en) * 2018-09-14 2019-01-15 北京字节跳动网络技术有限公司 Method and apparatus for generating face critical point detection model
CN110309815A (en) * 2019-07-11 2019-10-08 广州华多网络科技有限公司 A kind of processing method and system of facial recognition data

Also Published As

Publication number Publication date
CN110826421B (en) 2023-09-05

Similar Documents

Publication Publication Date Title
CN108898186B (en) Method and device for extracting image
CN108509915B (en) Method and device for generating face recognition model
CN108038880B (en) Method and apparatus for processing image
CN109214343B (en) Method and device for generating face key point detection model
CN108830235B (en) Method and apparatus for generating information
CN108229419B (en) Method and apparatus for clustering images
WO2020062493A1 (en) Image processing method and apparatus
CN109711508B (en) Image processing method and device
CN107609506B (en) Method and apparatus for generating image
CN108491823B (en) Method and device for generating human eye recognition model
CN110163096B (en) Person identification method, person identification device, electronic equipment and computer readable medium
CN109583389B (en) Drawing recognition method and device
CN111242097A (en) Face recognition method and device, computer readable medium and electronic equipment
CN108491812B (en) Method and device for generating face recognition model
CN108509994B (en) Method and device for clustering character images
CN109214501B (en) Method and apparatus for identifying information
CN109272543B (en) Method and apparatus for generating a model
CN108388889B (en) Method and device for analyzing face image
CN108229375B (en) Method and device for detecting face image
CN110059624B (en) Method and apparatus for detecting living body
CN110414502B (en) Image processing method and device, electronic equipment and computer readable medium
CN108510084B (en) Method and apparatus for generating information
CN110516598B (en) Method and apparatus for generating image
CN109345460B (en) Method and apparatus for rectifying image
CN108399401B (en) Method and device for detecting face image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant