WO2018219180A1 - 确定人脸图像质量的方法和装置、电子设备和计算机存储介质 - Google Patents

确定人脸图像质量的方法和装置、电子设备和计算机存储介质 Download PDF

Info

Publication number
WO2018219180A1
WO2018219180A1 PCT/CN2018/087915 CN2018087915W WO2018219180A1 WO 2018219180 A1 WO2018219180 A1 WO 2018219180A1 CN 2018087915 W CN2018087915 W CN 2018087915W WO 2018219180 A1 WO2018219180 A1 WO 2018219180A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
score
image
angle
size
Prior art date
Application number
PCT/CN2018/087915
Other languages
English (en)
French (fr)
Inventor
徐丽飞
于晨笛
刘文志
Original Assignee
深圳市商汤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市商汤科技有限公司 filed Critical 深圳市商汤科技有限公司
Priority to SG11201909737V priority Critical patent/SG11201909737VA/en
Priority to KR1020197030468A priority patent/KR102320649B1/ko
Priority to JP2019556650A priority patent/JP6871416B2/ja
Publication of WO2018219180A1 publication Critical patent/WO2018219180A1/zh
Priority to US16/655,235 priority patent/US11182589B2/en
Priority to US17/452,062 priority patent/US20220044005A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/20—Drawing from basic elements, e.g. lines or circles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/12—Bounding box

Definitions

  • the present application relates to computer vision technology, and more particularly to a method and apparatus for determining the quality of a face image, an electronic device, and a computer storage medium.
  • face recognition technology With the development of computer vision technology, face recognition technology has greatly improved in performance in recent years. For face recognition in non-extreme scenes, it can reach the level close to the artificial recognition result. The more the face recognition technology comes The more widely it is applied to various scenes in life.
  • the embodiment of the present application provides a technical solution for determining the quality of a face image.
  • a method for determining a quality of a face image including:
  • the quality information of the face in the image is acquired based on the posture angle information and/or the size information of the face.
  • an apparatus for determining a quality of a face image includes:
  • a first acquiring module configured to acquire posture angle information and/or size information of a face in the image
  • a second acquiring module configured to acquire quality information of the face in the image based on the posture angle information and/or the size information of the face.
  • an electronic device comprising the apparatus for determining a face image quality according to any of the above-mentioned applications.
  • another electronic device including:
  • a memory for storing executable instructions
  • a processor configured to communicate with the memory to execute the executable instructions to perform the operations of the method for determining face image quality as described in any of the above.
  • a computer storage medium for storing computer readable instructions, when the instructions are executed, implementing the method for determining face image quality according to any of the above-mentioned applications of the present application. operating.
  • the method and apparatus for determining the quality of a face image, the electronic device, and the computer storage medium provided by the above embodiments of the present application, acquiring posture angle information and/or size information of a face in the image, according to the posture angle information of the face and/or The size of the face information, the quality of the face in the image.
  • a method for evaluating face image quality based on key factors affecting face recognition results face definition, face size, face orientation
  • obtaining an index for evaluating key factors affecting face recognition results a posture angle of a face for reflecting whether the face is positive, and a size of a face for reflecting the face definition and the face size, and performing the face according to the posture angle information of the face and the size information of the face
  • the method for image quality evaluation the technical solution for determining the quality of the face image in the embodiment of the present application, can objectively evaluate the quality of the face image, and the accuracy of the evaluation result of the face image quality is high; in addition, the embodiment of the present application obtains the person
  • the size information of the face reflects the face definition that affects the face recognition result instead of directly obtaining the face definition in the image, which is advantageous for improving the computational efficiency compared to directly obtaining the face definition in the image, thereby Conducive to improving the real-time performance of face quality assessment.
  • FIG. 1 is a flow chart of an embodiment of a method for determining a face image quality according to the present application.
  • FIG. 2 is a flow chart of another embodiment of a method for determining face image quality according to the present application.
  • FIG. 3 is a flow chart of still another embodiment of a method for determining face image quality according to the present application.
  • FIG. 4 is a flow chart of a specific application embodiment of a method for determining a face image quality according to the present application.
  • FIG. 5 is a schematic structural diagram of an apparatus for determining a face image quality according to the present application.
  • FIG. 6 is a schematic structural diagram of another embodiment of an apparatus for determining a face image quality according to the present application.
  • FIG. 7 is a schematic structural diagram of still another embodiment of a device for determining a face image quality according to the present application.
  • FIG. 8 is a schematic structural diagram of an embodiment of an electronic device according to the present application.
  • Embodiments of the present application can be applied to electronic devices such as terminal devices, computer systems/servers, etc., which can operate with numerous other general purpose or special purpose computing system environments or configurations.
  • Examples of well-known terminal devices, computing systems, environments, and/or configurations suitable for use with electronic devices such as terminal devices, computer systems/servers, and the like include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients Machines, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, small computer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above, and the like.
  • Electronic devices such as terminal devices, computer systems/servers, etc., can be described in the general context of computer system executable instructions (such as program modules) being executed by a computer system.
  • program modules may include routines, programs, target programs, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices that are linked through a communication network.
  • program modules may be located on a local or remote computing system storage medium including storage devices.
  • FIG. 1 is a flow chart of an embodiment of a method for determining a face image quality according to the present application. As shown in FIG. 1, the method for determining the quality of a face image of this embodiment includes:
  • the attitude angle of the human face is also the angle of the human head, including the yaw angle and the pitch angle of the face in the normalized spherical coordinates of the head (ie, the image acquisition coordinate system), wherein the yaw angle is used for Indicates the angle of the face on the face in the horizontal direction, and the pitch angle is used to indicate the angle at which the face is lowered or raised in the vertical direction.
  • the yaw angle and the pitch angle the more positive the face, the easier the face recognition, the higher the accuracy of face recognition, and the yaw angle and the pitch angle are both 0. Face is the most positive, face recognition has the highest accuracy.
  • the size of the face is also the size of the face pixel.
  • the higher the quality of the face in the image the better the quality of the face in the image; on the contrary, the lower the quality of the face in the image, the worse the quality of the face in the image.
  • the false recognition rate is inseparable from the quality of the face image.
  • the side face angle is too large, the face pixel is too small, etc.
  • the accuracy of face recognition is usually significantly reduced, and the false recognition rate is high.
  • the method of determining the quality of the face image contributes to the improvement of the face recognition rate and is very important.
  • the definition criteria of the defined face image quality should make the face easy to recognize.
  • the face is easy to recognize when it is required to satisfy the conditions of high definition, large face, and positive face.
  • the sharpness of the face image comes from two aspects: one is that the image captured by the camera itself is blurred and unclear, and the other is that the face image itself is too small. Since the size of the face image needs to be uniformly scaled to the standard size before the face image recognition is performed, when the small face image is enlarged to the standard size, there is blurring due to pixel interpolation.
  • the captured image itself is clear, so the resolution of the face image and the face size are positively correlated without considering the image captured by the camera.
  • the larger the resolution, the higher the resolution, and the face size can be used to evaluate the sharpness of the face.
  • the method for determining the quality of a face image in the embodiment of the present application is evaluated from the perspective of easy face recognition, based on key factors affecting the face recognition result (for example, face definition, face size, face is positive).
  • Face image quality obtaining indicators for evaluating key factors affecting face recognition results: the attitude angle of the face and the size of the face, wherein the degree of the face is determined by the posture angle of the face, through the face
  • the size of the face is determined, and the face image quality is evaluated according to the posture angle information of the face and the size information of the face.
  • the technical solution for determining the image quality of the face in the embodiment of the present application can objectively evaluate the quality of the face image.
  • the accuracy of the evaluation result of the face image quality is high; in addition, the embodiment of the present application responds to the face resolution affecting the face recognition result by acquiring the size information of the face, instead of directly obtaining the face of the image. Degree, compared with the face sharpness in the direct acquisition image, improves the computational efficiency and improves the real-time performance of the face quality assessment.
  • the posture angle information of the face in the image is obtained, which can be specifically implemented as follows:
  • the image may be subjected to face detection, a face detection frame is obtained, and key points (eg, eye corners, mouth angles, and the like) of the face in the face detection frame are positioned to obtain key point coordinates of the face;
  • the key point coordinates of the face acquire the posture angle information of the face.
  • the posture angle information of the face includes a yaw angle and a pitch angle of the face.
  • obtaining the size information of the face in the image may be implemented by acquiring the size information of the face according to the size of the face detection frame.
  • the size of the face detection frame includes the length and/or width of the face detection frame.
  • the operation 104 may include: acquiring a score of the face pose angle according to the posture angle information of the face; and acquiring a score of the face size according to the size information of the face; the score according to the angle of the face pose The score of the face size obtains the quality score of the face in the image.
  • FIG. 2 is a flow chart of another embodiment of a method for determining face image quality according to the present application. As shown in FIG. 2, the method for determining the quality of a face image of this embodiment includes:
  • the face detection frame includes a face image detected from the image.
  • a face detection algorithm may be used to perform face detection on an image to obtain a face detection frame
  • the key point detection algorithm can be used to locate the key points of the face in the face detection frame to obtain the key point coordinates of the face.
  • the size of the face detection frame includes the length and/or width of the face detection frame.
  • the size of the face detection frame is the size of the face.
  • the attitude angle of the human face is also the angle of the human head, including the yaw angle and the pitch angle of the face in the normalized spherical coordinates of the head (ie, the image acquisition coordinate system), and the yaw angle is used to indicate the level.
  • the angle of the face on the face in the direction, and the pitch angle is used to indicate the angle at which the face is lowered or raised in the vertical direction.
  • the smaller the yaw angle and the pitch angle the more positive the face is, the easier it is to face recognition.
  • the accuracy of face recognition is usually higher, and the yaw angle and the pitch angle are both 0.
  • the face is the most positive, and the accuracy of face recognition is usually the highest.
  • the size of the face is also the size of the face pixel.
  • the score of the face pose angle can be obtained by: according to the yaw angle and the pitch angle of the face, Calculate the score Q yaw of the yaw angle yaw of the face, Calculate the score Q pitch of the pitch angle pitch of the face.
  • the score of the face size may be obtained by acquiring a score of the face size based on at least one of the length, the width, and the area of the face detection frame.
  • the area of the face detection frame is obtained by multiplying the length and the width of the face detection frame.
  • the length, width, and area of the face detection frame correspond to the size of the face image, and thus, the score of the face size can be determined by at least one of the length, the width, and the area of the face detection frame.
  • the score of the face size is obtained based on at least one of the length, the width and the area of the face detection frame, for example, the smaller value min of the length and the width of the face detection frame may be selected; according to the length And the smaller value min in the width, passed Calculate the score Q rect of the face size.
  • the size of the face can be better determined, and the score of the face size can be calculated based on the smaller value of the length and width of the face detection frame, which can be more Objectively respond to the size of the face.
  • the higher the quality score of the face in the image the better the quality of the face in the image; conversely, the lower the quality score of the face in the image, the worse the quality of the face in the image.
  • the operation 208 can be implemented as follows:
  • the quality score of the face in the image is calculated.
  • the weight of the score of the yaw angle, the weight of the score of the pitch angle, and the weight of the score of the face size may be preset, and may be adjusted according to actual needs. In general, the yaw angle has the greatest influence on the accuracy of the face recognition result. In a specific application, the weight of the yaw angle score can be set to the weight of the pitch angle and the weight of the face size.
  • the quality score of the face in the obtained image can more accurately and objectively reflect the quality of the face in an image.
  • the method for determining the quality of the face image evaluates the face based on the key factors affecting the face recognition result (face definition, face size, face face) from the perspective of easy face recognition.
  • Image quality obtaining indicators for evaluating key factors affecting face recognition results: the attitude angle of the face and the size of the face, wherein the degree of the face is determined by the posture angle of the face, and the size of the face is passed.
  • the embodiment of the present application responds to the face resolution affecting the face recognition result by acquiring the size information of the face instead of Directly obtaining the face definition in the image improves the computational efficiency and improves the face quality compared to directly obtaining the face definition in the image. Real-time assessment.
  • FIG. 3 is a flow chart of still another embodiment of a method for determining face image quality according to the present application. As shown in FIG. 3, the method for determining the quality of a face image of this embodiment includes:
  • the confidence of the key point coordinates is used to indicate the accuracy of the key point coordinates of the face. The greater the value of the confidence, the more accurate the key point coordinates of the face.
  • the operation 302 can be implemented by a pre-trained first neural network.
  • the face When the first neural network receives the input image, the face can be outputted by performing face detection and key point detection.
  • the detection frame, the key point coordinates of the face determined according to the face detection frame, and the confidence of the key point coordinates, the confidence of the key point coordinates may be based on the performance of the first neural network by the first neural network according to a preset manner
  • the size of the face detection frame is determined, the better the performance of the first neural network, the larger the size of the face detection frame (ie, the larger the face image and the clearer the face), the key of the determined face
  • the accuracy of the point coordinates is higher.
  • the posture angle information of the face includes a yaw angle and a pitch angle of the face.
  • the score of the face pose angle can be obtained as follows:
  • the score of the face size may be obtained by acquiring a score of the face size based on at least one of the length, the width, and the area of the face detection frame.
  • the area of the face detection frame is obtained by multiplying the length and the width of the face detection frame.
  • the length, width, and area of the face detection frame correspond to the size of the face image, and thus, the score of the face size can be determined by at least one of the length, the width, and the area of the face detection frame.
  • the score of the face size is obtained based on at least one of the length, the width and the area of the face detection frame, for example, the smaller value min of the length and the width of the face detection frame may be selected; according to the length And the smaller value min in the width, passed Calculate the score Q rect of the face size.
  • the size of the face can be better determined, and the score of the face size can be calculated based on the smaller value of the length and width of the face detection frame, which can be more Objectively respond to the size of the face.
  • the confidence of the key point coordinates can be utilized to pass with Calculating a score Q yaw of the corrected yaw angle and a score Q pitch of the corrected pitch angle; wherein Qalign indicates the confidence of the key point coordinates.
  • the operation 308 may be performed simultaneously with the operation of acquiring the score of the face size, before or after the operation of acquiring the score of the face size, and there is no execution time limit therebetween.
  • the attitude angle information of the face obtained by the coordinates of the key points is also inaccurate, in order to solve the problem that the estimation of the attitude angle information of the face is inaccurate due to the inaccurate coordinate of the key points.
  • the score of the calculated face pose angle is corrected according to the confidence degree of the key point coordinates of the face, thereby eliminating the posture angle information of the face caused by the inaccuracy of the key point coordinates.
  • the inaccuracy of the estimation and thus the effect on the final determination of the quality of the face image, improves the accuracy and reliability of the results of determining the quality of the face image.
  • FIG. 4 is a flow chart of a specific application embodiment of a method for determining a face image quality according to the present application. As shown in FIG. 4, the method for determining the quality of a face image of this embodiment includes:
  • the operations 402-404 can be implemented by a pre-trained first neural network.
  • the first neural network receives the input image
  • the image can be detected by the face and the key point is detected.
  • the face detection frame, the key point coordinates of the face and the confidence of the key point coordinates, the confidence of the key point coordinates may be based on the performance of the first neural network and the size of the face detection frame by the first neural network according to a preset manner. If the situation is determined, the better the performance of the first neural network, the larger the size of the face detection frame (ie, the larger the face image and the clearer the face), the higher the accuracy of the key coordinates of the determined face. .
  • the operation 406 can be implemented by a pre-trained second neural network, and when the second neural network receives the key point coordinates of the face, the key point coordinate calculation of the face can be performed, and the output person The yaw angle and pitch angle of the face.
  • the size of the face detection frame is obtained, including the length and width of the face detection frame.
  • the smaller value min of the length and width of the face detection frame is selected.
  • Q is the quality of the face in the image
  • Q yaw represents the score of the corrected yaw angle yaw
  • Q pitch represents the score of the corrected pitch angle pitch
  • Q rect represents the score of the face size
  • W1, w2, and w3 represent the weight of the score of the yaw angle, the weight of the score of the pitch angle, and the weight of the score of the face size, respectively.
  • the yaw angle has the greatest influence on the accuracy of the face recognition result, and the value of w1 can be set to 0.6; the weight w2 of the pitch angle score and the weight w3 of the face size score can be set to 0.2, also It can be adjusted according to the actual situation.
  • the method embodiments for determining the face image quality according to the present application may be performed on any of the plurality of images of the same face, respectively, to obtain the quality score of the face in the plurality of images.
  • the method further includes: selecting, according to the quality information of the face in the plurality of images, the image of the high quality of the at least one face for face detection.
  • the image with poor quality of the face and the image with high quality of the face are selected for face detection and recognition, and the face recognition rate of the image with high quality of the selected face is high. Therefore, it is advantageous to improve the accuracy of face recognition, and is advantageous for reducing the amount of operation data of face recognition, and is advantageous for improving the face recognition speed of an effective image.
  • FIG. 5 is a schematic structural diagram of an apparatus for determining a face image quality according to the present application.
  • the apparatus for determining the quality of a face image of this embodiment can be used to implement the method embodiments for determining the image quality of the face described above in the present application.
  • the apparatus for determining the quality of the face image of the embodiment includes: a first obtaining module 502 and a second acquiring module 504.
  • the first obtaining module 502 is configured to acquire posture angle information and size information of a face in the image.
  • the second obtaining module 504 is configured to acquire quality information of a face in the image based on the posture angle information and the size information of the face.
  • the device for determining the quality of the face image evaluates the image quality of the face based on key factors affecting the face recognition result (for example, face definition, face size, face is positive).
  • key factors affecting the face recognition result for example, face definition, face size, face is positive.
  • the embodiment provides a technical solution for determining the quality of the face image, and can objectively evaluate the quality of the face image, and the accuracy of the evaluation result of the face image quality is high.
  • the embodiment of the present application responds to the influence by acquiring the size information of the face.
  • the face sharpness of the face recognition result rather than directly obtaining the face sharpness in the image, is advantageous for improving the computational efficiency compared to directly obtaining the face sharpness in the image, thereby facilitating the evaluation of the face quality. Real time.
  • FIG. 6 is a schematic structural diagram of another embodiment of an apparatus for determining a face image quality according to the present application.
  • the first obtaining module 502 specifically includes: a face detecting sub-module 602 , a key point detecting sub-module 604 , and a first acquiring sub-module 606 .
  • the face detection sub-module 602 is configured to acquire a face detection frame in the image, where the face detection frame is used to determine a face in the image.
  • the face detection sub-module 602 can be configured to perform face detection on the image to obtain a face detection frame.
  • the key point detection sub-module 604 is configured to acquire key point coordinates of the face determined according to the face detection frame.
  • the key point detection sub-module 604 can be configured to perform key point positioning on the face image determined by the face detection frame to obtain key point coordinates of the face.
  • the first obtaining sub-module 606 is configured to acquire the posture angle information of the face according to the key point coordinates of the face, wherein the posture angle information of the face includes a yaw angle and a pitch angle of the face; and according to the face detection frame Size gets the size information of the face.
  • the size of the face detection frame includes the length and/or width of the face detection frame.
  • the face detection sub-module 602 is configured to perform face detection on the image, and obtain a face detection frame, where the face detection frame includes an image of the face, which is called : Face image.
  • the key point detection sub-module 604 is configured to perform key point positioning on the face image determined by the face detection frame to obtain key point coordinates of the face.
  • the second obtaining module 504 may include: a second obtaining submodule 608, a third obtaining submodule 610, and a fourth obtaining submodule. 612.
  • the second obtaining sub-module 608 is configured to obtain a score of the face posture angle according to the posture angle information of the face.
  • the second acquisition sub-module 608 is configured to pass the yaw angle and the pitch angle of the face. Calculate the score Q yaw of the yaw angle yaw of the face, Calculate the score Q pitch of the pitch angle pitch of the face.
  • the second acquisition module 608 may obtain the score of the face size based on at least one of the length, the width and the area of the face detection frame: selecting a smaller value of the length and the width of the face detection frame. Min; according to the smaller value min in length and width, pass Calculate the score Q rect of the face size.
  • the third obtaining sub-module 610 is configured to obtain a score of the face size according to the size information of the face.
  • the third obtaining sub-module 610 is configured to obtain a score of the face size based on at least one of a length, a width, and an area of the face detection frame; the area of the face detection frame is determined by the face detection frame. The product of length and width is obtained.
  • the fourth obtaining sub-module 612 is configured to obtain a quality score of the face in the image according to the score of the face pose angle and the score of the face size.
  • the fourth obtaining sub-module 612 is configured to calculate a face in the image according to the score of the yaw angle and its weight, the score of the pitch angle and its weight, the score of the face size, and the weight thereof. quality.
  • the weight of the score of the yaw angle may be set to the weight of the score of the pitch angle and the weight of the score of the face size.
  • FIG. 7 is a schematic structural diagram of still another embodiment of a device for determining a face image quality according to the present application.
  • the apparatus for determining the quality of the face image further includes: a fourth obtaining module 506 and a correcting module 508, in the embodiment, compared with the apparatus for determining the quality of the face image in the above embodiments of the present application. among them:
  • the fourth obtaining module 506 is configured to acquire the confidence of the key point coordinates. Among them, the confidence of the key point coordinates is used to indicate the accuracy of the key point coordinates of the face.
  • the fourth acquisition module 506 can be integrally configured with the keypoint detection sub-module 604, which can be implemented by a neural network.
  • the correction module 508 is configured to correct the score of the face pose angle obtained by the second acquisition sub-module 608 by using the confidence of the key point coordinates.
  • the correction module 508 is configured to: pass the confidence of the key point coordinates, respectively with Calculating a score Q yaw of the corrected yaw angle and a score Q pitch of the corrected pitch angle; wherein Qalign indicates the confidence of the key point coordinates.
  • the fourth obtaining sub-module 612 is configured to acquire the quality of the face in the image according to the score of the corrected face pose angle and the score of the face size.
  • the embodiment of the present application further provides an electronic device, including the device for determining the quality of a face image according to any of the above embodiments of the present application.
  • an electronic device including the device for determining the quality of a face image according to any of the above embodiments of the present application.
  • the embodiment of the present application responds to the face resolution affecting the face recognition result by acquiring the size information of the face instead of directly acquiring
  • the sharpness of the face in the image is better for improving the computational efficiency than directly obtaining the sharpness of the face in the image, which is beneficial to improving the real-time performance of the face quality assessment.
  • the selection module and the face detection module may also be included. among them:
  • a selection module configured to select at least one image of a high quality of the face according to the quality information of the face in the image in the plurality of images output by the device for determining the quality of the face image
  • the face detection module is configured to perform face detection on the selected at least one image.
  • the image with poor quality of the face and the image with high quality of the face are selected for face detection and recognition, and the face recognition rate of the image with high quality of the selected face is high. It is beneficial to improve the accuracy of face recognition, and to reduce the amount of computational data of face recognition, thereby facilitating the improvement of face recognition speed for effective images.
  • the embodiment of the present application further provides another electronic device, including: a memory for storing executable instructions; and a processor, configured to communicate with the memory to execute executable instructions, thereby completing any of the above embodiments of the present application.
  • a memory for storing executable instructions
  • a processor configured to communicate with the memory to execute executable instructions, thereby completing any of the above embodiments of the present application.
  • the electronic device of each of the above embodiments of the present application may be, for example, a mobile terminal, a personal computer (PC), a tablet computer, a server, or the like.
  • the embodiment of the present application further provides a computer storage medium for storing a computer readable instruction, when the instruction is executed, implementing the operation of the method for determining the face image quality of any of the above embodiments of the present application.
  • FIG. 8 is a schematic structural diagram of an embodiment of an electronic device according to the present application.
  • the electronic device includes one or more processors, communication units, etc., such as one or more central processing units (CPUs) 801, and/or one or more An image processor (GPU) 813 or the like, the processor may execute various kinds according to executable instructions stored in a read only memory (ROM) 802 or executable instructions loaded from the storage portion 808 into the random access memory (RAM) 803.
  • the communication unit 812 can include, but is not limited to, a network card, which can include, but is not limited to, an IB (Infiniband) network card.
  • IB Infiniband
  • the processor can communicate with the read-only memory 802 and/or the random access memory 803 to execute executable instructions, connect to the communication unit 812 via the bus 804, and communicate with other target devices via the communication unit 812, thereby completing the embodiments of the present application.
  • the operation corresponding to any one of the methods, for example, acquiring the posture angle information and the size information of the face in the image; and acquiring the quality information of the face in the image based on the posture angle information and the size information of the face.
  • RAM 803 various programs and data required for the operation of the device can be stored.
  • the CPU 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804.
  • ROM 802 is an optional module.
  • the RAM 803 stores executable instructions, or writes executable instructions to the ROM 802 at runtime, and the executable instructions cause the central processing unit (CPU) 801 to perform operations corresponding to the above-described communication methods.
  • An input/output (I/O) interface 805 is also coupled to bus 804.
  • the communication unit 812 may be integrated or may be provided with a plurality of sub-modules (for example, a plurality of IB network cards) and on the bus link.
  • the following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a storage portion 808 including a hard disk or the like. And a communication portion 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the Internet.
  • Driver 810 is also coupled to I/O interface 805 as needed.
  • a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, is mounted on the drive 810 as needed so that a computer program read therefrom is installed into the storage portion 808 as needed.
  • FIG. 8 is only an optional implementation manner.
  • the number and types of the components in FIG. 8 may be selected, deleted, added, or replaced according to actual needs; Different function component settings may also be implemented by separate settings or integrated settings.
  • the GPU 813 and the CPU 801 may be separately configured or the GPU 813 may be integrated on the CPU 801.
  • the communication unit may be separately configured or integrated on the CPU 801 or the GPU 813. and many more.
  • an embodiment of the present disclosure includes a computer program product comprising a computer program tangibly embodied on a machine readable medium, the computer program comprising program code for executing the method illustrated in the flowchart, the program code comprising Executing instructions corresponding to the method steps provided in the embodiments of the present application, for example, acquiring an instruction for the posture angle information and the size information of the face in the image; and acquiring the quality information of the face in the image based on the posture angle information and the size information of the face Instructions.
  • the computer program can be downloaded and installed from the network via communication portion 809, and/or installed from removable media 811.
  • the computer program is executed by the central processing unit (CPU) 801, the above-described functions defined in the method of the present application are performed.
  • the embodiment of the present application can be applied to: in the field of cell monitoring or security monitoring, the product of the capture machine or the face recognition, and the face detection by the image collected by the camera (ie, the image in the embodiment of the present application).
  • To identify the face image in order to improve the accuracy of face recognition, reduce the false recognition rate and the miss recognition rate, and avoid unnecessary recognition, it is necessary to first provide the image to the device or device for determining the image quality of the face.
  • the image is filtered and filtered to screen out high quality face images.
  • an image with a large side face or a low head or a face pixel that is too small ie, the face size is too small
  • the method, apparatus, or device for determining the quality of the face image in the embodiment of the present application can obtain the quality of the face in each image, effectively filtering out the image of the face with low quality and the above-mentioned image that is not suitable for face recognition.
  • the number of face recognition is reduced, and the face recognition efficiency is improved.
  • the embodiment of the present application is more effective in the scenario where the face recognition is time-consuming in the embedded device.
  • the embodiments of the present application have at least the following beneficial technical effects: the embodiment of the present application fully considers the face image requirement that is easy to face recognition, estimates the face posture angle and combines the face size to design an evaluation index, and combines the face yaw angle.
  • the pan angle and the face size are used to comprehensively evaluate the face image quality, and the situation that may cause the face pose angle estimation is corrected is not only real-time, but also easy to apply, and the accuracy of the evaluation method is ensured.
  • Reliability by obtaining the size information of the face to reflect the face definition affecting the face recognition result instead of directly obtaining the face definition in the image, it is advantageous for directly obtaining the face definition in the image.
  • Improve the efficiency of the operation which is conducive to improving the real-time performance of face quality evaluation; it is beneficial to improve the accuracy of face recognition by eliminating the poor quality image of the face and selecting the high quality image of the face for face detection and recognition. Rate, and is conducive to reducing the amount of computational data for face recognition, thereby facilitating the improvement of face recognition speed for effective images. .
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the foregoing steps include the steps of the foregoing method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
  • the methods and apparatus of the present application may be implemented in a number of ways.
  • the methods and apparatus of the present application can be implemented in software, hardware, firmware, or any combination of software, hardware, and firmware.
  • the above-described sequence of steps for the method is for illustrative purposes only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specifically stated.
  • the present application can also be implemented as a program recorded in a recording medium, the programs including machine readable instructions for implementing the method according to the present application.
  • the present application also covers a recording medium storing a program for executing the method according to the present application.

Abstract

本申请实施例公开了一种确定人脸图像质量的方法和装置、电子设备和计算机存储介质，其中，方法包括：获取图像中人脸的姿态角度信息和大小信息；基于所述人脸的姿态角度信息和大小信息获取图像中人脸的质量信息。

Description

确定人脸图像质量的方法和装置、电子设备和计算机存储介质
本申请要求在2017年5月31日提交中国专利局、申请号为201710405232.9、申请名称为“确定人脸图像质量的方法和装置、电子设备和计算机存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机视觉技术，尤其是涉及一种确定人脸图像质量的方法和装置、电子设备和计算机存储介质。
背景技术
随着计算机视觉技术的发展，人脸识别技术近几年在性能上有很大的提高，对于非极端场景下的人脸识别可以达到与人工识别结果接近的水平，人脸识别技术越也来越广泛地应用到生活中的各个场景。
发明内容
本申请实施例提供一种用于进行确定人脸图像质量的技术方案。
根据本申请实施例的一个方面，提供一种确定人脸图像质量的方法，包括：
获取图像中人脸的姿态角度信息和/或大小信息；
基于所述人脸的姿态角度信息和/或大小信息，获取图像中人脸的质量信息。
根据本申请实施例的另一个方面，提供一种确定人脸图像质量的装置，包括：
第一获取模块，用于获取图像中人脸的姿态角度信息和/或大小信息；
第二获取模块，用于基于所述人脸的姿态角度信息和/或大小信息，获取图像中人脸的质量信息。
根据本申请实施例的又一个方面，提供一种电子设备，包括本申请上述任一所述的确定人脸图像质量的装置。
根据本申请实施例的又一个方面，提供另一种电子设备，包括：
存储器，用于存储可执行指令；以及
处理器，用于与所述存储器通信以执行所述可执行指令从而完成本申请上述任一所述的确定人脸图像质量的方法的操作。
根据本申请实施例的再一个方面，提供一种计算机存储介质，用于存储计算机可读取的指令，所述指令被执行时实现本申请上述任一所述的确定人脸图像质量的方法的操 作。
基于本申请上述实施例提供的确定人脸图像质量的方法和装置、电子设备和计算机存储介质，获取图像中人脸的姿态角度信息和/或大小信息，根据人脸的姿态角度信息和/或人脸的大小的信息，获取图像中人脸的质量。
本公开的实施例提供的技术方案可以包括以下有益效果：
通过基于影响人脸识别结果的关键性因素(人脸清晰度、人脸大小、人脸是否正面)来评估人脸图像质量的方法，获取用于评价影响人脸识别结果的关键性因素的指标：用于反应人脸是否正面的人脸的姿态角度、以及用于反应人脸清晰度和人脸大小的人脸的大小，并根据人脸的姿态角度信息和人脸的大小信息进行人脸图像质量评价的方法，本申请实施例确定人脸图像质量的技术方案，可以客观评估人脸图像质量，对人脸图像质量的评估结果的精准率较高；另外，本申请实施例通过获取人脸的大小信息来反应影响人脸识别结果的人脸清晰度而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，有利于提高运算效率，从而有利于提升人脸质量评估的实时性。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。下面通过附图和实施例，对本申请的技术方案做进一步的详细描述。
附图说明
构成说明书的一部分的附图描述了本申请的实施例，并且连同描述一起用于解释本申请的原理。
参照附图，根据下面的详细描述，可以更加清楚地理解本申请，其中：
图1为本申请确定人脸图像质量的方法一个实施例的流程图。
图2为本申请确定人脸图像质量的方法另一个实施例的流程图。
图3为本申请确定人脸图像质量的方法又一个实施例的流程图。
图4为本申请确定人脸图像质量的方法一个具体应用实施例的流程图。
图5为本申请确定人脸图像质量的装置一个实施例的结构示意图。
图6为本申请确定人脸图像质量的装置另一个实施例的结构示意图。
图7为本申请确定人脸图像质量的装置又一个实施例的结构示意图。
图8为本申请电子设备一个实施例的结构示意图。
具体实施例
现在将参照附图来详细描述本申请的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本申请的范围。
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本申请及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本申请实施例可以应用于终端设备、计算机系统/服务器等电子设备，其可与众多其它通用或专用计算系统环境或配置一起操作。适于与终端设备、计算机系统/服务器等电子设备一起使用的众所周知的终端设备、计算系统、环境和/或配置的例子包括但不限于：个人计算机系统、服务器计算机系统、瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统﹑大型计算机系统和包括上述任何系统的分布式云计算技术环境，等等。
终端设备、计算机系统/服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。
图1为本申请确定人脸图像质量的方法一个实施例的流程图。如图1所示，该实施例的确定人脸图像质量的方法包括：
102，获取图像中人脸的姿态角度信息和/或大小信息。
其中，人脸的姿态角度也即人的头部姿态角度，包括头部归一化球坐标(即：图像采集坐标系)中人脸的偏航角度和俯仰角度，其中，偏航角度用于表示水平方向上人脸侧脸的角度，俯仰角度用于表示竖直方向上人脸低头或仰头的角度。在人脸大小一定的情况下，偏航角度、俯仰角度越小，人脸越正，越易于人脸识别，人脸识别的准确率越 高，偏航角度和俯仰角度均为0时，人脸最正，人脸识别的准确率最高。
其中的人脸大小也即人脸像素的大小，人脸越大、清晰度越高，越易于人脸识别，人脸识别的准确率越高。
104，基于人脸的姿态角度信息和/或大小信息，获取图像中人脸的质量信息。
其中，图像中人脸的质量越高，说明图像中人脸的质量越好；反之，图像中人脸的质量越低，说明图像中人脸的质量越差。
目前，在人脸识别过程中容易产生误识别的情况。误识别率除了和识别算法模型有关之外，和人脸图像的质量也是分不开的。在人脸图像质量较差的情况下，例如侧脸角度过大、人脸像素过小等，人脸识别的准确率通常会明显降低，误识别率较高。在实际场景中，大部分的误识别和漏识别都是因为人脸图像质量不够高造成的。因此，确定人脸图像质量的方法有助于提高人脸识别率而且非常重要。
现有的图像质量评价方法可以分为2大类：主观评价方法和客观评价方法。随着自动化程度的提高，在很多领域需要人工参与的主观评价方法存在诸多不便，成本高，周期长，于是客观评价方法逐渐发展起来。当前确定人脸图像质量的方法并没有引起足够的重视，关于人脸图像质量的客观评价方法尚不成熟，对人脸图像质量的评估结果不够精准。
为了评价人脸图像质量，需要建立人脸图像质量评价指标，需要定义好的人脸图像质量的评价标准。为了提高人脸识别率，定义好的人脸图像质量的评价标准应该使人脸易于识别，例如，人脸在需要满足清晰度高、人脸大、人脸正等条件时，易于识别。在实际应用场景中，人脸图像的清晰度来源于两个方面的影响：一是摄像头采集的图像本身是模糊不清晰的，二是人脸图像本身太小。由于在进行人脸图像识别前需要先将人脸图像大小统一缩放到标准大小，当小的人脸图像放大到标准大小时，会存在由于像素插值造成的模糊。一般情况下，根据应用场景选择合适的摄像头后，其采集的图像本身是清晰，于是在不考虑摄像头采集图像不清晰的情况下，人脸图像的清晰度和人脸大小是正相关的，人脸越大其清晰度也越高，可以用人脸大小来评价人脸的清晰度。
本申请实施例的确定人脸图像质量的方法，从易于人脸识别角度出发，基于影响人脸识别结果的关键性因素(例如，人脸清晰度、人脸大小、人脸是否正面)来评估人脸图像质量，获取用于评价影响人脸识别结果的关键性因素的指标：人脸的姿态角度和人脸的大小，其中，通过人脸的姿态角度确定人脸正面的程度，通过人脸大小确定人脸清晰度，并根据人脸的姿态角度信息和人脸的大小信息进行人脸图像质量评价，本申请实 施例确定人脸图像质量的技术方案，可以客观评估人脸图像质量，对人脸图像质量的评估结果的精准率较高；另外，本申请实施例通过获取人脸的大小信息，来反应影响人脸识别结果的人脸清晰度，而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，提高了运算效率，提升了人脸质量评估的实时性。
在本申请确定人脸图像质量的方法实施例的其中一个可选示例中，操作102中，获取图像中人脸的姿态角度信息，具体可以通过如下方式实现：
获取图像中的人脸检测框和根据所述人脸检测框确定的人脸的关键点坐标。例如，可以对图像进行人脸检测，获得人脸检测框，以及对人脸检测框中的人脸进行关键点(例如，眼角、嘴角等)定位，获得人脸的关键点坐标；根据上述人脸的关键点坐标获取人脸的姿态角度信息。其中，人脸的姿态角度信息包括人脸的偏航角度和俯仰角度。
在另一个可选示例中，操作102中，获取图像中人脸的大小信息，可以通过如下方式实现：根据人脸检测框的大小获取人脸的大小信息。其中，人脸检测框的大小包括人脸检测框的长度和/或宽度。
在又一个可选示例中，操作104可以包括：根据人脸的姿态角度信息获取人脸姿态角度的分数；以及根据人脸的大小信息获取人脸大小的分数；根据人脸姿态角度的分数和所述人脸大小的分数，获取图像中人脸的质量分数。
图2为本申请确定人脸图像质量的方法另一个实施例的流程图。如图2所示，该实施例的确定人脸图像质量的方法包括：
202，获取图像中的人脸检测框和根据该人脸检测框确定的人脸的关键点坐标。
其中，人脸检测框包括从图像中检测到的人脸图像。
例如，可以通过人脸检测算法，对图像进行人脸检测，获得人脸检测框；
例如，可以通过关键点检测算法，对人脸检测框中的人脸进行关键点定位，获得人脸的关键点坐标。
204，根据人脸的关键点坐标获取人脸的姿态角度信息，以及根据人脸检测框的大小获取人脸的大小信息。
其中，人脸检测框的大小包括人脸检测框的长度和/或宽度。在一个具体示例中，人脸检测框的大小即人脸的大小。其中，人脸的姿态角度也即人的头部姿态角度，包括头部归一化球坐标(即：图像采集坐标系)中人脸的偏航角度和俯仰角度，偏航角度用于表示水平方向上人脸侧脸的角度，俯仰角度用于表示竖直方向上人脸低头或仰头的角度。在人脸大小一定的情况下，偏航角度、俯仰角度越小，人脸越正，通常越易于人脸识别， 人脸识别的准确率通常越高，偏航角度和俯仰角度均为0时，人脸最正，人脸识别的准确率通常最高。
其中的人脸的大小也即人脸像素的大小，人脸越大、清晰度越高，通常越易于人脸识别，人脸识别的准确率通常越高。
206，根据人脸的姿态角度信息获取人脸姿态角度的分数，以及根据人脸的大小信息获取人脸大小的分数。
在其中一个可选示例中，可以通过如下方式获取人脸姿态角度的分数：根据人脸的偏航角度和俯仰角度，通过
Figure PCTCN2018087915-appb-000001
计算获得人脸的偏航角度yaw的分数Q yaw，通过
Figure PCTCN2018087915-appb-000002
计算获得人脸的俯仰角度pitch的分数Q pitch。
在另一个可选示例中，可以通过如下方式获取人脸大小的分数：基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数。其中，人脸检测框的面积由人脸检测框的长度与宽度的乘积获得。人脸检测框的长度、宽度、面积对应于人脸图像的大小，由此，通过人脸检测框的长度、宽度与面积中的至少一项可以确定人脸大小的分数。
进一步示例性地，基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数，例如可以是：选取人脸检测框的长度和宽度中的较小值min；根据长度和宽度中的较小值min，通过
Figure PCTCN2018087915-appb-000003
计算获得人脸大小的分数Q rect。
通过人脸检测框的长度和宽度中的较小值，可以更好的确定人脸的大小，从而基于人脸检测框的长度和宽度中的较小值计算获得人脸大小的分数，可以更加客观的反应人脸大小。
208，根据人脸姿态角度的分数和人脸大小的分数获取上述图像中人脸的质量分数。
其中，图像中人脸的质量分数越高，说明图像中人脸的质量越好；反之，图像中人脸的质量分数越低，说明图像中人脸的质量越差。
在又一个可选示例中，该操作208可以通过如下方式实现：
根据偏航角度的分数及其权重、俯仰角度的分数及其权重、人脸大小的分数及其权重，计算获得图像中人脸的质量分数。
其中，偏航角度的分数的权重、俯仰角度的分数的权重、人脸大小的分数的权重可 以预先设置，并且可以根据实际需求进行调整。通常情况下，偏航角度对人脸识别结果准确性的影响最大，在具体应用中，可以设置偏航角度的分数的权重大于俯仰角度的分数的权重和人脸大小的分数的权重，从而使得获得的图像中人脸的质量分数可以更加准确、客观的反应一个图像中人脸的质量高低。
本申请实施例的确定人脸图像质量的方法，从易于人脸识别角度出发，基于影响人脸识别结果的关键性因素(人脸清晰度、人脸大小、人脸是否正面)来评估人脸图像质量，获取用于评价影响人脸识别结果的关键性因素的指标：人脸的姿态角度和人脸的大小，其中，通过人脸的姿态角度确定人脸正面的程度，通过人脸的大小确定人脸清晰度，并进一步获取人脸姿态角度的分数和人脸大小的分数，根据人脸姿态角度的分数和人脸大小的分数确定图像中人脸的质量分数，从而可以更精确、客观的评估图像中人脸的质量，对人脸图像质量的评估结果的精准率更高；另外，本申请实施例通过获取人脸的大小信息来反应影响人脸识别结果的人脸清晰度而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，提高了运算效率，提升了人脸质量评估的实时性。
图3为本申请确定人脸图像质量的方法又一个实施例的流程图。如图3所示，该实施例的确定人脸图像质量的方法包括：
302，获取图像中的人脸检测框、根据该人脸检测框确定的人脸的关键点坐标和关键点坐标的置信度。
其中，关键点坐标的置信度用于表示人脸的关键点坐标的准确率，置信度的数值越大，表示人脸的关键点坐标越准确。
可选地，该操作302可以通过一个预先训练好的第一神经网络实现，该第一神经网络接收到输入的图像时，便可以通过对该图像进行人脸检测和关键点检测，输出人脸检测框、根据该人脸检测框确定的人脸的关键点坐标和关键点坐标的置信度，关键点坐标的置信度可以根据预设方式，由第一神经网络基于该第一神经网络的性能和人脸检测框的大小等情况确定，第一神经网络的性能越好，人脸检测框的大小较大(即：人脸图像较大、人脸较清晰)时，确定的人脸的关键点坐标的准确率越高。
304，根据人脸的关键点坐标获取人脸的姿态角度信息，以及根据人脸检测框的大小获取人脸的大小信息。其中，人脸的姿态角度信息包括人脸的偏航角度和俯仰角度。
306，根据人脸的姿态角度信息获取人脸姿态角度的分数，以及根据人脸的大小信息获取人脸大小的分数。
在其中一个可选示例中，可以通过如下方式获取人脸姿态角度的分数：
根据人脸的偏航角度和俯仰角度，通过
Figure PCTCN2018087915-appb-000004
计算获得人脸的偏航角度yaw的分数Q yaw，通过
Figure PCTCN2018087915-appb-000005
计算获得人脸的俯仰角度pitch的分数Q pitch。
在另一个可选示例中，可以通过如下方式获取人脸大小的分数：基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数。其中，人脸检测框的面积由人脸检测框的长度与宽度的乘积获得。
人脸检测框的长度、宽度、面积对应于人脸图像的大小，由此，通过人脸检测框的长度、宽度与面积中的至少一项可以确定人脸大小的分数。
进一步示例性地，基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数，例如可以是：选取人脸检测框的长度和宽度中的较小值min；根据长度和宽度中的较小值min，通过
Figure PCTCN2018087915-appb-000006
计算获得人脸大小的分数Q rect。
通过人脸检测框的长度和宽度中的较小值，可以更好的确定人脸的大小，从而基于人脸检测框的长度和宽度中的较小值计算获得人脸大小的分数，可以更加客观的反应人脸大小。
308，利用关键点坐标的置信度对人脸的姿态角度的分数进行修正。
示例性地，可以利用关键点坐标的置信度，分别通过
Figure PCTCN2018087915-appb-000007
和
Figure PCTCN2018087915-appb-000008
计算获得修正后的偏航角度的分数Q yaw和修正后的俯仰角度的分数Q pitch；其中，
Figure PCTCN2018087915-appb-000009
Qalign表示关键点坐标的置信度。可选地，该操作308可以与获取人脸大小的分数的操作同时进行、在获取人脸大小的分数的操作之前或之后进行，二者之间不存在执行时间限制。
310，根据修正后的人脸的姿态角度的分数和人脸大小的分数获取图像中人脸的质量分数。
在人脸的关键点坐标不准确时，由该关键点坐标获取到的人脸的姿态角度信息也会不准确，为了解决由于关键点坐标不准确造成人脸的姿态角度信息估计不准确的问题，本申请实施例中，根据该人脸的关键点坐标的置信度的大小对计算获得的人脸姿态角度的分数进行相应修正，从而消除了由于关键点坐标不准确造成人脸的姿态角度信息估计不准确、以及由此对最终确定人脸图像质量的结果的影响，提升了确定人脸图像质量的结果的准确性和可靠性。
图4为本申请确定人脸图像质量的方法一个具体应用实施例的流程图。如图4所示，该实施例的确定人脸图像质量的方法包括：
402，对图像进行人脸检测，获得人脸检测框。
404，对人脸检测框中的人脸进行关键点定位，获得人脸的关键点坐标以及关键点坐标的置信度。其中，关键点坐标的置信度用于表示人脸的关键点坐标的准确率。
可选地，操作402～404可以通过一个预先训练好的第一神经网络实现，该第一神经网络接收到输入的图像时，便可以通过对该图像进行人脸检测和关键点检测，输出人脸检测框、人脸的关键点坐标和关键点坐标的置信度，关键点坐标的置信度可以按照预设方式，由第一神经网络基于该第一神经网络的性能和人脸检测框的大小等情况确定，第一神经网络的性能越好，人脸检测框的大小较大(即：人脸图像较大、人脸较清晰)时，确定的人脸的关键点坐标的准确率越高。
之后，分别执行操作406和406’。
406，根据人脸的关键点坐标获取人脸的姿态角度信息。其中，人脸的姿态角度信息包括人脸的偏航角度和俯仰角度。可选地，该操作406可以通过一个预先训练好的第二神经网络实现，该第二神经网络接收到人脸的关键点坐标时，便可以通过对该人脸的关键点坐标计算，输出人脸的偏航角度和俯仰角度。
408，根据人脸的偏航角度和俯仰角度，通过
Figure PCTCN2018087915-appb-000010
计算获得人脸的偏航角度yaw的分数Q yaw，通过
Figure PCTCN2018087915-appb-000011
计算获得人脸的俯仰角度 pitch的分数Q pitch。
410，利用关键点坐标的置信度对人脸姿态角度的分数进行修正。
示例性地，利用关键点坐标的置信度，分别通过
Figure PCTCN2018087915-appb-000012
和
Figure PCTCN2018087915-appb-000013
计算获得修正后的偏航角度的分数Q yaw和修正后的俯仰角度的分数Q pitch；其中，
Figure PCTCN2018087915-appb-000014
Qalign表示关键点坐标的置信度。
之后，执行操作412。
406’,获取人脸检测框的大小，包括人脸检测框的长度和宽度。
408’,选取人脸检测框的长度和宽度中的较小值min。
410’，根据长度和宽度中的较小值min，通过
Figure PCTCN2018087915-appb-000015
计算获得人脸大小的分数Q rect。
其中，操作406～410和操作406’～410’之间不存在执行时间顺序关系限制，二者之间可以以任意时间和顺序执行。
412，根据修正后的偏航角度的分数及其权重、修正后的俯仰角度的分数及其权重、人脸大小的分数及其权重，计算获得图像中人脸的质量。
例如，可以通过Q＝w1*Q yaw+w2*Q pitch+w3*Q rect，计算获得图像中人脸的质量。
其中，Q为图像中人脸的质量，Q yaw表示修正后的偏航角度yaw的分数，Q pitch表示修正后的俯仰角度pitch的分数，Q rect表示人脸大小的分数。w1、w2、w3分别表示偏航角度的分数的权重、俯仰角度的分数的权重和人脸大小的分数的权重。通常情况下，偏航角度对人脸识别结果准确性的影响最大，w1的取值可以设置为0.6；俯仰角度的分数的权重w2和人脸大小的分数的权重w3可以均设置为0.2，也可以根据实际情况 进行调整。
进一步地，可以分别针对同一人脸的多张图像中的任一图像，执行本申请上述各确定人脸图像质量的方法实施例，从而获得多张图像中的人脸的质量分数。在本申请确定人脸图像质量的方法再一实施例中，还可以包括：根据上述多张图像中人脸的质量信息，选取至少一张人脸的质量高的图像进行人脸检测。
基于该实施例，可以剔除人脸的质量差的图像、选取出人脸的质量高的图像进行人脸检测和识别，由于选取出的人脸的质量高的图像的人脸识别率较高，因此有利于提高人脸识别的准确率，并且有利于降低人脸识别的运算数据量，有利于提高对有效图像的人脸识别速度。
图5为本申请确定人脸图像质量的装置一个实施例的结构示意图。该实施例的确定人脸图像质量的装置可用于实现本申请上述各确定人脸图像质量的方法实施例。如图5所示，该实施例的确定人脸图像质量的装置包括：第一获取模块502和第二获取模块504。
第一获取模块502，用于获取图像中人脸的姿态角度信息和大小信息。
第二获取模块504，用于基于人脸的姿态角度信息和大小信息获取图像中人脸的质量信息。
基于本申请上述实施例提供的确定人脸图像质量的装置，基于影响人脸识别结果的关键性因素(例如，人脸清晰度、人脸大小、人脸是否正面)来评估人脸图像质量，获取用于评价影响人脸识别结果的关键性因素的指标：人脸的姿态角度和人脸的大小，并根据人脸的姿态角度和人脸的大小，确定图像中人脸的质量，本申请实施例进行确定人脸图像质量的技术方案，可以客观评估人脸图像质量，对人脸图像质量的评估结果的精准率较高；另外，本申请实施例通过获取人脸的大小信息来反应影响人脸识别结果的人脸清晰度，而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，有利于提高运算效率，从而有利于提升人脸质量评估的实时性。
图6为本申请确定人脸图像质量的装置另一个实施例的结构示意图。如图6所示，该实施例中，第一获取模块502具体包括：人脸检测子模块602，关键点检测子模块604和第一获取子模块606。
人脸检测子模块602，用于获取图像中的人脸检测框，其中，该人脸检测框用于确定图像中的人脸。可选地，人脸检测子模块602可用于对图像进行人脸检测，获得人脸检测框。
关键点检测子模块604，用于获取根据人脸检测框确定的人脸的关键点坐标。可选 地，关键点检测子模块604可用于对由人脸检测框确定的人脸图像进行关键点定位，获得人脸的关键点坐标。
第一获取子模块606，用于根据人脸的关键点坐标获取人脸的姿态角度信息，其中，人脸的姿态角度信息包括人脸的偏航角度和俯仰角度；以及根据人脸检测框的大小获取人脸的大小信息。其中，人脸检测框的大小包括人脸检测框的长度和/或宽度。
另外，在该确定人脸图像质量的装置实施例中，人脸检测子模块602，用于对图像进行人脸检测，获得人脸检测框，该人脸检测框包括人脸的图像，称为：人脸图像。相应地，关键点检测子模块604，用于对由人脸检测框确定的人脸图像进行关键点定位，获得人脸的关键点坐标。
另外，再参见图6，在本申请确定人脸图像质量的装置又一个实施例中，第二获取模块504可以包括：第二获取子模块608、第三获取子模块610和第四获取子模块612。
第二获取子模块608，用于根据人脸的姿态角度信息获取人脸姿态角度的分数。
在其中一个可选示例中，第二获取子模块608用于根据人脸的偏航角度和俯仰角度，通过
Figure PCTCN2018087915-appb-000016
计算获得人脸的偏航角度yaw的分数Q yaw，通过
Figure PCTCN2018087915-appb-000017
计算获得人脸的俯仰角度pitch的分数Q pitch。
进一步地示例性地，第二获取模块608基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数可以为：选取人脸检测框的长度和宽度中的较小值min；根据长度和宽度中的较小值min，通过
Figure PCTCN2018087915-appb-000018
计算获得人脸大小的分数Q rect。
第三获取子模块610，用于根据人脸的大小信息获取人脸大小的分数。
在其中一个可选示例中，第三获取子模块610用于基于人脸检测框的长度、宽度与面积中的至少一项获取人脸大小的分数；人脸检测框的面积由人脸检测框的长度与宽度的乘积获得。
第四获取子模块612，用于根据人脸姿态角度的分数和人脸大小的分数获取图像中人脸的质量分数。在其中一个可选示例中，第四获取子模块612用于根据偏航角度的分数及其权重、俯仰角度的分数及其权重、人脸大小的分数及其权重，计算获得图像中人脸的质量。在实际应用中，由于人脸的偏航角度对人脸识别结果准确性的影响最大，可以设置偏航角度的分数的权重大于俯仰角度的分数的权重和人脸大小的分数的权重。
图7为本申请确定人脸图像质量的装置又一个实施例的结构示意图。如图7所示，与本申请上述各实施例的确定人脸图像质量的装置相比，该实施例中，确定人脸图像质量的装置还包括：第四获取模块506和修正模块508。其中：
第四获取模块506，用于获取关键点坐标的置信度。其中，关键点坐标的置信度用于表示人脸的关键点坐标的准确率。
示例性地，该第四获取模块506可以与关键点检测子模块604一体设置，二者可以通过一个神经网络实现。
修正模块508，用于利用关键点坐标的置信度对第二获取子模块608获得的人脸姿态角度的分数进行修正。
在其中一个可选示例中，修正模块508用于：利用关键点坐标的置信度，分别通过
Figure PCTCN2018087915-appb-000019
和
Figure PCTCN2018087915-appb-000020
计算获得修正后的偏航角度的分数Q yaw和修正后的俯仰角度的分数Q pitch；其中，
Figure PCTCN2018087915-appb-000021
Qalign表示关键点坐标的置信度。
相应地，该实施例中，第四获取子模块612，用于根据修正后的人脸姿态角度的分数和人脸大小的分数获取图像中人脸的质量。
本申请实施例还提供了一种电子设备，包括本申请上述任一实施例的确定人脸图像质量的装置。通过获取用于评价影响人脸识别结果的关键性因素的指标：人脸的姿态角度和人脸的大小，并根据人脸的姿态角度信息和人脸的大小信息进行人脸图像质量评价，可以客观评估人脸图像质量，对人脸图像质量的评估结果的精准率较高；另外，本申请实施例通过获取人脸的大小信息来反应影响人脸识别结果的人脸清晰度而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，有利于提高运算效率， 有利于提升人脸质量评估的实时性。
进一步地，在上述电子设备实施例中，还可以包括选取模块和人脸检测模块。其中：
选取模块，用于根据确定人脸图像质量的装置输出的多张图像中图像中人脸的质量信息，选取至少一张人脸的质量高的图像；
人脸检测模块，用于对选取出的至少一张图像进行人脸检测。
基于该实施例，可以剔除人脸的质量差的图像、选取出人脸的质量高的图像进行人脸检测和识别，由于选取出的人脸的质量高的图像的人脸识别率较高，有利于提高人脸识别的准确率，并且以利于降低人脸识别的运算数据量，从而有利于提高对有效图像的人脸识别速度。
本申请实施例还提供了另一种电子设备，包括：存储器，用于存储可执行指令；以及，处理器，用于与存储器通信以执行可执行指令，从而完成本申请上述任一实施例的确定人脸图像质量的方法的操作。
本申请上述各实施例的电子设备，例如可以是移动终端、个人计算机(PC)、平板电脑、服务器等。
本申请实施例还提供了一种计算机存储介质，用于存储计算机可读取的指令，该指令被执行时，实现本申请上述任一实施例的确定人脸图像质量的方法的操作。
图8为本申请电子设备一个实施例的结构示意图。下面参考图8，其示出了适于用来实现本申请实施例的终端设备或服务器的电子设备的结构示意图。如图8所示，该电子设备包括一个或多个处理器、通信部等，所述一个或多个处理器例如：一个或多个中央处理单元(CPU)801，和/或一个或多个图像处理器(GPU)813等，处理器可以根据存储在只读存储器(ROM)802中的可执行指令或者从存储部分808加载到随机访问存储器(RAM)803中的可执行指令而执行各种适当的动作和处理。通信部812可包括但不限于网卡，所述网卡可包括但不限于IB(Infiniband)网卡，
处理器可与只读存储器802和/或随机访问存储器803中通信以执行可执行指令，通过总线804与通信部812相连、并经通信部812与其他目标设备通信，从而完成本申请实施例提供的任一项方法对应的操作，例如，获取图像中人脸的姿态角度信息和大小信息；基于所述人脸的姿态角度信息和大小信息获取图像中人脸的质量信息。
此外，在RAM 803中，还可存储有装置操作所需的各种程序和数据。CPU801、ROM802以及RAM803通过总线804彼此相连。在有RAM803的情况下，ROM802为可选模块。RAM803存储可执行指令，或在运行时向ROM802中写入可执行指令，可执行 指令使中央处理单元(CPU)801执行上述通信方法对应的操作。输入/输出(I/O)接口805也连接至总线804。通信部812可以集成设置，也可以设置为具有多个子模块(例如多个IB网卡)，并在总线链接上。
以下部件连接至I/O接口805：包括键盘、鼠标等的输入部分806；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分807；包括硬盘等的存储部分808；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分809。通信部分809经由诸如因特网的网络执行通信处理。驱动器810也根据需要连接至I/O接口805。可拆卸介质811，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器810上，以便于从其上读出的计算机程序根据需要被安装入存储部分808。
需要说明的，如图8所示的架构仅为一种可选实现方式，在具体实践过程中，可根据实际需要对上述图8的部件数量和类型进行选择、删减、增加或替换；在不同功能部件设置上，也可采用分离设置或集成设置等实现方式，例如GPU813和CPU801可分离设置或者可将GPU813集成在CPU801上，通信部可分离设置，也可集成设置在CPU801或GPU813上，等等。这些可替换的实施方式均落入本申请公开的保护范围。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括有形地包含在机器可读介质上的计算机程序，计算机程序包含用于执行流程图所示的方法的程序代码，程序代码可包括对应执行本申请实施例提供的方法步骤对应的指令，例如，获取图像中人脸的姿态角度信息和大小信息的指令；基于所述人脸的姿态角度信息和大小信息获取图像中人脸的质量信息的指令。在这样的实施例中，该计算机程序可以通过通信部分809从网络上被下载和安装，和/或从可拆卸介质811被安装。在该计算机程序被中央处理单元(CPU)801执行时，执行本申请的方法中限定的上述功能。
本申请实施例可以可选的应用于：小区监控或者安防监控领域，抓拍机或者人脸识别相关的产品，对摄像头采集得到的图像(即：本申请实施例中的图像)进行人脸检测，并对人脸图像进行识别，为了提高人脸识别的准确率，降低误识别率和漏识别率、以及避免不必要的识别，就需要先将图像提供给确定人脸图像质量的装置或设备，对图像进行筛选过滤，以便筛选出高质量的人脸图像。通过进行人脸图像质量的评估，大侧脸或者低头很严重或者人脸像素太小(即：人脸大小太小)的图像，由于较难正确识别而可以筛除。通过本申请实施例的确定人脸图像质量的方法、装置或设备，可以获得各图像中人脸的质量，有效过滤掉图像中人脸的质量较低、上述不适合人脸识别的图像，从而 减少人脸识别次数，提高人脸识别效率，本申请实施例应用于在嵌入式设备中进行人脸识别比较耗时的场景下，效果更为明显。
本申请实施例至少具有以下有益技术效果：本申请实施例充分考虑了易于人脸识别的人脸图像需求，估计出人脸姿态角度并结合人脸大小来设计评价指标，结合人脸偏航角度和俯仰角度、以及人脸大小来综合评价人脸图像质量，并且对于可能造成人脸姿态角度估计不准的情况进行了修正，不仅实时性高，易于应用，且保证了评价方法的准确性和可靠性；通过获取人脸的大小信息来反应影响人脸识别结果的人脸清晰度而非直接获取图像中的人脸清晰度，相对于直接获取图像中的人脸清晰度来说，有利于提高运算效率，从而有利于提升人脸质量评估的实时性；通过剔除人脸的质量差的图像、选取出人脸的质量高的图像进行人脸检测和识别，有利于提高人脸识别的准确率，并且有利于降低人脸识别的运算数据量，从而有利于提高对有效图像的人脸识别速度。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
本说明书中各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似的部分相互参见即可。对于系统实施例而言，由于其与方法实施例基本对应，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
可能以许多方式来实现本申请的方法和装置。例如，可通过软件、硬件、固件或者软件、硬件、固件的任何组合来实现本申请的方法和装置。用于所述方法的步骤的上述顺序仅是为了进行说明，本申请的方法的步骤不限于以上具体描述的顺序，除非以其它方式特别说明。此外，在一些实施例中，还可将本申请实施为记录在记录介质中的程序，这些程序包括用于实现根据本申请的方法的机器可读指令。因而，本申请还覆盖存储用于执行根据本申请的方法的程序的记录介质。
本申请的描述是为了示例和描述起见而给出的，而并不是无遗漏的或者将本申请限于所公开的形式。很多修改和变化对于本领域的普通技术人员而言是显然的。选择和描述实施例是为了更好说明本申请的原理和实际应用，并且使本领域的普通技术人员能够理解本申请从而设计适于特定用途的带有各种修改的各种实施例。

Claims (29)

  1. 一种确定人脸图像质量的方法，其特征在于，包括：
    获取图像中人脸的姿态角度信息和/或大小信息；
    基于所述人脸的姿态角度信息和/或大小信息，获取所述图像中人脸的质量信息。
  2. 根据权利要求1所述的方法，其特征在于，所述获取图像中人脸的姿态角度信息，包括：
    获取所述图像中的人脸检测框和根据所述人脸检测框确定的所述人脸的关键点坐标；根据所述人脸的关键点坐标获取所述人脸的姿态角度信息，所述人脸的姿态角度信息包括所述人脸的偏航角度和俯仰角度。
  3. 根据权利要求2所述的方法，其特征在于，获取所述图像中的人脸检测框和根据所述人脸检测框确定的所述人脸的关键点坐标，包括：
    对所述图像进行人脸检测，获得所述人脸检测框；
    对所述人脸检测框中的人脸进行关键点定位，获得所述人脸的关键点坐标。
  4. 根据权利要求2～3任一所述的方法，其特征在于，获取所述图像中人脸的大小信息，包括：根据所述人脸检测框的大小获取所述人脸的大小信息；所述人脸检测框的大小包括所述人脸检测框的长度和/或宽度。
  5. 根据权利要求1～4任一所述的方法，其特征在于，所述基于所述人脸的姿态角度信息和/或大小信息，获取所述图像中人脸的质量信息，包括：
    根据所述人脸的姿态角度信息获取人脸姿态角度的分数；以及根据所述人脸的大小信息获取人脸大小的分数；根据所述人脸姿态角度的分数和所述人脸大小的分数，获取所述图像中人脸的质量分数。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述人脸的姿态角度信息获取人脸姿态角度的分数，包括：根据所述人脸的偏航角度和俯仰角度计算获得所述人脸的偏航角度yaw的分数Q yaw，和所述人脸的俯仰角度pitch的分数Q pitch。
  7. 根据权利要求5～6任一所述的方法，其特征在于，所述根据所述人脸的大小信息获取人脸大小的分数，包括：基于所述人脸检测框的长度、宽度与面积中的至少一项获取所述人脸大小的分数；所述人脸检测框的面积由所述人脸检测框的长度与宽度的乘积获得。
  8. 根据权利要求7所述的方法，其特征在于，基于所述人脸检测框的长度、宽度与面积中的至少一项获取所述人脸大小的分数，包括：
    选取所述人脸检测框的长度和宽度中的较小值min；
    根据所述较小值min，计算获得所述人脸大小的分数Qrect。
  9. 根据权利要求5～8任一所述的方法，其特征在于，根据所述人脸姿态角度的分数和所述人脸大小的分数，获取所述图像中人脸的质量分数，包括：
    根据所述偏航角度的分数及其权重、所述俯仰角度的分数及其权重、所述人脸大小的分数及其权重，计算获得所述图像中人脸的质量分数。
  10. 根据权利要求9所述的方法，其特征在于，所述偏航角度的分数的权重大于所述俯仰角度的分数的权重以及所述人脸大小的分数的权重。
  11. 根据权利要求5～10任一所述的方法，其特征在于，所述方法还包括：
    获取所述关键点坐标的置信度，所述关键点坐标的置信度用于表示所述人脸的关键点坐标的准确率；
    获取所述人脸姿态角度的分数之后，还包括：
    利用所述关键点坐标的置信度对所述人脸姿态角度的分数进行修正；
    根据所述人脸姿态角度的分数和所述人脸大小的分数，获取图像中人脸的质量分数，包括：根据修正后的人脸姿态角度的分数和所述人脸大小的分数，获取图像中人脸的质量分数。
  12. 根据权利要求11所述的方法，其特征在于，利用所述关键点坐标的置信度对所述人脸姿态角度的分数进行修正，包括：
    利用所述关键点坐标的置信度确定所述人脸的偏航角度yaw的分数Q yaw和所述人脸的俯仰角度pitch的分数Q pitch的修正参数a，并计算修正参数a分别与所述Q yaw和Q pitch的乘积，所述乘积被作为修正后的偏航角度的分数以及修正后的俯仰角度的分数；
    其中，在关键点坐标的置信度小于预定值的情况下，a的取值为第一值，在关键点坐标的置信度不小于预定值的情况下，a的取值为第二值，所述第一值小于第二值。
  13. 根据权利要求1～12任一所述的方法，其特征在于，分别针对多张图像中的至少一图像，执行所述获取图像中人脸的姿态角度信息和/或大小信息、以及基于所述人脸的姿态角度信息和/或大小信息获取图像中人脸的质量信息的操作；
    所述方法还包括：根据所述多张图像中人脸的质量信息，选取至少一张人脸的质量高的图像进行人脸检测。
  14. 一种确定人脸图像质量的装置，其特征在于，包括：
    第一获取模块，用于获取图像中人脸的姿态角度信息和/或大小信息；
    第二获取模块，用于基于所述人脸的姿态角度信息和/或大小信息，获取图像中人脸的质量信息。
  15. 根据权利要求14所述的装置，其特征在于，所述第一获取模块包括：
    人脸检测子模块，用于获取所述图像中的人脸检测框，所述人脸的姿态角度信息包括所述人脸的偏航角度和俯仰角度；
    关键点检测子模块，用于获取根据所述人脸检测框确定的所述人脸的关键点坐标；
    第一获取子模块，用于根据所述人脸的关键点坐标获取所述人脸的姿态角度信息，所述人脸的姿态角度信息包括所述人脸的偏航角度和俯仰角度；以及根据所述人脸检测框的大小获取所述人脸的大小信息；所述人脸检测框的大小包括所述人脸检测框的长度和宽度。
  16. 根据权利要求15所述的装置，其特征在于，所述人脸检测子模块，进一步用于对所述图像进行人脸检测，获得所述人脸检测框；
    所述关键点检测子模块，进一步用于对所述人脸检测框中的人脸进行关键点定位，获得所述人脸的关键点坐标。
  17. 根据权利要求15～16任一所述的装置，其特征在于，所述第二获取模块包括：
    第二获取子模块，用于根据所述人脸的姿态角度信息获取人脸姿态角度的分数；
    第三获取子模块，用于根据所述人脸的大小信息获取人脸大小的分数；
    第四获取子模块，用于根据所述人脸姿态角度的分数和所述人脸大小的分数，获取所述图像中人脸的质量分数。
  18. 根据权利要求17所述的装置，其特征在于，所述第二获取子模块进一步用于：根据所述人脸的偏航角度和俯仰角度，计算获得所述人脸的偏航角度yaw的分数Q yaw和所述人脸的俯仰角度pitch的分数Q pitch。
  19. 根据权利要求17～18任一所述的装置，其特征在于，所述第三获取子模块进一步用于：基于所述人脸检测框的长度、宽度与面积中的至少一项获取所述人脸大小的分数；所述人脸检测框的面积由所述人脸检测框的长度与宽度的乘积获得。
  20. 根据权利要求19所述的装置，其特征在于，所述第三获取子模块进一步用于：
    选取所述人脸检测框的长度和宽度中的较小值min；
    根据所述较小值min，计算获得人脸大小的分数Qrect。
  21. 根据权利要求17～20任一所述的装置，其特征在于，所述第四获取子模块，进一步用于根据所述偏航角度的分数及其权重、所述俯仰角度的分数及其权重、所述人脸大小的分数及其权重，计算获得所述图像中人脸的质量分数。
  22. 根据权利要求21所述的装置，其特征在于，所述偏航角度的分数的权重大于所述俯仰角度的分数的权重以及所述人脸大小的分数的权重。
  23. 根据权利要求17～22任一所述的装置，其特征在于，还包括：
    第三获取模块，用于获取所述关键点坐标的置信度，所述关键点坐标的置信度用于表示所述人脸的关键点坐标的准确率；
    修正模块，用于利用所述关键点坐标的置信度对所述人脸姿态角度的分数进行修正；
    所述第四获取子模块，进一步用于根据修正后的人脸姿态角度的分数和所述人脸大小的分数，获取所述图像中人脸的质量分数。
  24. 根据权利要求23所述的装置，其特征在于，所述修正模块进一步用于：
    利用所述关键点坐标的置信度，确定所述人脸的偏航角度yaw的分数Q yaw和所述人脸的俯仰角度pitch的分数Q pitch的修正参数a，并计算修正参数a分别与所述Q yaw和Q pitch的乘积，所述乘积被作为修正后的偏航角度的分数以及修正后的俯仰角度的分数；其中，在所述关键点坐标的置信度小于预定值的情况下，a的取值为第一值，在关键点坐标的置信度不小于预定值的情况下，a的取值为第二值，所述第一值小于第二值。
  25. 一种电子设备，其特征在于，包括权利要求14～24任一所述的确定人脸图像质量的装置。
  26. 根据权利要求25所述的电子设备，其特征在于，还包括：
    选取模块，用于根据所述确定人脸图像质量的装置输出的多张图像中人脸的质量信息，选取至少一张人脸的质量高的图像；
    人脸检测模块，用于对选取出的至少一张图像进行人脸检测。
  27. 一种电子设备，其特征在于，包括：
    存储器，用于存储可执行指令；以及，处理器，用于与所述存储器通信以执行所述可执行指令从而完成权利要求1～13任一所述方法的操作。
  28. 一种计算机存储介质，用于存储计算机可读取的指令，其特征在于，所述指令 被执行时实现权利要求1～13任一所述方法的操作。
  29. 一种计算机程序，包括计算机指令，当所述计算机指令在设备的处理器中运行时，所述处理器执行用于实现权利要求1-13中任一所述方法的操作。
PCT/CN2018/087915 2017-05-31 2018-05-22 确定人脸图像质量的方法和装置、电子设备和计算机存储介质 WO2018219180A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
SG11201909737V SG11201909737VA (en) 2017-05-31 2018-05-22 Methods and apparatuses for determining facial image quality, electronic devices, and computer storage media
KR1020197030468A KR102320649B1 (ko) 2017-05-31 2018-05-22 얼굴 이미지 품질을 결정하는 방법과 장치, 전자 기기 및 컴퓨터 저장 매체
JP2019556650A JP6871416B2 (ja) 2017-05-31 2018-05-22 顔画像品質を決定する方法および装置、電子機器ならびにコンピュータ記憶媒体
US16/655,235 US11182589B2 (en) 2017-05-31 2019-10-17 Methods and apparatuses for determining face image quality, electronic devices, and computer storage media
US17/452,062 US20220044005A1 (en) 2017-05-31 2021-10-23 Methods and apparatuses for determining face image quality, electronic devices, and computer storage media

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710405232.9A CN108230293A (zh) 2017-05-31 2017-05-31 确定人脸图像质量的方法和装置、电子设备和计算机存储介质
CN201710405232.9 2017-05-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/655,235 Continuation US11182589B2 (en) 2017-05-31 2019-10-17 Methods and apparatuses for determining face image quality, electronic devices, and computer storage media

Publications (1)

Publication Number Publication Date
WO2018219180A1 true WO2018219180A1 (zh) 2018-12-06

Family

ID=62656626

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/087915 WO2018219180A1 (zh) 2017-05-31 2018-05-22 确定人脸图像质量的方法和装置、电子设备和计算机存储介质

Country Status (6)

Country Link
US (2) US11182589B2 (zh)
JP (1) JP6871416B2 (zh)
KR (1) KR102320649B1 (zh)
CN (1) CN108230293A (zh)
SG (1) SG11201909737VA (zh)
WO (1) WO2018219180A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112651321A (zh) * 2020-12-21 2021-04-13 浙江商汤科技开发有限公司 档案处理方法、装置及服务器

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110837750B (zh) * 2018-08-15 2023-11-03 华为技术有限公司 一种人脸质量评价方法与装置
CN109101646B (zh) * 2018-08-21 2020-12-18 北京深瞐科技有限公司 数据处理方法、装置、系统及计算机可读介质
CN110909568A (zh) * 2018-09-17 2020-03-24 北京京东尚科信息技术有限公司 用于面部识别的图像检测方法、装置、电子设备及介质
CN111199165B (zh) * 2018-10-31 2024-02-06 浙江宇视科技有限公司 图像处理方法及装置
CN109409962A (zh) * 2018-11-08 2019-03-01 北京旷视科技有限公司 图像处理方法、装置、电子设备、计算机可读存储介质
CN109376684B (zh) * 2018-11-13 2021-04-06 广州市百果园信息技术有限公司 一种人脸关键点检测方法、装置、计算机设备和存储介质
CN109614910B (zh) * 2018-12-04 2020-11-20 青岛小鸟看看科技有限公司 一种人脸识别方法和装置
CN109753886B (zh) * 2018-12-17 2024-03-08 北京爱奇艺科技有限公司 一种人脸图像的评价方法、装置及设备
CN109816628B (zh) * 2018-12-20 2021-09-14 深圳云天励飞技术有限公司 人脸评价方法及相关产品
CN109754461A (zh) * 2018-12-29 2019-05-14 深圳云天励飞技术有限公司 图像处理方法及相关产品
CN111739084B (zh) * 2019-03-25 2023-12-05 上海幻电信息科技有限公司 图片处理方法、图集处理方法、计算机设备和存储介质
CN110147744A (zh) * 2019-05-09 2019-08-20 腾讯科技（深圳）有限公司 一种人脸图像质量评估方法、装置及终端
CN110796108B (zh) * 2019-11-04 2022-05-17 北京锐安科技有限公司 一种人脸质量检测的方法、装置、设备及存储介质
CN111028216A (zh) * 2019-12-09 2020-04-17 Oppo广东移动通信有限公司 图像评分方法、装置、存储介质及电子设备
CN111104909A (zh) * 2019-12-20 2020-05-05 深圳市商汤科技有限公司 图像处理方法、装置、计算机设备及存储介质
KR20210153826A (ko) * 2020-06-11 2021-12-20 삼성전자주식회사 디스플레이장치 및 그 제어방법
CN112200804A (zh) * 2020-11-09 2021-01-08 北京地平线信息技术有限公司 图像检测方法、装置、计算机可读存储介质及电子设备
CN113297423A (zh) * 2021-05-24 2021-08-24 深圳市优必选科技股份有限公司 一种推送方法、推送装置及电子设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1573660A (zh) * 2003-05-30 2005-02-02 微软公司 头部姿势估算方法和系统
US20120114198A1 (en) * 2010-11-08 2012-05-10 Yang Ting-Ting Facial image gender identification system and method thereof
CN102799877A (zh) * 2012-09-11 2012-11-28 上海中原电子技术工程有限公司 人脸图像筛选方法及系统
CN103824068A (zh) * 2014-03-19 2014-05-28 上海看看智能科技有限公司 人脸支付认证系统及方法
CN105046227A (zh) * 2015-07-24 2015-11-11 上海依图网络科技有限公司 一种针对人像视频系统的关键帧获取方法
CN105631439A (zh) * 2016-02-18 2016-06-01 北京旷视科技有限公司 人脸图像处理方法和装置
CN106295585A (zh) * 2016-08-16 2017-01-04 深圳云天励飞技术有限公司 一种兼顾实时性及人脸质量的过滤选择方法及系统
CN106446851A (zh) * 2016-09-30 2017-02-22 厦门大图智能科技有限公司 一种基于可见光的人脸优选方法及系统

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6937745B2 (en) * 2001-12-31 2005-08-30 Microsoft Corporation Machine vision system and method for estimating and tracking facial pose
US7747040B2 (en) * 2005-04-16 2010-06-29 Microsoft Corporation Machine vision system and method for estimating and tracking facial pose
JP2007028555A (ja) * 2005-07-21 2007-02-01 Sony Corp カメラシステム，情報処理装置，情報処理方法，およびコンピュータプログラム
JP2007206898A (ja) * 2006-01-31 2007-08-16 Toshiba Corp 顔認証装置および入退場管理装置
US7916897B2 (en) * 2006-08-11 2011-03-29 Tessera Technologies Ireland Limited Face tracking for controlling imaging parameters
GB2457194A (en) 2006-11-08 2009-08-12 Cryptometrics Inc System and method for parallel image processing
US10169646B2 (en) * 2007-12-31 2019-01-01 Applied Recognition Inc. Face authentication to mitigate spoofing
JP5127531B2 (ja) * 2008-03-27 2013-01-23 セコム株式会社 画像監視装置
JP5225317B2 (ja) * 2010-04-06 2013-07-03 キヤノン株式会社 撮像装置及び撮像装置の制御方法
JP5766564B2 (ja) * 2011-09-15 2015-08-19 株式会社東芝 顔認証装置及び顔認証方法
US8687880B2 (en) * 2012-03-20 2014-04-01 Microsoft Corporation Real time head pose estimation
US8254647B1 (en) * 2012-04-16 2012-08-28 Google Inc. Facial image quality assessment
US9552374B2 (en) * 2013-08-19 2017-01-24 Kodak Alaris, Inc. Imaging workflow using facial and non-facial features
GB2516512B (en) * 2013-10-23 2015-10-14 Imagination Tech Ltd Face detection
CN110263642B (zh) * 2013-10-28 2022-04-19 谷歌有限责任公司 用于替换图像的部分的图像缓存

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1573660A (zh) * 2003-05-30 2005-02-02 微软公司 头部姿势估算方法和系统
US20120114198A1 (en) * 2010-11-08 2012-05-10 Yang Ting-Ting Facial image gender identification system and method thereof
CN102799877A (zh) * 2012-09-11 2012-11-28 上海中原电子技术工程有限公司 人脸图像筛选方法及系统
CN103824068A (zh) * 2014-03-19 2014-05-28 上海看看智能科技有限公司 人脸支付认证系统及方法
CN105046227A (zh) * 2015-07-24 2015-11-11 上海依图网络科技有限公司 一种针对人像视频系统的关键帧获取方法
CN105631439A (zh) * 2016-02-18 2016-06-01 北京旷视科技有限公司 人脸图像处理方法和装置
CN106295585A (zh) * 2016-08-16 2017-01-04 深圳云天励飞技术有限公司 一种兼顾实时性及人脸质量的过滤选择方法及系统
CN106446851A (zh) * 2016-09-30 2017-02-22 厦门大图智能科技有限公司 一种基于可见光的人脸优选方法及系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112651321A (zh) * 2020-12-21 2021-04-13 浙江商汤科技开发有限公司 档案处理方法、装置及服务器

Also Published As

Publication number Publication date
KR20190129947A (ko) 2019-11-20
US20220044005A1 (en) 2022-02-10
US20200050835A1 (en) 2020-02-13
JP2020517027A (ja) 2020-06-11
CN108230293A (zh) 2018-06-29
JP6871416B2 (ja) 2021-05-12
KR102320649B1 (ko) 2021-11-03
US11182589B2 (en) 2021-11-23
SG11201909737VA (en) 2019-11-28

Similar Documents

Publication Publication Date Title
WO2018219180A1 (zh) 确定人脸图像质量的方法和装置、电子设备和计算机存储介质
US11107232B2 (en) Method and apparatus for determining object posture in image, device, and storage medium
US11182592B2 (en) Target object recognition method and apparatus, storage medium, and electronic device
WO2018137623A1 (zh) 图像处理方法、装置以及电子设备
WO2019100608A1 (zh) 摄像装置、人脸识别的方法、系统及计算机可读存储介质
US20200410074A1 (en) Identity authentication method and apparatus, electronic device, and storage medium
CN106778453B (zh) 人脸图像中检测眼镜佩戴的方法及装置
JP6071002B2 (ja) 信頼度取得装置、信頼度取得方法および信頼度取得プログラム
JP6822482B2 (ja) 視線推定装置、視線推定方法及びプログラム記録媒体
WO2019061659A1 (zh) 人脸图像眼镜去除方法、装置及存储介质
US11694331B2 (en) Capture and storage of magnified images
WO2019205633A1 (zh) 眼睛状态的检测方法、电子设备、检测装置和计算机可读存储介质
WO2021084972A1 (ja) 物体追跡装置および物体追跡方法
CN108289191B (zh) 图像识别方法及装置
CN107798292B (zh) 对象识别方法、计算机程序、存储介质及电子设备
CN111784660B (zh) 一种用于人脸图像的正脸程度的分析方法和系统
CN116580432A (zh) 一种在线考试监控方法、系统、计算设备及存储介质
US10902628B1 (en) Method for estimating user eye orientation using a system-independent learned mapping
JP7103443B2 (ja) 情報処理装置、情報処理方法、およびプログラム
CN111898610B (zh) 卡片缺角检测方法、装置、计算机设备及存储介质
CN109214317B (zh) 一种信息量确定方法及装置
WO2023048153A1 (ja) 情報処理方法、コンピュータプログラム及び情報処理装置
CN114973365A (zh) 一种人像偏转矫正的方法、设备及介质
CN115086648A (zh) 一种摄像头清晰度检测方法和装置
CN117671290A (zh) 一种图像识别方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18808894

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20197030468

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2019556650

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 23.04.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18808894

Country of ref document: EP

Kind code of ref document: A1