WO2016110030A1 - 一种人脸图像的检索系统及方法 - Google Patents

一种人脸图像的检索系统及方法 Download PDF

Info

Publication number
WO2016110030A1
WO2016110030A1 PCT/CN2015/078302 CN2015078302W WO2016110030A1 WO 2016110030 A1 WO2016110030 A1 WO 2016110030A1 CN 2015078302 W CN2015078302 W CN 2015078302W WO 2016110030 A1 WO2016110030 A1 WO 2016110030A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
image
face image
frontal
modeling
Prior art date
Application number
PCT/CN2015/078302
Other languages
English (en)
French (fr)
Inventor
王春茂
张文聪
浦世亮
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2016110030A1 publication Critical patent/WO2016110030A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content

Definitions

  • the present invention relates to the field of intelligent security technologies, and in particular, to a system and method for retrieving face images.
  • Face retrieval is a focus of related research in face information processing and content-based retrieval. It is a very active research direction in recent years. It has extremely wide application value in the fields of intelligent human-machine interface, content-based retrieval, digital video processing, security and so on.
  • the face retrieval technology mainly outputs the first N face images most similar to the input face image from the database (the database size is usually millions of orders) through a certain modeling and comparison method, and according to similarity
  • the N face images are sorted to obtain a similar face sequence according to the similarity from high to low, that is, the search result.
  • N is a positive integer greater than or equal to 1.
  • Face retrieval is one of the classic applications of face recognition. Although face recognition technology has been developed for many years, the current face recognition technology still has some shortcomings, such as when the image is not clear or there are factors such as attitude deflection that are not conducive to recognition, the correct rate of retrieval is lower. In the case of clear images or no posture deflection, the retrieval accuracy rate will drop sharply, and this downward trend will become more prominent when the retrieval database is increased.
  • the technical problem to be solved by the present invention is to overcome the problem that the face recognition technology in the prior art has a low retrieval accuracy when the image is unclear or there is a posture deflection which is unfavorable for recognition.
  • an embodiment of the present invention first provides a method for retrieving a face image, the method comprising: performing global image quality preprocessing on a face image to obtain a pre-processed face image; Processing a face image for face detection, obtaining a face in the pre-processed face image; performing feature point positioning on the face in the pre-processed face image to obtain a feature point of the face; The feature points perform face correction on the face in the pre-processed face image to obtain a frontal face image; perform face modeling on the front face image to obtain a face model; and adopt the face model Face matching is performed on the face image database, and similar face sequences arranged in order of similarity are obtained.
  • performing face correction on the face in the pre-processed face image comprises performing geometric correction and/or image quality correction on the face of the pre-processed face image.
  • the method comprises accepting an interactive operation of the user in at least one of the global image quality preprocessing, the face detection, the feature point location, and the face correction.
  • accepting the user's interactive operations includes accepting the interactive operation by the user using template selection and/or parameter adjustment.
  • performing face modeling on the frontal face image to obtain a face model includes: modeling a face of the frontal face image to obtain a high-dimensional feature of the frontal face image; The high-dimensional features of the frontal face image are dimensionally reduced to obtain the face model.
  • An embodiment of the present invention further provides a method for retrieving a face image, the method comprising: acquiring an average face model; and using a semantic description technique to adjust a face component in the average face model to obtain a frontal person a face image; performing a face modeling on the frontal face image to obtain a face model; using the face model to perform face comparison on the face image database, and obtaining similar faces arranged in a similarity order sequence.
  • the adjusting the face component in the average face model comprises: performing the adjustment on the face component in the average face model by using template selection and/or parameter adjustment.
  • performing face modeling on the frontal face image to obtain a face model includes: modeling a face of the frontal face image to obtain a high-dimensional feature of the frontal face image; The high-dimensional features of the frontal face image are dimensionally reduced to obtain the face model.
  • An embodiment of the present invention further provides a retrieval system for a face image, the system comprising: a pre-processing module that performs global image quality pre-processing on a face image to obtain a pre-processed face image; and a detection module Pre-processing face Performing face detection on the image to obtain a face in the pre-processed face image; and positioning a module, performing feature point positioning on the face in the pre-processed face image to obtain a feature point of the face; and a correction module And performing face correction on the face in the pre-processed face image according to the feature point to obtain a standard frontal face image; and modeling a module, performing face modeling on the frontal face image to obtain a person The face model; the comparison module adopts the face model to perform face comparison on the face image database, and obtain similar face sequences arranged in order of similarity.
  • the correction module performs geometric correction and/or image quality correction on the face of the pre-processed face image.
  • the system comprises: an interactive operation module, accepting an interactive operation of the user in at least one of the global image quality preprocessing, the face detection, the feature point location, and the face correction .
  • the interactive operation module accepts the interactive operation by the user using template selection and/or parameter adjustment.
  • the modeling module includes: a modeling unit that performs face modeling on the frontal face image to obtain a high-dimensional feature of the frontal face image; a dimension reduction unit that faces the frontal face image The high-dimensional feature is dimension-reduced to obtain the face model.
  • An embodiment of the present invention further provides a retrieval system for a face image, the system comprising: an acquisition module for acquiring an average face model; and an adjustment module for using the semantic description technique for the face component in the average face model Adjusting to obtain a standard frontal face image; modeling module, performing face modeling on the frontal face image to obtain a face model; comparing the module, using the face model to perform a face image database Faces are compared, and similar face sequences arranged in order of similarity are obtained.
  • the adjustment module performs the adjustment on the face component in the average face model by means of template selection and/or parameter adjustment.
  • the modeling module includes: a modeling unit that performs face modeling on the frontal face image to obtain a high-dimensional feature of the frontal face image; a dimension reduction unit that faces the frontal face image The high-dimensional feature is dimension-reduced to obtain the face model.
  • embodiments of the present invention can generate a standard, clear frontal face image from a face image that is unfavorable to retrieval factors, such as a blurred or presence gesture.
  • Embodiments of the present invention may generate a face image that conforms to the descriptor's description requirements through an interactive process of language description.
  • the embodiment of the present invention solves the problem that the current face recognition technology has a low recognition accuracy rate when the image is not clear or the face is deviated in the image, such as posture deflection, and the image is overcome in the prior art. When the factors are identified, the correct rate will be sharply The problem of falling has improved the accuracy and accuracy of face recognition.
  • the face image generated by the embodiment of the present invention can perform face search, which makes up for the defect that face search cannot be performed without a face image that can be input as input.
  • FIG. 1 is a schematic flow chart of an embodiment of a method for retrieving a face image according to the present invention.
  • FIG. 2 is a schematic flow chart of another embodiment of a method for retrieving a face image according to the present invention.
  • FIG. 3 is a schematic view showing the configuration of an embodiment of a face image retrieval system of the present invention.
  • FIG. 4 is a schematic structural view of another embodiment of a face image retrieval system of the present invention.
  • the input of the face retrieval application is a face image, which requires the presence of a face image, Used as input.
  • the face retrieval system cannot be used.
  • it is possible to generate a virtual face image by sketching and then make up for this deficiency as an input the virtual face image and true Real face images are very different and are often used as input for retrieval.
  • An embodiment of the method for retrieving a face image of the present invention generates a face image that can be used for face retrieval based on a real face image.
  • the actual face image as the input may not be a standard, clear frontal face image, and may still be obtained when the face image is not clear or the face in the face image has a posture deflection or the like which is not conducive to recognition. Clear, standard frontal face image.
  • the embodiment of the retrieval method mainly includes the following steps.
  • Step S110 Perform global image quality preprocessing on the real face image to improve the sharpness of the image and reduce the influence of image noise, illumination and other unfavorable factors on subsequent processing.
  • the image quality preprocessing mainly includes the processing of denoising, de-lighting effects, contrast and sharpness adjustment (such as improving contrast and sharpness) on the real face image.
  • Step S120 Perform face detection on the obtained pre-processed face image, and detect the obtained face from the face image (that is, the pre-processed face image) that has passed the global image quality pre-processing.
  • Step S130 performing feature point positioning on the face in the pre-processed face image obtained by the detection, and finding a feature point of the face in the real face image.
  • Step S140 correcting a face in the pre-processed face image based on the located feature points, and obtaining a standard frontal face image. Since the face image is subjected to image quality processing, the obtained face image is clear. Therefore, efficient and accurate face retrieval can be performed by using the standard frontal face image obtained by the correction.
  • the correction of the face in this step mainly includes geometric correction of the face and/or image quality correction of the face.
  • the geometric correction of the face mainly includes adjusting the face with the attitude deflection to the standard frontal face by 3D matching and/or 3D rotation technology.
  • the image quality correction of the face mainly includes processing according to the detected characteristics of the face itself, such as illumination correction (such as de-lighting or enhancing the illumination effect), and sharpness correction (boosting or reducing the resolution).
  • Step S150 performing face modeling based on the frontal face image of the standard.
  • the facial modeling is based on a five-guane Gabor/LBP feature extraction algorithm (Gabor features and LBP features are two commonly used image features in image processing technology), and the front face image of the standard is obtained. High dimensional features.
  • Step S160 using a feature reduction technique of the nonlinear subspace to obtain a standard frontal face image
  • the high-dimensional features are subjected to dimensionality reduction to obtain a face model.
  • the dimension of the face model may be 5000 dimensions or the like, for example.
  • Step S170 using the face model, performing face matching on the face image database, and obtaining a similar face sequence in descending order of similarity.
  • the face image database is face-to-face comparison, which can be a face-to-face comparison of different images.
  • face-to-face comparison can be a face-to-face comparison of different images.
  • the focus is that when there are large differences between the original images of the two models, the better similarity calculation results can still be obtained.
  • the learning method based on Metric Learning and calculating the Mahalanobis distance by using the metric matrix obtained by learning, even if the difference in face quality between synthetic face and library is large, the similarity obtained is still relatively accurate.
  • the user's interactive operation can be accepted for any intermediate processing result or the finally obtained corrected face image as needed, so as to improve the effect of the finally obtained face image.
  • the user adjusts the automatically positioned feature points, and corrects the feature points according to the adjustment performed by the user, so as to improve the detected features in the face.
  • the accuracy of the points lays a good foundation for the subsequent improvement of face correction quality.
  • the adjustment parameters between the two layers of the eyelids input by the user through the interactive operation are received, and the adjustment parameters between the two layers of the eyelids are determined according to the adjustment parameters input by the user.
  • the distance is finely adjusted; when the feature point is positioned on the right eye corner of the face, the angle parameter of the eye corner input by the user through the interactive operation is received, and according to the angle parameter of the eye angle input by the user, the right eye of the face is Fine-tuning the corners of the eyes, and so on.
  • the steps involved in the method of the present invention may be adjusted sequentially or selectively.
  • the image quality correction and geometric correction of the detected face can be adjusted in the order of the two, and the image quality correction can be performed first, or the geometric correction can be performed first. Quality correction.
  • the sequential adjustment or selective execution of these steps is primarily directed to flexible selection in different situations, and one of ordinary skill in the art can determine as desired.
  • the face, nose, cheekbones, eyes, and faces of the face can also be based on the face semantic description technique. Carefully adjust the components such as eyebrows and ears. In these steps, allowing the user to intervene in an interactive manner can further improve the accuracy of the face image processing results.
  • ⁇ can operate interactively.
  • a common image processing method to improve the overall image quality including denoising, de-lighting effects, contrast and clarity.
  • smoothing, median filtering, etc. may be selected for denoising processing
  • Retinex filtering, histogram equalization, and the like may be selected to perform de-lighting effects processing
  • a histogram correction algorithm may be selected to enhance image contrast.
  • Select algorithms such as image sharpening to improve the sharpness of the image.
  • the user may select a specific processing algorithm of the foregoing preprocessing according to different image characteristics.
  • the face position may be detected in the face image preprocessed by the Adaboost face detection algorithm, etc., so as to further perform subsequent processing on the face.
  • the user can select each of the faces one by one through an interactive operation, especially when the automatic detection fails, the user can select in the face image by means of frame selection.
  • the face to be treated.
  • a plurality of feature points are finely located in the face image to describe The contours of the face and the contours of the cheeks.
  • the user can modify the coordinates of the feature point through an interactive operation to accurately adjust the position of the feature point.
  • the face in the geometric correction process of the face, the face can be corrected to a standard frontal face based on an optimized 3D face matching algorithm such as 3D matching and 3D rotation. Since different face models have subtle differences for subsequent processing, the user can select an appropriate 3D face model through interactive operation to improve the accuracy and accuracy of geometric correction.
  • 3D face matching algorithm such as 3D matching and 3D rotation.
  • the 3D texture map and the learning-based super-resolution algorithm can be used to synthesize the face image onto the 3D model.
  • the face can be independently processed in accordance with the component.
  • the face can be divided into forehead, cheek, chin, ears, eyes, eyes, nose, nose, nose, mouth, mouth, hair, beard (upper lip), beard (chin) and glasses.
  • beard upper lip
  • beard chin
  • accepting the user's adjustment of the interactive operation of the face component may be an interactive operation provided by the user in various ways such as template selection and/or parameter adjustment.
  • the template selection refers to generating a plurality of templates for each component of the face in advance according to different parameters.
  • the user selects and replaces the corresponding component template that is considered to be close to the required component.
  • Template selection This method can usually be used to adjust the appearance of different parts.
  • the above parameter adjustment is to directly modify the parameters of the component, which can usually be used to adjust the position and size of the component.
  • a reasonable face template can be constructed in advance, that is, The entire part's appearance space can be evenly covered by a small number of templates.
  • Embodiments of the present invention perform principal component analysis (PCA) modeling for each component using more than 6 million true standard frontal face images, and then take the first 3 components for each component's PCA model (software corresponds to Three-dimensional arrangement, so that you can operate quickly by up, down, left and right and page keys, each component is in accordance with Take 7 samples and get a total of 343 templates.
  • PCA principal component analysis
  • ⁇ is a feature value corresponding to each principal component in the main component analysis, and is used to define a normalized component variation range of the corresponding principal component.
  • the first 10 components of the PCA model of each component can be provided as control parameters, so that more realistic components can be obtained.
  • a face image that can be used for face retrieval can also be obtained based on the average face model.
  • another embodiment of the method for searching a face image of the present invention mainly includes the following steps.
  • Step S210 obtaining an average face model.
  • the average face model may be obtained by counting front face image data based on a large amount of standards in advance.
  • Step S220 using a semantic description technique to adjust a face component in the average face model to obtain a standard frontal face image.
  • one component can be selected each time. Parts that have been previously adjusted can be adjusted after other parts have been adjusted.
  • the face can be independently processed in accordance with the component.
  • the face can be divided into forehead, cheek, chin, ears, eyes, eyes, nose, nose, nose, mouth, mouth, hair, beard (upper lip), beard (chin) and glasses.
  • beard upper lip
  • beard chin
  • the adjustment of the face components can be performed in a variety of ways, such as template selection and/or parameter adjustment.
  • the template selection refers to generating a plurality of templates for each component of the face in advance according to different parameters.
  • the user selects and replaces the corresponding component template that is considered to be close to the required component.
  • Template selection This method can usually be used to adjust the appearance of different parts.
  • the above parameter adjustment is to directly modify the parameters of the component, which can usually be used to adjust the position and size of the component. To construct a reasonable face template in advance, please refer to the content of the foregoing embodiment of the present invention.
  • Step S230 performing face modeling based on the frontal face image of the standard.
  • the facial modeling is based on a five-guane Gabor/LBP feature extraction algorithm (Gabor features and LBP features are two commonly used image features in image processing technology), and the front face image of the standard is obtained. High dimensional features.
  • step S240 the feature reduction technique of the nonlinear subspace is used to perform dimensionality reduction on the high dimensional features of the obtained standard frontal face image to obtain a face model.
  • the dimension of the face model may be 5000 dimensions or the like, for example.
  • Step S250 using the face model, performing face comparison on the face image database, and obtaining a similar face sequence from high to low in order of similarity.
  • the face image database is face-to-face comparison, which can be a face-to-face comparison of different images.
  • face-to-face comparison can be a face-to-face comparison of different images.
  • the focus is that when there are large differences between the original images of the two models, the better similarity calculation results can still be obtained.
  • the learning method based on Metric Learning and calculating the Mahalanobis distance by using the metric matrix obtained by learning, even if the difference in face quality between synthetic face and library is large, the similarity obtained is still relatively accurate.
  • the adjustment of the components of the face whether it is template selection or parameter adjustment, the entire process of semantic input of the synthesized face image is highly involved by the descriptor (ie, the user or the user). Interaction process.
  • the system can quickly synthesize different face images, and the descriptors subjectively judge whether they match the expectations.
  • the order of adjustment is not required. It is usually possible to make preliminary adjustments to the contours, etc., to obtain a relatively close appearance, and then repeatedly select the remaining components to make adjustments, and finally obtain a face that is consistent with the semantic description of the descriptor.
  • the difficulty of the adjustment can be reduced, thereby facilitating the implementation of the interaction process.
  • the division of the 15 components listed above in the embodiment of the present invention is only one of a plurality of division manners; the similar division manner should be considered equivalent to the present invention, and all are protected by the present invention. Within the scope.
  • the standard frontal face image obtained by the above embodiment of the present invention can be used for face retrieval. It should be noted that the foregoing positive face image based on a real face image or a standard generated based on the semantic description is not a real face image, but a virtual face image obtained after a certain process. Therefore, it is possible to introduce a heterogeneous face matching technique to obtain a better similarity ordering, and finally obtain a more accurate similar face sequence.
  • the standard frontal face image obtained according to the face image or the semantic description is actually a virtual face image.
  • the face image existing in the face image database is usually a standard photo (such as an ID card photo).
  • the embodiment of the present invention adopts a machine learning method, so that the similarity calculation expression used for face matching can tolerate a certain image quality difference. Add two faces of the same person, one is a standard photo, one is a raw If you don't do this, the similarity between the two may be very low, and the similarity will still be high considering the difference in image quality.
  • Metric Learning is one of the solutions to different quality.
  • the specific method is to directly obtain the measurement matrix considering the difference in image quality through large-scale sample direct learning, and then directly replace the covariance matrix in the standard Mahalanobis distance calculation.
  • the virtual face image (that is, the standard frontal face image in the foregoing embodiment) is compared with the face image in the face image database such as the standard document photo, and the person is not used.
  • the face image in the face image database is subjected to virtualization processing, such as the scheme of obtaining a standard frontal face image according to the input face image in the foregoing embodiment of the present invention, thereby avoiding the low efficiency and processing of the scheme in implementation.
  • virtualization processing such as the scheme of obtaining a standard frontal face image according to the input face image in the foregoing embodiment of the present invention, thereby avoiding the low efficiency and processing of the scheme in implementation.
  • the inherent defects of some information are easily lost in the process.
  • the face image retrieval system of the present invention mainly includes a preprocessing module 310, a detection module 320, a positioning module 330, a correction module 340, a modeling module 350, a comparison module 360, and the like.
  • the pre-processing module 310 performs global image quality pre-processing on the real face image to obtain a pre-processed face image.
  • the detecting module 320 is connected to the pre-processing module 310, and performs face detection on the pre-processed face image to obtain a face in the pre-processed face image.
  • the positioning module 330 is connected to the detecting module 320, and performs feature point positioning on the face in the pre-processed face image to obtain feature points of the face.
  • the correction module 340 is connected to the positioning module 330, and performs face correction on the face in the pre-processed face image according to the feature point to obtain a standard frontal face image.
  • the correction module 340 performs geometric correction and/or image quality correction on the face of the pre-processed face image.
  • the modeling module 350 is connected to the correction module 340, and performs face modeling on the standard frontal face image obtained by the correction module 340 to obtain a face model.
  • the comparison module 360 is connected to the modeling module 350, and uses the face model obtained by the modeling module 350 to perform face matching on the face image database to obtain a similar face sequence arranged in the order of similarity.
  • the modeling module 350 includes a modeling unit 351 and a dimension reduction unit 352.
  • the modeling unit 351 is connected to the correction module 340, and performs face modeling on the standard frontal face image obtained by the correction module 340 to obtain a high-dimensional feature of the standard frontal face image.
  • the dimension reduction unit 352 is connected to the modeling unit 351 and the comparison module 360, and performs dimensionality reduction on the high-dimensional features of the standard frontal face image obtained by the modeling unit 351 to obtain a face model.
  • the face image retrieval system of the present invention may further include an interactive operation module 370, and the pre-processing module 310, the detection module 320, the positioning module 330, the correction module 340, and the modeling module 350.
  • the module unit 351 and the dimension reduction unit 352, and the comparison module 360 are connected, in at least one process of global image quality preprocessing, face detection, feature point positioning, face correction, face modeling, and comparison. Accept user interactive actions.
  • the interactive operation module 370 accepts interactive operations by the user using template selection and/or parameter adjustment.
  • an embodiment of another retrieval system for a face image of the present invention includes an acquisition module 410, an adjustment module 420, a modeling module 430, a comparison module 440, and the like.
  • the acquisition module 410 obtains an average face model.
  • the adjustment module 420 is connected to the acquisition module 410, and uses a semantic description technique to adjust the face component in the average face model to obtain a standard frontal face image. Specifically, the adjustment module 420 adjusts the face component in the average face model by using template selection and/or parameter adjustment.
  • the modeling module 430 is connected to the adjustment module 420, and performs face modeling on the standard frontal face image obtained by the adjustment module 420 to obtain a face model.
  • the comparison module 440 is connected to the modeling module 430, and uses the face model obtained by the modeling module 430 to perform face comparison on the face image database to obtain a similar face sequence arranged in the order of similarity.
  • the modeling module 430 includes a modeling unit 431 and a dimension reduction unit 432.
  • the modeling unit 431 is connected to the adjustment module 420, and performs face modeling on the standard frontal face image obtained by the adjustment module 420 to obtain a high-dimensional feature of the standard frontal face image.
  • the dimension reduction unit 432 is connected to the modeling unit 431 and the comparison module 440, and performs dimensionality reduction on the high-dimensional features of the standard frontal face image obtained by the modeling unit 431 to obtain a face model.
  • Embodiments of the present invention satisfactorily solves the problem that the previous face retrieval technology has a significantly low image retrieval accuracy rate which is disadvantageous to face recognition factors such as blurring and posture deflection.
  • Embodiments of the present invention are capable of generating a standard frontal face image that is most similar to the description content by interactive operations. Moreover, the standard frontal face image can be used as an input for face retrieval.
  • a clearer face image can be generated by the blurred face image
  • a frontal face image can be generated for the face image with the posture deflection
  • the interactive description can be performed through the language.
  • Generating face images these three are not mutually exclusive, but can choose any one according to actual needs. Or any two or all of the three ways to generate a standard frontal face image. Further, it is also possible to perform a face search by using the generated face image on the front side as a search input, and obtain a highly accurate search result.
  • the automatic reconstruction method can be applied to the case where there is input but the input image quality is not high.
  • the semantic description is mainly for the case where no face image is available for input, such as generating a virtual face image according to the description of a person (such as a witness, etc.).
  • For automatic refactoring there may be some deviations due to the completion of the automation, which can be corrected by the human semantic description to ensure the accuracy of the final output of the face image and the subsequent retrieval.
  • semantic description because it is difficult for a person to accurately describe an image at a time, it is inevitable that the process of interaction is required to gradually change a part until it meets the expected expectations.

Landscapes

  • Engineering & Computer Science (AREA)
  • Library & Information Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种人脸图像的检索系统及方法，克服现有技术中的人脸识别技术在图像不清晰或者存在姿态偏转等不利于识别的因素时检索正确率较低的问题。该方法包括：对人脸图像进行全局的画质预处理；对预处理人脸图像进行人脸检测，获得其中的人脸并进行特征点定位，获得人脸的特征点；根据特征点对人脸进行人脸校正，获得正面人脸图像；对正面人脸图像进行人脸建模，获得人脸模型；采用人脸模型对人脸图像数据库进行人脸比对，得到相似人脸序列。本发明的实施例解决了当前人脸识别技术在图像不清晰或者图像中的人脸存在姿态偏转等不利于识别的因素时识别正确率较低的问题，提高了人脸识别的正确率和准确性。

Description

一种人脸图像的检索系统及方法
相关申请的交叉引用
本申请要求享有于2015年1月9日提交的名称为“一种人脸图像的检索系统及方法”的中国专利申请CN201510013920.1的优先权，该申请的全部内容通过引用并入本文中。
技术领域
本发明涉及智能安防技术领域，尤其涉及一种人脸图像的检索系统及方法。
背景技术
人脸作为图像与视频中最重要的视觉对象(Visual Object)之一，在计算机视觉、模式识别、多媒体技术研究中占有重要的地位。人脸检索是人脸信息处理及基于内容的检索等相关研究中的一个焦点问题，是近年来研究十分活跃的一个方向。它在智能人机接口、基于内容的检索、数字视频处理、保安等领域有着极为广泛的应用价值。
人脸检索技术主要是通过一定的建模和比对方法，从数据库(数据库规模通常上百万量级)中输出与所输入的人脸图像最相似的前N张人脸图像，并按照相似度对这N张人脸图像进行排序，得到按照相似度从高到低的相似人脸序列即检索结果。其中，N为大于等于1的正整数。
人脸检索是人脸识别的经典应用之一。虽然人脸识别技术已经有了多年的发展，但是目前的人脸识别技术仍然存在一些缺陷，比如当图像不清晰或者存在姿态偏转等不利于识别的因素时，检索的正确率较低，相比清晰图像或者没有姿态偏转的情形而言，检索正确率会急剧下降，而且这一下降的趋势在检索数据库增大时会显得更加突出。
发明内容
本发明所要解决的技术问题是为了克服现有技术中的人脸识别技术在图像不清晰或者存在姿态偏转等不利于识别的因素时检索正确率较低的问题。
为了解决上述技术问题，本发明的实施例首先提供了一种人脸图像的检索方法，该方法包括：对人脸图像进行全局的画质预处理，得到预处理人脸图像；对所述预处理人脸图像进行人脸检测，获得所述预处理人脸图像中的人脸；对所述预处理人脸图像中的人脸进行特征点定位，获得所述人脸的特征点；根据所述特征点对所述预处理人脸图像中的人脸进行人脸校正，获得正面人脸图像；对所述正面人脸图像进行人脸建模，获得人脸模型；采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
优选地，对所述预处理人脸图像中的人脸进行人脸校正，包括：对所述预处理人脸图像中的人脸进行人脸进行几何校正和/或画质校正。
优选地，该方法包括：在所述全局的画质预处理、所述人脸检测、所述特征点定位以及所述人脸校正的至少一个过程中接受用户的交互式操作。
优选地，接受用户的交互式操作，包括：接受用户采用模板选择和/或参数调整的所述交互式操作。
优选地，对所述正面人脸图像进行人脸建模，获得人脸模型，包括：对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
本发明的实施例还提供了一种人脸图像的检索方法，该方法包括：获取平均人脸模型；利用语义描述技术对所述平均人脸模型中的人脸部件进行调整，获得正面人脸图像；对所述正面人脸图像进行人脸建模，获得人脸模型；采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
优选地，对所述平均人脸模型中的人脸部件进行调整，包括：采用模板选择和/或参数调整的方式，对所述平均人脸模型中的人脸部件进行所述调整。
优选地，对所述正面人脸图像进行人脸建模，获得人脸模型，包括：对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
本发明的实施例还提供了一种人脸图像的检索系统，该系统包括：预处理模块，对人脸图像进行全局的画质预处理，得到预处理人脸图像；检测模块，对所述预处理人脸 图像进行人脸检测，获得所述预处理人脸图像中的人脸；定位模块，对所述预处理人脸图像中的人脸进行特征点定位，获得所述人脸的特征点；校正模块，根据所述特征点对所述预处理人脸图像中的人脸进行人脸校正，获得标准的正面人脸图像；建模模块，对所述正面人脸图像进行人脸建模，获得人脸模型；比对模块，采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
优选地，所述校正模块对所述预处理人脸图像中的人脸进行人脸进行几何校正和/或画质校正。
优选地，该系统包括：交互式操作模块，在所述全局的画质预处理、所述人脸检测、所述特征点定位以及所述人脸校正的至少一个过程中接受用户的交互式操作。
优选地，所述交互式操作模块接受用户采用模板选择和/或参数调整的所述交互式操作。
优选地，所述建模模块包括：建模单元，对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；降维单元，对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
本发明的实施例还提供了一种人脸图像的检索系统，该系统包括：获取模块，获取平均人脸模型；调整模块，利用语义描述技术对所述平均人脸模型中的人脸部件进行调整，获得标准的正面人脸图像；建模模块，对所述正面人脸图像进行人脸建模，获得人脸模型；比对模块，采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
优选地，所述调整模块采用模板选择和/或参数调整的方式，对所述平均人脸模型中的人脸部件进行所述调整。
优选地，所述建模模块包括：建模单元，对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；降维单元，对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
与现有技术相比，本发明的实施例可以根据一张模糊的或者存在姿态偏转等不利于检索因素的人脸图像生成一张标准的、清晰的正面人脸图像。本发明的实施例可以通过语言描述的交互过程生成一张符合描述者描述需求的人脸图像。本发明的实施例解决了当前人脸识别技术在图像不清晰或者图像中的人脸存在姿态偏转等不利于识别的因素时识别正确率较低的问题，克服了现有技术中图像存在不利于识别的因素时正确率就会急剧下 降的问题，提高了人脸识别的正确率和准确性。利用本发明的实施例所生成的人脸图像可以进行人脸检索，弥补了当前没有可以作为输入的人脸图像就无法进行人脸检索的缺陷。
本发明的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本发明的技术方案而了解。本发明的目的和其他优点可通过在说明书、权利要求书以及附图中所特别指出的结构和/或流程来实现和获得。
附图说明
附图用来提供对本发明的技术方案或现有技术的进一步理解，并且构成说明书的一部分。其中，表达本发明实施例的附图与本发明的实施例一起用于解释本发明的技术方案，但并不构成对本发明技术方案的限制。
图1为本发明的人脸图像的检索方法的实施例的流程示意图。
图2为本发明的人脸图像的检索方法的另一实施例的流程示意图。
图3为本发明的人脸图像的检索系统的实施例的构造示意图。
图4为本发明的人脸图像的检索系统的另一实施例的构造示意图。
具体实施方式
以下将结合附图及实施例来详细说明本发明的实施方式，借此对本发明如何应用技术手段来解决技术问题，并达成相应技术效果的实现过程能充分理解并据以实施。本发明实施例以及实施例中的各个特征，在不相冲突前提下可以相互结合，所形成的技术方案均在本发明的保护范围之内。
附图所示出的本发明的实施例的方法所包含的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且，虽然本发明的实施例的方法在流程图中示出了逻辑顺序，但是在某些情况下，本发明的实施例的方法也可以以不同于附图所示的顺序执行所示出或描述的步骤。
本发明的发明人在对现有技术进行研究时，还发现了现有技术存在如下的技术缺陷：人脸检索应用的输入是一张人脸图像，这就需要存在一张人脸图像，以用来作为输入。但是当没有可以作为输入的人脸图像时，就无法使用人脸检索系统。虽然可以通过素描生成一张虚拟的人脸图像然后作为输入来弥补这一不足，但是虚拟的人脸图像与真 实的人脸图像差异非常大，用作检索的输入，往往效果不佳。
本发明的人脸图像的检索方法的实施例，基于一张真实的人脸图像生成一张可以用于人脸检索的人脸图像。该作为输入的真实的人脸图像，可以不是标准的、清晰的正面人脸图像，在人脸图像不清晰或者人脸图像中的人脸存在姿态偏转等不利于识别的因素时，仍然可以获得清晰的、标准的正面人脸图像。如图1所示，该检索方法的实施例主要包括如下步骤。
步骤S110，对真实的人脸图像进行全局的画质预处理，提高图像的清晰度，降低图像噪声、光照等不利因素对后续处理的影响。
接收真实的人脸图像的输入，或者主动获取一张真实的人脸图像。然后对该真实的人脸图像进行全局的画质预处理，得到预处理人脸图像。该画质预处理，主要包括对该真实的人脸图像进行去噪点、去光照影响、对比度和清晰度调整(如提升对比度和清晰度)等处理。
步骤S120，对所得到的预处理人脸图像进行人脸检测，从经过了上述全局的画质预处理的人脸图像(也即预处理人脸图像)中检测获得人脸。
步骤S130，对检测获得的预处理人脸图像中的人脸进行特征点定位，找到该幅真实的人脸图像中人脸的特征点。
步骤S140，基于所定位到的特征点，对预处理人脸图像中的人脸进行校正，获得标准的正面人脸图像。由于对人脸图像进行了画质处理，因此所得到的人脸图像，是清晰的。从而，利用通过该校正所得到的标准的正面人脸图像，就可以进行高效、准确的人脸检索了。
本步骤中的对人脸进行校正，主要包括对人脸的几何校正和/或对人脸的画质校正。对人脸的几何校正，主要包括将存在姿态偏转的人脸通过3D匹配和/或3D旋转技术调整成标准的正面人脸。人脸的画质校正，主要包括根据所检测出的人脸本身的特性，进行光照校正(如去光照影响或者增强光照效果)、清晰度校正(提升或者降低清晰度)等处理。
步骤S150，基于该标准的正面人脸图像进行人脸建模。
本发明的实施例中，人脸建模采用的是基于五官的Gabor/LBP特征提取算法(Gabor特征和LBP特征是图像处理技术中两种常用的图像特征)，获得该标准的正面人脸图像的高维特征。
步骤S160，采用非线性子空间的特征降维技术，对所获得的标准的正面人脸图像的 高维特征进行降维处理，得到人脸模型。本发明的实施例中，该人脸模型的维度比如可以是5000维等。
步骤S170，采用该人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序从高到低的相似人脸序列。其中，对人脸图像数据库进行人脸比对，可以是异画质的人脸比对。异画质的人脸比对，重点在于当两个模型的原始图片存在较大差异时仍然能够得到比较好的相似度计算结果。采用基于Metric Learning的学习方法，并结合学习得到的度量矩阵计算马氏距离，即便合成人脸和库中人脸画质差异较大，得到的相似度仍然比较准确。
本发明的实施例，可以根据需要，针对上述的任何中间处理结果或者最后得到的经过校正的人脸图像，接受用户的交互式操作，以便提高最后得到的人脸图像的效果。比如，对所检测出的人脸进行特征点定位时，接收用户对自动定位出的特征点的调整，并根据用户所进行的调整对特征点进行修正，以提高所检测出的人脸中特征点的准确性，为后续提高人脸校正质量打好基础。具体地，对人脸中的双眼皮进行特征点的定位时，接收用户通过交互式操作所输入的两层眼皮之间的调整参数，并根据用户所输入的调整参数对两层眼皮之间的距离进行微调；在对人脸中右眼眼角进行特征点的定位时，接收用户通过交互式操作所输入的眼角的角度参数，并根据用户所输入的眼角的角度参数，对人脸中右眼的眼角进行微调，等等。
上述的本发明的实施例，只是一种较佳的实施方式，可以较高效率地得到较高质量的标准的正面人脸图像。在另一些实施例中，本发明的方法所包含的步骤，可以进行顺序上的调整或者选择性地执行。比如，对所检测出的人脸进行的画质校正和几何校正，二者执行的先后顺序是可以调整的，既可以先进行画质校正再进行几何校正，也可以先进行几何校正在进行画质校正。当然，还可以一边进行画质校正一边进行几何校正。再如，在对人脸进行画质校正时，既可以是仅进行光照校正，也可以是仅进行清晰度校正，还可以是既进行光照校正又进行清晰度校正。本发明的普通技术人员能够理解，这些步骤的顺序调整或者选择性地执行，主要是针对不同情形下的灵活选择，本领域的普通技术人员根据需要可以自行确定。
如图1所示，对真实的人脸图像进行预处理、人脸检测、特征定位以及人脸校正时，还可以基于人脸语义描述技术，对人脸的嘴巴、鼻子、颧骨、眼睛、眉毛、耳朵等部件进行细致的调整。在这些步骤中，允许用户以交互式操作方式的介入，可以进一步提高人脸图像处理结果的准确性。
比如，在对真实的人脸图像进行全局的画质预处理过程中，用户可以通过交互式操作 来选择通用的图像处理方法，以提升全局的图像画质，包括去噪点、去光照影响、提升对比度和清晰度等。具体地，可以选择平滑滤波、中值滤波等技术进行去噪点处理，可以选择Retinex滤波、直方图均衡等算法来进行去光照影响的处理，可以选择直方图校正等算法来提升图像的对比度，可以选择图像锐化等算法来提升图像的清晰度。本发明的实施例中，用户可以根据不同的图像特性，来选择上述预处理的具体处理算法。
再如，可以通过Adaboost人脸检测算法等在经过画质预处理的人脸图像中检测出人脸位置，以便进一步地针对人脸进行后续处理。在这个处理过程中，对于检测多个人脸的情形，用户可以通过交互式操作逐一选择其中的每个人脸，尤其是在自动检测失败时，用户可以通过框选的方式在人脸图像中选择给出待处理的人脸。
又如，在人脸的特征定位过程中，基于回归的人脸特征点定位方法，在人脸图像中精细定位出多个(本发明的一个实施例中用到了87个)特征点，以描出人脸五官和脸颊的轮廓等。当存在特征点定位不够精准时，用户可以通过交互式操作来修改特征点的坐标，以精确调整特征点的位置。
还如，在人脸的几何校正过程中，可以基于优化的3D人脸匹配算法，如3D匹配和3D旋转等，将人脸校正为标准的正面人脸。由于不同的人脸模型对于后续的处理存在着细微的差异，用户可以通过交互式操作，选择合适的3D人脸模型来提高几何校正的精度和准确度。
在人脸的画质校正过程中，可以采用3D纹理贴图和基于学习的超分辨率算法，将人脸图像合成到3D模型上。
本发明的实施例中，可以将人脸按照部件进行独立化处理。比如，可以将人脸分为额头、脸颊、下巴、双耳、双眼、双眉、鼻梁、鼻、鼻翼、嘴、嘴角、头发、胡须(上唇)、胡须(下巴)以及眼镜等15个部件。在对人脸的部件进行调整时，可以每次选中其中一个部件来进行。
基于人脸语义描述技术，接受用户对人脸的部件进行交互式操作的调整，可以是接受用户采用模板选择和/或参数调整等多种方式所提供的交互式操作。模板选择是指预先按照不同参数为人脸的各个部件生成众多模板，在进行人脸部件的调整时，用户选择其认为与所需要的部件比较接近的相应的部件模板并进行替换。模板选择这种方式通常可以用来调整不同的部件外观。上述的参数调整则是直接修改部件的参数，这种方式通常可以用来调整部件的位置、大小等。
对于模板选择的调整方法，为了提高调整效率，可以预先构造出合理的人脸模板，即 通过少数种类的模板就能够均匀覆盖整个部件外观空间。
本发明的实施例采用超过600万的真实的标准的正面人脸图像针对每个部件进行主元分析(PCA)建模，再对每个部件的PCA模型取前3个分量(软件上对应于三维排列，从而通过上下、左右和翻页键即可进行快速操作)，每个分量在按照
Figure PCTCN2015078302-appb-000001
取7个样点，共计得到343个模板。其中，λ为主元分析中每个主成分对应的特征值，用来限定对应主成分归一化变化范围。
对于参数调整方式，除位置和大小参数外，每个部件的PCA模型的前10个分量都可以作为控制参数进行提供，从而可以获得更加逼真的部件。
本发明的人脸图像的检索方法的实施例中，也可以基于平均人脸模型来获得可以用来进行人脸检索的人脸图像。
如图2所示，本发明的人脸图像的检索方法的另一实施例，主要包括如下步骤。
步骤S210，获取平均人脸模型。
本发明的实施例中，平均人脸模型可以是预先基于海量的标准的正面人脸图像数据统计得到。
步骤S220，利用语义描述技术对平均人脸模型中的人脸部件进行调整，获得标准的正面人脸图像。在调整过程中，每次可以选择一个部件来进行。先前进行过调整的部件，在对另一些部件进行调整之后，又可以继续进行调整。
本发明的实施例中，可以将人脸按照部件进行独立化处理。比如，可以将人脸分为额头、脸颊、下巴、双耳、双眼、双眉、鼻梁、鼻、鼻翼、嘴、嘴角、头发、胡须(上唇)、胡须(下巴)以及眼镜等15个部件。在对人脸的部件进行调整时，可以每次选中其中一个部件来进行。
对人脸的部件进行调整，可以采用模板选择和/或参数调整等多种方式。模板选择是指预先按照不同参数为人脸的各个部件生成众多模板，在进行人脸部件的调整时，用户选择其认为与所需要的部件比较接近的相应的部件模板并进行替换。模板选择这种方式通常可以用来调整不同的部件外观。上述的参数调整则是直接修改部件的参数，这种方式通常可以用来调整部件的位置、大小等。预先构造出合理的人脸模板，请参考本发明前述实施例中的内容。
步骤S230，基于该标准的正面人脸图像进行人脸建模。
本发明的实施例中，人脸建模采用的是基于五官的Gabor/LBP特征提取算法(Gabor特征和LBP特征是图像处理技术中两种常用的图像特征)，获得该标准的正面人脸图像的高维特征。
步骤S240，采用非线性子空间的特征降维技术，对所获得的标准的正面人脸图像的高维特征进行降维处理，得到人脸模型。本发明的实施例中，该人脸模型的维度比如可以是5000维等。
步骤S250，采用该人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序从高到低的相似人脸序列。其中，对人脸图像数据库进行人脸比对，可以是异画质的人脸比对。异画质的人脸比对，重点在于当两个模型的原始图片存在较大差异时仍然能够得到比较好的相似度计算结果。采用基于Metric Learning的学习方法，并结合学习得到的度量矩阵计算马氏距离，即便合成人脸和库中人脸画质差异较大，得到的相似度仍然比较准确。
本发明的实施例，对人脸的部件所进行的调整，无论是模板选择还是参数调整的方式，语义输入合成人脸图像的整个过程都是描述者(也即用户或者使用者)高度参与的交互过程。系统通过模板或者参数，可以快速地合成出不同的人脸图像，并由描述者主观判断是否与期望吻合。
对于不同的部件，调整的先后顺序并没有要求。通常可以先对轮廓等进行初步的调整，得到一个比较接近的外观，然后再反复选择剩下的各种部件来进行调整，最终可以得到与描述者语义描述一致的人脸。
本发明的实施例中，对于语义描述合成人脸图像时按照部件来进行调整，可以降低调整的难度，从而使交互过程便于实施。需要说明的是，本发明的实施例在上述所列举的15个部件的划分，仅是众多划分方式的一种；对于与之类似的划分方式应当认为与本发明等同，均在本发明的保护范围内。
利用本发明的上述实施例所得到的标准的正面人脸图像，可以用来进行人脸检索。需要说明的是，前述基于真实的人脸图像或者基于语义描述所生成的标准的正面人脸图像，毕竟不是真实的人脸图像，而是经过一定处理后得到的虚拟的人脸图像。因此，可以引入异画质人脸比对等技术，得到比较好的相似度排序，最终得到较为准确的相似人脸序列。
本发明的实施例，根据人脸图像或者语义描述所得到的标准的正面人脸图像，实际是虚拟的人脸图像。而人脸图像数据库中所存在的人脸图像，通常是标准的证件照(如身份证照片等)。本发明的实施例采用机器学习的方法，让用来进行人脸比对的相似度计算表达式能够容忍一定的画质差异。加入两张同一个人的人脸图像，一张是标准照，一张是生 成的虚拟人脸，如果不做此处理，则二者之间相似度可能会很低，而考虑了画质差异则相似度仍然会很高。
Metric Learning是解决异画质的方案之一。具体做法是通过大规模的样本直接学习得到考虑了画质差异的度量矩阵，然后就可以直接用来替换标准的马氏距离计算中的协方差矩阵。
本发明的技术方案，采用虚拟的人脸图像(即前述实施例中的标准的正面人脸图像)与标准证件照等人脸图像数据库中的人脸图像进行的比对，并没有采用对人脸图像数据库中的人脸图像进行虚拟化处理，比如本发明前述实施例中根据所输入的人脸图像获得标准的正面人脸图像的方案，避免了这种方案在实现时效率较低以及处理过程中容易丢失部分信息的固有缺陷。
如图3所示，本发明的人脸图像的检索系统，主要包括有预处理模块310、检测模块320、定位模块330、校正模块340、建模模块350以及比对模块360等。
预处理模块310，对真实的人脸图像进行全局的画质预处理，得到预处理人脸图像。
检测模块320，与预处理模块310相连，对预处理人脸图像进行人脸检测，获得预处理人脸图像中的人脸。
定位模块330，与检测模块320相连，对预处理人脸图像中的人脸进行特征点定位，获得人脸的特征点。
校正模块340，与定位模块330相连，根据特征点对预处理人脸图像中的人脸进行人脸校正，获得标准的正面人脸图像。校正模块340对预处理人脸图像中的人脸进行人脸进行几何校正和/或画质校正。
建模模块350，与校正模块340相连，对校正模块340获得的标准的正面人脸图像进行人脸建模，获得人脸模型。
比对模块360，与建模模块350相连，采用建模模块350所获得的人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
如图3所示，建模模块350包括建模单元351和降维单元352。建模单元351与校正模块340相连，对校正模块340获得的标准的正面人脸图像进行人脸建模，获得标准的正面人脸图像的高维特征。降维单元352，与建模单元351及比对模块360相连，对建模单元351获得的标准的正面人脸图像的高维特征进行降维，获得人脸模型。
本发明的人脸检索系统的实施例，还请参考前述图1所描述的本发明的人脸检索方法 的实施例。
如图3所示，本发明的人脸图像的检索系统，还可以包括交互式操作模块370，与预处理模块310、检测模块320、定位模块330、校正模块340、建模模块350中的建模单元351和降维单元352、以及比对模块360等均相连，在全局的画质预处理、人脸检测、特征点定位、人脸校正、人脸建模以及比对的至少一个过程中接受用户的交互式操作。该交互式操作模块370接受用户采用模板选择和/或参数调整的交互式操作。
如图4所示，本发明的人脸图像的另一种检索系统的实施例，包括获取模块410、调整模块420、建模模块430以及比对模块440等。
获取模块410获取平均人脸模型。
调整模块420，与获取模块410相连，利用语义描述技术对平均人脸模型中的人脸部件进行调整，获得标准的正面人脸图像。具体地，该调整模块420采用模板选择和/或参数调整的方式，对平均人脸模型中的人脸部件进行调整。
建模模块430，与调整模块420相连，对调整模块420获得的标准的正面人脸图像进行人脸建模，获得人脸模型。
比对模块440，与建模模块430相连，采用建模模块430所获得的人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
如图4所示，建模模块430包括建模单元431和降维单元432。建模单元431与调整模块420相连，对调整模块420获得的标准的正面人脸图像进行人脸建模，获得标准的正面人脸图像的高维特征。降维单元432，与建模单元431及比对模块440相连，对建模单元431获得的标准的正面人脸图像的高维特征进行降维，获得人脸模型。
本发明的人脸图像的检索系统的实施例，还请参考前述图2所描述的本发明的人脸图像的检索方法的实施例。
本发明的实施例良好地解决了先前人脸检索技术对存在模糊、姿态偏转等不利于人脸识别因素的图像检索正确率显著低下的问题。本发明的实施例能够通过交互式操作生成一张与描述内容最相似的标准的正面人脸图像。而且，该标准的正面人脸图像可以作为人脸检索的输入。
本发明的实施例中，基于自动重构的技术，可以通过模糊的人脸图像生成更加清晰的人脸图像，对存在姿态偏转的人脸图像生成正面的人脸图像，以及通过语言描述交互式生成人脸图像，这三者并不是相互排斥的，而是可以根据实际需要，选择其中任意一种、 或者其中任意两种、或者全部三种方式来生成标准的正面的人脸图像。而且，还可以把所生成的标准的正面的人脸图像作为检索输入来进行人脸检索，获得高准确度的检索结果。
基于人脸图像，采用自动重构的方式可以适用于有输入但输入图像质量不高的情况。语义描述主要是针对没有人脸图像可供输入的情况，比如根据人(如目击证人等)的描述生成虚拟的人脸图像。对于自动重构，由于自动化完成可能存在一些偏差，可以通过人的语义描述进行修正，保证最后输出的人脸图像以及后续所进行的检索的准确性。对于语义描述，因为人很难是一次性精确描述一个图像，所以必然需要交互的过程，逐渐地改动某个局部，直到符合描述着的预期。
本领域的技术人员应该明白，上述的本发明实施例所提供的系统的各组成部分，以及方法中的各步骤，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上。可选地，它们可以用计算装置可执行的程序代码来实现。从而，可以将它们存储在存储装置中由计算装置来执行，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。
虽然本发明所揭露的实施方式如上，但所述的内容仅为便于理解本发明技术方案而采用的实施方式，并非用以限定本发明。任何本发明所属领域内的技术人员，在不脱离本发明所揭露的精神和范围的前提下，可以在实施的形式及细节上进行任何的修改与变化，但本发明的专利保护范围，仍须以所附的权利要求书所界定的范围为准。

Claims (16)

  1. 一种人脸图像的检索方法，该方法包括：
    对人脸图像进行全局的画质预处理，得到预处理人脸图像；
    对所述预处理人脸图像进行人脸检测，获得所述预处理人脸图像中的人脸；
    对所述预处理人脸图像中的人脸进行特征点定位，获得所述人脸的特征点；
    根据所述特征点对所述预处理人脸图像中的人脸进行人脸校正，获得正面人脸图像；
    对所述正面人脸图像进行人脸建模，获得人脸模型；
    采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
  2. 根据权利要求1所述的方法，其中，对所述预处理人脸图像中的人脸进行人脸校正，包括：
    对所述预处理人脸图像中的人脸进行人脸进行几何校正和/或画质校正。
  3. 根据权利要求1所述的方法，其中，该方法包括：
    在所述全局的画质预处理、所述人脸检测、所述特征点定位以及所述人脸校正的至少一个过程中接受用户的交互式操作。
  4. 根据权利要求3所述的方法，其中，接受用户的交互式操作，包括：
    接受用户采用模板选择和/或参数调整的所述交互式操作。
  5. 根据权利要求1所述的方法，其中，对所述正面人脸图像进行人脸建模，获得人脸模型，包括：
    对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；
    对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
  6. 一种人脸图像的检索方法，该方法包括：
    获取平均人脸模型；
    利用语义描述技术对所述平均人脸模型中的人脸部件进行调整，获得正面人脸图像；
    对所述正面人脸图像进行人脸建模，获得人脸模型；
    采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
  7. 根据权利要求6所述的方法，其中，对所述平均人脸模型中的人脸部件进行调整，包括：
    采用模板选择和/或参数调整的方式，对所述平均人脸模型中的人脸部件进行所述调整。
  8. 根据权利要求6所述的方法，其中，对所述正面人脸图像进行人脸建模，获得人脸模型，包括：
    对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；
    对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
  9. 一种人脸图像的检索系统，该系统包括：
    预处理模块，对人脸图像进行全局的画质预处理，得到预处理人脸图像；
    检测模块，对所述预处理人脸图像进行人脸检测，获得所述预处理人脸图像中的人脸；
    定位模块，对所述预处理人脸图像中的人脸进行特征点定位，获得所述人脸的特征点；
    校正模块，根据所述特征点对所述预处理人脸图像中的人脸进行人脸校正，获得标准的正面人脸图像；
    建模模块，对所述正面人脸图像进行人脸建模，获得人脸模型；
    比对模块，采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
  10. 根据权利要求9所述的系统，其中：
    所述校正模块对所述预处理人脸图像中的人脸进行人脸进行几何校正和/或画质校正。
  11. 根据权利要求9所述的系统，其中，该系统包括：
    交互式操作模块，在所述全局的画质预处理、所述人脸检测、所述特征点定位以及所 述人脸校正的至少一个过程中接受用户的交互式操作。
  12. 根据权利要求11所述的系统，其中：
    所述交互式操作模块接受用户采用模板选择和/或参数调整的所述交互式操作。
  13. 根据权利要求9所述的系统，其中，所述建模模块包括：
    建模单元，对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；
    降维单元，对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
  14. 一种人脸图像的检索系统，该系统包括：
    获取模块，获取平均人脸模型；
    调整模块，利用语义描述技术对所述平均人脸模型中的人脸部件进行调整，获得标准的正面人脸图像；
    建模模块，对所述正面人脸图像进行人脸建模，获得人脸模型；
    比对模块，采用所述人脸模型，对人脸图像数据库进行人脸比对，得到按相似度顺序进行排列的相似人脸序列。
  15. 根据权利要求14所述的系统，其中：
    所述调整模块采用模板选择和/或参数调整的方式，对所述平均人脸模型中的人脸部件进行所述调整。
  16. 根据权利要求14所述的系统，其中，所述建模模块包括：
    建模单元，对所述正面人脸图像进行人脸建模，获得所述正面人脸图像的高维特征；
    降维单元，对所述正面人脸图像的高维特征进行降维，获得所述人脸模型。
PCT/CN2015/078302 2015-01-09 2015-05-05 一种人脸图像的检索系统及方法 WO2016110030A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510013920.1A CN105825163A (zh) 2015-01-09 2015-01-09 一种人脸图像的检索系统及方法
CN201510013920.1 2015-01-09

Publications (1)

Publication Number Publication Date
WO2016110030A1 true WO2016110030A1 (zh) 2016-07-14

Family

ID=56355454

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/078302 WO2016110030A1 (zh) 2015-01-09 2015-05-05 一种人脸图像的检索系统及方法

Country Status (2)

Country Link
CN (1) CN105825163A (zh)
WO (1) WO2016110030A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110377389A (zh) * 2019-07-12 2019-10-25 北京旷视科技有限公司 图像标注导引方法、装置、计算机设备和存储介质
CN110782419A (zh) * 2019-10-18 2020-02-11 杭州趣维科技有限公司 一种基于图形处理器的三维人脸融合方法及系统
CN111951373A (zh) * 2020-06-30 2020-11-17 重庆灵翎互娱科技有限公司 一种人脸图像的处理方法和设备
CN112733680A (zh) * 2020-12-31 2021-04-30 南京视察者智能科技有限公司 一种基于监控视频流生成高质量人脸图像的模型训练方法、提取方法、装置和终端设备
CN113378779A (zh) * 2021-06-30 2021-09-10 广州卓腾科技有限公司 一种基于图像识别的户籍图像判断方法、系统和可读存储介质
US11120252B2 (en) 2016-12-28 2021-09-14 Zhejiang Dahua Technology Co., Ltd. Methods, systems, and media for evaluating images
CN114168684A (zh) * 2021-12-10 2022-03-11 南威软件股份有限公司 一种基于异步机制的人脸建模入库服务实现方法及装置

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106408658B (zh) * 2016-10-31 2021-06-15 北京旷视科技有限公司 人脸建模的方法及装置
CN107358573A (zh) * 2017-06-16 2017-11-17 广东欧珀移动通信有限公司 图像美颜处理方法和装置
CN107679235B (zh) * 2017-10-25 2021-09-21 成都尽知致远科技有限公司 基于云平台的检索系统
CN107947871A (zh) * 2017-11-21 2018-04-20 韩劝劝 一种控制路由器的辐射量的方法
CN108874889B (zh) * 2018-05-15 2021-01-12 中国科学院自动化研究所 基于目标体图像的目标体检索方法、系统及装置
CN109086703A (zh) * 2018-07-23 2018-12-25 广州跨行网络科技有限公司 一种非接触式掌纹识别系统及其实施方法
CN108946349A (zh) * 2018-07-23 2018-12-07 广州跨行网络科技有限公司 一种人脸识别和云视频对讲的电梯控制系统及其实施方法
CN109508660A (zh) * 2018-10-31 2019-03-22 上海交通大学 一种基于视频的au检测方法
CN110457998B (zh) * 2019-06-27 2020-07-28 北京旷视科技有限公司 影像数据关联方法和设备、数据处理设备及介质
CN113313631B (zh) * 2021-06-10 2024-05-10 北京百度网讯科技有限公司 图像渲染方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008064395A1 (en) * 2006-08-18 2008-06-05 National Ict Australia Limited Facial feature processing
CN101377814A (zh) * 2007-08-27 2009-03-04 索尼株式会社 人脸图像处理设备、人脸图像处理方法以及计算机程序
CN103632147A (zh) * 2013-12-10 2014-03-12 公安部第三研究所 实现面部特征标准化语义描述的系统及方法
CN104112152A (zh) * 2013-10-30 2014-10-22 北京安捷天盾科技发展有限公司 二维码生成装置、人像识别装置和身份验证装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100458831C (zh) * 2006-06-01 2009-02-04 北京中星微电子有限公司 人脸模型训练模块及方法、人脸实时认证系统及方法
CN102004899B (zh) * 2010-11-03 2012-09-26 无锡中星微电子有限公司 一种人脸认证系统及方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008064395A1 (en) * 2006-08-18 2008-06-05 National Ict Australia Limited Facial feature processing
CN101377814A (zh) * 2007-08-27 2009-03-04 索尼株式会社 人脸图像处理设备、人脸图像处理方法以及计算机程序
CN104112152A (zh) * 2013-10-30 2014-10-22 北京安捷天盾科技发展有限公司 二维码生成装置、人像识别装置和身份验证装置
CN103632147A (zh) * 2013-12-10 2014-03-12 公安部第三研究所 实现面部特征标准化语义描述的系统及方法

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11120252B2 (en) 2016-12-28 2021-09-14 Zhejiang Dahua Technology Co., Ltd. Methods, systems, and media for evaluating images
CN110377389A (zh) * 2019-07-12 2019-10-25 北京旷视科技有限公司 图像标注导引方法、装置、计算机设备和存储介质
CN110377389B (zh) * 2019-07-12 2022-07-26 北京旷视科技有限公司 图像标注导引方法、装置、计算机设备和存储介质
CN110782419A (zh) * 2019-10-18 2020-02-11 杭州趣维科技有限公司 一种基于图形处理器的三维人脸融合方法及系统
CN110782419B (zh) * 2019-10-18 2022-06-21 杭州小影创新科技股份有限公司 一种基于图形处理器的三维人脸融合方法及系统
CN111951373A (zh) * 2020-06-30 2020-11-17 重庆灵翎互娱科技有限公司 一种人脸图像的处理方法和设备
CN111951373B (zh) * 2020-06-30 2024-02-13 重庆灵翎互娱科技有限公司 一种人脸图像的处理方法和设备
CN112733680A (zh) * 2020-12-31 2021-04-30 南京视察者智能科技有限公司 一种基于监控视频流生成高质量人脸图像的模型训练方法、提取方法、装置和终端设备
CN112733680B (zh) * 2020-12-31 2024-02-13 南京视察者智能科技有限公司 一种基于监控视频流生成高质量人脸图像的模型训练方法、提取方法、装置和终端设备
CN113378779A (zh) * 2021-06-30 2021-09-10 广州卓腾科技有限公司 一种基于图像识别的户籍图像判断方法、系统和可读存储介质
CN114168684A (zh) * 2021-12-10 2022-03-11 南威软件股份有限公司 一种基于异步机制的人脸建模入库服务实现方法及装置
CN114168684B (zh) * 2021-12-10 2023-08-08 清华大学 一种基于异步机制的人脸建模入库服务实现方法及装置

Also Published As

Publication number Publication date
CN105825163A (zh) 2016-08-03

Similar Documents

Publication Publication Date Title
WO2016110030A1 (zh) 一种人脸图像的检索系统及方法
Ling et al. Face verification across age progression using discriminative methods
WO2019128508A1 (zh) 图像处理方法、装置、存储介质及电子设备
US9449432B2 (en) System and method for identifying faces in unconstrained media
US9317785B1 (en) Method and system for determining ethnicity category of facial images based on multi-level primary and auxiliary classifiers
US9818024B2 (en) Identifying facial expressions in acquired digital images
Lee et al. Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos
US9443325B2 (en) Image processing apparatus, image processing method, and computer program
US20120288167A1 (en) Pose-robust recognition
CN105139000B (zh) 一种去除眼镜痕迹的人脸识别方法及装置
EP2115662A2 (en) Separating directional lighting variability in statistical face modelling based on texture space decomposition
CN111178195A (zh) 人脸表情识别方法、装置及计算机可读存储介质
WO2020037962A1 (zh) 面部图像校正方法、装置及存储介质
WO2010133661A1 (en) Identifying facial expressions in acquired digital images
Zhao et al. Applying contrast-limited adaptive histogram equalization and integral projection for facial feature enhancement and detection
US9514354B2 (en) Facial analysis by synthesis and biometric matching
CN110647782A (zh) 三维人脸重建与多姿态人脸识别方法及装置
KR20210145781A (ko) 고운 결로 피부 텍스처를 향상시키는 안면 유연화 시스템 및 방법
Zhou et al. Shape-appearance-correlated active appearance model
Mahmood et al. Multi-order statistical descriptors for real-time face recognition and object classification
US20220207917A1 (en) Facial expression image processing method and apparatus, and electronic device
CN117557688A (zh) 肖像生成模型训练方法、装置、计算机设备和存储介质
Ayinde et al. Region-based face detection
US9786030B1 (en) Providing focal length adjustments
Zamuner et al. A pose-adaptive constrained local model for accurate head pose tracking

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15876536

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 03.01.2018)

122 Ep: pct application non-entry in european phase

Ref document number: 15876536

Country of ref document: EP

Kind code of ref document: A1