WO2018076495A1

WO2018076495A1 - Method and system for retrieving face image

Info

Publication number: WO2018076495A1
Application number: PCT/CN2016/110134
Authority: WO
Inventors: 李祥明; 王俊; 吴祖玉
Original assignee: 广州炒米信息科技有限公司
Priority date: 2016-10-28
Filing date: 2016-12-15
Publication date: 2018-05-03
Also published as: CN106503686A

Abstract

Disclosed are a method and system for retrieving a face image. The method comprises: determining feature point position information of a face image to be retrieved according to a pre-set face feature point model (S101); performing calculation to obtain a facial feature of the face image to be retrieved according to the feature point position information (S102); in combination with the facial feature, obtaining a feature vector of the face image to be retrieved (S103); and retrieving face images pre-stored in a database according to the feature vector to obtain a face image most similar to the face image to be retrieved (S104). Feature point position information of a face image to be retrieved is determined according to a pre-set face feature point model, and then a feature vector of the face image to be retrieved is obtained according to the determined feature point position information. Determined feature points of the face image to be retrieved are limited, and the determined feature point position information of the face image to be retrieved is two-dimensional coordinate information. Thus, a face image most similar to the face image to be retrieved can be rapidly and efficiently retrieved from a large quantity of face images according to the feature vector.

Description

Method and system for retrieving face images

Technical field

The present invention relates to the field of image recognition technologies, and in particular, to a method and system for retrieving a face image.

Background technique

In recent years, with the rapid growth of the Internet, the explosive growth of Internet pictures and the increasing popularity of security monitoring devices, massive amounts of face image data are generated every day. In such a large-scale face database, you can quickly retrieve your own interest. Part of the face image has become an urgent need.

At present, face detection and recognition technology has been widely used in various fields and has become a research hotspot. Similar face search, that is, given a face to be found, it is necessary to find a result similar to its appearance from an image database containing hundreds of thousands or more faces, and return a sequence of pictures sorted according to their similarity. Faced with a large amount of face image data, it is necessary to effectively organize the face data and search and analyze the face data, so as to efficiently search for face images. The traditional method is to extract high-dimensional complex features such as LBP features and ORB features of face images and to traverse the entire face database linearly to find the most similar faces, and the retrieval speed is slow.

Summary of the invention

Based on this, a method and system for retrieving a face image are provided to overcome the problem of slow face retrieval.

A method for retrieving a face image includes: determining feature point position information of a face image to be retrieved according to a preset face feature point model; and calculating a face feature of the face image to be retrieved according to the feature point position information; Combining the facial features to obtain a feature vector of the face image to be retrieved; and searching for a face image saved in advance in the database according to the feature vector to obtain a similarity with the face image to be retrieved The highest face image.

In view of the deficiencies of the conventional technology, a system for retrieving face images is also provided.

A system for retrieving a face image includes: a feature point position information determining module, a face feature calculating module, a feature vector acquiring module, and a retrieving module; and the feature point position information determining module is configured to use the preset face feature point The model determines the feature point location information of the face image to be retrieved; the facial feature calculation module is configured to calculate the facial feature of the face image to be retrieved according to the feature point location information; and the feature vector acquisition module is configured to combine The facial feature obtains a feature vector of the face image to be retrieved; the searching module is configured to retrieve a face image pre-saved in the database according to the feature vector, and obtain a face with the highest similarity to the face image to be retrieved image.

The beneficial effect of the solution is that the feature point position information of the face image to be retrieved is determined according to the preset face feature point model, and then the feature vector of the face image to be retrieved is obtained according to the determined feature point position information; The feature points of the face image are limited, and the determined feature point position information of the face image to be retrieved is two-dimensional coordinate information. Therefore, according to the feature vector, the face image with the highest similarity to the face image to be retrieved can be quickly and efficiently retrieved from the massive face image in the database.

DRAWINGS

FIG. 1 is a schematic flowchart of a method for retrieving a face image according to an embodiment; FIG.

2 is a schematic flowchart of a method for retrieving a face image according to another embodiment;

3 is a schematic diagram of a face feature point of the embodiment of FIG. 2;

4 is a schematic structural diagram of a system for retrieving a face image according to an embodiment.

detailed description

In order to further illustrate the technical means and effects achieved by the present invention, the following is combined with the accompanying drawings. The preferred embodiments of the present invention are described clearly and completely.

FIG. 1 is a schematic flow chart of a method for retrieving a face image according to an embodiment. As shown in FIG. 1, a method for retrieving a face image includes:

S101. Determine feature point location information of the face image to be retrieved according to the preset face feature point model.

In the embodiment, the method for determining the preset face feature point model is to determine a plurality of feature points in the face contour, the eyebrow contour, the nose contour, the eye contour, and the mouth contour of the face image. In the face detection, the feature of the face image to be retrieved may be extracted according to the preset face feature point model by using ASM (Active Shape Model), AAM (Active Appreance Model) or DLIB method. point. DLIB is a machine learning C++ library that contains many algorithms commonly used in machine learning. In this embodiment, the face feature points of the face image to be retrieved are extracted by DLIB, and then the feature point position information of the extracted face image to be retrieved is obtained; the extracted face feature points of the face image to be retrieved may be face contours. At least two types of feature points of the feature point, the feature point of the eyebrow contour, the feature point of the nose contour, the feature point of the eye contour, and the feature point of the mouth contour are generally extracted from the five types of feature points of the face image to be retrieved. come out. In addition to determining the feature point position information, the face position information can also be determined.

S102. Calculate a facial feature of the face image to be retrieved according to the feature point location information.

In this embodiment, the feature point location information is two-dimensional coordinate information, and the facial features of the face image to be retrieved can be obtained conveniently and quickly by a simple four-step operation, the facial features including a face shape, an eyebrow shape, and a chin. Type, eyebrow sag, nose length to width ratio, eyebrow distance, eyebrow maximum point, angle between the inner and outer eye line and the horizontal line, the angle between the eyebrow and the width of the eye.

S103. Combine the facial features to obtain a feature vector of the face image to be retrieved.

In this embodiment, the feature vector contains a number of elements, each of which represents each facial feature. For example, the feature vector is

The first element a ₁ represents the face shape, the second element a ₂ represents the eyebrow type, the third element a ₃ represents the eyebrow sag, the fourth element a ₄ represents the eyebrow length ratio, and the fifth element a ₅ represents the eyebrow distance, and so on.

S104. Search for a face image saved in advance in the database according to the feature vector, and obtain a face image with the highest similarity to the face image to be retrieved.

The face image pre-stored in the database is generally a tens of thousands of massive face images, and the face image pre-stored in the database is retrieved according to the feature vector, and the face image with the highest similarity to the face image to be retrieved is obtained. Is the face image most similar to the face image to be retrieved.

Determining the feature point position information of the face image to be retrieved according to the preset face feature point model, and then obtaining the feature vector of the face image to be retrieved according to the determined feature point position information; the feature point of the face image to be retrieved determined It is limited that the determined feature point position information of the face image to be retrieved is two-dimensional coordinate information. Therefore, according to the feature vector, the face image with the highest similarity to the face image to be retrieved can be quickly and efficiently retrieved from the massive face image in the database.

FIG. 2 is a schematic flowchart of a method for retrieving a face image according to another embodiment. As shown in FIG. 2, a method for retrieving a face image includes:

S201. Determine feature point location information of the face image to be retrieved according to the preset face feature point model;

FIG. 3 is a schematic diagram of a face feature point of the present embodiment. As shown in FIG. 3, 60 face feature points are extracted according to the method for extracting face feature points by DLIB. Among them, the face contour feature point number is 0-16, a total of 16, the left eyebrow contour feature point number is 17-21, a total of 5, the right eyebrow contour feature point number is 22-26, a total of 5, nose The number of the contour feature points is 27-35, a total of 9, the left eye contour feature points are 36-41, a total of 6, the right eye contour feature points are numbered 42-47, a total of 6, mouth contour feature points The 13 serial numbers are 48-60, a total of 13.

S202. Correct the face of the face image to be retrieved according to the feature point location information.

In this embodiment, the face of the face image to be retrieved is corrected to the posture according to the feature point position information, and the frontal posture of the face is that the two eyes are on one horizontal line, the left and right faces are substantially uniform in size, and there is no pitch. By correcting the face to the frontal posture, the feature point position coordinates on the face image can be unified to the vertical coordinate system, and more accurate feature point position information can be obtained.

S203. Calculate a facial feature of the face image to be retrieved according to the feature point location information.

The face parameter is calculated according to the feature point position information of the face contour. The face is divided into a round face, an elliptical face, a duck face, a pear face, a national face and others. In this embodiment, the face parameter of the face image to be retrieved can be obtained by:

Obtaining the distance data D _{1 of the} feature point 1 and the feature point 15 as the face width data; acquiring the distance data D ₂ from the midpoint of the line defined by the feature point 21 and the feature point 22 to the

feature point

8, 3*D ₂ / 2 as the face length data; the face length data is divided by the face width data to obtain the face length face width ratio; according to the preset face length face width ratio and the face type correspondence, the corresponding face shape is obtained.

According to the feature point position information of the eyebrow contour, the eyebrow shape parameter is calculated. The eyebrow parameter is mainly represented by the highest point of the eyebrow, the sag of the eyebrow and the angle of the eyebrow. The eyebrow shape can be a straight eyebrow and a curved eyebrow. According to the brow and the eyebrow, The eyebrow feature point position information determines the eyebrow shape. The vertical axis coordinate data of the eyebrow contour feature point is obtained, and the point at which the vertical axis coordinate data is the largest is determined as the highest point of the eyebrow. The difference data D ₃ between the feature point 18 and the vertical axis coordinate of the feature point 17 is obtained, and the distance data D _{4 of the} feature point 17 and the feature point 21 is obtained, and D ₃ /D _{4 is taken} as the eyebrow sag. Obtaining the connection l _{1 of the} feature point 20 and the feature point 21, acquiring the feature point 22 and the feature point 23 connecting the line l ₂ , and obtaining the angle data of the connection lines l ₁ and l ₂ as the angle inside the eyebrow.

The chin type parameter is calculated according to the feature point position information of the face contour; the chin type can be divided into a pointed chin and a round chin; the chin type parameter is determined by: obtaining the connection point _{3 of the} feature point 5 and the feature point 7 to acquire the feature point 9 is connected to the feature point 11 l ₄ , and the angle data D _{5 of the} connection line l ₃ and the connection line l ₄ is obtained; and the distance data D _{6 of the} feature point 5 and the feature point 11 is obtained as the chin width parameter. Combine the angle data D ₅ and the chin width parameter to obtain the chin type parameter. Further, if the chin width parameter is small and the angle data D _{5 is} small, it is a pointed chin parameter, otherwise it is a round chin parameter.

The nose aspect ratio is calculated from the feature point position information of the nose contour. Obtaining feature points and feature point 27 31 D ₇ and the distance data 31 and the feature point from the feature point data 35 D _8, the D ₇ / D ₈ as a nasal aspect ratio.

According to the feature point position information of the eye contour and the position information of the eyebrow contour feature point, the angle between the inner and outer eye line and the horizontal line, the eyebrow distance and the eye width are calculated. The distance data D ₁₀ between the feature point 20 and the feature point 38 from the data D ₉ and the feature point 23 and the feature point 43 is obtained, and (D ₉ + D ₁₀ )/2 is taken as the eyebrow distance. The line _{16 of the} feature point 36 to the feature point 39 is obtained, and the angle data of the line ₁₆ and the horizontal line is taken as the angle between the inner and outer eye line and the horizontal line. Obtaining feature points from the feature data D 36 points 39 42 ₁₁ and the feature point from the feature point data 45 D _12, the _{_{(D 11 + D 12) /}} 2 as eye width.

S204. The facial features are ranked according to a predetermined level standard to obtain a hierarchical level of the facial features.

As a preferred embodiment, the level criteria of the facial features are determined prior to grading the facial features by a predetermined level of criteria. For example, the method for determining the level standard of the eyebrows is to count the eyebrow thickness of a certain number of faces, which may be 10,000 people, calculate the mean value, variance, and quantize to level 5. All of the set facial features are ranked according to a predetermined level standard to obtain a hierarchical level of all facial features.

S205: Determine a semantic feature corresponding to the facial feature according to a preset semantic feature determining rule and a hierarchical level; and obtain a semantic feature of the facial image to be retrieved according to the semantic feature corresponding to each facial feature.

As a preferred embodiment, the semantic feature corresponding to the facial feature is obtained according to a preset semantic feature determining rule and a hierarchical level; for example, if the eyebrow thickness level is 4, the corresponding eyebrow is determined. The semantic feature is “enhanced eyebrow”, and the eye size level is 4, then the semantic feature corresponding to the eye is determined to be “big eye”; the semantic feature of the combination of the two “rich eyebrows” can be used as the semantics of the face image to be retrieved. feature.

S206. Combine the number of hierarchical levels corresponding to each facial feature to obtain a feature vector of the face image to be retrieved.

For example, combining the number of levels corresponding to each facial feature, and obtaining the feature vector of the face image to be retrieved is

The first element 4 represents the face series, the second element 2 represents the eyebrow series, the third element 5 represents the eyebrow sag level, the fourth element 1 represents the nose aspect ratio, and the fifth element 3 Represents the eyebrow eye level, and so on.

S207. Perform a search in the entire database according to the semantic features of the face image to be retrieved, and determine a search range from a face image saved in advance in the database.

As a preferred embodiment, the semantic features of the face, eyebrow, nose, eyes, and chin of the face image to be retrieved are searched in the entire database, and the search range is determined from the face image saved in advance in the database. Can greatly reduce the search space.

S208. Compare the feature vector of the face image to be retrieved with the feature vector of each face image in the search range to obtain a vector difference value; and find a face image with the smallest vector difference value in the search range. And determining a face image having the highest similarity with the face image to be retrieved, that is, a face image most similar to the face image to be retrieved.

In this embodiment, the feature vector of the face image to be retrieved is compared with the feature vector of each face image in the search range to obtain a vector difference value, and the database is waiting according to the obtained vector difference value. The face image in the search range of the face image is searched, and the face image with the small difference between the vector and the face image to be retrieved is ranked in front, and the face image with the highest ranking is selected to be similar to the face image to be searched. The highest degree of face image.

The beneficial effect of the embodiment is that the image of the face to be retrieved is determined according to the preset face feature point model. The feature point position information is obtained according to the determined feature point position information, and the feature vector and the semantic feature of the face image to be retrieved are firstly retrieved according to the semantic feature of the face image to be retrieved, and are retrieved from the database in advance. The search range is determined in the face image; and then the feature vector of the face image to be retrieved can be used to retrieve the face image with the highest similarity to the face image to be retrieved within the search range more quickly and efficiently.

4 is a schematic structural diagram of a system for retrieving a face image according to an embodiment. As shown in FIG. 4, a system for retrieving a face image includes: a feature point location information determining module 101, a facial feature computing module 102, a feature vector obtaining module 103, and a retrieval module 104; and the feature point location information determining module 101 And determining feature point position information of the face image to be retrieved according to the preset face feature point model; the face feature calculation module 102, configured to calculate a face of the face image to be retrieved according to the feature point position information a feature vector acquiring module 103, configured to combine the facial features to obtain a feature vector of a face image to be retrieved, and the searching module 104, configured to retrieve a pre-saved face image in the database according to the feature vector , obtaining a face image with the highest similarity to the face image to be retrieved.

As a preferred embodiment, the system for retrieving a face image further includes: a semantic feature acquisition module (not shown); the semantic feature acquisition module, configured to obtain a face image to be retrieved according to the facial feature a semantic feature; the retrieval module is further configured to perform a retrieval in the entire database according to the semantic feature of the face image to be retrieved, and determine a retrieval range from a pre-saved face image in the database; The search is performed within the search range to obtain a face image with the highest similarity to the face image to be retrieved.

As a preferred embodiment, the system for retrieving a face image further includes: a grading module (not shown); the grading module is configured to classify the facial features according to a predetermined grading standard, and obtain a Deriving the hierarchical level of the facial features; determining the rules and ranking levels according to the preset semantic features The semantic feature corresponding to the facial feature; combining the semantic features corresponding to each facial feature to obtain a semantic feature of the face image to be retrieved. The feature vector obtaining module is further configured to combine the hierarchical levels corresponding to the facial features to obtain a feature vector of the face image to be retrieved.

As a preferred embodiment, the searching module is further configured to compare the feature vector of the face image to be retrieved with the feature vector of each face image in the search range to obtain a vector difference value; The face image with the smallest vector difference in the range is determined as the face image with the highest similarity to the face image to be retrieved.

As a preferred embodiment, the system for retrieving a face image further includes: a face pose correction module (not shown); the face pose correction module, configured to retrieve a map based on the feature point location information The face of the face image is corrected, and in general, the face is corrected to the frontal posture. The frontal posture of the face is that the two eyes are on one horizontal line, the left and right faces are the same size, and there is no pitch.

In a preferred embodiment, the preset facial feature point model includes: a feature point of the face contour, a feature point of the eyebrow contour, a feature point of the nose contour, a feature point of the eye contour, and at least a feature point of the mouth contour. Two types of feature points.

The facial feature calculation module 102 is further configured to calculate the face type and the chin type according to the position information of the face contour feature point; calculate the eye shape according to the position information of the eye contour feature point; calculate the nose according to the feature point position information of the nose contour Aspect ratio; according to the feature point position information of the eyebrow contour and the feature point position information of the eye contour, the angle between the inner and outer eye line and the horizontal line, the eyebrow distance and the eye width are calculated.

The technical features of the above-described embodiments may be arbitrarily combined. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be considered as the scope of this manual.

The above described embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed. It is not to be construed as limiting the scope of the invention patent. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims

A method for retrieving a face image, comprising:

Determining feature point position information of the face image to be retrieved according to the preset face feature point model;

Calculating a facial feature of the face image to be retrieved according to the feature point position information;

Combining the facial features to obtain a feature vector of the face image to be retrieved;

The face image pre-stored in the database is retrieved according to the feature vector, and a face image with the highest similarity to the face image to be retrieved is obtained.
The method for retrieving a face image according to claim 1, wherein the step of calculating a facial feature of the face image to be retrieved according to the feature point position information further comprises:

Obtaining a semantic feature of the face image to be retrieved according to the facial feature;

The step of retrieving the face image pre-stored in the database according to the feature vector, and obtaining the face image with the highest similarity to the face image to be retrieved includes:

Performing a search in the database according to the semantic feature of the face image to be retrieved, and determining a search range corresponding to the semantic feature from a face image saved in advance in the database;

The feature vector is used to perform a search within the search range, and a face image with the highest similarity to the face image to be retrieved is obtained.
The method for retrieving a face image according to claim 2, wherein the step of obtaining a semantic feature of the face image to be retrieved according to the facial feature comprises:

And classifying the facial features according to a predetermined level standard to obtain a hierarchical level of the facial features;

Determining rules according to preset semantic features, hierarchical levels to obtain semantic features corresponding to the facial features;

Combining the semantic features corresponding to each facial feature, the semantic features of the face image to be retrieved are obtained.
The method for retrieving a face image according to claim 3, wherein the step of combining the facial features to obtain a feature vector of the face image to be retrieved comprises:

The number of hierarchical levels corresponding to each facial feature is combined to obtain a feature vector of the face image to be retrieved.
The method for retrieving a face image according to claim 2, wherein the step of performing a search within the search range by using the feature vector to obtain a face image having the highest similarity with the face image to be retrieved comprises:

Comparing the feature vector of the face image to be retrieved with the feature vector of each face image in the search range to obtain a vector difference value;

Finding a face image with the smallest difference between the vectors in the search range, and determining a face image having the highest similarity with the face image to be retrieved.
The method for retrieving a face image according to claim 1, wherein the step of determining feature point position information of the face image to be retrieved according to the preset face feature point model and calculating according to the feature point position information The steps of obtaining facial features of the face image to be retrieved include:

The face of the face image to be retrieved is corrected according to the feature point position information.
The method for retrieving a face image according to any one of claims 1 to 6, wherein the preset face feature point model comprises a feature point of a face contour, a feature point of an eyebrow contour, and a feature of a nose contour. At least two types of feature points of the point, the feature point of the eye contour, and the feature point of the mouth contour.
The method for retrieving a face image according to claim 7, wherein the preset face feature point model comprises a feature point of a face contour, a feature point of an eyebrow contour, a feature point of a nose contour, and an eye contour The feature point and the feature point of the mouth contour, and the step of calculating the facial feature of the face image to be retrieved according to the feature point position information includes:

Calculating the face parameter and the chin type parameter according to the feature point position information of the face contour;

Calculating the eyebrow shape parameter according to the feature point position information of the eyebrow contour;

Calculating the nose aspect ratio according to the feature point position information of the nose contour;

According to the feature point position information of the eyebrow contour and the feature point position information of the eye contour, the angle between the inner and outer eye line and the horizontal line, the eyebrow distance and the eye width are calculated.
A system for retrieving a face image, comprising: a feature point location information determining module, a facial feature computing module, a feature vector acquiring module, and a retrieval module;

The feature point location information determining module is configured to determine feature point location information of the face image to be retrieved according to the preset face feature point model;

The facial feature calculation module is configured to calculate a facial feature of the face image to be retrieved according to the feature point location information;

The feature vector obtaining module is configured to combine the facial features to obtain a feature vector of a face image to be retrieved;

The searching module is configured to retrieve a face image saved in advance in the database according to the feature vector, and obtain a face image with the highest similarity to the face image to be retrieved.
The system for retrieving a face image according to claim 9, further comprising: a semantic feature acquisition module;

The semantic feature acquiring module is configured to obtain a semantic feature of the face image to be retrieved according to the facial feature;

The search module is further configured to perform a search in the entire database according to the semantic features of the face image to be retrieved, determine a search range from a face image saved in advance in the database; and use the feature vector in the search range according to the feature vector The search is performed internally to obtain a face image with the highest similarity to the face image to be retrieved.