CN110110126B - Method, device and server for inquiring face image of person - Google Patents
Method, device and server for inquiring face image of person Download PDFInfo
- Publication number
- CN110110126B CN110110126B CN201910357244.8A CN201910357244A CN110110126B CN 110110126 B CN110110126 B CN 110110126B CN 201910357244 A CN201910357244 A CN 201910357244A CN 110110126 B CN110110126 B CN 110110126B
- Authority
- CN
- China
- Prior art keywords
- image
- face image
- extraction model
- feature extraction
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Image Analysis (AREA)
Abstract
The disclosure relates to a method, a device and a server for inquiring facial images of people, belonging to the technical field of face recognition, wherein the method comprises the steps of training a first image feature extraction model based on image reconstruction to obtain a second image feature extraction model; respectively extracting feature information of each facial image in a database based on the second image feature extraction model, wherein the feature information comprises an identity feature value and a state feature value; when an image query request carrying a target face image sent by a terminal is received, extracting feature information of the target face image based on the second image feature extraction model; and determining a face image meeting a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and sending the face image to the terminal. By adopting the method and the device, the fineness of inquiring the face image can be improved.
Description
Technical Field
The present disclosure relates to the field of face recognition technologies, and in particular, to a method, an apparatus, and a server for querying a face image of a person.
Background
The rapid development of internet technology provides great convenience for people's work and life, for example, users can perform image query work through the internet.
In one application scenario, when a user intends to query a facial image having a certain feature, information of the feature, such as melon seed face, double-eye eyelid, etc., may be input in a search engine, and a server may return a query result to a terminal based on the input feature information.
In the related art, the method for querying the face image of the person queries through the keyword, and the keyword can only roughly reflect certain characteristics of the face image, so that only rough query can be achieved, and fine query cannot be achieved.
Disclosure of Invention
The disclosure provides a method, a device and a server for inquiring a face image of a person, which can solve the problem that the method for inquiring the face image of the person in the related art can only realize rough inquiry and cannot realize fine inquiry.
According to a first aspect of embodiments of the present disclosure, there is provided a method of querying a face image of a person, the method including:
training the first image feature extraction model based on image reconstruction to obtain a second image feature extraction model;
respectively extracting feature information of each facial image in a database based on the second image feature extraction model, wherein the feature information comprises an identity feature value and a state feature value;
when an image query request carrying a target face image sent by a terminal is received, extracting feature information of the target face image based on the second image feature extraction model;
and determining a face image meeting a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and sending the face image to the terminal.
Optionally, the training the first image feature extraction model based on image reconstruction to obtain a second image feature extraction model includes:
the method comprises the steps of obtaining sample face images one by one, extracting feature information of the sample face images on the basis of a first image feature extraction model every time one sample face image is obtained, inputting the feature information into a pre-trained image reconstruction model to obtain a reconstructed face image of the sample face images, determining image reconstruction loss on the basis of the sample face images and the reconstructed face images, carrying out parameter training and updating on the first image feature extraction model on the basis of the image reconstruction loss, and determining a first image feature extraction model which is trained at present as a second image feature extraction model until a cycle ending condition is met.
Optionally, the extracting feature information of the sample face image based on the first image feature extraction model includes: extracting feature information and a prediction classification identifier of the sample facial image based on a first image feature extraction model;
the method further comprises the following steps: determining a classification loss based on the prediction classification identifier and the obtained actual classification identifier of the sample facial image;
based on the image reconstruction loss, performing parameter training and updating on the first image feature extraction model, wherein the parameter training and updating comprises the following steps: and performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss.
Optionally, performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss, including:
based on the formula L ═ a × L1+b×L2Determining the total loss, wherein L is the total loss, L1To classify the loss, L2The image reconstruction loss is represented by a weight coefficient of classification loss and b is represented by a weight coefficient of image reconstruction loss;
and performing parameter training and updating on the first image feature extraction model based on the total loss.
Optionally, the method further includes:
acquiring a plurality of pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image;
training an initial image feature extraction model based on the plurality of pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image to obtain a first image feature extraction model.
Optionally, the determining, based on the feature information of the target face image and the feature information of each face image in the database, a face image that satisfies a preset similarity condition with the target face image, and sending the determined face image to the terminal includes:
determining similarity of the target face image and each face image in the database respectively based on the feature information of the target face image and the feature information of each face image in the database;
and sending the face image corresponding to the similarity greater than the similarity threshold value to the terminal.
Optionally, the determining, based on the feature information of the target face image and the feature information of each face image in the database, a face image that satisfies a preset similarity condition with the target face image, and sending the determined face image to the terminal includes:
determining similarity of the target face image and each face image in the database respectively based on the feature information of the target face image and the feature information of each face image in the database;
and sending the face image corresponding to the maximum similarity to the terminal.
According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for querying a face image of a person, including:
the training module is configured to perform image reconstruction-based training on the first image feature extraction model to obtain a second image feature extraction model;
a first extraction module configured to perform extraction of feature information of each face image in a database based on the second image feature extraction model, wherein the feature information includes an identity feature value and a status feature value;
the second extraction module is configured to extract the feature information of the target face image based on the second image feature extraction model when an image query request carrying the target face image sent by a receiving terminal is executed;
and the determining module is configured to determine a face image meeting a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and transmit the face image to the terminal.
Optionally, the training module is specifically configured to perform:
the method comprises the steps of obtaining sample face images one by one, extracting feature information of the sample face images on the basis of a first image feature extraction model every time one sample face image is obtained, inputting the feature information into a pre-trained image reconstruction model to obtain a reconstructed face image of the sample face images, determining image reconstruction loss on the basis of the sample face images and the reconstructed face images, carrying out parameter training and updating on the first image feature extraction model on the basis of the image reconstruction loss, and determining a first image feature extraction model which is trained at present as a second image feature extraction model until a cycle ending condition is met.
Optionally, the training module is specifically configured to perform: extracting feature information and a prediction classification identifier of the sample facial image based on a first image feature extraction model;
the training module further configured to perform: determining a classification loss based on the prediction classification identifier and the obtained actual classification identifier of the sample facial image;
the training module is specifically configured to perform: and performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss.
Optionally, the training module is specifically configured to perform:
based on the formula L ═ a × L1+b×L2Determining the total loss, wherein L is the total loss, L1To classify the loss, L2The image reconstruction loss is represented by a weight coefficient of classification loss and b is represented by a weight coefficient of image reconstruction loss;
and performing parameter training and updating on the first image feature extraction model based on the total loss.
Optionally, the apparatus further comprises:
an acquisition module configured to perform acquisition of a plurality of pairs of sample face images, a reference similarity of each pair of sample face images, and an actual classification identification of each sample face image;
and the initial training module is configured to train an initial image feature extraction model based on the plurality of pairs of sample facial images, the reference similarity of each pair of sample facial images and the actual classification identification of each sample facial image, so as to obtain a first image feature extraction model.
Optionally, the determining module includes:
a determination unit configured to perform determination of similarities of the target face image with the respective face images in the database, respectively, based on the feature information of the target face image and the feature information of the respective face images in the database;
a transmitting unit configured to perform transmitting a face image corresponding to a similarity greater than a similarity threshold to the terminal.
Optionally, the determining module includes:
a determination unit configured to perform determination of similarities of the target face image with the respective face images in the database, respectively, based on the feature information of the target face image and the feature information of the respective face images in the database;
a transmitting unit configured to perform transmitting the face image corresponding to the maximum similarity to the terminal.
According to a third aspect of the embodiments of the present disclosure, there is provided a server, including:
one or more processors;
one or more memories for storing the one or more processor-executable instructions;
wherein the one or more processors are configured to perform the above-described method of querying a face image of a person.
According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, wherein instructions of the storage medium, when executed by a processor of a server, enable the server to perform the above-described method of querying a face image of a person.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising one or more instructions executable by a processor of a server to perform the method steps of the first aspect described above.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
in the embodiment of the present disclosure, when the server queries a face image similar to a person corresponding to the target face image, feature information of the target face image may be extracted based on the second image feature extraction model; then, a face image satisfying a preset similarity condition with the target face image is determined based on the feature information of the target face image and the feature information of each face image in the database. When the method is used for inquiring the face image of a person, the target face image is used as the base point of inquiry, and compared with the key words in the related technology, the target face image can reflect the characteristics to be inquired more finely, so that the fineness of inquiring the face image can be obviously improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow diagram illustrating a method of querying a face image of a person, according to an example embodiment;
FIG. 2 is a flow diagram illustrating a method of querying a face image of a person, according to an example embodiment;
FIG. 3 is a flow diagram illustrating a method of querying a face image of a person, according to an example embodiment;
FIG. 4 is a flow diagram illustrating a method of training a first image feature extraction model in accordance with an exemplary embodiment;
FIG. 5 is a flow diagram illustrating a method of training a first image feature extraction model in accordance with an exemplary embodiment;
FIG. 6 is a block diagram illustrating an apparatus for querying a facial image of a person, according to an example embodiment;
FIG. 7 is a block diagram illustrating an apparatus for querying a facial image of a person, according to an example embodiment;
FIG. 8 is a block diagram illustrating an apparatus for querying a facial image of a person, according to an example embodiment;
fig. 9 is a block diagram illustrating an apparatus for querying a face image of a person according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
Fig. 1 is a flowchart illustrating a method of querying a face image of a person, which may be used in a server, as shown in fig. 1, according to an exemplary embodiment, including the following steps.
In step 101, the server trains the first image feature extraction model based on image reconstruction to obtain a second image feature extraction model.
The first image feature extraction model and the second image feature extraction model are both models that are trained by technicians in advance through machine learning and are used for extracting feature information of facial images, wherein the first image feature extraction model may be a model trained on the basis of an initial image feature extraction model, the second image feature extraction model may be a model trained on the basis of the first image feature extraction model, and specific training processes of the first image feature extraction model and the second image feature extraction model will be described in detail below.
In step 102, the server extracts feature information of each face image in the database based on the second image feature extraction model, respectively.
The feature information may include an identity feature value and a state feature value, and specifically, the feature information may be a feature vector, for example, a 1024-dimensional feature vector, and the plurality of elements in the feature vector may include the identity feature value and the state feature value.
The identity characteristic value may be a specific value of a facial attribute capable of representing the identity of a person, the identity characteristic is a facial growth phase characteristic of the person, and may be a value of a characteristic capable of representing the identity of the person on a face such as an eye shape, a width of two eyes, a mouth shape, an eyebrow shape and the like, for example, the eye shape length is 50 mm, the eye shape length is an identity characteristic, and 50 mm is an eye shape length which is an identity characteristic. The state feature value may be a specific value of a face state, and the face state may include an expression condition, a posture condition, and a lighting condition of a face image of the corresponding person, for example, a smile face image, where the smile is a state feature value of a face.
In implementation, the server may input each facial image in the database into the second image feature extraction model in advance, sequentially obtain and store feature information of each facial image, and when the feature information of each facial image is required by a subsequent server, the server may obtain the feature information from the database. The feature information extracted by the second image feature extraction model not only includes facial growth features of a person, but also includes current expression state information of the person, for example, the current expression state information of the person in the facial image is deep, smiling, laughing, crying or the like, and the information can be represented by the feature information.
In step 103, when receiving an image query request carrying a target face image sent by a terminal, the server extracts feature information of the target face image based on a second image feature extraction model.
In implementation, an application for querying a face image of a person may be installed on a terminal of a user, and after the user opens the application and logs in, a target face image may be uploaded on a query interface, where the target face image may be a face image of the user himself or a face image of another person. In this way, the terminal can send an image query request carrying the target face image to the server, and when the server receives the image query request sent by the terminal, the server can extract the feature information of the target face image based on the second image feature extraction model. The image query request may also carry an account identifier of the terminal, so as to return a query result to the terminal.
In step 104, the server determines a face image satisfying a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and transmits the face image to the terminal.
The feature information may be represented by using feature vectors, and accordingly, the similarity between the target face image and each face image may be determined by a cosine distance between two feature vectors, or may be calculated by an euclidean distance between two feature vectors, and the like.
In implementation, after the server extracts the feature information of the target face image through the second image feature extraction model, firstly, the similarity between the feature information of the target face image and the feature information of each face image can be determined, then, a face image meeting a preset similarity condition with the target face image can be determined from all the similarities, and finally, the determined face image is sent to the terminal.
In one possible application, the server may send one or more queried face images to the terminal, e.g., the server may send face images corresponding to a similarity greater than a similarity threshold to the terminal, e.g., the server may send face images corresponding to a similarity greater than 85% to the terminal. For another example, the server may transmit, to the terminal, a face image corresponding to a similarity before a preset name among all the similarities, and for example, may transmit, to the terminal, a face image corresponding to a similarity before the top ten names. Alternatively, for example, the server may transmit the face image corresponding to the maximum similarity to the terminal.
In this way, the terminal can display the face image after receiving the face image which is transmitted by the server and has the similarity with the face image of the target person and meets the preset similarity condition.
In one possible application, after receiving the image query request carrying the target face image, as shown in fig. 2, the server may input the target face image into the second image feature extraction model to obtain feature information of the target face image, then obtain the feature information of each face image from the stored database of the feature information of each face image, and then input the feature information of the target face image and the feature vectors of each face image into the similarity calculation module pair by pair to obtain the similarity between the target face image and each face image.
Based on the above, compared with the related art in which the features of the target face image are described in text and then the face image similar to the target face image is searched for by using the key words, the method for searching for the face image similar to the target face image according to the target face image uses the target face image as the query base point, and the target face image can reflect the features to be queried more finely than the key words in the related art, so that the fineness of the query face image can be obviously improved.
In addition, by querying the face image of the person using the above method, not only a face image similar in growth of the person corresponding to the target face image can be found, but also a face image similar in expression state to the person in the target face image can be queried. For example, the target face image is an image of a person 1 who is laughing, and a face image that is similar in size to the person 1 and is also laughing can be searched using the above method. That is, with this method, it is possible to query a face image whose person corresponding to the target face image is similar in growth and whose expression state is also similar.
The above part is a process of querying a face image of a person using this method, and specific training processes of the second image feature extraction model and the first image feature extraction model will be described below:
the process of training the first image feature extraction model to obtain the second image feature extraction model may be as follows:
as described above, the second image feature extraction model may be a model trained on the first image feature extraction model, and specifically, the server may acquire sample face images one by one, each sample face image is acquired, and for each sample face image, the following operations may be performed according to the flow shown in fig. 3, and the flow diagram shown in fig. 4 may also be referred to (fig. 3 and 4 are each a flow performed for each sample face image).
Wherein the following steps are a loop training process, and are performed for each sample face image in each loop as follows.
In step 1011, the server extracts feature information of the sample face image based on the first image feature extraction model.
In the implementation of the first image feature extraction model, after the server acquires a sample face image, the server may input the sample face image into the first image feature extraction model trained in advance to obtain feature information of the sample face image.
In step 1012, the server inputs the feature information extracted via the first image feature extraction model into a pre-trained image reconstruction model, resulting in a reconstructed face image of the sample face image.
In implementation, after obtaining the feature information of the sample face image through the first image feature extraction model, the server may reconstruct the sample face image according to the feature information, for example, the obtained feature information is input into a pre-trained image reconstruction model, so as to obtain a reconstructed face image of the sample face image.
In step 1013, the server determines an image reconstruction loss based on the sample face image and its reconstructed face image.
Wherein, the sample face image refers to the face image input to the first image feature extraction model, and may also be referred to as an original face image; the sample face image is input to the feature information obtained by the first image feature extraction model, and the face image obtained by inputting the sample face image to the image reconstruction model is called a reconstructed face image of the sample face image.
In an implementation, the server may determine an image reconstruction loss between the sample face image and its reconstructed face image, for example, a difference between pixel values of corresponding positions between the sample face image and its reconstructed face image may be taken as the image reconstruction loss therebetween. The smaller the image reconstruction loss is, the more similar the reconstructed face image is to the original sample face image, and correspondingly, the higher the precision of the first image feature extraction model is, the higher the accuracy of the extracted feature information is.
In step 1014, the server performs parameter training and updating on the first image feature extraction model based on the image reconstruction loss until a cycle end condition is satisfied, and then determines the currently trained first image feature extraction model as a second image feature extraction model.
The condition that the loop end is satisfied may be that when the image reconstruction loss satisfies a preset condition, for example, the image reconstruction loss is smaller than a preset value; the condition that the loop ending is satisfied may also be that the number of loops satisfies a condition, for example, when the server detects that the number of loops of parameter training update of the first image feature extraction model reaches a preset value.
In implementation, after the server determines the image reconstruction loss between the sample face image and the reconstructed face image thereof, the first image feature extraction model may be subjected to parameter adjustment based on the image reconstruction loss, and training optimization updating is performed. When the first image feature extraction model is optimized to a certain degree, namely when the circulation end condition is met, the server determines the currently trained first image feature extraction model as a second image feature extraction model, so that the accuracy of the second image feature extraction model in extracting feature information can be improved, and the accuracy of inquiring the face image of a person can be improved.
Optionally, the server may perform optimization adjustment on the first image feature extraction model based on the classification loss during the parameter training and updating of the first image feature extraction model, and as shown in fig. 5, in step 1011, the server may extract feature information of the sample facial image based on the first image feature extraction model, and may also extract a prediction classification identifier of the sample facial image based on the first image feature extraction model. Accordingly, prior to step 1014, the method may further comprise the server determining a classification loss based on the predicted classification identifier and the actual classification identifier of the acquired sample facial image. After the server determines the image reconstruction loss and the classification loss of the sample face image, the parameter training and updating can be carried out on the first image feature extraction model based on the image reconstruction loss and the classification loss.
The classification identifier is an identifier of a person corresponding to the sample face image, that is, an identifier of a certain person, and is an identifier for distinguishing different persons. Each face image is uniquely corresponding to a classification mark, a plurality of sample face images can correspond to the same classification mark, but one classification mark can only correspond to one person, and the plurality of sample face images belong to the same person. For example, the number of sample face images is 50 thousands, and each person has 50 face images corresponding to each person, and accordingly, there are 1 ten thousands of classification labels in these sample face images, that is, there are as many classification labels as how many persons the sample face images relate to.
The predicted classification identifier is a classification identifier of a person corresponding to the face image predicted by the first image feature extraction model, the actual classification identifier is a real actual classification identifier of a person corresponding to a certain face image, and the actual classification identifiers of the predicted classification identifiers may be the same or different.
The classification loss is the difference between the actual classification identifier and the predicted classification identifier of the same sample face image. If the actual classification identifier is the same as the predicted classification identifier, the classification loss is 0, and if the actual classification identifier is not the same as the predicted classification identifier, the classification loss is 1.
In an implementation, the server may be based on the formula L ═ a × L1+b×L2Determining the total loss; the server may then perform a parameter training update on the first image feature extraction model based on the total loss. Wherein L is the total loss, L1To classify the loss, L2For image reconstruction loss, a is the weight coefficient of classification loss, and b is the weight of image reconstruction lossAnd (4) weight coefficient. The values of a and b can be set by a technician according to actual requirements, for example, a can be 1, b can be 0.1, and the like.
Based on the above, referring to fig. 5 again, each time the server obtains a sample face image, the server performs feature extraction using the first image feature extraction model that has been updated last time to obtain feature information and preset classification identifiers. Secondly, the server inputs the feature information extracted by the first image feature extraction model into an image reconstruction model to obtain a reconstructed face image, and calculates the image reconstruction loss between the sample face image and the reconstructed face image; the server also determines a classification loss based on the predicted classification identifier and the acquired actual classification identifier of the sample facial image. And then, the server determines the total loss of the sample face image based on the image reconstruction loss and the classification loss, and performs parameter optimization on the first image feature extraction model based on the total loss of the sample face image. In this way, the server performs the above operation once every time the server acquires a sample face image until the end condition of the cycle is satisfied, and then determines the currently trained first image feature extraction model as the second image feature extraction model.
In this way, the server may obtain the second image feature extraction model based on the image reconstruction and the first image feature extraction model.
The above is the process of training the first image feature extraction model to obtain the second image feature extraction model, and the following describes the process of training the initial image feature extraction model to obtain the first image feature extraction model:
firstly, a server acquires a plurality of pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image; then, the server trains the initial image feature extraction model based on the multiple pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image to obtain a first image feature extraction model.
The reference similarity of each pair of sample facial images may be a score of similarity degree of the pair of sample facial images by a technician according to identity features (i.e., facial growth features) and state features (i.e., facial expression states) of two persons in the pair of sample facial images.
In an implementation, after the server obtains a plurality of sample face images, each sample face image may be first cropped and aligned to obtain a fixed resolution face image, for example, unify the sample face images into a 128 × 128 resolution image. The technician may also add an actual classification label to each sample facial image based on the persons in each sample facial image. Then, the technician can combine the plurality of sample face images two by two to obtain a plurality of pairs of sample face images, and for each pair of sample face images, the technician scores the similarity of the pair of sample face images according to the facial growth and expression states of two people in the pair of sample face images, and gives a reference similarity to the two sample face images. And finally, training the initial image feature extraction model by a technician according to the plurality of pairs of sample facial images, the reference similarity of each pair of sample facial images and the actual classification identification of each sample facial image to obtain a first image feature extraction model.
In this way, as shown in fig. 5, when the trained first image feature extraction model is applied, a facial image is input to the first image feature extraction model, so that feature information and a prediction classification flag corresponding to the facial image can be obtained.
Optionally, in a possible application, after the user queries the face image according to the method, the user may further obtain identification information of a person corresponding to the face image, and further obtain, according to the identification information, a face image of a person that is not the same as the person corresponding to the target face image, and correspondingly, the server may determine, based on the feature information and the identification information of the target face image and the feature information and the identification information of each face image in the database, that the face image that satisfies the preset similarity condition with the target face image and the identification information of which is not matched, and send the determined face image to the terminal.
The identification information may be information related to a person corresponding to the face image, information capable of representing the person or information capable of contacting the person, and may include communication information, location information, and terminal model information.
The communication information can be account information of a social network where the face image is located, names of people corresponding to the face image, communication mode information and the like; the position information may include position information at the time of uploading of the face image, and may also include position information at the time of photographing of the face image; the terminal model information may be model information of a terminal used to capture the face image.
In implementation, after the server determines at least one face image satisfying a preset similarity condition with the target face image, first, the identification information of the person corresponding to each of the face images and the identification information of the target face image are obtained, then, face images not matching the identification information of the target face image are screened out from all face images satisfying the preset similarity condition with the target face image, and the screened face images and the corresponding identification information are sent to the terminal.
The face image that does not match the identification information of the target face image may be a face image that does not match all of the information contained in the identification information, or a face image that does not match part of the information contained in the identification information, and the specific situation may be set by a technician or selected by the user.
For example, the user may set the query level in the query process, and if the query level is higher (e.g., a high-level query), all of the information included in the identification information may not match, and if the query level is lower (e.g., a simple query), some of the information included in the identification information may not match.
In an implementation, the identification information may include communication information, location information, and terminal model information, and after the server determines a face image satisfying a preset similarity condition with the target face image, for example, the server determines five face images, which are respectively denoted as face image 1, face image 2, face image 3, face image 4, and face image 5. Then, the server acquires the identification information of the face image and the identification information of the target face image, for example, the identification information of the target face image is marked as identification information 0, and the identification information of the face image is sequentially marked as identification information 1, identification information 2, identification information 3, identification information 4 and identification information 5.
If the identity recognition information 0 is different from the information in the identity recognition information 1 and the identity recognition information 2 respectively; the identification information 0 is respectively the same as the information parts in the identification information 3 and the identification information 4; the identification information 0 is identical to the identification information 5 (the probability that the target face image and the face image 5 belong to the same person is relatively high).
Under the condition that the query level is relatively high, namely under the condition that the restriction condition is relatively strict, the facial image determined by the server is the facial image 1 corresponding to the identification information 1 and the facial image 2 corresponding to the identification information 2, and then the server can send the facial image 1, the identification information 1, the facial image 2 and the identification information 2 to the terminal.
In the case that the query level is relatively low, that is, the restriction condition is relatively loose, the server determines that the face image is the face image 1 corresponding to the identification information 1, the face image 2 corresponding to the identification information 2, the face image 3 corresponding to the identification information 3, and the face image 4 corresponding to the identification information 4, and then the server may send the face image 1 and the identification information 1, the face image 2 and the identification information 2, the face image 3 and the identification information 3, and the face image 4 and the identification information 4 to the terminal.
In this way, after the terminal receives the face image which is sent by the server and has the similarity with the face image of the target person meeting the preset similarity condition and the identity identification information of the target person is not matched, the face image can be displayed. The user can acquire the information of the people corresponding to the face image according to the identity identification information of the face image, and then can get contact with the people, so that the entertainment of an application program used for inquiring the face image on the terminal can be further improved, and the retention rate of the application program can be further improved.
The server filters the face images with the similarity meeting the preset similarity condition but with unmatched identity identification information, so that the face images meeting the similarity condition and belonging to the same person as the target face images can be filtered, further, the face images inquired in the way do not belong to the same person basically with the target face images, and then, the user can use the method to search other persons similar to the target face images, and further the interestingness of entertainment can be increased.
Based on the above, in a possible application environment, when a user wants to query a person having a length similar to that of the user, the user may open an application program installed on the terminal for querying a face image of the person, log in the application program, upload the face image of the user, and further send an image query request carrying the face image of the user to the server. When the server receives an image query request sent by the terminal, the server can determine face images meeting the conditions based on the method and send the face images and corresponding identification information to the terminal. After the terminal receives the image, the image can be displayed, and then the user can obtain a face image which is similar to the face image grown by the user. The user can further acquire the communication mode of the corresponding person based on the identification information corresponding to the facial images so as to contact other people and expand the social circle of the user.
In the embodiment of the present disclosure, when the server queries a face image similar to a person corresponding to the target face image, feature information of the target face image may be extracted based on the second image feature extraction model; then, the server determines a face image satisfying a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database. When the method is used for inquiring the face image of a person, the target face image is used as the base point of inquiry, and compared with the key words in the related technology, the target face image can reflect the characteristics to be inquired more finely, so that the fineness of inquiring the face image can be obviously improved.
Fig. 6 is a block diagram illustrating an apparatus for querying a face image of a person according to an example embodiment. Referring to fig. 6, the apparatus includes a training module 610, a first extraction module 620, a second extraction module 630, and a determination module 640.
The training module 610 is configured to perform training of a first image feature extraction model based on image reconstruction, resulting in a second image feature extraction model;
the first extraction module 620 is configured to perform extraction of feature information of each face image in a database based on the second image feature extraction model, wherein the feature information includes an identity feature value and a status feature value;
the second extraction module 630 is configured to extract feature information of a target facial image based on the second image feature extraction model when receiving an image query request carrying the target facial image sent by a terminal;
the determining module 640 is configured to determine a face image satisfying a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and transmit the face image to the terminal.
Optionally, the training module 610 is specifically configured to perform:
the method comprises the steps of obtaining sample face images one by one, extracting feature information of the sample face images on the basis of a first image feature extraction model every time one sample face image is obtained, inputting the feature information into a pre-trained image reconstruction model to obtain a reconstructed face image of the sample face images, determining image reconstruction loss on the basis of the sample face images and the reconstructed face images, carrying out parameter training and updating on the first image feature extraction model on the basis of the image reconstruction loss, and determining a first image feature extraction model which is trained at present as a second image feature extraction model until a cycle ending condition is met.
Optionally, the training module 610 is specifically configured to perform: extracting feature information and a prediction classification identifier of the sample facial image based on a first image feature extraction model;
the training module 610 is further configured to perform: determining a classification loss based on the prediction classification identifier and the obtained actual classification identifier of the sample facial image;
the training module 610 is specifically configured to perform: and performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss.
Optionally, the training module 610 is specifically configured to perform:
based on the formula L ═ a × L1+b×L2Determining the total loss, wherein L is the total loss, L1To classify the loss, L2The image reconstruction loss is represented by a weight coefficient of classification loss and b is represented by a weight coefficient of image reconstruction loss;
and performing parameter training and updating on the first image feature extraction model based on the total loss.
Optionally, as shown in fig. 7, the apparatus further includes an obtaining module 608 and an initial training module 609, where:
the obtaining module 608 configured to perform obtaining a plurality of pairs of sample face images, a reference similarity of each pair of sample face images, and an actual classification identification of each sample face image;
the initial training module 609 is configured to perform training on an initial image feature extraction model based on the plurality of pairs of sample face images, the reference similarity of each pair of sample face images, and the actual classification identifier of each sample face image, so as to obtain a first image feature extraction model.
Optionally, as shown in fig. 8, the determining module 640 includes:
a determination unit 641 configured to perform determination of the similarity of the target face image with each of the face images in the database, respectively, based on the feature information of the target face image and the feature information of each of the face images in the database;
a transmitting unit 642 configured to perform transmitting a face image corresponding to a similarity greater than a similarity threshold to the terminal.
Optionally, referring to fig. 8 again, the determining module 640 includes:
a determination unit 641 configured to perform determination of the similarity of the target face image with each of the face images in the database, respectively, based on the feature information of the target face image and the feature information of each of the face images in the database;
a transmitting unit 642 configured to perform transmitting the face image corresponding to the maximum similarity to the terminal.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 9 is a schematic structural diagram of a computer device 900 according to an embodiment of the present disclosure, where the computer device 900 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 901 and one or more memories 902, where the memory 902 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 901 to implement the following method steps for determining user characteristic data:
training the first image feature extraction model based on image reconstruction to obtain a second image feature extraction model;
respectively extracting feature information of each facial image in a database based on the second image feature extraction model, wherein the feature information comprises an identity feature value and a state feature value;
when an image query request carrying a target face image sent by a terminal is received, extracting feature information of the target face image based on the second image feature extraction model;
and determining a face image meeting a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and sending the face image to the terminal.
There is also provided, in accordance with an embodiment of the present disclosure, a non-transitory computer-readable storage medium, wherein instructions of the storage medium, when executed by a processor of a server, enable the server to perform the above-described method of querying a face image of a person.
According to an embodiment of the present disclosure, there is provided a computer program product comprising one or more instructions executable by a processor of a server to perform the method steps described above.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (16)
1. A method of querying a face image of a person, the method comprising:
training the first image feature extraction model based on image reconstruction to obtain a second image feature extraction model;
respectively extracting feature information of each facial image in a database based on the second image feature extraction model, wherein the feature information comprises an identity feature value and a state feature value;
when an image query request carrying a target face image sent by a terminal is received, extracting feature information of the target face image based on the second image feature extraction model;
determining at least one face image meeting a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, and acquiring the identity identification information of a person corresponding to each face image in the at least one face image and the identity identification information of the person corresponding to the target face image, wherein the identity identification information refers to information capable of being connected with the person; and screening out a face image which is not matched with the identification information of the target face image from the at least one face image, and sending the screened face image and the corresponding identification information to the terminal.
2. The method of claim 1, wherein training the first image feature extraction model based on image reconstruction to obtain the second image feature extraction model comprises:
the method comprises the steps of obtaining sample face images one by one, extracting feature information of the sample face images on the basis of a first image feature extraction model every time one sample face image is obtained, inputting the feature information into a pre-trained image reconstruction model to obtain a reconstructed face image of the sample face images, determining image reconstruction loss on the basis of the sample face images and the reconstructed face images, carrying out parameter training and updating on the first image feature extraction model on the basis of the image reconstruction loss, and determining a first image feature extraction model which is trained at present as a second image feature extraction model until a cycle ending condition is met.
3. The method of claim 2, wherein extracting feature information of the sample facial image based on the first image feature extraction model comprises: extracting feature information and a prediction classification identifier of the sample facial image based on a first image feature extraction model;
the method further comprises the following steps: determining a classification loss based on the prediction classification identifier and the obtained actual classification identifier of the sample facial image;
based on the image reconstruction loss, performing parameter training and updating on the first image feature extraction model, wherein the parameter training and updating comprises the following steps: and performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss.
4. The method of claim 3, wherein performing a parametric training update on the first image feature extraction model based on the image reconstruction penalty and the classification penalty comprises:
based on the formula L ═ a × L1+b×L2Determining the total loss, wherein L is the total loss, L1To classify the loss, L2The image reconstruction loss is represented by a weight coefficient of classification loss and b is represented by a weight coefficient of image reconstruction loss;
and performing parameter training and updating on the first image feature extraction model based on the total loss.
5. The method of claim 3, further comprising:
acquiring a plurality of pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image;
training an initial image feature extraction model based on the plurality of pairs of sample face images, the reference similarity of each pair of sample face images and the actual classification identification of each sample face image to obtain a first image feature extraction model.
6. The method according to any one of claims 1-5, further comprising:
determining similarity of the target face image and each face image in the database respectively based on the feature information of the target face image and the feature information of each face image in the database;
and sending the face image corresponding to the similarity greater than the similarity threshold value to the terminal.
7. The method according to any one of claims 1-5, further comprising:
determining similarity of the target face image and each face image in the database respectively based on the feature information of the target face image and the feature information of each face image in the database;
and sending the face image corresponding to the maximum similarity to the terminal.
8. An apparatus for querying a face image of a person, comprising:
the training module is configured to perform image reconstruction-based training on the first image feature extraction model to obtain a second image feature extraction model;
a first extraction module configured to perform extraction of feature information of each face image in a database based on the second image feature extraction model, wherein the feature information includes an identity feature value and a status feature value;
the second extraction module is configured to extract the feature information of the target face image based on the second image feature extraction model when an image query request carrying the target face image sent by a receiving terminal is executed;
a determination module configured to perform, after determining at least one face image satisfying a preset similarity condition with the target face image based on the feature information of the target face image and the feature information of each face image in the database, acquiring identification information of a person corresponding to each face image in the face images satisfying the preset similarity condition and identification information of a person corresponding to the target face image, wherein the identification information represents information that can be linked to the person corresponding to the face image; and screening out the face images which are not matched with the identity identification information of the target face image from all the face images meeting the preset similarity condition, and sending the screened face images and the corresponding identity identification information to the terminal.
9. The apparatus of claim 8, wherein the training module is specifically configured to perform:
the method comprises the steps of obtaining sample face images one by one, extracting feature information of the sample face images on the basis of a first image feature extraction model every time one sample face image is obtained, inputting the feature information into a pre-trained image reconstruction model to obtain a reconstructed face image of the sample face images, determining image reconstruction loss on the basis of the sample face images and the reconstructed face images, carrying out parameter training and updating on the first image feature extraction model on the basis of the image reconstruction loss, and determining a first image feature extraction model which is trained at present as a second image feature extraction model until a cycle ending condition is met.
10. The apparatus of claim 9, wherein the training module is specifically configured to perform: extracting feature information and a prediction classification identifier of the sample facial image based on a first image feature extraction model;
the training module further configured to perform: determining a classification loss based on the prediction classification identifier and the obtained actual classification identifier of the sample facial image;
the training module is specifically configured to perform: and performing parameter training and updating on the first image feature extraction model based on the image reconstruction loss and the classification loss.
11. The apparatus of claim 10, wherein the training module is specifically configured to perform:
based on the formula L ═ a × L1+b×L2Determining the total loss, wherein L is the total loss, L1To classify the loss, L2The image reconstruction loss is represented by a weight coefficient of classification loss and b is represented by a weight coefficient of image reconstruction loss;
and performing parameter training and updating on the first image feature extraction model based on the total loss.
12. The apparatus of claim 10, further comprising:
an acquisition module configured to perform acquisition of a plurality of pairs of sample face images, a reference similarity of each pair of sample face images, and an actual classification identification of each sample face image;
and the initial training module is configured to train an initial image feature extraction model based on the plurality of pairs of sample facial images, the reference similarity of each pair of sample facial images and the actual classification identification of each sample facial image, so as to obtain a first image feature extraction model.
13. The apparatus according to any one of claims 8-12, wherein the determining module comprises:
a determination unit configured to perform determination of similarities of the target face image with the respective face images in the database, respectively, based on the feature information of the target face image and the feature information of the respective face images in the database;
a transmitting unit configured to perform transmitting a face image corresponding to a similarity greater than a similarity threshold to the terminal.
14. The apparatus according to any one of claims 8-12, wherein the determining module comprises:
a determination unit configured to perform determination of similarities of the target face image with the respective face images in the database, respectively, based on the feature information of the target face image and the feature information of the respective face images in the database;
a transmitting unit configured to perform transmitting the face image corresponding to the maximum similarity to the terminal.
15. A server, comprising:
one or more processors;
one or more memories for storing the one or more processor-executable instructions;
wherein the one or more processors are configured to perform the method of querying the facial image of a person of any one of claims 1-7.
16. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a server, enable the server to perform the method of querying a face image of a person of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910357244.8A CN110110126B (en) | 2019-04-29 | 2019-04-29 | Method, device and server for inquiring face image of person |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910357244.8A CN110110126B (en) | 2019-04-29 | 2019-04-29 | Method, device and server for inquiring face image of person |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110110126A CN110110126A (en) | 2019-08-09 |
| CN110110126B true CN110110126B (en) | 2021-08-27 |
Family
ID=67487593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910357244.8A Active CN110110126B (en) | 2019-04-29 | 2019-04-29 | Method, device and server for inquiring face image of person |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110110126B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112329797A (en) * | 2020-11-13 | 2021-02-05 | 杭州海康威视数字技术股份有限公司 | Target object retrieval method, device, server and storage medium |
| CN114360008B (en) * | 2021-12-23 | 2023-06-20 | 上海清鹤科技股份有限公司 | Face authentication model generation method, authentication method, equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824059B (en) * | 2014-02-28 | 2017-02-15 | 东南大学 | Facial expression recognition method based on video image sequence |
| US11593632B2 (en) * | 2016-12-15 | 2023-02-28 | WaveOne Inc. | Deep learning based on image encoding and decoding |
| CN107180248A (en) * | 2017-06-12 | 2017-09-19 | 桂林电子科技大学 | Strengthen the hyperspectral image classification method of network based on associated losses |
| CN108537743B (en) * | 2018-03-13 | 2022-05-20 | 杭州电子科技大学 | Face image enhancement method based on generation countermeasure network |
| CN108875818B (en) * | 2018-06-06 | 2020-08-18 | 西安交通大学 | Zero sample image classification method based on combination of variational self-coding machine and antagonistic network |
-
2019
- 2019-04-29 CN CN201910357244.8A patent/CN110110126B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110110126A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9514356B2 (en) | Method and apparatus for generating facial feature verification model | |
| CN108288051B (en) | Pedestrian re-recognition model training method and device, electronic equipment and storage medium | |
| JP5857073B2 (en) | System and method for relevance of image texting and text imaging | |
| CN109213882A (en) | Picture sort method and terminal | |
| CN101305368A (en) | Semantic visual search engine | |
| CN106557728B (en) | Query image processing and image search method and device and monitoring system | |
| CN110110126B (en) | Method, device and server for inquiring face image of person | |
| CN111339812A (en) | Pedestrian identification and re-identification method based on whole or partial human body structural feature set, electronic equipment and storage medium | |
| CN111742342A (en) | Image generation method, image generation device, and image generation system | |
| KR20210033940A (en) | How to Train Neural Networks for Human Facial Recognition | |
| US11263436B1 (en) | Systems and methods for matching facial images to reference images | |
| CN110598097B (en) | Hair style recommendation system, method, equipment and storage medium based on CNN | |
| CN106485220A (en) | Face identification method, the intelligent glasses with face identification functions and server | |
| CN111401193B (en) | Method and device for acquiring expression recognition model, and expression recognition method and device | |
| CN109670423A (en) | A kind of image identification system based on deep learning, method and medium | |
| CN110737885A (en) | Method and device for authenticating identity of livestock | |
| CN110427870B (en) | Eye picture recognition method, target recognition model training method and device | |
| JP4340860B2 (en) | Face matching system | |
| CN110675312A (en) | Image data processing method, image data processing device, computer equipment and storage medium | |
| CN112364946B (en) | Training method of image determination model, and method, device and equipment for image determination | |
| US11783587B2 (en) | Deep learning tattoo match system based | |
| CN115578765A (en) | Target identification method, device, system and computer readable storage medium | |
| CN111738157B (en) | Face action unit data set construction method and device and computer equipment | |
| KR101720685B1 (en) | Apparatus and Method for Web Data based Identification System for Object Identification Performance Enhancement | |
| CN111444374B (en) | Human body retrieval system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |