CN112149601A - Occlusion-compatible face attribute identification method and device and electronic equipment - Google Patents

Occlusion-compatible face attribute identification method and device and electronic equipment Download PDF

Info

Publication number
CN112149601A
CN112149601A CN202011059698.6A CN202011059698A CN112149601A CN 112149601 A CN112149601 A CN 112149601A CN 202011059698 A CN202011059698 A CN 202011059698A CN 112149601 A CN112149601 A CN 112149601A
Authority
CN
China
Prior art keywords
image
face
attribute
target
facial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011059698.6A
Other languages
Chinese (zh)
Inventor
柳天驰
程禹
申省梅
马原
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Pengsi Technology Co ltd
Original Assignee
Beijing Pengsi Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Pengsi Technology Co ltd filed Critical Beijing Pengsi Technology Co ltd
Priority to CN202011059698.6A priority Critical patent/CN112149601A/en
Publication of CN112149601A publication Critical patent/CN112149601A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • G06V40/171Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification

Abstract

The embodiment of the application provides a method and a device for identifying facial attributes compatible with occlusion and electronic equipment, wherein the method comprises the following steps: acquiring a face image shot by a target user; determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user. The method can improve the accuracy and robustness of facial attribute identification.

Description

Occlusion-compatible face attribute identification method and device and electronic equipment
Technical Field
The application relates to the technical field of data processing, in particular to a method and a device for identifying facial attributes compatible with occlusion and electronic equipment.
Background
The technology for recognizing the face attribute according to the face image is widely concerned, the accuracy of face attribute recognition is seriously influenced under the condition that the face is shielded, the inherent structure and geometric features of the face are damaged by shielding, and the richness of face feature information is seriously reduced.
When the model is used for identifying the attributes of the face, the model is directly used for identifying the face image, namely, the feature extraction and the attribute classification are directly carried out, so that error information is introduced, and the accuracy of model identification is low.
Disclosure of Invention
In view of this, an object of the embodiments of the present application is to provide a method, an apparatus and an electronic device for identifying facial attributes compatible with occlusion, which reduce the influence of occlusion on facial attribute identification and improve the accuracy and robustness of facial attribute identification when identifying an occluded facial image and an unoccluded facial image through one model.
In a first aspect, an embodiment of the present application provides an occlusion-compatible facial attribute identification method, where the method includes:
acquiring a face image shot by a target user;
determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user;
and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
In one embodiment, inputting the global target image or the partial target image to a pre-trained face property recognition model comprises:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
In one embodiment, determining a partial target image from the facial image comprises:
and setting the pixel value of the pixel point of the shielding area in the face image as a preset value.
In one embodiment, after acquiring a face image captured for a target user, the method further includes:
detecting facial keypoint information from the facial image;
calibrating the facial image based on the keypoint information.
In one embodiment, the face attribute recognition model is obtained by pre-training through the following steps:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the face attribute recognition training model acquires a plurality of image pairs in a training sample library, wherein one image pair comprises a global target image and a partial target image, the global target image comprises a complete face image of a reference object, and the partial target image comprises a partial face image of the reference object;
carrying out feature extraction on the global target image to obtain a first feature map, and carrying out feature extraction on part of the target image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
In one embodiment, adjusting the model parameters comprises:
and adjusting model parameters of a face attribute recognition training model which is not trained completely according to the principle that a first loss value between the first prediction attribute and the actual attribute of the first sample image and a second loss value between the second prediction attribute and the actual attribute of the second sample image are simultaneously minimum.
In a second aspect, an embodiment of the present application provides an occlusion-compatible facial attribute recognition apparatus, including:
the acquisition module is used for acquiring a face image shot by a target user;
the processing module is used for determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
In one embodiment, the processing module is configured to input the global target image or the partial target image to a pre-trained face property recognition model according to the following steps:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
In an embodiment, the processing module is further configured to set a pixel value of a pixel point of a blocking area in the face image to a preset value.
In one embodiment, the processing module is further configured to, after acquiring a face image captured for a target user, detect face key point information from the face image, and calibrate the face image based on the key point information.
In one embodiment, the apparatus further comprises a facial attribute recognition model pre-training module, which is used for pre-training the facial attribute recognition model by the following steps:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the method comprises the steps that a face attribute recognition training model obtains a plurality of image pairs in a training sample library, wherein one image pair comprises a first sample image and a second sample image, the first sample image comprises a complete face image of a reference object, and the second sample image comprises a partial face image of the reference object;
performing feature extraction on the first sample image to obtain a first feature map, and performing feature extraction on the second sample image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
In an embodiment, the facial attribute recognition model pre-training module is further configured to, when adjusting model parameters, adjust model parameters of a facial attribute recognition training model that is not trained according to a principle that the first loss value and the second loss value are simultaneously minimum.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a storage medium storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions to perform the steps of the method of the first aspect as described above when the electronic device is operated.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, performs the steps of the method of the first aspect.
According to the occlusion-compatible face attribute identification method provided by the embodiment of the application, a face image obtained by shooting for a target user is obtained, and a global target image or a partial target image is determined from the face image according to the occlusion condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; the method comprises the steps of inputting a global target image or a part of target images into a pre-trained facial attribute recognition model to obtain an attribute recognition result of a target user, inputting the global target image or the part of target images of the same target user into a facial attribute recognition training model to be trained in a correlation mode, and training to obtain a final model for recognizing facial attributes, so that the influence of a shielding area on facial attribute recognition is avoided, and on the premise of not reducing the model recognition accuracy, the purpose that one model recognizes the shielded facial images and the non-shielded facial images is achieved.
Further, according to an embodiment, whether the face is shielded or not is judged, and the part of the face area which is not shielded is reserved for attribute recognition, so that targeted processing can be ensured according to the shielding condition of the face. For example, for a face with local shielding, such as a face with lower shielding, a more representative face upper non-shielding area is selected, the feature information of the face non-shielding part is fully utilized, the influence of a shielding object on the whole face attribute recognition is reduced, and the accuracy and the robustness of the face attribute recognition are improved. The facial attribute identification method is compatible with the shielded/non-shielded face scene, ensures the identification accuracy of the non-shielded face scene, can be well adapted to the scene with the shielded face, and expands the application range and the scene of the face attribute identification algorithm.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a first flowchart illustrating a method for identifying an occlusion-compatible facial attribute according to an embodiment of the present application;
FIG. 2 is a second flowchart illustrating a method for identifying an occlusion-compatible facial attribute according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of an occlusion-compatible facial property recognition apparatus according to an embodiment of the present application;
fig. 4 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
When facial attributes such as facial attributes (such as gender, age, expression and the like) are recognized, images with shielding and images without shielding are generally recognized through different models, for shielding images, when a model for recognizing shielding images is trained, a local non-shielding face is generally used as a sample to train the model, so that the robustness of the trained model is poor, and the recognition accuracy of the model is low.
Based on this, the application proposes a method for identifying an attribute of an occluded face, the method comprising: acquiring a face image shot by a target user; determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user. A complete face image or a face image with occlusions is identified by a model. The face attribute identification model is obtained by training at least one training sample, each training sample consists of a global target image and a partial target image of the same reference object, the global target image is an image of the reference object which is not shielded, the partial target image and the global target image are the same in part, different parts are in a shielding state, and namely the partial target image is a partial shielding image of the global target image. Therefore, the global target image and the partial target image of the same reference object are input into the facial attribute recognition model to be trained in a correlated mode, the model of the final recognition attribute is obtained through training, the influence of a shielding area on the facial attribute recognition is avoided, and on the premise that the model recognition accuracy is not reduced, the recognition of the shielded facial image and the non-shielded facial image by one model is achieved. The embodiments of the present application will be described in detail based on this idea.
In the embodiment of the present application, the facial attribute recognition model is obtained by pre-training through the following steps:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the method comprises the steps that a face attribute recognition training model obtains a plurality of image pairs in a training sample library, wherein one image pair comprises a first sample image and a second sample image, the first sample image comprises a complete face image of a reference object, and the second sample image comprises a partial face image of the reference object;
performing feature extraction on the first sample image to obtain a first feature map, and performing feature extraction on the second sample image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
Here, the predicted attribute and the actual attribute are both face attributes, and the face attribute includes at least one of an attribute under a gender category, an attribute under an age category, or an attribute under an expression category; the first feature map can be a feature value matrix, and feature values in the matrix represent face attribute feature information included in a corresponding region in the first sample image; the second feature map may also be a feature matrix, and feature values in the matrix represent face attribute feature information included in corresponding regions in the second sample image, where the size of the region in the first sample image and the size of the region in the second sample image may be determined according to actual conditions; the feature extraction model may be a combination of layers of convolutional neural network, activation function (e.g., ReLU, Tanh, Sigmoid, etc.), Batch Normalization, etc.; the first prediction attribute and the second prediction attribute are both attributes obtained through prediction, and the attribute category of the prediction attribute is the same as that of the actual attribute.
In a specific implementation process, when a training sample library is obtained, a large number of sample images can be obtained, and actual attributes of reference objects included in the sample images are labeled manually.
In consideration of the fact that when a sample image is shot, a camera is not necessarily shooting a face of a reference object, therefore, the reference object in the sample image needs to be calibrated, so that the calibrated sample image is a standard face image of the reference object (the standard face image is obtained by shooting the face of the reference object just by the camera), and the calibrated sample image is used as a sample image in a training sample library.
When the sample image is calibrated, the face key point information, that is, the face key points (for example, eyes, nose, mouth, etc.), of the image of the reference object included in the sample image may be detected, the position information of the face key points in the sample image may be acquired, and the reference object in the sample image may be face-aligned using the acquired position information, that is, the face of the reference object in the sample image is calibrated to the standard face, so that the positions of the reference object in different sample images may be ensured to be the same.
After the first sample image is obtained, a partial region in the first sample image may be masked, the region to be masked is a fixed region or a non-fixed region, the position of the fixed region in the image is fixed for the fixed region, and the size of the fixed region is preset and may be determined according to the proportion of the masked portion of the object in the face image.
When a partial region in the first sample image is shielded, the pixel value of the pixel point of the partial region can be set as a preset value, the partial region can also be shielded by using a blank image, the remaining unshielded images are reserved, and the image after the first sample image is shielded is used as a second sample image.
After the first sample image and the second sample image are obtained, the first sample image and the second sample image of the same reference object may be associated and input to a feature extraction model in the unfinished facial attribute recognition model, that is, the first sample image and the second sample image are input to the feature extraction model in the unfinished facial attribute recognition model as an image pair, and the feature extraction model extracts feature information from the first sample image and feature information from the second sample image respectively to obtain a first feature map corresponding to the first sample image and a second feature map corresponding to the second sample image. The first feature map comprises more face attribute information than the second feature map.
And respectively associating and inputting the first feature map and the second feature map to an attribute classification model in the unfinished training face attribute recognition model, wherein the attribute classification model outputs a first prediction attribute corresponding to the first sample image and a second prediction attribute corresponding to the second sample image.
Determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition.
And under the condition that the training is determined to be not finished, adjusting the model parameters and carrying out the next iterative training. When the model parameters of the unfinished facial attribute recognition model are adjusted, a first loss value between the first prediction attribute and the actual attribute of the first sample image and a second loss value between the second prediction attribute and the actual attribute of the second sample image can be calculated, and the model parameters of the unfinished facial attribute recognition model are adjusted according to the principle that the first loss value and the second loss value are simultaneously minimum. Wherein the first loss value may be a distance or a similarity between the first prediction attribute and an actual attribute of the first sample image, the second loss value may be a distance or a similarity between the second prediction attribute and an actual attribute of the second sample image, and the algorithm for calculating the distance and the similarity is a conventional algorithm in the art, wherein the updating of the parameters in the model may be accomplished by using a conventional gradient descent algorithm, and for the L1 loss function, all of which are not described in detail herein.
In order to improve the recognition accuracy of the facial attribute recognition model, when adjusting the model parameters, the similarity between the first feature map and the second feature map may be further considered, so that the second feature map is infinitely close to the first feature map, that is, the feature information extracted from the partial face image is close to the feature information extracted from the global face image, that is, a third loss value between the first feature map and the second feature map is calculated, and the model parameters of the facial attribute recognition model are adjusted according to the principle that the first loss value, the second loss value and the third loss value are simultaneously minimum, so as to obtain the trained facial attribute recognition model. Wherein the third loss value may be a distance or a similarity between the first feature map and the second feature map.
And obtaining the face attribute recognition model under the condition that the training is determined to be completed.
After the face attribute identification model is obtained, face attribute identification can be performed using the face attribute identification model. In view of this situation, an embodiment of the present application provides an occlusion-compatible facial attribute identification method, as shown in fig. 1, the method includes the following steps:
s101, a face image obtained by shooting aiming at a target user is obtained.
Here, the facial image is collected by an image collecting device, the image collecting device may be a camera, a photographing device, and the like, the image collecting device may be a device in an access control system, for example, an access control system set by an enterprise, an access control system set by a residential community, or an image collecting device set indoors or outdoors, for example, a mall entrance, an office building entrance, a public transportation security check, and the like, and may be determined according to actual situations, but it should be understood that the present application is not limited thereto.
S102, determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user;
here, the captured face image may have an occlusion, and in the case of an occlusion, for example, when the target user wears a mask, the mask wearing area is an occlusion area in the face image, and the face image is regarded as a partial target image, and in the case of no occlusion, the face image in the case of no occlusion is regarded as a global target image.
Here, if it is determined that the target user in the face image is occluded, determining a partial target image from the face image, and inputting the partial target image to a face attribute recognition model; the part of the target image is the same as the part of the face image, and different parts are in an occlusion state;
and if the target user in the face image is determined not to be blocked, inputting the face image into a pre-trained face attribute recognition model.
Here, the algorithm for detecting whether the facial image has occlusion includes an image classification method based on deep learning, and the like, and the above method may be a conventional method in the art, and the present application does not limit the method.
The size of the partial target image may be the same as that of the face image, and the partial target image is obtained by performing a blocking process on the face image, that is, blocking a fixed area in the face image, which will be described in detail below.
When it is determined that the target user in the face image is occluded, determining a partial target image from the face image according to the following steps:
before determining the partial target image from the face image, face key point information may be detected from the face image, where the key point information is face key point information, the face key points include eyes, a nose, a mouth, and the like, and the face key point information is position information of the face key points in the face image.
The method includes the steps that a target user in a face image is calibrated by using face key point information, namely, when the face of the target user in the face image is not a standard face, the face of the target user in the face image needs to be calibrated to the standard face to obtain a calibrated face image, and the calibrated face image meets the input requirement of a model on the size of the image. In this way, the model identification is facilitated.
Because the shielding area in the shielding image which can be identified by the trained facial attribute identification model is fixed, after the calibrated facial image is obtained, the pixel value of the pixel point in the image of the designated area is set as the preset value, so that part of the target image is obtained. The position of the designated area in the face image is preset, for example, the designated area may be an area below the nose or an area where the eyes are located, and the shape of the designated area may be a quadrangle (e.g., a rectangle), which may be determined according to actual situations.
S103, inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
Here, the attribute identification result includes at least one of the attribute identification result in the gender category, the attribute identification result in the expression category, and the attribute identification result in the age category, and for example, when the attribute identification result is the attribute identification result in the gender category, the result may be a gender male or a female, when the attribute identification result is the attribute identification result in the age category, the result may be an age 15, an age 27, or the like, and when the attribute identification result is the attribute identification result in the expression category, the result may be an expression happy state, an expression sadness, or the like.
The global target image and the partial target image are both face images, the size of the global target image is the same as that of the partial target image, and the difference between the global target image and the partial target image is that the pixel value of the pixel point of the partial image in the shielding state in the partial target image is a preset value, the preset value can be set according to the actual situation, for example, the preset value is 0, 1, 20, and the like, and when the preset value is 0, the shielded part in the global target image is represented as black.
As an optional embodiment, the area corresponding to the part of the target image in the shielding state is a fixed area, for example, when the target user wears a mask, the part of the target image in the shielding state is an area shielded by the mask, and for example, when the target user wears sunglasses, the part of the target image in the shielding state is an area shielded by the sunglasses; the area where the part of the target image in the face image is in the shielding state is determined to be the shielded area in the face image which can be recognized by the face attribute recognition model, namely, the shielded area in the face image of the target user is the same as the shielded area in the sample image, and for the images, the recognition accuracy of the face attribute recognition model is higher.
After obtaining the partial target image, inputting the partial target image into the facial attribute recognition model for feature extraction to obtain a feature map corresponding to the partial target image, and inputting the obtained feature map into the attribute classification model of the facial attribute recognition model to obtain an attribute recognition result of the target user.
For example, the target user is a woman, the face image is an image of a mask, and after the face image is acquired, the pixel value of a mask covering area in the face image is set to 0 to obtain a partial target image, and the partial target image is input to the face attribute identification model to obtain that the attribute identification result of the target user is a woman and a mask.
When the target user in the facial image is not blocked, the facial image may be input to the feature extraction model in the facial attribute recognition model, global feature information is extracted, and the feature extraction result is input to the attribute classification model of the facial attribute classification model, so as to obtain an attribute recognition result of the target user, for example, the attribute recognition result is female, no mask is worn, or the like.
In an embodiment, referring to fig. 2, after obtaining a face image to be recognized, detecting face key point information, calibrating/aligning the face image based on the face key point information, determining whether an occlusion exists in the calibrated/aligned face image, if so, retaining a non-occlusion region of a face in the face image to obtain a partial target image, inputting the partial target image to a face attribute recognition model to obtain an attribute recognition result, and if not, inputting the entire face image as a global target image to the face attribute recognition model to obtain an attribute recognition result.
Through the facial attribute recognition model, the facial image with the face being shielded can be recognized, and the facial image with the face not being shielded can also be recognized.
Referring to fig. 3, a schematic diagram of an occlusion-compatible facial attribute recognition apparatus provided in an embodiment of the present application is shown, where the apparatus includes:
an acquisition module 31 configured to acquire a face image captured by a target user;
the processing module 32 is used for determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
In one embodiment, the processing module 32 is configured to input the global target image or the partial target image to a pre-trained face property recognition model according to the following steps:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
In one embodiment, the processing module 32 is further configured to:
detecting facial keypoint information from the facial image;
calibrating the facial image based on the keypoint information.
The processing module 32 is configured to determine a target image from the facial image according to the following steps:
and in the calibrated face image, setting the pixel values of the pixel points in the image of the designated area as preset values.
In one embodiment, the apparatus further includes a training module 33 (i.e., a facial attribute recognition model pre-training module) for pre-training a facial attribute recognition model by:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the method comprises the steps that a face attribute recognition training model obtains a plurality of image pairs in a training sample library, wherein one image pair comprises a first sample image and a second sample image, the first sample image comprises a complete face image of a reference object, and the second sample image comprises a local face image of the reference object;
performing feature extraction on the first sample image to obtain a first feature map, and performing feature extraction on the second sample image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
In one embodiment, the training module 33 adjusts the model parameters of the untrained face attribute recognition training model according to a simultaneous minimum rule of the first loss value and the second loss value when adjusting the model parameters.
In an embodiment, in the second sample image, a pixel value of a pixel point of the partial image in the shielding state is a preset value.
An embodiment of the present application further provides an electronic device 40, as shown in fig. 4, which is a schematic structural diagram of the electronic device 40 provided in the embodiment of the present application, and includes:
a processor 41, a memory 42, and a bus 43. The memory 42 stores machine-readable instructions executable by the processor 41 (for example, execution instructions corresponding to the acquisition module 31 and the processing module 32 in the apparatus in fig. 3, and the like), when the electronic device 40 runs, the processor 41 communicates with the memory 42 through the bus 43, and when the processor 41 executes the following processing:
acquiring a face image shot by a target user;
determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user;
and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
In one possible embodiment, the instructions executed by processor 41 to input the global target image or the partial target image into a pre-trained face property recognition model include:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
In one possible embodiment, the instructions executed by processor 41,
determining a partial target image from the facial image, including:
and setting the pixel value of the pixel point of the shielding area in the face image as a preset value.
In one possible embodiment, the instructions executed by the processor 41 further include, after acquiring a face image captured for a target user:
detecting facial keypoint information from the facial image;
calibrating the facial image based on the keypoint information.
In one possible implementation, processor 41 executes instructions that train the facial attribute recognition model according to the following steps:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the method comprises the steps that a face attribute recognition training model obtains a plurality of image pairs in a training sample library, wherein one image pair comprises a first sample image and a second sample image, the first sample image comprises a complete face image of a reference object, and the second sample image comprises a partial face image of the reference object;
performing feature extraction on the first sample image to obtain a first feature map, and performing feature extraction on the second sample image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
In one possible embodiment, the instructions executed by processor 41 adjust the type parameter, including:
and adjusting model parameters of the face attribute recognition training model which is not trained completely according to the principle that the first loss value and the second loss value are simultaneously minimum.
As is known to those skilled in the art, as computer hardware evolves, the specific implementation and nomenclature of the bus may change, and the bus as referred to herein conceptually encompasses any information transfer line capable of servicing components within an electronic device, including, but not limited to, FSB, HT, QPI, Infinity Fabric, etc.
In the embodiment of the present application, the processor may be a general-purpose processor including a Central Processing Unit (CPU), and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a neural Network Processor (NPU), a Tensor Processor (TPU), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the occlusion-compatible face property identification method are performed.
Specifically, the storage medium can be a general storage medium, such as a mobile magnetic disk, a hard disk, or the like, and when a computer program on the storage medium is executed, the occlusion-compatible face attribute identification method can be executed; the global target image comprises a complete facial image of a target user, the partial target image comprises a partial facial image which is not shielded by the target user, the global target image or the partial target image is input into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user, therefore, the global target image and the partial target image of the same reference object are input into a facial attribute recognition model to be trained in a correlation mode, a model of a final recognition attribute is obtained through training, the influence of a shielding area on attribute recognition is avoided, and on the premise that the model recognition accuracy is not reduced, the recognition of the shielded facial image and the non-shielded facial image by one model is realized. The method can improve the accuracy and robustness of facial attribute identification.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing an electronic device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. An occlusion-compatible facial attribute recognition method, comprising:
acquiring a face image shot by a target user;
determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user;
and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
2. The method of claim 1, wherein inputting the global target image or partial target image to a pre-trained facial property recognition model comprises:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
3. The method of claim 2, wherein determining a partial target image from the facial image comprises:
and setting the pixel value of the pixel point of the shielding area in the face image as a preset value.
4. The method of claim 1, wherein after acquiring the image of the face captured for the target user, further comprising:
detecting facial keypoint information from the facial image;
calibrating the facial image based on the keypoint information.
5. The method of claim 1, wherein the facial attribute recognition model is pre-trained by a method comprising:
the face attribute recognition training model carries out a plurality of times of iterative training according to a training sample library, and one time of iterative training comprises the following steps:
the method comprises the steps that a face attribute recognition training model obtains a plurality of image pairs in a training sample library, wherein one image pair comprises a first sample image and a second sample image, the first sample image comprises a complete face image of a reference object, and the second sample image comprises a partial face image of the reference object;
performing feature extraction on the first sample image to obtain a first feature map, and performing feature extraction on the second sample image to obtain a second feature map;
predicting according to the first characteristic diagram to obtain a first prediction attribute, and predicting according to the second characteristic diagram to obtain a second prediction attribute;
determining a first loss value between the first prediction attribute and the actual attribute of the first sample image, determining a second loss value between the second prediction attribute and the actual attribute of the second sample image, and determining whether the training of the face attribute recognition training model is finished according to the first loss value, the second loss value and a preset convergence condition;
under the condition that the training is determined to be not finished, adjusting model parameters, and carrying out the next iterative training;
and obtaining the face attribute recognition model under the condition that the training is determined to be completed.
6. The method of claim 5, wherein adjusting model parameters comprises:
and adjusting model parameters of the face attribute recognition training model which is not trained completely according to the principle that the first loss value and the second loss value are simultaneously minimum.
7. An occlusion-compatible facial attribute recognition apparatus, comprising:
the acquisition module is used for acquiring a face image shot by a target user;
the processing module is used for determining a global target image or a partial target image from the face image according to the shielding condition of the face image; the global target image comprises a complete face image of a target user, and the partial target image comprises a partial face image which is not blocked by the target user; and inputting the global target image or the partial target image into a pre-trained facial attribute recognition model to obtain an attribute recognition result of the target user.
8. The apparatus of claim 7, wherein the processing module is to input the global target image or partial target image to a pre-trained facial property recognition model according to:
judging whether the face in the face image is shielded or not;
if the occlusion is determined to exist, determining to obtain a part of target images from the face images, and inputting the part of target images into a pre-trained face attribute recognition model;
and if the fact that the occlusion does not exist is determined, determining the face image as a global target image, and inputting the global target image into a pre-trained face attribute recognition model.
9. An electronic device, comprising: a processor and a storage medium storing machine-readable instructions executable by the processor to perform the steps of the method of any one of claims 1 to 6 when the electronic device is operated.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 6.
CN202011059698.6A 2020-09-30 2020-09-30 Occlusion-compatible face attribute identification method and device and electronic equipment Pending CN112149601A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011059698.6A CN112149601A (en) 2020-09-30 2020-09-30 Occlusion-compatible face attribute identification method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011059698.6A CN112149601A (en) 2020-09-30 2020-09-30 Occlusion-compatible face attribute identification method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN112149601A true CN112149601A (en) 2020-12-29

Family

ID=73951244

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011059698.6A Pending CN112149601A (en) 2020-09-30 2020-09-30 Occlusion-compatible face attribute identification method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN112149601A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112418190A (en) * 2021-01-21 2021-02-26 成都点泽智能科技有限公司 Mobile terminal medical protective shielding face recognition method, device, system and server
CN113469216A (en) * 2021-05-31 2021-10-01 浙江中烟工业有限责任公司 Retail terminal poster identification and integrity judgment method, system and storage medium
CN113610106A (en) * 2021-07-01 2021-11-05 北京大学 Feature compatible learning method and device between models, electronic equipment and medium

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160110586A1 (en) * 2014-10-15 2016-04-21 Nec Corporation Image recognition apparatus, image recognition method and computer-readable medium
EP3379458A2 (en) * 2017-03-23 2018-09-26 Samsung Electronics Co., Ltd. Facial verification method and apparatus
CN109034078A (en) * 2018-08-01 2018-12-18 腾讯科技(深圳)有限公司 Training method, age recognition methods and the relevant device of age identification model
CN109271884A (en) * 2018-08-29 2019-01-25 厦门理工学院 Face character recognition methods, device, terminal device and storage medium
CN109784255A (en) * 2019-01-07 2019-05-21 深圳市商汤科技有限公司 Neural network training method and device and recognition methods and device
CN109886167A (en) * 2019-02-01 2019-06-14 中国科学院信息工程研究所 One kind blocking face identification method and device
US20190205616A1 (en) * 2017-12-29 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for detecting face occlusion
CN110110681A (en) * 2019-05-14 2019-08-09 哈尔滨理工大学 It is a kind of for there is the face identification method blocked
CN110119723A (en) * 2019-05-17 2019-08-13 北京深醒科技有限公司 One kind carries out blocking facial expression recognizing method based on ACNN
CN111291604A (en) * 2018-12-07 2020-06-16 深圳光启空间技术有限公司 Face attribute identification method, device, storage medium and processor
CN111310624A (en) * 2020-02-05 2020-06-19 腾讯科技(深圳)有限公司 Occlusion recognition method and device, computer equipment and storage medium
CN111339930A (en) * 2020-02-25 2020-06-26 四川翼飞视科技有限公司 Face recognition method combining mask attribute loss function
CN111339812A (en) * 2019-06-29 2020-06-26 北京澎思科技有限公司 Pedestrian identification and re-identification method based on whole or partial human body structural feature set, electronic equipment and storage medium
CN111401222A (en) * 2020-03-12 2020-07-10 河南威虎智能科技有限公司 Feature learning method for combined multi-attribute information of shielded face
CN111523431A (en) * 2020-04-16 2020-08-11 支付宝(杭州)信息技术有限公司 Face recognition method, device and equipment
CN111582141A (en) * 2020-04-30 2020-08-25 京东方科技集团股份有限公司 Face recognition model training method, face recognition method and device
CN111666925A (en) * 2020-07-02 2020-09-15 北京爱笔科技有限公司 Training method and device for face recognition model

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160110586A1 (en) * 2014-10-15 2016-04-21 Nec Corporation Image recognition apparatus, image recognition method and computer-readable medium
EP3379458A2 (en) * 2017-03-23 2018-09-26 Samsung Electronics Co., Ltd. Facial verification method and apparatus
US20190205616A1 (en) * 2017-12-29 2019-07-04 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for detecting face occlusion
CN109034078A (en) * 2018-08-01 2018-12-18 腾讯科技(深圳)有限公司 Training method, age recognition methods and the relevant device of age identification model
CN109271884A (en) * 2018-08-29 2019-01-25 厦门理工学院 Face character recognition methods, device, terminal device and storage medium
CN111291604A (en) * 2018-12-07 2020-06-16 深圳光启空间技术有限公司 Face attribute identification method, device, storage medium and processor
CN109784255A (en) * 2019-01-07 2019-05-21 深圳市商汤科技有限公司 Neural network training method and device and recognition methods and device
CN109886167A (en) * 2019-02-01 2019-06-14 中国科学院信息工程研究所 One kind blocking face identification method and device
CN110110681A (en) * 2019-05-14 2019-08-09 哈尔滨理工大学 It is a kind of for there is the face identification method blocked
CN110119723A (en) * 2019-05-17 2019-08-13 北京深醒科技有限公司 One kind carries out blocking facial expression recognizing method based on ACNN
CN111339812A (en) * 2019-06-29 2020-06-26 北京澎思科技有限公司 Pedestrian identification and re-identification method based on whole or partial human body structural feature set, electronic equipment and storage medium
CN111310624A (en) * 2020-02-05 2020-06-19 腾讯科技(深圳)有限公司 Occlusion recognition method and device, computer equipment and storage medium
CN111339930A (en) * 2020-02-25 2020-06-26 四川翼飞视科技有限公司 Face recognition method combining mask attribute loss function
CN111401222A (en) * 2020-03-12 2020-07-10 河南威虎智能科技有限公司 Feature learning method for combined multi-attribute information of shielded face
CN111523431A (en) * 2020-04-16 2020-08-11 支付宝(杭州)信息技术有限公司 Face recognition method, device and equipment
CN111582141A (en) * 2020-04-30 2020-08-25 京东方科技集团股份有限公司 Face recognition model training method, face recognition method and device
CN111666925A (en) * 2020-07-02 2020-09-15 北京爱笔科技有限公司 Training method and device for face recognition model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
WU, G等: "Occluded Face Recognition Based on the Deep Learning", 《PROCEEDINGS OF THE 2019 31ST CHINESE CONTROL AND DECISION CONFERENCE (CCDC 2019)》, 31 December 2019 (2019-12-31), pages 793 - 797 *
王浩: "基于局部特征聚类损失和多类特征融合的面部表情识别", 《模式识别与人工智能》, no. 03, 31 March 2019 (2019-03-31), pages 268 - 276 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112418190A (en) * 2021-01-21 2021-02-26 成都点泽智能科技有限公司 Mobile terminal medical protective shielding face recognition method, device, system and server
CN112418190B (en) * 2021-01-21 2021-04-02 成都点泽智能科技有限公司 Mobile terminal medical protective shielding face recognition method, device, system and server
CN113469216A (en) * 2021-05-31 2021-10-01 浙江中烟工业有限责任公司 Retail terminal poster identification and integrity judgment method, system and storage medium
CN113469216B (en) * 2021-05-31 2024-02-23 浙江中烟工业有限责任公司 Retail terminal poster identification and integrity judgment method, system and storage medium
CN113610106A (en) * 2021-07-01 2021-11-05 北京大学 Feature compatible learning method and device between models, electronic equipment and medium
CN113610106B (en) * 2021-07-01 2023-10-24 北京大学 Feature compatible learning method and device between models, electronic equipment and medium

Similar Documents

Publication Publication Date Title
CN109815845B (en) Face recognition method and device and storage medium
WO2021077984A1 (en) Object recognition method and apparatus, electronic device, and readable storage medium
US10762376B2 (en) Method and apparatus for detecting text
US11487995B2 (en) Method and apparatus for determining image quality
CN111428581B (en) Face shielding detection method and system
CN110363047B (en) Face recognition method and device, electronic equipment and storage medium
CN112149601A (en) Occlusion-compatible face attribute identification method and device and electronic equipment
US20180157899A1 (en) Method and apparatus detecting a target
WO2018170864A1 (en) Face recognition and tracking method
CN107832700A (en) A kind of face identification method and system
CN111461165A (en) Image recognition method, recognition model training method, related device and equipment
US20230033052A1 (en) Method, apparatus, device, and storage medium for training image processing model
CN111444744A (en) Living body detection method, living body detection device, and storage medium
US11475537B2 (en) Method and apparatus with image normalization
CN111274916A (en) Face recognition method and face recognition device
CN106372624B (en) Face recognition method and system
CN111767900A (en) Face living body detection method and device, computer equipment and storage medium
US10007678B2 (en) Image processing apparatus, image processing method, and recording medium
CN111914748A (en) Face recognition method and device, electronic equipment and computer readable storage medium
CN112633221A (en) Face direction detection method and related device
CN113344000A (en) Certificate copying and recognizing method and device, computer equipment and storage medium
CN112560584A (en) Face detection method and device, storage medium and terminal
CN112200056A (en) Face living body detection method and device, electronic equipment and storage medium
JP5648452B2 (en) Image processing program and image processing apparatus
CN112949785B (en) Object detection method, device, equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination