CN111414879B - Face shielding degree identification method and device, electronic equipment and readable storage medium - Google Patents

Face shielding degree identification method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN111414879B
CN111414879B CN202010225939.3A CN202010225939A CN111414879B CN 111414879 B CN111414879 B CN 111414879B CN 202010225939 A CN202010225939 A CN 202010225939A CN 111414879 B CN111414879 B CN 111414879B
Authority
CN
China
Prior art keywords
face
image
shielding
area
faces
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010225939.3A
Other languages
Chinese (zh)
Other versions
CN111414879A (en
Inventor
王诗吟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Douyin Vision Co Ltd
Original Assignee
Douyin Vision Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Douyin Vision Co Ltd filed Critical Douyin Vision Co Ltd
Priority to CN202010225939.3A priority Critical patent/CN111414879B/en
Publication of CN111414879A publication Critical patent/CN111414879A/en
Application granted granted Critical
Publication of CN111414879B publication Critical patent/CN111414879B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The disclosure provides a face shielding degree identification method, a device, an electronic device and a readable storage medium, comprising: acquiring an image to be identified; inputting the image to be identified into a neural network model, and acquiring the face shielding degree of the image to be identified based on the output result of the trained neural network model; the neural network model is obtained through training of face shielding data of a preset number, and the face shielding data is obtained through the following modes: acquiring a preset number of original images; after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained. According to the scheme, the acquisition of the face shielding data is simpler and more convenient and efficient, and the neural network model for the face shielding degree recognition can be obtained through more efficient training, so that the neural network model is more beneficial to being applied to the face shielding degree recognition.

Description

Face shielding degree identification method and device, electronic equipment and readable storage medium
Technical Field
The disclosure relates to the technical field of computers, and in particular relates to a face shielding degree identification method, a face shielding degree identification device, electronic equipment and a readable storage medium.
Background
In the live video industry, a live video platform often needs a user to upload a head portrait picture, and the platform can identify the identity of the user through a face in the head portrait picture and the like. The head portrait pictures uploaded by the users often contain shielding objects such as caps, glasses, masks and hairs, the shielding objects can shield part of the areas of the faces of the users, and if the area of the faces shielded by the shielding objects in the head portrait pictures uploaded by the users is too large, the platform cannot identify the identity of the users, and then the platform can judge that the head portrait pictures of the users are unqualified head portrait pictures.
At present, many platforms use a neural network to identify the shielding degree of a human face in a head portrait picture uploaded by a user, and if the shielding degree is greater than a specific threshold value, the head portrait picture is judged to be unqualified. Before the neural network is utilized to identify the face shielding degree, massive face shielding data (namely images marked with the face shielding degree) are needed to be used as training samples for training the neural network, the existing face shielding data are mostly obtained through manual marking, manpower and material resources are consumed, the marking efficiency is low, and the neural network is not beneficial to identifying the face shielding degree.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In a first aspect, an embodiment of the present disclosure provides a method for identifying a face shielding degree, including:
acquiring an image to be identified;
inputting the image to be identified into a trained neural network model, and acquiring an identification result of the face shielding degree of the image to be identified based on an output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
In a second aspect, an embodiment of the present disclosure provides a face shielding degree recognition device, including:
the image acquisition module to be identified is used for acquiring the image to be identified;
the recognition result acquisition module is used for inputting the image to be recognized into the trained neural network model and acquiring a face shielding degree recognition result of the image to be recognized based on the output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
In a third aspect, embodiments of the present disclosure provide an electronic device comprising a memory and a processor;
a memory having a computer program stored therein;
a processor for executing a computer program to implement the method provided in the first aspect embodiment or any of the alternative embodiments of the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the method provided in the embodiment of the first aspect or any of the alternative embodiments of the first aspect.
The beneficial effects that this disclosure provided technical scheme brought are:
according to the scheme provided by the embodiment of the disclosure, the image to be recognized is input into the neural network model to obtain the probability that the image to be recognized corresponds to the shielding proportion of each face, the shielding degree of the face corresponding to the image to be recognized is obtained based on the probability of the shielding proportion of each face, before the neural network model is used, the neural network model is trained by using the preset number of face shielding data, in the training process, a large number of images containing the shielded faces are obtained by arranging and combining a plurality of original images containing the non-shielded faces with various shielding object images, and the faces in the original images are aligned before the images containing the shielded faces are obtained, so that the acquisition of the face shielding data is simpler and more efficient, the neural network model for recognizing the shielding degree of the face can be obtained by more efficiently completing the training of the neural network model, and the neural network model is more beneficial to be applied to the recognition of the shielding degree of the face.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings that are required to be used in the description of the embodiments of the present disclosure will be briefly introduced below.
Fig. 1 is a flowchart of a face shielding degree recognition method provided in an embodiment of the present disclosure;
fig. 2 is a block diagram of a face shielding degree recognition device according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present disclosure and are not to be construed as limiting the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
For the purposes of clarity, technical solutions and advantages of the present disclosure, the following further details the embodiments of the present disclosure with reference to the accompanying drawings.
In the live video industry, a live video platform often needs a user to upload a head portrait picture, and the platform can identify the identity of the user through a face in the head portrait picture and the like. The head portrait pictures uploaded by the users often contain shielding objects such as caps, glasses, masks and hairs, the shielding objects can shield part of the areas of the faces of the users, and if the area of the faces shielded by the shielding objects in the head portrait pictures uploaded by the users is too large, the platform cannot identify the identity of the users, and then the platform can judge that the head portrait pictures of the users are unqualified head portrait pictures.
At present, many platforms use a neural network to identify the shielding degree of a human face in a head portrait picture uploaded by a user, and if the shielding degree is greater than a specific threshold value, the head portrait picture is judged to be unqualified. Before the neural network is utilized to identify the shielding degree of the face, massive face shielding data (namely, images marked with the face shielding degree) are needed to be used as training samples to train the neural network, the existing face shielding data are mostly obtained through manual work to obtain images with shielded faces, then the face shielding proportion of the faces in the images is manually determined, and the face shielding proportion is marked in the images to obtain corresponding face shielding data. The generation process of the face shielding data needs a large number of images containing the shielded faces, but a large amount of manpower and material resources are consumed for obtaining a large number of images containing the shielded faces, angles of the faces in the images are different, face shielding proportion corresponding to the images is determined to be low in efficiency, marking efficiency is low, and the face shielding degree is not easy to identify by using a neural network. In view of the above problems, an embodiment of the present disclosure provides a face shielding degree recognition method.
Fig. 1 is a flowchart of a face shielding degree recognition method provided by an embodiment of the present disclosure, where, as shown in fig. 1, the method may include:
step S101, an image to be identified is acquired.
The image to be identified is an image containing a face, for example, the image to be identified may be a head portrait image uploaded to the video live broadcast platform by a user, and in the application scenario, the video live broadcast platform needs to identify the face shielding degree in the head portrait image.
It should be noted that the image to be identified may include a plurality of faces, only one of the faces may be identified in the subsequent identification process, one of the faces may be screened out from the plurality of faces as an identification object through a preset rule, and the identification result of the face may be used as the identification result of the final face shielding degree; the faces may be respectively used as recognition objects, and the final recognition result of the face shielding degree may be obtained according to the recognition result of the faces, for example, an average value of the recognition results of the faces may be taken as the final recognition result of the face shielding degree. Hereinafter, the scheme of the present disclosure will be described in the case where the image to be recognized includes only one face, and it is understood that the scope of the present disclosure is not limited thereto.
Step S102, inputting the image to be identified into a trained neural network model, and acquiring an identification result of the face shielding degree of the image to be identified based on the output result of the trained neural network model.
The face shielding degree is used for indicating the size of the area shielded by the face, and generally, the ratio of the area shielded by the face to the total area of the face can be expressed, the larger the ratio is, the larger the area shielded by the face is, namely, the higher the face shielding degree is, otherwise, the smaller the area shielded by the face is, namely, the lower the face shielding degree is.
The neural network model is a neural network model with a classification function, and the output result is the probability that the face shielding proportion in the image to be recognized is the shielding proportion of each preset face (or the shielding proportion range of each preset face), wherein the number of the shielding proportion of the preset face (or the shielding proportion range of the preset face) (corresponding to the number of neurons of the output layer of the neural network model) can be set according to the actual requirement. The trained neural network model is a neural network model determined by the trained network parameters, and the trained neural network model can be directly used for identifying the face shielding degree of the image to be identified.
Specifically, after the probability that the image to be recognized corresponds to each face shielding proportion (or each preset face shielding proportion range) is obtained, the corresponding face shielding proportion (or face shielding proportion range) of the image to be recognized can be determined according to the probabilities, and then the corresponding face shielding degree is obtained.
The trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
In order to obtain a trained neural network model, a plurality of sample images are required to train the original neural network model, face shielding proportion (or face shielding proportion range) of each sample image corresponds to a preset face shielding proportion (or face shielding proportion range), and each sample image is face shielding data. The number of the original images (preset number) used in the training process can be set according to actual requirements, and is not limited herein, and it can be understood that the larger the preset number is, the larger the amount of face shielding data is, and the higher the accuracy of the output result of the trained neural network model obtained by training is.
Specifically, since the original image containing the non-occluded face and the occlusion object image are both easier to obtain than the original image containing the occluded face, in order to obtain the face occlusion data more simply and conveniently, the method and the device respectively obtain the original images containing the non-occluded face and a plurality of occlusion object images in preset quantity, and obtain the images containing the occluded face in preset quantity through the arrangement and combination of the original images containing the non-occluded face and the occlusion object images. Meanwhile, because angles of faces in most face images are inconsistent, in order to determine face shielding proportion corresponding to each image more efficiently, the faces in the original image are aligned before the faces are shielded by using the shielding object image. It can be understood that the steps of alignment, occlusion, labeling and the like in the face occlusion data generation process can be completed in the corresponding image processing device.
According to the scheme provided by the embodiment of the disclosure, the image to be recognized is input into the neural network model to obtain the probability that the image to be recognized corresponds to the shielding proportion of each face, the shielding degree of the face corresponding to the image to be recognized is obtained based on the probability of the shielding proportion of each face, before the neural network model is used, the neural network model is trained by using the preset number of face shielding data, in the training process, a large number of images containing the shielded faces are obtained by arranging and combining a plurality of original images containing the non-shielded faces with various shielding object images, and the faces in the original images are aligned before the images containing the shielded faces are obtained, so that the acquisition of the face shielding data is simpler and more efficient, the neural network model for recognizing the shielding degree of the face can be obtained by more efficiently completing the training of the neural network model, and the neural network model is more beneficial to be applied to the recognition of the shielding degree of the face. In an optional embodiment of the present disclosure, after performing alignment processing on faces in each original image, performing occlusion processing on the aligned faces by using an occlusion object image, and acquiring a preset number of face occlusion data based on each aligned face and each occlusion processed face, including:
Based on at least three appointed positions on the face in each original image, carrying out alignment treatment on each face to obtain a first image corresponding to each original image;
the face in each first image is shielded by using the shielding object image, and a second image corresponding to each first image is obtained;
based on each second image and the corresponding first image, acquiring the face shielding proportion of each second image;
and generating face shielding data of a preset quantity based on the second images and the corresponding face shielding proportion.
Specifically, the process of acquiring the preset number of face shielding data can be summarized as follows: face alignment processing, face side processing, face shielding proportion acquisition and data labeling.
The face angles in the face images are different, so that the images containing the blocked faces can be conveniently generated in a subsequent batch mode, the corresponding face blocking proportion in the generated images containing the blocked faces can be conveniently calculated, and the faces can be aligned based on at least three specified positions on the faces in the original images, so that the faces are all at the same angle. In the face alignment processing step, the at least three specified positions are typically positions where corresponding organs of each face are located, for example, a position where eyes are located, a position where a nose is located, a position where a mouth is located, a position where ears are located, and the like. Specifically, after the face images in the original images are aligned, first images corresponding to the original images are obtained, angles of faces in all the first images are consistent, and the faces are not blocked.
In order to further obtain an image containing the blocked faces, the faces in the first images with the preset number are blocked by utilizing different blocking object images, so that the second images with the preset number can be obtained, and the faces in the second images are blocked by blocking objects. In the face shielding step, the shielding object may be a hat, glasses, mask, or the like.
In the step of calculating the face shielding proportion, the area of the face in the first image is extracted, the first area of the face can be obtained, the area of the shielding object in the second image is extracted, and the second area of the shielding object can be obtained. And obtaining the face shielding proportion corresponding to each second image according to the first area and the second area.
In the data labeling step, after a second image containing a blocked face and a face blocking proportion of the second image are obtained, the second image is labeled by utilizing the face blocking proportion, and corresponding face blocking data are obtained.
In an alternative embodiment of the present disclosure, the at least three designated locations include: the method for aligning the faces based on at least three appointed positions of the faces in the original images to obtain first images corresponding to the original images comprises the following steps:
Setting a first designated point, a second designated point and a third designated point which are not on a straight line on a preset canvas;
and mapping the human faces in the original images onto a preset canvas to obtain a first image corresponding to each original image, wherein the positions of the right eyes in the human faces in the original images are mapped to a first designated point, the positions of the left eyes are mapped to a second designated point, and the positions of the mouth are mapped to a third designated point.
The position of the right eye, the position of the left eye and the position of the mouth on the face are specific positions of each face, the three positions are not generally in a straight line, the angles of the faces can be determined through planes formed by the three positions, and if the angles of the planes formed by the three positions in each face are determined, the angles of the faces are unified.
The first, second and third designated points which are not on a straight line on the preset canvas are used as position references for alignment processing, wherein the first, second and third designated points can be points on an isosceles triangle, and the third designated point is positioned at the vertex position of the isosceles triangle.
Specifically, the first designated point, the second designated point and the third designated point on the preset canvas are taken as fixed points, and the position of the right eye, the position of the left eye and the position of the mouth on the face in each original image are respectively mapped to the first designated point, the second designated point and the third designated point, so that a plane formed by the position of the right eye, the position of the left eye and the position of the mouth on the face is parallel to a reference plane where the preset canvas is located. The obtained first image is convenient for subsequent shielding treatment and face shielding proportion calculation.
In an optional embodiment of the disclosure, acquiring a face occlusion proportion of each second image based on each second image and the corresponding first image includes:
acquiring a first area of an area where a face is located in each first image and a second area of an area where the face is blocked in a corresponding second image;
and acquiring the face shielding proportion corresponding to each second image based on each first area and the corresponding second area.
The method comprises the steps of extracting a region where a face in a first image is located and a region where the face in a corresponding second image is blocked by an edge extraction mode, and further calculating a first area and a second area.
Specifically, a ratio of the second area corresponding to each second image to the second area corresponding to the first image may be used as the face shielding ratio corresponding to the second image.
In an optional embodiment of the disclosure, obtaining a face shielding proportion corresponding to each second image based on each first area and the corresponding second area includes:
based on the first areas, obtaining average face areas of the faces;
and taking the ratio of each second area to the average face area as the face shielding ratio corresponding to each second image.
Specifically, in the face alignment process, the number of the adopted designated positions is always limited, so that errors may exist in the subsequently calculated first areas, and therefore the average face area of each face can be obtained according to each first area, and the average face area is used as the area of the area where each face is not shielded. Further, the ratio of each second area to the average face area may be used as the face shielding ratio corresponding to each second image.
In an optional embodiment of the disclosure, generating the preset number of face occlusion data based on each second image and the corresponding face occlusion proportion includes:
And obtaining the face shielding data of the preset quantity by using the second image entry labels corresponding to the face shielding proportion.
In an alternative embodiment of the present disclosure, the training process of the neural network model includes:
dividing the face shielding data into at least two types based on the size of the face shielding proportion, wherein each type of face shielding data corresponds to the same preset face shielding proportion range;
and training the neural network model by using the face shielding data of each type to obtain a trained neural network model.
In the face alignment process, the number of the adopted designated positions is always limited, so that errors may exist in the face shielding proportion corresponding to each second image calculated later, and in order to avoid the influence of the errors on the subsequent training process, face shielding data corresponding to each preset face shielding proportion range can be used as face shielding data. Then, even if errors exist in the face shielding proportion corresponding to the face shielding data, the actual value of the face shielding proportion can still be within the corresponding preset face shielding proportion range. In order to further reduce the influence of the error on the subsequent training process, a certain difference value (the difference value can be determined according to actual requirements) can be formed between the maximum value of one range and the minimum value of the other range in two adjacent preset face shielding proportion ranges, and it can be understood that face shielding data corresponding to the face shielding proportion between the maximum value and the minimum value is not included in any type of face shielding data, namely is not used for the subsequent training process.
For example, the face shielding data may be divided into three types, the first type is face shielding data with a corresponding face shielding ratio not greater than 20%, the second type is face shielding data with a corresponding face shielding ratio not less than 40% and not greater than 60%, and the third type is face shielding data with a corresponding face shielding ratio not less than 80%. It can be seen that if the face shielding data with the shielding proportion of 10% has errors and the real face shielding proportion of 11% in the first type face shielding data, the face shielding proportion corresponding to the face shielding data belongs to the first type face shielding data no matter whether errors exist, in other words, the errors of the face shielding proportion of the face shielding data have no influence on the subsequent training process. Further, the face shielding proportion range of the various face shielding data is different by 20%, and by the arrangement, if the face shielding data with the face shielding proportion close to 20% in the first face shielding data is not the face shielding data in the second face shielding data even if errors exist, the accuracy in model training by utilizing the various face shielding data is ensured, and similarly, the face shielding proportion range between the second face shielding data and the second face shielding data is different by 20% due to the same consideration.
The method may further comprise:
setting a corresponding face shielding degree value for each preset face shielding proportion range;
inputting the image to be identified into a trained neural network model, and acquiring a face shielding degree identification result of the image to be identified based on an output result of the trained neural network model, wherein the method comprises the following steps:
inputting the image to be identified into a trained neural network model, and outputting the probability that the face shielding proportion corresponding to the image to be identified is in the range of each preset face shielding proportion;
and acquiring a face shielding degree recognition result of the image to be recognized based on each probability and the corresponding face shielding degree value.
Specifically, since each output result is the probability that the image to be recognized corresponds to each preset face shielding proportion range, in order to quantify the face shielding degree, a corresponding face shielding degree value can be set for each face shielding proportion range, so that the face shielding degree of each output result is more visual. For example, a face shielding degree value of a preset face shielding proportion range in which the face shielding proportion is not more than 20% may be set to 0, a face shielding degree value of a preset face shielding proportion range in which the face shielding proportion is not less than 40% and not more than 60% may be set to 1, and a face shielding degree value of a preset face shielding proportion range in which the face shielding proportion is not less than 80% may be set to 2.
After the probability that the face shielding proportion corresponding to the image to be recognized is in each preset face shielding proportion range is obtained, the face shielding proportion range corresponding to the maximum probability and/or the corresponding face shielding degree value can be used as a corresponding face shielding degree recognition result.
In an optional embodiment of the present disclosure, obtaining a face occlusion degree recognition result of an image to be recognized based on each probability and a corresponding face occlusion degree value includes:
and obtaining an expected value of the face shielding degree value based on each probability and the corresponding face shielding degree value, and taking the expected value as a face shielding degree recognition result of the image to be recognized.
Specifically, the expected value of the face shielding degree value can be used as the face shielding degree recognition result of the image to be recognized. For example, if the face shielding degree value of the preset face shielding proportion range with the face shielding proportion not greater than 20% is set to 0, and the probability corresponding to the output result is P 0 Setting the face shielding degree value of the preset face shielding proportion range with the face shielding proportion not less than 40% and not more than 60% as 1, and setting the probability corresponding to the output result as P 1 Setting the face shielding degree value of the preset face shielding proportion range with the face shielding proportion not less than 80% as 2, and setting the probability corresponding to the output result as P 2 The face shielding degree recognition result of the image to be recognized can be obtained by the following expression: 0*p 0 +1*p 1 +2*p 2 Wherein p is 0 +p 1 +p 2 ＝1。
Fig. 2 is a block diagram of a face shielding degree recognition device according to an embodiment of the present disclosure, where the device 20 may include: an image acquisition module 201 to be identified, an identification result acquisition module 202, wherein:
the image to be identified acquisition module 201 is configured to acquire an image to be identified;
the recognition result obtaining module 202 is configured to input an image to be recognized into a trained neural network model, and obtain a recognition result of a face shielding degree of the image to be recognized based on an output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
According to the scheme provided by the embodiment of the disclosure, the image to be recognized is input into the neural network model to obtain the probability that the image to be recognized corresponds to the shielding proportion of each face, the shielding degree of the face corresponding to the image to be recognized is obtained based on the probability of the shielding proportion of each face, before the neural network model is used, the neural network model is trained by using the preset number of face shielding data, in the training process, a large number of images containing the shielded faces are obtained by arranging and combining a plurality of original images containing the non-shielded faces with various shielding object images, and the faces in the original images are aligned before the images containing the shielded faces are obtained, so that the acquisition of the face shielding data is simpler and more efficient, the neural network model for recognizing the shielding degree of the face can be obtained by more efficiently completing the training of the neural network model, and the neural network model is more beneficial to be applied to the recognition of the shielding degree of the face.
In an optional embodiment of the present disclosure, after performing alignment processing on faces in each original image, performing occlusion processing on the aligned faces by using an occlusion object image, and acquiring a preset number of face occlusion data based on each aligned face and each occlusion processed face, including:
Based on at least three appointed positions on the face in each original image, carrying out alignment treatment on each face to obtain a first image corresponding to each original image;
the face in each first image is shielded by using the shielding object image, and a second image corresponding to each first image is obtained;
based on each second image and the corresponding first image, acquiring the face shielding proportion of each second image;
and generating face shielding data of a preset quantity based on the second images and the corresponding face shielding proportion.
In an alternative embodiment of the present disclosure, the at least three designated locations include: the method for aligning the faces based on at least three appointed positions of the faces in the original images to obtain first images corresponding to the original images comprises the following steps:
setting a first designated point, a second designated point and a third designated point which are not on a straight line on a preset canvas;
and mapping the human faces in the original images onto a preset canvas to obtain a first image corresponding to each original image, wherein the positions of the right eyes in the human faces in the original images are mapped to a first designated point, the positions of the left eyes are mapped to a second designated point, and the positions of the mouth are mapped to a third designated point.
In an optional embodiment of the disclosure, acquiring a face occlusion proportion of each second image based on each second image and the corresponding first image includes:
acquiring a first area of an area where a face is located in each first image and a second area of an area where the face is blocked in a corresponding second image;
and acquiring the face shielding proportion corresponding to each second image based on each first area and the corresponding second area.
In an optional embodiment of the disclosure, obtaining a face shielding proportion corresponding to each second image based on each first area and the corresponding second area includes:
based on the first areas, obtaining average face areas of the faces;
and taking the ratio of each second area to the average face area as the face shielding ratio corresponding to each second image.
In an optional embodiment of the disclosure, generating the preset number of face occlusion data based on each second image and the corresponding face occlusion proportion includes:
and obtaining the face shielding data of the preset quantity by using the second image entry labels corresponding to the face shielding proportion.
In an alternative embodiment of the present disclosure, the training process of the neural network model includes:
Dividing the face shielding data into at least two types based on the size of the face shielding proportion, and setting the face shielding data of each type as the same preset face shielding proportion range;
training the neural network model by using each type of face shielding data respectively to obtain a trained neural network model;
the method further comprises the steps of:
setting a corresponding face shielding degree value for each preset face shielding proportion range;
inputting the image to be identified into a trained neural network model, and acquiring a face shielding degree identification result of the image to be identified based on an output result of the trained neural network model, wherein the method comprises the following steps:
inputting the image to be identified into a trained neural network model, and outputting the probability that the face shielding proportion corresponding to the image to be identified is in the range of each preset face shielding proportion;
and acquiring a face shielding degree recognition result of the image to be recognized based on each probability and the corresponding face shielding degree value.
In an optional embodiment of the present disclosure, obtaining a face occlusion degree recognition result of an image to be recognized based on each probability and a corresponding face occlusion degree value includes:
and obtaining an expected value of the face shielding degree value based on each probability and the corresponding face shielding degree value, and taking the expected value as a face shielding degree recognition result of the image to be recognized.
Referring now to fig. 3, there is shown a schematic diagram of an electronic device (e.g., a terminal device or server performing the method shown in fig. 1) 30 suitable for use in implementing embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 3 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
An electronic device includes: a memory and a processor, where the processor may be referred to as a processing device 301 described below, the memory may include at least one of a Read Only Memory (ROM) 302, a Random Access Memory (RAM) 303, and a storage device 308 described below, as follows:
as shown in fig. 3, the electronic device 30 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 301 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 30 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
In general, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 308 including, for example, magnetic tape, hard disk, etc.; and communication means 309. The communication means 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. While fig. 3 shows an electronic device having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing means 301.
It should be noted that the computer readable storage medium described above in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
acquiring an image to be identified; inputting the image to be identified into a trained neural network model, and acquiring an identification result of the face shielding degree of the image to be identified based on an output result of the trained neural network model; the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode: acquiring a preset number of original images, wherein the original images contain non-occluded faces; after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules or units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The name of a module or unit is not limited to the unit itself in some cases, and for example, the image acquisition module to be recognized may also be described as "a module that acquires an image to be recognized".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present disclosure, the present disclosure provides a face occlusion degree recognition method, including:
acquiring an image to be identified;
inputting the image to be identified into a trained neural network model, and acquiring an identification result of the face shielding degree of the image to be identified based on an output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
According to the face shielding degree recognition method provided by the present disclosure, after performing alignment processing on faces in each original image, performing shielding processing on the aligned faces by using shielding object images, and acquiring a preset number of face shielding data based on the aligned faces and the shielding processed faces, including:
Based on at least three appointed positions on the face in each original image, carrying out alignment treatment on each face to obtain a first image corresponding to each original image;
the face in each first image is shielded by using the shielding object image, and a second image corresponding to each first image is obtained;
based on each second image and the corresponding first image, acquiring the face shielding proportion of each second image;
and generating face shielding data of a preset quantity based on the second images and the corresponding face shielding proportion.
According to the face shielding degree recognition method provided by the present disclosure, at least three designated positions include: the method for aligning the faces based on at least three appointed positions of the faces in the original images to obtain first images corresponding to the original images comprises the following steps:
setting a first designated point, a second designated point and a third designated point which are not on a straight line on a preset canvas;
and mapping the human faces in the original images onto a preset canvas to obtain a first image corresponding to each original image, wherein the positions of the right eyes in the human faces in the original images are mapped to a first designated point, the positions of the left eyes are mapped to a second designated point, and the positions of the mouth are mapped to a third designated point.
According to the face shielding degree recognition method provided by the disclosure, based on each second image and the corresponding first image, the face shielding proportion of each second image is obtained, and the face shielding degree recognition method comprises the following steps:
acquiring a first area of an area where a face is located in each first image and a second area of an area where the face is blocked in a corresponding second image;
and acquiring the face shielding proportion corresponding to each second image based on each first area and the corresponding second area.
According to the face shielding degree recognition method provided by the present disclosure, based on each first area and the corresponding second area, the face shielding proportion corresponding to each second image is obtained, including:
based on the first areas, obtaining average face areas of the faces;
and taking the ratio of each second area to the average face area as the face shielding ratio corresponding to each second image.
According to the face shielding degree identification method provided by the disclosure, based on each second image and the corresponding face shielding proportion, face shielding data with preset quantity are generated, and the face shielding degree identification method comprises the following steps:
and obtaining the face shielding data of the preset quantity by using the second image entry labels corresponding to the face shielding proportion.
According to the face shielding degree recognition method provided by the disclosure, the training process of the neural network model comprises the following steps:
Dividing the face shielding data into at least two types based on the size of the face shielding proportion, and setting the face shielding data of each type as the same preset face shielding proportion range;
training the neural network model by using each type of face shielding data respectively to obtain a trained neural network model;
the method further comprises the steps of:
setting a corresponding face shielding degree value for each preset face shielding proportion range;
inputting the image to be identified into a trained neural network model, and acquiring a face shielding degree identification result of the image to be identified based on an output result of the trained neural network model, wherein the method comprises the following steps:
inputting the image to be identified into a trained neural network model, and outputting the probability that the face shielding proportion corresponding to the image to be identified is in the range of each preset face shielding proportion;
and acquiring a face shielding degree recognition result of the image to be recognized based on each probability and the corresponding face shielding degree value.
According to the face shielding degree recognition method provided by the disclosure, based on each probability and the corresponding face shielding degree value, a face shielding degree recognition result of an image to be recognized is obtained, and the face shielding degree recognition method comprises the following steps:
and obtaining an expected value of the face shielding degree value based on each probability and the corresponding face shielding degree value, and taking the expected value as a face shielding degree recognition result of the image to be recognized.
According to one or more embodiments of the present disclosure, the present disclosure provides a face occlusion degree recognition apparatus, including:
the image acquisition module to be identified is used for acquiring the image to be identified;
the recognition result acquisition module is used for inputting the image to be recognized into the trained neural network model and acquiring a face shielding degree recognition result of the image to be recognized based on the output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and the face shielding data are obtained in the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the faces in the original images are aligned, the aligned faces are shielded by using the shielding object images, and based on the aligned faces and the shielded faces, the shielding data of the faces with preset quantity are obtained.
In accordance with one or more embodiments of the present disclosure, the present disclosure provides an electronic device including a memory and a processor;
a memory having a computer program stored therein;
a processor for executing a computer program to implement the methods in one or more embodiments described above.
According to one or more embodiments of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, implements the method of one or more of the embodiments described above.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.

Claims (11)

1. The face shielding degree recognition method is characterized by comprising the following steps of:
acquiring an image to be identified;
inputting the image to be identified into a trained neural network model, and acquiring an identification result of the face shielding degree of the image to be identified based on an output result of the trained neural network model;
the trained neural network model is obtained through training of face shielding data of a preset number, and each face shielding data is obtained through the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the face in each original image is aligned, the aligned face is shielded by using a shielding object image, shielding proportion is obtained based on the face after the alignment and the face after the shielding, and the shielding data of the faces with the preset quantity are generated based on the shielding proportion; wherein the shielding proportion is obtained by the ratio of the average face area obtained by calculation based on the first area to the second area; the first area is the area of the face area after the alignment treatment, and the second area is the area of the shielding object after the shielding treatment.
2. The method according to claim 1, wherein after the aligning faces in the original images, performing occlusion processing on the aligned faces by using an occlusion object image, and obtaining an occlusion ratio based on the aligned faces and the occlusion-processed faces, and generating the preset number of face occlusion data based on the occlusion ratio, includes:
based on at least three appointed positions on a face in each original image, carrying out alignment processing on each face to obtain a first image corresponding to each original image;
the face in each first image is shielded by using the shielding object image, and a second image corresponding to each first image is obtained;
acquiring face shielding proportion of each second image based on each second image and the corresponding first image;
and generating the face shielding data of the preset quantity based on the second images and the corresponding face shielding proportion.
3. The method of claim 2, wherein the at least three specified locations comprise: the alignment processing is performed on each face based on at least three specified positions on the face in each original image to obtain a first image corresponding to each original image, wherein the first image comprises:
Setting a first designated point, a second designated point and a third designated point which are not on a straight line on a preset canvas;
and mapping the human face in each original image onto the preset canvas to obtain a first image corresponding to each original image, wherein the position of the right eye in the human face in each original image is mapped to the first specified point, the position of the left eye is mapped to the second specified point, and the position of the mouth is mapped to the third specified point.
4. The method of claim 2, wherein the obtaining the face mask ratio of each of the second images based on each of the second images and the corresponding first image comprises:
acquiring a first area of an area where a face is located in each first image and a second area of an area where the face is blocked in a corresponding second image;
and acquiring face shielding proportion corresponding to each second image based on each first area and the corresponding second area.
5. The method of claim 4, wherein the obtaining the face occlusion ratio corresponding to each of the second images based on each of the first areas and the corresponding second areas comprises:
based on the first areas, obtaining average face areas of the faces;
And taking the ratio of each second area to the average face area as the face shielding proportion corresponding to each second image.
6. The method of claim 2, wherein generating the predetermined amount of face occlusion data based on each of the second images and the corresponding face occlusion scale comprises:
and obtaining the face shielding data of the preset quantity by using second image entry labels corresponding to the face shielding proportion.
7. The method of claim 1, wherein the training process of the neural network model comprises:
dividing the face shielding data into at least two types based on the size of the face shielding proportion, and setting the face shielding data of each type as the same preset face shielding proportion range;
training the neural network model by using each type of face shielding data to obtain the trained neural network model;
the method further comprises the steps of:
setting a corresponding face shielding degree value for each preset face shielding proportion range;
inputting the image to be identified into a trained neural network model, and based on the output result of the trained neural network model, obtaining a face shielding degree identification result of the image to be identified, wherein the method comprises the following steps:
Inputting the image to be identified into a trained neural network model, and outputting the probability that the face shielding proportion corresponding to the image to be identified is in the range of each preset face shielding proportion;
and acquiring a face shielding degree recognition result of the image to be recognized based on each probability and the corresponding face shielding degree value.
8. The method according to claim 7, wherein the obtaining the face occlusion degree recognition result of the image to be recognized based on each probability and the corresponding face occlusion degree value includes:
and obtaining an expected value of the face shielding degree value based on each probability and the corresponding face shielding degree value, and taking the expected value as a face shielding degree recognition result of the image to be recognized.
9. A face shielding degree recognition device, characterized by comprising:
the image acquisition module to be identified is used for acquiring the image to be identified;
the recognition result acquisition module is used for inputting the image to be recognized into a trained neural network model and acquiring a face shielding degree recognition result of the image to be recognized based on an output result of the trained neural network model;
The trained neural network model is obtained through training of face shielding data of a preset number, and each face shielding data is obtained through the following mode:
acquiring a preset number of original images, wherein the original images contain non-occluded faces;
after the face in each original image is aligned, the aligned face is shielded by using a shielding object image, shielding proportion is obtained based on the face after the alignment and the face after the shielding, and the shielding data of the faces with the preset quantity are generated based on the shielding proportion; wherein the shielding proportion is obtained by the ratio of the average face area obtained by calculation based on the first area to the second area; the first area is the area of the face area after the alignment treatment, and the second area is the area of the shielding object after the shielding treatment.
10. An electronic device comprising a memory and a processor;
the memory stores a computer program;
the processor for executing the computer program to implement the method of any one of claims 1 to 8.
11. A non-transitory computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1 to 8.
CN202010225939.3A 2020-03-26 2020-03-26 Face shielding degree identification method and device, electronic equipment and readable storage medium Active CN111414879B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010225939.3A CN111414879B (en) 2020-03-26 2020-03-26 Face shielding degree identification method and device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010225939.3A CN111414879B (en) 2020-03-26 2020-03-26 Face shielding degree identification method and device, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN111414879A CN111414879A (en) 2020-07-14
CN111414879B true CN111414879B (en) 2023-06-09

Family

ID=71493385

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010225939.3A Active CN111414879B (en) 2020-03-26 2020-03-26 Face shielding degree identification method and device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN111414879B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112052789A (en) * 2020-09-03 2020-12-08 腾讯科技（深圳）有限公司 Face recognition method and device, electronic equipment and storage medium
CN111931712B (en) * 2020-09-18 2023-05-26 杭州海康威视数字技术股份有限公司 Face recognition method, device, snapshot machine and system
CN113011298B (en) * 2021-03-09 2023-12-22 阿波罗智联(北京)科技有限公司 Truncated object sample generation, target detection method, road side equipment and cloud control platform
CN112990032B (en) * 2021-03-23 2022-08-16 中国人民解放军海军航空大学航空作战勤务学院 Face image processing method and device
CN113327284B (en) * 2021-05-27 2022-08-26 北京百度网讯科技有限公司 Image recognition method and device, electronic equipment and storage medium
CN113537054B (en) * 2021-07-15 2022-11-01 重庆紫光华山智安科技有限公司 Face shielding degree calculation method and device, electronic equipment and computer readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6128397A (en) * 1997-11-21 2000-10-03 Justsystem Pittsburgh Research Center Method for finding all frontal faces in arbitrarily complex visual scenes
CN107292287A (en) * 2017-07-14 2017-10-24 深圳云天励飞技术有限公司 Face identification method, device, electronic equipment and storage medium
WO2019033572A1 (en) * 2017-08-17 2019-02-21 平安科技（深圳）有限公司 Method for detecting whether face is blocked, device and storage medium
CN109960974A (en) * 2017-12-22 2019-07-02 北京市商汤科技开发有限公司 Face critical point detection method, apparatus, electronic equipment and storage medium
WO2020015477A1 (en) * 2018-07-16 2020-01-23 阿里巴巴集团控股有限公司 Face recognition method and terminal device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105243356B (en) * 2015-09-10 2019-02-12 北京大学 A kind of method and device that establishing pedestrian detection model and pedestrian detection method
CN106803069B (en) * 2016-12-29 2021-02-09 南京邮电大学 Crowd happiness degree identification method based on deep learning
CN109840477B (en) * 2019-01-04 2020-11-24 苏州飞搜科技有限公司 Method and device for recognizing shielded face based on feature transformation
CN109871802A (en) * 2019-02-15 2019-06-11 深圳和而泰数据资源与云技术有限公司 Image detecting method and image detection device
CN110909654A (en) * 2019-11-18 2020-03-24 深圳市商汤科技有限公司 Training image generation method and device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6128397A (en) * 1997-11-21 2000-10-03 Justsystem Pittsburgh Research Center Method for finding all frontal faces in arbitrarily complex visual scenes
CN107292287A (en) * 2017-07-14 2017-10-24 深圳云天励飞技术有限公司 Face identification method, device, electronic equipment and storage medium
WO2019033572A1 (en) * 2017-08-17 2019-02-21 平安科技（深圳）有限公司 Method for detecting whether face is blocked, device and storage medium
CN109960974A (en) * 2017-12-22 2019-07-02 北京市商汤科技开发有限公司 Face critical point detection method, apparatus, electronic equipment and storage medium
WO2020015477A1 (en) * 2018-07-16 2020-01-23 阿里巴巴集团控股有限公司 Face recognition method and terminal device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Robust PCA for Face Recognition with Occlusion Using Symmetry Information;Libin Cao等;《IEEE Xplore》;第323-328页 *
基于稀疏表示的遮挡人脸识别算法的研究与实现;芦鑫;万方数据库;全文 *

Also Published As

Publication number Publication date
CN111414879A (en) 2020-07-14

Similar Documents

Publication Publication Date Title
CN111414879B (en) Face shielding degree identification method and device, electronic equipment and readable storage medium
CN111368685B (en) Method and device for identifying key points, readable medium and electronic equipment
CN111476309B (en) Image processing method, model training method, device, equipment and readable medium
CN108427939B (en) Model generation method and device
CN111369427B (en) Image processing method, image processing device, readable medium and electronic equipment
CN107622240B (en) Face detection method and device
US11436863B2 (en) Method and apparatus for outputting data
CN111783626B (en) Image recognition method, device, electronic equipment and storage medium
CN110211195B (en) Method, device, electronic equipment and computer-readable storage medium for generating image set
CN110062157B (en) Method and device for rendering image, electronic equipment and computer readable storage medium
WO2022233223A1 (en) Image splicing method and apparatus, and device and medium
WO2023179310A1 (en) Image restoration method and apparatus, device, medium, and product
CN111402122A (en) Image mapping processing method and device, readable medium and electronic equipment
CN111209856B (en) Invoice information identification method and device, electronic equipment and storage medium
CN114004905B (en) Method, device, equipment and storage medium for generating character style pictogram
CN115731341A (en) Three-dimensional human head reconstruction method, device, equipment and medium
CN110288691B (en) Method, apparatus, electronic device and computer-readable storage medium for rendering image
CN109816791B (en) Method and apparatus for generating information
CN111353470B (en) Image processing method and device, readable medium and electronic equipment
CN111340813B (en) Image instance segmentation method and device, electronic equipment and storage medium
CN110033413B (en) Image processing method, device, equipment and computer readable medium of client
CN112085035A (en) Image processing method, image processing device, electronic equipment and computer readable medium
CN112488204A (en) Training sample generation method, image segmentation method, device, equipment and medium
CN114049417B (en) Virtual character image generation method and device, readable medium and electronic equipment
CN112991147B (en) Image processing method, device, electronic equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant after: Tiktok vision (Beijing) Co.,Ltd.

Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd.

Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant after: Douyin Vision Co.,Ltd.

Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing.

Applicant before: Tiktok vision (Beijing) Co.,Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant