CN112926424A - Face occlusion recognition method and device, readable medium and equipment - Google Patents

Face occlusion recognition method and device, readable medium and equipment Download PDF

Info

Publication number
CN112926424A
CN112926424A CN202110183312.0A CN202110183312A CN112926424A CN 112926424 A CN112926424 A CN 112926424A CN 202110183312 A CN202110183312 A CN 202110183312A CN 112926424 A CN112926424 A CN 112926424A
Authority
CN
China
Prior art keywords
image
face
area
key area
recognized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110183312.0A
Other languages
Chinese (zh)
Other versions
CN112926424B (en
Inventor
岳凯宇
周峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Aibee Technology Co Ltd
Original Assignee
Beijing Aibee Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Aibee Technology Co Ltd filed Critical Beijing Aibee Technology Co Ltd
Priority to CN202110183312.0A priority Critical patent/CN112926424B/en
Priority claimed from CN202110183312.0A external-priority patent/CN112926424B/en
Publication of CN112926424A publication Critical patent/CN112926424A/en
Application granted granted Critical
Publication of CN112926424B publication Critical patent/CN112926424B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The application discloses a method, a device, a readable medium and equipment for identifying face shielding, wherein the method comprises the steps of obtaining an image to be identified; identifying the image of the image to be identified to obtain the pixel value of each key area in the image to be identified and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face; and determining the specific part of the face in the shielded state in the image to be recognized by using the face shielding recognition result of the image to be recognized. The face shielding identification model is adopted to identify the key area of the face, so that the accuracy of face part shielding identification is improved.

Description

Face occlusion recognition method and device, readable medium and equipment
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a method, an apparatus, a readable medium, and a device for recognizing facial occlusion.
Background
In the prior art, a method for identifying the occlusion of a face part comprises the following steps: and identifying a face area and a non-face area in the image, and if a specific area where the face part is located is identified as the non-face area, identifying that the specific area is blocked. For example, if the region where the mouth is located is recognized as a non-human face region, a recognition result that the mouth part of the face is blocked can be obtained.
However, in the conventional method for recognizing the occlusion of the face part, since the face region and the non-face region are recognized by the pixel value of the image, when the face part is occluded by using a substance close to the color of the face, the occluded part cannot be correctly recognized. For example, when a hand is used to block a mouth, since the color of the hand is close to the color of a human face, a region blocked by the hand is easily recognized as a human face region, and the final recognition result is a recognition result in which a face portion is not blocked and a mouth portion is not accurately recognized. Therefore, the conventional method for recognizing the face part by occlusion has low accuracy, and cannot accurately recognize the part of the face part which is occluded.
Disclosure of Invention
Based on the defects of the prior art, the application provides a face occlusion recognition method, a face occlusion recognition device, a readable medium and a device, so as to accurately recognize the occluded part of the face.
The application discloses a method for identifying face occlusion in a first aspect, which comprises the following steps:
acquiring an image to be identified;
identifying the image of the image to be identified to obtain the pixel value of each key area in the image to be identified and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face;
and determining the specific part of the face in the shielded state in the image to be recognized by using the face shielding recognition result of the image to be recognized.
Optionally, in the method for identifying face occlusion, the generating a face occlusion identification result of the image to be identified; wherein, the face occlusion recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in an occluded state, and the method comprises the following steps:
judging whether the key area is in an unshielded state or not by using the pixel value of the key area;
if the key area is judged to be in the non-shielded state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image; wherein the target image and the image to be identified have the same specification.
Optionally, in the method for identifying face occlusion, if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value of a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
Optionally, in the method for identifying face occlusion, the determining, by using the pixel value of the key region, whether the key region is in an unoccluded state includes:
and identifying a pixel relation structure in the image to be identified, and determining whether each key area is in a shielded state.
Optionally, in the method for identifying face occlusion, after the determining, by using the pixel value of the key region, whether the key region is in an unoccluded state, the method further includes:
aiming at each determined key area which is not in a shielded state, calculating the proportion of the number of target pixel points in the key area to the pixel points of the image to be identified; the target pixel points are pixel points which meet the requirements of specific parts of the face referred by the key area in the key area;
wherein: generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image, wherein the method comprises the following steps:
generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image; wherein, the requirement of the proportion is as follows: the incomplete condition of the specific image area is inversely related to the value of the ratio.
Optionally, in the method for identifying face occlusion, the identifying an image of the image to be identified, obtaining a pixel value of each key region in the image to be identified, and generating a face occlusion identification result of the image to be identified includes:
inputting the image to be recognized into a face shielding recognition model, recognizing the image of the image to be recognized by the face shielding recognition model to obtain a pixel value of each key area in the image to be recognized, and generating a face shielding recognition result of the image to be recognized; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face; the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
The second aspect of the present application discloses a face-shielding recognition device, including:
the device comprises an acquisition unit, a recognition unit and a processing unit, wherein the acquisition unit is used for acquiring an image to be recognized;
the identification unit is used for identifying the image of the image to be identified, obtaining the pixel value of each key area in the image to be identified and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face;
and the determining unit is used for determining the specific part of the face in the shielded state in the image to be recognized by utilizing the face shielding recognition result of the image to be recognized.
Alternatively, in the above-described face occlusion recognition apparatus, the recognition unit performs face occlusion recognition result generation of the image to be recognized; wherein, when the face occlusion recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in an occluded state, the face occlusion recognition result is used for:
judging whether the key area is in an unshielded state or not by using the pixel value of the key area; if the key area is judged to be in the non-shielded state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image; wherein the target image and the image to be identified have the same specification.
Optionally, in the face-occluded recognition device, if there are a plurality of key regions in the image to be recognized in the non-occluded state, for each key region in the image to be recognized in the non-occluded state, a pixel value set in a specific image region generated at a position of the target image corresponding to the key region is unique, and a size of the specific image region is a preset size.
Optionally, in the above face occlusion recognition apparatus, the recognition unit is configured to, when determining whether the key region is in an unoccluded state by using the pixel value of the key region,:
and identifying a pixel relation structure in the image to be identified, and determining whether each key area is in a shielded state.
Optionally, the above face occlusion recognition device further includes:
the calculating unit is used for calculating the ratio of the number of target pixel points in each determined key area in the unblocked state to the number of the pixel points of the image to be identified; the target pixel points are pixel points which meet the requirements of specific parts of the face referred by the key area in the key area;
the identification unit is used for generating a specific image area at a position of the target image corresponding to the key area, setting a pixel value of the specific image area as a preset value, and when the adjusted target image is obtained, the identification unit is used for:
generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image; wherein, the requirement of the proportion is as follows: the incomplete condition of the specific image area is inversely related to the value of the ratio.
Optionally, in the above apparatus for recognizing facial occlusion, the recognizing unit includes:
the identification subunit is used for inputting the image to be identified into a face shielding identification model, identifying the image of the image to be identified by the face shielding identification model, obtaining the pixel value of each key area in the image to be identified, and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face; the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
A third aspect of the application discloses a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements the method as described in any of the first aspects above.
The fourth aspect of the present application discloses an apparatus comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as in any one of the first aspects above.
According to the technical scheme, in the face occlusion recognition method provided by the embodiment of the application, the pixel value of each key area in the image to be recognized can be obtained by recognizing the image of the image to be recognized, and the face occlusion recognition result of the image to be recognized is generated. And the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state, and the key area is an image area of a specific part of the face. Compared with the mode of identifying the face area and the non-face area in the image to be identified in the prior art, the method and the device for identifying the face area and the non-face area in the image to be identified can obtain the pixel value of each key area in the image to be identified, and further can obtain the result of whether each key area in the image to be identified is in the shielded state, so that whether the face part is shielded can be accurately determined even if an object close to the face color is used for shielding the face part, and the accuracy of shielding and identifying the face part is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for identifying facial occlusion according to an embodiment of the present application;
FIG. 2a is a diagram illustrating an image to be recognized according to an embodiment of the present application;
FIG. 2b is a diagram illustrating an adjusted target image obtained from the image to be recognized in FIG. 2a according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a method for outputting an adjusted target image according to an embodiment of the present disclosure;
FIG. 4a is a schematic diagram of another image to be recognized according to an embodiment of the present disclosure;
FIG. 4b is a diagram illustrating an adjusted target image obtained from the image to be recognized in FIG. 4a according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a method for creating a face occlusion recognition model according to an embodiment of the present application;
FIG. 6a is a schematic diagram of a training image during processing according to an embodiment of the present disclosure;
FIG. 6b is a schematic diagram of an actual face occlusion recognition result of the training image shown in FIG. 6a according to an embodiment of the present application;
FIG. 7 is a flowchart illustrating another method for identifying facial occlusion according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a device for recognizing facial occlusion according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of another face occlusion recognition device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the embodiment of the present application discloses a method for identifying facial occlusion, which specifically includes the following steps:
and S101, acquiring an image to be identified.
The image to be recognized refers to a face image in which an unrecognized face is occluded or not. The image to be recognized may specifically refer to a face image indicating whether the face of an unrecognized human face is occluded or not, or may refer to a face image indicating whether the face of an unrecognized animal is occluded or not. The image to be recognized has pixel value information and position information of each pixel point.
There are many ways to acquire the image to be recognized, for example, the image to be recognized may be acquired by a camera, and then the acquired image to be recognized may be acquired. For another example, a video of a shot face is acquired, then a plurality of video frames are obtained from the video, and each video frame is acquired as an image to be recognized.
There are many scenes for acquiring the image to be recognized, for example, in the process of performing face occlusion recognition in the scene in which the camera shoots the face, the image to be recognized in the shooting process of the camera is acquired. Or processing the video into a plurality of video frames in a scene of processing the video, and taking each video frame as an image to be identified.
S102, inputting an image to be recognized into a face shielding recognition model, recognizing the pixel value of each key area in the image to be recognized by the face shielding recognition model, and obtaining and outputting a face shielding recognition result of the image to be recognized, wherein the face shielding recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in a shielded state, the key area is an image area of a specific part of a face, and the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
The face occlusion recognition model is used for recognizing whether a part occluded in the face of the image to be recognized exists, and specifically, the description is performed through an output face occlusion recognition result. The face occlusion recognition result can indicate whether each key area in the image to be recognized is in an occluded state. The key area is an image area of a specific part of the face, that is, the face-shielding recognition result of the image to be recognized output by the face-shielding recognition model can indicate whether the specific part of each face in the image to be recognized is in a shielded state. The specific part of the face may include a mouth, a left eye, a right eye and a nose, that is, the face occlusion recognition model recognizes whether a key region of the mouth, a key region of the left eye, a key region of the right eye and a key region of the nose are in an occluded state.
Specifically, an image to be recognized is input into a face shielding recognition model, the face shielding recognition model recognizes pixel points in the image to be recognized, and pixel values of each key area in the image to be recognized are recognized. The pixel relation structure refers to a special pixel structure constructed by the arrangement relation between pixel points and pixel points, when a key area is in an unblocked state, the pixel relation structure of the key area has specific characteristics, so that the face blocking recognition model can recognize the pixel value of each key area by recognizing whether the image to be recognized has the pixel relation structure characteristics of each key area, recognize the pixel value of the key area, recognize whether the key area is in the blocked state by the characteristics of the pixel relation structure in the key area, the unblocked pixel value occupation condition in the key area and other factors, and output the face blocking recognition result after obtaining the result of whether each key area is in the blocked state. Specifically, each key area in the image to be recognized can be recognized through the pixel value and the position of each pixel point in the image to be recognized, and then whether each key area is in a shielded state or not can be recognized through the pixel value in each key area.
In the prior art, when face occlusion recognition is performed, a face region and a non-face region in an image to be recognized are recognized, and the recognition principle specifically includes that the face region and the non-face region can be recognized according to the difference between the characteristics of the pixel values of the face region and the characteristics of the pixel values of the non-face region, and if a face part is recognized as the non-face region, the face part is considered to be occluded. However, the face occlusion recognition method has a disadvantage of low accuracy. When the face is shielded by using things with the color similar to the color of the face, all face parts are mistakenly considered not to be in the shielded state, namely, a non-face area does not exist in the image, and the shielded result of the face parts cannot be accurately obtained.
In the embodiment of the application, the face shielding identification model identifies the pixel value of each key area in the image to be identified, and the identification result of whether each key area is in the shielded state is obtained, namely the face shielding model has the capability of identifying whether each face part is in the shielded state, but not only a face area and a non-face area as in the prior art. If in the prior art, when a certain face part is shielded by using things with similar colors to the human face, the face shielding recognition model can recognize that the pixel value of the face part does not accord with the pixel relation structure of the face part in the non-shielding state when recognizing the key area of the face part, and then the face part can be accurately recognized to be in the shielding state.
Optionally, in a specific embodiment of the present application, an implementation manner of executing step S102 includes:
inputting an image to be recognized into a face shielding recognition model, recognizing the pixel value of each key area in the image to be recognized by the face shielding recognition model, if the key area is determined to be in an unshielded state by using the pixel value of the key area, generating a specific image area at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image.
The target image and the image to be recognized have the same specification. The target image and the image to be recognized have the same specification, which means that the target image and the image to be recognized have the same resolution. For example, if the image to be recognized is an image of 108 × 108 standard, the target image is also an image of 108 × 108.
Specifically, the face occlusion recognition model recognizes a pixel value of each key area in the image to be recognized, if the key area is determined to be in an unoccluded state by using the pixel value of the key area, a specific image area is generated at a position of the target image corresponding to the key area, the pixel value of the specific image area is set to be a preset value, the key area is distinguished from other areas in the image by generating the specific image area with the pixel value being the preset value, and the key area can be indicated to be in the unoccluded state. The target image may be another image with the same specification as the image to be recognized, or may be the image to be recognized after the pixel value setting processing. The specific image area may be a circular area with a preset size, or may be an area with another shape with a preset size.
Optionally, in a specific embodiment of the present application, the pixel values of all other image areas in the target image where the non-occluded state is not identified may be set to another preset value different from the pixel values of the specific image area. For example, if it is recognized that two key areas in the image to be recognized are in an unblocked state, a specific image area is generated at a position corresponding to the two key areas in the target image, and a pixel value is set to a preset value, for example, the pixel value is 1. And the image areas other than the two key areas can be set to have a pixel value of 2 at the corresponding target image position. After the setting is finished, the adjusted target image can be obtained, the pixel values of two key areas in the non-shielded state in the adjusted target image are different from the pixel values of other positions, and the two key areas in the non-shielded state can be explained from the target image. If the face has four key areas, but the target image shows that two key areas are in the non-blocked state, it indicates that the other two key areas are blocked and not in the non-blocked state, and therefore, the adjusted target image output in the embodiment of the present application can indicate the face-blocking recognition result of the image to be recognized.
Optionally, in a specific embodiment of the present application, if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value set in a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
If the number of the key areas in the image to be identified in the non-occluded state is multiple, in order to differently illustrate the non-occluded state of the multiple key areas in the target image, for each key area in the image to be identified in the non-occluded state, the pixel value set in the specific image area generated at the position of the target image corresponding to the key area is unique, that is, the pixel value of the specific image area generated at the position of each key area is a specific and unique preset value. For example, if the critical regions of the face share the critical region of the left eye, the critical region of the right eye, the critical region of the nose, and the critical region of the mouth. If the key area of the left eye is in the unblocked state, generating a specific image area with a pixel value of 1 at the key area position of the left eye in the target image, if the key area of the right eye is in the unblocked state, generating a specific image area with a pixel value of 2 at the key area position of the right eye in the target image, if the key area of the nose is in the unblocked state, generating a specific image area with a pixel value of 3 at the key area position of the nose in the target image, and if the key area of the mouth is in the unblocked state, generating a specific image area with a pixel value of 5 at the key area position of the mouth in the target image. Wherein, the size of the specific image area is a preset size. For example, the specific image areas for the left and right eyes are both circular image areas with a dot radius of 10 pixels, and the specific image areas for the nose and mouth are both circular image areas with a dot radius of 13 pixels. Alternatively, the pixel value may be set to 5 in other areas where the non-occluded state is not recognized.
For example, as shown in fig. 2a, in the image to be recognized, since the key region of the left eye portion, the key region of the right eye portion, the key region of the nose portion, and the key region of the mouth portion in the image to be recognized are all in an unobstructed state, a circular specific image area with a preset radius is generated on each key region in the adjusted target image shown in fig. 2b, and the pixel values of the specific image areas at different key region positions are different. Therefore, it can be known from fig. 2b that the left eye, the right eye, the nose, and the mouth of the image to be recognized are all in the non-occluded state.
Optionally, referring to fig. 3, in an embodiment of the present application, an implementation manner that an image to be recognized is input to a face occlusion recognition model, the face occlusion recognition model recognizes a pixel value of each key region in the image to be recognized, if it is determined that the key region is in an unoccluded state by using the pixel value of the key region, a specific image region is generated at a position of the target image corresponding to the key region, and the pixel value of the specific image region is set to be a preset value, so as to obtain an adjusted target image, and output the adjusted target image includes:
s301, the image to be recognized is input into a face shielding recognition model, the pixel relation structure in the image to be recognized is recognized by the face shielding recognition model, and each key area of the image to be recognized in an unblocked state is determined.
After the image to be recognized is input into the face shielding recognition model, the face shielding recognition model recognizes pixels of the image to be recognized. When different key areas are in an unblocked state, a specific pixel relation structure corresponding to the key areas can be displayed in the image to be identified, so that the position of each key area in the image to be identified in the unblocked state can be determined by identifying the pixel relation structure in the image to be identified. Namely, the step of determining the key area of each of the images to be recognized in the non-blocked state refers to the step of determining the position of the key area of each of the images to be recognized in the non-blocked state.
If the key area is in a blocked state, when the pixel relation structure in the image to be identified is identified, the specific pixel relation structure corresponding to the key area cannot be identified, and therefore the position of the key area cannot be determined. The face shielding recognition model learns the specific pixel relation structure corresponding to each key area through training, so that the face shielding recognition model has the capability of determining each key area of the image to be recognized in an unblocked state. Wherein, the specific pixel relation structure may refer to a complete pixel relation structure specific to the key region. For example, a critical region of the mouth, and the specific pixel relation structure of the critical region of the mouth may refer to the complete pixel structure of the mouth. The specific pixel relationship structure may also refer to a partial pixel relationship structure specific to the key region. For example, the critical region of the mouth, and the specific pixel relation structure of the critical region of the mouth may refer to the pixel structure of the upper lip or the pixel structure of the lower lip.
By way of example, the face has a total of four critical areas, left eye, right eye, nose, and mouth. When the key zone of the left eye is in an unblocked state, the face blocking recognition model can recognize the specific complete pixel relation structure of the left eye when recognizing the pixel relation structure of the image to be recognized, and the position of the key zone of the left eye is determined. And determining the position of the key zone of the left eye, namely determining that the key zone of the left eye is in an unblocked state. The other key areas are the same, and are not described herein again.
S302, aiming at each determined key area which is not in the shielded state, generating a specific image area at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image.
Wherein, the size of the specific image area is a preset size. After each key area in the non-occluded state is determined in step S301, for each key area in the non-occluded state, a specific image area is generated at a position of the target image corresponding to the key area, and a pixel value of the specific image area is set to a preset value, thereby indicating that the key area is in the non-occluded state. And after the pixel value is set, the adjusted target image is obtained and output. Alternatively, after setting the pixel value of the specific image area to a preset value, the pixel values of areas other than the specific image area may also be set to a pixel value different from the preset value of the key area.
Optionally, in a specific embodiment of the present application, after the step S301 is executed, the method further includes:
and calculating the ratio of the number of target pixel points in the key area to the pixel points of the image to be identified aiming at each determined key area which is not in the sheltered state.
The target pixel points are pixel points in the key area which meet the requirements of specific parts of the face indicated by the key area.
And calculating the proportion of the number of target pixel points in the key area to the pixel points of the image to be recognized aiming at each determined key area in the non-shielded state, wherein the proportion can indicate the proportion of the image area part in the non-shielded state in the key area to the image to be recognized. The pixel points meeting the requirement of the specific part specified by the key region can be pixel points which are considered to meet the requirement of the unshielded state of the specific part corresponding to the key region in the key region. For example, the key region may be a pixel point that meets the requirement of a specific pixel relationship structure corresponding to the key region. Although it is determined that the key area is in the non-blocked state, some pixel points which are not in the non-blocked state may exist in the key area, and in order to accurately reflect the blocked degree of the key area, the proportion of the number of target pixel points in the key area to the pixel points of the image to be identified can be calculated, and the blocked degree in the key area can be reflected by using the proportion. The higher the ratio of the number of the target pixel points to the pixel points of the image to be identified is, the higher the unblocked degree of the key area is, namely, the more the pixel points in the unblocked state in the key area are.
For example, if the mouth in the image to be recognized blocks the upper lip, when the pixel relationship structure of the image to be recognized is recognized, the pixel relationship structure of the lower lip which is not blocked is recognized, and the key area position of the mouth is determined, and if the mouth is not completely blocked, the ratio of the number of the target pixel points in the number of the pixel points of the image to be recognized is considered to be 30%, and the ratio of the number of the target pixel points in the calculated key area in the number of the pixel points of the image to be recognized is 15%.
Wherein, when executing step S302, the method includes:
and generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image. Wherein, the requirement of the proportion is as follows: the instances of disability in a particular image area are inversely related to the value of the occupancy ratio.
The incomplete condition of the generated specific image area relative to the key area is inversely related to the ratio value, namely the size of the specific image area is positively related to the ratio value. The smaller the occupation ratio value is, the larger the incomplete condition of the key area is, the larger the occupation ratio value is, and the smaller the incomplete condition of the key area is. For example, if the mouth in the image to be recognized blocks the upper lip, when the pixel relationship structure of the image to be recognized is recognized, the complete pixel relationship structure of the lower lip which is not blocked is recognized, and the key area position of the mouth is determined, if the mouth is not blocked at all, the proportion of the number of target pixel points in the number of pixel points of the image to be recognized is 30%, and at this time, when the upper lip is blocked, the proportion of the number of target pixel points in the key area calculated in the number of pixel points of the image to be recognized is 15%, then a round specific image area which is half defective can be generated, the pixel value of the round specific image area is a preset value, and the size of the specific image area is a preset size.
The size of the generated specific image area is a preset size, and the incomplete condition of the specific image area is in negative correlation with the value of the proportion, so the shielding degree of the corresponding key area can be reflected by the generated specific image area.
Optionally, in an embodiment of the present application, if it is not identified that the key area is in an unobstructed state, the specific image area is not generated. For example, the image to be recognized shown in fig. 4a is input into a face occlusion recognition model, a key region of a mouth part in the image to be recognized shown in fig. 4a is occluded, after the face occlusion recognition model performs recognition, an adjusted target image shown in fig. 4b is output, in the adjusted target image, a circle (i.e., a specific image area) is generated only on key regions of a left eye, a right eye and a nose part, and the key region of the mouth part does not have a circle, so that the adjusted target image can indicate whether each key region in the image to be recognized is in an occluded state.
Optionally, referring to fig. 5, in an embodiment of the present application, a method for creating a face occlusion recognition model includes:
s501, constructing a training image set.
Wherein, training the image set, includes: multiple training images and the actual face shielding recognition result of each training image.
The training image refers to a face image for which face occlusion recognition is not performed. Need not including the training image who shelters from the face among many training images, also need including not sheltering from the training image of face, and then realize training out the face that can accurately accomplish face's face and shelter from the discernment and shelter from the identification model through abundant training image sample. Optionally, in order to improve the recognition accuracy of the trained facial occlusion recognition model, the training images in the training image set may further include training images with different facial parts occluded. The richer the training image is, the higher the recognition accuracy of the trained face occlusion recognition model is.
The actual face occlusion recognition result of the training image is used for explaining whether each actual key area of the training image is in an occluded state or not. The actual face occlusion recognition result may be presented in many forms, for example, the image may be used to indicate whether each key area is in an occluded state, or whether each key area is in an occluded state may be respectively indicated by a numerical value corresponding to each key area. For example, referring to fig. 6a, before training, all the key points in the training image shown in fig. 6a are labeled, and the key points are feature points in the key area to which the key points belong. Then, for each key area in the training image shown in fig. 6a, a minimum circumscribed circle is calculated by using all key points in the key area, so that all key points fall into the calculated minimum circumscribed circle, after the minimum circumscribed circle is calculated, the minimum circumscribed circle with a pixel value of a preset value is generated at the position of the minimum circumscribed circle of each key area in the target image, and the actual face occlusion recognition result of the training image shown in fig. 6b is obtained. As can be seen from the actual face occlusion recognition result of the training image shown in fig. 6b, since each key zone in fig. 6a is not in an occluded state, the circle of each key zone in fig. 6b is generated on the image, the circle pixel value at the position of the key zone for the left eye is 1, the circle pixel value at the position of the key zone for the right eye is 2, the circle pixel value at the position of the key zone for the nose is 3, the circle pixel value at the position of the key zone for the mouth is 4, and the pixel values of other regions are 5. As can be seen from fig. 6b, each of the actual critical areas of the training image is in an unobstructed state.
S502, inputting each training image in the training image set into a neural network model respectively, and obtaining and outputting a face shielding recognition result of each training image by the neural network model respectively.
And respectively inputting each training image in the training image set into the neural network model, and respectively obtaining and outputting a face shielding recognition result of each training image by the neural network model. The face shielding recognition result of the training image output by the neural network model is used for explaining whether each key area in the training image recognized by the neural network model is in a shielded state. The neural network model outputs face occlusion recognition results in many forms, for example, in the form of an output image or in the form of an output matrix.
S503, continuously adjusting parameters in the neural network model according to the error between the face shielding recognition result of each training image output by the neural network model and the actual face shielding recognition result of the training image until the error between the face shielding recognition result of each training image output by the adjusted neural network model and the actual face shielding recognition result of the training image meets a preset convergence condition, and determining the adjusted neural network model as the face shielding recognition model.
For each training image, an error exists between the face occlusion recognition result of the training image output by the neural network model and the actual face occlusion recognition result of the training image, so that parameters in the neural network model need to be continuously adjusted, the error between the face occlusion recognition result of the training image output by the neural network model and the actual face occlusion recognition result of the training image can meet a preset convergence condition, and then the adjusted neural network model is determined as the face occlusion recognition model.
S103, determining a specific part of the face in the shielded state in the image to be recognized by using the face shielding recognition result of the image to be recognized.
Because the face shielding identification result of the image to be identified can indicate whether each key area in the image to be identified is in the shielded state, which key areas are in the shielded state can be determined according to the face shielding identification result of the image to be identified, and the key areas are image areas of specific parts of the face, so that the specific parts of the face in the shielded state in the image to be identified are determined.
According to the method for identifying the face shielding, the image to be identified is input into the face shielding identification model, the pixel value of each key area in the image to be identified is identified by the face shielding identification model, and the face shielding identification result of the image to be identified is obtained and output. The face shielding recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in a shielded state or not, the key area is an image area of a specific part of the face, and the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image, so that the specific part of the face in the shielded state in the image to be recognized is determined through the face shielding recognition result of the image to be recognized obtained through the face shielding recognition model. Compared with the mode of identifying the face area and the non-face area in the image to be identified in the prior art, the mode of identifying the key area of the face through the face shielding identification model is adopted in the application, so that whether the face part is shielded or not can be accurately determined even if the face part is shielded by an object close to the face color, and the accuracy of face part shielding identification is improved.
Referring to fig. 7, the embodiment of the present application further discloses another method for identifying facial occlusion, which specifically includes the following steps:
and S701, acquiring an image to be identified.
The principle and the execution process of step S701 are the same as those of step S101 shown in fig. 1, and reference may be made to these steps, which are not described herein again.
S702, identifying the image of the image to be identified, obtaining the pixel value of each key area in the image to be identified, and generating a face shielding identification result of the image to be identified, wherein the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state, and the key area is an image area of a specific part of a face.
Specifically, the pixel points in the image to be recognized are recognized, and the pixel value of each key area in the image to be recognized is recognized. The pixel relation structure refers to a special pixel structure constructed by the arrangement relation between pixel points and pixel points, when a key area is in an unshielded state, the pixel relation structure of the key area has specific characteristics, so that the pixel value of each key area can be identified by identifying whether the image to be identified has the pixel relation structure characteristics of each key area, the pixel value of the key area is identified, whether the key area is in the shaded state is identified by the characteristics of the pixel relation structure in the key area, the proportion condition of the unshielded pixel values in the key area and other factors, and then the face shading identification result of the image to be identified can be generated.
Optionally, each key area in the image to be recognized may be identified by recognizing the pixel value and position of each pixel point in the image to be recognized, so as to obtain the pixel value of each key area, and then whether each key area is in a shielded state is recognized by the pixel value in each key area, so as to generate a face shielding recognition result of the image to be recognized.
In the prior art, when face occlusion recognition is performed, a face region and a non-face region in an image to be recognized are recognized, and the recognition principle specifically includes that the face region and the non-face region can be recognized according to the difference between the characteristics of the pixel values of the face region and the characteristics of the pixel values of the non-face region, and if a face part is recognized as the non-face region, the face part is considered to be occluded. However, the face occlusion recognition method has a disadvantage of low accuracy. When the face is shielded by using things with the color similar to the color of the face, all face parts are mistakenly considered not to be in the shielded state, namely, a non-face area does not exist in the image, and the shielded result of the face parts cannot be accurately obtained.
In the embodiment of the application, the pixel value of each key area in the image to be recognized is recognized, and the recognition result of whether each key area is in a shielded state or not is obtained, namely whether each face part is in a shielded state or not is recognized, rather than only a face area and a non-face area can be recognized as in the prior art, and the accuracy of face shielding recognition is improved by recognizing the pixel value of each key area. If in the prior art, when a certain face part is shielded by using things with similar colors to the human face, when a key area of the face part is identified, the pixel value of the face part can be identified to be not in accordance with the pixel relation structure of the face part in the non-shielded state, and then the face part can be accurately identified to be in the shielded state.
Optionally, when step S702 is executed, the step S702 may be implemented by using a neural network model having a capability of identifying whether each key region is in an occlusion state, or the step S702 may also be implemented by using some image recognition algorithms to recognize an image of the image to be recognized, obtain a pixel value of each key region in the image to be recognized, and generate a face occlusion recognition result of the image to be recognized. It should be noted that, there are many specific ways to execute step S702, including but not limited to what is proposed in the embodiments of the present application.
Optionally, in a specific embodiment of the present application, an implementation manner of executing step S702 includes:
and inputting the image to be recognized into a face shielding recognition model, recognizing the image of the image to be recognized by the face shielding recognition model, obtaining the pixel value of each key area in the image to be recognized, and generating a face shielding recognition result of the image to be recognized.
The face shielding recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in a shielded state or not, the key area is an image area of a specific part of a face, and the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
It should be noted that the execution process and principle of inputting the image to be recognized to the face occlusion recognition model, recognizing the image of the image to be recognized by the face occlusion recognition model to obtain the pixel value of each key region in the image to be recognized, and generating the face occlusion recognition result of the image to be recognized are the same as those of step S102 shown in fig. 1, and are referred to here and will not be described again.
Optionally, in a specific embodiment of the present application, the step S702 of generating a face-occlusion recognition result of the image to be recognized, where the face-occlusion recognition result of the image to be recognized is used to describe whether each key region in the image to be recognized is in an occluded state, includes:
and judging whether the key area is in an unshielded state or not by utilizing the pixel value of the key area, if so, generating a specific image area at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, and obtaining the adjusted target image. And the target image and the image to be recognized have the same specification.
It should be noted that the execution process and principle of "using the pixel value of the key area to determine whether the key area is in the non-occluded state, if the key area is determined to be in the non-occluded state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area to a preset value to obtain an adjusted target image" is similar to the execution process and principle of "inputting the image to be recognized to the face occlusion recognition model, recognizing the pixel value of each key area in the image to be recognized by the face occlusion recognition model, if the key area is determined to be in the non-occluded state by using the pixel value of the key area, generating the specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area to a preset value to obtain an adjusted target image, and outputting the adjusted target image" mentioned in step S102, for reference, further description is omitted here.
It should be noted that, when performing the implementation of "determining whether the key area is in the non-blocked state by using the pixel value of the key area, if the key area is determined to be in the non-blocked state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area to the preset value to obtain the adjusted target image", except that "inputting the image to be recognized to the face blocking recognition model, recognizing the pixel value of each key area in the image to be recognized by the face blocking recognition model" as mentioned in the above-mentioned one implementation of step S102, if the key area is determined to be in the non-blocked state by using the pixel value of the key area, generating the specific image area at the position of the key area corresponding to the target image, setting the pixel value of the specific image area to the preset value to obtain the adjusted target image, and outputting the adjusted target image ", the implementation is implemented by using the face blocking recognition model, the implementation and principles may also be implemented using some other form of image recognition algorithm, image processing algorithm, etc.
Optionally, in a specific embodiment of the present application, if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value set in a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
Optionally, in an embodiment of the present application, an implementation manner of determining whether the key region is in an unobstructed state by using a pixel value of the key region is performed, including:
and identifying a pixel relation structure in the image to be identified, and determining whether each key area is in a blocked state.
It should be noted that, in the embodiment of the present application, an execution process and a principle of "identifying a pixel relationship structure in an image to be identified and determining whether each key region is in a blocked state" are similar to those of step S301 shown in fig. 3, and may be referred to, and details are not repeated here.
It should be noted that, when "recognizing the pixel relationship structure in the image to be recognized and determining whether each key region is in the blocked state" is executed, the execution process and principle may be implemented by using some image recognition algorithms, image processing algorithms, and other manners, besides the face blocking recognition model adopted in step S301.
Optionally, in a specific embodiment of the present application, after further determining whether the key area is in an unobstructed state by using a pixel value of the key area, the method further includes:
and calculating the ratio of the number of target pixel points in the key area to the pixel points of the image to be identified aiming at each determined key area which is not in the sheltered state.
The target pixel points are pixel points in the key area which meet the requirements of specific parts of the face indicated by the key area. Generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image, wherein the method comprises the following steps: and aiming at each determined key area in the non-shielded state, generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image. Wherein the requirements of the proportion are as follows: the instances of disability in a particular image area are inversely related to the value of the occupancy ratio.
It should be noted that, in the embodiment of the present application, an execution process and a principle of "calculating, for each determined key region in the non-blocked state, a ratio of the number of target pixels in the key region to pixels of the image to be recognized" are similar to an execution principle and a process of "calculating, for each determined key region in the non-blocked state, a ratio of the number of target pixels in the key region to pixels of the image to be recognized" that are also executed after step S301 is executed in the embodiment shown in fig. 3, and thus, reference may be made to the execution principle and the process, and details are not repeated here.
In the embodiment of the application, "for each determined key area in an unshaded state, a specific image area meeting the requirement of the proportion is generated at the position of the target image corresponding to the key area, the pixel value of the specific image area is set to be a preset value, an adjusted target image is obtained, and the adjusted target image is output. Wherein the requirements of the proportion are as follows: the execution process and principle of the negative correlation between the incomplete condition of the specific image area and the numerical value of the occupation ratio are the same as that of the step S302, that is, the specific image area meeting the requirement of the occupation ratio is generated at the position of the target image corresponding to the key area for each determined key area in the non-blocked state, the pixel value of the specific image area is set to be a preset value, the adjusted target image is obtained, and the adjusted target image is output. Wherein, the requirement of the proportion is as follows: the implementation and principle of the embodiment of "the incomplete condition of the specific image area is inversely related to the value of the occupation ratio" are similar, and are not described herein again.
It should be further noted that, when "calculating the ratio of the number of target pixels in the key region to the pixels of the image to be recognized for each determined key region in the non-blocked state" in the embodiment of the present application is executed, "calculating the number of target pixels in the key region to the ratio of the pixels of the image to be recognized for each determined key region in the non-blocked state" that is also executed after step S301 in the embodiment shown in fig. 3 is implemented by using a face-blocking recognition model, "in addition, some image recognition algorithms, image processing algorithms, and other manners may also be used to implement the execution process and principle.
S703, determining the specific part of the face in the shielded state in the image to be recognized by using the face shielding recognition result of the image to be recognized.
It should be noted that the execution process and principle of step S703 are the same as step S103 shown in fig. 1, and reference may be made to this step, which is not described herein again.
In the method for identifying the face occlusion provided by the embodiment of the application, the pixel value of each key area in the image to be identified can be obtained by identifying the image of the image to be identified, and the face occlusion identification result of the image to be identified is generated. And the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state, and the key area is an image area of a specific part of the face. Compared with the mode of identifying the face area and the non-face area in the image to be identified in the prior art, the method and the device for identifying the face area and the non-face area in the image to be identified can obtain the pixel value of each key area in the image to be identified, and further can obtain the result of whether each key area in the image to be identified is in the shielded state, so that whether the face part is shielded can be accurately determined even if an object close to the face color is used for shielding the face part, and the accuracy of shielding and identifying the face part is improved.
Referring to fig. 8, based on the method for identifying facial occlusion provided in the embodiment shown in fig. 1, the embodiment of the present application correspondingly discloses an apparatus for identifying facial occlusion, including: a first acquisition unit 801, a first recognition unit 802, and a first determination unit 803.
A first acquiring unit 801, configured to acquire an image to be identified.
The first identification unit 802 is configured to input the image to be identified to a face occlusion identification model, identify a pixel value of each key region in the image to be identified by the face occlusion identification model, and obtain and output a face occlusion identification result of the image to be identified. The face shielding recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in a shielded state or not, the key area is an image area of a specific part of a face, and the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
Optionally, in a specific embodiment of the present application, the first identifying unit 802 includes:
the first identification subunit is used for inputting the image to be identified to the face shielding identification model, identifying the pixel value of each key area in the image to be identified by the face shielding identification model, if the key area is determined to be in an unshielded state by using the pixel value of the key area, generating a specific image area at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image. The target image and the image to be recognized have the same specification.
Optionally, in a specific embodiment of the present application, the first identifying subunit includes: a first determining subunit and a setting subunit.
The first determining subunit is configured to input the image to be recognized to the face occlusion recognition model, recognize the pixel relationship structure in the image to be recognized by the face occlusion recognition model, and determine a key region in which each of the image to be recognized is in an unoccluded state.
And the setting subunit is used for generating a specific image area at the position of the target image corresponding to the key area aiming at each determined key area which is not in the shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image. Wherein, the size of the specific image area is a preset size.
Optionally, in a specific embodiment of the present application, if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value set in a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
Optionally, in a specific embodiment of the present application, the method further includes:
and the first calculating unit is used for calculating the ratio of the number of target pixel points in the key area to the pixel points of the image to be identified aiming at each determined key area which is in the unblocked state. The target pixel points are pixel points in the key area which meet the requirements of specific parts of the face indicated by the key area. A setup subunit comprising: and the generating subunit is used for generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining the adjusted target image, and outputting the adjusted target image. Wherein, the requirement of the proportion is as follows: the instances of disability in a particular image area are inversely related to the value of the occupancy ratio.
Optionally, in a specific embodiment of the present application, the method further includes: the device comprises a construction unit, a training recognition unit and an adjusting unit.
And the construction unit is used for constructing a training image set. Wherein, training the image set, includes: multiple training images and the actual face shielding recognition result of each training image.
And the training identification unit is used for respectively inputting each training image in the training image set into the neural network model, and respectively obtaining and outputting the face shielding identification result of each training image by the neural network model.
And the adjusting unit is used for continuously adjusting parameters in the neural network model according to the error between the face shielding recognition result of each training image output by the neural network model and the actual face shielding recognition result of the training image until the error between the face shielding recognition result of each training image output by the adjusted neural network model and the actual face shielding recognition result of the training image meets a preset convergence condition, and determining the adjusted neural network model as the face shielding recognition model.
A first determining unit 803, configured to determine a specific part of the face in the occluded state in the image to be recognized, using the face occlusion recognition result of the image to be recognized.
The specific principle and the execution process of each unit in the device for identifying face occlusion disclosed in the embodiment of the present application are the same as those of the method for identifying face occlusion disclosed in the embodiment of the present application, and reference may be made to corresponding parts in the method for identifying face occlusion disclosed in the embodiment of the present application, which are not described herein again.
In the identification apparatus for face occlusion provided in the embodiment of the application, the image to be identified is input to the face occlusion identification model through the first identification unit 802, the pixel value of each key region in the image to be identified is identified by the face occlusion identification model, and a face occlusion identification result of the image to be identified is obtained and output. The face occlusion recognition result of the image to be recognized is used to indicate whether each key region in the image to be recognized is in an occluded state, the key region is an image region of a specific part of the face, and the face occlusion recognition model is obtained by training the neural network model with a plurality of training images and an actual face occlusion recognition result of each training image, so that the first determining unit 803 can determine the specific part of the face in the occluded state in the image to be recognized according to the face occlusion recognition result of the image to be recognized, which is obtained by the face occlusion recognition model. Compared with the mode of identifying the face area and the non-face area in the image to be identified in the prior art, the mode of identifying the key area of the face through the face shielding identification model is adopted in the application, so that whether the face part is shielded or not can be accurately determined even if the face part is shielded by an object close to the face color, and the accuracy of face part shielding identification is improved.
Referring to fig. 9, based on the method for identifying facial occlusion proposed by the embodiment shown in fig. 7, the embodiment of the present application correspondingly discloses an apparatus for identifying facial occlusion, which includes: an acquisition unit 901, a recognition unit 902 and a determination unit 903.
An acquiring unit 901, configured to acquire an image to be recognized.
The identification unit 902 is configured to identify an image of the image to be identified, obtain a pixel value of each key region in the image to be identified, and generate a face occlusion identification result of the image to be identified, where the face occlusion identification result of the image to be identified is used to indicate whether each key region in the image to be identified is in an occluded state; the key region is an image region of a specific part of the face.
Optionally, in a specific embodiment of the present application, the identifying unit 902 performs generating a face occlusion identification result of the image to be identified; the method for recognizing the face occlusion of the image to be recognized includes the following steps:
and judging whether the key area is in an unshielded state or not by utilizing the pixel value of the key area, if so, generating a specific image area at the position of the target image corresponding to the key area, setting the pixel value of the specific image area as a preset value, and obtaining the adjusted target image. The target image and the image to be recognized have the same specification.
Optionally, in a specific embodiment of the present application, if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value set in a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
Optionally, in an embodiment of the present application, the identifying unit 902 performs, when determining whether the key area is in an unblocked state by using a pixel value of the key area, to:
and identifying a pixel relation structure in the image to be identified, and determining whether each key area is in a blocked state.
Optionally, in a specific embodiment of the present application, the method further includes:
and the calculating unit is used for calculating the ratio of the number of target pixel points in the key area to the pixel points of the image to be identified aiming at each determined key area which is not in the sheltered state. The target pixel points are pixel points in the key area which meet the requirements of specific parts of the face indicated by the key area.
The identifying unit 902 executes to generate a specific image area at a position of the target image corresponding to the key area, and sets a pixel value of the specific image area to a preset value, so as to, when obtaining the adjusted target image:
and generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image. Wherein, the requirement of the proportion is as follows: the incomplete condition of the specific image area is inversely related to the value of the occupation ratio.
Optionally, in a specific embodiment of the present application, the identifying unit 902 includes:
and the identification subunit is used for inputting the image to be identified into the face shielding identification model, identifying the image of the image to be identified by the face shielding identification model, obtaining the pixel value of each key area in the image to be identified, and generating the face shielding identification result of the image to be identified. The face shielding recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in a shielded state or not, the key area is an image area of a specific part of a face, and the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
A determining unit 903, configured to determine a specific part of the face in the occluded state in the image to be recognized, using the face occlusion recognition result of the image to be recognized.
The specific principle and the execution process of each unit in the device for identifying face occlusion disclosed in the embodiment of the present application are the same as those of the method for identifying face occlusion disclosed in the embodiment of the present application, and reference may be made to corresponding parts in the method for identifying face occlusion disclosed in the embodiment of the present application, which are not described herein again.
In the identification device for face occlusion provided by the embodiment of the application, since the image of the image to be identified can be identified by the identification unit 902 in the application, the pixel value of each key area in the image to be identified is obtained, and the face occlusion identification result of the image to be identified is generated. And the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state, and the key area is an image area of a specific part of the face. Compared with the mode of identifying the face area and the non-face area in the image to be identified in the prior art, the method and the device for identifying the face area and the non-face area in the image to be identified can obtain the pixel value of each key area in the image to be identified, and further can obtain the result of whether each key area in the image to be identified is in the shielded state, so that whether the face part is shielded can be accurately determined even if an object close to the face color is used for shielding the face part, and the accuracy of shielding and identifying the face part is improved.
The embodiment of the application provides a computer readable medium, on which a computer program is stored, wherein the program is executed by a processor to implement the method for identifying facial occlusion provided by the above method embodiments.
An embodiment of the present application provides an apparatus, including: one or more processors, a storage device, having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method for identifying facial occlusions provided by the above method embodiments.
Those skilled in the art can make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Claims (10)

1. A method for identifying facial occlusion, comprising:
acquiring an image to be identified;
identifying the image of the image to be identified to obtain the pixel value of each key area in the image to be identified and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face;
and determining the specific part of the face in the shielded state in the image to be recognized by using the face shielding recognition result of the image to be recognized.
2. The method according to claim 1, wherein the generating of the face occlusion recognition result of the image to be recognized; wherein, the face occlusion recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in an occluded state, and the method comprises the following steps:
judging whether the key area is in an unshielded state or not by using the pixel value of the key area;
if the key area is judged to be in the non-shielded state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image; wherein the target image and the image to be identified have the same specification.
3. The method according to claim 2, wherein if there are a plurality of key areas in the image to be identified in the non-occluded state, for each key area in the image to be identified in the non-occluded state, a pixel value of a specific image area generated at a position of the target image corresponding to the key area is unique, and a size of the specific image area is a preset size.
4. The method according to claim 2, wherein the determining whether the key region is in an unobstructed state by using the pixel values of the key region comprises:
and identifying a pixel relation structure in the image to be identified, and determining whether each key area is in a shielded state.
5. The method according to claim 2, wherein after determining whether the key region is in an unobstructed state by using the pixel values of the key region, the method further comprises:
aiming at each determined key area which is not in a shielded state, calculating the proportion of the number of target pixel points in the key area to the pixel points of the image to be identified; the target pixel points are pixel points which meet the requirements of specific parts of the face referred by the key area in the key area;
wherein: generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image, wherein the method comprises the following steps:
generating a specific image area meeting the requirement of the proportion at the position of the target image corresponding to the key area aiming at each determined key area in the non-shielded state, setting the pixel value of the specific image area as a preset value, obtaining an adjusted target image, and outputting the adjusted target image; wherein, the requirement of the proportion is as follows: the incomplete condition of the specific image area is inversely related to the value of the ratio.
6. The method according to claim 1, wherein the recognizing the image of the image to be recognized, obtaining a pixel value of each key area in the image to be recognized, and generating a face-occlusion recognition result of the image to be recognized comprises:
inputting the image to be recognized into a face shielding recognition model, recognizing the image of the image to be recognized by the face shielding recognition model to obtain a pixel value of each key area in the image to be recognized, and generating a face shielding recognition result of the image to be recognized; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face; the face shielding recognition model is obtained by training a neural network model through a plurality of training images and the actual face shielding recognition result of each training image.
7. An apparatus for recognizing facial occlusion, comprising:
the device comprises an acquisition unit, a recognition unit and a processing unit, wherein the acquisition unit is used for acquiring an image to be recognized;
the identification unit is used for identifying the image of the image to be identified, obtaining the pixel value of each key area in the image to be identified and generating a face shielding identification result of the image to be identified; the face shielding identification result of the image to be identified is used for explaining whether each key area in the image to be identified is in a shielded state; the key area is an image area of a specific part of the face;
and the determining unit is used for determining the specific part of the face in the shielded state in the image to be recognized by utilizing the face shielding recognition result of the image to be recognized.
8. The apparatus according to claim 7, wherein the recognition unit performs generation of a face occlusion recognition result of the image to be recognized; wherein, when the face occlusion recognition result of the image to be recognized is used for explaining whether each key area in the image to be recognized is in an occluded state, the face occlusion recognition result is used for:
judging whether the key area is in an unshielded state or not by using the pixel value of the key area; if the key area is judged to be in the non-shielded state, generating a specific image area at the position of the target image corresponding to the key area, and setting the pixel value of the specific image area as a preset value to obtain an adjusted target image; wherein the target image and the image to be identified have the same specification.
9. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1 to 6.
10. An apparatus, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
CN202110183312.0A 2021-02-10 Face shielding recognition method, device, readable medium and equipment Active CN112926424B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110183312.0A CN112926424B (en) 2021-02-10 Face shielding recognition method, device, readable medium and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110183312.0A CN112926424B (en) 2021-02-10 Face shielding recognition method, device, readable medium and equipment

Publications (2)

Publication Number Publication Date
CN112926424A true CN112926424A (en) 2021-06-08
CN112926424B CN112926424B (en) 2024-05-31

Family

ID=

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120113912A (en) * 2011-04-06 2012-10-16 한국수자원공사 The detection and recovery method of occlusion of a face image using a correlation based method
CN108319953A (en) * 2017-07-27 2018-07-24 腾讯科技(深圳)有限公司 Occlusion detection method and device, electronic equipment and the storage medium of target object
CN111027504A (en) * 2019-12-18 2020-04-17 上海眼控科技股份有限公司 Face key point detection method, device, equipment and storage medium
CN111160269A (en) * 2019-12-30 2020-05-15 广东工业大学 Face key point detection method and device
CN111428581A (en) * 2020-03-05 2020-07-17 平安科技(深圳)有限公司 Face shielding detection method and system
CN111797773A (en) * 2020-07-07 2020-10-20 广州广电卓识智能科技有限公司 Method, device and equipment for detecting occlusion of key parts of human face
CN111814569A (en) * 2020-06-12 2020-10-23 深圳禾思众成科技有限公司 Method and system for detecting human face shielding area

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120113912A (en) * 2011-04-06 2012-10-16 한국수자원공사 The detection and recovery method of occlusion of a face image using a correlation based method
CN108319953A (en) * 2017-07-27 2018-07-24 腾讯科技(深圳)有限公司 Occlusion detection method and device, electronic equipment and the storage medium of target object
CN111027504A (en) * 2019-12-18 2020-04-17 上海眼控科技股份有限公司 Face key point detection method, device, equipment and storage medium
CN111160269A (en) * 2019-12-30 2020-05-15 广东工业大学 Face key point detection method and device
CN111428581A (en) * 2020-03-05 2020-07-17 平安科技(深圳)有限公司 Face shielding detection method and system
CN111814569A (en) * 2020-06-12 2020-10-23 深圳禾思众成科技有限公司 Method and system for detecting human face shielding area
CN111797773A (en) * 2020-07-07 2020-10-20 广州广电卓识智能科技有限公司 Method, device and equipment for detecting occlusion of key parts of human face

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王振华等: "基于深度学习的部分遮挡人脸识别", 电子技术与软件工程, no. 02, pages 145 - 147 *

Similar Documents

Publication Publication Date Title
CN110443192B (en) Non-interactive human face living body detection method and system based on binocular image
CN108520219B (en) Multi-scale rapid face detection method based on convolutional neural network feature fusion
US20200380279A1 (en) Method and apparatus for liveness detection, electronic device, and storage medium
JP4974788B2 (en) Image processing apparatus, image processing method, program, and storage medium
EP1650711B1 (en) Image processing device, imaging device, image processing method
EP2378486B1 (en) Multi-mode region-of-interest video object segmentation
US20210303919A1 (en) Image processing method and apparatus for target recognition
US20240021015A1 (en) System and method for selecting images for facial recognition processing
JPH0424503A (en) Apparatus for detecting position of eye
CN111914748B (en) Face recognition method, device, electronic equipment and computer readable storage medium
CN110047059B (en) Image processing method and device, electronic equipment and readable storage medium
CN110895802A (en) Image processing method and device
CN112836625A (en) Face living body detection method and device and electronic equipment
US20060269128A1 (en) Image correction method and apparatus
CN111325058A (en) Driving behavior detection method, device and system and storage medium
CN114529962A (en) Image feature processing method and device, electronic equipment and storage medium
CN111832464A (en) Living body detection method and device based on near-infrared camera
CN112926424A (en) Face occlusion recognition method and device, readable medium and equipment
CN111553271A (en) Face feature fusion method
JPH11306348A (en) Method and device for object detection
JP2005316743A (en) Image processing method and device
CN112926424B (en) Face shielding recognition method, device, readable medium and equipment
CN112966575B (en) Target face recognition method and device applied to smart community
JPH04174309A (en) Driver's eye position detecting apparatus and condition detecting apparatus
JP6851241B2 (en) Image analyzer

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant