WO2021237968A1 - 一种应用于人脸遮挡场景的活体检测方法及装置 - Google Patents
一种应用于人脸遮挡场景的活体检测方法及装置 Download PDFInfo
- Publication number
- WO2021237968A1 WO2021237968A1 PCT/CN2020/112222 CN2020112222W WO2021237968A1 WO 2021237968 A1 WO2021237968 A1 WO 2021237968A1 CN 2020112222 W CN2020112222 W CN 2020112222W WO 2021237968 A1 WO2021237968 A1 WO 2021237968A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- living body
- detection
- infrared
- rgb
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 299
- 230000001815 facial effect Effects 0.000 claims abstract description 94
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000002310 reflectometry Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000001727 in vivo Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 210000000988 bone and bone Anatomy 0.000 description 3
- 230000009977 dual effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 210000000214 mouth Anatomy 0.000 description 3
- 210000001331 nose Anatomy 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 210000000887 face Anatomy 0.000 description 2
- 210000001847 jaw Anatomy 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000001755 vocal effect Effects 0.000 description 2
- 210000000216 zygoma Anatomy 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 210000000088 lip Anatomy 0.000 description 1
- 210000000537 nasal bone Anatomy 0.000 description 1
- 210000003928 nasal cavity Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 210000001584 soft palate Anatomy 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 210000002105 tongue Anatomy 0.000 description 1
- 210000000515 tooth Anatomy 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 210000001260 vocal cord Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
- G06V40/45—Detection of the body part being alive
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/02—Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Definitions
- the present disclosure relates to computer technology, and in particular, to a living body detection method and device applied to a scene where a human face is occluded.
- the live detection link can effectively identify the following hidden attack hazards: photo attacks, screen attacks, mask attacks, etc. .
- the living body detection technology has tended to be perfect, but in special applications, inaccurate detection may still occur.
- everyone wears masks resulting in an excessively large area of occlusion of faces and reduced facial features, which in turn affects face detection and recognition and living body detection links to a large extent.
- the embodiments of the present disclosure provide a living body detection method and device applied to a face occluded scene, so as to improve the accuracy of living body detection in a face occluded scene.
- a living body detection method applied to a scene where a face is occluded includes:
- For the target face take RGB images and take infrared images, and use a preset facial feature recognition model to perform face recognition on the RGB image and the infrared image respectively, and obtain the corresponding RGB image facial feature recognition results And infrared image facial feature recognition results;
- both the first detection result and the second detection result indicate the presence of a living body
- the preset facial feature recognition model is adopted, and before the face recognition is performed on the RGB image and the infrared image respectively, the following operations are further performed Any one or combination of:
- Performing obstruction detection on the RGB image determining the obstruction area of the obstruction on the target face in the RGB image, and the ratio of the face area of the target face to the second set threshold.
- a preset living body detection model to perform living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result to obtain the first detection result, including:
- the living body detection model is used to determine that the infrared image facial feature recognition result includes living body features, and when the number of facial features included in the partial RGB image facial feature recognition result reaches a preset number threshold , It is determined that the first detection result indicates the presence of a living body.
- using the living body detection model to perform living body detection based on the infrared reflectance of the RGB eye image and the infrared reflectance of the infrared image eye image to obtain the second detection result includes:
- the second detection result represents the existence of a living body.
- the method before determining that the first detection result and the second detection result both indicate the presence of a living body, the method further includes:
- the area of the RGB image eye image is larger than the area of the RGB image eye image
- the area of the infrared image eye image is larger than the area of the infrared image eye image
- Using a preset live detection model based on the texture recognition result of the spliced edge of the RGB image eye image, or/and, based on the texture recognition result of the spliced edge of the infrared image eye image, perform the live detection, and obtain the first Three detection results, wherein if the third detection result indicates that there is no texture information of the spliced edge, it is determined that there is a living body;
- the determination to pass the living body detection includes:
- both the first detection result and the second detection result characterize the presence of a living body
- the third detection result also characterizes the presence of a living body
- Acquire audio data entered by a person corresponding to the target face extract voiceprint features of the audio data, and determine that there is a living body based on the voiceprint features.
- a living body detection device applied to a scene where a face is occluded includes:
- the photographing unit is used to photograph RGB images and infrared images of the target face, and adopt a preset facial feature recognition model to perform face recognition on the RGB image and the infrared image respectively to obtain the corresponding RGB image Facial feature recognition results and infrared image facial feature recognition results;
- the first detection unit is configured to determine the position of a non-occluded object according to the detection result of the occluder of the RGB image, and filter out the part of the RGB image corresponding to the position of the non-occluded object from the facial feature recognition result of the RGB image A facial feature recognition result, and using a preset living body detection model, performing living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result, to obtain a first detection result;
- the second detection unit is used to extract an RGB image eye image from the RGB image, and an infrared image eye image from the infrared image, using the living body detection model, based on the infrared reflectivity of the RGB eye image and Performing live body detection on the infrared reflectance of the infrared image eye image to obtain a second detection result;
- the judging unit is configured to determine that when the first detection result and the second detection result both indicate the presence of a living body, it is determined to pass the living body detection.
- the photographing unit further Used to perform any one or combination of the following operations:
- Performing obstruction detection on the RGB image determining the obstruction area of the obstruction on the target face in the RGB image, and the ratio of the face area of the target face to the second set threshold.
- a preset living body detection model is used to perform living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result, and when the first detection result is obtained, the first detection unit Used for:
- the living body detection model is used to determine that the infrared image facial feature recognition result includes living body features, and when the number of facial features included in the partial RGB image facial feature recognition result reaches a preset number threshold , It is determined that the first detection result indicates the presence of a living body.
- the living body detection model is used to perform living body detection based on the infrared reflectance of the RGB eye image and the infrared reflectance of the infrared image eye image, and when the second detection result is obtained, the second detection unit uses At:
- the second detection result represents the existence of a living body.
- it further includes:
- the third detection unit is configured to perform the following operations before determining that both the first detection result and the second detection result indicate the presence of a living body:
- the area of the RGB image eye image is larger than the area of the RGB image eye image
- the area of the infrared image eye image is larger than the area of the infrared image eye image
- Using a preset live detection model based on the texture recognition result of the spliced edge of the RGB image eye image, or/and, based on the texture recognition result of the spliced edge of the infrared image eye image, perform the live detection, and obtain the first Three detection results, wherein if the third detection result indicates that there is no texture information of the spliced edge, it is determined that there is a living body;
- the determination unit is configured to:
- both the first detection result and the second detection result characterize the presence of a living body
- the third detection result also characterizes the presence of a living body
- it further includes:
- the voiceprint detection unit is used to obtain the audio data entered by the person corresponding to the target face, extract the voiceprint feature of the audio data, and determine the presence of a living body based on the voiceprint feature before determining to pass the live body detection.
- a living body detection device includes:
- Memory used to store executable instructions
- the processor is configured to read and execute executable instructions stored in the memory to implement the method for media data encryption processing according to any one of the first aspect.
- a computer-readable storage medium when an instruction in the computer-readable storage medium is executed by a processor, enables the method for media data encryption processing as described in any one of the first aspect to be executed.
- dual cameras are used to photograph the occluded face, and an RGB image and an infrared image are obtained respectively. Then, the facial feature recognition result and the infrared image of the unoccluded area of the face in the RGB image are combined. According to the facial feature recognition result in the image, the first detection result of living body detection is obtained, and further, the second detection result of living body detection is obtained based on the infrared reflectance of the RGB image eye image and the infrared reflectance of the infrared image eye image, thereby When both the first detection result and the second detection result indicate the presence of a living body, it is determined to pass the living body detection.
- the recognition result of the RGB image to compensate for the recognition result of the infrared image in the case of the reduction of the facial features due to the occlusion of the human face, and at the same time, combine the unoccluded eye parts in the RGB image and the infrared image
- the difference of infrared reflectance can quickly perform live body detection on the passing target person, which effectively improves the detection efficiency of live body detection, and also ensures the detection accuracy.
- FIG. 1 is a flowchart of living body recognition in a face occlusion scene in an embodiment of the disclosure
- FIG. 2A is a schematic diagram of an unobstructed target face in an embodiment of the disclosure.
- 2B is a schematic diagram of a target face that is blocked in an embodiment of the disclosure.
- 3A is a schematic diagram of comparison between RGB images and infrared images of real human eyes in an embodiment of the disclosure
- FIG. 3B is a schematic diagram of a comparison between an RGB image and an infrared image of a virtual human eye in an embodiment of the disclosure
- FIG. 4 is a schematic diagram of hole attack detection in an embodiment of the disclosure.
- FIG. 5 is a schematic diagram of the logical structure of a living body detection device in an embodiment of the disclosure.
- FIG. 6 is a schematic diagram of the physical structure of the living body detection device in an embodiment of the disclosure.
- dual cameras are used to photograph the occluded human face, and a red, green and blue (RGB) image and a red, green, and blue (RGB) image are obtained respectively.
- RGB red, green and blue
- RGB red, green, and blue
- the living body detection process applied to the scene occluded by the human face is specifically as follows:
- Step 101 Take an RGB image through a normal camera, and take an infrared image through an infrared camera.
- an ordinary camera can be used to collect an RGB image under normal illumination.
- a red camera can also be used to synchronously collect a grayscale image under infrared light.
- Step 102 Determine whether the target human face is not included in the RGB image and the infrared image, if yes, end the current process; otherwise, go to step 103.
- Step 103 Determine whether the RGB image contains a target face? If yes, go to step 105; otherwise, go to step 104.
- the infrared image must contain the target face. Therefore, only the RGB image needs to be judged. If there is no target face in the RGB image, and the infrared image If the target face is included, it is possible that the person has moved during the shooting process, so it is necessary to re-shoot the RGB image.
- steps 102 and 103 are set for the situation where the target face is not captured by accident. In actual applications, if a person must stand still and take a photo when passing by, there is no need to perform steps 102 and 103. This is just an example. ,No longer.
- Step 104 Re-shoot the RGB image through the ordinary camera.
- the face whose face area is below the preset area threshold can be regarded as a non-target face and deleted, so as to avoid the subsequent special face recognition process , It affects the accuracy of the recognition result of the target face.
- step 105 does not need to be performed. This is only an example, and will not be repeated.
- Step 106 Determine whether the Intersection-over-Union (IOU) area of the target face contained in the RGB image and the target face contained in the infrared image reaches a first set threshold? If yes, combine the current process; otherwise, go to step 107.
- IOU Intersection-over-Union
- step 106 may not be performed. This is only an example and will not be repeated.
- Step 107 Use the preset occlusion detection model to perform occlusion detection on the target face in the RGB image, determine the occlusion area of the occlusion object on the target face based on the occlusion detection result, and determine the occlusion area and the occlusion area. Is the ratio of the face area of the target face lower than the second set threshold? If yes, go to step 108, otherwise, end the current process.
- the obstruction detection model can be obtained based on deep neural network training. After the RGB image is input, the detection coordinate position of the obstruction, such as a detection frame, will be output.
- the target person does not wear a mask on the face, it will be processed according to the ordinary living body detection process, which will not be introduced too much in the embodiment of the present disclosure.
- the ratio of the area of the mask worn on the target person’s face to the face area exceeds the second set threshold, it means that the mask is unreasonably worn. You can end the current process and prompt the target person to re-adjust the mask before triggering the live detection again. Process.
- the ratio of the area of the mask worn on the target person's face to the face area does not exceed the second set threshold, it means that the mask is worn reasonably and the subsequent living body detection process can be performed.
- Step 108 Use a preset face feature recognition model to perform face recognition on the RGB image and the infrared image, respectively, to obtain corresponding RGB image face feature recognition results and infrared image face feature recognition results.
- an affine transformation method may be used to perform normalization processing on the RGB image and the infrared image to ensure the convenience of subsequent data processing, which will not be repeated here.
- the pre-trained facial feature recognition model can be used to recognize facial features in RGB images, and can also be used to recognize facial features in infrared images.
- the difference is that the RGB image contains With more pixels, the details are clearer, so more accurate facial features can be recognized.
- the person wears a mask because the person wears a mask, only half of the face can be accurately identified through the RGB image, for example, the brow bone, eye frame, and part of the face can be accurately identified The characteristics of the nose.
- the facial feature recognition results of RGB images still contain the full face features. Except for the above-mentioned accurately recognized features, the remaining features are all predictions, that is, the recognition results are not accurate, such as cheekbones, nose, and mouth. , Jaw, chin and other features are all predictive features.
- the infrared image contains fewer pixels and few details, it is not restricted by the mask occlusion. Therefore, through the infrared image, the full facial features of the face can still be recognized, that is, the infrared image facial feature recognition result , Also contains the full-face features of a human face.
- Step 109 Determine the position of the non-occluded object according to the detection result of the occluder, and filter out the facial feature recognition result of the partial RGB image corresponding to the position of the non-occluded object from the facial feature recognition result of the RGB image.
- Step 110 Use a preset living body detection model to perform living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result, to obtain a first detection result.
- the living body detection model is obtained after pre-training based on a deep neural network. Although there are only some facial features in the RGB image, it can play a very good auxiliary role for the living body detection process.
- the living body detection device encounters a photo attack or a screen attack, the other party will hold the face photo or hold the face screen.
- the details in the face are blurred, such as the brow bone, eye frame, and nasal bone. It is not clear, therefore, using some of the facial features in the RGB image to assist in living body detection can play a very good role.
- the first detection can be determined The result indicates the presence of a living body.
- the partial RGB image facial feature recognition result and the infrared image facial feature recognition result can be input into the above-mentioned living body detection model in a way of left and right stitching, top and bottom stitching, or simultaneous input.
- Step 111 Extract an RGB image eye image from the RGB image, and extract an infrared image eye image from the infrared image, based on the infrared reflectance of the RGB image eye image and the infrared reflectance of the infrared image eye image Perform a live test and obtain the second test result.
- the infrared reflectivity of the RGB image eye image can be compared with the infrared reflectance of the infrared image eye image to distinguish the virtual human eye.
- the infrared reflectance in the RGB image eye image is different from the infrared reflectance in the infrared image eye image, and has a significant difference.
- the infrared reflectance in the RGB image eye image is the same as the infrared reflectance in the infrared image eye image, and there is no significant difference.
- the second detection result represents the presence of a living body.
- the eye image of the RGB image and the eye image of the infrared image may be inputted into the above-mentioned living body detection model in a manner of left and right splicing, up and down splicing, or simultaneous input.
- Step 112 When it is determined that the first detection result and the second detection result both indicate the presence of a living body, it is determined that the living body detection is passed.
- step 111 in order to further prevent hole attacks, optionally, after step 111 is performed, the following steps may be further performed before step 112:
- An RGB image eye image is further extracted from the RGB image, the area of the RGB image eye image is larger than the area of the RGB image eye image; the infrared image eye image is further extracted from the infrared image, so The area of the eye image of the infrared image is larger than the area of the eye image of the infrared image, and further, a preset living body detection model is used, based on the texture recognition result of the spliced edge of the eye image of the RGB image, or/and, Based on the texture recognition result of the spliced edge of the infrared image eye image, the living body detection is performed, and the third detection result is obtained.
- the third detection result indicates that there is texture information of the splicing edge
- it is determined that there is a hole attack that is, any one of the eye image of the RGB image and the infrared image of the eye image indicates that the texture information of the splicing edge is present, then it is determined There is a hole attack.
- the third detection result indicates that there is no texture information of any spliced edge, it is determined that there is no hole attack, that is, there is a living body, that is, both the RGB image eye image and the infrared image eye image indicate that there is no texture information of the spliced edge. It is determined that there is no hole attack.
- the opponent when performing a hole attack, the opponent may hold a face photo, but have holes in the eyes to avoid the live detection of the photo attack and the screen attack described in step 111.
- the texture information of the face photo part and the texture information of the real face eye are not uniformly transitioned, and there must be a splicing edge. Therefore, by detecting the texture information of the splicing edge, you can know whether there is a hole attack.
- step 112 After the third detection result is obtained, when step 112 is performed, it is required that the first detection result, the second detection result, and the third detection result all characterize the existence of a living body, in order to determine that the person finally passes the living body detection.
- the voiceprint recognition technology can be further used to determine whether there is a living body before finally passing the living body detection.
- the audio data entered by the person corresponding to the target human face may be acquired, the voiceprint feature of the audio data may be extracted, and based on the voiceprint feature, it is determined that there is a living body.
- vocal control organs of living body include vocal cords, soft palate, tongue, teeth, lips, etc.; vocal resonators include pharyngeal cavity, oral cavity, and nasal cavity. Small differences in these organs will cause vocalization.
- the change of airflow causes the difference in sound quality and tone.
- the habit of sound production in the living body is also fast and slow, and the force varies greatly, and it also causes the difference in sound intensity and length.
- the audio data can be converted into electrical signals, and then drawn into corresponding spectral graphs through changes in the intensity, wavelength, frequency, and rhythm of the electrical signals, thereby forming a voiceprint diagram.
- the visual form of the live sound can be reflected.
- Features such as tone quality, tone color, tone intensity, and tone length.
- the above-mentioned voiceprint detection step may be before the first detection result is obtained; it may also be after the first detection result is obtained, but before the second detection result is obtained; it may also be after the second detection result is obtained, but before the second detection result is obtained. 3. Before the detection result; the specific execution timing is not limited, it only needs to be executed before the final determination of the existence of a living body, and I will not repeat it here.
- the living body detection device at least includes: a photographing unit 50, a first detection unit 51, a second detection unit 52, and a determination unit 54, wherein,
- the photographing unit 50 is used for photographing RGB images and infrared images for the target face, and adopts a preset facial feature recognition model to perform face recognition on the RGB image and the infrared image respectively to obtain the corresponding RGB Image facial feature recognition results and infrared image facial feature recognition results;
- the first detection unit 51 is configured to determine the position of a non-occluded object according to the detection result of the occluder of the RGB image, and filter out the RGB image face feature recognition result, the part of the RGB corresponding to the position of the non-occluded object Image facial feature recognition result, and using a preset living body detection model, performing living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result, to obtain a first detection result;
- the second detection unit 52 is configured to extract an RGB image eye image from the RGB image, and an infrared image eye image from the infrared image, using the living body detection model based on the infrared reflectivity of the RGB eye image Performing live detection with the infrared reflectance of the infrared image of the eye image to obtain a second detection result;
- the determining unit 54 is configured to determine that when both the first detection result and the second detection result indicate the presence of a living body, it is determined to pass the living body detection.
- the photographing unit 50 it is further used to perform any one or combination of the following operations:
- Performing obstruction detection on the RGB image determining the obstruction area of the obstruction on the target face in the RGB image, and the ratio of the face area of the target face to the second set threshold.
- a preset living body detection model is used to perform living body detection based on the partial RGB image facial feature recognition result and the infrared image facial feature recognition result, and when the first detection result is obtained, the first detection unit 51 is used for:
- the living body detection model is used to determine that the infrared image facial feature recognition result includes living body features, and when the number of facial features included in the partial RGB image facial feature recognition result reaches a preset number threshold , It is determined that the first detection result indicates the presence of a living body.
- the living body detection model is used to perform living body detection based on the infrared reflectance of the RGB eye image and the infrared reflectance of the infrared image eye image, and when the second detection result is obtained, the second detection unit 52 Used for:
- the second detection result represents the existence of a living body.
- the device further includes:
- the third detection unit 53 is configured to perform the following operations before determining that both the first detection result and the second detection result indicate the presence of a living body:
- the area of the RGB image eye image is larger than the area of the RGB image eye image
- the area of the infrared image eye image is larger than the area of the infrared image eye image
- Using a preset live detection model based on the texture recognition result of the spliced edge of the RGB image eye image, or/and, based on the texture recognition result of the spliced edge of the infrared image eye image, perform the live detection, and obtain the first Three detection results, wherein if the third detection result indicates that there is no texture information of the spliced edge, it is determined that there is a living body;
- the determination unit 54 is configured to:
- both the first detection result and the second detection result characterize the presence of a living body
- the third detection result also characterizes the presence of a living body
- it further includes:
- the voiceprint detection unit 55 is configured to obtain the audio data entered by the person corresponding to the target face before determining to pass the live body detection, extract the voiceprint feature of the audio data, and determine the presence of a live body based on the voiceprint feature .
- an embodiment of the present disclosure provides a living body detection device, including:
- the memory 60 is used to store executable instructions
- the processor 61 is configured to read and execute executable instructions stored in the memory to implement any one of the methods introduced in the foregoing embodiments.
- embodiments of the present application provide a computer-readable storage medium.
- instructions in the computer-readable storage medium are executed by a processor, any one of the methods introduced in the foregoing embodiments can be executed.
- dual cameras are used to photograph the occluded face, and an RGB image and an infrared image are obtained respectively, and then, the face in the unoccluded area of the face in the RGB image is combined.
- the feature recognition result and the facial feature recognition result in the infrared image are obtained to obtain the first detection result of the living body detection.
- the first detection result of the living body detection is obtained.
- the second detection result so that when the first detection result and the second detection result both indicate the presence of a living body, it is determined that the living body detection is passed.
- the recognition result of the RGB image to compensate for the recognition result of the infrared image in the case of the reduction of the facial features due to the occlusion of the human face, and at the same time, combine the unoccluded eye parts in the RGB image and the infrared image
- the difference of infrared reflectance can quickly perform live body detection on the passing target person, which effectively improves the detection efficiency of live body detection, and also ensures the detection accuracy.
- the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
- computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
- These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
- the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
- These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
- the instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Collating Specific Patterns (AREA)
Abstract
Description
Claims (14)
- 一种应用于人脸遮挡场景的活体检测方法,其特征在于,包括:针对目标人脸,拍摄红绿蓝RGB图像以及拍摄红外图像,并采用预设的人脸特征识别模型,分别对所述RGB图像和所述红外图像进行人脸识别,获得相应的RGB图像人脸特征识别结果和红外图像人脸特征识别结果;根据针对所述RGB图像的遮挡物检测结果确定非遮挡物位置,并从所述RGB图像人脸特征识别结果中筛选出,所述非遮挡物位置对应的部分RGB图像人脸特征识别结果,以及采用预设的活体检测模型,基于所述部分RGB图像人脸特征识别结果和所述红外图像人脸特征识别结果进行活体检测,获得第一检测结果;从所述RGB图像中提取RGB图像眼睛图像,以及从所述红外图像中提取红外图像眼睛图像,采用所述活体检测模型,基于所述RGB图像眼睛图像的红外线反射率和所述红外图像眼睛图像的红外线反射率进行活体检测,获得第二检测结果;判定所述第一检测结果和所述第二检测结果均表征存在活体时,确定通过活体检测。
- 如权利要求1所述的方法,其特征在于,针对目标人脸,拍摄RGB图像以及拍摄红外图像之后,采用预设的人脸特征识别模型,分别对所述RGB图像和所述红外图像进行人脸识别之前,进一步执行以下操作中的任意一种或组合:对所述RGB图像和所述红外图像进行扫描,确定所述RGB图像和所述红外图像均包含所述目标人脸;从所述RGB图像和所述红外图像中,删除非目标人脸,所述非目标人脸为人脸面积未达到预设的面积门限值的人脸;对所述RGB图像和所述红外图像进行扫描,确定所述RGB图像包含的目标人脸和所述红外图像包含的目标人脸的交并比IOU面积达到第一设定阈 值;对所述RGB图像进行遮挡物检测,确定所述RGB图像中的目标人脸上的遮挡物的遮挡面积,与所述目标人脸的人脸面积的比值于第二设定阈值。
- 如权利要求1所述的方法,其特征在于,采用预设的活体检测模型,基于所述部分RGB图像人脸特征识别结果和所述红外图像人脸特征识别结果进行活体检测,获得第一检测结果,包括:采用所述活体检测模型,判定所述红外图像人脸特征识别结果中包含有活体特征,以及所述部分RGB图像人脸特征识别结果包含的人脸特征的数目达到预设的数目门限值时,确定第一检测结果表征存在活体。
- 如权利要求1所述的方法,其特征在于,采用所述活体检测模型,基于所述RGB眼睛图像的红外线反射率和所述红外图像眼睛图像的红外线反射率进行活体检测,获得第二检测结果,包括:采用所述活体检测模型,判定所述RGB眼睛图像的红外线反射率和所述红外图像眼睛图像的红外线反射率不同时,确定所述第二检测结果表征存在活体。
- 如权利要求1-4任一项所述的方法,其特征在于,判定所述第一检测结果和所述第二检测结果均表征存在活体之前,进一步包括:从所述RGB图像中进一步提取出RGB图像眼部图像,所述RGB图像眼部图像的面积大于所述RGB图像眼睛图像的面积;从所述红外图像中进一步提取出红外图像眼部图像,所述红外图像眼部图像的面积大于所述红外图像眼睛图像的面积;采用预设的活体检测模型,基于所述RGB图像眼部图像的拼接边缘的纹理识别结果,或/和,基于所述红外图像眼部图像的拼接边缘的纹理识别结果,进行活体检测,获得第三检测结果,其中,若所述第三检测结果表征不存在任何拼接边缘的纹理信息,则确定存在活体;判定所述第一检测结果和所述第二检测结果均表征存在活体时,确定通过活体检测时,包括:判定所述第一检测结果和所述第二检测结果均表征存在活体,以及所述第三检测结果也表征存在活体时,确定通过活体检测。
- 如权利要求5所述的方法,其特征在于,在确定通过活体检测之前,进一步包括:获取所述目标人脸对应的人物录入的音频数据,提取所述音频数据的声纹特征,并基于所述声纹特征,确定存在活体。
- 一种应用于人脸遮挡场景的活体检测装置,其特征在于,包括:拍摄单元,用于针对目标人脸,拍摄红绿蓝RGB图像以及拍摄红外图像,并采用预设的人脸特征识别模型,分别对所述RGB图像和所述红外图像进行人脸识别,获得相应的RGB图像人脸特征识别结果和红外图像人脸特征识别结果;第一检测单元,用于根据针对所述RGB图像的遮挡物检测结果确定非遮挡物位置,并从所述RGB图像人脸特征识别结果中筛选出,所述非遮挡物位置对应的部分RGB图像人脸特征识别结果,以及采用预设的活体检测模型,基于所述部分RGB图像人脸特征识别结果和所述红外图像人脸特征识别结果进行活体检测,获得第一检测结果;第二检测单元,用于从所述RGB图像中提取RGB图像眼睛图像,以及从所述红外图像中提取红外图像眼睛图像,采用所述活体检测模型,基于所述RGB眼睛图像的红外线反射率和所述红外图像眼睛图像的红外线反射率进行活体检测,获得第二检测结果;判定单元,用于判定所述第一检测结果和所述第二检测结果均表征存在活体时,确定通过活体检测。
- 如权利要求7所述的装置,其特征在于,针对目标人脸,拍摄RGB图像以及拍摄红外图像之后,采用预设的人脸特征识别模型,分别对所述RGB图像和所述红外图像进行人脸识别之前,所述拍摄单元进一步用于执行以下操作中的任意一种或组合:对所述RGB图像和所述红外图像进行扫描,确定所述RGB图像和所述 红外图像均包含所述目标人脸;从所述RGB图像和所述红外图像中,删除非目标人脸,所述非目标人脸为人脸面积未达到预设的面积门限值的人脸;对所述RGB图像和所述红外图像进行扫描,确定所述RGB图像包含的目标人脸和所述红外图像包含的目标人脸的交并比IOU面积达到第一设定阈值;对所述RGB图像进行遮挡物检测,确定所述RGB图像中的目标人脸上的遮挡物的遮挡面积,与所述目标人脸的人脸面积的比值于第二设定阈值。
- 如权利要求7所述的装置,其特征在于,采用预设的活体检测模型,基于所述部分RGB图像人脸特征识别结果和所述红外图像人脸特征识别结果进行活体检测,获得第一检测结果时,所述第一检测单元用于:采用所述活体检测模型,判定所述红外图像人脸特征识别结果中包含有活体特征,以及所述部分RGB图像人脸特征识别结果包含的人脸特征的数目达到预设的数目门限值时,确定第一检测结果表征存在活体。
- 如权利要求7所述的装置,其特征在于,采用所述活体检测模型,基于所述RGB眼睛图像的红外线折射率和所述红外图像眼睛图像的红外线折射率进行活体检测,获得第二检测结果时,所述第二检测单元用于:采用所述活体检测模型,判定所述RGB眼睛图像的红外线折射率和所述红外图像眼睛图像的红外线折射率不同时,确定所述第二检测结果表征存在活体。
- 如权利要求7-10任一项所述的装置,其特征在于,进一步包括:第三检测单元,用于在判定所述第一检测结果和所述第二检测结果均表征存在活体之前,执行以下操作:从所述RGB图像中进一步提取出RGB图像眼部图像,所述RGB图像眼部图像的面积大于所述RGB图像眼睛图像的面积;从所述红外图像中进一步提取出红外图像眼部图像,所述红外图像眼部图像的面积大于所述红外图像眼睛图像的面积;采用预设的活体检测模型,基于所述RGB图像眼部图像的拼接边缘的纹理识别结果,或/和,基于所述红外图像眼部图像的拼接边缘的纹理识别结果,进行活体检测,获得第三检测结果,其中,若所述第三检测结果表征不存在任何拼接边缘的纹理信息,则确定存在活体;判定所述第一检测结果和所述第二检测结果均表征存在活体时,确定通过活体检测,所述判定单元用于:判定所述第一检测结果和所述第二检测结果均表征存在活体,以及所述第三检测结果也表征存在活体时,确定通过活体检测。
- 如权利要求11所述的装置,其特征在于,进一步包括:声纹检测单元,用于在确定通过活体检测之前,获取所述目标人脸对应的人物录入的音频数据,提取所述音频数据的声纹特征,并基于所述声纹特征,确定存在活体。
- 一种活体检测装置,其特征在于,包括:存储器,用于存储可执行指令;处理器,用于读取并执行存储器中存储的可执行指令,以实现如权利要求1至6中任一项所述的媒体数据加密处理的方法。
- 一种计算机可读存储介质,其特征在于,当所述计算机可读存储介质中的指令由处理器执行时,使得能够执行如权利要求1至6中任一项所述媒体数据加密处理的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010468727.8 | 2020-05-28 | ||
CN202010468727.8A CN111582238B (zh) | 2020-05-28 | 2020-05-28 | 一种应用于人脸遮挡场景的活体检测方法及装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021237968A1 true WO2021237968A1 (zh) | 2021-12-02 |
Family
ID=72114228
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/112222 WO2021237968A1 (zh) | 2020-05-28 | 2020-08-28 | 一种应用于人脸遮挡场景的活体检测方法及装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN111582238B (zh) |
WO (1) | WO2021237968A1 (zh) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111582238B (zh) * | 2020-05-28 | 2021-04-02 | 上海依图网络科技有限公司 | 一种应用于人脸遮挡场景的活体检测方法及装置 |
CN112115852A (zh) * | 2020-09-17 | 2020-12-22 | 广东光速智能设备有限公司 | 一种使用rgb红外相机的活体检测方法 |
CN112215113A (zh) * | 2020-09-30 | 2021-01-12 | 张成林 | 人脸识别方法及装置 |
CN112418865A (zh) * | 2020-11-13 | 2021-02-26 | 上海臻面智能信息科技有限公司 | 一种基于安全支付模组的活体检测方法 |
CN112329720A (zh) * | 2020-11-26 | 2021-02-05 | 杭州海康威视数字技术股份有限公司 | 人脸活体检测方法、装置及设备 |
TWI818824B (zh) * | 2022-12-07 | 2023-10-11 | 財團法人工業技術研究院 | 用於計算遮蔽人臉影像的人臉擺動方向的裝置及方法 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9594969B1 (en) * | 2015-11-19 | 2017-03-14 | Intel Corporation | Iris recognition including liveness testing |
CN108710843A (zh) * | 2018-05-14 | 2018-10-26 | 安徽质在智能科技有限公司 | 用于考勤的面部检测方法和装置 |
CN110516616A (zh) * | 2019-08-29 | 2019-11-29 | 河南中原大数据研究院有限公司 | 一种基于大规模rgb以及近红外数据集的双重认证人脸防伪方法 |
CN111046703A (zh) * | 2018-10-12 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | 人脸防伪检测方法、装置及多目相机 |
CN111079470A (zh) * | 2018-10-18 | 2020-04-28 | 杭州海康威视数字技术股份有限公司 | 人脸活体检测的方法和装置 |
CN111582238A (zh) * | 2020-05-28 | 2020-08-25 | 上海依图网络科技有限公司 | 一种应用于人脸遮挡场景的活体检测方法及装置 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1353292B1 (en) * | 2002-04-12 | 2011-10-26 | STMicroelectronics (Research & Development) Limited | Biometric sensor apparatus and methods |
US10027883B1 (en) * | 2014-06-18 | 2018-07-17 | Amazon Technologies, Inc. | Primary user selection for head tracking |
CN105243386B (zh) * | 2014-07-10 | 2019-02-05 | 汉王科技股份有限公司 | 人脸活体判断方法以及系统 |
CN106650666A (zh) * | 2016-12-26 | 2017-05-10 | 北京旷视科技有限公司 | 活体检测的方法及装置 |
CN108985134B (zh) * | 2017-06-01 | 2021-04-16 | 重庆中科云从科技有限公司 | 基于双目摄像机的人脸活体检测及刷脸交易方法及系统 |
CN110532957B (zh) * | 2019-08-30 | 2021-05-07 | 北京市商汤科技开发有限公司 | 人脸识别方法及装置、电子设备和存储介质 |
CN111079576B (zh) * | 2019-11-30 | 2023-07-28 | 腾讯科技(深圳)有限公司 | 活体检测方法、装置、设备及存储介质 |
-
2020
- 2020-05-28 CN CN202010468727.8A patent/CN111582238B/zh active Active
- 2020-08-28 WO PCT/CN2020/112222 patent/WO2021237968A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9594969B1 (en) * | 2015-11-19 | 2017-03-14 | Intel Corporation | Iris recognition including liveness testing |
CN108710843A (zh) * | 2018-05-14 | 2018-10-26 | 安徽质在智能科技有限公司 | 用于考勤的面部检测方法和装置 |
CN111046703A (zh) * | 2018-10-12 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | 人脸防伪检测方法、装置及多目相机 |
CN111079470A (zh) * | 2018-10-18 | 2020-04-28 | 杭州海康威视数字技术股份有限公司 | 人脸活体检测的方法和装置 |
CN110516616A (zh) * | 2019-08-29 | 2019-11-29 | 河南中原大数据研究院有限公司 | 一种基于大规模rgb以及近红外数据集的双重认证人脸防伪方法 |
CN111582238A (zh) * | 2020-05-28 | 2020-08-25 | 上海依图网络科技有限公司 | 一种应用于人脸遮挡场景的活体检测方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN111582238A (zh) | 2020-08-25 |
CN111582238B (zh) | 2021-04-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021237968A1 (zh) | 一种应用于人脸遮挡场景的活体检测方法及装置 | |
US20230334894A1 (en) | Detecting device, detecting method, and recording medium | |
WO2018049952A1 (zh) | 一种照片获取方法及装置 | |
CN105612533B (zh) | 活体检测方法、活体检测系统以及计算机程序产品 | |
WO2019127262A1 (zh) | 基于云端的人脸活体检测方法、电子设备和程序产品 | |
US20190080149A1 (en) | Occlusion detection for facial recognition processes | |
JP2004317699A (ja) | デジタルカメラ | |
JP2004320287A (ja) | デジタルカメラ | |
CN101390128B (zh) | 脸部器官的位置的检测方法及检测系统 | |
CN109508706B (zh) | 一种基于微表情识别和无感人脸识别的静默活体检测方法 | |
CN111382648A (zh) | 人脸动态表情的检测方法、装置、设备及存储介质 | |
JP2004320286A (ja) | デジタルカメラ | |
JP2007274207A (ja) | 画像表示装置、撮像装置および画像表示方法 | |
JP2022105583A (ja) | 顔生体検出方法、装置、電子機器、記憶媒体及びコンピュータプログラム | |
CN111353404A (zh) | 一种人脸识别方法、装置及设备 | |
JP2004320285A (ja) | デジタルカメラ | |
US20240104965A1 (en) | Face liveness detection methods and apparatuses | |
CN111860394A (zh) | 一种基于姿态估计和动作检测的动作活体识别方法 | |
CN111178233A (zh) | 基于活体认证的身份认证方法及装置 | |
TW201411564A (zh) | 瞌睡提醒裝置 | |
CN107862298B (zh) | 一种基于红外摄像装置下眨眼的活体检测方法 | |
CN112434647A (zh) | 一种人脸活体检测方法 | |
JP2007159610A (ja) | 登録装置、認証装置、登録認証装置、登録方法、認証方法、登録プログラムおよび認証プログラム | |
TWI466070B (zh) | 眼睛搜尋方法及使用該方法的眼睛狀態檢測裝置與眼睛搜尋裝置 | |
JP2004110813A (ja) | 人物認証装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20937774 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20937774 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06.07.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20937774 Country of ref document: EP Kind code of ref document: A1 |