CN117831105A - Living body detection method, living body detection device, electronic equipment and storage medium - Google Patents

Abstract

The application discloses a living body detection method, a living body detection device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: extracting a face area of the first visible light image to obtain a second visible light image; inputting the second visible light image into a first living body detection model to obtain a first detection result output by the first living body detection model; converting the second visible light image from the visible light domain to the infrared light domain to obtain an infrared light image corresponding to the second visible light image; inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model; and obtaining a living body detection result of the face of the object to be identified based on the first detection result and the second detection result. According to the method, the living body detection is carried out on the shot visible light image and the infrared light image obtained through light domain conversion, so that the effectiveness and reliability of living body detection can be improved, the attack of non-living bodies is prevented, and the safety of living body identification is guaranteed.

Description

Living body detection method, living body detection device, electronic equipment and storage medium

Technical Field

The application belongs to the technical field of image processing, and particularly relates to a living body detection method, a living body detection device, electronic equipment and a storage medium.

Background

Living detection is a technique for confirming that a person in a video is actually present and has a vital sign, and is recognized by analyzing living actions such as blinking, opening mouth, and turning head in the video.

Living body detection is widely applied in many scenes, particularly in the security sensitive fields of identity verification, face recognition, financial transaction and the like, however, the technology of living body detection through video has the problems of high detection failure rate and large occupied resources.

Disclosure of Invention

The present application aims to solve at least one of the technical problems existing in the prior art. Therefore, the application provides a living body detection method, a living body detection device, an electronic device and a storage medium, which can effectively carry out living body detection, and have high detection success rate and less occupied resources.

In a first aspect, the present application provides a method of in vivo detection, the method comprising:

acquiring a first visible light image of the face of an object to be identified;

extracting a face area of the first visible light image to obtain a second visible light image;

inputting the second visible light image into a first living body detection model to obtain a first detection result output by the first living body detection model;

Converting the second visible light image from a visible light domain to an infrared light domain to obtain an infrared light image corresponding to the second visible light image;

inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model;

obtaining a living body detection result of the face of the object to be identified based on the first detection result and the second detection result;

the first living body detection model is trained based on a visible light image sample set, and the second living body detection model is trained based on an infrared light image sample set.

According to the living body detection method, the living body detection is carried out on the shot visible light image and the infrared light image obtained through light domain conversion, so that the living body detection effectiveness and reliability can be improved, the non-living body attack can be prevented, the living body identification safety is guaranteed, the face area is extracted, the living body detection is carried out on the face, the interference of complex background can be reduced, the living body detection effect is improved, the detection success rate is high, the occupied resources are small, no additional hardware equipment for collecting the infrared light image is added, and the cost is reduced.

According to an embodiment of the present application, the converting the second visible light image from the visible light domain to the infrared light domain, to obtain an infrared light image corresponding to the second visible light image, includes:

and inputting the second visible light image into a countermeasure generation network, and converting the second visible light image from a visible light domain to an infrared light domain through the countermeasure generation network to obtain the infrared light image.

According to one embodiment of the application, the countermeasure generation network is constructed based on a loop generation network.

According to an embodiment of the present application, the extracting the face area from the first visible light image to obtain a second visible light image includes:

and carrying out background segmentation and face area extraction on the first visible light image based on a stable video matting algorithm to obtain the second visible light image.

According to one embodiment of the present application, the second living body detection model is used for detecting a living body face, a picture face and a mask, and the first living body detection model is used for detecting a living body face, a picture face and a mask.

According to one embodiment of the application, the visible light image sample set includes a plurality of visible light image samples, the infrared light image sample set includes a plurality of infrared light image samples, the plurality of infrared light image samples and the plurality of visible light image samples are in one-to-one correspondence, and the infrared light image samples are obtained by countermeasure generation based on the visible light image samples.

According to one embodiment of the present application, the acquiring a first visible light image of a face of an object to be identified includes:

obtaining a visible light image to be detected of the face of the object to be identified;

performing face detection on the visible light image to be detected, and taking the visible light image to be detected as the first visible light image under the condition that the visible light image to be detected comprises facial features;

and under the condition that the visible light image to be detected does not comprise facial features, re-acquiring the visible light image to be detected and performing face detection until the re-acquired visible light image to be detected is determined to comprise facial features, and taking the visible light image to be detected comprising the facial features as the first visible light image.

In a second aspect, the present application provides a living body detection apparatus, comprising:

the acquisition module is used for acquiring a first visible light image of the face of the object to be identified;

the first processing module is used for extracting the face area of the first visible light image to obtain a second visible light image;

the second processing module is used for inputting the second visible light image into the first living body detection model to obtain a first detection result output by the first living body detection model;

The third processing module is used for converting the second visible light image from the visible light domain to the infrared light domain to obtain an infrared light image corresponding to the second visible light image;

the fourth processing module is used for inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model;

a fifth processing module, configured to obtain a living body detection result of the face of the object to be identified based on the first detection result and the second detection result;

the first living body detection model is trained based on a visible light image sample set, and the second living body detection model is trained based on an infrared light image sample set.

According to the living body detection device, through carrying out living body detection to the visible light image of shooting and the infrared light image that the light domain conversion obtained, can improve living body detection's validity and reliability to prevent the attack of non-living body, ensure living body identification's security, through extracting facial region, carry out living body detection to the face, can reduce the interference of complicated background, promoted living body detection's effect, the detection success rate is high and occupy the resource less, does not increase the hardware equipment of extra collection infrared light image, reduce cost.

In a third aspect, the present application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the living detection method according to the first aspect described above when executing the computer program.

In a fourth aspect, the present application provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the living body detection method as described in the first aspect above.

In a fifth aspect, the present application provides a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to execute a program or instructions to implement the living body detection method according to the first aspect.

In a sixth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the living body detection method as described in the first aspect above.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic flow chart of a living body detection method according to an embodiment of the present application;

FIG. 2 is a flow chart of facial region extraction provided by an embodiment of the present application;

FIG. 3 is a second flow chart of a living body detection method according to an embodiment of the present disclosure;

FIG. 4 is a third flow chart of a living body detection method according to an embodiment of the present disclosure;

fig. 5 is a schematic structural view of a living body detection apparatus provided in an embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The living body detection method, the living body detection device, the electronic device and the readable storage medium provided by the embodiment of the application are described in detail below by specific embodiments and application scenes thereof with reference to the accompanying drawings.

The living body detection method can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet having a touch sensitive surface (e.g., a touch screen display and/or a touch pad). It should also be appreciated that in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).

In the following various embodiments, a terminal including a display and a touch sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

The execution body of the living body detection method provided by the embodiment of the application may be an electronic device or a functional module or a functional entity in the electronic device capable of implementing the living body detection method, and the electronic device mentioned in the embodiment of the application includes, but is not limited to, a mobile phone, a tablet computer, a camera, a wearable device, and the like.

As shown in fig. 1, the living body detection method includes: steps 110 to 160.

Step 110, a first visible light image of the face of the object to be identified is acquired.

The object to be identified is a subject in a living body detection scene, and may be a user, and the face of the object to be identified may be a live face of a living body, or may be a non-living body such as a photo, a video, a three-dimensional model, a mask, and the like.

For example, when a user uses a service in a financial field such as a bank, securities, insurance, etc., in an authentication scene, the user's face is photographed by using a mobile phone camera, so as to obtain a first visible light image.

For another example, in authentication scenes such as government service, enterprise attendance or border inspection of exit and entrance ports, a camera is used for shooting the face of the object to be identified, so as to obtain a first visible light image.

And 120, extracting a face area of the first visible light image to obtain a second visible light image.

Since the complex and changeable background easily causes deterioration of the living body detection effect in the face recognition, in this embodiment, background removal of the obtained first visible light image is required, and only the face area is extracted, facilitating living body detection.

It will be appreciated that the background in the second visible light image has been removed, leaving behind an image of the facial region in the first visible light image.

The face region extraction of the first visible light image to obtain a second visible light image can be realized through edge detection, color space conversion, segmentation algorithms and other image matting algorithms, can also be realized through segmentation and image matting methods based on deep learning such as convolutional neural networks (Convolutional Neural Network, CNN) and semantic segmentation networks (Semantic Segmentation Network), and can also be realized through segmentation and image matting methods based on cloud APIs.

And 130, inputting a second visible light image into a first living body detection model to obtain a first detection result output by the first living body detection model, wherein the first living body detection model is trained based on a visible light image sample set.

It can be appreciated that the set of visible light image samples includes a plurality of visible light image samples, and the first living body detection model is trained by using the set of visible light image samples, so that the first living body detection model has the capability of carrying out living body detection on the visible light images.

In actual execution, the second visible light image is input to the first living body detection model, and after the first living body detection model receives the second visible light image, reasoning and living body detection are carried out on the second visible light image, and a first detection result is generated and output.

And 140, converting the second visible light image from the visible light domain to the infrared light domain to obtain an infrared light image corresponding to the second visible light image.

The infrared light domain can be a near infrared light domain, and the infrared light image can be a near infrared light image, so that the near infrared light has a shorter wavelength, higher resolution can be provided, and more accurate detailed information of the face surface can be included, so that the recognition accuracy is improved.

It will be appreciated that since the infrared light image is converted from the second visible light image, the infrared light image includes image information of the face region and does not include image information of the background in the first visible light image.

In actual implementation, the second visible light image may be converted into a grayscale image, and a corresponding color mapping scheme may be selected according to the desired infrared effect.

For example, the red channel may be used to represent infrared information and the green and blue channels may remain in the original visible image.

Based on the color mapping scheme, the infrared information is superimposed on the second visible light image, and a corresponding infrared light image can be obtained.

And 150, inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model, wherein the second living body detection model is obtained by training based on an infrared light image sample set.

Wherein both the first and second live detection models may be loaded and run based on a deep learning framework such as TensorFlow, pyTorch.

It is understood that the infrared light image sample set includes a plurality of infrared light image samples, and the second living body detection model is trained by using the infrared light image sample set, so that the second living body detection model has the capability of carrying out living body detection on the infrared light images.

In actual execution, the infrared light image is input to a second living body detection model, and after the infrared light image is received, the second living body detection model carries out reasoning and living body detection on the infrared light image, and generates and outputs a second detection result.

Step 160, obtaining a living body detection result of the face of the object to be identified based on the first detection result and the second detection result.

The first detection result and the second detection result may be the predicted probability for each category, for example, in the first detection result, the probability of being predicted as a living body is a, the probability of being predicted as a non-living body is b, and in the second detection result, the probability of being predicted as a living body is c, the probability of being predicted as a non-living body is d.

The living body detection result may be determined according to a weighted sum of the prediction probabilities of each classification, for example, the first prediction result has a weight of x and the second prediction result has a weight of y, and the calculated probability predicted as living body is ax+cy and the calculated probability predicted as non-living body is bx+dy.

For example, if ax+cy > bx+dy, the living body detection result may be that the face of the object to be identified is a living body, and if ax+cy is equal to or less than bx+dy, the living body detection result may be that the face of the object to be identified is a non-living body.

For another example, the probability threshold R may be set, and the determination result of the living body may be that the face of the object to be identified is a living body in the case of ax+cy+.r, and the determination result of the living body may be that the face of the object to be identified is a non-living body in the case of ax+cy+.r.

In the related art, a visible light image and an infrared light image are respectively acquired by using a binocular camera, and then living body detection is respectively performed on the acquired visible light image and infrared light image.

On one hand, the related technology needs to use the binocular camera to perform image acquisition, so that the hardware cost of living body detection is improved, the use scene is also greatly limited, the difference of angles, light rays and the like of images shot by different cameras can influence the living body detection effect, and on the other hand, the related technology directly uses the acquired visible light image and infrared light image to perform living body detection and is greatly influenced by complex background.

According to the living body detection method provided by the embodiment of the application, the living body detection is carried out on the shot visible light image and the infrared light image obtained through light domain conversion, so that the living body detection effectiveness and reliability can be improved, the non-living body attack is prevented, the living body identification safety is guaranteed, the face area is extracted, the living body detection is carried out on the face, the interference of a complex background can be reduced, the living body detection effect is improved, the detection success rate is high, the occupied resources are small, no additional hardware equipment for collecting the infrared light image is added, and the cost is reduced.

In some embodiments, step 110, acquiring a first visible light image of the face of the object to be identified, includes:

obtaining a visible light image to be detected of the face of an object to be identified;

performing face detection on the visible light image to be detected, and taking the visible light image to be detected as a first visible light image under the condition that the visible light image to be detected comprises facial features;

and under the condition that the visible light image to be detected does not comprise the facial features, re-acquiring the visible light image to be detected and performing face detection until the re-acquired visible light image to be detected comprises the facial features, and taking the visible light image to be detected comprising the facial features as a first visible light image.

The to-be-detected visible light image may be obtained by performing image acquisition on the face of the to-be-identified object according to a certain frequency by using a mobile phone camera or a camera, and because the to-be-identified object may suddenly twist or move during image acquisition, the acquired to-be-detected visible light image does not contain the face of the to-be-identified image, or the face of the to-be-detected visible light image is blurred, so that the to-be-detected visible light image does not contain facial features.

It will be appreciated that the face detection of the visible light image to be detected may be achieved by a face detection algorithm which may detect facial features in the image and give corresponding position and bounding box information.

The Face detection algorithm may be implemented based on conventional computer vision methods, such as Haar cascade classifiers, HOG features+svms, etc., and deep learning-based methods, such as Multi-cascade convolutional neural networks (Multi-task Cascaded Convolutional Networks, MTCNN), single-shot Multi-frame detectors (Single Shot MultiBox Detector, SSD), retinal Face detection algorithms (Retina Face), center-point Face detectors (Center Face), etc.

In actual execution, after a to-be-detected visible light image of the face of the object to be identified is acquired, a face detection algorithm is called to carry out face detection on the to-be-detected visible light image so as to determine whether the to-be-detected visible light image contains facial features, and the to-be-detected visible light image is used as a first visible light image for face region extraction under the condition that the to-be-detected visible light image contains the facial features.

And under the condition that the visible light image to be detected does not comprise the facial features, re-acquiring the visible light image to be detected and calling a face detection algorithm to carry out face detection until the fact that the re-acquired visible light image to be detected comprises the facial features is determined, and taking the re-acquired visible light image to be detected containing the facial features as a first visible light image.

In this embodiment, the first visible light image determined to contain the facial features is obtained by performing face detection on the visible light image to be detected, which provides a basis for face region extraction and living body detection.

In some embodiments, step 120, performing facial region extraction on the first visible light image to obtain a second visible light image includes:

and carrying out background segmentation and face area extraction on the first visible light image based on a stable video matting algorithm to obtain a second visible light image.

In the first visible light image, the face of the object to be identified is used as a foreground entity, and the other faces are used as backgrounds.

As shown in fig. 2, a stable video matting (RVM) algorithm is called to build a background model, each pixel in the first visible light image 210 is classified, all pixels are divided into a foreground and a background, a face area in the first visible light image 210 is determined according to a result of background segmentation, and the face area is separated from the first visible light image 210 through clipping and masking operations according to position information of the face area, so as to obtain a face image as the second visible light image 220.

In this embodiment, the first visible light image 210 is extracted from the face region at the pixel level by using the stable video matting algorithm, so that the segmentation accuracy is high, the face region is extracted accurately, the processing speed is high, and the interference of the complex background on the living body detection is reduced.

In some embodiments, step 140 of converting the second visible light image 220 from the visible light domain to the infrared light domain, to obtain an infrared light image corresponding to the second visible light image 220 includes:

the second visible light image 220 is input to the countermeasure generation network, and the second visible light image 220 is converted from the visible light domain to the infrared light domain through the countermeasure generation network, resulting in an infrared light image.

In some embodiments, the countermeasure generation network is constructed based on a loop generation network.

The countermeasure generation network may be trained on a cyclic generation network (cyclic-Consistent Generative Adversarial Network, cyclic gan) by a plurality of first visible light image 210 samples and a plurality of first infrared light image samples corresponding to the plurality of first visible light image 210 samples.

It should be noted that, the binocular camera includes a visible light camera and a near infrared light camera, and after image acquisition is performed on the faces of the plurality of sample objects by using the binocular camera, a first visible light image 210 sample and a corresponding first infrared light image sample can be obtained at the same time.

The CycleGAN includes a first generator, a second generator, a first arbiter, and a second arbiter.

The first generator for mapping of visible light to infrared light and the second generator for mapping of infrared light to visible light may employ an encoder-decoder architecture, wherein the encoder encodes the input image into a latent feature representation and the decoder decodes the latent feature representation into an image of the target light domain.

The first discriminator is used for discriminating between the generated infrared light image and the input infrared light image, the second discriminator is used for discriminating between the generated visible light image and the input visible light image, and the first discriminator and the second discriminator can adopt a Markov discriminator (PatchGAN) structure for discriminating the image part.

A contrast loss for making the generated image look true in the target domain, a cyclical consistency loss for maintaining consistency of the image conversion, and an identity loss for ensuring that the generator can retain information of the input image can be introduced in the CycleGAN.

The CycleGAN should be able to restore the incoming visible light image back by inputting the infrared light image generated by the first generator into the second generator again to ensure the bi-directionality and consistency of the mapping.

Likewise, the visible light image generated by the second generator is again input to the first generator, and the input infrared light image should also be restored back.

The cyclical consistency loss may help the first generator and the second generator learn the corresponding mapping.

In the process of performing the countermeasure training by the first generator, the second generator, the first discriminator and the second discriminator, the first generator attempts to generate a realistic infrared light image to fool the first discriminator, the second generator attempts to generate a realistic visible light image to fool the second discriminator, and the first discriminator and the second discriminator attempt to accurately judge the distinction between the generated image and the real image, by which the first generator and the second generator gradually improve the quality of the generated infrared light image and the visible light image.

The challenge-generating network may be trained based on the following steps:

acquiring a plurality of first visible light image 210 samples acquired by a binocular camera on the faces of a plurality of sample objects, and a plurality of first infrared light image samples corresponding to the plurality of first visible light image 210 samples;

invoking RVM algorithm to extract facial regions of the plurality of first visible light image 210 samples to obtain a plurality of second visible light image 220 samples, and invoking RVM algorithm to extract facial regions of the plurality of first infrared light image samples to obtain a plurality of second infrared light image samples;

inputting a plurality of second visible light image 220 samples and a plurality of second infrared light image samples to a countermeasure generator network;

converting the plurality of second visible light image 220 samples from the visible light domain to the infrared light domain through the first generator to obtain a plurality of anti-infrared light images, and converting the plurality of second infrared light image samples from the infrared light domain to the visible light domain through the second generator to obtain a plurality of anti-visible light images;

performing authenticity judgment on a plurality of anti-infrared light images through a first discriminator to obtain the anti-loss and the cycle consistency loss of a first generator;

performing authenticity judgment on the multiple countermeasure visible light images through a second discriminator to obtain countermeasure loss and cycle consistency loss of a second generator;

The weights of the first generator and the second generator are updated according to the countermeasures and the loop consistency losses to minimize a total loss of the generators, which may be a sum of the countermeasures and the loop consistency losses.

The weights of the first and second discriminators are updated to minimize the loss of the discriminators, including the discrimination loss for the real images in the visible and infrared light domains and the discrimination loss for the corresponding generated images.

And repeatedly updating weights of the first generator, the second generator, the first discriminator and the second discriminator until the preset iteration times or loss convergence is reached, so as to obtain the trained countermeasure generation network.

By alternately training the generator and the arbiter, the CycleGAN can learn the mapping relationship between the visible light domain and the infrared light domain, and can convert images with high quality.

After training is completed, the resulting countermeasure generation network may convert the second visible light image 220 from the visible light domain to the infrared light domain to enable generation of the infrared light image.

In this embodiment, the second visible light image 220 is converted from the visible light domain to the infrared light domain through the trained countermeasure generation network, so as to obtain the infrared light image, and no external hardware equipment is needed, so that the acquisition cost of the infrared light image can be effectively reduced, a higher-quality infrared light image is obtained, and the target recognition and detection capability can be enhanced.

In some embodiments, the set of visible light image samples includes a plurality of visible light image samples, the set of infrared light image samples includes a plurality of infrared light image samples, the plurality of infrared light image samples are in one-to-one correspondence with the plurality of visible light image samples, and the infrared light image samples are generated by antagonism based on the visible light image samples.

In actual execution, face image acquisition is performed on a plurality of sample objects, a RVM algorithm is invoked to extract face areas, a plurality of visible light image samples are obtained, the plurality of visible light image samples are input into a trained countermeasure generation network, the countermeasure generation network converts the plurality of visible light image samples from a visible light domain to an infrared light domain, and a plurality of infrared light image samples can be obtained.

In this embodiment, by obtaining an infrared light image sample corresponding to the visible light image sample that is generated against, the field characteristics of the visible light region and the infrared light region can be effectively retained.

In some embodiments, the second living detection model is for detecting a living face, a picture face, and a mask, and the first living detection model is for detecting a living face, a picture face, and a mask.

Wherein the first living body detection model and the second living body detection model can be three-classification models.

It is understood that the visible light image sample set includes a plurality of visible light image samples, and sample labels corresponding to the plurality of visible light image samples, the sample labels including a living body face label, a picture face label, and a mask label.

And converting the plurality of second visible light image 220 samples from the visible light domain to the infrared light domain to obtain a plurality of infrared light image samples, and constructing an infrared light image sample set based on the anti-infrared light image samples and sample labels corresponding to the anti-infrared light image samples.

In actual execution, training a first living body detection model through a visible light image sample set to obtain a trained first living body detection model, and training a second living body detection model through an infrared light image sample set to obtain a trained second living body detection model.

It should be noted that the first detection result and the second detection result may be probabilities that the face of the object to be detected is a living face, a picture face, and a mask, respectively.

In the embodiment, the living body detection and classification are carried out through the first living body detection model and the second living body detection model, so that the subsequent processing is convenient, the safety of living body detection is effectively improved, and the living body detection method is suitable for multiple application scenes.

In the related art, whether the face is a living body is detected by training the visible light living body detection model and the infrared light living body detection model, however, when the background of recognition is complicated, the living body detection of a real person is difficult to recognize as a real person, and there is a possibility of erroneous judgment.

According to the embodiment provided by the application, the face area is extracted from the image, the background is removed, and then the living body algorithm training is carried out, so that the recognition rate of living body detection is improved.

A specific embodiment is described below.

As shown in fig. 3, in the training phase for the first living body detection model and the second living body detection model, image acquisition is performed on the faces of a plurality of sample objects, so as to obtain a plurality of visible light images.

And (3) invoking a face detection algorithm to detect the facial features of the visible light image, if the facial features are not included, finishing acquisition, and if the facial features are included, invoking a RVM algorithm to extract the facial regions of the visible light image, so as to realize face matting and obtain a plurality of visible light image samples.

And inputting the plurality of visible light image samples into the countermeasure generation network to obtain a plurality of infrared light image samples corresponding to the plurality of visible light image samples output by the countermeasure generation network.

The method comprises the steps of dividing a plurality of collected visible light image samples into three types, namely a living body face, a picture face and a mask, and correspondingly marking the living body face, the picture face and the mask to form a visible light image sample set, and equally dividing a plurality of infrared light image samples into the living body face, the picture face and the mask to correspondingly mark the living body face, the picture face and the mask to form an infrared light image sample set.

The first living body detection model is trained through the visible light image sample set, a three-classification model can be obtained, and the second living body detection model is trained through the infrared light image sample set, and another three-classification model can be obtained.

As shown in fig. 4, a visible light image to be detected of the face of the object to be identified is acquired, a face detection algorithm is invoked to perform face detection on the visible light image to be detected to determine whether the visible light image to be detected contains facial features, and the visible light image to be detected is used as a first visible light image 210 for face extraction when the visible light image to be detected is determined to include the facial features; in the case that it is determined that the visible light image to be detected does not include the facial feature, the visible light image to be detected is re-acquired and a face detection algorithm is invoked to perform face detection until it is determined that the re-acquired visible light image to be detected includes the facial feature, and the re-acquired visible light image to be detected including the facial feature is taken as the first visible light image 210.

The RVM algorithm is invoked to extract the facial region of the first visible image 210, resulting in a second visible image 220.

In one aspect, the second visible light image 220 is input to a first living body detection model, the first living body detection model performs reasoning and living body detection on the second visible light image 220 after receiving the second visible light image 220, and the first living body detection model generates and outputs a first detection result.

On the other hand, the infrared light image is input to a second living body detection model, the second living body detection model performs reasoning and living body detection on the infrared light image after receiving the infrared light image, and the second living body detection model generates and outputs a second detection result.

And performing weighted calculation on the first detection result and the second detection result to obtain a living body detection result of the face of the object to be identified, wherein the living body detection result is used for judging whether the face of the object to be identified is a living body or not.

In this embodiment, the visible light camera is used to collect the face, and the first visible light image 210 is used to extract the face area, so as to reduce the interference of the background, improve the recognition accuracy of the face living body detection under the complex background environment, and solve the problem that the face living body detection recognition effect is reduced because of the complex background.

According to the living body detection method provided by the embodiment of the application, the execution body can be a living body detection device. In the embodiment of the present application, a living body detection device provided in the embodiment of the present application is described by taking a living body detection method performed by the living body detection device as an example.

The embodiment of the application also provides a living body detection device.

As shown in fig. 5, the living body detection apparatus includes: an acquisition module 510, a first processing module 520, a second processing module 530, a third processing module 540, a fourth processing module 550, and a fifth processing module 560.

An acquisition module 510, configured to acquire a first visible light image 210 of a face of an object to be identified;

the first processing module 520 is configured to extract a face area of the first visible light image 210 to obtain a second visible light image 220;

the second processing module 530 is configured to input the second visible light image 220 to the first living body detection model, and obtain a first detection result output by the first living body detection model;

the third processing module 540 is configured to convert the second visible light image 220 from the visible light domain to the infrared light domain, so as to obtain an infrared light image corresponding to the second visible light image 220;

a fourth processing module 550, configured to input the infrared light image to the second living body detection model, so as to obtain a second detection result output by the second living body detection model;

A fifth processing module 560, configured to obtain a living body detection result of the face of the object to be identified based on the first detection result and the second detection result; the first living body detection model is trained based on a visible light image sample set, and the second living body detection model is trained based on an infrared light image sample set.

According to the living body detection device provided by the embodiment of the application, through carrying out living body detection on the shot visible light image and the infrared light image obtained through light domain conversion, the living body detection effectiveness and reliability can be improved, so that the attack of non-living bodies is prevented, the living body identification safety is guaranteed, the face area is extracted, the living body detection is carried out on the face, the interference of complex background can be reduced, the living body detection effect is improved, the detection success rate is high, the occupied resources are small, the hardware equipment for additionally collecting the infrared light image is not added, and the cost is reduced.

In some embodiments, the third processing module 540 is further to:

the second visible light image 220 is input to the countermeasure generation network, and the second visible light image 220 is converted from the visible light domain to the infrared light domain through the countermeasure generation network, resulting in an infrared light image.

In some embodiments, the countermeasure generation network is constructed based on a loop generation network.

In some embodiments, the first processing module 520 is further to:

based on the stable video matting algorithm, background segmentation and face region extraction are performed on the first visible light image 210, and a second visible light image 220 is obtained.

In some embodiments, the second living detection model is for detecting a living face, a picture face, and a mask, and the first living detection model is for detecting a living face, a picture face, and a mask.

In some embodiments, the set of visible light image samples includes a plurality of visible light image samples, the set of infrared light image samples includes a plurality of infrared light image samples, the plurality of infrared light image samples are in one-to-one correspondence with the plurality of visible light image samples, and the infrared light image samples are generated by antagonism based on the visible light image samples.

In some embodiments, the obtaining module 510 is further configured to:

obtaining a visible light image to be detected of the face of an object to be identified;

performing face detection on the visible light image to be detected, and taking the visible light image to be detected as a first visible light image 210 under the condition that the visible light image to be detected is determined to comprise facial features;

In the case where it is determined that the visible light image to be measured does not include the facial feature, the visible light image to be measured is re-acquired and face detection is performed until it is determined that the re-acquired visible light image to be measured includes the facial feature, and the visible light image to be measured including the facial feature is taken as the first visible light image 210.

The living body detection device in the embodiment of the application can be an electronic device, and also can be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.

The living body detection apparatus in the embodiment of the present application may be an apparatus having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The living body detection device provided in the embodiment of the present application can implement each process implemented in the living body detection method embodiments of fig. 1 to 4, and in order to avoid repetition, a detailed description is omitted here.

In some embodiments, as shown in fig. 6, the embodiment of the present application further provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and capable of running on the processor 601, where the program when executed by the processor 601 implements the processes of the living body detection method embodiment described above, and the same technical effects can be achieved, and for avoiding repetition, a description is omitted herein.

The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.

The embodiment of the present application further provides a non-transitory computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements each process of the above embodiment of the living body detection method, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as computer readable memory ROM, random access memory RAM, magnetic or optical disks, and the like.

The embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements the above-described living detection method.

The processor is a processor in the electronic device in the above embodiment. Readable storage media include computer readable storage media such as computer readable memory ROM, random access memory RAM, magnetic or optical disks, and the like.

The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or instructions, the processes of the embodiment of the living body detection method can be realized, the same technical effects can be achieved, and in order to avoid repetition, the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods of the embodiments of the present application.

In the description of the present application, the meaning of "plurality" is two or more.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.

In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

Claims (10)

1. A living body detecting method, characterized by comprising:

acquiring a first visible light image of the face of an object to be identified;

extracting a face area of the first visible light image to obtain a second visible light image;

Inputting the second visible light image into a first living body detection model to obtain a first detection result output by the first living body detection model;

converting the second visible light image from a visible light domain to an infrared light domain to obtain an infrared light image corresponding to the second visible light image;

inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model;

obtaining a living body detection result of the face of the object to be identified based on the first detection result and the second detection result;

the first living body detection model is trained based on a visible light image sample set, and the second living body detection model is trained based on an infrared light image sample set.

2. The living body detection method according to claim 1, wherein the converting the second visible light image from the visible light domain to the infrared light domain to obtain an infrared light image corresponding to the second visible light image includes:

and inputting the second visible light image into a countermeasure generation network, and converting the second visible light image from a visible light domain to an infrared light domain through the countermeasure generation network to obtain the infrared light image.

3. The in-vivo detection method according to claim 2, wherein said challenge-generation network is constructed based on a loop-generation network.

4. The living body detection method according to claim 1, wherein the performing face area extraction on the first visible light image to obtain a second visible light image includes:

and carrying out background segmentation and face area extraction on the first visible light image based on a stable video matting algorithm to obtain the second visible light image.

5. The living body detection method according to claim 1, wherein the second living body detection model is for detecting a living body face, a picture face, and a mask, and the first living body detection model is for detecting a living body face, a picture face, and a mask.

6. The living body detection method according to any one of claims 1 to 5, wherein the visible light image sample set includes a plurality of visible light image samples, the infrared light image sample set includes a plurality of infrared light image samples, the plurality of infrared light image samples are in one-to-one correspondence with the plurality of visible light image samples, and the infrared light image samples are obtained by countermeasure generation based on the visible light image samples.

7. The living body detection method according to any one of claims 1 to 5, wherein the acquiring a first visible light image of the face of the object to be identified includes:

obtaining a visible light image to be detected of the face of the object to be identified;

performing face detection on the visible light image to be detected, and taking the visible light image to be detected as the first visible light image under the condition that the visible light image to be detected comprises facial features;

and under the condition that the visible light image to be detected does not comprise facial features, re-acquiring the visible light image to be detected and performing face detection until the re-acquired visible light image to be detected is determined to comprise facial features, and taking the visible light image to be detected comprising the facial features as the first visible light image.

8. A living body detecting device, characterized by comprising:

the acquisition module is used for acquiring a first visible light image of the face of the object to be identified;

the first processing module is used for extracting the face area of the first visible light image to obtain a second visible light image;

the second processing module is used for inputting the second visible light image into the first living body detection model to obtain a first detection result output by the first living body detection model;

The third processing module is used for converting the second visible light image from the visible light domain to the infrared light domain to obtain an infrared light image corresponding to the second visible light image;

the fourth processing module is used for inputting the infrared light image into a second living body detection model to obtain a second detection result output by the second living body detection model;

a fifth processing module, configured to obtain a living body detection result of the face of the object to be identified based on the first detection result and the second detection result;

the first living body detection model is trained based on a visible light image sample set, and the second living body detection model is trained based on an infrared light image sample set.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the living detection method according to any of claims 1-7 when executing the program.

10. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the living detection method according to any of claims 1-7.