WO2020124994A1

WO2020124994A1 - Liveness detection method and apparatus, electronic device, and storage medium

Info

Publication number: WO2020124994A1
Application number: PCT/CN2019/095081
Authority: WO
Inventors: 王旭
Original assignee: 北京字节跳动网络技术有限公司
Priority date: 2018-12-18
Filing date: 2019-07-08
Publication date: 2020-06-25
Also published as: CN109684974A

Abstract

A liveness detection method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining an image frame in a video in real time upon determining that a liveness detection starting condition is met (S110); recognizing at least one organ area of a user in the image frame, wherein each organ area corresponds to at least one feature value set, and each feature value set comprises at least one feature value (S120); determining an image frame in which the at least one organ area meets a condition of being uncovered as a target image frame, calculating a feature value matching each organ area recognized in the target image frame, updating and storing the calculated feature value to a feature value set corresponding to the organ area to which the feature value belongs (S140); and performing detection of at least one target action condition on the feature value set in real time, and when it is determined that a liveness detection ending condition is met, determining that the user is a live person if the detection result of the at least one target action condition is "verification succeeds" (S160).

Description

Living body detection method, device, electronic equipment and storage medium

This application requires the priority of the Chinese patent application with the application number 201811549518.5 submitted to the China Patent Office on December 18, 2018. The entire content of this application is incorporated by reference in this application.

Technical field

Embodiments of the present disclosure relate to image processing technology, for example, to a living body detection method, device, electronic device, and storage medium.

Background technique

People are often required to perform identity authentication in various activities to ensure the security of information. With the development of computers and networks, the popularity of Internet electronic devices has made ID authentication work more efficient. Usually, ID authentication needs to determine that the subject is a living body.

In the financial system and face recognition system, it is often necessary to perform in vivo judgment. In this process, the user is generally required to complete basic facial movements, for example, blinking movements. Related technologies determine whether a user is a living user by detecting changes in posture parameters of face parts (for example, eyes or mouth, etc.) in multiple photos including the user's face image, and use face key point positioning and human Face tracking and other technologies to verify whether the user is operating himself.

But the above method is easy to be attacked by photos. For example, by closing the eyes of the legal user, and quickly placing a black pen on the user's eyes, and quickly leaving, to simulate the closed eyes of the legal user, resulting in the user's photo Being misjudged as a living body, the related art methods cannot guarantee the accuracy of the living body detection and reduce the security of identity authentication.

Summary of the invention

Embodiments of the present disclosure provide a living body detection method, device, electronic equipment, and storage medium, which can accurately identify a living body and improve the security of identity authentication.

In an embodiment, an embodiment of the present disclosure provides a living body detection method, which includes:

When it is determined that the living condition detection start condition is satisfied, the image frames in the video are obtained in real time;

Identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value;

An image frame in which the at least one organ region satisfies the unoccluded condition is determined as a target image frame, and feature values matching each organ region identified in the target image frame are calculated respectively, and the calculated feature The value update is stored in the feature value set corresponding to the organ region to which the feature value belongs;

At least one target action condition is detected on the feature value set in real time, and when it is determined that the living body detection end condition is satisfied, the detection results of the at least one target action condition are all verified to determine that the user is a living body.

In an embodiment, an embodiment of the present disclosure also provides a living body detection device, which includes:

The image frame acquisition module is set to acquire the image frames in the video in real time when it is determined that the living body detection start condition is satisfied;

An organ region identification module, configured to identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value;

The feature value set update module is configured to determine the image frames in which the at least one organ region satisfies the unoccluded condition as the target image frame, and calculate feature values matching each organ region identified in the target image frame, And update and store the calculated feature value to the feature value set corresponding to the organ region to which the feature value belongs;

The living body detection module is configured to perform real-time detection of the at least one target action condition on the feature value set, and when it is determined that the living body detection end condition is satisfied, the detection results based on the at least one target action condition are all verified and determined The user is a living body.

In an embodiment, an embodiment of the present disclosure also provides an electronic device, the electronic device includes:

One or more processors;

Memory, set to store one or more programs;

The one or more programs are executed by the one or more processors, so that the one or more processors implement the method described in the foregoing embodiment.

In an embodiment, an embodiment of the present disclosure further provides a computer-readable storage medium that stores a computer program on the computer-readable storage medium, and the computer program is executed by a processor to implement the method described in the foregoing embodiment.

BRIEF DESCRIPTION

FIG. 1 is a flowchart of a living body detection method provided in Embodiment 1 of the present disclosure;

2 is a flowchart of a living body detection method provided in Embodiment 2 of the present disclosure;

3 is a schematic structural diagram of a living body detection device provided in Embodiment 3 of the present disclosure;

4 is a schematic structural diagram of an electronic device according to Embodiment 4 of the present disclosure.

detailed description

The disclosure will be described below with reference to the drawings and embodiments. The embodiments described herein are only for explaining the present disclosure, but not for limiting the present disclosure. For ease of description, only some parts but not all structures related to the present disclosure are shown in the drawings.

Example one

FIG. 1 is a flowchart of a living body detection method according to Embodiment 1 of the present disclosure. This embodiment can be applied to detect whether a user in a recorded video is a living body. The method can be performed by a living body detection device. It can be implemented by software and/or hardware, and the device can be configured in an electronic device, such as a computer. As shown in FIG. 1, the method includes S110 to S160.

S110, when it is determined that the living body detection start condition is satisfied, the image frames in the video are acquired in real time.

The living body detection start condition may refer to a condition for determining to start performing the living body detection operation. Exemplarily, when the living body detection start instruction is received, it is determined that the living body detection start condition is satisfied.

Generally speaking, the video is formed by a series of static image frames continuously displayed at a very fast speed. Thus, the video can be split into a series of image frames, and each image frame can be used as an image. In the embodiment of the present disclosure, the image frame is an image including a user's face image. When performing living body detection, a video including the user's face is recorded, so as to obtain information about the user's face according to the image frames in the video.

Living body detection is usually a detection process performed in real time. Therefore, the video is a video being recorded in real time, and each image frame recorded in the video can be acquired and processed in real time, thereby ensuring the timeliness of living body detection.

S120. Identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value.

The organ area may refer to the area where the organ in the user's face is located. For example, the organ area includes at least one of the following: an eye area, a mouth area, and a nose area.

Usually in live detection, it is necessary to determine whether the face pose captured in the video conforms to the pose characteristics of the live body. In one embodiment, face pose action detection can be achieved through one or more key points in the face, and based on Face gesture detection detects whether the user is alive. The key points may refer to key points in the human face that have an identification function. For example, the key points may include key points of the left eyeball, key points of the mouth, key points of the nostril, key points of the brow and tail, and key points of the contour of the face. The key points can be identified by a pre-trained machine learning model, or by other methods, such as Active Shape Model (ASM)-based methods, which are not specifically limited in the embodiments of the present disclosure.

According to the identified key points in each organ region, the characteristic value of each organ region can be calculated. Optionally, if the organ region is an eye region, the characteristic value of the organ region includes the closed value of the eye region ; If the organ area is a mouth area, the characteristic value of the organ area includes the opening and closing of the mouth area; if the organ area is a nose area, the characteristic value of the organ area includes a pitch angle of the nose area And/or rotation angle. In addition, the feature value of the nose area may also include the yaw angle. The feature value may be determined according to the relative positions between multiple key points. For example, the closing value of the eye area may be the distance between the highest key point of the eyeball and the lowest keypoint of the eyeball. In addition, a corresponding machine learning model can be trained for each feature value to calculate the feature value, or a method for calculating the feature value according to the key point can be determined according to requirements, and this embodiment of the present disclosure does not specifically limit this.

The feature values of each organ region in the calculated image frame are correspondingly added to at least one feature value set matching the organ region to which the feature value belongs, and the feature value set corresponding to each organ region is updated in real time. Each feature value set can store a fixed number of feature values, but the fixed number of feature values in the feature value set may be the same or different. In an embodiment, each feature value set may be regarded as a queue, redundant feature values are deleted at the head of the queue, and new feature values are inserted at the end of the queue. The storage space of each queue may be the same or different.

In an embodiment, the feature value of the eye area is a closed value, and the corresponding feature value set is a closed value set; the feature value of the mouth area is a contraction degree, and the corresponding feature value set is a contraction degree set; the feature value of the nose area is The pitch angle, rotation angle and yaw angle respectively correspond to the set of characteristic values as pitch angle set, rotation angle set and yaw angle set. Exemplarily, the closed value set stores up to 5 feature values, the pitch angle set stores up to 15 feature values, and the rotation angle set stores up to 15 feature values.

S130. Determine whether each organ region identified in the image frame satisfies the unoccluded condition until all image frames in the video acquired in real time are completed. If yes, execute S140; otherwise, execute S150.

The unoccluded condition refers to the condition that the organ region identified in the image frame is not covered to block other items. Occlusion judgment is performed on the acquired image frame in real time. After the judgment of the current image frame acquired in real time is completed, if the image frame is not acquired again, the judgment is ended, and when the image frame is acquired again, the occlusion judgment of the image frame is continued.

Optionally, determining that the at least one organ region satisfies the unoccluded condition as the target image frame may include: inputting the image frame into a pre-trained occlusion judgment network model that matches each organ region And obtain the output of the occlusion judgment network model, for each organ region in the image frame, the occlusion judgment result; if the occlusion judgment result for each organ region is unoccluded, determine the at least one organ region All meet the unoccluded condition. When each of the organ regions identified by the image frame satisfies the unoccluded condition, the image frame is determined to be the target image frame.

Among them, the occlusion judgment network model may be a machine learning model. In an embodiment, for each organ region, an occlusion image corresponding to the organ region is obtained as a training sample, and an occlusion judgment network model matching the organ region is trained. The image frame is input into the occlusion judgment network model matching the organ region, and the occlusion judgment result matching the organ region in the image frame output by the model can be obtained. In a case where the matching judgment result of all the organ regions identified in the image frame is unoccluded, it is determined that each organ region identified in the image frame satisfies the unoccluded condition, and the image frame is determined as the target image frame.

The occlusion judgment network model realizes the occlusion judgment of each organ region in the image frame, which can improve the accuracy of the occlusion judgment of each organ region, so that the living body detection is performed when each organ region is unoccluded, thereby improving the living body detection Accuracy.

S140, using the image frame as a target image frame, respectively calculating a feature value matching each organ region identified in the target image frame, and updating and storing the calculated feature value to the feature value In the feature value set corresponding to the organ region to which it belongs, S160 is executed.

In an embodiment, each set of feature values is updated every time an image frame judged by occlusion is acquired. With the real-time video recording, the image frames are continuously acquired in real time, so that the feature value set is continuously updated, making the liveness detection time-sensitive.

S150. Acquire the next image frame in real time.

S160: Perform at least one target action condition detection on the feature value set in real time, and when it is determined that the living body detection end condition is satisfied, the detection results of the at least one target action condition are all verified to determine that the user is a living body .

Among them, the target action condition may refer to a condition that defines the action for detecting the captured user as a living body. Exemplarily, the target action condition may include at least one of the following: open-eye and close-eye action, open-mouth and close-mouth action, nodding action, and shaking head action.

The biopsy end condition may refer to a condition for limiting the end of the target action condition detection. Exemplarily, when the target action condition detection time ends, the target action condition detection is successful, or the biopsy stop instruction is received, it is determined that the biopsy detection is satisfied End condition. The test results include verification passed and verification failed.

Usually, the user is prompted on the display screen of the electronic device to make an action that matches the target action condition within a set time. If the user is detected to make the action, continue to the next target action condition detection; if the user is not detected to make the action within the set time, determine that the detection result of the target action condition is verification failure, and continue to the next target action Condition detection. When the detection of all target action conditions is completed, the living body detection is ended. When the detection result of each target action condition is verified, that is, it is detected that the user has made all the target actions defined by at least one target action condition, and the user is determined to pass the live detection, That is, the user is a living body.

Among them, at least one target action condition can be detected according to the change of each feature value in the feature value set. For example, the target action condition is an action of opening and closing eyes, which can be stored in the closed value set matching the eye area. The closing value determines the maximum and minimum closing values of the eyes. At the same time, it is determined that the change from the maximum closing value to the minimum closing value or the maximum closing value to the minimum closing value is a monotonous change. The detection result of the action of opening and closing the eyes is verified.

Optionally, the detecting in real time at least one target action condition on the feature value set includes: if it is determined that the at least one feature value set matching the target action condition satisfies the target action condition, determining the target The test result of the operating condition is verified.

In an embodiment, the feature value set that needs to be detected can be determined according to the target action condition currently being detected, to avoid detecting all feature value sets at the same time, reduce the amount of detected data, and improve the efficiency of target action detection.

In an embodiment, the living body detection includes at least one of the following: opening and closing eyes, opening and closing mouths, nodding, and shaking heads. Generally, only one action can be detected based on one set of feature values. When the illegal person uses the photo or video of the legitimate user to simulate the motion in the biopsy, the timing of the motion detection in the biopsy cannot be determined. Therefore, when the biopsy prompts to detect the target action, the unrelated actions that are not related to the target action are displayed. This extraneous motion is not a natural continuous motion of the user (for example blinking).

For example, in any motion detection process, blinking motions will continue to appear.

In another example, when performing a biopsy, the user will only perform one action that matches the biopsy according to the biopsy prompt. The living body detection prompts the user to make a nodding motion, but detects that the user makes an opening and closing motion.

Therefore, in addition to the feature value set that matches the specified action of the living body detection, additional multiple feature value sets need to be detected. For example, when a reasonable action other than the specified action is detected, the target action condition is verified. Pass; for example, when an unreasonable action other than the specified action is detected, it is determined that the user is not a living body. It can be understood that multiple target value sets need to be detected in the target action condition to determine whether the user has made the target action, and at the same time determine whether the user has made unreasonable actions. When the set of characteristic values matching the unreasonable action condition in the target action condition determines that the unreasonable action condition is satisfied, it is determined that the detection result of the target action condition is verification failure.

By detecting the target action condition based on the set of feature values matching the target action condition, you can avoid detecting all the feature value sets at the same time, reduce the amount of data detected, improve the efficiency of target action detection, and at the same time, avoid detecting the target action At the same time, it also detects the unreasonable action through the living body detection, thereby improving the accuracy of the living body detection.

Optionally, the determining that the at least one feature value set matching the target action condition satisfies the target action condition includes: if the maximum feature value and the minimum feature in each feature value set matching the target action condition The value satisfies the maximum value condition matching each feature value set, and at least one feature value between the maximum feature value, the minimum feature value, and the maximum feature value and the minimum feature value satisfies the monotonic change condition, Then, it is determined that at least one feature value set matching the target action condition meets the target action condition.

Among them, the maximum value condition refers to a condition that limits the maximum value threshold and the minimum value threshold of the feature value set. The maximum value threshold and the minimum value threshold defined in the maximum value condition of different feature value sets may be different or the same. When the maximum feature value determined by the feature value set is greater than the maximum value threshold, and the minimum feature value is less than the minimum value threshold, it is determined that the feature value set satisfies the maximum value condition.

The monotonic change condition may refer to a condition in which the change in the size of continuous feature values within a set range is monotonic. The maximum eigenvalue, minimum eigenvalue, and at least one eigenvalue between the maximum eigenvalue and the minimum eigenvalue satisfy the monotonic change condition, which means that the size change of the eigenvalue in the range from the maximum eigenvalue to the minimum eigenvalue is a monotonically decreasing or minimum feature The change in the size of the feature value from the value to the largest feature value is monotonously increasing.

According to the maximum value condition and the monotonic change condition, it is determined that there is a maximum feature value and a minimum feature value in the feature value set, and the maximum feature value and the minimum feature value are continuously monotonously changing, and it is determined that the user has made a continuous and standard-compliant target action. Exemplarily, in the action of opening and closing the eyes, the maximum feature value is the feature value when the eyes are opened and the distance between the upper and lower eyelids is the largest, the minimum feature value is the feature value when the eyes are closed and the distance between the upper and lower eyelids is the smallest, and the maximum feature value is The size change between the minimum feature values is that the distance between the upper and lower eyelids decreases monotonously, and it is determined that the closed value set satisfies the action of opening and closing the eyes.

By simultaneously satisfying the maximum value condition and the monotonous change condition, it is determined that the target action condition is satisfied, to ensure that the user has made the action prompted by the target action condition, and at the same time to ensure that the user is a continuous action, thereby achieving accurate detection of the target action condition.

In the embodiments of the present disclosure, the image frames in the video recording the user’s face image are obtained in real time, and for each image frame, the occlusion judgment of the organ region is performed first, and when all the organ regions meet the unoccluded condition, at least one is updated according to real-time A set of feature values corresponding to the organ region, and at least one target action condition is detected according to the updated set of feature values, and whether the user is alive is determined based on the detection result of the at least one target action condition, which solves the occlusion photo method used in the related art Through the problem of living body detection, the false judgment of living body detection is reduced, and the accuracy rate of living body detection is improved, thereby improving the security of identity authentication.

Example 2

FIG. 2 is a flowchart of a living body detection method according to Embodiment 2 of the present disclosure.

The method of this embodiment may include S210 to S2120.

S210, when it is determined that the living body detection start condition is satisfied, the image frames in the video are acquired in real time.

The living body detection, video, image frame, organ area, feature value, feature value set, unoccluded condition, living body detection end condition and detection result in this embodiment can all refer to the description in the above embodiment.

S220. Identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value, wherein the organ region includes at least one of the following : Eye area, mouth area and nose area; the feature value set corresponding to the eye area includes the eye closure value set; the feature value set corresponding to the mouth area includes the mouth opening and closing degree set; the feature value corresponding to the nose area The set includes a set of pitch angles and/or a set of rotation angles.

S230, input the image frame into a pre-trained occlusion judgment network model matching each organ region, and obtain the occlusion judgment result for each organ region in the image frame output by the occlusion judgment network model .

S240: In a case where the result of the occlusion judgment of each organ region is unoccluded, determine that the at least one organ region satisfies the unoccluded condition.

S250. Determine whether each organ region identified in the image frame satisfies the unoccluded condition until the image frames in the video acquired in real time are all determined. If yes, execute S260; otherwise, execute S270.

S260, using the image frame as a target image frame, respectively calculating feature values matching each organ region identified in the target image frame, and updating and storing the calculated feature values to the feature values In the feature value set corresponding to the organ region to which it belongs, S280 is executed.

S270, acquiring the next image frame in real time. Return to execute S250.

S280, determine the currently detected target action condition from the at least one target action condition, and determine in real time whether at least one feature value set matching the currently detected target action condition meets the target action condition, and if so, execute S290 ; Otherwise, execute S2100.

Optionally, the determining that the at least one feature value set matching the target action condition meets the target action condition may include: if the maximum feature value and the minimum value in each feature value set matching the target action condition The feature value satisfies the maximum value condition matching each set of feature values, and at least one feature value between the maximum feature value, the minimum feature value, and the maximum feature value and the minimum feature value satisfies the monotonic change condition , It is determined that at least one feature value set matching the target action condition meets the target action condition.

In an embodiment, if at least one feature value set matching the target action condition does not satisfy the target action condition, then continue to determine whether the target action condition is met according to the at least one feature value set updated in real time. If the detection time of the target operating condition ends, at least one feature value set matching the target operating condition does not always satisfy the target operating condition, and it is determined that the detection result of the target operating condition is a failure.

S290: Determine that the detection result of the target action condition is verified.

S2100. Determine whether the time end condition corresponding to the currently detected target action condition is satisfied. If yes, execute S2110; otherwise, execute S280.

The time end condition may refer to a condition that limits the detection duration of the currently detected target operating condition. If the duration of the detection corresponding to the currently detected target action condition, the user has never made a facial action gesture defined by the target action condition, it is determined that the detection result of the currently detected target action condition is failure. If the image frame acquired in real time at the current time is still within the detection duration corresponding to the currently detected target action condition, continue to perform the current detection of the target action condition according to the multiple feature value sets updated corresponding to the next image frame acquired in real time Detection.

S2110. Determine that the detection result of the target action condition is verification failure.

S2120. When it is determined that the living body detection end condition is satisfied, determine whether the user is a living body according to the detection result of the at least one target action condition.

Example Three

FIG. 3 is a schematic structural diagram of a living body detection device according to an embodiment of the present disclosure. This embodiment can be applied to detect whether a user in a recorded video is a living body. The device may be implemented in software and/or hardware, and the device may be configured in an electronic device. As shown in FIG. 3, the apparatus may include: an image frame acquisition module 310, an organ region identification module 320, a feature value set update module 330, and a living body detection module 340.

The image frame acquisition module 310 is set to acquire the image frames in the video in real time when it is determined that the living body detection start condition is satisfied;

The organ region identification module 320 is configured to identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value;

The feature value set update module 330 is configured to determine the image frames in which the at least one organ region satisfies the unoccluded condition as the target image frame, and then separately calculate the features matching each organ region identified in the target image frame Value, and update and store the calculated feature value to the feature value set corresponding to the organ region to which the feature value belongs;

The living body detection module 340 is configured to detect at least one target action condition on the feature value set in real time, and when it is determined that the living body detection end condition is satisfied, the detection results according to the at least one target action condition are all verified and determined The user is a living body.

In the embodiment of the present disclosure, the image frames in the video recording the face image of the user are acquired in real time, and the occlusion judgment of the organ region is performed for each image frame first, and when all the organ regions meet the unoccluded condition, at least one organ region is updated in real time Corresponding feature value set, and detecting at least one target action condition according to the updated feature value set, and judging whether the user is a living body according to the detection result of the at least one target action condition, which solves the related art of using the occlusion photo method to pass the living body The problem of detection reduces the misjudgment of live detection, improves the accuracy of live detection, and thus improves the security of identity authentication.

In an embodiment, the feature value set updating module 330 includes: an occlusion judgment module configured to input the image frame into a pre-trained occlusion judgment network model that matches each organ region, and obtain the The output of the occlusion judgment network model is the occlusion judgment result of each organ region in the image frame; the unocclusion condition judgment module is set to determine that the occlusion judgment result of each organ region is unoccluded. The at least one organ region satisfies the unoccluded condition; the target image frame determination module is configured to determine the image frame when it is recognized that the at least one organ region in the image frame satisfies the unoccluded condition Is the target image frame.

In an embodiment, the living body detection module 340 includes a target action condition detection module configured to determine the target action if it is determined that at least one feature value set matching the target action condition satisfies the target action condition The test result of the condition is verified.

In an embodiment, the target action condition detection module includes: if the maximum feature value and the minimum feature value in each feature value set matching the target action condition satisfy the maximum value matching each feature value set Conditions, and at least one feature value between the maximum feature value, the minimum feature value, and the maximum feature value and the minimum feature value satisfies the monotonic change condition, then determine at least one feature that matches the target action condition The set of values meets the target action condition.

In an embodiment, the organ region includes an eye region, and the characteristic value of the organ region includes a closed value of the eye region.

In an embodiment, the organ region includes a mouth region, and the characteristic value of the organ region includes a degree of opening and closing of the mouth region.

In an embodiment, the organ area includes a nose area, and the characteristic value of the organ area includes a pitch angle and/or a rotation angle of the nose area.

The living body detection device provided by the embodiment of the present disclosure belongs to the same concept as the living body detection method provided by the embodiment one. For technical details not described in detail in the embodiments of the present disclosure, please refer to embodiment one, and the embodiment and the embodiment one of the present disclosure Has the same effect.

Example 4

An embodiment of the present disclosure provides an electronic device. Referring to FIG. 4, a schematic structural diagram of an electronic device (eg, terminal device or server) 400 suitable for implementing the embodiment of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, for example, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant (PDA), tablet computers (Portable Android Device, PAD), portable multimedia players (Personal Multimedia Player, PMP), mobile terminals such as in-vehicle terminals (for example, car navigation terminals), and fixed terminals such as digital television (TV), desktop computers, and so on. The electronic device shown in FIG. 4 is only an example, and should not bring limitations on the functions and use scope of the embodiments of the present disclosure.

As shown in FIG. 4, the electronic device 400 may include a processing device (such as a central processing unit, a graphics processor, etc.) 401. The processing device 401 may be based on a program stored in a read-only memory (Read-only Memory, ROM) 402 or from The storage device 408 loads a program in a random access memory (Random Access Memory, RAM) 403 to perform various appropriate actions and processes. In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input/output (Input/Output, I/O) interface 405 is also connected to the bus 404.

Generally, the following devices can be connected to the I/O interface 405: including input devices 406 such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (Liquid Crystal Display, LCD) , An output device 407 of a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to perform wireless or wired communication with other devices to exchange data. Although FIG. 4 shows an electronic device 400 including various devices, it should be understood that it is not required to implement or have all the devices shown. The present disclosure may alternatively be implemented or equipped with more or fewer devices.

According to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 409, or from the storage device 408, or from the ROM 402. When the computer program is executed by the processing device 401, the above-described functions defined in the living body detection method of the embodiment of the present disclosure are executed.

Example 5

Embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or a combination of the two. The computer-readable storage medium may include, for example, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or a combination of the above. More specific examples of computer-readable storage media may include: electrical connection with one or more wires, portable computer disk, hard disk, RAM, ROM, erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM ) Or flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or a suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. This propagated data signal can take many forms, including electromagnetic signals, optical signals, or suitable combinations of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device . The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: electric wires, optical cables, radio frequency (RF), etc., or a suitable combination of the foregoing.

The computer-readable medium may be included in the electronic device; or it may exist alone without being assembled into the electronic device.

The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: when it is determined that the living condition detection start condition is satisfied, acquire the image frames in the video in real time; At least one organ region of the user is identified in the image frame, and each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value; an image frame in which at least one organ region satisfies the unoccluded condition It is determined that the target image frame respectively calculates the feature value matching each organ region identified in the target image frame, and updates and stores the calculated feature value to the feature value corresponding to the organ region to which the feature value belongs In the set; at least one target action condition is detected on the feature value set in real time, and when it is determined that the in-vivo detection end condition is satisfied, the detection results based on the at least one target action condition are all verified and determined that the user is Living body.

The present disclosure may write computer program code for performing the operations of the present disclosure in one or more programming languages or a combination thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include Conventional procedural programming language-such as "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN) or a wide area network (Wide Area Network, WAN), or it can be connected to an external computer ( For example, using an Internet service provider to connect through the Internet).

The flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, functions, and operations of systems, methods, and computer program products according to one or more embodiments of the present disclosure. Each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing a prescribed logical function. In some alternative implementations, the functions marked in the blocks may also occur in an order different from that marked in the drawings. For example, two blocks represented in succession may actually be executed in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. Each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented with dedicated hardware-based systems that perform specified functions or operations, or can be implemented with dedicated hardware It is implemented in combination with computer instructions.

The modules described in the embodiments of the present disclosure may be implemented in software or hardware. Among them, the name of the module does not constitute a limitation on the module itself under certain circumstances. For example, the image frame acquisition module can also be described as "a module that acquires image frames in a video in real time when it is determined that the living condition detection start condition is satisfied. ".

Claims

A living body detection method, including:

When it is determined that the living condition detection start condition is satisfied, the image frames in the video are obtained in real time;

Identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value;

An image frame in which the at least one organ region satisfies the unoccluded condition is determined as a target image frame, and feature values matching each organ region identified in the target image frame are calculated respectively, and the calculated feature The value update is stored in the feature value set corresponding to the organ region to which the feature value belongs;

At least one target action condition is detected on the feature value set in real time, and when it is determined that the living body detection end condition is satisfied, the detection results of the at least one target action condition are all verified to determine that the user is a living body.
The method according to claim 1, wherein the determining of the image frames in a case where all of the at least one organ region satisfies the unoccluded condition as the target image frame includes:

Input the image frame into a pre-trained occlusion judgment network model matching each organ region, and obtain the occlusion judgment result for each organ region in the image frame output by the occlusion judgment network model;

In the case that the result of the occlusion determination of each organ region is unoccluded, it is determined that the at least one organ region satisfies the unoccluded condition;

When it is recognized that all the at least one organ region in the image frame satisfies the unoccluded condition, the image frame is determined to be the target image frame.
The method according to claim 1, wherein the detecting in real time at least one target action condition on the feature value set includes:

In response to determining that at least one feature value set matching the target action condition satisfies the target action condition, it is determined that the detection result of the target action condition is verified.
The method according to claim 3, wherein the determination that at least one feature value set matching the target action condition satisfies the target action condition includes:

In response to the maximum feature value and the minimum feature value in each feature value set matching the target action condition satisfying the maximum value condition matching each feature value set, and the maximum feature value, the minimum feature value and At least one feature value between the maximum feature value and the minimum feature value satisfies a monotonic change condition, and it is determined that at least one feature value set matching the target action condition satisfies the target action condition.
The method according to claim 1, wherein the organ region includes an eye region, and the characteristic value of the organ region includes a closed value of the eye region.
The method according to claim 1, wherein the organ region includes a mouth region, and the characteristic value of the organ region includes a degree of opening and closing of the mouth region.
The method according to claim 1, wherein the organ area includes a nose area, and the characteristic value of the organ area includes at least one of the following: a pitch angle of the nose area and a rotation angle of the nose area.
A living body detection device, including:

The image frame acquisition module is set to acquire the image frames in the video in real time when it is determined that the living body detection start condition is satisfied;

An organ region identification module, configured to identify at least one organ region of the user in the image frame, each organ region corresponds to at least one feature value set, and each feature value set includes at least one feature value;

The feature value set update module is configured to determine the image frames in which the at least one organ region satisfies the unoccluded condition as the target image frame, and calculate feature values matching each organ region identified in the target image frame, And update and store the calculated feature value to the feature value set corresponding to the organ region to which the feature value belongs;

The living body detection module is configured to perform real-time detection of the at least one target action condition on the feature value set, and when it is determined that the living body detection end condition is satisfied, the detection results based on the at least one target action condition are all verified and determined The user is a living body.
The apparatus according to claim 8, wherein the feature value set update module includes:

The occlusion judgment module is configured to input the image frame into a pre-trained occlusion judgment network model matching each organ region, and obtain the output of the occlusion judgment network model for each organ region in the image frame Occlusion judgment results;

An unoccluded condition judgment module, configured to determine that the at least one organ area satisfies the unoccluded condition when the result of the occlusion judgment of each organ area is unoccluded;

The target image frame determination module is configured to determine that the image frame is the target image frame when it is recognized that the at least one organ region in the image frame satisfies the unoccluded condition.
The device according to claim 8, wherein the living body detection module comprises:

The target action condition detection module is configured to, in response to determining that at least one feature value set matching the target action condition satisfies the target action condition, determine that the detection result of the target action condition is verified.
The apparatus according to claim 10, wherein the target action condition detection module comprises:

In response to the maximum feature value and the minimum feature value in each feature value set matching the target action condition satisfying the maximum value condition matching each feature value set, and the maximum feature value, the minimum feature value and At least one feature value between the maximum feature value and the minimum feature value satisfies a monotonic change condition, and it is determined that at least one feature value set matching the target action condition satisfies the target action condition.
The apparatus according to claim 8, wherein the organ region includes an eye region, and the characteristic value of the organ region includes a closed value of the eye region.
The apparatus according to claim 8, wherein the organ region includes a mouth region, and the characteristic value of the organ region includes a degree of opening and closing of the mouth region.
The apparatus according to claim 8, wherein the organ area includes a nose area, and the characteristic value of the organ area includes at least one of the following: a pitch angle of the nose area, and a rotation angle of the nose area.
An electronic device, including:

One or more processors;

Memory, set to store one or more programs;

The one or more programs are executed by the one or more processors, so that the one or more processors implement the method according to any one of claims 1-7.
A computer-readable storage medium storing a computer program on the computer-readable storage medium, the computer program being executed by a processor to implement the method according to any one of claims 1-7.