WO2021196831A1

WO2021196831A1 - Data verification method based on video information, device, and storage medium

Info

Publication number: WO2021196831A1
Application number: PCT/CN2021/071987
Authority: WO
Inventors: 李伟; 赵之砚
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-03-30
Filing date: 2021-01-15
Publication date: 2021-10-07
Also published as: CN111553189A

Abstract

A data verification method based on video information, a device, and a storage medium, relating to the technical field of voice processing. The method comprises: collecting data to be verified, said data comprising first video information containing an action subject (S110); respectively performing real person identification and first identity verification on the action subject in said data according to the first video information, so as to determine whether the action subject is a real person and whether the action subject is consistent with a pre-stored identification photograph (S120); if the action subject is a real person and is consistent with the pre-stored identification photograph, pushing a verification problem to the action subject, and collecting second video information when the action subject answers the verification problem (S130); and respectively performing second identity verification and micro-expression analysis on the action subject according to the second video information, so as to determine whether the action subject is consistent with the action subject in the first video and whether the action subject has a spoofing behavior in the process of answering the verification problem (S140). The method determines, by using an image and voice recognition technology, whether an answerer has a spoofing behavior when answering the Health Notice.

Description

Data verification method, device and storage medium based on video information

This application claims the priority of the Chinese patent filed with the Chinese Patent Office on March 30, 2020, the application number is 202010236490.0, and the invention title is "Data verification method, device and storage medium based on video information", the entire content of which is incorporated by reference In this application.

Technical field

This application relates to the field of video processing technology, and in particular to a data verification method, device and storage medium based on video information.

Background technique

At present, the main filling mode of the health notice is: operate in the APP or official account or H5 link, and manually select the answer items for the content in the health notice.

The insured (or the insured) will be required to truthfully fill in the health notice when purchasing insurance. Insured persons who are not insured with a healthy body may have to pay more premiums or be refused insurance. Therefore, in the actual operation process, there will be the following situations in which the health notice is not truthfully filled in:

1. The insured deliberately concealed the medical history and failed to fill in the health notification form truthfully;

2. Due to sloppyness, the insured does not check the health notification carefully, or just glances at it casually, always feels that he (or the insured) is in good health, and fills it out at will;

3. In order to complete the insured index, the insurance broker fills in on behalf of the insured person.

The inventor realized that for the underwriters unable to judge the sub-insurance, the insurance policy may be "postponed" by the insurance company to review the relevant information of the customer, re-fill the notice, and re-review, which reduces the work efficiency of the insurance company.

Therefore, there is an urgent need for a data information verification method to prevent misfilling the health notification form.

Summary of the invention

In view of the above problems, the purpose of this application is to provide a data verification method, device and storage medium based on video information. It can be used in the entire process of responding to the health notice, using image and voice detection and recognition technology to prevent malicious insurance from fraudulent insurance, and pre-position the underwriting work, which can save a lot of manpower and material resources to do inspections, underwriting, etc. The economic cost and time cost brought about.

According to one aspect of this application, a data verification method based on video information is provided, which includes the following steps:

S110: Collect data to be verified, where the data to be verified includes first video information including an acting subject;

S120: According to the first video information, perform real person identification and first identity verification on the actors in the data to be verified to determine whether the actors are real people and whether they are consistent with the pre-stored ID photos ; If the subject is a real person and is consistent with the pre-stored ID photo, then go to S130; if the subject is not a real person and/or is inconsistent with the pre-stored ID photo, then go to S110 ；

S130: Push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

S140: According to the second video information, perform second identity verification and micro-expression analysis on the actor respectively to determine whether the actor is consistent with the actor in the first video, and determine the behavior Whether the subject has cheated in the process of answering the verification question; if the subject is inconsistent with the subject in the first video, stop pushing new verification questions to the subject; if the subject is If the actors in the first video are the same, but the micro-expression analyzes that the actors have deceptive behaviors in the process of answering the verification question, the verification question is marked as an abnormal question.

According to another aspect of the present application, a data verification system based on video information is provided, including:

The first video acquisition unit is configured to collect data to be verified, where the data to be verified includes first video information including an action subject;

The real person recognition and first identity verification unit is configured to perform real person recognition and first identity verification on the actors in the data to be verified according to the first video information to determine whether the actors are real people , And whether it is consistent with the pre-stored ID photo; if the actor is a real person and is consistent with the pre-stored ID photo, then the second video collection and verification question push is performed; if the actor is not a real person , And/or is inconsistent with the pre-stored ID photo, then restart the first video capture unit;

A verification question push and second video acquisition unit, configured to push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

The deceptive behavior determination unit is configured to perform second identity verification and micro-expression analysis on the behavior subject according to the second video information to determine whether the behavior subject is consistent with the behavior subject in the first video, And determine whether the actor has cheated in the process of answering the verification question; if the actor is inconsistent with the actor in the first video, stop pushing new verification questions to the actor; if The actor is consistent with the actor in the first video, but the micro-expression analyzes that the actor has cheating in the process of answering the verification question, and the verification question is marked as an abnormal question.

According to another aspect of the present application, there is provided an electronic device, comprising: a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the following steps are implemented: S110: collecting data to be verified , The data to be verified includes the first video information including the behavior subject;

According to another aspect of the present application, there is provided a computer-readable storage medium in which a data verification program based on video information is stored, and when the data verification program based on video information is executed by a processor , The following steps are implemented: S110: Collect data to be verified, where the data to be verified includes the first video information including the behavior subject;

Using the above-mentioned video information-based data verification method, device, and storage medium according to this application, it integrates AI technology capabilities such as face recognition, face comparison, voice recognition, semantic understanding, smart recording, and micro-expressions to change the previous health notification Operation process, using video method to verify and fill in the data of the health notice. It has the following advantages: 1. To prevent others from answering under false names, and to ensure that I answer by the real person; 2. Use micro-expression anti-fraud technology to shut out false answers; 3. The whole health notification response is recorded in the whole process and stored for a long time for possible future generation Provide a basis for disputes; 4. Every health notification question will be broadcast to the insured (or the insured) to reduce the insured (or the insured)’s wrong answers due to carelessness and carelessness.

In order to achieve the above and related objects, one or more aspects of the present application include the features that will be described in detail later and specifically pointed out in the claims. The following description and drawings illustrate some exemplary aspects of the present application in detail. However, these aspects indicate only some of the various ways in which the principles of this application can be used. Furthermore, this application is intended to include all these aspects and their equivalents.

Description of the drawings

By referring to the following description in conjunction with the accompanying drawings and the content of the claims, and with a more comprehensive understanding of the application, other purposes and results of the application will be more clear and easier to understand. In the attached picture:

FIG. 1 is a flowchart of a data verification method based on video information according to Embodiment 1 of the present application;

2 is a schematic diagram of the logical structure of a data verification system based on video information according to Embodiment 2 of the present application;

FIG. 3 is a schematic diagram of a logical structure of an electronic device according to Embodiment 3 of the present application.

The same reference numerals in all the drawings indicate similar or corresponding features or functions.

Detailed ways

In the following description, for illustrative purposes, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is obvious that these embodiments can also be implemented without these specific details. In other examples, for the convenience of describing one or more embodiments, well-known structures and devices are shown in the form of block diagrams.

Glossary:

Silent multi-frame live detection model: Based on the latest deep convolutional neural network, combined with hundreds of millions of real face data and non-real face data training. Non-real face pictures have certain characteristics, such as moiré, picture reflections, distortions, abnormal backgrounds, etc. The silent multi-frame live detection model detects multiple pictures from the video to determine whether it is taken by a real person.

RPPG heartbeat detection: Remote Photoplethysmography (RPPG) uses the reflected ambient light to measure the subtle brightness changes of the skin. The subtle changes in skin brightness are caused by the blood flow caused by the beating heart. Generally, we can get a signal similar to BVP (blood volume pulse) through RPPG, and the heart rate can be predicted through this signal.

FFmpeg: FFmpeg is an open source computer program that can be used to record, convert digital audio and video, and convert them into streams. It provides a complete solution for recording, converting, and streaming audio and video.

The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Example 1

Fig. 1 is a flowchart of a data verification method based on video information according to Embodiment 1 of the present application.

As shown in Figure 1, the method for data verification based on video information provided in this embodiment includes the following steps:

S110: Collect the data to be verified, and the data to be verified includes the first video information including the actors;

The actor in the data to be verified may be the respondent who will reply to the health notice. A front camera is set on the display screen, and the front camera shoots the first video information of the respondent, which can be displayed on the display screen. The first video information is used for real person identification and identity verification of the respondent.

S120: According to the first video information, perform real person identification and first identity verification on the respondent to determine whether the respondent is a real person and whether it is consistent with the pre-stored ID photo; if the respondent is a real person and is consistent with the pre-stored If the ID photo is consistent, go to s130; if the respondent is not a real person, and/or is inconsistent with the pre-stored ID photo, go to s110;

In this step, it is judged whether it is a real person who is recording the video, and whether it is the person on the submitted ID card who is recording the video. If the two conditions are met at the same time, the following health question answering stage can be carried out. If one of the conditions is not met, it means that the respondent has a tendency to cheat insurance, and the following health questions will no longer be displayed, stop the question and answer of the health notice, and record the first video of other respondents.

S130: Push verification questions to the answerer in turn, and collect the second video information when the answerer answers the verification question;

The verification question can be a single health question, a single health question is displayed on the display screen one by one, and the second video information when the respondent answers the single health question is collected and displayed on the display screen. The display screen only displays one health question. When the respondent answers this health question, the front camera collects the video of the respondent as the second video information. The second video information and this health question are displayed on the display at the same time. After the respondent has answered this health question, the next health question will be displayed.

S140: According to the second video information, perform the second identity verification and micro-expression analysis on the respondent to determine whether the respondent is consistent with the respondent in the first video, and whether the respondent cheated in the process of answering the health question Behavior; if the answerer is inconsistent with the answerer in the first video, stop pushing new verification questions to the answerer; if the answerer is consistent with the answerer in the first video, but the micro-expression analysis answerer is answering healthy If there is deceptive behavior during the problem process, the health problem in this article will be marked as an abnormal problem.

When the respondent answers each health question, a corresponding second video information is collected, which is used to judge whether the respondent has cheated when answering each health question, and record it and save it. It is convenient for the auditor to review the health notice in a targeted manner. After all the questions have been answered, save all the second videos, so that the video files recorded by the respondent when answering the questions can be kept for a long time, providing powerful supporting evidence for possible disputes in the future.

Specifically, in step S120:

First, the multimedia video processing tool FFmpeg is used to separate the image information and the voice information of the first video information to obtain the first image information and the first voice information.

According to the first video information, real person recognition is performed on the answerer to determine whether the answerer is a real person, including the following process:

Intercept at least 20,000 first face pictures from the first image information through frame extraction technology, and calculate the first face picture according to the silent multi-frame liveness detection model to determine whether it is a real person shooting, and the silent multi-frame liveness detection This method can effectively prevent other people from using paper photos, mobile phone photos, and mobile phone videos to collect faces, and truly achieve real face collection.

Use the RPPG heartbeat detection method to obtain the heart rate of the respondent when intercepting each first face picture, and determine whether it is a real person shooting;

If the obtained heart rate is within the set range, calculate the average heart rate fluctuation value according to each heart rate value. If the heart rate is less than the average heart rate fluctuation value, and the result calculated according to the silent multi-frame live detection model is a real person, It is judged that the answerer is a real person, and the real person is recognized and passed;

If the obtained heart rate is not within the set range, it is judged that the respondent is not a real person, and the average heart rate fluctuation value is no longer calculated, and the filling of the health notice is completed. The range of heart rate setting can be: (50-160) beats/min.

The calculation process of the average fluctuation value of heart rate includes:

Divide the first face pictures into M groups according to the time sequence of the interception. Each group includes N first face pictures. The maximum heart rate value in each group is subtracted from the minimum heart rate value to obtain the heart rate difference. The heart rate difference of each group is added and the average value is calculated, and the average value is the average fluctuation value of the heart rate. Calculated as follows:

Among them: M is the number of shooting groups, M can be greater than 10,000, and each group has N face pictures; H ₁ , H ₂ , H ₃ ,..., H _N is a group of intercepting each face picture The heart rate value obtained through the RPPG heartbeat detection at time; A is the average heart rate fluctuation value.

Real person recognition adopts the combination of heartbeat detection and silent multi-frame live detection model at the same time, which increases the reliability.

According to the first video information, real person recognition is performed on the respondent to determine whether the respondent is a real person. Voice recognition and lip language recognition can also be used, including:

Perform voice recognition on the first voice information to obtain the semantic information corresponding to the first voice information; perform frame processing on the first image information to obtain the position of the lips in each frame of the image after the frame; the position of the lips in each frame of the image Perform lip recognition to obtain the semantic information corresponding to the lip of each frame of image; use the time alignment algorithm to calculate the similarity value between the semantic information corresponding to the first speech information and the semantic information corresponding to the lip recognition, and determine the corresponding value according to the similarity value. State whether the person who answered the question is a real person, if it is a real person, the real person will be recognized and passed.

According to the first video information, the first identity verification is performed on the respondent to determine whether the respondent is consistent with the pre-stored ID photo, including the following process:

The face image detection algorithm model can be used to detect the quality of each first face picture, or a few pictures can be randomly selected, and at least one first face picture meeting the preset quality conditions can be selected and stored as a standard face picture ; Compare the standard face picture with the ID photo stored by the respondent to obtain the similarity between the standard face picture and the ID photo. If the similarity is higher than the preset similarity of the ID, the first identity verification is passed.

The face image detection algorithm model can detect the entire face image. The preset quality conditions can be: whether the face features are available, whether the proportion of the face meets the requirements (20%-70%), and whether the overall image pixels meet the setting Require. The purpose of detection is to determine whether the extracted first face image meets the conditions for comparison, and to provide a higher-quality image as a standard face image.

In this step, it is judged whether it is the person on the ID card who is recording the video, and the health question will be asked. If the person on the ID card is recording the video and the real person is recording the video, then the following health question answering stage can be carried out.

In step S130, while the display screen displays the health problem, the voice announcement of the displayed health problem can also be performed automatically through the speaker. Respondents can listen to every health question, avoiding that some respondents do not read the question carefully and answer at will.

Specifically, in step S140:

First, the multimedia video processing tool FFmpeg is used to separate the image information and the voice information of the second video information to obtain the second image information and the second voice information.

According to the second video information, the second identity verification and micro-expression analysis are performed on the respondent to determine whether the respondent is consistent with the respondent in the first video, and to determine whether the respondent is cheating in the process of answering the health question. It includes the following steps:

The second image information in the second video information is framed within a set time to obtain a second face picture, and the second face picture is compared with the standard face image obtained in s120 for second identity verification, If the comparison results are different, the respondent has cheated during the course of answering the health question, stop pushing the health question to the respondent, and stop responding to the respondent’s health notice. If the comparison result is the same, the answerer is the same as the answerer in the first video, and there is no change in the middle, and the health notice will not stop responding. At the same time, input the second face image into the facial expression classification model based on convolutional neural network for micro-expression analysis to determine whether the respondent has cheated in answering health questions. If the second face image corresponds to the micro-expression analysis result If it shows deceptive behavior, it has a tendency to deceive. Mark this health problem as an abnormal problem and save it.

Draw frames within the set time, which can be determined according to the actual situation. In this application, frames can be drawn after the health question is displayed, when the answerer starts to answer the question, that is, when the answerer utters a voice, and when the answer to a health question is completed. Or it can draw frames every 2 seconds.

Since users often reveal more than one information when answering, including their language, micro-expressions, body movements, and conscious touch, they will more clearly reflect the user's intentions. Therefore, multiple channels of information are collected and mutually determined at the same time. It helps the system accurately understand whether users are honest when answering questions.

Respondents answer a health question. If there is a violation of ordinary people’s answering habits, it will be marked as an abnormal question. After all the questions and answers of the entire health notice are completed, the auditor will record the abnormality according to which health question has been recorded before. Knowing that the person's emotional and psychological changes are large, there is a possibility of falsification in the answer to this health question. After all the questions and answers of the entire health notice are completed, the auditor will record the abnormality based on which health problem has been recorded before, and know at which link there is a tendency to deceive. If there is a substitution response, the response to the health notice will be stopped. By checking the recorded abnormal questions, the auditor can easily figure out whether the respondent has false answers when answering each health notification question. If there are false answers, they can conduct data review and so on.

It may further include obtaining and recording the answer to a single health question based on the second video information, including the following process:

Perform voice recognition on the second voice information in the second video information, obtain the semantic information corresponding to the second voice information through the semantic recognition engine, and then use the deep learning neural network model to compare the semantic information with the answer to a single health question pre-stored in the database The matching is performed to obtain the matching rate of the semantic information and the answer, and the semantic information and the matching rate are saved.

The answer to a single health question stored in the database includes multiple answers that can be insured for that health question. The matching rate can indicate the similarity between the answer of the answerer and the answer stored in the database. A high similarity indicates that the answer of the answerer meets the insurance conditions of the health question, and a low similarity indicates that the answer of the answerer needs to be further reviewed by the insurance reviewer. The insurance auditor can determine which insurance condition of the customer needs to be further audited according to the matching rate of each stored health question answer, which improves the auditor's work efficiency and reduces the work intensity.

According to the second video information, obtain and record the answer of the respondent to a single health question. Lip recognition can also be used, which specifically includes the following process: lip recognition is performed on the second image information, and lips are obtained through the face recognition model Then, the lip language recognition algorithm model established by the deep learning neural network matches the lip language model with the lip language model of the single health question answer stored in the database, and obtains the lip language model of the lip motion and answer through the face recognition model Match rate, save lip action and match rate.

The lip language recognition algorithm model mainly uses deep learning model algorithms based on time series recognition such as RNN (recurrent neural network) + LSTM (long short-term memory network).

In this application, the health question to be answered and the video and audio of the respondent recorded by the camera are displayed on the screen at the same time. When the respondent reads and listens to the health question, the respondent’s face, voice, lip print, expression and other information are collected for real person recognition, identity authentication, confirmation of answer content, and recognition of fraudulent responses, which improves insurance and verification. Guaranteed efficiency.

Example 2

Fig. 2 is a flowchart of a data verification method based on video information according to Embodiment 1 of the present application.

As shown in Figure 2, a data verification system based on video information provided by this embodiment includes: a first video collection unit 201, a real person recognition and first identity verification unit 202, a verification question push and a second video collection unit 203 And the deceptive behavior judging unit 204.

The first video collection unit 201 is configured to collect data to be verified, and the data to be verified includes first video information including an action subject;

The real person recognition and first identity verification unit 202 is configured to perform real person recognition and first identity verification on the actors in the data to be verified according to the first video information, to determine whether the actors are real persons and whether they are related to pre-stored certificates If the actor is a real person and it is consistent with the pre-stored ID photo, then the second video collection and verification problem will be pushed; if the actor is not a real person, and/or is inconsistent with the pre-stored ID photo, it will be renewed Perform the first video capture unit;

The verification question push and second video collection unit 203 is configured to push at least one verification question to the actor in turn, and collect the second video information when the actor answers the verification question;

The deceptive behavior judging unit 204 is configured to perform second identity verification and micro-expression analysis on the actor according to the second video information, determine whether the actor is consistent with the actor in the first video, and determine whether the actor is answering the verification question Whether there is deceptive behavior in the process; if the actor is inconsistent with the actor in the first video, stop pushing new verification questions to the actor; if the actor is consistent with the actor in the first video, but the micro-expression analysis of the actor If there is fraud in answering the verification question, the verification question will be marked as an abnormal question.

Example 3

As shown in Fig. 3, an electronic device 1 includes a memory 3 and a processor 2. A computer program 4 is stored in the memory, and the computer program 4 is executed by the processor 3 as follows:

Example 4

A computer-readable storage medium including a data verification program based on video information. When the data verification program based on video information is executed by a processor, the following steps are implemented:

The computer-readable storage medium may be non-volatile or volatile.

As above, the method, device, and storage medium for data verification based on video information according to the present application are described by way of example with reference to FIGS. 1, 2 and 3. However, those skilled in the art should understand that various improvements can be made without departing from the content of this application for the above-mentioned data verification method, device and storage medium based on video information proposed in this application. Therefore, the protection scope of this application should be determined by the content of the appended claims.

Claims

A data verification method based on video information, which includes the following steps:

S110: Collect data to be verified, where the data to be verified includes first video information including an acting subject;

S120: According to the first video information, perform real person identification and first identity verification on the actors in the data to be verified to determine whether the actors are real people and whether they are consistent with the pre-stored ID photos ; If the subject is a real person and is consistent with the pre-stored ID photo, then go to S130; if the subject is not a real person and/or is inconsistent with the pre-stored ID photo, then go to S110 ；

S130: Push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

S140: According to the second video information, perform second identity verification and micro-expression analysis on the actor respectively to determine whether the actor is consistent with the actor in the first video, and determine the behavior Whether the subject has cheated in the process of answering the verification question; if the subject is inconsistent with the subject in the first video, stop pushing new verification questions to the subject; if the subject is If the actors in the first video are the same, but the micro-expression analyzes that the actors have deceptive behaviors in the process of answering the verification question, the verification question is marked as an abnormal question.
The method for data verification based on video information according to claim 1, wherein said first video information is used to perform real person recognition on the subject in the data to be verified to determine whether the subject is real Of people, including the following processes:

Intercepting at least 20,000 first face pictures from the first image information in the first video information by decimating frames, and calculating the first face pictures according to a silent multi-frame living body detection model;

Acquiring, by using the RPPG heartbeat detection method, the heart rate of the subject when each of the first face pictures is intercepted;

If the heart rate is within the set range, the average heart rate fluctuation value is calculated according to the heart rate, if the heart rate is all less than the average heart rate fluctuation value, and the calculation result of the silent multi-frame living body detection model is For a real person, it is judged that the subject of the act is a real person;

If the heart rate is not within the set range, it is determined that the subject is not a real person.
The method for data verification based on video information according to claim 2, wherein the calculation process of the average fluctuation value of the heart rate comprises:

Divide the first face pictures into M groups according to the time sequence of the interception, each group includes N first face pictures, and subtract the smallest heart rate value from the largest heart rate value in each group to obtain the heart rate difference. , The heart rate difference values of each group are added together and the average value is calculated, and the average value is the average fluctuation value of the heart rate.
The method for data verification based on video information according to claim 2, wherein the first identity verification is performed on the subject according to the first video information, and it is determined whether the subject is consistent with a pre-stored ID photo , Including the following processes:

Using a face image detection algorithm model to perform quality detection on each of the first face pictures, and select at least one first face picture that meets a preset quality condition and store it as a standard face picture;

The standard face picture is compared with the ID photo pre-stored by the actor, and the similarity between the standard face picture and the ID photo is obtained. If the similarity is higher than the preset similarity of the ID, Then the first identity verification is passed.
The method for data verification based on video information according to claim 4, wherein the second identity verification and micro-expression analysis are performed on the actor according to the second video information to determine whether the actor is Consistent with the actor in the first video, and judging whether the actor has cheated in the process of answering the verification question, including the following process:

The second image information in the second video information is framed within a set time to obtain a second face picture, and the second face picture is compared with the standard face picture. If the comparison result is If they are the same, the behavior subject is consistent with the behavior subject in the first video;

The second face picture is input into an expression classification model based on a convolutional neural network for micro-expression analysis, and if the micro-expression analysis result shows fraudulent behavior, the verification question is marked as an abnormal question.
The method for data verification based on video information according to claim 1, further comprising obtaining and recording the answer to the verification question by the behavior subject according to the second video information.
The method for data verification based on video information according to claim 1, wherein said S130 further comprises: broadcasting the verification question through a speaker.
A data verification system based on video information, including:

The first video acquisition unit is configured to collect data to be verified, where the data to be verified includes first video information including an action subject;

The real person recognition and first identity verification unit is configured to perform real person recognition and first identity verification on the actors in the data to be verified according to the first video information to determine whether the actors are real people , And whether it is consistent with the pre-stored ID photo; if the actor is a real person and is consistent with the pre-stored ID photo, then the second video collection and verification question push is performed; if the actor is not a real person , And/or is inconsistent with the pre-stored ID photo, then restart the first video capture unit;

A verification question push and second video collection unit, configured to push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

The deceptive behavior determination unit is configured to perform second identity verification and micro-expression analysis on the behavior subject according to the second video information to determine whether the behavior subject is consistent with the behavior subject in the first video, And determine whether the actor has cheated in the process of answering the verification question; if the actor is inconsistent with the actor in the first video, stop pushing new verification questions to the actor; if The actor is consistent with the actor in the first video, but the micro-expression analyzes that the actor has cheating in the process of answering the verification question, and the verification question is marked as an abnormal question.
An electronic device, which includes a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the following steps are implemented: S110: Collect data to be verified, and the data to be verified includes The first video information of the actor;

S120: According to the first video information, perform real person identification and first identity verification on the actors in the data to be verified to determine whether the actors are real people and whether they are consistent with the pre-stored ID photos ; If the subject is a real person and is consistent with the pre-stored ID photo, then go to S130; if the subject is not a real person and/or is inconsistent with the pre-stored ID photo, then go to S110 ；

S130: Push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

S140: According to the second video information, perform second identity verification and micro-expression analysis on the actor respectively to determine whether the actor is consistent with the actor in the first video, and determine the behavior Whether the subject has cheated in the process of answering the verification question; if the subject is inconsistent with the subject in the first video, stop pushing new verification questions to the subject; if the subject is If the actors in the first video are the same, but the micro-expression analyzes that the actors have deceptive behaviors in the process of answering the verification question, the verification question is marked as an abnormal question.
9. The electronic device according to claim 9, wherein the real person recognition of the subject in the data to be verified according to the first video information to determine whether the subject is a real person includes the following process:

Intercepting at least 20,000 first face pictures from the first image information in the first video information by decimating frames, and calculating the first face pictures according to a silent multi-frame living body detection model;

Acquiring, by using the RPPG heartbeat detection method, the heart rate of the subject when each of the first face pictures is intercepted;

If the heart rate is within the set range, the average heart rate fluctuation value is calculated according to the heart rate, if the heart rate is all less than the average heart rate fluctuation value, and the calculation result of the silent multi-frame living body detection model is For a real person, it is judged that the subject of the act is a real person;

If the heart rate is not within the set range, it is determined that the subject is not a real person.
The electronic device of claim 10, wherein the calculation process of the average fluctuation value of the heart rate comprises:

Divide the first face pictures into M groups according to the time sequence of the interception, each group includes N first face pictures, and subtract the smallest heart rate value from the largest heart rate value in each group to obtain the heart rate difference. , The heart rate difference values of each group are added together and the average value is calculated, and the average value is the average fluctuation value of the heart rate.
10. The electronic device according to claim 10, wherein said performing a first identity verification on said actor according to said first video information, and determining whether said actor is consistent with a pre-stored ID photo, comprises the following process:

Using a face image detection algorithm model to perform quality detection on each of the first face pictures, and select at least one first face picture that meets a preset quality condition and store it as a standard face picture;

The standard face picture is compared with the ID photo pre-stored by the actor, and the similarity between the standard face picture and the ID photo is obtained. If the similarity is higher than the preset similarity of the ID, Then the first identity verification is passed.
The electronic device according to claim 12, wherein, according to the second video information, the second identity verification and micro-expression analysis are performed on the behavior subject respectively to determine whether the behavior subject is the same as the first The actors in the video are consistent, and judging whether the actors have cheated in the process of answering the verification question, including the following process:

The second image information in the second video information is framed within a set time to obtain a second face picture, and the second face picture is compared with the standard face picture. If the comparison result is If they are the same, the behavior subject is consistent with the behavior subject in the first video;

The second face picture is input into an expression classification model based on a convolutional neural network for micro-expression analysis, and if the micro-expression analysis result shows fraudulent behavior, the verification question is marked as an abnormal question.
9. The electronic device according to claim 9, further comprising obtaining and recording the answer to the verification question by the agent according to the second video information.
9. The electronic device of claim 9, wherein the S130 further comprises: broadcasting the verification question through a speaker.
A computer-readable storage medium, wherein a data verification program based on video information is stored in the computer-readable storage medium, and when the data verification program based on video information is executed by a processor, the following steps are implemented: S110 : Collect data to be verified, where the data to be verified includes the first video information including the subject;

S120: According to the first video information, perform real person identification and first identity verification on the actors in the data to be verified to determine whether the actors are real people and whether they are consistent with the pre-stored ID photos ; If the subject is a real person and is consistent with the pre-stored ID photo, then go to S130; if the subject is not a real person and/or is inconsistent with the pre-stored ID photo, then go to S110 ；

S130: Push at least one verification question to the actor in turn, and collect second video information when the actor answers the verification question;

S140: According to the second video information, perform second identity verification and micro-expression analysis on the actor respectively to determine whether the actor is consistent with the actor in the first video, and determine the behavior Whether the subject has cheated in the process of answering the verification question; if the subject is inconsistent with the subject in the first video, stop pushing new verification questions to the subject; if the subject is If the actors in the first video are the same, but the micro-expression analyzes that the actors have deceptive behaviors in the process of answering the verification question, the verification question is marked as an abnormal question.
16. The computer-readable storage medium according to claim 16, wherein, according to the first video information, real person recognition is performed on the subject in the data to be verified to determine whether the subject is a real person , Including the following processes:

Intercepting at least 20,000 first face pictures from the first image information in the first video information by decimating frames, and calculating the first face pictures according to a silent multi-frame living body detection model;

Acquiring, by using the RPPG heartbeat detection method, the heart rate of the subject when each of the first face pictures is intercepted;

If the heart rate is within the set range, the average heart rate fluctuation value is calculated according to the heart rate, if the heart rate is all less than the average heart rate fluctuation value, and the calculation result of the silent multi-frame living body detection model is For a real person, it is judged that the subject of the act is a real person;

If the heart rate is not within the set range, it is determined that the subject is not a real person.
The computer-readable storage medium according to claim 17, wherein the calculation process of the average fluctuation value of the heart rate comprises:

Divide the first face pictures into M groups according to the time sequence of the interception, each group includes N first face pictures, and subtract the smallest heart rate value from the largest heart rate value in each group to obtain the heart rate difference. , The heart rate difference values of each group are added together and the average value is calculated, and the average value is the average fluctuation value of the heart rate.
17. The computer-readable storage medium according to claim 17, wherein said performing a first identity verification on said subject according to said first video information, and determining whether said subject is consistent with a pre-stored ID photo, comprises The following process:

Using a face image detection algorithm model to perform quality detection on each of the first face pictures, and select at least one first face picture that meets a preset quality condition and store it as a standard face picture;

The standard face picture is compared with the ID photo pre-stored by the actor, and the similarity between the standard face picture and the ID photo is obtained. If the similarity is higher than the preset similarity of the ID, Then the first identity verification is passed.
17. The computer-readable storage medium according to claim 17, wherein said performing a first identity verification on said subject according to said first video information, and determining whether said subject is consistent with a pre-stored ID photo, comprises The following process:

Using a face image detection algorithm model to perform quality detection on each of the first face pictures, and select at least one first face picture that meets a preset quality condition and store it as a standard face picture;

The standard face picture is compared with the ID photo pre-stored by the actor, and the similarity between the standard face picture and the ID photo is obtained. If the similarity is higher than the preset similarity of the ID, Then the first identity verification is passed.