WO2021190086A1

WO2021190086A1 - Face-to-face examination risk control method and apparatus, computer device, and storage medium

Info

Publication number: WO2021190086A1
Application number: PCT/CN2021/070931
Authority: WO
Inventors: 王盼
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-03-26
Filing date: 2021-01-08
Publication date: 2021-09-30
Also published as: CN111429267A

Abstract

A face-to-face examination risk control method and apparatus, a computer device, and a storage medium. Said method comprises: acquiring video information of a target object replying to stress questions, the stress questions being a question set generated according to collected personal information of the target object (S1000); identifying, in the video information, a time node where each stress question is proposed, and segmenting the video information into several video segments according to the time nodes (S2000); on the basis of image information and audio information in the video segments, extracting from each video segment response information of the target object when replying to a corresponding stress question (S3000); and generating a face-to-face examination score value of the target object according to the response information (S4000). The performance of a target object is scored according to different questions, providing objective evaluation of a target object; the whole process needs no manual operation, reducing manpower costs.

Description

Face-to-face audit risk control method, device, computer equipment and storage medium

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on March 26, 2020, the application number is CN202010225123.0, and the invention title is "face-to-face risk control methods, devices, computer equipment and storage media", all of which The content is incorporated in this application by reference.

Technical field

This application relates to the field of intelligent identification technology. Specifically, this application relates to a face-to-face risk control method, device, computer equipment, and storage medium.

Background technique

With the development of the financial industry and economic development, various emerging businesses and projects continue to unfold. When many companies or individuals implement these projects or businesses and need funds, they will apply for loans from banks and other financial institutions to solve the current problems. The dilemma of companies or individuals using money. In order to ensure that the loan is used in the appropriate place for the purpose, and to ensure that the money taken out can be recovered, the bank and other financial institutions will conduct a qualification review of the lender before the loan. The qualification review includes a review of historical credit information and personal identity. Review and on-site interview performance, etc.

technical problem

The inventor realizes that in the prior art, on-site interviews are generally conducted manually, and the interviewers use their own experience to ask questions about the review materials submitted by the lender, observe the expressions and answers of the lender to determine whether the lender is authentic Answer the questions raised and evaluate the honesty of the lender. However, due to the large number of loan personnel, different face-to-face reviewers have different experience levels, different criteria for judging people, and there are too many subjective factors in manual review, which cannot guarantee the consistency of the lender’s evaluation standards, and manual review is time-consuming and labor-intensive. Excessive labor costs. In order to reduce the subjective factors of the lender’s evaluation, the entire process of manual interviews is recorded by video recording, or the robots are used directly for the interviews. In the existing machine interviews, the problems during the interviews are usually extracted randomly from the database. After conducting a background check on the lender manually, the stress question is manually formulated, the question is output by voice through the robot, and the whole process of answering the lender’s question is recorded in video. Later, the above-mentioned recorded video file is passed Processing, identifying the micro-expression of the lender when answering questions, in order to objectively score the performance of the lender’s answer. However, in this way, the stress question to the lender cannot be automatically generated by the robot according to the individual's own situation, and the evaluation of the micro-expression of the lender answering the question is evaluated by the overall performance of the lender in the entire video, not by the computer The evaluation is performed automatically according to the specific problems, and the accuracy of the evaluation results is not high.

Technical solutions

A face-to-face risk control method, including:

Acquiring video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

Identify the time node raised by each stress question in the video information, and divide the video information into a number of video segments according to the time node, where the video segment includes the character that the target object is answering the corresponding The image information of the facial expression change during the stress question and the audio information representing the voice change of the target object and the reply answer when answering the corresponding stress question;

Based on the image information and audio information, extract the response information when the target object in each video clip responds to the corresponding stress question;

The face-to-face review score value of the target object is generated according to each response information.

A face-to-face audit risk control device, including:

Obtaining module: configured to perform acquisition of video information of a target object responding to a stress question, wherein the stress question is a set of questions generated based on the collected personal information of the target object;

Decomposition module: configured to execute the time node proposed according to the identification of each stress problem in the video information, and divide the video information into several video segments according to the time node, where the video segment includes Image information that characterizes the change in expression of the target object when answering the corresponding stress question and audio information that characterizes the change in voice of the target object and the reply answer when answering the corresponding stress question;

Extraction module: configured to perform extraction of response information when the target object in each video segment replies to a corresponding stress question based on the image information and audio information;

The scoring module is configured to execute the generation of the score value of the face-to-face review of the target object according to each response information.

A computer device includes a memory and a processor, and computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the processor executes the following steps:

A storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

This application eliminates the phenomenon of unfair scoring due to manual subjectivity, and the entire process is completed by a computer without manual operation, which saves labor costs.

Description of the drawings

The above and/or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, in which:

Figure 1 is the flow chart of the risk control method for face-to-face examination of the application;

Figure 2 is a flow chart of the scoring method for face-to-face review of the application;

Figure 3 is a flowchart of a method for generating a micro-expression feature set in this application;

Figure 4 is a flow chart of a method for generating face-to-face review scores based on multiple videos in this application;

Figure 5 is a flow chart of the method for generating stress problems in this application;

Figure 6 is a flowchart of a method for listing stress data to generate stress problems in this application;

FIG. 7 is a flow chart of the method for obtaining the video information of the target object responding to the stress problem in this application;

Figure 8 is a schematic diagram of the application face-to-face risk control device module;

Figure 9 is a block diagram of the basic structure of the computer equipment of this application.

Embodiments of the present invention

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are only used to explain the present application, and cannot be construed as a limitation to the present application.

Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the described features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and/or groups of them. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may also be present. In addition, “connected” or “coupled” used herein may include wireless connection or wireless coupling. The term "and/or" as used herein includes all or any unit and all combinations of one or more associated listed items.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and unless specifically defined as here, they will not be idealized or overly Explain the formal meaning.

Those skilled in the art can understand that the "terminal" and "terminal equipment" used herein include both wireless signal receiver equipment, which only has equipment with wireless signal receivers without transmitting capability, and also includes receiving and transmitting hardware equipment. A device that has a device capable of performing two-way communication receiving and transmitting hardware on a two-way communication link. Such equipment may include: cellular or other communication equipment, which has a single-line display or multi-line display or cellular or other communication equipment without a multi-line display; PCS (Personal Communications Service, personal communication system), which can combine voice and data Processing, fax and/or data communication capabilities; PDA (Personal Digital Assistant), which can include radio frequency receivers, pagers, Internet/Intranet access, web browsers, notebooks, calendars, and/or GPS (Global Positioning System (Global Positioning System) receiver; a conventional laptop and/or palmtop computer or other device that has and/or includes a radio frequency receiver. The "terminal" and "terminal equipment" used here may be portable, transportable, installed in vehicles (aviation, sea and/or land), or suitable and/or configured to operate locally, and/or In a distributed form, it runs on the earth and/or any other location in space. The "terminal" and "terminal device" used here can also be communication terminals, Internet terminals, music/video playback terminals, such as PDA, MID (Mobile Internet Device, mobile Internet device) and/or music/video playback Functional mobile phones can also be devices such as smart TVs and set-top boxes.

Specifically, please refer to Figure 1. This application discloses a face-to-face risk control method, including:

S1000. Obtain video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

The target object is a person who needs to conduct a face-to-face review. The application scenario of this application is to use a computer to conduct a full face-to-face review of the target object. The stress question in the face-to-face interview process is generated by the personal information of the target object, and then interacts with the target object in the way of voice playback. While the target object responds to the stress question, the process of the target object answering the stress question is recorded. To collect real-time video information.

S2000. Identify the time node raised by each stress question in the video information, and divide the video information into a number of video segments according to the time node, where the video segment includes a character that the target object is Image information of facial expression changes when answering the corresponding stress question and audio information representing the voice change of the target object and the reply answer when answering the corresponding stress question;

Since the stress problem is played through the computer by voice playback, and there is video recording during the entire interview process, and the video recording is collected in a timeline manner, the computer can identify each stress problem raised in the video information Time node, the time period between the time node of the previous stress problem and the time node of the next stress problem is the time period for the target object to reply to the previous stress problem, and the video of this time period can be intercepted. Obtain a video clip for the stress problem. In one embodiment, there are multiple stress questions, so multiple video clips can be segmented, and each video clip includes image information that characterizes the change in expression of the target object in answering the stress question, and image information that characterizes the target object’s answer to the stress question. The voice change when stimulating the question and the audio message for replying to the answer.

S3000: Based on the image information and audio information, extract response information when the target object in each video clip replies to a corresponding stress question;

The above-mentioned segmented video segment includes image information that characterizes the change in expression of the target object and audio information that characterizes the change in voice of the target object and a reply answer. Different stress problems involve different aspects, the target object will have different reactions, and the target object’s response to different stress problems is of different importance to the face-to-face review, for example, the stress problem during the face-to-face loan review of the target object For the age, if the target object’s actual age is 35 but deliberately said to be 32 years old, and the stress problem is the annual income, the target object’s actual annual income is 20W, but it is said to be 250,000, if all are identified by micro-expression The target person lied. When scoring the entire interview process, the severity of annual income lie is higher than the severity of age lie, so the greater the proportion in the final face-to-face scoring. For this reason, the video information is divided into stress problems, and scores are based on the performance of the target object under a single stress problem. The performance of the target user is the facial expressions and body expressions recognized by the image information and the audio information. The voice intonation and question answer are comprehensively reflected by multiple factors. Based on these factors, a comprehensive face-to-face review score value is calculated to make the face-to-face review score more objective.

An application of the technical solution of this application is to conduct AI interviews with target objects, where the stress question is raised and the video recording is done by the computer itself, and the stress question of the target object is based on the collected target object’s individual The computer outputs the stress problem in the form of voice broadcast, and records the video information of the target responding to the stress problem. The computer decomposes the collected video information into several video clips according to the time node of the stress problem. The performance of the target object and the response information in the video clip are scored, and then the total face-to-face review score is calculated. The performance of the target object is scored according to different questions, which can evaluate the target object objectively and eliminate the artificial The subjectivity of scoring leads to the phenomenon of unfair scoring, and the whole process is completed by computer without manual operation, which saves labor costs.

In one embodiment, referring to FIG. 2, the response information includes a micro-expression feature set, a voice feature set, and a reply answer, and the extracted response information when the target object responds to a corresponding stress question in each video clip includes:

S3100: Generate a micro-expression feature set that characterizes the video clip from the facial and body actions of the target object in the image information in the video clip;

S3200. Extract a voice feature set that characterizes the target object of the video clip and a reply answer from the audio information in the video clip, where the voice feature set includes voice, intonation, speech rate, and amplitude information;

S3300. Mapping the micro-expression feature set, the sound feature set and the reply answer in a time axis manner to obtain response information corresponding to the stress question.

In one embodiment, the response information is extracted by the target object's micro-expression and audio information to evaluate the question. The micro expressions are the facial expressions and body movements of the target object. In one embodiment, please refer to FIG. 3. The micro expressions that characterize the video clip are generated from the facial and body movements of the target object in the image information in the video clip. The feature set includes:

S3110. Input the image information into a micro-expression recognition model to output the micro-expression in the image information;

S3120: Mapping the recognized micro-expression and the time point in the corresponding video segment to form a micro-expression feature set.

In the process of individual micro-expression recognition, the image information of the video clip is first input into the micro-expression recognition model. The micro-expression recognition model is a neural network model that is pre-trained to a convergent state and can recognize various expressions of the target object. Summarize the facial expressions of the human face, and map the changes in facial features when the human body produces the facial expressions to perform facial expression recognition. The change value of the facial features is the movement feature of the facial expression. In this embodiment, the facial expression recognition is performed through the video image through the movement feature between the contour of the face and the facial features in each frame of the video connected first. The change is recognized. Specifically, the specific performance of the motion characteristics of the micro-expressions can be found in Table 1:

Table 1: Specific performance of micro-expressions

Facial expression recognition based on neural network is mainly divided into three steps: face detection, expression feature extraction and expression classification. Normally, there is interference information from other environments in the image for facial expression recognition, so face detection is equivalent to detecting the face in the video or image to be processed and segmenting it from the image, thereby effectively reducing facial expression recognition Interference information that may appear in the process. Face detection is mainly based on the unique characteristics of the face, detecting whether there is a face in the image area to be tested, and comparing the detection possibility with a threshold, so as to locate the coordinate information of the face and segment the face. Location.

When the face area is determined, it is equivalent to reducing the detection area for facial expression recognition, and feature extraction of the facial expression information is performed on the facial area. At present, the more popular expression feature extraction methods include principal component analysis (PCA), local binary mode (LBP), and some other feature extraction methods based on motion and deformation.

At this stage, it is necessary to select a suitable classifier for the problem to be asked for training to obtain a classifier with accurate classification and strong generalization ability, and use the image to be detected as input, and then face detection, expression feature extraction and classifier classification to obtain the face The category of the emoticon.

After the micro-expression in the associated frame of the video is identified through the neural network model, the time axis information corresponding to the micro-expression is obtained, and the time-axis information is matched with the corresponding micro-expression to form a micro-expression feature set.

The method of recognizing the specific content of voice information can also be processed by neural network model, for example, using Kaldi network model to extract features, output voice content in text, and recognize the voice, intonation, and speed of the target object when responding And amplitude information, etc., these voice information constitute a set of sound features of the target object in the video segment. The combination of voice feature set and micro-expression feature set can be used as a factor to evaluate the credit score of the video clip. For example, when the micro-expression is recognized as the face turns red and the eyes turn left and right unconsciously, and the voice feature at the same micro-expression time point is stuttering, it can be recognized that the current state of the target object is nervous and may be suspected of lying. Inputting the recognized micro-expression feature set and voice feature set into the scoring model, the corresponding credit score score can be obtained. In one embodiment, the scoring model is a pre-trained model that scores based on stress questions and possible responses and expressions.

S4000: Generate a face-to-face review score value of the target object according to each response information.

After each video clip passes step S3000, a corresponding response message is generated, and each response message is processed, and finally the face-to-face review score value of the target object can be generated.

In an embodiment, referring to FIG. 4, generating the face-to-face review score value of the target object according to each response information includes:

S4100. Input each of the response information into a scoring model in turn to output the credit scores of the corresponding video clips in turn, where the scoring model is pre-trained to a convergent state, and according to the stress question and its possible response answers, A model for scoring micro-expression features and voice features;

S4200: Obtain the scoring weight value of the stress problem corresponding to each video clip;

S4300: According to the scoring weight value and the corresponding credit scoring value, a weighted average method is used to calculate and generate a face-to-face review score value.

In one embodiment, after the video information is disassembled into a single video segment, the performance of the target object in the video segment is scored through a neural network model to obtain a credit score score. The credit score is the value of the probability that the target object may lie when answering the question.

Since the above-mentioned disclosures of different stress issues have different importance to the face-to-face audit results, different scoring weight values are formulated for different stress issues to highlight the seriousness of fraud on certain issues. After the computer obtains the credit scoring value of the corresponding video clip, it obtains the scoring weight value of the stress problem, multiplying all the credit scoring scores by the corresponding scoring weight value, and then weighting the average to get the final face-to-face review The score value. For example, there are three stress problems including A, B, and C, and the corresponding evaluation weights are 0.3, 0.6, and 0.1 respectively. The corresponding credit scores of stress problems A, B, and C are 80, 60, and 50, respectively. Then the score value of face-to-face review=(0.3*80+0.6*60+0.1*50)/(0.3+0.6+0.1)=65, and the final score value of face-to-face review is 65 points.

In one embodiment, referring to FIG. 5, generating the stress question according to the collected personal information of the target object includes:

S1100. Acquire personal information of the target object;

S1200: Matching stress data in a preset stress database according to the personal information;

S1300. List the stress data according to a preset rule to generate a stress problem.

In one embodiment, during the face-to-face interview process, different target objects have different stress problems, so as to improve the referenceability of the face-to-face interview. One method of generating stress problems is to generate corresponding stress problems according to preset rules through the collected personal information of the target object. Personal information includes, but is not limited to, personal identity information, historical behavior data, and historical credit data, etc. Personal identity information includes age, marital status, occupation, income, etc., historical behavior data includes asset purchases, loan records, and historical credit data including Credit rating, etc. Different personal information corresponds to different stress data, and different stress data includes one or more different questions. In one embodiment, please refer to FIG. Methods of listing to generate stress problems include:

S1310. Obtain the priority level of the stress data;

S1320. List the stress data according to the priority level to generate a stress problem.

In order to ask questions about stress in an orderly manner, it is necessary to arrange the obtained stress data. Since the stress data is matched from the stress database according to personal information, it can be responded to according to a preset rule. The questions in the stress data are listed. In one embodiment, the preset rule is to list the questions according to the priority level of the stress data to generate stress questions. Therefore, in addition to mapping the association relationship between personal information and stress data in the preset stress database, the priority level of the stress data is also mapped. As an example, please refer to Table 2:

Table 2: List of the relationship between personal information and stress data

The above multiple databases are combined to form a preset stress database. In one embodiment, the corresponding databases are selected based on the personal identity information of the target object. Since there are multiple databases, each database contains multiple stress issues. Therefore, the lender can be asked in turn for each matching database.

In one embodiment, the stress data is a combination of multiple questions listed according to a preset association relationship, and the combination relationship is arranged according to the logical relationship between the questions. For example, in a married database, the first The question is whether there are children. If not, ask when you plan to have children; if so, ask the number of children, then ask how old each child is, and then ask the child’s monthly expenditure percentage, etc., in the database of the number of fixed assets , You can ask about the type of fixed assets, and then according to the identified type, look up the next question in the corresponding type database. For example, if the lender answers that the fixed asset is a real estate, then ask about the number of real estate, then ask about the location, and then ask about the real estate. Ask about the total area of the property, ask about the purchase period of the real estate, ask whether the nature of the real estate is shared or exclusive, whether it is an apartment or a residence, and ask about the mortgage of the real estate. In the process of questioning, the questions in the same stress database are asked in the order in which they are arranged.

In one embodiment, referring to FIG. 7, the stress data obtained is a combination of multiple questions listed according to a preset association relationship, and the obtaining video information of the target object responding to the stress question includes:

S1400: Acquire image information and audio information of the target object's response to the stress problem, and associate the image information and audio information in a time axis manner;

S1500: Identify the reply answer corresponding to the target object in the audio information;

S1600. Select a combination of questions in the stress data according to the reply answer to raise subsequent stress questions, and obtain video information synchronously.

Since the above-disclosed stress data includes multiple questions, each question is arranged in accordance with the logical relationship between each other. Therefore, the method of obtaining the video information of the target object's response to the stress question is to first obtain the stress with the highest priority. Ask the first question in the data. At the same time, the image information and audio information of the target object's reply are obtained, and the audio information is recognized. After the reply answer is identified, the follow-up question combination in the stress data is selected according to the reply answer. The question is played by voice, and the video information of the target object's reply is obtained synchronously. When all the stressful questions are answered, the video shooting ends.

On the other hand, please refer to Figure 8. This application discloses a face-to-face risk control device, including:

Obtaining module 1000: configured to perform acquisition of video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

Decomposition module 2000: configured to execute the time node proposed according to the identification of each stress problem in the video information, and divide the video information into several video segments according to the time node, wherein, in the video segment Including image information that characterizes the change in expression of the target object when answering the corresponding stress question and audio information that characterizes the change in voice of the target object and the answer to the answer when answering the corresponding stress question;

Extraction module 3000: configured to perform, based on the image information and audio information, extract response information when the target object in each video clip responds to a corresponding stress question

The scoring module 4000 is configured to execute the generation of the score value of the face-to-face review of the target object according to each response information.

Optionally, the response information includes a micro-expression feature set, a voice feature set, and a reply answer, and the extraction module includes:

The first feature generation module: configured to execute the generation of a micro-expression feature set based on the facial and body movements of the target object in the image information;

The second feature generation module: configured to perform extraction of a voice feature set characterizing the target object of the video clip and a reply answer from the audio information in the video clip, wherein the voice feature set includes voice, intonation, and speech rate And amplitude information;

Response information generation module: configured to perform mapping of the micro-expression feature set, the sound feature set and the response answer to each other in a time axis manner to obtain response information corresponding to the stress question.

Optionally, the scoring module includes:

Credit score output module: configured to input each of the response information into the scoring model in turn to output the credit score scores of the corresponding video clips in turn, where the scoring model is pre-trained to a convergent state, according to the application Models for scoring exciting questions and possible response answers, micro-expression characteristics and voice characteristics;

Weight acquisition module: configured to execute and acquire the scoring weight value of the stress problem corresponding to each video clip;

The weighted calculation module is configured to perform calculation and generate the face-to-face review score value in a weighted average manner according to the score weight value and the corresponding credit score score value.

Optionally, the first feature set generating module includes:

Image input module: configured to perform input of the image information into the micro-expression recognition model to output the micro-expression in the image information;

Mapping module: configured to perform mutual mapping of the recognized micro-expression and the time point that appears in the video to form a micro-expression feature set.

Optionally, it also includes:

Personal information acquisition module: configured to perform acquisition of the personal information of the target object;

Matching module: configured to perform matching of stress data in a preset stress database according to the personal information;

The first generating module is configured to perform listing of the stress data according to preset rules to generate stress questions.

Optionally, the first generating module includes:

Priority acquisition module: configured to execute the priority acquisition of the stress data;

The second generating module is configured to perform listing of the stress data according to the priority level to generate stress questions.

Optionally, the stress data is a combination of multiple questions listed according to a preset association relationship, and the acquisition module includes:

Associating module: configured to perform acquisition of image information and audio information of the target object's response to the stress problem, and to associate the image information and audio information in a time axis manner;

Recognition module: configured to perform recognition of the reply answer of the target object in the audio information;

Selection module: configured to perform selection of a combination of questions in the stress data according to the reply answer to raise subsequent stress questions, and obtain video information synchronously.

Please refer to FIG. 9 for the basic structure block diagram of the computer equipment provided in the embodiment of the present application.

The computer equipment includes a processor, a non-volatile storage medium, a memory, and a network interface connected through a system bus. Wherein, the non-volatile storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, the processor can realize a A kind of face-to-face audit risk control method. The processor of the computer equipment is used to provide calculation and control capabilities, and supports the operation of the entire computer equipment. The computer readable instructions may be stored in the memory of the computer device, and when the computer readable instructions are executed by the processor, the processor may execute the following steps:

The specific embodiments of the computer equipment of this application are basically the same as the embodiments of the above-mentioned face-to-face risk control method, and will not be repeated here.

The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art can understand that the structure shown in FIG. 9 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

The computer device receives the status information of the prompt behavior sent by the associated client, that is, whether the associated terminal opens the prompt and whether the user closes the prompt task. By verifying whether the above-mentioned task conditions are fulfilled, the corresponding preset instruction is sent to the associated terminal, so that the associated terminal can perform corresponding operations according to the preset instruction, thereby realizing effective supervision of the associated terminal. At the same time, when the prompt information state is different from the preset state instruction, the server side controls the associated terminal to continue ringing, so as to prevent the prompt task of the associated terminal from automatically terminating after a period of execution.

The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

The specific embodiments of the storage medium of the computer-readable instructions of the present application are basically the same as the embodiments of the above-mentioned face-to-face risk control method, and will not be repeated here.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or a volatile storage medium, or a random access memory (Random Access Memory). , RAM) etc.

In another embodiment, the face-to-face audit risk control method provided in this application further ensures the privacy and security of all the above-mentioned data, all the above-mentioned data can also be stored in a node of a blockchain. For example, video information and audio information, etc., these data can be stored in the blockchain node.

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

It should be understood that although the various steps in the flowchart of the drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the order of execution is also It is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

The above are only part of the implementation of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of this application, several improvements and modifications can be made, and these improvements and modifications are also Should be regarded as the scope of protection of this application.

Claims

A face-to-face risk control method, which includes:

Acquiring video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

Identify the time node raised by each stress question in the video information, and divide the video information into a number of video segments according to the time node, where the video segment includes the character that the target object is answering the corresponding The image information of the facial expression change during the stress question and the audio information representing the voice change of the target object and the reply answer when answering the corresponding stress question;

Based on the image information and audio information, extract the response information when the target object in each video clip responds to the corresponding stress question;

The face-to-face review score value of the target object is generated according to each response information.
The face-to-face audit risk control method according to claim 1, wherein the response information includes a set of micro-expression features, a set of voice features, and a response answer, and the extraction of each video clip when the target object responds to the corresponding stress question The response information includes:

Generating a set of micro-expression features that characterize the video segment from the facial and body movements of the target object in the image information in the video segment;

Extracting a voice feature set that characterizes the target object of the video clip and a reply answer from the audio information in the video clip, where the voice feature set includes voice, intonation, speech rate, and amplitude information;

The micro-expression feature set, the voice feature set and the reply answer are mapped to each other in a time axis manner to obtain response information corresponding to the stress question.
The method for controlling face-to-face audit risk according to claim 2, wherein said generating the face-to-face audit score value of the target object according to each response information comprises:

Each of the response information is sequentially input into the scoring model to sequentially output the credit scoring value of the corresponding video clip, wherein the scoring model is pre-trained to a convergent state, based on the stress question and its possible response answers, micro-expressions A model for scoring features and voice features;

Obtain the scoring weight value of the stress problem corresponding to each video clip;

According to the scoring weight value and the corresponding credit scoring value, a weighted average method is used to calculate and generate a face-to-face review score value.
The face-to-face audit risk control method according to claim 2, wherein said generating a micro-expression feature set characterizing the video clip from the facial and body actions of the target object in the image information in the video clip comprises:

Inputting the image information into a micro-expression recognition model to output the micro-expression in the image information;

The identified micro-expression and the time point in the corresponding video segment are mapped to each other to form a micro-expression feature set.
The face-to-face audit risk control method according to claim 1, wherein generating the stress question according to the collected personal information of the target object comprises:

Obtaining personal information of the target object;

Matching stress data in a preset stress database according to the personal information;

List the stress data according to preset rules to generate stress problems.
The face-to-face audit risk control method according to claim 5, wherein the listing the stress data according to a preset rule to generate a stress problem comprises:

Acquiring the priority level of the stress data;

The stress data is listed according to the priority level to generate stress problems.
The face-to-face audit risk control method according to claim 6, wherein the stress data is a combination of a plurality of questions listed according to a preset association relationship, and the obtaining the video information of the target object responding to the stress question comprises:

Acquiring image information and audio information of the target object in reply to the stress question, and associating the image information and audio information in a time axis manner;

Identifying the reply answer of the target object in the audio information;

According to the reply answer, a combination of questions in the stress data is selected to raise subsequent stress questions, and video information is obtained synchronously.
A face-to-face audit risk control device, which includes:

An acquisition module: configured to perform acquisition of video information of a target object responding to a stress question, wherein the stress question is a question set generated according to the collected personal information of the target object;

Decomposition module: configured to execute the time node proposed according to the identification of each stress problem in the video information, and divide the video information into several video segments according to the time node, where the video segment includes Image information that characterizes the change in expression of the target object when answering the corresponding stress question and audio information that characterizes the change in voice of the target object and the reply answer when answering the corresponding stress question;

Extraction module: configured to perform extraction of response information when the target object in each video segment replies to a corresponding stress question based on the image information and audio information;

The scoring module is configured to execute the generation of the score value of the face-to-face review of the target object according to each response information.
A computer device includes a memory and a processor. The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the following steps:

Acquiring video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

Identify the time node raised by each stress question in the video information, and divide the video information into a number of video segments according to the time node, where the video segment includes the character that the target object is answering the corresponding The image information of the facial expression change during the stress question and the audio information representing the voice change of the target object and the reply answer when answering the corresponding stress question;

Based on the image information and audio information, extract the response information when the target object in each video clip responds to the corresponding stress question;

The face-to-face review score value of the target object is generated according to each response information.
The computer device according to claim 9, wherein the response information includes a micro-expression feature set, a sound feature set, and a response answer, and the extracted response information when the target object in each video clip responds to a corresponding stress question includes :

Generating a set of micro-expression features that characterize the video segment from the facial and body movements of the target object in the image information in the video segment;

Extracting a voice feature set that characterizes the target object of the video clip and a reply answer from the audio information in the video clip, where the voice feature set includes voice, intonation, speech rate, and amplitude information;

The micro-expression feature set, the voice feature set and the reply answer are mapped to each other in a time axis manner to obtain response information corresponding to the stress question.
11. The computer device according to claim 10, wherein said generating the face-to-face review score value of the target object according to each response information comprises:

Each of the response information is sequentially input into the scoring model to sequentially output the credit scoring value of the corresponding video clip, wherein the scoring model is pre-trained to a convergent state, based on the stress question and its possible response answers, micro-expressions A model for scoring features and voice features;

Obtain the scoring weight value of the stress problem corresponding to each video clip;

According to the scoring weight value and the corresponding credit scoring value, a weighted average method is used to calculate and generate a face-to-face review score value.
10. The computer device according to claim 10, wherein said generating a set of micro-expression features characterizing the video clip from the facial and body actions of the target object in the image information in the video clip comprises:

Inputting the image information into a micro-expression recognition model to output the micro-expression in the image information;

The identified micro-expression and the time point in the corresponding video segment are mapped to each other to form a micro-expression feature set.
The computer device according to claim 9, wherein generating the stress question based on the collected personal information of the target object comprises:

Obtaining personal information of the target object;

Matching stress data in a preset stress database according to the personal information;

List the stress data according to preset rules to generate stress problems.
The computer device according to claim 13, wherein the listing the stress data according to a preset rule to generate a stress question comprises:

Acquiring the priority level of the stress data;

The stress data is listed according to the priority level to generate stress problems.
The computer device according to claim 14, wherein the stress data is a combination of a plurality of questions listed according to a preset association relationship, and the obtaining the video information of the target object responding to the stress question comprises:

Acquiring image information and audio information of the target object in reply to the stress question, and associating the image information and audio information in a time axis manner;

Identifying the reply answer of the target object in the audio information;

According to the reply answer, a combination of questions in the stress data is selected to raise subsequent stress questions, and video information is obtained synchronously.
A storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Acquiring video information of a target object responding to a stress question, where the stress question is a set of questions generated based on the collected personal information of the target object;

Identify the time node raised by each stress question in the video information, and divide the video information into a number of video segments according to the time node, where the video segment includes the character that the target object is answering the corresponding The image information of the facial expression change during the stress question and the audio information representing the voice change of the target object and the reply answer when answering the corresponding stress question;

Based on the image information and audio information, extract the response information when the target object in each video clip responds to the corresponding stress question;

The face-to-face review score value of the target object is generated according to each response information.
The storage medium according to claim 16, wherein the response information includes a micro-expression feature set, a sound feature set, and a response answer, and the extracted response information when the target object in each video clip responds to a corresponding stress question includes :

Generating a set of micro-expression features that characterize the video segment from the facial and body movements of the target object in the image information in the video segment;

Extracting a voice feature set that characterizes the target object of the video clip and a reply answer from the audio information in the video clip, where the voice feature set includes voice, intonation, speech rate, and amplitude information;

The micro-expression feature set, the voice feature set and the reply answer are mapped to each other in a time axis manner to obtain response information corresponding to the stress question.
18. The storage medium according to claim 17, wherein said generating the face-to-face review score value of the target object according to each response information comprises:

Each of the response information is sequentially input into the scoring model to sequentially output the credit scoring value of the corresponding video clip, wherein the scoring model is pre-trained to a convergent state, based on the stress question and its possible response answers, micro-expressions A model for scoring features and voice features;

Obtain the scoring weight value of the stress problem corresponding to each video clip;

According to the scoring weight value and the corresponding credit scoring value, a weighted average method is used to calculate and generate a face-to-face review score value.
18. The storage medium according to claim 17, wherein said generating a set of micro-expression features characterizing the video clip from the facial and body movements of the target object in the image information in the video clip comprises:

Inputting the image information into a micro-expression recognition model to output the micro-expression in the image information;

The identified micro-expression and the time point in the corresponding video segment are mapped to each other to form a micro-expression feature set.
The storage medium according to claim 16, wherein generating the stress question according to the collected personal information of the target object comprises:

Obtaining personal information of the target object;

Matching stress data in a preset stress database according to the personal information;

List the stress data according to preset rules to generate stress problems.