WO2019184125A1 - 基于微表情的风险识别方法、装置、设备及介质 - Google Patents

基于微表情的风险识别方法、装置、设备及介质 Download PDF

Info

Publication number
WO2019184125A1
WO2019184125A1 PCT/CN2018/094217 CN2018094217W WO2019184125A1 WO 2019184125 A1 WO2019184125 A1 WO 2019184125A1 CN 2018094217 W CN2018094217 W CN 2018094217W WO 2019184125 A1 WO2019184125 A1 WO 2019184125A1
Authority
WO
WIPO (PCT)
Prior art keywords
expression recognition
recognition result
test
standard
detection model
Prior art date
Application number
PCT/CN2018/094217
Other languages
English (en)
French (fr)
Inventor
戴磊
张国辉
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2019184125A1 publication Critical patent/WO2019184125A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/175—Static expression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/03—Credit; Loans; Processing thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Definitions

  • the present application relates to the field of face recognition, and in particular, to a method, device, device and medium for risk identification based on micro-expression.
  • each loan loan is subject to risk management (ie risk control) to determine whether a loan can be issued to the lender.
  • risk management ie risk control
  • a key step in the traditional risk control approach of the financial industry is the face-to-face communication between the creditor and the lender to determine the accuracy of the information provided by the lender in the process of processing the loan, thereby determining the risk of the loan.
  • the credit reviewer may not understand the facial expressions of the lender because of inattention or lack of understanding of the facial expressions of the person. These subtle expression changes may reflect the lender's communication.
  • Psychological activities (such as lying) make the risk control results issued by the credit reviewer less credible due to neglecting the micro-expression of the lender's credit review process.
  • the embodiment of the present application provides a method, a device, a device and a medium for risk identification based on micro-expressions, so as to solve the problem that the risk control result is not highly credible due to neglecting the change of the micro-expression of the lender.
  • an embodiment of the present application provides a risk recognition method based on a micro-expression, including:
  • a risk identification result is obtained based on the standard expression recognition result and the test expression recognition result.
  • the embodiment of the present application provides a micro-expression based risk identification device, including:
  • a video data acquiring module to be used for acquiring video data to be identified, where the video data to be identified includes at least two frames of video images to be identified;
  • the video data dividing module to be identified is configured to divide at least two frames of the video image to be identified into a basic problem feature set and a sensitive problem feature set;
  • a standard expression recognition result obtaining module configured to input the image to be recognized in each frame of the basic problem feature set into at least two micro-expression recognition models that are pre-trained to obtain a corresponding standard expression recognition result;
  • the test expression recognition result obtaining module is configured to input the image to be recognized in each frame of the sensitive problem feature set into at least two micro-expression recognition models that are pre-trained to obtain a corresponding test expression recognition result;
  • the risk identification result obtaining module is configured to obtain a risk identification result based on the standard expression recognition result and the test expression recognition result.
  • an embodiment of the present application provides a computer device, including a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, where the processor executes the computer The following steps are implemented when reading the instruction:
  • a risk identification result is obtained based on the standard expression recognition result and the test expression recognition result.
  • the embodiment of the present application provides one or more non-volatile readable storage media storing computer readable instructions, when the computer readable instructions are executed by one or more processors, such that the one or Multiple processors perform the following steps:
  • a risk identification result is obtained based on the standard expression recognition result and the test expression recognition result.
  • FIG. 1 is a flowchart of a micro-expression-based risk identification method provided in Embodiment 1 of the present application;
  • FIG. 2 is a specific schematic view of step S10 of Figure 1;
  • FIG. 3 is a specific schematic view of step S30 of Figure 1;
  • FIG. 4 is a specific schematic view of step S40 of Figure 1;
  • FIG. 5 is a specific schematic view of step S50 of Figure 1;
  • FIG. 6 is a schematic block diagram of a micro-expression based risk identification device provided in Embodiment 2 of the present application.
  • FIG. 7 is a schematic diagram of a computer device provided in Embodiment 4 of the present application.
  • FIG. 1 is a flow chart showing a micro-expression based risk recognition method in the present embodiment.
  • the micro-expression-based risk identification method can be applied to financial institutions such as banks, securities, insurance, etc., and can effectively assist the credit reviewer to control the risk of the lender, thereby determining whether the loan can be issued to the lender.
  • the micro-expression-based risk identification method includes the following steps:
  • S10 Acquire video data to be identified, where the video data to be identified includes at least two frames of video images to be identified.
  • the video data to be identified refers to the video data obtained by preprocessing the original video data.
  • the original video data is used to record the unprocessed video data of the lender during the credit review process.
  • the video data to be identified is video data composed of at least two frames of video images to be identified.
  • the video data to be identified (ie, the original video data) needs to be divided according to the video data to be recognized by the target client before the identification of the video data to be identified. Therefore, the video data to be identified includes at least two frames of the video image to be identified. In order to determine the micro-feature feature of the face in each frame of the video image to be identified, to determine whether the user is lying for risk management.
  • S20 Divide at least two frames of the video image to be identified into a basic problem feature set and a sensitive problem feature set.
  • the basic problem feature set refers to a set of basic questions set based on some personal information of the target customer, such as an ID number, a mobile phone number of a relative, and a home address.
  • a sensitive problem feature set is a collection of basic questions used to determine whether a target customer is at risk, such as loan use, personal income, and willingness to repay.
  • the division of the basic problem feature set and the sensitive question feature set is divided according to the condition that the question has a standard answer. Take the bank as an example. If the target customer pre-stores some personal information (such as ID number, family phone number and home address) in financial institutions such as banks, securities, insurance, etc., based on these personal information pre-stored with standard answers. The proposed question is replied to the set of video images to be identified corresponding to the basic problem feature set. If the target customer does not pre-store the information in the financial institutions such as banks, securities, insurance, etc., it is considered that there is no standard answer to the part of the information, and the set of the video image to be identified corresponding to the question raised by the part of the information is regarded as sensitive. Problem feature set.
  • the basic problem feature set includes at least one frame to be identified video image;
  • the sensitive problem feature set includes at least one frame of the to-be-identified video image, so as to be subsequently performed based on the recognition result of the basic problem feature set and the recognition result of the sensitive problem feature set. Judging, thereby achieving the purpose of risk control, improving the accuracy of risk identification, and the number of video frames in the feature set of the basic problem is the same as the number of video frames in the feature set of the sensitive problem, so that the subsequent calculation of the recognition result is convenient.
  • the micro-expression recognition model is a pre-trained model for acquiring the micro-expression features of the target client.
  • the standard expression recognition result is a recognition result obtained by using the micro-expression recognition model to identify each frame of the video image to be recognized in the basic problem feature set. Specifically, each frame to be recognized in the basic problem feature set is input to at least two micro-expression recognition models that are pre-trained for recognition, to obtain corresponding standard expression recognition results output by each micro-expression recognition model,
  • the standard expression recognition result reflects the micro-expression of the target customer when speaking the truth to a certain extent, and can be used as a basis for judging whether the target customer is telling the truth when replying to a sensitive question.
  • each frame of the to-be-identified video image in the basic problem feature set is input to at least two micro-expression recognition models for recognition, so as to obtain a corresponding standard expression recognition result, so as to improve the accuracy of the risk recognition, so that the auxiliary effect is obtained.
  • S40 Input each image to be recognized in the sensitive problem feature set into at least two micro-expression recognition models that are pre-trained for recognition, and obtain corresponding test expression recognition results.
  • the test expression recognition result is a recognition result obtained by using the micro-expression recognition model to identify each frame of the to-be-identified video image in the feature set of the sensitive problem. Specifically, each frame of the to-be-identified video image in the sensitive problem feature set is input to at least two micro-expression recognition models that are pre-trained for recognition to obtain a corresponding test expression recognition result output by each micro-expression recognition model.
  • the test expression recognition result reflects to some extent the micro-expression of the target customer who speaks truth or lie when replying to sensitive questions.
  • the video image to be identified in each of the sensitive problem feature sets is input to at least two micro-expression recognition models for recognition, so as to obtain a corresponding test expression recognition result, and the accuracy of the risk recognition is improved, so that the auxiliary effect is better.
  • S50 Obtain a risk identification result based on the standard expression recognition result and the test expression recognition result.
  • the standard expression recognition results corresponding to each of the frames of the basic problem feature set to be identified are summarized as reference data.
  • the test expression recognition result corresponding to each frame of the to-be-identified video image in the sensitive problem feature set is summarized as test data, and the reference data is compared with the test data, if the test data is compared with the reference data difference multiples and presets The thresholds are compared to obtain a risk level, and then the risk identification result is obtained.
  • the to-be-identified video data includes at least two frames of the to-be-identified video image, so as to divide at least two frames of the to-be-identified video image into equal-scale basic problem feature sets and sensitive problem feature sets, so that When the statistics of the recognition result are subsequently calculated, the calculation is convenient. Then, the video image to be recognized in each frame of the basic problem feature set is input to at least two micro-expression recognition models that are pre-trained for recognition, and the corresponding standard expression recognition result is obtained; and the sensitive problem feature is concentrated on each frame of the to-be-recognized video image.
  • step S10 the video data to be identified is obtained, which specifically includes the following steps:
  • the original video data is used to record the unprocessed video data of the lender during the credit review process.
  • the credit reviewer can perform a video chat with the target client (ie, the lender), and in the video chat process, the target client is asked based on the preset question to obtain the video data of the target customer replying to the problem, that is, the original video data.
  • S12 Perform framing and normalization processing on the original video data to obtain video data to be identified.
  • the framing processing refers to dividing the original video data according to a preset time to acquire at least one frame of the to-be-identified video image.
  • normalization is a way to simplify the calculation, that is, a dimensional expression, transformed into a dimensionless expression, becomes a scalar.
  • the face area of the target client is required to extract the micro-feature feature of the target client. Therefore, the pixels of the video image to be identified after the frame is normalized to 260*260 pixels. The pixels are unified so that each frame of the video image to be identified is subsequently identified.
  • the target customer is questioned by means of video chat to obtain the video data of the target customer reply, that is, the original video data, so that the process of the credit review is intelligent, and the face-to-face communication between the creditor and the target customer is not required, thereby saving Labor cost.
  • the original video data is framed and normalized, and the pixels of the video image to be identified are unified for each frame, so that the video image to be identified for each frame is subsequently identified, thereby improving the accuracy of the risk identification.
  • the micro-expression recognition model in step S30 includes at least two of a face detection model, a feature point detection model, an emotion detection model, a head posture detection model, a blink detection model, and an iris edge detection model.
  • the face detection model is a model for extracting a face image of each frame of the video image to be recognized.
  • the feature point detection model is a model for identifying face feature points in a video image to be recognized for each frame.
  • the head pose detection model is a model for identifying the head offset direction of the target client in each frame of the video image to be identified.
  • the blink detection model is a model for identifying whether a target customer in each video frame to be recognized blinks.
  • the iris edge detection model is used to reflect the model of the eye movement of the target user in each frame of the video image to be identified.
  • the basic problem feature set and the sensitive feature set are respectively input into the face detection model, the feature point detection model, the emotion detection model, the head posture detection model, the blink detection model, and the iris edge detection model.
  • the model is identified to obtain the standard expression recognition result of the target customer and the test expression recognition result, and the purpose of the micro-expression based risk recognition is realized based on the standard expression recognition result and the test expression recognition result.
  • step S30 the video image to be recognized in each frame of the basic problem feature set is input to at least two micro-expression recognition models that are pre-trained for recognition, and the corresponding standard expression is obtained. Identifying the results includes the following steps:
  • S31 Input a video image to be recognized in each frame of the basic problem feature set into a face detection model for recognition, and obtain a standard face image.
  • the standard face image is a face image obtained by inputting a basic problem feature set into a face detection model for recognition. Specifically, the video image to be recognized in each of the basic problem feature sets is input into the face detection model, and the face position in the video image to be recognized is detected in each frame, thereby extracting the face image, that is, the standard face image, for subsequent Technical input is provided by the input of the model.
  • S32 Input a standard face image into the feature point detection model for recognition, and obtain a standard face feature point.
  • the standard face feature point is a feature coordinate point obtained by inputting a standard face image into the feature point detection model for recognition.
  • the facial feature points include five feature points including a left eye, a right eye, a nose tip, a left mouth corner, and a right mouth corner.
  • the standard face image is input to the feature point detection model for identification, and the feature point detection model obtains the coordinate positions of the above five feature points, and provides technical support for the input of the subsequent iris edge detection model.
  • S33 Input the standard face image into the emotion detection model for recognition, and obtain the first standard expression recognition result.
  • the first standard expression recognition result is a corresponding emotion recognition result obtained by inputting a standard face image into the emotion detection model for recognition.
  • the emotion detection model is capable of outputting probability values of seven emotions corresponding to the standard face image. These seven emotions include calm, anger, disgust, fear, happiness, sadness and surprise.
  • the standard face image is input to the emotion detection model for recognition, and the probability values of the seven emotions corresponding to the standard face image are obtained, and if the probability value of the certain emotion exceeds the corresponding preset threshold, the standard is obtained.
  • the emotion corresponding to the face image is the first standard expression recognition result, and provides technical support for subsequent risk control based on the first standard expression recognition result.
  • the second standard expression recognition result is a probability value of the head offset direction obtained by inputting the standard face image into the head posture model for recognition.
  • the head is offset in the six directions above, below, left, right, front, and back.
  • the standard face image is input to the head pose model for identification to obtain a probability value of the head offset direction. If the probability value of the head angle biased in a certain direction exceeds a corresponding preset threshold, the current determination is determined.
  • the face is offset in the corresponding direction.
  • the target customer's head posture can be well reflected by the target customer's eye line of sight direction or attention direction. For example, when asking a question, the target customer's head suddenly makes an abrupt movement. (such as sudden withdrawal or sudden tilt, etc.), then maybe he is lying. Therefore, by obtaining the target customer's head posture, technical support is provided for subsequent risk control, and the accuracy of risk control is improved.
  • S35 Input the standard face image into the blink detection model for recognition, and obtain the third standard expression recognition result.
  • the third standard expression recognition result is a recognition result that reflects the eye movement condition obtained by inputting the standard face image into the iris edge detection model for recognition.
  • the standard face image is input to the blink detection model for recognition, and the blink detection model outputs 0 (blink) or 1 (not blink) to represent whether the target user in the video image to be recognized in the frame blinks.
  • the number of blinks can reflect the current psychological activities (such as tension) of the target customers, and assist in the subsequent risk assessment of the target customers to further improve the accuracy of risk control.
  • S36 Input a standard facial feature point into the iris edge detection model for recognition, and obtain a fourth standard expression recognition result.
  • the fourth standard expression recognition result is a recognition result obtained by inputting the standard facial feature point into the iris edge detection model for recognizing the eye movement condition. Specifically, before the standard face feature point is input to the iris edge detection model for recognition, the eye region is first cut based on the human eye coordinate point in the face feature point, and then the iris region is detected by the iris edge detection model.
  • the center of the closed region formed based on the position of the iris edge point is the exact position of the center of the eye, and the center position of the eye is tracked relative to the position of the eyelid (corresponding to the central coordinate point of the eye through the feature point detection model)
  • the change of eyelid position can obtain the change of eye movement, and the obtained eye movement condition can be well reflected to provide technical support for subsequent risk control.
  • the standard expression recognition result includes a first standard expression recognition result, a second standard expression recognition result, a third standard expression recognition result, and a fourth standard expression recognition result.
  • the video image to be recognized of each frame in the basic problem feature set is first input into the face detection model for recognition, and the standard face image is obtained to remove other factors and improve the accuracy of the risk identification. Then, the standard face image is input into the feature point detection model for recognition, and five feature points of the face, that is, standard face feature points, are acquired, so that the standard face feature points are input into the iris edge detection model for recognition, and the target client is obtained.
  • the eye movement condition ie, the fourth standard expression recognition result
  • the standard face image is input into the emotion detection model for identification to obtain the probability value of the certain emotion corresponding to the target customer (ie, the first standard expression recognition result), and the technology for providing risk control based on the first standard expression recognition result is subsequently provided. stand by.
  • the standard face image is input into the head posture model for recognition to obtain the offset direction of the head posture (ie, the second standard expression recognition result), and the target customer's eye sight can be well reflected based on the target customer's head posture. Changes in direction or direction of attention provide technical support for subsequent risk control and improve the accuracy of risk control.
  • the standard face image is input into the blink detection model for identification to obtain the target customer's corresponding blink situation (ie, the third standard expression recognition result), so that the subsequent statistics blink time can reflect the target customer's current psychological activity (such as nervousness).
  • the target customer's current psychological activity such as nervousness
  • each frame of the to-be-identified video image in the sensitive problem feature set is input to at least two micro-expression recognition models that are pre-trained for recognition, and the corresponding test expression is obtained.
  • the identification result includes the following steps:
  • S41 Input a video image to be identified in each of the sensitive problem feature sets into a face detection model for recognition, and obtain a test face image.
  • the test face image is a face image obtained by inputting a sensitive problem feature set into a face detection model for recognition. Specifically, the video image to be identified in each of the sensitive problem feature sets is input into the face detection model, and the position of the face in the video image to be recognized in each frame is detected, and then the face image is extracted, that is, the test face image is taken as a follow-up Technical input is provided by the input of the model.
  • S42 Input the test face image into the feature point detection model for recognition, and obtain the test face feature point.
  • the test face feature point is a feature coordinate point obtained by inputting the test face image into the feature point detection model for recognition.
  • the test face feature points include five feature points including a left eye, a right eye, a nose tip, a left mouth corner, and a right mouth corner.
  • the test face image is input to the feature point detection model for identification, and the feature point detection model obtains the coordinate positions of the above five feature points, and provides technical support for the input of the subsequent iris edge detection model.
  • S43 Input the test face image into the emotion detection model for recognition, and obtain the first test expression recognition result.
  • the first test expression recognition result is a corresponding emotion recognition result obtained by inputting the test face image into the emotion detection model for recognition.
  • the emotion detection model is capable of outputting probability values of seven emotions corresponding to the test face image. These seven emotions include calm, anger, disgust, fear, happiness, sadness and surprise.
  • the test face image is input to the emotion detection model for recognition, and the probability values of the seven emotions corresponding to the test face image are obtained, and if the probability value of the certain emotion exceeds the corresponding preset threshold, the test is obtained.
  • the emotion corresponding to the face image ie, the first test expression recognition result
  • the second test expression recognition result is a probability value of the head offset direction obtained by inputting the test face picture to the head posture model for recognition.
  • the head is offset in the six directions above, below, left, right, front, and back.
  • the test face image is input to the head pose model for identification to obtain a probability value of the head offset direction. If the probability value of the head angle biased in a certain direction exceeds a corresponding preset threshold, determining the current The face is offset in the corresponding direction.
  • the target customer's eye line of sight direction or attention direction can be well reflected, and technical support for subsequent risk control is provided to improve the accuracy of risk control.
  • the third test expression recognition result is a recognition result that reflects the eye movement condition obtained by inputting the test face image into the iris edge detection model for recognition.
  • the test face image is input to the blink detection model for recognition, and the blink detection model outputs 0 (blink) or 1 (not blink) to represent whether the target user in the video image to be recognized in the frame blinks.
  • the number of blinks can reflect the current psychological activities (such as tension) of the target customers, and assist in the subsequent risk assessment of the target customers to further improve the accuracy of risk control.
  • the fourth test expression recognition result is a recognition result obtained by inputting the test face feature point into the iris edge detection model for detecting the eye movement condition. Specifically, before the test face feature point is input to the iris edge detection model for recognition, the eye region is first cut based on the human eye coordinate point in the face feature point, and then the iris region is detected by using the iris edge detection model.
  • the center of the closed region formed based on the position of the iris edge point is the exact position of the center of the eye, and the center position of the eye is tracked relative to the position of the eyelid (corresponding to the central coordinate point of the eye through the feature point detection model)
  • the change of eyelid position can obtain the change of eye movement, and the obtained eye movement condition can be well reflected to provide technical support for subsequent risk control.
  • the test expression recognition result includes a first test expression recognition result, a second test expression recognition result, a third test expression recognition result, and a fourth test expression recognition result.
  • each frame of the to-be-identified video image of the sensitive problem feature set is first input into the face detection model for recognition, and the test face image is acquired to remove other factors and improve the accuracy of the risk identification. Then, the test face image is input into the feature point detection model for recognition, and five feature points of the face are obtained, that is, test face feature points, so that the test face feature points are input into the iris edge detection model for identification, and the target client is obtained.
  • the eye movement condition ie, the fourth test expression recognition knot
  • the test face image is input into the emotion detection model for identification, so as to obtain the probability value of the certain emotion corresponding to the target customer (ie, the first test expression recognition result), and the technology for providing risk control based on the first test expression recognition result is subsequently provided.
  • Supporting; inputting the test face image into the head posture model for recognition to obtain the offset direction of the head posture (ie, the second test expression recognition result), and the target customer's head posture can well reflect the target customer's Changes in the direction of the eye's line of sight or direction of attention provide technical support for subsequent risk control and improve the accuracy of risk control.
  • the test face image is input into the blink detection model for identification to obtain the target customer's corresponding blink situation (ie, the third test expression recognition result), so that the subsequent statistics blink time can reflect the target customer's current psychological activity (such as nervousness).
  • the target customer's current psychological activity such as nervousness
  • step S30 or step S40 the face detection model is trained using CascadeCNN network.
  • CascadeCNN Feace Detection
  • Violajones is a face detection framework.
  • the CascadeCNN method is used to train the picture with the face position to obtain the face detection model, which improves the recognition efficiency of the face detection model.
  • the CascadeCNN method is used to train the picture with the face position as follows: the first stage of training, scanning the image with a 12-net network, rejecting more than 90% of the window, and inputting the remaining window to the 12-calibration-net
  • the network corrects and then corrects the corrected image using a non-maximum suppression algorithm to eliminate highly overlapping windows.
  • 12-net uses a 12 ⁇ 12 detection window, with a step size of 4, and slides on a W (wide) ⁇ H (high) picture to obtain a detection window.
  • the non-maximum suppression algorithm is a widely used method in the fields of target detection and localization. The essence of the algorithm principle is to search for local maxima and suppress non-maximum elements.
  • the window in the training data that is determined to be non-human face is used as a negative sample, and all real faces are exceeded (ie, The window of the preset threshold is used as a positive sample to obtain a corresponding detection window.
  • images are processed using a 24-net and 24-calibration-net network; 12-net and 24-net are networks that determine whether they are face regions.
  • the 12-calibration-net network and the 24-calibration-net network are correction networks.
  • the above-mentioned 24-net network is used for face detection on the training data, and the window determined as non-human face in the training data is taken as a negative sample, and all real faces are taken as positive samples.
  • the images input in the second stage of the training are processed by the 48-net and 48-calibration-net networks to complete the final stage of training to obtain corresponding face images from the video images to be identified.
  • the correction network is used to correct the area where the face is located, and the coordinates of the face area are obtained.
  • the actual face area is corrected according to each combination, and the corrected bounding box based on each combination has a score c n for the score above a certain set threshold (That is, when t), it is added to the original boundary, and the final result is averaged, which is the optimal bounding box.
  • a certain set threshold That is, when t
  • the three offset variables are as follows: Sn ⁇ (0.83, 0.91, 1.0, 1.10, 1.21), Xn ⁇ (-0.17, 0, 0.17), Yn ⁇ (-0.17, 0, 0.17), and the offset vector three
  • the parameters are corrected.
  • the specific correction formula is as follows:
  • step S30 or step S40 the feature point detection model is trained using DCNN network training.
  • DCNN Deep Convolutional Neural Network
  • the DCNN network is trained by taking pictures of the positions of the face features (the five features of the left eye, the right eye, the nose tip, the left mouth corner, and the right mouth corner) to obtain the feature point detection model.
  • the training process of the network is as follows: first select the training group, randomly select N samples from the training data as the training group, set the weight and the threshold to a random value close to 0, and initialize the learning rate; then, The training group is input into the DCNN network, and the predicted output of the network is obtained, and its real output is given; (x' denotes the predicted output; x denotes the true output corresponding to x'; i denotes the i-th feature; L denotes the length of the face frame) calculates the predicted output and the real output, obtains the output error, and sequentially based on the output error
  • the adjustment amount of each weight value and the adjustment amount of the threshold value are calculated, and the weight value and the threshold value in the DCNN model are respectively adjusted based on the adjustment amount of each weight value and the adjustment amount of the threshold value. After experiencing M iterations, it is judged whether the accuracy of the model satisfies the requirement, if it is not satisfied, the iteration is continued; if it is satisfied, the
  • step S30 or step S40 the emotion detection model is trained using the ResNet-80 network.
  • the ResNet-80 network refers to a network using the residual network idea, a total of 80 layers, which can be understood as an 80-layer residual network.
  • the Residual Network (ResNet) is a deep convolutional network.
  • an 80-layer residual network is used to train face images with seven emotions, and an emotion detection model is obtained to improve the accuracy of model recognition.
  • the seven emotions include calm, anger, disgust, fear, happiness, sadness and surprise.
  • the training step of training the face images marked with seven emotions by using the 80-layer deep convolution network is as follows: firstly, the face images (original training data) of the 7 emotions marked are normalized into 256*256 pixels. Then convert the face image and its corresponding image tag data into a unified format (such as "1" image tag data represents image data "angry") to obtain target training data, and randomly scramble for model training, so that the model can Based on the training data, the emotional characteristics are learned to improve the accuracy of model recognition. Then the target training data is input into the network, and the training is started. The value of the model parameters is adjusted by the gradient descent method. After several iterations until the test accuracy is stable at around 0.99, the training is stopped to obtain the emotion detection model.
  • the calculation formula of the gradient descent algorithm includes with Where ⁇ j represents the ⁇ value obtained for each iteration; h ⁇ (x) probability density function; x j represents the training data of the jth iteration; x (i) represents the positive and negative samples; y (i) represents the output result.
  • the gradient descent algorithm also known as the steepest descent algorithm, is the value of ⁇ when it is optimized by multiple iterations to obtain the value of the cost function J( ⁇ ), which is the required model parameter. Based on this model parameter, Obtaining the emotion detection model, the gradient descent algorithm is simple and easy to implement.
  • step S30 or step S40 the head posture detection model is trained using a 10-layer convolutional neural network.
  • CNN Convolutional Neural Network
  • the basic structure of CNN includes two layers, a convolution layer and a pooling layer.
  • the 10-layer convolutional neural network can achieve the training precision requirement in a short time.
  • the 10-layer convolutional neural network is used to train the data in the umdface database to obtain the head pose detection model, which greatly shortens the training time of the head pose model and improves the efficiency of model recognition.
  • the umdface database is an image database containing face information of different people, such as face frames and face poses.
  • the training process using a 10-layer convolutional neural network for training is as follows: A convolution operation (ie, feature extraction) is performed on the training data. Where * represents convolution; x j represents the jth input feature map; y j represents the jth output feature map; w ij is the convolution kernel between the ith input feature map and the jth output feature map ( Weight); b j represents the bias term of the jth output feature map. Then, the maximum pooled downsampling is used to downsample the convolved feature map to achieve dimensionality reduction on the feature map.
  • y j represents the ith output spectrum in the downsampling process (ie, the downsampled feature map), and each neuron in the downsampling process is from the ith input spectrum (the convolved feature map) It is obtained by local sampling of the downsampling frame of S*S; m and n respectively represent the step size of the moving of the downsampling frame.
  • step S30 or step S40 the blink detection model is trained using a logistic regression model.
  • Logistic Regression (LR) model is a classification model in machine learning.
  • the logistic regression model is trained using the image of the eye region marked with blinking and blinking in advance as training data.
  • the Sigmoid (S-type growth curve) function is chosen as the logic function.
  • the Sigmoid function is a common S-type function in biology. In information science, the Sigmoid function is often used due to its single addition and inverse function.
  • the function formula of the Sigmoid function is In which the Sigmoid function formula is substituted into the logistic regression hypothesis model, the above formula is Further, the cost function of the logistic regression model is Substituting Cost(h ⁇ (x), y) into the cost function yields the above formula, ie Since the logistic regression model is a two-class model, assuming that the probability of taking a positive class is p, then for an input, observe p/(1-p) to see if it is more likely to belong to a positive or negative class. The Sigmoid function can be very good. This reflects the characteristics of the logistic regression model, thus making the logistic regression model more efficient.
  • step S30 or step S40 the iris edge detection model is trained using a random forest algorithm.
  • the random forest is a classifier that uses multiple trees to train and predict samples (ie, training data).
  • the monocular picture of the iris area is marked with the preset color as the training data.
  • the implementation steps of the random forest are as follows: randomly select a pixel on the picture, and then continuously spread the surrounding pixel points, and then perform pixel point comparison. Since the iris is pre-labeled with a preset color, the color of the iris area is The color of the area is quite different, so the iris edge is considered to be the iris edge as long as it finds that the outermost periphery of one area is different from the other one of the other relatively large areas (for example, 20 pixels).
  • the human eye structure is composed of a portion such as a sclera, an iris, a pupillary lens, and a retina.
  • the iris is an annular portion between the black pupil lens and the white sclera, which contains many detailed features of interlaced spots, filaments, crowns, stripes, and crypts.
  • the training data is trained by the random forest algorithm to obtain the iris edge detection model, and the technical support for acquiring the position of the iris edge based on the iris edge detection model and acquiring the eye movement change is provided.
  • the CascadeCNN network training is used to train the picture with the face position to obtain the face detection model, which improves the recognition efficiency of the face detection model.
  • the deep convolutional neural network is trained by taking pictures of the positions of the face features (the left eye, the right eye, the nose tip, the left mouth corner and the right mouth corner) to obtain the feature point detection model and improve the feature point detection model recognition. Accuracy.
  • the 80-layer residual network is used to train the face images with seven emotions, and the emotion detection model is obtained to improve the accuracy of the emotion detection model recognition.
  • the 10-layer convolutional neural network is used to train the data in the umdface database to obtain the head pose detection model, which greatly shortens the training time of the head pose model and improves the efficiency of model recognition.
  • the logistic regression model is used to train the pre-labeled eye area image to obtain the blink detection model, which can well reflect the two-category problem (ie whether it blinks) and improve the accuracy of model recognition.
  • the random forest algorithm is used to train the monocular image of the iris area in the preset color to obtain the iris edge detection model, which is simple to implement and improves the training efficiency of the model.
  • the standard expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one standard emotion indicator.
  • the test expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one test emotion indicator.
  • standard emotional indicators include standard positive emotions and standard negative emotions.
  • Standard positive emotions are positive emotions expressed in the concentration of basic problem characteristics, such as happiness or rising of the mouth.
  • the standard negative emotion is the negative emotions presented in the basic problem feature set, such as anger or frown.
  • Testing emotional metrics includes testing positive emotions and testing negative emotions. Testing positive emotions is a positive emotion in the basic problem set, such as happy or rising. Testing negative emotions is a negative emotion, such as anger or frown, that is present in the basic problem feature set.
  • the standard expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one standard emotion indicator
  • the test expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one test emotion indicator
  • the standard emotion index of each frame of the to-be-identified video image in the basic problem feature set is counted, and the number of occurrences of the standard positive emotion or the standard negative emotion is taken as the first frequency in the standard expression recognition result corresponding to the basic problem feature set.
  • the standard emotion index of each frame of the to-be-identified video image in the feature set of the basic problem is counted, and the number of occurrences of each standard emotion indicator is determined to be the first frequency, which provides technical support for subsequent risk identification.
  • test emotion index of each frame of the to-be-identified video image in the sensitive problem feature set is counted, and the test expression recognition result corresponding to the sensitive problem feature set is obtained, and the number of occurrences of the positive emotion or the test negative emotion is tested as the second frequency.
  • the test emotion index of each frame of the to-be-identified video image in the feature set of the basic problem is counted, and the number of occurrences of each test emotion indicator is determined to be the second frequency, which provides technical support for subsequent risk identification.
  • S53 Obtain a risk identification result based on the first frequency and the second frequency.
  • the difference multiples of the first frequency and the second frequency are calculated to obtain a difference multiple of positive emotions or a difference multiple of negative emotions.
  • t 1 represents the first frequency (the frequency of occurrence of the standard positive emotion indicator or the frequency of the standard negative emotion indicator);
  • t 2 represents the second frequency (testing the frequency of occurrence of positive emotion indicators or testing the frequency of occurrence of negative emotion indicators). If it is necessary to obtain the difference multiple of the negative emotion, the test negative emotion indicator is divided by the standard negative emotion index to obtain the corresponding difference multiple, so that the difference multiple is compared with the first threshold, and if the difference multiple exceeds the first threshold, It is considered to be risky to obtain risk identification results.
  • the test positive emotion index is divided by the standard positive emotion to obtain the corresponding difference multiple, so as to compare the difference multiple with the second threshold, if the difference multiple exceeds the second threshold , identified as risky to obtain risk identification results.
  • the first threshold is set to 3 times and the second threshold is set to 2 times.
  • the obtaining the risk identification result further comprises: comparing the various benchmark data of the basic problem feature set with the test data of the sensitive problem feature set, and performing the one-to-one comparison to obtain the risk identification result.
  • the reference data is indicator data corresponding to the basic problem feature set, which includes blink, AU, mood, and head gesture.
  • the test data is indicator data corresponding to the feature set of the sensitive problem, including blink, AU, mood, and head gesture.
  • the number of occurrences of each basic indicator is compared with the number of occurrences of each test indicator. If the abnormality indicator exceeds a preset threshold (such as a first threshold or a second threshold), it is determined as a risk user.
  • a preset threshold such as a first threshold or a second threshold
  • the target customer is questioned by means of video chat to obtain the video data that the target customer replies, that is, the original video data, so that the credit review process is intelligent, labor cost is saved, and then, the original video data is framed and
  • the normalization process is performed to unify the pixels of the video image to be identified for each frame, so as to identify each video image to be identified for each frame, and improve the accuracy of the risk identification.
  • at least two frames of the to-be-recognized video image are divided into a proportional basic problem feature set and a sensitive problem feature set, so that the calculation is convenient when the recognition result is subsequently counted.
  • the video image to be recognized of each frame in the basic problem feature set is input into the face detection model for recognition, and the standard face image is obtained to remove other factors and improve the accuracy of the risk recognition.
  • the standard face image is input into the feature point detection model for recognition, and five feature points of the face (ie, standard face feature points) are acquired, so that the standard face feature points are input into the iris edge detection model for recognition and acquisition.
  • the eye movement of the target customer ie, the fourth standard expression recognition result
  • the standard face image is input into the emotion detection model for identification to obtain the probability value of the certain emotion corresponding to the target customer (ie, the first standard expression recognition result), and the technology for providing risk control based on the first standard expression recognition result is subsequently provided. stand by.
  • the standard face image is input into the head pose model for recognition to obtain the offset direction of the head (ie, the second standard expression recognition result), and the target client's head posture can be well reflected by the target client's eye posture. Sight direction or attention direction provides technical support for subsequent risk control and improves the accuracy of risk control.
  • the standard face image is input into the blink detection model for identification to obtain the target customer's corresponding blink situation (ie, the third standard expression recognition result), and the current psychological activity (such as tension) of the target customer can be reflected by the subsequent statistics blinking times.
  • the target customer's corresponding blink situation ie, the third standard expression recognition result
  • the current psychological activity such as tension
  • Fig. 6 is a block diagram showing the principle of a micro-expression based risk recognition apparatus corresponding to the micro-expression based risk recognition method in the first embodiment.
  • the micro-expression-based risk identification device includes a to-be-identified video data acquisition module 10, a to-be-identified video data division module 20, a standard expression recognition result acquisition module 30, a test expression recognition result acquisition module 40, and a risk recognition result.
  • Obtain module 50 The implementation functions of the to-be-identified video data acquisition module 10, the to-be-identified video data division module 20, the standard expression recognition result acquisition module 30, the test expression recognition result acquisition module 40, and the risk recognition result acquisition module 50 are based on the micro
  • the steps corresponding to the risk identification method of the expression correspond one-to-one. In order to avoid redundancy, the present embodiment will not be described in detail.
  • the to-be-identified video data acquiring module 10 is configured to acquire video data to be identified, and the to-be-identified video data includes at least two frames of the to-be-identified video image.
  • the to-be-identified video data dividing module 20 is configured to divide at least two frames of the to-be-identified video image into a basic problem feature set and a sensitive question feature set.
  • the standard expression recognition result obtaining module 30 is configured to input the video image to be recognized in each frame of the basic problem feature set into at least two micro-expression recognition models that are pre-trained for recognition, and obtain corresponding standard expression recognition results.
  • the test expression recognition result obtaining module 40 is configured to input the video image to be recognized of each frame in the sensitive problem feature set into at least two micro-expression recognition models that are pre-trained for recognition, and obtain a corresponding test expression recognition result.
  • the risk identification result obtaining module 50 is configured to obtain a risk identification result based on the standard expression recognition result and the test expression recognition result.
  • the to-be-identified video data acquiring module 10 includes an original video data acquiring unit 11 and a to-be-identified video data acquiring unit 12.
  • the original video data acquiring unit 11 is configured to acquire original video data.
  • the to-be-identified video data acquiring unit 12 performs frame-by-frame and normalization processing on the original video data to obtain video data to be identified.
  • the standard expression recognition result acquisition module 30 includes a standard face image acquisition unit 31, a standard face feature point acquisition unit 32, a first standard expression recognition result acquisition unit 33, a second standard expression recognition result acquisition unit 34, and a third.
  • the standard expression recognition result acquisition unit 35 and the fourth standard expression recognition result acquisition unit 36 are a standard face image acquisition unit 31 and a standard face feature point acquisition unit 32.
  • the standard face image obtaining unit 31 is configured to input the video image to be recognized of each frame in the basic problem feature set into the face detection model for recognition, and obtain a standard face image.
  • the standard face feature point acquiring unit 32 is configured to input a standard face image into the feature point detection model for recognition, and obtain a standard face feature point.
  • the first standard expression recognition result obtaining unit 33 is configured to input the standard face image into the emotion detection model for recognition, and obtain the first standard expression recognition result.
  • the second standard expression recognition result obtaining unit 34 is configured to input the standard face image into the head posture model for recognition, and obtain the second standard expression recognition result.
  • the third standard expression recognition result obtaining unit 35 is configured to input the standard face image into the iris edge detection model for recognition, and obtain the third standard expression recognition result.
  • the fourth standard expression recognition result obtaining unit 36 is configured to input the standard facial feature points into the blink detection model for recognition, and obtain the fourth standard expression recognition result.
  • the test expression recognition result acquisition module 40 includes a test face image acquisition unit 41, a test face feature point acquisition unit 42, a first test expression recognition result acquisition unit 43, a second test expression recognition result acquisition unit 44, and a third test.
  • the expression recognition result acquisition unit 45 and the fourth test expression recognition result acquisition unit 46 are included in the test expression recognition result acquisition module 40.
  • the test face image obtaining unit 41 is configured to input a video image to be recognized of each frame in the sensitive problem feature set into the face detection model for recognition, and obtain a test face image.
  • the test face feature point obtaining unit 42 is configured to input the test face image into the feature point detection model for recognition, and obtain the test face feature point.
  • the first test expression recognition result obtaining unit 43 is configured to input the test face image into the emotion detection model for recognition, and obtain the first test expression recognition result.
  • the second test expression recognition result obtaining unit 44 is configured to input the test face image into the head posture model for recognition, and obtain the second test expression recognition result.
  • the third test expression recognition result obtaining unit 45 is configured to input the test face image into the iris edge detection model for recognition, and obtain a third test expression recognition result.
  • the fourth test expression recognition result obtaining unit 46 is configured to input the test face feature point into the blink detection model for recognition, and obtain the fourth test expression recognition result.
  • the standard expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one standard emotion indicator.
  • the test expression recognition result corresponding to each frame of the to-be-identified video image corresponds to at least one test emotion indicator.
  • the risk identification result acquisition module 50 includes a first frequency acquisition unit 51, a second frequency acquisition unit 52, and a risk identification result acquisition unit 53.
  • the first frequency acquisition unit 51 determines, based on all the standard emotion recognition results, the number of occurrences of each of the standard emotion indicators as the first frequency.
  • the second frequency acquisition unit 52 is configured to determine, according to all the test emotion recognition results, the number of occurrences of each of the test emotion indicators as the second frequency.
  • the risk identification result obtaining unit 53 acquires the risk identification result based on the first frequency and the second frequency.
  • the embodiment provides one or more non-volatile readable storage media having computer readable instructions that, when executed by one or more processors, cause the one or more processors to execute The micro-expression-based risk identification method in Embodiment 1 is implemented. To avoid repetition, details are not described herein again. Alternatively, when the computer readable instructions are executed by one or more processors, causing the one or more processors to perform the functions of the modules/units in the micro-expression-based risk identification device of Embodiment 2, Avoid repetition, no more details here.
  • FIG. 7 is a schematic diagram of a computer device according to an embodiment of the present application.
  • computer device 70 of this embodiment includes a processor 71, a memory 72, and computer readable instructions 73 stored in memory 72 and executable on processor 71.
  • the processor 71 executes the steps in the various embodiments of the micro-expression-based risk identification method described above when executing the computer readable instructions 73, such as steps S10 through S50 shown in FIG.
  • the processor 81 implements the functions of the modules/units in the various apparatus embodiments described above when the computer readable instructions 73 are executed, such as the functions of the modules 10 through 50 shown in FIG.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Accounting & Taxation (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Finance (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

本申请公开了一种基于微表情的风险识别方法、装置、设备及介质。该基于微表情的风险识别方法包括：获取待识别视频数据，待识别视频数据包括至少两帧待识别视频图像。将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；将基本问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；将敏感问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；基于标准表情识别结果和测试表情识别结果，获取风险识别结果。该基于微表情的风险识别方法可有效解决目前风险控制的结果可信度不高，辅助效果不佳的问题。

Description

基于微表情的风险识别方法、装置、设备及介质
本专利申请以2018年3月30日提交的申请号为201810292475.0，名称为“基于微表情的风险识别方法、装置、设备及介质”的中国发明专利申请为基础，并要求其优先权。
技术领域
本申请涉及人脸识别领域，尤其涉及一种基于微表情的风险识别方法、装置、设备及介质。
背景技术
在金融行业，每一笔贷款资金的发放均需进行风险管控(即风险控制)，以确定能否给贷款人发放贷款。金融行业的传统风险控制方法中的关键一步是信审人与贷款人面对面的交流，以确定贷款人在办理贷款过程提供的资料的准确性，从而确定其贷款风险。但是在面对面的交流过程中，信审人可能因为注意力不集中或者对人的面部表情了解不深，忽略贷款人面部的一些细微的表情变化，这些细微的表情变化可能反映贷款人交流时的心理活动(如说谎)，使得信审人出具的风险控制结果因忽略贷款人信审过程的微表情而导致可信度不高。
发明内容
本申请实施例提供一种基于微表情的风险识别方法、装置、设备及介质，以解决当前因忽略贷款人微表情变化而导致风险控制结果可信度不高问题。
第一方面，本申请实施例提供一种基于微表情的风险识别方法，包括：
获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
第二方面，本申请实施例提供一种基于微表情的风险识别装置，包括：
待识别视频数据获取模块，用于获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
待识别视频数据划分模块，用于将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
标准表情识别结果获取模块，用于将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
测试表情识别结果获取模块，用于将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
风险识别结果获取模块，用于基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
第三方面，本申请实施例提供一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现如下步骤：
获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
第四方面，本申请实施例提供一个或多个存储有计算机可读指令的非易失性可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
本申请的一个或多个实施例的细节在下面的附图及描述中提出。本申请的其他特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例的描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例1中提供的基于微表情的风险识别方法的一流程图；
图2是图1中步骤S10的一具体示意图；
图3是图1中步骤S30的一具体示意图；
图4是图1中步骤S40的一具体示意图；
图5是图1中步骤S50的一具体示意图；
图6是本申请实施例2中提供的基于微表情的风险识别装置的一原理框图；
图7是本申请实施例4中提供的计算机设备的一示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
实施例1
图1示出本实施例中基于微表情的风险识别方法的流程图。该基于微表情的风险识别方法可应用在银行、证券、保险等金融机构上，能够有效辅助信审人对贷款人进行风险控制，从而确定能否给该贷款人发放贷款。如图1所示，该基于微表情的风险识别方法包括如下步骤：
S10：获取待识别视频数据，待识别视频数据包括至少两帧待识别视频图像。
其中，待识别视频数据是指对原始视频数据进行预处理后获取到的视频数据。其中，原始视频数据是用于记录贷款人在信审过程中的未经处理的视频数据。待识别视频数据是 由至少两帧待识别视频图像组成的视频数据。
本实施例中，由于后续在对待识别视频数据进行识别之前，需要针对目标客户所回复的视频数据(即原始视频数据)进行划分，因此，该待识别视频数据包括至少两帧待识别视频图像，以便判断每一帧待识别视频图像中的人脸的微表情特征，以确定用户是否在说谎，以便进行风险管控。
S20：将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集。
其中，基本问题特征集是指基于目标客户的一些个人信息所设置的基本问题的集合，例如身份证号、亲人手机号和家庭住址等。敏感问题特征集是用于判定目标客户是否存在风险的基本问题的集合，例如贷款用途、个人收入和偿还意愿等。
具体地，基本问题特征集和敏感问题特征集的划分是依据该问题是否存在标准答案的条件进行划分。以银行为例，若目标客户在银行、证券、保险等金融机构预存储了一些个人信息(如身份证号、亲人手机号和家庭住址等)，则基于这些预先存储有标准答案的个人信息所提出的问题进行回复所对应的待识别视频图像的集合作为基本问题特征集。而对于目标客户没有在银行、证券、保险等金融机构预存储的信息，则认为该部分信息没有标准答案，将基于该部分信息所提出的问题进行回复所对应的待识别视频图像的集合作为敏感问题特征集。
本实施例中，基本问题特征集包含至少一个帧待识别视频图像；敏感问题特征集包含至少一帧待识别视频图像，以便后续基于基本问题特征集的识别结果和敏感问题特征集的识别结果进行判断，从而达到风险控制的目的，提高风险识别的准确性，并且基本问题特征集中的视频帧数量与敏感问题特征集中的视频帧数量相同，以使后续对识别结果进行统计时，计算方便。
S30：将基本问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果。
其中，微表情识别模型是预先训练好的用于获取目标客户微表情特征的模型。标准表情识别结果是采用微表情识别模型对基本问题特征集中的每一帧待识别视频图像进行识别所获取到的识别结果。具体地，将基本问题特征集中的每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，以获取每个微表情识别模型输出的对应的标准表情识别结果，该标准表情识别结果在一定程度上反映目标客户说真话时的微表情，可以作为判断目标客户在回复敏感问题时是否说真话的评价依据。本实施例中，将基本问题特征集中的每一帧待识别视频图像输入到至少两个微表情识别模型进行识别，以获取对应的标准表情识别结果，以提高风险识别的准确率，使得辅助效果更佳。
S40：将敏感问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果。
其中，测试表情识别结果是采用微表情识别模型对敏感问题特征集中的每一帧待识别视频图像进行识别所获取到的识别结果。具体地，将敏感问题特征集中的每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，以获取每个微表情识别模型输出的对应的测试表情识别结果。该测试表情识别结果在一定程度上反映目标客户在回复敏感问题时说真话或说假话的微表情。本实施例中，将敏感问题特征集中每一帧待识别视频图像输入到至少两个微表情识别模型进行识别，以获取对应的测试表情识别结果，提高风险识别的准确率，使得辅助效果更佳。
S50：基于标准表情识别结果和测试表情识别结果，获取风险识别结果。
具体地，将基本问题特征集每一帧待识别视频图像对应的标准表情识别结果进行汇总作为基准数据。然后，将敏感问题特征集中的每一帧待识别视频图像对应的测试表情识别结果进行汇总作为测试数据，将基准数据与测试数据进行比对，若测试数据相对于基准数据差异的倍数与预设阈值进行比对，以获取风险等级，进而获取风险识别结果。
本实施例中，通过获取待识别视频数据，待识别视频数据包括至少两帧待识别视频图像，以便将至少两帧待识别视频图像划分成等比例的基本问题特征集和敏感问题特征集，以便后续对识别结果进行统计时，计算方便。然后，将基本问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；将敏感问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果，以提高风险识别的准确率，使得辅助效果更佳。最后，基于标准表情识别结果和测试表情识别结果，获取风险识别结果，以达到基于微表情的风险识别的目的，有效辅助信审人对贷款人进行风险控制。
在一具体实施方式中，如图2所示，步骤S10中，即获取待识别视频数据,具体包括如下步骤：
S11：获取原始视频数据。
其中，原始视频数据是用于记录贷款人在信审过程中的未经处理的视频数据。具体地，信审人可与目标客户(即贷款人)进行视频聊天，在视频聊天过程中基于预先设置的问题对目标客户进行提问，以获取目标客户回复问题的视频数据即原始视频数据。
S12：对原始视频数据进行分帧和归一化处理，获取待识别视频数据。
具体地，分帧处理是指按照预设时间对原始视频数据进行划分，以获取至少一帧的待识别视频图像。其中，归一化是一种简化计算的方式，即将有量纲的表达式，经过变换，化为无量纲的表达式，成为标量。例如本实施例中的原始视频数据中，需要有目标客户的面部区域，才能提取目标客户的微表情特征，因此需要将分帧后的待识别视频图像的像素归一化到260*260像素，统一像素，以便后续对每一帧待识别视频图像进行识别。
本实施例中，通过视频聊天的方式对目标客户进行提问，以获取目标客户回复的视频数据即原始视频数据，以使信审过程智能化，无需信审人与目标客户进行面对面交流，以节省人工成本。然后，对原始视频数据分帧和归一化处理，统一每一帧待识别视频图像的像素，以便后续对每一帧待识别视频图像进行识别，提高风险识别的准确率。
在一具体实施方式中，步骤S30中的微表情识别模型包括人脸检测模型、特征点检测模型、情绪检测模型、头部姿态检测模型、眨眼检测模型和虹膜边缘检测模型中的至少两个。
其中，人脸检测模型是用来提取每一帧待识别视频图像的人脸图片的模型。特征点检测模型是用来识别每一帧待识别视频图像中的人脸特征点的模型。头部姿态检测模型是用来识别每一帧待识别视频图像中目标客户的头部偏移方向的模型。眨眼检测模型是用来识别每一帧待识别视频图像中的目标客户是否眨眼的模型。虹膜边缘检测模型时用来反映每一帧待识别视频图像中的目标用户的眼动情况的模型。本实施例中，通过将基本问题特征集和敏感为题特征集分别输入到人脸检测模型、特征点检测模型、情绪检测模型、头部姿态检测模型、眨眼检测模型和虹膜边缘检测模型这七个模型中进行识别，以获取目标客户的标准表情识别结果和测试表情识别结果，并基于标准表情识别结果和测试表情识别结果实现基于微表情的风险识别的目的。
在一具体实施方式中，如图3所示，步骤S30中，即将基本问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果，具体包括如下步骤：
S31：将基本问题特征集中每一帧待识别视频图像输入到人脸检测模型进行识别，获取标准人脸图片。
其中，标准人脸图片是将基本问题特征集输入到人脸检测模型进行识别所得到的人脸图片。具体地，将基本问题特征集中每一帧待识别视频图像输入到人脸检测模型中，检测每一帧待识别视频图像中的人脸位置，进而提取人脸图片即标准人脸图片，为后续模型的输入提供技术支持。
S32：将标准人脸图片输入到特征点检测模型进行识别，获取标准人脸特征点。
其中，标准人脸特征点是将标准人脸图片输入到特征点检测模型进行识别所得到的特征坐标点。该人脸特征点包括左眼、右眼、鼻尖、左嘴角和右嘴角等五个特征点。具体地，将标准人脸图片输入到特征点检测模型进行识别，特征点检测模型会得出上述五个特征点的坐标位置，为后续虹膜边缘检测模型的输入提供技术支持。
S33：将标准人脸图片输入到情绪检测模型进行识别，获取第一标准表情识别结果。
其中，第一标准表情识别结果是将标准人脸图片输入到情绪检测模型进行识别所获取的对应的情绪识别结果。该情绪检测模型能够输出该标准人脸图片对应的七种情绪的概率值。这七种情绪包括平静、生气、厌恶、恐惧、高兴、难过和惊讶。具体地，将标准人脸图片输入到情绪检测模型进行识别，获取到该标准人脸图片对应的七种情绪的概率值，若某种情绪的概率值超过对应的预设阈值，则得到该标准人脸图片对应的情绪即为第一标准表情识别结果，为后续基于该第一标准表情识别结果进行风险控制提供技术支持。
S34：将标准人脸图片输入到头部姿态模型进行识别，获取第二标准表情识别结果。
其中，第二标准表情识别结果是将标准人脸图片输入到头部姿态模型进行识别所获取的头部偏移方向的概率值。头部偏移方向以上、下、左、右、前和后这六个方向来表示。具体地，将标准人脸图片输入到头部姿态模型进行识别，以获取头部偏移方向的概率值，若该头部角度偏向某一方向的概率值超过对应的预设阈值，则确定当前人脸向对应方向偏移。本实施例中，通过得出目标客户的头部姿态能够很好的反映目标客户的眼睛视线方向或注意力方向，例如当询问一个问题时，目标客户的头部突然做出了一个突兀的移动(如突然撤回或突然倾斜等)，那么可能他是在说谎。因此，通过得出目标客户的头部姿态，为后续进行风险控制提供技术支持，提高风险控制的准确率。
S35：将标准人脸图片输入到眨眼检测模型进行识别，获取第三标准表情识别结果。
其中，第三标准表情识别结果是将标准人脸图片输入到虹膜边缘检测模型进行识别所获取到的反映眼动情况的识别结果。具体地，将标准人脸图片输入到眨眼检测模型进行识别，眨眼检测模型会输出0(眨眼)或1(未眨眼)，以代表该帧待识别视频图像中目标用户是否眨眼。通过后续统计眨眼次数能够反映目标客户的当前的心理活动(如紧张)，为后续对目标客户做出风险评估做辅助，进一步提高风险控制的准确率。
S36：将标准人脸特征点输入到虹膜边缘检测模型进行识别，获取第四标准表情识别结果。
其中，第四标准表情识别结果是将标准人脸特征点输入到虹膜边缘检测模型进行识别所获取到的用来反映眼动情况的识别结果。具体地，将标准人脸特征点输入到虹膜边缘检测模型进行识别之前，先基于人脸特征点中的人眼坐标点，裁剪出眼睛区域，然后采用虹膜边缘检测模型对该眼睛区域进行检测，以获取虹膜边缘位置，则基于虹膜边缘点位置所形成的闭合区域的中心即为眼睛中心的准确位置，追踪眼睛中心位置相对于眼眶位置(通过特征点检测模型得到一眼球中心坐标点所对应的眼眶位置)的变化，即可得到眼动变化的情况，通过得到的眼动情况能够很好的反映为后续进行风险控制提供技术支持。
其中，标准表情识别结果包括第一标准表情识别结果、第二标准表情识别结果、第三标准表情识别结果和第四标准表情识别结果。
本实施例中，先将基本问题特征集中的每一帧待识别视频图像输入到人脸检测模型进行识别，获取标准人脸图片，以去除其他因素干扰，提高风险识别的准确率。然后，将标准人脸图片输入到特征点检测模型进行识别，获取人脸的五个特征点即标准人脸特征点，以便将标准人脸特征点输入到虹膜边缘检测模型进行识别，获取目标客户的眼动情况(即第四标准表情识别结果)，基于该眼动情况能够很好地为后续进行风险控制提供技术支持。将标准人脸图片输入到情绪检测模型进行识别，以获取目标客户对应的某种情绪的概率值(即第一标准表情识别结果)，为后续基于该第一标准表情识别结果进行风险控制提供技 术支持。将标准人脸图片输入到头部姿态模型进行识别，以获取头部姿态的偏移方向(即第二标准表情识别结果)，基于目标客户的头部姿态能够很好的反映目标客户的眼睛视线方向或注意力方向的变化，为后续进行风险控制提供技术支持，提高风险控制的准确率。将标准人脸图片输入到眨眼检测模型进行识别，以获取目标客户在对应的眨眼情况(即第三标准表情识别结果)，以便后续统计眨眼次数能够反映目标客户的当前的心理活动(如紧张)，为后续对目标客户做出风险评估做辅助，进一步提高风险控制的准确率。
在一具体实施方式中，如图4所示，步骤S40中，即将敏感问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果中，具体包括如下步骤：
S41：将敏感问题特征集中每一帧待识别视频图像输入到人脸检测模型进行识别，获取测试人脸图片。
其中，测试人脸图片是将敏感问题特征集输入到人脸检测模型进行识别所得到的人脸图片。具体地，将敏感问题特征集中每一帧待识别视频图像输入到人脸检测模型中，检测每一帧待识别视频图像中的人脸位置，进而提取人脸图片即测试人脸图片，为后续模型的输入提供技术支持。
S42：将测试人脸图片输入到特征点检测模型进行识别，获取测试人脸特征点。
其中，测试人脸特征点是将测试人脸图片输入到特征点检测模型进行识别所得到的特征坐标点。该测试人脸特征点包括左眼、右眼、鼻尖、左嘴角和右嘴角等五个特征点。具体地，将测试人脸图片输入到特征点检测模型进行识别，特征点检测模型会得出上述五个特征点的坐标位置，为后续虹膜边缘检测模型的输入提供技术支持。
S43：将测试人脸图片输入到情绪检测模型进行识别，获取第一测试表情识别结果。
其中，第一测试表情识别结果是将测试人脸图片输入到情绪检测模型进行识别所获取的对应的情绪识别结果。该情绪检测模型能够输出测试人脸图片对应的七种情绪的概率值。这七种情绪包括平静、生气、厌恶、恐惧、高兴、难过和惊讶。具体地，将测试人脸图片输入到情绪检测模型进行识别，获取到该测试人脸图片对应的七种情绪的概率值，若某种情绪的概率值超过对应的预设阈值，则得到该测试人脸图片对应的情绪(即第一测试表情识别结果)，为后续基于该第一测试表情识别结果进行风险控制提供技术支持。
S44：将测试人脸图片输入到头部姿态模型进行识别，获取第二测试表情识别结果。
其中，第二测试表情识别结果是将测试人脸图片输入到头部姿态模型进行识别所获取的头部偏移方向的概率值。头部偏移方向以上、下、左、右、前和后这六个方向来表示。具体地，将测试人脸图片输入到头部姿态模型进行识别，以获取头部偏移方向的概率值，若该头部角度偏向某一方向的概率值超过对应的预设阈值，则确定当前人脸向对应方向偏移。本实施例中，通过得出目标客户的头部姿态能够很好的反映目标客户的眼睛视线方向或注意力方向，为后续进行风险控制提供技术支持，提高风险控制的准确率。
S45：将测试人脸图片输入到眨眼检测模型进行识别，获取第三测试表情识别结果。
其中，第三测试表情识别结果是将测试人脸图片输入到虹膜边缘检测模型进行识别所获取到的反映眼动情况的识别结果。具体地，将测试人脸图片输入到眨眼检测模型进行识别，眨眼检测模型会输出0(眨眼)或1(未眨眼)代表该帧待识别视频图像中目标用户是否眨眼。通过后续统计眨眼次数能够反映目标客户的当前的心理活动(如紧张)，为后续对目标客户做出风险评估做辅助，进一步提高风险控制的准确率。
S46：将测试人脸特征点输入到虹膜边缘检测模型进行识别，获取第四测试表情识别结果。
其中，第四测试表情识别结果是将测试人脸特征点输入到虹膜边缘检测模型进行识别所获取到的用来反映眼动情况的识别结果。具体地，将测试人脸特征点输入到虹膜边缘检测模型进行识别之前，先基于人脸特征点中的人眼坐标点，裁剪出眼睛区域，然后采用虹 膜边缘检测模型对该眼睛区域进行检测，以获取虹膜边缘位置，则基于虹膜边缘点位置所形成的闭合区域的中心即为眼睛中心的准确位置，追踪眼睛中心位置相对于眼眶位置(通过特征点检测模型得到一眼球中心坐标点所对应的眼眶位置)的变化，即可得到眼动变化的情况，通过得到的眼动情况能够很好的反映为后续进行风险控制提供技术支持。
其中，测试表情识别结果包括第一测试表情识别结果、第二测试表情识别结果、第三测试表情识别结果和第四测试表情识别结果。
本实施例中，先将敏感问题特征集中的每一帧待识别视频图像输入到人脸检测模型进行识别，获取测试人脸图片，以去除其他因素干扰，提高风险识别的准确率。然后，将测试人脸图片输入到特征点检测模型进行识别，获取人脸的五个特征点即测试人脸特征点，以便将测试人脸特征点输入到虹膜边缘检测模型进行识别，获取目标客户的眼动情况(即第四测试表情识别结)果，基于该眼动情况能够很好的反映为后续进行风险控制提供技术支持。将测试人脸图片输入到情绪检测模型进行识别，以获取目标客户对应的某种情绪的概率值(即第一测试表情识别结果)，为后续基于该第一测试表情识别结果进行风险控制提供技术支持；将测试人脸图片输入到头部姿态模型进行识别，以获取头部姿态的偏移方向(即第二测试表情识别结果)，基于目标客户的头部姿态能够很好的反映目标客户的眼睛视线方向或注意力方向的变化，为后续进行风险控制提供技术支持，提高风险控制的准确率。将测试人脸图片输入到眨眼检测模型进行识别，以获取目标客户在对应的眨眼情况(即第三测试表情识别结果)，以便后续统计眨眼次数能够反映目标客户的当前的心理活动(如紧张)，为后续对目标客户做出风险评估做辅助，进一步提高风险控制的准确率。
在一具体实施方式中，步骤S30或步骤S40中，人脸检测模型采用CascadeCNN网络训练。
其中，CascadeCNN(人脸检测)是对经典的Violajones方法的深度卷积网络实现，是一种检测速度较快的人脸检测方法。Violajones是一种人脸检测框架。本实施例中，采用CascadeCNN方法对标注好人脸位置的图片进行训练，以获取人脸检测模型，提高了人脸检测模型的识别效率。
具体地，采用CascadeCNN方法对标注好人脸位置的图片进行训练的步骤如下：训练第一阶段，采用12-net网络扫描图像，并拒绝90％以上的窗口，将剩余窗口输入到12-calibration-net网络进行矫正，然后对采用非极大值抑制算法对矫正后的图像进行处理，以消除高度重叠窗口。其中，12-net是使用12×12的检测窗口，以步长为4，在W(宽)×H(高)的图片上滑动，得到检测窗口。非极大值抑制算法在目标检测和定位等领域是一种被广泛使用的方法，其算法原理的本质是搜索局部极大值并抑制非极大值元素。然后，利用上述的12-net网络对训练数据上作人脸检测，将训练数据中判为非人脸(即没有超过预设阈值的)的窗口作为负样本，将所有真实人脸(即超过预设阈值的)的窗口作为正样本，以获取对应的检测窗口。训练第二阶段，采用24-net和24-calibration-net网络对图像进行处理；其中，12-net和24-net是判断是否为人脸区的网络。12-calibration-net网络和24-calibration-net网络是矫正网络。最后，利用上述的24-net网络在训练数据上作人脸检测，将训练数据中判定为非人脸的窗口作为负样本，将所有真实人脸作为正样本。训练第三阶段，采用48-net和48-calibration-net网络对训练第二阶段输入的图像进行处理，以完成最后阶段的训练，以从待识别视频图像中获取对应的人脸图片。
具体地，矫正网络用于矫正人脸所在区域，得出人脸区域的坐标，其矫正的步骤如下：首先设定三个偏移变量：水平平移量(Xn)，垂直平移量(Yn)，宽高比缩放(Sn)。其中Xn设定3个值，Yn设定3个值，Sn设定5个值。根据对Xn，Yn，Sn的组合，一共能得出3x3x5＝45种组合。在数据集(训练数据)上将实际人脸区域，根据每一种组合矫正，基于每一种组合进行矫正后的边界框都有一个得分c n，对于得分高于某个设定的阈值(即t)时，将其 累加进原边界，最后结果取平均，就是最佳边界框。若三个偏移变量如下：Sn∈(0.83,0.91,1.0,1.10,1.21)、Xn∈(-0.17,0,0.17)、Yn∈(-0.17,0,0.17)，同时对偏移向量三个参数进行矫正，具体矫正公式如下：
Figure PCTCN2018094217-appb-000001
相应地，步骤S30或步骤S40中，特征点检测模型采用DCNN网络训练进行训练。
其中，DCNN(深度卷积神经网络)是一种深度卷积神经网络。本实施例中，采用标注好人脸特征(左眼、右眼、鼻尖、左嘴角和右嘴角等五个特征)位置的图片对DCNN网络进行训练，以获取特征点检测模型。
具体地，网络的训练过程如下：先选定训练组，从训练数据中随机选取N个样本作为训练组，将权值和阈值设置为接近于0的随机值，并初始化学习率；然后，将训练组输入到DCNN网络中，得到网络的预测输出，并给出它的真实输出；采用公式
Figure PCTCN2018094217-appb-000003
(x′表示预测输出；x表示x′对应的真实输出；i表示第i个特征；L表示人脸框的长度)对预测输出和真实输出进行计算，获取输出误差，并基于该输出误差依次计算出各权值的调整量和阈值的调整量，并基于各权值的调整量和阈值的调整量分别调整DCNN模型中的权值和阈值。当经历M次迭代后，判断模型的准确率是否满足要求，如果不满足，则继续迭代；如果满足，则训练结束，获取特征点检测模型。
相应地，步骤S30或步骤S40中，情绪检测模型采用ResNet-80网络进行训练。
其中，ResNet-80网络是指使用残差网络思想的网络，共80层，可以理解为80层的残差网络。残差网络(ResNet)是一种深度卷积网络。本实施例中，采用80层的残差网络对标注好七种情绪的人脸图片进行训练，获取情绪检测模型，提高模型识别的准确率。七种情绪包括平静、生气、厌恶、恐惧、高兴、难过和惊讶。
具体地，采用80层的深度卷积网络对标注好七种情绪的人脸图片进行训练的训练步骤如下：先将标注好的7种情绪的人脸图片(原始训练数据),归一化为256*256像素。然后将人脸图片及其对应图片标签数据转化成统一格式(如“1”图片标签数据代表图片数据“生气”)，以获取目标训练数据，并随机打乱，以便进行模型训练，使得模型能够基于该训练数据学习情绪特征，提高模型识别的准确率。然后将目标训练数据输入网络，开始训练,通过梯度下降法，调整模型参数的值，经过多次迭代直至测试精度稳定在0.99左右时，停止训练，以获取情绪检测模型。其中，梯度下降算法的计算公式包括
Figure PCTCN2018094217-appb-000004
和
Figure PCTCN2018094217-appb-000005
其中，θ j表示每次迭代得到的θ值；h θ(x)概率密度函数；x j表示第j次迭代的训练数据；x (i)表示正负样本；y (i)表示输出结果。梯度下降算法也称为最速下降算法，是对其进行多次迭代求导优化得到使代价函数J(θ)的值最小时的θ的值，即为所需的模型参数，基于此模型参数，获取情绪检测模型，梯度下降算法计算简单，容易实现。
相应地，步骤S30或步骤S40中，头部姿态检测模型采用10层的卷积神经网络进行训练。
其中，卷积神经网络(CNN)是一种多层神经网络，擅长处理图像尤其是大图像的相关机器学习问题。CNN的基本结构包括两层，卷积层和池化层。
本实施例中，由于神经网络的层数越多，计算时间越长，头部姿态区别度较高，采用10层卷积神经网络能够实现在较短时间内达到训练精度要求。采用10层卷积神经网络对umdface数据库中的数据进行训练，以获取头部姿态检测模型，大大缩短了头部姿态模型的训练时间，提高模型识别的效率。其中，umdface数据库是一种包含不同人的人脸信息(如人脸框和人脸姿势)的图像数据库。
具体地，采用10层的卷积神经网络进行训练的训练过程如下：采用公式
Figure PCTCN2018094217-appb-000006
对训练数据进行卷积运算(即特征提取)。其中，*代表卷积；x j代表第j个输入特征图；y j代表第j个输出特征图；w ij是第i个输入特征图与第j个输出特征图之间的卷积核(权重)；b j代表第j个输出特征图的偏置项。然后采用最大池化下采样对卷积后的特征图进行下采样操作以实现对特征图的降维，其计算公式为
Figure PCTCN2018094217-appb-000007
其中，y j表示下采样过程中的第i个输出谱(即下采样后的特征图)，下采样过程中的每一个神经元是从第i个输入谱(卷积后的特征图)中采用S*S的下采样框局部采样得到的；m与n分别表示下采样框移动的步长。
相应地，步骤S30或步骤S40中，眨眼检测模型采用逻辑回归模型进行训练。
其中，逻辑回归(Logistic Regression，LR)模型是机器学习中的一种分类模型。本实施例中，采用预先标注好眨眼和未眨眼的眼睛区域图片作为训练数据对逻辑回归模型进行训练。具体地，逻辑回归模型假设为h θ(x)＝g(θ mx)，其中g(θ mx)为逻辑函数，即某个数据属于某一类别(二分类问题)的概率。具体选用Sigmoid(S型生长曲线)函数作为逻辑函数，Sigmoid函数是一个在生物学中常见的S型的函数，在信息科学中，由于其单增以及反函数单增等性质，Sigmoid函数常被用作神经网络的阈值函数，将变量映射到0,1之间。Sigmoid函数的函数公式为
Figure PCTCN2018094217-appb-000008
其中将Sigmoid函数公式代入逻辑回归假设模型得到，上述公式即
Figure PCTCN2018094217-appb-000009
进一步地，逻辑回归模型的代价函数为
Figure PCTCN2018094217-appb-000010
将Cost(h θ(x),y)代入代价函数得到上述公式，即
Figure PCTCN2018094217-appb-000011
由于逻辑回归模型是二分类模型，假设取正类的概率为p，那么对一个输入，观察p/(1-p)就可以得出它更可能属于正类还是负类，Sigmoid函数可以很好的反映出逻辑回归模型的这种特点，因此使得逻辑回归模型训练的效率高。
相应地，步骤S30或步骤S40中，虹膜边缘检测模型采用随机森林算法进行训练。
其中，随机森林是是利用多棵树对样本(即训练数据)进行训练并预测的一种分类器。 本实施例中，采用预设颜色标注虹膜区域的单眼图片作为训练数据。具体地，随机森林的实现步骤如下：随机在图片上选取一个像素，再与其很接近的周围像素点不断扩散，然后进行像素点对比，由于预先用预设颜色标注虹膜，因此虹膜区域的颜色与其区域的颜色是截然不同的，因此，只要找到一个区域的最外围与其他周边一个相对较大的区域(例如20个像素)的颜色都不同，则认为是虹膜边缘。
具体地，人的眼睛结构由巩膜、虹膜、瞳孔晶状体和视网膜等部分组成。虹膜是位于黑色瞳孔晶状体和白色巩膜之间的圆环状部分，其包含有很多相互交错的斑点、细丝、冠状、条纹和隐窝等的细节特征。本实施例中，通过随机森林算法对训练数据进行训练，以获取虹膜边缘检测模型，为后续基于该虹膜边缘检测模型检测到虹膜边缘的位置，进而获取眼动变化提供技术支持。
本实施例中，通过采用CascadeCNN网络训练对标注好人脸位置的图片进行训练，以获取人脸检测模型，提高了人脸检测模型的识别效率。采用标注好人脸特征(左眼、右眼、鼻尖、左嘴角和右嘴角等五个特征)位置的图片对深度卷积神经网络进行训练，以获取特征点检测模型，提高特征点检测模型识别的准确率。采用80层的残差网络对标注好七种情绪的人脸图片进行训练，获取情绪检测模型，提高情绪检测模型识别的准确率。采用10层卷积神经网络对umdface数据库中的数据进行训练，以获取头部姿态检测模型，大大缩短了头部姿态模型的训练时间，提高模型识别的效率。采用逻辑回归模型对预先标注的眼睛区域图片进行训练，以获取眨眼检测模型，能够很好地反映二分类问题(即是否眨眼)，提高了模型识别的准确率。采用随机森林算法对预设颜色标注虹膜区域的单眼图片进行训练，以获取虹膜边缘检测模型，实现简单，提高了模型的训练效率。
在一具体实施方式中，每一帧待识别视频图像对应的标准表情识别结果对应至少一个标准情绪指标。每一帧待识别视频图像对应的测试表情识别结果对应至少一个测试情绪指标。
其中，标准情绪指标包括标准正面情绪和标准负面情绪。标准正面情绪是基本问题特征集中所呈现出的积极的情绪，如高兴或者嘴角上扬。标准负面情绪是基本问题特征集中所呈现出的负面的情绪，如愤怒或者皱眉。测试情绪指标包括测试正面情绪和测试负面情绪。测试正面情绪是基本问题特征集中所呈现出的积极的情绪，如高兴或者嘴角上扬。测试负面情绪是基本问题特征集中所呈现出的负面的情绪，如愤怒或者皱眉。
在一具体实施方式中，每一帧待识别视频图像对应的标准表情识别结果对应至少一个标准情绪指标；每一帧待识别视频图像对应的测试表情识别结果对应至少一个测试情绪指标；如图5所示，步骤S50中，即基于标准表情识别结果和测试表情识别结果，获取风险识别结果，具体包括如下步骤：
S51：基于所有的标准情绪识别结果，确定每一种标准情绪指标的出现次数为第一频次。
具体地，对基本问题特征集中的每一帧待识别视频图像的标准情绪指标进行统计，获取基本问题特征集对应的标准表情识别结果中，标准正面情绪或标准负面情绪的出现次数作为第一频次。本实施例中，统计基本问题特征集中的每一帧待识别视频图像的标准情绪指标，确定每一种标准情绪指标的出现次数为第一频次，为后续进行风险识别提供技术支持。
S52：基于所有的测试情绪识别结果，确定每一种测试情绪指标的出现次数为第二频次。
具体地，对敏感问题特征集中的每一帧待识别视频图像的测试情绪指标进行统计，获取敏感问题特征集对应的测试表情识别结果中，测试正面情绪或测试负面情绪的出现次数作为第二频次。本实施例中，统计基本问题特征集中的每一帧待识别视频图像的测试情绪指标，确定每一种测试情绪指标的出现次数为第二频次，为后续进行风险识别提供技术支持。
S53：基于第一频次和第二频次，获取风险识别结果。
具体地，采用公式
Figure PCTCN2018094217-appb-000012
对第一频次和第二频次的差异倍数进行计算，以获取正面情绪的差异倍数或者负面情绪的差异倍数。其中，t 1表示第一频次(标准正面情绪指标出现的频次或者标准负面情绪指标出现的频次)；t 2表示第二频次(测试正面情绪指标出现的频次或者测试负面情绪指标出现的频次)。若需要获取负面情绪的差异倍数时，将测试负面情绪指标与标准负面情绪指标相除即可获取其对应的差异倍数，以便将差异倍数与第一阈值进行比较，若差异倍数超过第一阈值，则认定为有风险，以获取风险识别结果。或者，若需要获取正面情绪的差异倍数时，将测试正面情绪指标与标准正面情绪相除即可获取其对应的差异倍数，以便将差异倍数与第二阈值进行比较，若差异倍数超过第二阈值，则认定为有风险，以获取风险识别结果。本实施例中，第一阈值设为3倍，第二阈值设为2倍。
进一步地，获取风险识别结果还包括如下方式：通过统计基本问题特征集的各项基准数据与敏感问题特征集的各项测试数据，进行一一比对，以获取风险识别结果。具体地，基准数据是基本问题特征集对应的指标数据，其包括眨眼、AU、情绪和头部姿态等。测试数据是敏感问题特征集对应的指标数据，其包括眨眼、AU、情绪和头部姿态等。最后，统计每一基本指标出现的次数与每一测试指标出现的次数进行比较，若出现异常指标超过预设阈值(如第一阈值或第二阈值)，则认定为风险用户。
本实施例中，通过视频聊天的方式对目标客户进行提问，以获取目标客户回复的视频数据即原始视频数据，以使信审过程智能化，节省人工成本，然后，对原始视频数据分帧和归一化处理，统一每一帧待识别视频图像的像素，以便后续对每一帧待识别视频图像进行识别，提高风险识别的准确率。然后，将至少两帧待识别视频图像划分成等比例的基本问题特征集和敏感问题特征集，以便后续对识别结果进行统计时，计算方便。将将基本问题特征集中的每一帧待识别视频图像输入到人脸检测模型进行识别，获取标准人脸图片，以去除其他因素干扰，提高风险识别的准确率。然后，将标准人脸图片输入到特征点检测模型进行识别，获取人脸的五个特征点(即标准人脸特征点)，以便将标准人脸特征点输入到虹膜边缘检测模型进行识别，获取目标客户的眼动情况(即第四标准表情识别结果)，以便通过得到的眼动情况为后续进行风险控制提供技术支持。将标准人脸图片输入到情绪检测模型进行识别，以获取目标客户对应的某种情绪的概率值(即第一标准表情识别结果)，为后续基于该第一标准表情识别结果进行风险控制提供技术支持。将标准人脸图片输入到头部姿态模型进行识别，以获取头部的偏移方向(即第二标准表情识别结果)，通过得出目标客户的头部姿态能够很好的反映目标客户的眼睛视线方向或注意力方向，为后续进行风险控制提供技术支持，提高风险控制的准确率。将标准人脸图片输入到眨眼检测模型进行识别，以获取目标客户在对应的眨眼情况(即第三标准表情识别结果)，通过后续统计眨眼次数能够反映目标客户的当前的心理活动(如紧张)，为后续对目标客户做出风险评估做辅助，进一步提高风险控制的准确率。最后，基于标准表情识别结果确定每一种标准情绪指标的出现次数为第一频次；基于所有的测试情绪识别结果，确定每一种测试情绪指标的出现次数为第二频次，通过计算第一频次和第二频次的差异倍数，通过将差异数据与第一阈值或第二阈值进行比较，获取风险识别结果，以达到基于微表情的风险识别的目的，有效辅助信审人对贷款人进行风险控制。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
实施例2
图6示出与实施例1中基于微表情的风险识别方法一一对应的基于微表情的风险识别 装置的原理框图。如图6所示，该基于微表情的风险识别装置包括待识别视频数据获取模块10、待识别视频数据划分模块20、标准表情识别结果获取模块30、测试表情识别结果获取模块40和风险识别结果获取模块50。其中，待识别视频数据获取模块10、待识别视频数据划分模块20、标准表情识别结果获取模块30、测试表情识别结果获取模块40和风险识别结果获取模块50的实现功能与实施例1中基于微表情的风险识别方法对应的步骤一一对应，为避免赘述，本实施例不一一详述。
待识别视频数据获取模块10，用于获取待识别视频数据，待识别视频数据包括至少两帧待识别视频图像。
待识别视频数据划分模块20，用于将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集。
标准表情识别结果获取模块30，用于将基本问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果。
测试表情识别结果获取模块40，用于将敏感问题特征集中每一帧待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果。
风险识别结果获取模块50，用于基于标准表情识别结果和测试表情识别结果，获取风险识别结果。
优选地，待识别视频数据获取模块10包括原始视频数据获取单元11和待识别视频数据获取单元12。
原始视频数据获取单元11，用于获取原始视频数据。
待识别视频数据获取单元12，对原始视频数据进行分帧和归一化处理，获取待识别视频数据。
优选地，标准表情识别结果获取模块30包括标准人脸图片获取单元31、标准人脸特征点获取单元32、第一标准表情识别结果获取单元33、第二标准表情识别结果获取单元34、第三标准表情识别结果获取单元35和第四标准表情识别结果获取单元36。
标准人脸图片获取单元31，用于将基本问题特征集中每一帧待识别视频图像输入到人脸检测模型进行识别，获取标准人脸图片。
标准人脸特征点获取单元32，用于将标准人脸图片输入到特征点检测模型进行识别，获取标准人脸特征点。
第一标准表情识别结果获取单元33，用于将标准人脸图片输入到情绪检测模型进行识别，获取第一标准表情识别结果。
第二标准表情识别结果获取单元34，用于将标准人脸图片输入到头部姿态模型进行识别，获取第二标准表情识别结果。
第三标准表情识别结果获取单元35，用于将标准人脸图片输入到虹膜边缘检测模型进行识别，获取第三标准表情识别结果。
第四标准表情识别结果获取单元36，用于将标准人脸特征点输入到眨眼检测模型进行识别，获取第四标准表情识别结果。
优选地，测试表情识别结果获取模块40包括测试人脸图片获取单元41测试人脸特征点获取单元42、第一测试表情识别结果获取单元43、第二测试表情识别结果获取单元44、第三测试表情识别结果获取单元45和第四测试表情识别结果获取单元46。
测试人脸图片获取单元41，用于将敏感问题特征集中每一帧待识别视频图像输入到人脸检测模型进行识别，获取测试人脸图片。
测试人脸特征点获取单元42，用于将测试人脸图片输入到特征点检测模型进行识别，获取测试人脸特征点。
第一测试表情识别结果获取单元43，用于将测试人脸图片输入到情绪检测模型进行识别，获取第一测试表情识别结果。
第二测试表情识别结果获取单元44，用于将测试人脸图片输入到头部姿态模型进行识别，获取第二测试表情识别结果。
第三测试表情识别结果获取单元45，用于将测试人脸图片输入到虹膜边缘检测模型进行识别，获取第三测试表情识别结果。
第四测试表情识别结果获取单元46，用于将测试人脸特征点输入到眨眼检测模型进行识别，获取第四测试表情识别结果。
每一帧待识别视频图像对应的标准表情识别结果对应至少一个标准情绪指标。每一帧待识别视频图像对应的测试表情识别结果对应至少一个测试情绪指标。
优选地，风险识别结果获取模块50包括第一频次获取单元51、第二频次获取单元52和风险识别结果获取单元53。
第一频次获取单元51，基于所有的标准情绪识别结果，确定每一种标准情绪指标的出现次数为第一频次。
第二频次获取单元52，用于基于所有的测试情绪识别结果，确定每一种测试情绪指标的出现次数为第二频次。
风险识别结果获取单元53，基于第一频次和第二频次，获取风险识别结果。
实施例3
本实施例提供一个或多个存储有计算机可读指令的非易失性可读存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行时实现实施例1中基于微表情的风险识别方法，为避免重复，这里不再赘述。或者，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行时实现实施例2中基于微表情的风险识别装置中各模块/单元的功能，为避免重复，这里不再赘述。
实施例4
图7是本申请一实施例提供的计算机设备的示意图。如图7所示，该实施例的计算机设备70包括：处理器71、存储器72以及存储在存储器72中并可在处理器71上运行的计算机可读指令73。处理器71执行计算机可读指令73时实现上述各个基于微表情的风险识别方法实施例中的步骤，例如图1所示的步骤S10至S50。或者，处理器81执行计算机可读指令73时实现上述各装置实施例中各模块/单元的功能，例如图6所示模块10至50的功能。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种基于微表情的风险识别方法，其特征在于，包括：
    获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
    将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
    将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
    基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
  2. 如权利要求1所述的基于微表情的风险识别方法，其特征在于，所述获取待识别视频数据，包括：
    获取原始视频数据；
    对所述原始视频数据进行分帧和归一化处理，获取所述待识别视频数据。
  3. 如权利要求1所述的基于微表情的风险识别方法，其特征在于，所述微表情识别模型包括人脸检测模型、特征点检测模型、情绪检测模型、头部姿态检测模型、眨眼检测模型和虹膜边缘检测模型。
  4. 如权利要求1所述的基于微表情的风险识别方法，其特征在于，所述将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果，包括：
    将所述基本问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取标准人脸图片；
    将所述标准人脸图片输入到所述特征点检测模型进行识别，获取标准人脸特征点；
    将所述标准人脸图片输入到所述情绪检测模型进行识别，获取第一标准表情识别结果；
    将所述标准人脸图片输入到所述头部姿态模型进行识别，获取第二标准表情识别结果；
    将所述标准人脸图片输入到所述眨眼检测模型进行识别，获取第三标准表情识别结果；
    将所述标准人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四标准表情识别结果；
    其中，所述标准表情识别结果包括所述第一标准表情识别结果、所述第二标准表情识别结果、所述第三标准表情识别结果和所述第四标准表情识别结果。
  5. 如权利要求1所述的基于微表情的风险识别方法，其特征在于，所述将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果，包括：
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取测试人脸图片；
    将所述测试人脸图片输入到所述特征点检测模型进行识别，获取测试人脸特征点；
    将所述测试人脸图片输入到所述情绪检测模型进行识别，获取第一测试表情识别结果；
    将所述测试人脸图片输入到所述头部姿态模型进行识别，获取第二测试表情识别结果；
    将所述测试人脸图片输入到所述眨眼检测模型进行识别，获取第三测试表情识别结果；
    将所述测试人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四测试表情识别结果；
    其中，所述测试表情识别结果包括所述第一测试表情识别结果、所述第二测试表情识别结果、所述第三测试表情识别结果和所述第四测试表情识别结果。
  6. 如权利要求3-5中的任一项所述的基于微表情的风险识别方法，其特征在于，所述人脸检测模型具体为采用CascadeCNN网络训练得到的人脸检测模型；
    所述特征点检测模型采用DCNN网络训练进行训练；
    所述情绪检测模型采用ResNet-80网络进行训练；
    所述头部姿态检测模型采用10层的卷积神经网络进行训练；
    所述眨眼检测模型采用逻辑回归模型进行训练；
    所述虹膜边缘检测模型采用随机森林算法进行训练。
  7. 如权利要求1所述的基于微表情的风险识别方法，其特征在于，每一帧所述待识别视频图像对应的所述标准表情识别结果对应至少一个标准情绪指标；
    每一帧所述待识别视频图像对应的所述测试表情识别结果对应至少一个测试情绪指标；
    所述基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果，包括：
    基于所有的所述标准情绪识别结果，确定每一种所述标准情绪指标的出现次数为第一频次；
    基于所有的所述测试情绪识别结果，确定每一种所述测试情绪指标的出现次数为第二频次；
    基于所述第一频次和所述第二频次，获取风险识别结果。
  8. 一种基于微表情的风险识别装置，其特征在于，包括：
    待识别视频数据获取模块，用于获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
    待识别视频数据划分模块，用于将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
    标准表情识别结果获取模块，用于将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
    测试表情识别结果获取模块，用于将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
    风险识别结果获取模块，用于基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
  9. 一种计算机设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机可读指令，其特征在于，所述处理器执行所述计算机可读指令时实现如下步骤：
    获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
    将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
    将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
    基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
  10. 如权利要求9所述的计算机设备，其特征在于，所述微表情识别模型包括人脸检 测模型、特征点检测模型、情绪检测模型、头部姿态检测模型、眨眼检测模型和虹膜边缘检测模型。
  11. 如权利要求9所述的计算机设备，其特征在于，所述将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果，包括：
    将所述基本问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取标准人脸图片；
    将所述标准人脸图片输入到所述特征点检测模型进行识别，获取标准人脸特征点；
    将所述标准人脸图片输入到所述情绪检测模型进行识别，获取第一标准表情识别结果；
    将所述标准人脸图片输入到所述头部姿态模型进行识别，获取第二标准表情识别结果；
    将所述标准人脸图片输入到所述眨眼检测模型进行识别，获取第三标准表情识别结果；
    将所述标准人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四标准表情识别结果；
    其中，所述标准表情识别结果包括所述第一标准表情识别结果、所述第二标准表情识别结果、所述第三标准表情识别结果和所述第四标准表情识别结果。
  12. 如权利要求9所述的计算机设备，其特征在于，所述将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果，包括：
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取测试人脸图片；
    将所述测试人脸图片输入到所述特征点检测模型进行识别，获取测试人脸特征点；
    将所述测试人脸图片输入到所述情绪检测模型进行识别，获取第一测试表情识别结果；
    将所述测试人脸图片输入到所述头部姿态模型进行识别，获取第二测试表情识别结果；
    将所述测试人脸图片输入到所述眨眼检测模型进行识别，获取第三测试表情识别结果；
    将所述测试人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四测试表情识别结果；
    其中，所述测试表情识别结果包括所述第一测试表情识别结果、所述第二测试表情识别结果、所述第三测试表情识别结果和所述第四测试表情识别结果。
  13. 如权利要求10-12所述的计算机设备，其特征在于，所述人脸检测模型具体为采用CascadeCNN网络训练得到的人脸检测模型；
    所述特征点检测模型采用DCNN网络训练进行训练；
    所述情绪检测模型采用ResNet-80网络进行训练；
    所述头部姿态检测模型采用10层的卷积神经网络进行训练；
    所述眨眼检测模型采用逻辑回归模型进行训练；
    所述虹膜边缘检测模型采用随机森林算法进行训练。
  14. 如权利要求9所述的计算机设备，其特征在于，每一帧所述待识别视频图像对应的所述标准表情识别结果对应至少一个标准情绪指标；
    每一帧所述待识别视频图像对应的所述测试表情识别结果对应至少一个测试情绪指标；
    所述基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果，包括：
    基于所有的所述标准情绪识别结果，确定每一种所述标准情绪指标的出现次数为第一频次；
    基于所有的所述测试情绪识别结果，确定每一种所述测试情绪指标的出现次数为第二频次；
    基于所述第一频次和所述第二频次，获取风险识别结果。
  15. 一个或多个存储有计算机可读指令的非易失性可读存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如下步骤：
    获取待识别视频数据，所述待识别视频数据包括至少两帧待识别视频图像；
    将至少两帧待识别视频图像划分成基本问题特征集和敏感问题特征集；
    将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果；
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果；
    基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果。
  16. 如权利要求15所述的非易失性可读存储介质，其特征在于，所述微表情识别模型包括人脸检测模型、特征点检测模型、情绪检测模型、头部姿态检测模型、眨眼检测模型和虹膜边缘检测模型。
  17. 如权利要求15所述的非易失性可读存储介质，其特征在于，所述将所述基本问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的标准表情识别结果，包括：
    将所述基本问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取标准人脸图片；
    将所述标准人脸图片输入到所述特征点检测模型进行识别，获取标准人脸特征点；
    将所述标准人脸图片输入到所述情绪检测模型进行识别，获取第一标准表情识别结果；
    将所述标准人脸图片输入到所述头部姿态模型进行识别，获取第二标准表情识别结果；
    将所述标准人脸图片输入到所述眨眼检测模型进行识别，获取第三标准表情识别结果；
    将所述标准人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四标准表情识别结果；
    其中，所述标准表情识别结果包括所述第一标准表情识别结果、所述第二标准表情识别结果、所述第三标准表情识别结果和所述第四标准表情识别结果。
  18. 如权利要求15所述的非易失性可读存储介质，其特征在于，所述将所述敏感问题特征集中每一帧所述待识别视频图像输入到预先训练好的至少两个微表情识别模型进行识别，获取对应的测试表情识别结果，包括：
    将所述敏感问题特征集中每一帧所述待识别视频图像输入到所述人脸检测模型进行识别，获取测试人脸图片；
    将所述测试人脸图片输入到所述特征点检测模型进行识别，获取测试人脸特征点；
    将所述测试人脸图片输入到所述情绪检测模型进行识别，获取第一测试表情识别结果；
    将所述测试人脸图片输入到所述头部姿态模型进行识别，获取第二测试表情识别结果；
    将所述测试人脸图片输入到所述眨眼检测模型进行识别，获取第三测试表情识别结 果；
    将所述测试人脸特征点输入到所述虹膜边缘检测模型进行识别，获取第四测试表情识别结果；
    其中，所述测试表情识别结果包括所述第一测试表情识别结果、所述第二测试表情识别结果、所述第三测试表情识别结果和所述第四测试表情识别结果。
  19. 如权利要求16-18所述的非易失性可读存储介质，其特征在于，所述人脸检测模型具体为采用CascadeCNN网络训练得到的人脸检测模型；
    所述特征点检测模型采用DCNN网络训练进行训练；
    所述情绪检测模型采用ResNet-80网络进行训练；
    所述头部姿态检测模型采用10层的卷积神经网络进行训练；
    所述眨眼检测模型采用逻辑回归模型进行训练；
    所述虹膜边缘检测模型采用随机森林算法进行训练。
  20. 如权利要求15所述的非易失性可读存储介质，其特征在于，每一帧所述待识别视频图像对应的所述标准表情识别结果对应至少一个标准情绪指标；
    每一帧所述待识别视频图像对应的所述测试表情识别结果对应至少一个测试情绪指标；
    所述基于所述标准表情识别结果和所述测试表情识别结果，获取风险识别结果，包括：
    基于所有的所述标准情绪识别结果，确定每一种所述标准情绪指标的出现次数为第一频次；
    基于所有的所述测试情绪识别结果，确定每一种所述测试情绪指标的出现次数为第二频次；
    基于所述第一频次和所述第二频次，获取风险识别结果。
PCT/CN2018/094217 2018-03-30 2018-07-03 基于微表情的风险识别方法、装置、设备及介质 WO2019184125A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810292475.0A CN108537160A (zh) 2018-03-30 2018-03-30 基于微表情的风险识别方法、装置、设备及介质
CN201810292475.0 2018-03-30

Publications (1)

Publication Number Publication Date
WO2019184125A1 true WO2019184125A1 (zh) 2019-10-03

Family

ID=63482484

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094217 WO2019184125A1 (zh) 2018-03-30 2018-07-03 基于微表情的风险识别方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN108537160A (zh)
WO (1) WO2019184125A1 (zh)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110781810A (zh) * 2019-10-24 2020-02-11 合肥盛东信息科技有限公司 一种人脸情绪识别方法
CN111062074A (zh) * 2019-12-11 2020-04-24 同济大学 建筑空间品质虚拟模拟及智能评价方法
CN111274447A (zh) * 2020-01-13 2020-06-12 深圳壹账通智能科技有限公司 基于视频的目标表情生成方法、装置、介质、电子设备
CN111783620A (zh) * 2020-06-29 2020-10-16 北京百度网讯科技有限公司 表情识别方法、装置、设备及存储介质
CN111860154A (zh) * 2020-06-12 2020-10-30 歌尔股份有限公司 一种基于视觉的额头检测方法、装置和电子设备
CN111950449A (zh) * 2020-08-11 2020-11-17 合肥工业大学 基于走路姿态的情绪识别方法
CN112329663A (zh) * 2020-11-10 2021-02-05 西南大学 一种基于人脸图像序列的微表情时刻检测方法及装置
CN112381036A (zh) * 2020-11-26 2021-02-19 厦门大学 一种应用于刑侦的微表情与宏表情片段识别方法
CN112417414A (zh) * 2020-12-04 2021-02-26 支付宝(杭州)信息技术有限公司 一种基于属性脱敏的隐私保护方法、装置以及设备
CN112487904A (zh) * 2020-11-23 2021-03-12 成都尽知致远科技有限公司 一种基于大数据分析的视频图像处理方法及系统
CN112699774A (zh) * 2020-12-28 2021-04-23 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113191196A (zh) * 2021-04-01 2021-07-30 北京睿芯高通量科技有限公司 一种新型智能安防系统中的轨迹分析方法及系统
CN113276827A (zh) * 2021-05-26 2021-08-20 朱芮叶 电动汽车能源回收系统控制方法、系统以及汽车
CN113313048A (zh) * 2021-06-11 2021-08-27 北京百度网讯科技有限公司 脸部表情识别方法和装置
CN113822229A (zh) * 2021-10-28 2021-12-21 重庆科炬企业孵化器有限公司 一种面向表情识别的用户体验评价建模方法及装置
CN113901915A (zh) * 2021-10-08 2022-01-07 无锡锡商银行股份有限公司 一种轻量网络与MagFace在视频中的表情检测方法
CN114287938A (zh) * 2021-12-13 2022-04-08 重庆大学 建筑环境中人体参数的安全区间获得方法和设备
CN114973362A (zh) * 2022-05-20 2022-08-30 厦门大学 一种应用于社交机器人的动态延长编码微表情识别方法
CN116824280A (zh) * 2023-08-30 2023-09-29 安徽爱学堂教育科技有限公司 基于微表情变化的心理预警方法

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109472206B (zh) * 2018-10-11 2023-07-07 平安科技（深圳）有限公司 基于微表情的风险评估方法、装置、设备及介质
CN109472269A (zh) * 2018-10-17 2019-03-15 深圳壹账通智能科技有限公司 图像特征配置及校验方法、装置、计算机设备及介质
CN109635838B (zh) * 2018-11-12 2023-07-11 平安科技（深圳）有限公司 人脸样本图片标注方法、装置、计算机设备及存储介质
CN111241887B (zh) * 2018-11-29 2024-04-16 北京市商汤科技开发有限公司 目标对象关键点识别方法及装置、电子设备和存储介质
CN109784170B (zh) * 2018-12-13 2023-11-17 平安科技（深圳）有限公司 基于图像识别的车险定损方法、装置、设备及存储介质
CN109711297A (zh) * 2018-12-14 2019-05-03 深圳壹账通智能科技有限公司 基于面部图片的风险识别方法、装置、计算机设备及存储介质
CN109461073A (zh) * 2018-12-14 2019-03-12 深圳壹账通智能科技有限公司 智能识别的风险管理方法、装置、计算机设备及存储介质
CN109584050A (zh) * 2018-12-14 2019-04-05 深圳壹账通智能科技有限公司 基于微表情识别的用户风险程度分析方法及装置
CN109766419A (zh) * 2018-12-14 2019-05-17 深圳壹账通智能科技有限公司 基于语音分析的产品推荐方法、装置、设备及存储介质
CN109509088A (zh) * 2018-12-15 2019-03-22 深圳壹账通智能科技有限公司 基于微表情识别的贷款审核方法、装置、设备及介质
CN109766461A (zh) * 2018-12-15 2019-05-17 深圳壹账通智能科技有限公司 基于微表情的照片管理方法、装置、计算机设备及介质
CN109697665A (zh) * 2018-12-15 2019-04-30 深圳壹账通智能科技有限公司 基于人工智能的贷款审核方法、装置、设备及介质
CN109509087A (zh) * 2018-12-15 2019-03-22 深圳壹账通智能科技有限公司 智能化的贷款审核方法、装置、设备及介质
CN109767317A (zh) * 2018-12-15 2019-05-17 深圳壹账通智能科技有限公司 基于会员等级评定的贷款审核方法、装置、设备及介质
CN109829362A (zh) * 2018-12-18 2019-05-31 深圳壹账通智能科技有限公司 安检辅助分析方法、装置、计算机设备和存储介质
CN109784185A (zh) * 2018-12-18 2019-05-21 深圳壹账通智能科技有限公司 基于微表情识别的客户餐饮评价自动获取方法及装置
CN109754312A (zh) * 2018-12-18 2019-05-14 深圳壹账通智能科技有限公司 产品推送方法、装置、计算机设备和存储介质
CN109767290A (zh) * 2018-12-18 2019-05-17 深圳壹账通智能科技有限公司 产品推送方法、装置、计算机设备和存储介质
CN109793526B (zh) * 2018-12-18 2022-08-02 深圳壹账通智能科技有限公司 测谎方法、装置、计算机设备和存储介质
CN109766917A (zh) * 2018-12-18 2019-05-17 深圳壹账通智能科技有限公司 面试视频数据处理方法、装置、计算机设备和存储介质
CN109584051A (zh) * 2018-12-18 2019-04-05 深圳壹账通智能科技有限公司 基于微表情识别的客户逾期风险判断方法及装置
CN109711982A (zh) * 2019-01-04 2019-05-03 深圳壹账通智能科技有限公司 面核提问方法、装置、计算机设备和可读存储介质
CN109831665B (zh) * 2019-01-16 2022-07-08 深圳壹账通智能科技有限公司 一种视频质检方法、系统及终端设备
CN109800703A (zh) * 2019-01-17 2019-05-24 深圳壹账通智能科技有限公司 基于微表情的风险审核方法、装置、计算机设备及存储介质
CN109886111A (zh) * 2019-01-17 2019-06-14 深圳壹账通智能科技有限公司 基于微表情的比赛监控方法、装置、计算机设备及存储介质
CN109858405A (zh) * 2019-01-17 2019-06-07 深圳壹账通智能科技有限公司 基于微表情的满意度评价方法、装置、设备及存储介质
CN109919426A (zh) * 2019-01-24 2019-06-21 平安科技（深圳）有限公司 稽核访谈测谎方法、服务器及计算机可读存储介质
CN109992505A (zh) * 2019-03-15 2019-07-09 平安科技（深圳）有限公司 应用程序测试方法、装置、计算机设备及存储介质
CN110097004B (zh) * 2019-04-30 2022-03-29 北京字节跳动网络技术有限公司 面部表情识别方法和装置
CN110490424A (zh) * 2019-07-23 2019-11-22 阿里巴巴集团控股有限公司 一种基于卷积神经网络的进行风险评估的方法和装置
CN110427881B (zh) * 2019-08-01 2021-11-26 东南大学 基于人脸局部区域特征学习的跨库微表情识别方法及装置
CN110674684A (zh) * 2019-08-15 2020-01-10 深圳壹账通智能科技有限公司 微表情分类模型生成、图像识别方法、装置、设备及介质
US11552914B2 (en) 2019-10-06 2023-01-10 International Business Machines Corporation Filtering group messages
CN110889332A (zh) * 2019-10-30 2020-03-17 中国科学院自动化研究所南京人工智能芯片创新研究院 一种基于面试中微表情的说谎检测方法
CN111339940B (zh) * 2020-02-26 2023-07-21 中国工商银行股份有限公司 视频风险识别方法及装置
CN111767779A (zh) * 2020-03-18 2020-10-13 北京沃东天骏信息技术有限公司 图像处理方法、装置、设备及计算机可读存储介质
CN111540440B (zh) * 2020-04-23 2021-01-15 深圳市镜象科技有限公司 基于人工智能的心理体检方法、装置、设备和介质
CN111597301B (zh) * 2020-04-24 2023-06-09 北京百度网讯科技有限公司 文本预测方法、装置以及电子设备
CN112183946A (zh) * 2020-09-07 2021-01-05 腾讯音乐娱乐科技（深圳）有限公司 多媒体内容评估方法、装置及其训练方法
CN112084992B (zh) * 2020-09-18 2021-04-13 北京中电兴发科技有限公司 一种人脸关键点检测模块中人脸框选取方法
CN112614583A (zh) * 2020-11-25 2021-04-06 平安医疗健康管理股份有限公司 抑郁等级测试系统
CN113158978B (zh) * 2021-05-14 2022-04-08 无锡锡商银行股份有限公司 一种微表情识别在视频审核中的风险预警方法
CN113243918B (zh) * 2021-06-11 2021-11-30 深圳般若计算机系统股份有限公司 基于多模态隐匿信息测试的风险检测方法及装置
CN115526888A (zh) * 2022-11-17 2022-12-27 博奥生物集团有限公司 眼图数据识别方法、装置、存储介质及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102103617A (zh) * 2009-12-22 2011-06-22 华为终端有限公司 获取表情含义的方法和装置
CN103258204A (zh) * 2012-02-21 2013-08-21 中国科学院心理研究所 一种基于Gabor和EOH特征的自动微表情识别方法
CN104820495A (zh) * 2015-04-29 2015-08-05 姜振宇 一种异常微表情识别与提醒方法及装置
CN105913046A (zh) * 2016-05-06 2016-08-31 姜振宇 微表情识别装置及方法
US20180005272A1 (en) * 2016-06-30 2018-01-04 Paypal, Inc. Image data detection for micro-expression analysis and targeted data services

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107480622A (zh) * 2017-08-07 2017-12-15 深圳市科迈爱康科技有限公司 微表情识别方法、装置及存储介质
CN107679526B (zh) * 2017-11-14 2020-06-12 北京科技大学 一种人脸微表情识别方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102103617A (zh) * 2009-12-22 2011-06-22 华为终端有限公司 获取表情含义的方法和装置
CN103258204A (zh) * 2012-02-21 2013-08-21 中国科学院心理研究所 一种基于Gabor和EOH特征的自动微表情识别方法
CN104820495A (zh) * 2015-04-29 2015-08-05 姜振宇 一种异常微表情识别与提醒方法及装置
CN105913046A (zh) * 2016-05-06 2016-08-31 姜振宇 微表情识别装置及方法
US20180005272A1 (en) * 2016-06-30 2018-01-04 Paypal, Inc. Image data detection for micro-expression analysis and targeted data services

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110781810A (zh) * 2019-10-24 2020-02-11 合肥盛东信息科技有限公司 一种人脸情绪识别方法
CN110781810B (zh) * 2019-10-24 2024-02-27 合肥盛东信息科技有限公司 一种人脸情绪识别方法
CN111062074A (zh) * 2019-12-11 2020-04-24 同济大学 建筑空间品质虚拟模拟及智能评价方法
CN111062074B (zh) * 2019-12-11 2023-04-07 同济大学 建筑空间品质虚拟模拟及智能评价方法
CN111274447A (zh) * 2020-01-13 2020-06-12 深圳壹账通智能科技有限公司 基于视频的目标表情生成方法、装置、介质、电子设备
CN111860154A (zh) * 2020-06-12 2020-10-30 歌尔股份有限公司 一种基于视觉的额头检测方法、装置和电子设备
CN111860154B (zh) * 2020-06-12 2024-05-28 歌尔股份有限公司 一种基于视觉的额头检测方法、装置和电子设备
CN111783620A (zh) * 2020-06-29 2020-10-16 北京百度网讯科技有限公司 表情识别方法、装置、设备及存储介质
CN111950449A (zh) * 2020-08-11 2020-11-17 合肥工业大学 基于走路姿态的情绪识别方法
CN111950449B (zh) * 2020-08-11 2024-02-13 合肥工业大学 基于走路姿态的情绪识别方法
CN112329663A (zh) * 2020-11-10 2021-02-05 西南大学 一种基于人脸图像序列的微表情时刻检测方法及装置
CN112487904A (zh) * 2020-11-23 2021-03-12 成都尽知致远科技有限公司 一种基于大数据分析的视频图像处理方法及系统
CN112381036A (zh) * 2020-11-26 2021-02-19 厦门大学 一种应用于刑侦的微表情与宏表情片段识别方法
CN112417414A (zh) * 2020-12-04 2021-02-26 支付宝(杭州)信息技术有限公司 一种基于属性脱敏的隐私保护方法、装置以及设备
CN112699774A (zh) * 2020-12-28 2021-04-23 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN112699774B (zh) * 2020-12-28 2024-05-24 深延科技(北京)有限公司 视频中人物的情绪识别方法及装置、计算机设备及介质
CN113052064A (zh) * 2021-03-23 2021-06-29 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113052064B (zh) * 2021-03-23 2024-04-02 北京思图场景数据科技服务有限公司 基于面部朝向、面部表情及瞳孔追踪的注意力检测方法
CN113191196A (zh) * 2021-04-01 2021-07-30 北京睿芯高通量科技有限公司 一种新型智能安防系统中的轨迹分析方法及系统
CN113276827A (zh) * 2021-05-26 2021-08-20 朱芮叶 电动汽车能源回收系统控制方法、系统以及汽车
CN113313048A (zh) * 2021-06-11 2021-08-27 北京百度网讯科技有限公司 脸部表情识别方法和装置
CN113313048B (zh) * 2021-06-11 2024-04-09 北京百度网讯科技有限公司 脸部表情识别方法和装置
CN113901915B (zh) * 2021-10-08 2024-04-02 无锡锡商银行股份有限公司 一种轻量网络与MagFace在视频中的表情检测方法
CN113901915A (zh) * 2021-10-08 2022-01-07 无锡锡商银行股份有限公司 一种轻量网络与MagFace在视频中的表情检测方法
CN113822229A (zh) * 2021-10-28 2021-12-21 重庆科炬企业孵化器有限公司 一种面向表情识别的用户体验评价建模方法及装置
CN114287938B (zh) * 2021-12-13 2024-02-13 重庆大学 建筑环境中人体参数的安全区间获得方法和设备
CN114287938A (zh) * 2021-12-13 2022-04-08 重庆大学 建筑环境中人体参数的安全区间获得方法和设备
CN114973362A (zh) * 2022-05-20 2022-08-30 厦门大学 一种应用于社交机器人的动态延长编码微表情识别方法
CN116824280B (zh) * 2023-08-30 2023-11-24 安徽爱学堂教育科技有限公司 基于微表情变化的心理预警方法
CN116824280A (zh) * 2023-08-30 2023-09-29 安徽爱学堂教育科技有限公司 基于微表情变化的心理预警方法

Also Published As

Publication number Publication date
CN108537160A (zh) 2018-09-14

Similar Documents

Publication Publication Date Title
WO2019184125A1 (zh) 基于微表情的风险识别方法、装置、设备及介质
US10574883B2 (en) System and method for guiding a user to take a selfie
US11176393B2 (en) Living body recognition method, storage medium, and computer device
US11633148B2 (en) Systems and methods for hair analysis
JP7078803B2 (ja) 顔写真に基づくリスク認識方法、装置、コンピュータ設備および記憶媒体
CN111310705A (zh) 图像识别方法、装置、计算机设备及存储介质
CN110634116B (zh) 一种面部图像评分方法及摄像机
Hu et al. Research on abnormal behavior detection of online examination based on image information
US20220237943A1 (en) Method and apparatus for adjusting cabin environment
Florea et al. Can Your Eyes Tell Me How You Think? A Gaze Directed Estimation of the Mental Activity.
Yu et al. An eye detection method based on convolutional neural networks and support vector machines
KR20200012355A (ko) Clm과 가버 웨이블렛을 이용한 얼굴 인증 과정을 구비한 온라인 강의 모니터링 방법
CN111259815A (zh) 一种人脸图像质量评估方法、系统、设备及介质
Robin et al. Improvement of face and eye detection performance by using multi-task cascaded convolutional networks
Saif et al. Robust drowsiness detection for vehicle driver using deep convolutional neural network
Chin et al. Facial skin image classification system using Convolutional Neural Networks deep learning algorithm
Vitek et al. Exploring bias in sclera segmentation models: A group evaluation approach
Kumar et al. Facial emotion recognition and detection using cnn
RU2768797C1 (ru) Способ и система для определения синтетически измененных изображений лиц на видео
Zhou Eye-Blink Detection under Low-Light Conditions Based on Zero-DCE
CN113920563A (zh) 在线考试作弊识别方法、装置、计算机设备和存储介质
CN112487980A (zh) 基于微表情治疗方法、装置、系统与计算机可读存储介质
CN112182537A (zh) 监控方法、装置、服务器、系统以及存储介质
EP3794541A1 (en) Systems and methods for hair analysis
CN113408406B (zh) 一种视线追踪方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18912610

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21/01/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18912610

Country of ref document: EP

Kind code of ref document: A1