WO2019109530A1 - 情绪识别方法、装置及存储介质 - Google Patents

情绪识别方法、装置及存储介质 Download PDF

Info

Publication number
WO2019109530A1
WO2019109530A1 PCT/CN2018/077346 CN2018077346W WO2019109530A1 WO 2019109530 A1 WO2019109530 A1 WO 2019109530A1 CN 2018077346 W CN2018077346 W CN 2018077346W WO 2019109530 A1 WO2019109530 A1 WO 2019109530A1
Authority
WO
WIPO (PCT)
Prior art keywords
test
feature vector
video
question
distance
Prior art date
Application number
PCT/CN2018/077346
Other languages
English (en)
French (fr)
Inventor
韦峰
徐国强
Original Assignee
深圳壹账通智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳壹账通智能科技有限公司 filed Critical 深圳壹账通智能科技有限公司
Publication of WO2019109530A1 publication Critical patent/WO2019109530A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Definitions

  • the present application relates to the field of video processing technologies, and in particular, to an emotion recognition method, apparatus, and computer readable storage medium.
  • CQT The comparison question test
  • CQT also known as the standard question test method or the control question test
  • CQT was invented in 1947 by John Reid of Chicago, USA.
  • CQT usually involves three types of problems: neutral issues, guidelines, and related issues.
  • Neutral issues also known as irrelevant issues, are not related to the test subject and do not cause psychological stress on the test subject.
  • the criterion problem is that the test subject will or will be able to make a dishonest answer, which can cause certain psychological pressure on the test subject, but it has nothing to do with the test subject and is used to compare with the related question.
  • the related question that is, the question related to the test subject, is the question to be identified by the test.
  • the theoretical basis of CQT is: honest people are afraid of the problem of the standard, and will have a greater psychological reaction to the rope problem, while those who conceal the truth are more afraid of related problems and will have a greater psychological reaction to related problems.
  • the honesty's emotions are real, and the one who conceals the truth will deliberately hide the true emotions.
  • the recognition of the test subject's mood depends either on the tester's experience or on the multi-channel tester to record the test subject's physiological response map for each problem.
  • the judgment result of the former is often inaccurate and objective, and the latter is contact-type. It is easy to infringe on the human rights of the test subject, and the test object is also prone to conflict, which affects the test results.
  • the present application provides an emotion recognition method, apparatus, and computer readable storage medium, and combines CQT to analyze an expression feature of a test object when answering different types of questions, thereby realizing objective and non-contact type. Emotional recognition.
  • the present application provides an emotion recognition method, which is applied to an electronic device, and the method includes:
  • Question bank construction steps automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and build a test question bank;
  • Questionnaire generation step generate a test questionnaire according to the test question bank
  • Video cutting step recording the video of the test subject answering the test questionnaire, cutting the video in units of a single question, and obtaining a video segment in which the test object answers each question;
  • Feature extraction step extracting an emoticon feature vector of each video segment, and treating the emoticon feature vector of each video segment as a corresponding feature vector of each question;
  • Calculation step calculating the center point feature vector of the neutral problem, the center point feature vector of the criterion problem, and the first distance between the feature vector of each related problem and the center point feature vector of the neutral problem, and the characteristics of each related problem a second distance between the vector and the center point feature vector of the criterion problem;
  • the step of identifying when the first distance is greater than the second distance, determining that the test object answers the relevant question hides the real emotion, and when the first distance is less than the second distance, determining the emotion displayed by the test object when answering the related question It is true.
  • the application also provides an electronic device comprising a memory and a processor, the memory including an emotion recognition program.
  • the electronic device is directly or indirectly connected to the imaging device, and the imaging device transmits the recorded video to the electronic device.
  • the processor of the electronic device executes the emotion recognition program in the memory, the following steps are implemented:
  • Question bank construction steps automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and build a test question bank;
  • Questionnaire generation step generate a test questionnaire according to the test question bank
  • Video cutting step recording the video of the test subject answering the test questionnaire, cutting the video in units of a single question, and obtaining a video segment in which the test object answers each question;
  • Feature extraction step extracting an emoticon feature vector of each video segment, and treating the emoticon feature vector of each video segment as a corresponding feature vector of each question;
  • Calculation step calculating the center point feature vector of the neutral problem, the center point feature vector of the criterion problem, and the first distance between the feature vector of each related problem and the center point feature vector of the neutral problem, and the characteristics of each related problem a second distance between the vector and the center point feature vector of the criterion problem;
  • the step of identifying when the first distance is greater than the second distance, determining that the test object answers the relevant question hides the real emotion, and when the first distance is less than the second distance, determining the emotion displayed by the test object when answering the related question It is true.
  • the present application further provides a computer readable storage medium including an emotion recognition program, when the emotion recognition program is executed by a processor, implementing emotion recognition as described above Any step in the method.
  • the emotion recognition method, device and computer readable storage medium provided by the present application automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and then select a certain number of three types of questions and sort them to generate a test questionnaire. .
  • test object When the test object answers the test questionnaire, extract the expression feature vector of the video segment of the test object to answer each question, regard it as the corresponding feature vector of each question, calculate the central point feature vector of the neutral problem and the criterion problem, and each The first distance between the feature vector of the related problem and the center point feature vector of the neutral problem and the second distance between the center point feature vector of the criterion problem, and when the first distance is greater than the second distance, the test is determined When the object answers the related question, the true emotion is hidden. When the first distance is smaller than the second distance, it is determined that the test subject responds to the related question and the emotion displayed is true. With the present application, the emotional state of the test subject when answering related questions can be objectively and non-contactly identified.
  • FIG. 1 is an application environment diagram of a first preferred embodiment of an electronic device of the present application.
  • FIG. 2 is an application environment diagram of a second preferred embodiment of the electronic device of the present application.
  • FIGS. 1 and 2 are a block diagram showing the program of the emotion recognition program in FIGS. 1 and 2.
  • FIG. 4 is a flow chart of a preferred embodiment of the emotion recognition method of the present application.
  • the camera 3 is connected to the electronic device 1 via the network 2, and the camera 3 records the video of the test subject answering the test questionnaire (mainly the face video of the test subject), and transmits it to the electronic device 1 via the network 2, electronically.
  • the device 1 analyzes the video using the emotion recognition program 10 provided by the present application to obtain an emotional recognition result for the test subject.
  • the electronic device 1 may be a terminal device having a storage and computing function such as a server, a smart phone, a tablet computer, a portable computer, a desktop computer, or the like.
  • the electronic device 1 includes a memory 11, a processor 12, a network interface 13, and a communication bus 14.
  • the camera device 3 is installed in a specific place, such as an interrogation room, a laboratory, a credit review place, and the like, for recording a video of the test subject answering the test questionnaire, and then transmitting the video to the memory 11 through the network 2.
  • the network interface 13 may include a standard wired interface, a wireless interface (such as a WI-FI interface).
  • Communication bus 14 is used to implement connection communication between these components.
  • the memory 11 includes at least one type of readable storage medium.
  • the at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card type memory, or the like.
  • the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1.
  • the readable storage medium may also be an external memory 11 of the electronic device 1, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (SMC). , Secure Digital (SD) card, Flash Card, etc.
  • SMC smart memory card
  • SD Secure Digital
  • the memory 11 stores the program code of the emotion recognition program 10, the video recorded by the camera 3, and other data to which the processor 12 executes the program code of the emotion recognition program 10 and the last output data. Wait.
  • Processor 12 may be a Central Processing Unit (CPU), microprocessor or other data processing chip in some embodiments.
  • CPU Central Processing Unit
  • microprocessor or other data processing chip in some embodiments.
  • Figure 1 shows only the electronic device 1 with components 11-14, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.
  • the electronic device 1 may further include a user interface
  • the user interface may include an input unit such as a keyboard, a voice input device such as a microphone, a device with a voice recognition function, a voice output device such as an audio, a headphone, and the like.
  • the user interface may also include a standard wired interface and a wireless interface.
  • the electronic device 1 may further include a display.
  • the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch sensor, or the like in some embodiments.
  • the display is used to display information processed by the electronic device 1 and a visualized user interface.
  • the electronic device 1 further comprises a touch sensor.
  • the area provided by the touch sensor for the user to perform a touch operation is referred to as a touch area.
  • the touch sensor described herein may be a resistive touch sensor, a capacitive touch sensor, or the like.
  • the touch sensor includes not only a contact type touch sensor but also a proximity type touch sensor or the like.
  • the touch sensor may be a single sensor or a plurality of sensors arranged, for example, in an array.
  • a user such as a tester, a test subject, or the like, can activate the emotion recognition program 10 by touch.
  • the electronic device 1 may further include a radio frequency (RF) circuit, a sensor, an audio circuit, and the like, and details are not described herein.
  • RF radio frequency
  • FIG. 2 it is an application environment diagram of a second preferred embodiment of the electronic device of the present application.
  • the test object and the tester remotely execute the test link through the terminal 3, and the camera device 30 of the terminal 3 records the video of the test object answer test questionnaire and transmits it to the electronic device 1 through the network 2, and the processor 12 of the electronic device 1 executes the memory 11
  • the stored program code of the emotion recognition program 10 analyzes the video to obtain an emotional recognition result for the test object for reference by the tester.
  • the terminal 3 can be a terminal device having a storage and computing function, such as a smart phone, a tablet computer, a portable computer, and a desktop computer.
  • the emotion recognition program 10 of Figures 1 and 2 when executed by the processor 12, implements the following steps:
  • Question bank construction steps automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and build a test question bank;
  • Questionnaire generation step generate a test questionnaire according to the test question bank
  • Video cutting step recording the video of the test subject answering the test questionnaire, cutting the video in units of a single question, and obtaining a video segment in which the test object answers each question;
  • Feature extraction step extracting an emoticon feature vector of each video segment, and treating the emoticon feature vector of each video segment as a corresponding feature vector of each question;
  • Calculation step calculating the center point feature vector of the neutral problem, the center point feature vector of the criterion problem, and the first distance between the feature vector of each related problem and the center point feature vector of the neutral problem, and the characteristics of each related problem a second distance between the vector and the center point feature vector of the criterion problem;
  • the step of identifying when the first distance is greater than the second distance, determining that the test object answers the relevant question hides the real emotion, and when the first distance is less than the second distance, determining the emotion displayed by the test object when answering the related question It is true.
  • FIG. 3 it is a program block diagram of the emotion recognition program 10 in Figs.
  • the emotion recognition program 10 is divided into a plurality of modules, which are stored in the memory 11 and executed by the processor 12 to complete the present application.
  • a module as referred to in this application refers to a series of computer program instructions that are capable of performing a particular function.
  • the emotion recognition program 10 can be divided into a question bank construction module 110, a questionnaire generation module 120, a video cutting module 130, a feature extraction module 140, a calculation module 150, and an identification module 160.
  • the question bank building module 110 is configured to automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and construct a test question bank. Before the test session is officially started, the tester will learn about the various aspects of the test object through various channels, including basic information such as ID card information, contact telephone number, address information, education, and occupation, as well as personal information related to the test subject. For example, credit history, history of crimes, etc.
  • the question bank building module 110 automatically generates a large number of neutral problems, criteria problems, and related problems based on the information of the test objects. For example, the question bank building module 110 can generate the following neutral questions:
  • the question of the criteria that the question bank building module 110 can generate is as follows:
  • the related questions that the question bank building module 110 can generate are as follows:
  • the above introduction to the generation of massive neutral issues, guidelines and related issues is only a partial example and is not exhaustive.
  • the generated problems and the material information on which the problems are generated are stored in the memory 11 of the electronic device 1.
  • the questionnaire generating module 120 is configured to generate a test questionnaire according to the test question bank. After the completion of the test question bank construction, the selection and sequence of the questions become an important factor affecting the quality of the test questionnaire. The quality of the test questionnaire directly affects the accuracy and reliability of the emotional recognition results.
  • the test questionnaire includes at least two criteria problems, the number of the criterion problems is less than the number of related questions and the number of neutral problems, respectively, and the same type of questions are not adjacent.
  • the number of questions in each test questionnaire should be greater than the preset number (for example, 15) to ensure the scope of the test.
  • the first and last questions of the test questionnaire should be set to neutral questions to help test object adjustment. Emotions, relaxation.
  • the video cutting module 130 is configured to cut a video of the test object to answer the test questionnaire, and obtain a video segment of the test object to answer each question.
  • the camera device 3 of FIG. 1 or the camera device 30 of the terminal 3 of FIG. 2 records a video of the test subject answering the test questionnaire, and the video cutting module 130 cuts the video in units of a single question to obtain a video segment in which the test object answers each question. . Recording Test Subjects When answering a video of a test questionnaire, you can set a time limit for each question (for example, 20 seconds), and the next question is displayed when the preset answer time is exceeded.
  • the feature extraction module 140 is configured to extract an emoticon feature vector of each video segment.
  • the feature extraction module 140 extracts the expression feature vector of a video segment, first extracts the action features such as the head orientation, the eye orientation, and the action unit (AU) from the video segment, and then counts each action feature in the video segment.
  • the number of occurrences and the duration of the occurrence, the high-order expression features of the video segment are constructed according to the statistical results, and then the feature selection algorithm is used to select the optimal feature subset from the high-order expression features of the video segment, and finally the optimal feature is selected.
  • the subset is subjected to dimensionality reduction processing to obtain an expression feature vector of the video segment in a two-dimensional space. Since there is a one-to-one correspondence between the video clip and the test questionnaire, for the sake of convenience, we regard the emoticon feature vector of the video clip as the feature vector of the corresponding question.
  • the feature screening algorithm may be a sequence forward selection (SFS) algorithm, a sequence backward selection (SBS) algorithm, a bidirectional search (BDS) algorithm, and filtering.
  • the filter feature selection algorithm may also be another feature screening algorithm.
  • the dimensionality reduction processing uses a t-SNE algorithm to project a high-dimensional (for example, 4710-dimensional) expression feature into a two-dimensional space, and obtains an expression feature vector of the video segment in a two-dimensional space to facilitate visual display.
  • the calculation module 150 is configured to calculate a center point feature vector of the neutral problem, a center point feature vector of the criterion problem, and a first distance and a correlation between the feature vector of each related problem and the center point feature vector of the neutral problem The second distance between the feature vector of the problem and the center point feature vector of the criterion problem.
  • the central point feature vector of the neutral problem and the center point feature vector of the criterion problem can be calculated by the K-means algorithm.
  • the mean value of the eigenvectors of the neutral problem and the mean of the eigenvectors of the criterion problem can also be calculated. The mean is used as the center point feature vector. Then, a first distance between the feature vector of each related problem and the center point feature vector of the neutral problem and a second distance between the feature vector of each related problem and the center point feature vector of the criterion problem are calculated.
  • n1 neutral questions contains n1 neutral questions, n2 criteria problems, and n3 related questions
  • the center point feature vector of the n1 neutral problems is calculated as (x i , y i )
  • the n2 criteria The center point feature vector of the problem is (x c , y c )
  • the first distance between the feature vector (x r , y r ) of any related problem and the center point feature vector of the neutral problem can be expressed as:
  • the second distance between the feature vector (x r , y r ) of the related problem and the center point feature vector of the criterion problem can be expressed as:
  • the identification module 160 is configured to identify an emotional state when the test object answers each related question. When the first distance is greater than the second distance, determining that the test object answers the related question hides the real emotion; when the first distance is smaller than the second distance, determining that the test subject responds to the related question is an actual emotion .
  • FIG. 4 it is a flowchart of a preferred embodiment of the emotion recognition method of the present application.
  • the electronic device 1 is activated, and the processor 12 executes the emotion recognition program 10 stored in the memory 11 to implement the following steps:
  • step S10 the question bank building module 110 is used to generate a massive neutral problem, a criterion problem, and related problems, and a test question bank is constructed.
  • a massive neutral problem a criterion problem, and related problems
  • a test question bank is constructed.
  • the questionnaire generation module 120 is used to select three types of questions in the test question database to form a test questionnaire.
  • the test questionnaire includes at least two criteria problems, the number of the criterion problems is less than the number of related questions and the number of neutral problems, respectively, and the same type of questions are not adjacent.
  • the number of questions in each test questionnaire should be greater than the preset number to ensure the scope of the test.
  • the first and last questions of the test questionnaire should be set to neutral issues to help the test subjects adjust their mood and relax their mood.
  • step S30 the test subject answers the video of the test questionnaire, and the video is cut by the video cutting module 130 to obtain a video segment in which the test object answers each question.
  • the video of the test subject answering test questionnaire is recorded by the camera device 3 of FIG. 1 or the camera device 30 of the terminal 3 of FIG. 2, and the video cutting module 130 cuts the video in units of a single question to obtain a video of the test subject answering each question. Fragment.
  • step S40 the feature extraction module 140 extracts the expression feature vector of each video segment, and considers the expression feature vector of each video segment as the feature vector of each question.
  • the feature extraction module 140 extracts the expression feature vector of each video segment, and considers the expression feature vector of each video segment as the feature vector of each question.
  • Step S50 using the calculation module 150 to calculate a center point feature vector of the neutral problem, a center point feature vector of the criterion problem, and a first distance between the feature vector of each related problem and the center point feature vector of the neutral problem, each The second distance between the feature vector of the related problem and the center point feature vector of the criterion problem.
  • a center point feature vector of the neutral problem a center point feature vector of the criterion problem
  • a first distance between the feature vector of each related problem and the center point feature vector of the neutral problem each The second distance between the feature vector of the related problem and the center point feature vector of the criterion problem.
  • step S60 the identification module 160 is used to identify the emotional state of the test subject when answering each related question.
  • the first distance is greater than the second distance, determining that the test object answers the related question hides the real emotion; when the first distance is smaller than the second distance, determining that the test subject responds to the related question is an actual emotion .
  • the embodiment of the present application further provides a computer readable storage medium, which may be a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read only memory (ROM), and an erasable programmable Any combination or combination of any one or more of read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, and the like.
  • the computer readable storage medium includes a test question bank, material information on which the test question bank is constructed, and an emotion recognition program 10, and the emotion recognition program 10 is executed by the processor to perform the following operations:
  • Question bank construction steps automatically generate a large number of neutral problems, criteria problems and related problems according to the information of the test object, and build a test question bank;
  • Questionnaire generation step generate a test questionnaire according to the test question bank
  • Video cutting step recording the video of the test subject answering the test questionnaire, cutting the video in units of a single question, and obtaining a video segment in which the test object answers each question;
  • Feature extraction step extracting an emoticon feature vector of each video segment, and treating the emoticon feature vector of each video segment as a corresponding feature vector of each question;
  • Calculation step calculating the center point feature vector of the neutral problem, the center point feature vector of the criterion problem, and the first distance between the feature vector of each related problem and the center point feature vector of the neutral problem, and the characteristics of each related problem a second distance between the vector and the center point feature vector of the criterion problem;
  • the step of identifying when the first distance is greater than the second distance, determining that the test object answers the relevant question hides the real emotion, and when the first distance is less than the second distance, determining the emotion displayed by the test object when answering the related question It is true.
  • a disk including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the various embodiments of the present application.
  • a terminal device which may be a mobile phone, a computer, a server, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Multimedia (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Probability & Statistics with Applications (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种情绪识别方法、装置及存储介质。该方法包括以下步骤：生成海量中性问题、准绳问题和相关问题，构建测试题库（S10）；根据测试题库生成测试问卷（S20）；切割测试对象回答测试问卷的视频，得到测试对象回答每个问题的视频片段（S30）；提取每个视频片段的表情特征向量，将其视为对应的每个问题的特征向量（S40）；计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量的第二距离（S50）；当第一距离大于第二距离时，判定该测试对象隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象表现出的情绪是真实的（S60）。

Description

情绪识别方法、装置及存储介质
本申请要求于2017年12月8日提交中国专利局、申请号为201711289932.2、发明名称为“情绪识别方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在申请中。
技术领域
本申请涉及视频处理技术领域，尤其涉及一种情绪识别方法、装置及计算机可读存储介质。
背景技术
对照问题测试法(comparison question test，CQT)，也称准绳问题测试法或控制问题测试法(control question test)，由美国芝加哥的理德(John Reid)于1947年发明。CQT通常涉及三种类型的问题：中性问题、准绳问题和相关问题。中性问题又称无关问题，是与测试主题无关，不会引起测试对象心理压力的问题。准绳问题是测试对象必然会或者很有可能会做出不诚实回答的问题，能够给测试对象造成一定的心理压力，但与测试主题无关，用于和相关问题作比较。相关问题即和测试主题有关的问题，是测试所要甄别的问题。CQT的理论基础是：诚实者害怕准绳问题，会对准绳问题产生较大的心理反应，而隐瞒真相者更害怕相关问题，会对相关问题产生更大的心理反应。
在测试对象回答相关问题时，诚实者的情绪是真实的，而隐瞒真相者会刻意隐藏真实情绪。然而，目前缺少识别测试对象情绪的客观有效的方法。通常，对测试对象情绪的识别要么依赖测试人员的经验，要么通过多通道测试仪记录测试对象对每个问题的生理反应图谱。前者的判断结果往往不准确客观，后者是接触式的，容易对测试对象的人权构成侵犯，测试对象也容易产生抵触心理，从而影响测试结果。
发明内容
为解决现有技术存在的不足，本申请提供一种情绪识别方法、装置及计算机可读存储介质，通过结合CQT，对测试对象回答不同类型问题时的表情 特征进行分析，实现客观、非接触式的情绪识别。
为实现上述目的，本申请提供一种情绪识别方法，应用于电子装置，该方法包括：
题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
问卷生成步骤：根据测试题库生成测试问卷；
视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
本申请还提供一种电子装置，该电子装置包括存储器和处理器，所述存储器中包括情绪识别程序。该电子装置直接或间接地与摄像装置相连接，摄像装置将录制的视频传送至电子装置。该电子装置的处理器执行存储器中的情绪识别程序时，实现以下步骤：
题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
问卷生成步骤：根据测试题库生成测试问卷；
视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距 离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
此外，为实现上述目的，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质中包括情绪识别程序，所述情绪识别程序被处理器执行时，实现如上所述的情绪识别方法中的任意步骤。
本申请提供的情绪识别方法、装置及计算机可读存储介质，根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，然后，选取一定数量的三类问题并排序，生成测试问卷。在测试对象回答测试问卷时，提取测试对象回答每个问题的视频片段的表情特征向量，将其视为对应的每个问题的特征向量，计算中性问题、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离和与准绳问题的中心点特征向量之间的第二距离，当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。利用本申请，可以客观、非接触地识别测试对象回答相关问题时的情绪状态。
附图说明
图1为本申请电子装置第一较佳实施例的应用环境图。
图2为本申请电子装置第二较佳实施例的应用环境图。
图3为图1、图2中情绪识别程序的程序模块图。
图4为本申请情绪识别方法较佳实施例的流程图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
下面将参考若干具体实施例来描述本申请的原理和精神。应当理解，此 处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
参照图1所示，为本申请电子装置第一较佳实施例的应用环境图。在该实施例中，摄像装置3通过网络2连接电子装置1，摄像装置3录制测试对象回答测试问卷的视频(主要是拍摄测试对象的正脸视频)，通过网络2传送至电子装置1，电子装置1利用本申请提供的情绪识别程序10分析所述视频，得到对测试对象的情绪识别结果。
电子装置1可以是服务器、智能手机、平板电脑、便携计算机、桌上型计算机等具有存储和运算功能的终端设备。
该电子装置1包括存储器11、处理器12、网络接口13及通信总线14。
摄像装置3安装于特定场所，如审讯室、实验室、信用审核场所等，用于录制测试对象回答测试问卷的视频，然后通过网络2将所述视频传输至存储器11。网络接口13可以包括标准的有线接口、无线接口(如WI-FI接口)。通信总线14用于实现这些组件之间的连接通信。
存储器11包括至少一种类型的可读存储介质。所述至少一种类型的可读存储介质可为如闪存、硬盘、多媒体卡、卡型存储器等的非易失性存储介质。在一些实施例中，所述可读存储介质可以是所述电子装置1的内部存储单元，例如该电子装置1的硬盘。在另一些实施例中，所述可读存储介质也可以是所述电子装置1的外部存储器11，例如所述电子装置1上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。
在本实施例中，所述存储器11存储所述情绪识别程序10的程序代码、摄像装置3录制的视频，以及处理器12执行情绪识别程序10的程序代码应用到的其他数据以及最后输出的数据等。
处理器12在一些实施例中可以是一中央处理器(Central Processing Unit,CPU)，微处理器或其他数据处理芯片。
图1仅示出了具有组件11-14的电子装置1，但是应理解的是，并不要求实施所有示出的组件，可以替代的实施更多或者更少的组件。
可选地，该电子装置1还可以包括用户接口，用户接口可以包括输入单元比如键盘(Keyboard)、语音输入装置比如麦克风(microphone)等具有语音识别功能的设备、语音输出装置比如音响、耳机等，可选地用户接口还可 以包括标准的有线接口、无线接口。
可选地，该电子装置1还可以包括显示器。显示器在一些实施例中可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode，有机发光二极管)触摸器等。显示器用于显示电子装置1处理的信息以及可视化的用户界面。
可选地，该电子装置1还包括触摸传感器。所述触摸传感器所提供的供用户进行触摸操作的区域称为触控区域。此外，这里所述的触摸传感器可以为电阻式触摸传感器、电容式触摸传感器等。而且，所述触摸传感器不仅包括接触式的触摸传感器，也可包括接近式的触摸传感器等。此外，所述触摸传感器可以为单个传感器，也可以为例如阵列布置的多个传感器。用户，例如测试人员、测试对象等，可以通过触摸启动情绪识别程序10。
该电子装置1还可以包括射频(Radio Frequency，RF)电路、传感器和音频电路等等，在此不再赘述。
参照图2所示，为本申请电子装置第二较佳实施例的应用环境图。测试对象、测试人员通过终端3远程执行测试环节，终端3的摄像装置30录制测试对象回答测试问卷的视频，并通过网络2传送至所述电子装置1，电子装置1的处理器12执行存储器11存储的情绪识别程序10的程序代码，对视频进行分析，得到对测试对象的情绪识别结果，供测试人员参考。
图2中电子装置1的组件，例如图中示出的存储器11、处理器12、网络接口13及通信总线14，以及图中未示出的组件，请参照关于图1的介绍。
所述终端3可以为智能手机、平板电脑、便携计算机、桌上型计算机等具有存储和运算功能的终端设备。
图1、图2中的情绪识别程序10，在被处理器12执行时，实现以下步骤：
题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
问卷生成步骤：根据测试题库生成测试问卷；
视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
关于上述步骤的详细介绍，请参照下述图3关于情绪识别程序10的程序模块图及图4关于情绪识别方法较佳实施例的流程图的说明。
参照图3所示，为图1、图2中情绪识别程序10的程序模块图。在本实施例中，情绪识别程序10被分割为多个模块，该多个模块被存储于存储器11中，并由处理器12执行，以完成本申请。本申请所称的模块是指能够完成特定功能的一系列计算机程序指令段。
所述情绪识别程序10可以被分割为：题库构建模块110、问卷生成模块120、视频切割模块130、特征提取模块140、计算模块150和识别模块160。
题库构建模块110，用于根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库。测试环节正式开始前，测试人员会通过多种途径了解到测试对象的多方面信息，包括身份证信息、联系电话、住址信息、学历、从事职业等基本信息，也包括与测试主题相关的个人信息，例如信用历史、有无作案史等等。题库构建模块110根据测试对象的这些信息自动生成海量的中性问题、准绳问题和相关问题。例如，题库构建模块110可以生成如下中性问题：
您的生日是几月几号？
离您家最近的公园的名字是什么？
您每天上下班在路上的时间是多长？
您小学的校名是什么？
您家附近有“丰巢”智能快递柜吗？
假设测试对象为贷款申请者，题库构建模块110可以生成的准绳问题如下：
您有过向朋友借零钱，后来忘记还的事吗？
您有过把自己的过错推给别人的事吗？
您拖欠过水电费吗？
您以前说过谎吗？
您有过为了避免受到惩罚而掩盖事实真相的经历吗？
依上述例子，假设测试对象为贷款申请者，题库构建模块110可以生成的相关问题如下：
您本次申请贷款的用途是什么？
您的家人支持您申请贷款吗？
您家未来几个月有大额资金支出计划吗？会不会影响您的还贷信心？
您近期还有其他贷款申请计划吗？
上述关于生成海量中性问题、准绳问题和相关问题的介绍仅是提供部分例子，未能穷举。生成的问题以及生成这些问题依据的资料信息，存储于电子装置1的存储器11中。
问卷生成模块120，用于根据所述测试题库生成测试问卷。测试题库构建完成后，问题的选取和顺序安排成为影响测试问卷质量的重要因素，测试问卷的质量则直接影响情绪识别结果的准确度和可靠性。在本实施例中，所述测试问卷至少包括两个准绳问题，所述准绳问题的数量分别少于相关问题数量和中性问题数量，且同一类型的问题不相邻。
“同一类型的问题不相邻”的要求，是为了凸显测试对象在面对不同类型问题时的表情变化。除此之外，每份测试问卷的问题数量应该大于预设数量(例如15个)，以保证测试范围，测试问卷的第一题和最后一题应该设置为中性问题，以帮助测试对象调节情绪、放松心情。
视频切割模块130，用于切割测试对象回答测试问卷的视频，得到测试对象回答每个问题的视频片段。图1的摄像装置3或图2中终端3的摄像装置30录制测试对象回答测试问卷的视频，视频切割模块130以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段。录制测试对象 回答测试问卷的视频时，可为每个问题设置答题限制时间(例如20秒)，超过预设答题限制时间则展示下一个问题。
特征提取模块140，用于提取每个视频片段的表情特征向量。特征提取模块140提取一个视频片段的表情特征向量时，先从该视频片段中提取头部朝向、眼球朝向和面部动作单元(action unit，AU)等动作特征，再统计各动作特征在该视频片段中出现的次数和持续的时长，根据统计结果构造该视频片段的高阶表情特征，然后利用特征筛选算法从该视频片段的高阶表情特征中筛选出最优特征子集，最后对最优特征子集进行降维处理，得到该视频片段在二维空间中的表情特征向量。由于视频片段和测试问卷的问题存在一一对应的关系，为了便于表述，我们把视频片段的表情特征向量视为对应的问题的特征向量。
在本实施例中，所述特征筛选算法可以是序列前向选择(Sequential Forward Selection，SFS)算法、序列后向选择(Sequential Backward Selection，SBS)算法、双向搜索(Bidirectional Search，BDS)算法、过滤特征选择(filter feature selection)算法，也可以是其他特征筛选算法。所述降维处理采用t-SNE算法，将高维(例如4710维)的表情特征向二维空间投影，得到视频片段在二维空间中的表情特征向量，以便于可视化展示。
计算模块150，用于计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离。中性问题的中心点特征向量和准绳问题的中心点特征向量，可以利用K-means算法计算得到，也可以计算中性问题的特征向量的均值以及准绳问题的特征向量的均值，以特征向量的均值作为中心点特征向量。然后，计算每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离以及每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离。
例如，假设一份测试问卷包含n1个中性问题、n2个准绳问题和n3个相关问题，计算得到该n1个中性问题的中心点特征向量为(x i，y i)，该n2个准绳问题的中心点特征向量为(x c，y c)，则任意一个相关问题的特征向量(x r，y r)与中性问题的中心点特征向量之间的第一距离可以表示为：
Figure PCTCN2018077346-appb-000001
相关问题的特征向量(x r，y r)与准绳问题的中心点特征向量之间的第二 距离可以表示为：
Figure PCTCN2018077346-appb-000002
识别模块160，用于识别测试对象回答每个相关问题时的情绪状态。当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪；当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
依上述例子，当d ri＞d rc时，表明该测试对象在回答该相关问题时，其心理压力和情绪波动与回答中性问题时的心理压力和情绪波动差异较大，与回答准绳问题时的心理压力和情绪波动差异较小，故该测试对象回答该相关问题时隐藏了真实情绪；当d ri＜d rc时，表明该测试对象在回答该相关问题时，其心理压力和情绪波动与回答中性问题时的心理压力和情绪波动差异较小，与回答准绳问题时的心理压力和情绪波动差异较大，故该测试对象回答该相关问题时表现出的情绪是真实的；当d ri＝d rc时，表明该测试对象在回答该相关问题时，其心理压力和情绪波动与回答中性问题、准绳问题时的心理压力和情绪波动差异相同，无法判断该测试对象回答该相关问题时的情绪状态，但这种情况出现的可能性非常小，可以对测试问卷做出适当调整，重复测试环节，重新判断测试对象的情绪状态。
参照图4所示，为本申请情绪识别方法较佳实施例的流程图。利用图1或图2所示的架构，启动电子装置1，处理器12执行存储器11中存储的情绪识别程序10，实现如下步骤：
步骤S10，利用题库构建模块110生成海量中性问题、准绳问题和相关问题，构建测试题库。中性问题、准绳问题和相关问题的生成方式请参照上述题库构建模块110的详细介绍。
步骤S20，利用问卷生成模块120选取测试题库中的三类问题组成测试问卷。在本实施例中，所述测试问卷至少包括两个准绳问题，所述准绳问题的数量分别少于相关问题数量和中性问题数量，且同一类型的问题不相邻。此外，每份测试问卷的问题数量应该大于预设数量，以保证测试范围，测试问卷的第一题和最后一题应该设置为中性问题，以帮助测试对象调节情绪、放松心情。
步骤S30，录制测试对象回答测试问卷的视频，利用视频切割模块130对该视频进行切割，得到测试对象回答每个问题的视频片段。利用图1的摄像装置3或图2中终端3的摄像装置30录制测试对象回答测试问卷的视频，视频切割模块130以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段。
步骤S40，利用特征提取模块140提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量。表情特征向量的具体提取方法请参照上述提取模块140的详细介绍。
步骤S50，利用计算模块150计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离。具体的计算方法请参照上述计算模块150的详细介绍。
步骤S60，利用识别模块160识别测试对象回答每个相关问题时的情绪状态。当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪；当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
此外，本申请实施例还提出一种计算机可读存储介质，所述计算机可读存储介质可以是硬盘、多媒体卡、SD卡、闪存卡、SMC、只读存储器(ROM)、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器等等中的任意一种或者几种的任意组合。所述计算机可读存储介质中包括测试题库、构建测试题库依据的资料信息以及情绪识别程序10，所述情绪识别程序10被处理器执行时实现如下操作：。
题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
问卷生成步骤：根据测试题库生成测试问卷；
视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
本申请之计算机可读存储介质的具体实施方式与上述情绪识别方法以及电子装置1的具体实施方式大致相同，在此不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、装置、物品或者方法不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、装置、物品或者方法所固有的要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种情绪识别方法，应用于电子装置，其特征在于，该方法包括：
    题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
    问卷生成步骤：根据测试题库生成测试问卷；
    视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
    特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
    计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
    识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
  2. 如权利要求1所述的情绪识别方法，其特征在于，所述测试问卷至少包括两个准绳问题，所述准绳问题的数量分别少于相关问题数量和中性问题数量，且同一类型的问题不相邻。
  3. 如权利要求1所述的情绪识别方法，其特征在于，所述特征提取步骤中提取每个视频片段的表情特征向量包括以下步骤：
    提取每个视频片段的动作特征；
    统计每个视频片段中各动作特征出现的次数及持续的时长；
    根据统计结果构造每个视频片段的高阶表情特征；
    利用特征筛选算法从每个视频片段的高阶表情特征中筛选出特征子集；
    对所述特征子集进行降维处理，得到每个视频片段的表情特征向量。
  4. 如权利要求3所述的情绪识别方法，其特征在于，所述动作特征包括头部朝向、眼球朝向和面部动作单元。
  5. 如权利要求1所述的情绪识别方法，其特征在于，所述录制测试对象回答测试问卷的视频的步骤，具体包括：
    通过现场摄像装置录制测试对象现场回答测试问卷的视频，或者通过远程摄像装置录制测试对象远程回答测试问卷的视频。
  6. 如权利要求2至4任意一项所述的情绪识别方法，其特征在于，所述录制测试对象回答测试问卷的视频的步骤，具体包括：
    通过现场摄像装置录制测试对象现场回答测试问卷的视频，或者通过远程摄像装置录制测试对象远程回答测试问卷的视频。
  7. 如权利要求6所述的情绪识别方法，其特征在于，为每个问题设置答题限制时间，超过预设答题限制时间则展示下一个问题。
  8. 一种电子装置，包括存储器和处理器，其特征在于，所述存储器中包括情绪识别程序，所述情绪识别程序被所述处理器执行时实现如下步骤：
    题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
    问卷生成步骤：根据测试题库生成测试问卷；
    视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
    特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
    计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离；及
    识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
  9. 如权利要求8所述的电子装置，其特征在于，所述测试问卷至少包括两个准绳问题，所述准绳问题的数量分别少于相关问题数量和中性问题数量，且同一类型的问题不相邻。
  10. 如权利要求8所述的电子装置，其特征在于，所述特征提取步骤中 提取每个视频片段的表情特征向量包括以下步骤：
    提取每个视频片段的动作特征；
    统计每个视频片段中各动作特征出现的次数及持续的时长；
    根据统计结果构造每个视频片段的高阶表情特征；
    利用特征筛选算法从每个视频片段的高阶表情特征中筛选出特征子集；
    对所述特征子集进行降维处理，得到每个视频片段的表情特征向量。
  11. 如权利要求10所述的电子装置，其特征在于，所述动作特征包括头部朝向、眼球朝向和面部动作单元。
  12. 如权利要求8所述的电子装置，其特征在于，所述录制测试对象回答测试问卷的视频的步骤，具体包括：
    通过现场摄像装置录制测试对象现场回答测试问卷的视频，或者通过远程摄像装置录制测试对象远程回答测试问卷的视频。
  13. 如权利要求9-11任一项所述的电子装置，其特征在于，所述录制测试对象回答测试问卷的视频的步骤，具体包括：
    通过现场摄像装置录制测试对象现场回答测试问卷的视频，或者通过远程摄像装置录制测试对象远程回答测试问卷的视频。
  14. 如权利要求13所述的电子装置，其特征在于，为每个问题设置答题限制时间，超过预设答题限制时间则展示下一个问题。
  15. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中包括情绪识别程序，所述情绪识别程序被处理器执行时，实现如下步骤：
    题库构建步骤：根据测试对象的信息自动生成海量的中性问题、准绳问题和相关问题，构建测试题库；
    问卷生成步骤：根据测试题库生成测试问卷；
    视频切割步骤：录制测试对象回答测试问卷的视频，以单个问题为单位对该视频进行切割，得到测试对象回答每个问题的视频片段；
    特征提取步骤：提取每个视频片段的表情特征向量，将每个视频片段的表情特征向量视为对应的每个问题的特征向量；
    计算步骤：计算中性问题的中心点特征向量、准绳问题的中心点特征向量以及每个相关问题的特征向量与中性问题的中心点特征向量之间的第一距离、每个相关问题的特征向量与准绳问题的中心点特征向量之间的第二距离； 及
    识别步骤：当第一距离大于第二距离时，判定该测试对象回答该相关问题时隐藏了真实情绪，当第一距离小于第二距离时，判定该测试对象回答该相关问题时表现出的情绪是真实的。
  16. 如权利要求15所述的计算机可读存储介质，其特征在于，所述测试问卷至少包括两个准绳问题，所述准绳问题的数量分别少于相关问题数量和中性问题数量，且同一类型的问题不相邻。
  17. 如权利要求15所述的计算机可读存储介质，其特征在于，所述特征提取步骤中提取每个视频片段的表情特征向量包括以下步骤：
    提取每个视频片段的动作特征；
    统计每个视频片段中各动作特征出现的次数及持续的时长；
    根据统计结果构造每个视频片段的高阶表情特征；
    利用特征筛选算法从每个视频片段的高阶表情特征中筛选出特征子集；
    对所述特征子集进行降维处理，得到每个视频片段的表情特征向量。
  18. 如权利要求17所述的计算机可读存储介质，其特征在于，所述动作特征包括头部朝向、眼球朝向和面部动作单元。
  19. 如权利要求15-18任一项所述的计算机可读存储介质，其特征在于，所述录制测试对象回答测试问卷的视频的步骤，具体包括：
    通过现场摄像装置录制测试对象现场回答测试问卷的视频，或者通过远程摄像装置录制测试对象远程回答测试问卷的视频。
  20. 如权利要求19所述的计算机可读存储介质，其特征在于，为每个问题设置答题限制时间，超过预设答题限制时间则展示下一个问题。
PCT/CN2018/077346 2017-12-08 2018-02-27 情绪识别方法、装置及存储介质 WO2019109530A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711289932.2A CN108052889B (zh) 2017-12-08 2017-12-08 情绪识别方法、装置及存储介质
CN201711289932.2 2017-12-08

Publications (1)

Publication Number Publication Date
WO2019109530A1 true WO2019109530A1 (zh) 2019-06-13

Family

ID=62122467

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/077346 WO2019109530A1 (zh) 2017-12-08 2018-02-27 情绪识别方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN108052889B (zh)
WO (1) WO2019109530A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115607156A (zh) * 2022-10-21 2023-01-17 厦门诸格量科技有限公司 基于多模态的心理认知筛查测评方法、系统及存储介质
CN116058841A (zh) * 2023-01-04 2023-05-05 北京火神永创科技有限公司 基于皮电波的心理测试自动评分方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109389489B (zh) * 2018-09-25 2023-04-18 平安科技（深圳）有限公司 欺诈行为的识别方法、计算机可读存储介质及终端设备
CN109259733A (zh) * 2018-10-25 2019-01-25 深圳和而泰智能控制股份有限公司 一种睡眠中呼吸暂停检测方法、装置和检测设备
CN109816518A (zh) * 2019-01-04 2019-05-28 深圳壹账通智能科技有限公司 面核结果获取方法、装置、计算机设备和可读存储介质
CN109697227A (zh) * 2019-01-28 2019-04-30 深圳壹账通智能科技有限公司 过程优化的电子问卷动态生成方法、装置和计算机设备
CN112331110A (zh) * 2020-11-05 2021-02-05 大连民族大学 一种可互动的智能讲解器及其使用方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732203A (zh) * 2015-03-05 2015-06-24 中国科学院软件研究所 一种基于视频信息的情绪识别与跟踪方法
US20150193718A1 (en) * 2015-03-23 2015-07-09 Looksery, Inc. Emotion recognition for workforce analytics
US20170154211A1 (en) * 2015-03-18 2017-06-01 Victor Shaburov Emotion recognition in video conferencing
CN107085709A (zh) * 2017-04-25 2017-08-22 广东数相智能科技有限公司 一种基于视频信息的自信度判断方法、装置及系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8032356B2 (en) * 2006-05-25 2011-10-04 University Of Southern California Spoken translation system using meta information strings
CN1996344A (zh) * 2006-12-22 2007-07-11 北京航空航天大学 人脸表情信息的提取方法及处理
US10095917B2 (en) * 2013-11-04 2018-10-09 Facebook, Inc. Systems and methods for facial representation
CN104504365A (zh) * 2014-11-24 2015-04-08 闻泰通讯股份有限公司 视频序列中的笑脸识别系统及方法
CN105913046A (zh) * 2016-05-06 2016-08-31 姜振宇 微表情识别装置及方法
CN107220591A (zh) * 2017-04-28 2017-09-29 哈尔滨工业大学深圳研究生院 多模态智能情绪感知系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732203A (zh) * 2015-03-05 2015-06-24 中国科学院软件研究所 一种基于视频信息的情绪识别与跟踪方法
US20170154211A1 (en) * 2015-03-18 2017-06-01 Victor Shaburov Emotion recognition in video conferencing
US20150193718A1 (en) * 2015-03-23 2015-07-09 Looksery, Inc. Emotion recognition for workforce analytics
CN107085709A (zh) * 2017-04-25 2017-08-22 广东数相智能科技有限公司 一种基于视频信息的自信度判断方法、装置及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115607156A (zh) * 2022-10-21 2023-01-17 厦门诸格量科技有限公司 基于多模态的心理认知筛查测评方法、系统及存储介质
CN115607156B (zh) * 2022-10-21 2023-06-27 厦门诸格量科技有限公司 基于多模态的心理认知筛查测评方法、系统及存储介质
CN116058841A (zh) * 2023-01-04 2023-05-05 北京火神永创科技有限公司 基于皮电波的心理测试自动评分方法
CN116058841B (zh) * 2023-01-04 2023-09-19 北京火神永创科技有限公司 基于皮电波的心理测试自动评分方法

Also Published As

Publication number Publication date
CN108052889A (zh) 2018-05-18
CN108052889B (zh) 2020-07-24

Similar Documents

Publication Publication Date Title
WO2019109530A1 (zh) 情绪识别方法、装置及存储介质
US10490195B1 (en) Using system command utterances to generate a speaker profile
CN107704834B (zh) 微表情面审辅助方法、装置及存储介质
US10762299B1 (en) Conversational understanding
US10839790B2 (en) Sequence-to-sequence convolutional architecture
CN107430858B (zh) 传送标识当前说话者的元数据
WO2019085329A1 (zh) 基于循环神经网络的人物性格分析方法、装置及存储介质
US10733387B1 (en) Optimizing machine translations for user engagement
CN110020009B (zh) 在线问答方法、装置及系统
WO2019085330A1 (zh) 人物性格分析方法、装置及存储介质
KR20190024711A (ko) 정보 검증 방법 및 장치
WO2019085331A1 (zh) 欺诈可能性分析方法、装置及存储介质
US20180322264A1 (en) Verification system
Khan et al. A novel audio forensic data-set for digital multimedia forensics
US20170318013A1 (en) Method and system for voice-based user authentication and content evaluation
US10043069B1 (en) Item recognition using context data
US20230215418A1 (en) Terminal and Operating Method Thereof
KR20170126667A (ko) 회의 기록 자동 생성 방법 및 그 장치
US10410655B2 (en) Estimating experienced emotions
US10755171B1 (en) Hiding and detecting information using neural networks
WO2021047376A1 (zh) 数据处理方法、数据处理装置及相关设备
WO2020252930A1 (zh) 问答系统重复问题识别方法、装置、设备及存储介质
US20170091859A1 (en) System for real-time data structuring and storage
US11704585B2 (en) System and method to determine outcome probability of an event based on videos
CN108460077B (zh) 一种指标分析方法、系统以及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18885062

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12/10/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18885062

Country of ref document: EP

Kind code of ref document: A1