CN113243918B - Risk detection method and device based on multi-mode hidden information test - Google Patents
Risk detection method and device based on multi-mode hidden information test Download PDFInfo
- Publication number
- CN113243918B CN113243918B CN202110653116.5A CN202110653116A CN113243918B CN 113243918 B CN113243918 B CN 113243918B CN 202110653116 A CN202110653116 A CN 202110653116A CN 113243918 B CN113243918 B CN 113243918B
- Authority
- CN
- China
- Prior art keywords
- question
- model
- matrix
- scene
- risk detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/164—Lie detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/03—Credit; Loans; Processing thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Strategic Management (AREA)
- Theoretical Computer Science (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Technology Law (AREA)
- Entrepreneurship & Innovation (AREA)
- Social Psychology (AREA)
- Surgery (AREA)
- Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Educational Technology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Psychiatry (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Developmental Disabilities (AREA)
- Educational Administration (AREA)
- Child & Adolescent Psychology (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
Abstract
The application discloses a risk detection method and a device based on multi-mode hidden information test, wherein the risk detection method comprises the following steps: constructing a structured questionnaire and a scene questionnaire feedback matrix; sequentially playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire; acquiring physiological characteristic data and a normalized time sequence thereof based on audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence; constructing a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix to obtain a result matrix; and respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by using the result matrix to obtain a multi-dimensional risk detection result. The method and the device can automatically detect and identify the pressure abnormity of the testee when answering the specific question.
Description
Technical Field
The present application relates to the field of risk management technologies, and in particular, to a risk detection method and apparatus based on a multi-modal hidden information test, and a computer-readable storage medium.
Background
The financial wind control process of bank credit, insurance claim settlement and the like generally comprises three steps: (1) and (5) identity authentication. Used for judging whether the operation is the personal operation; (2) and checking basic information. The system is used for checking the completeness and accuracy of basic information provided by a user; (3) covert information detection (true intent recognition). Whether the client has subjective information concealing behaviors, such as whether the loan application is actual or not, whether an accident actually occurs or not, is detected. True intent recognition typically relies on empirical judgment or the provision of relevant documentation. Also, for example, resume history also has a problem of fraud, and focusing on job duties and personal performances of job seekers, it generally exaggerates their contribution to a certain project. Psychological stress testing techniques can be incorporated into the relevant links to identify subjective information that hides behaviors or intentions from view.
The psychological test technology is that under a specific scene, a psychological tester generates continuous 'stimulation-response' action on a tested person, and extracts a series of physiological characteristics of the tested person, such as language, action, expression, emotion and the like, so as to judge whether the language expression of the tested person is consistent with the mental thinking and the real objective fact or not and whether the probability condition of emotional abnormality exists or not.
In the practical application process, the testee produces psychological abnormal reaction, which is not necessarily caused by lie-scattering, but also possibly caused by the influence of some external environmental factors: (1) the pressure stimuli differ from scene to scene. For example, in a recruitment interview scene, the stress stimulation brought by the environment to the tested person is obviously different; (2) stress stimulation of the psychological tester itself. Different psychological testers or interviewers, or some expression languages and body behaviors which are irrelevant to evaluation or random can bring different external pressure stimulation to the testee, even when the testees face a plurality of psychological testers or interviewers at the same time.
However, the influence of external environmental factors is rarely considered in the prior art, so that the pressure abnormality of the testee in answering the questionnaire cannot be accurately detected.
Disclosure of Invention
The application provides a risk detection method and device based on multi-mode hidden information test and a computer readable storage medium.
In order to solve the above technical problem, a first technical solution provided by the present application is: the risk detection method based on the multi-mode hidden information test comprises the following steps: constructing a structured questionnaire and a scene questionnaire feedback matrix; sequentially playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire, and acquiring audio and video data of a tested person in an interactive process; acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by using the normalized time sequence; constructing a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, and optimizing model parameters of the first model according to the second model to obtain a result matrix; and respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by using the result matrix to obtain a multi-dimensional risk detection result.
In order to solve the above technical problem, a second technical solution provided by the present application is: the risk detection device comprises a questionnaire processing module, a time sequence processing module, a model parameter estimation model and a risk detection model, wherein the questionnaire processing module is used for acquiring a preset questionnaire; the time sequence processing module is used for playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire in sequence and acquiring audio and video data of a tested person in an interactive process; the device is also used for acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence; the model parameter estimation model is used for constructing a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, and optimizing the model parameters of the first model according to the second model to obtain a result matrix; the risk detection model is used for respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by utilizing the result matrix to obtain multi-aspect risk detection results.
In order to solve the above technical problem, a third technical solution provided by the present application is: providing another risk detection device based on a multi-mode hidden information test, wherein the risk detection device comprises a processor and a memory connected with the processor, and the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the risk detection method described above.
In order to solve the above technical problem, a fourth technical solution provided by the present application is: there is provided a computer readable storage medium storing program instructions that, when executed, implement the risk detection method described above.
According to the risk detection method, a structured question questionnaire is constructed; sequentially playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire; acquiring physiological characteristic data and a normalized time sequence thereof based on audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence; constructing a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, and optimizing and obtaining model parameters of the first model according to the second model to obtain a result matrix; and respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by using the result matrix to obtain a multi-dimensional risk detection result. The method and the device can automatically detect and identify the pressure abnormity of the testee when answering the questionnaire.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flowchart of an embodiment of a risk detection method based on a multi-modal hidden information test provided in the present application;
FIG. 2 is a flow diagram of sub-steps of step S11 of the risk detection method of FIG. 1
FIG. 3 is a schematic sequence diagram of one embodiment of a questionnaire provided herein;
FIG. 4 is a timing diagram illustrating an embodiment of a conversational stimulus matrix provided herein;
FIG. 5 is a flow diagram of sub-steps of step S14 of the risk detection method of FIG. 1;
fig. 6 is a schematic structural diagram of an embodiment of the risk detection apparatus based on a multi-modal hidden information test provided in the present application;
fig. 7 is a schematic structural diagram of another embodiment of the risk detection apparatus based on the multi-modal hidden information test provided in the present application;
fig. 8 is a schematic structural diagram of a computer-readable storage medium of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The psychological test technique comprises three elements: (1) test scenarios (e.g., inquiries, recruiting interviews, bank face-signs, etc.); (2) psychological testers and test questionnaires; (3) the tested person.
The compilation of the psychological test questionnaire is the core content of the test, and the quasi-rope Question test method CQT (control Question test) and the secret Information test method CIT (confidential Information test) are the two most commonly used test methods. The CQT test question consists of three types of related questions, unrelated questions and quasi-rope questions; the CIT test question consists of a key question (related question) and a companionship question (background question). Different from CQT, the question of CIT is in unit of group, a complete set of CIT test questions is composed of a plurality of groups, and each group is composed of a plurality of questions. Theoretically, the CIT can be used as long as the hidden information known only by the testee exists, namely the testee trying to hide the information can easily distinguish the key problem and the companioning problem, and if the physiological responses of the testee to the key problem and the companioning problem are not different, the possibility that the testee has smaller hidden possibility to the key problem can be inferred; if the physiological response of the key question is significantly stronger than that of the companionship question, it is inferred that the tested person has a greater possibility of hiding the key question. The test structure of the two types of test methods contains related and contrast two types of stimulation, and the core mechanism of the test methods adopts different problems to arouse the physiological characteristic abnormity of a tested person. Two types of testing methods are widely applied to the lie detection field, and the lie detection technology does not identify the lie itself, but detects the abnormality of psychological stress through physiological index variation under certain external pressure.
Based on the principle, the risk detection method based on the multi-mode hidden information test can be applied to the related fields of financial risk management, human resource interview and the like, and can automatically detect and identify the psychological or emotional stress abnormality of the testee when answering the specific key question.
The present application will be described in detail with reference to the accompanying drawings and examples.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a risk detection method based on a multi-modal hidden information test according to an embodiment of the present application.
The risk detection method is applied to the terminal equipment, wherein the terminal equipment can be a server, can also be mobile equipment, and can also be a system formed by the cooperation of the server and the mobile equipment. Accordingly, each part, such as each unit, sub-unit, module, and sub-module, included in the terminal device may be all disposed in the server, may be all disposed in the mobile device, and may be disposed in the server and the mobile device, respectively.
Further, the server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules, for example, software or software modules for providing distributed servers, or as a single software or software module, and is not limited herein.
The risk detection method based on the multi-mode hidden information test comprises the following specific steps:
step S11: and constructing a structured question questionnaire and a scene questionnaire feedback matrix.
The risk detection device of the embodiment of the present application normalizes and structures the heart test question text of the CQT or CIT first, and the processing method includes, but is not limited to: removal of stop words, spelling correction, rewriting of abbreviations, etc. Further, the staff can also set the maximum word length of the centering question sentence text asLOn the one hand for a number of words greater than LThe question text is cut off, and on the other hand, the number of corresponding words is less than LThe question text is filled, so that the effect of measuring the length of the question text at a unified level is achieved.
The risk detection device may obtain the preset questionnaire in this embodiment based on the CQT or CIT heart-measuring questionnaire text in combination with the above processing method, or may introduce a scene questionnaire without actual semantics, recombine and define the CQT or CIT heart-measuring questionnaire text, and generate a questionnaire that can be directed to different scenes, specifically please refer to fig. 2 continuously, and fig. 2 is a sub-step flowchart of step S11 of the risk detection method shown in fig. 1.
Step S11 in the embodiment of the present application may specifically include the following sub-steps:
step S21: defining a dictionary matrix, traversing words in the dictionary matrix, and defining a corresponding dictionary matrix.
Wherein the risk detection means defines a dictionary matrix
Total number of words in the dictionary matrix isN。
Is an identity matrix
To (1) akThe columns of the image data are,
。
get the first in the dictionary matrixiThe word is
Wherein
Definition dictionary matrix abbreviation
Setting the first word of the dictionary matrix to be an empty character, i.e.
Representing a null character.
Step S22: question and answer libraries are defined.
The risk detection device utilizes a dictionary matrix to define a question bank and an answer bank, wherein a plurality of questions are defined in the question bank
The answer library has several defined answer sentences
The number of question lines in the question bank is
Number of answers in answer library
. Wherein, the firstiThe dictionary structure of the heart-shaped question sentence is expressed as
Of 1 atiThe dictionary structure of the answer of the testee of the heart-shaped question sentence is expressed as
。
Step S23: the dictionary matrix is used to define an initial scene question and a smooth scene question.
Wherein the risk detection means further defines two types of special questions, namely an initial scene question
And smooth scene question
Wherein, in the step (A),
. Therefore, only the scene question in the current scene needs to be marked for distinguishing from other questions, and actual text content is not needed. Smooth scene question
May be provided as a plurality of different sentences (corresponding to each question-evaluating answer). To simplify the illustration, embodiments of the present application smooth scene question after each evaluation question answer
The simplified process is set to be the same statement, and the generality and the universality of the whole algorithm and process are not lost.
Note that the initial scene question sentence
Only one question is used in a complete interactive process, and a smooth scene question is used once after the answer of the testee is completed, specifically referring to fig. 3, where fig. 3 is a sequence diagram of an embodiment of a question questionnaire provided in the present application. At the beginning of the interactive process, the risk detection device collects the scene pressure of the testee through an initial scene question sentence, and then the scene pressure of the testee is collected through a smooth scene question sentence after a question stage, a thinking stage and an answer stage in sequenceAnd collecting, and so on.
According to the risk detection method, two types of scene question sentences without actual semantics are introduced, and the psychological pressure brought to a tested person by a current scene or a current environment can be analyzed and extracted by combining stimulation-response model parameter estimation, so that the risk detection method can be directly suitable for different scenes.
Step S24: a scene questionnaire feedback matrix is defined using a question bank, an answer bank, an initial scene question, and a smooth scene question.
Wherein the risk detection means define
Scene questionnaire feedback matrix
Of 1 atjThe column vector is
，
. Wherein the content of the first and second substances,lconstructing a scene questionnaire from a questionnaire library
Extracting the number of a plurality of question sentences, i.e. arranging to form a question sub-sentence library
. Corresponding sub-answer sentence library
From a library of answers by interaction
And dynamic extraction, wherein a plurality of answers correspond to a question sentence.
Specific correspondence may be referred to as follows:
……
Step S25: and defining the attribute of the heart measuring question-answer matrix by using the attribute classification of the question library.
Wherein the risk detection means definem×lCardiac survey question-answer matrix attributeB，mAnd classifying the attributes of various question sentences. Library of presentation questions
To middlejWhether the question is of the firstiAnd if the attribute is positive, the value is 1, and if the attribute is negative, the value is 0.
For example, the test question of CQT consists of three categories, related questions, unrelated questions, and quasi-rope questions; the CIT test question consists of a key question (related question) and a companionship question (background question). One question sentence can belong to both CQT related questions and CIT key questions, and in the risk control field, the CQT related questions or the CIT key questions are key wind control points.
Step S12: and sequentially playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire, and acquiring audio and video data of the tested person in the interactive process.
After the structured processing of the questionnaire is completed, a background evaluation system carried on the risk detection device establishes network communication with a foreground user terminal, and a testee receives the information of the questionnaire through the foreground user terminal and acquires the audio and video data of the testee through the foreground user terminal. The foreground user terminal of the embodiment of the application includes, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices and the like.
After the testee enters the conversation, the foreground user terminal can start to collect audio and video stream and record the start time
Independently collectzThe physiological characteristics of the tested person, whereinjThe time sequence of the physiological characteristic is
At a frequency of
。
In the real-time interactive process, the foreground user terminal obtains the 0 th question in the question questionnaire, namely the initial scene question, and carries out random delay operation. As shown in fig. 3, (1) a question phase. Foreground user terminal obtaining first in questionnaireiA question sentence, using TTS (text to speech ) to perform speech synthesis broadcast, simultaneously displaying the question sentence and optional answer at the foreground user terminal, recording the start time of broadcast display
And broadcast end time
. (2) A thought phase and an answer phase. Respectively recording the initial response time of the testee
And completion of the response time
The foreground user terminal continuously collects audio and video stream and converts the collected answer sentence voice data into text answer
And updating the scene questionnaire feedback matrixQ， 
The time is the feedback of the testee's thought. (3) A mood smoothing phase. Delay waiting, structure
Corresponding block session stimulus matrix
To accomplish the followingjTime series processing of heterogeneous non-uniform frequency signals of individual physiological characteristics. Order counteri=i+1, wheniLess than the total number of question sentences evaluated this time 3lAnd +1, circularly executing the interactive process.
Step S13: and acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence.
The acquisition of the physiological characteristic data based on the audio and video data in the embodiment of the application can be specifically based on the acquisition of non-contact or contact physiological characteristic data such as audio and video, electroencephalogram and the like through human-computer automatic interaction.
Because the identification error of the single-dimension physiological signal and the derived characteristic index thereof is large, physiological characteristic data with different dimensions, such as a dynamically increased blood volume pulse BVP signal, a Mel frequency cepstrum MFCC coefficient and the like, can be dynamically introduced in the practical application process, and currently, the related technology lacks general processing on the physiological characteristic data with different frequencies, so that the dynamic increase or decrease of the physiological characteristic dimension cannot be dynamically self-adapted.
Therefore, the non-contact or contact physiological characteristic signals such as audio/video signals, electroencephalogram signals and the like acquired in the embodiment of the application are various original coefficients or parameters, including but not limited to audio/video signals, electroencephalogram signals and other signals which can be time-sequenced, for example, an average optical flow direction histogram of a two-dimensional face ROI area, a one-dimensional blood volume pulse BVP signal and the like; speech signals, e.g., one-dimensional volume, multi-dimensional mel-frequency cepstrum MFCC coefficients, etc.; user behavior information, such as one-dimensional time information of a handwritten signature, and the like.
The embodiment of the application can directly use derived characteristics or parameters of non-contact or contact physiological characteristic signals, such as micro expressions, eye movements, heart rate and the like, and also can directly use original coefficients or parameters thereof, such as MHOOF of a corresponding region ROI can extract micro expression characteristics and eye movement characteristics, and a BVP signal can extract heart rate. The latter is recommended because: (1) in the process of extracting or training the derived features or parameters, loss and classification errors exist in the information of the original coefficients or parameters, so that the accuracy of stress emotion evaluation is influenced; (2) some derived features or parameters are inherently less stable.
Wherein the risk detection means defines individual physiological characteristics of the subjectjNormalized time series of
At a frequency of
. Further, a normalized time series is constructed
Corresponding to
Conversational stimulus matrix
。
In particular for different frequencies
Time series vector of
All have corresponding conversational stimulus matrix
Wherein the total time length
，
To evaluate the total length of time. The tested person enters the conversation with the time
Of 1 atiThe starting time of the question report and display is
The broadcast end time is
(ii) a The subject answers the firstiThe starting time of a question is
Completion of the answer time
Wherein, in the step (A),
including a questioning phase and a thinking phase.
Therefore, the conversational stimulus matrix of the embodiment of the present application can be expressed as:
wherein the content of the first and second substances,
in total 3l+1 block matrix
Each block matrix
A size of
，
Column values are all
Column vectors, i.e.
， 
Is the product of the Kronecker reaction,
is element 1, length
Referring to fig. 4, fig. 4 is a timing diagram of an embodiment of a conversational stimulation matrix provided in the present application.
Wherein the content of the first and second substances,
，
，
，
， 
in addition, the corresponding time sequence of the physiological characteristic signal of the discontinuous type
Need to be aligned with
And (5) carrying out structure adjustment. For example, speech signals mainly occur only in the answer phase, other phases not
Then will be
The corresponding block matrixes of other non-initial scene stages are all set to 0, i.e.
Then according to the question vector pairs of the preceding and following paragraphs
Structural adjustment is carried out according to the ratio of 2:3:1, namely
I.e. by
，
Generating conversational stimulus matrices for improved performance
The risk detection means may also use ADF root-of-unit testing for each feature time series
Performing a stationarity check if
If it is a non-stationary time sequence, then it is determined for the non-stationary time sequence
Performing difference processing to
Is converted into and
stationary sequence of equal time length
Will be sequenced
The front of the site is filled with 0.
According to the embodiment of the application, through heterogeneous non-same-frequency signal time sequence processing, the calculation processing of the dimension-variable physiological characteristic data can be self-adapted. As long as different frequencies are determined
Time series vector of
Can generate corresponding conversational stimulus matrix
。
Step S14: and constructing a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, and optimizing the model parameters of the first model according to the second model to obtain a result matrix.
In which the risk detection device needs to further perform "stimulus-response" model parameter estimation, please refer to fig. 5, and fig. 5 is a flowchart of the sub-steps of step S14 of the risk detection method shown in fig. 1.
Step S14 in the embodiment of the present application may specifically include the following sub-steps:
step S31: the model parameters of the first model comprise model residuals and parameters to be estimated.
Wherein the risk detection device is based on physiological characteristics of single testeejTime series of
And session stimulus matrix
The following first model was constructed:
wherein the content of the first and second substances,
and
are all time lag variable parameters; residual error
Is a stationary white noise with a mean of 0 and a constant variance;
is the session stimulation matrixtTime
Column vector components representing different question and answer sentences, in commonb=2l+2, their corresponding parameters to be estimated
Namely, scene environment and different question-pair physiological characteristics at different stages
Influence of fluctuations, e.g.
And
is the current initial scene question
And smooth scene question
To pair
The influence of the wave motion is such that,
and
is to ask questions and ask sentences
And answer sentences
To pair
The effect of the fluctuations. Through the model, a psycho-physiological dynamic mechanism model under a stimulation-response paradigm is introduced, and an individual can be directly evaluated.
Then, the risk detection device determines the optimal model order for the first model according to an AIC (Akaike information criterion) criterion and a BIC (Bayesian information criterion) criterion
And performing least square estimation on the model parameters. Can be initially selected
Determining an optimal model order for the first model based on the AIC criterion and the BIC criterion
. The BIC criterion under the condition that the time series observed values are enough, the selected orders of the BIC can converge to the actual orders, namely, the consistency exists. The AIC criterion does not have consistency and the AIC is usually chosen to be of a larger order than BIC, so the model scaling can be done using the BIC criterion only.
Step S32: and carrying out autocorrelation and heteroscedasticity test on the model residual.
Step S33: and if the residual sequence corresponding to the model residual has heteroscedasticity, establishing a second model based on the residual sequence of the model residual.
Further, the risk detection device performs Ljung-Box Q test and Eagle ARCH test on the standardized residual error of the first model, performs autocorrelation and heteroscedasticity test on the model residual error, and if the residual error sequence is not a white noise sequence and has heteroscedasticity, the following second model needs to be established:
wherein the content of the first and second substances,
and
are all time lag variable parameters. Higher order second models increase model instability, and thus the initial selection
Determining the optimal model order for the second model according to the AIC criterion and the BIC criterion
。
Step S34: and obtaining the optimal model order of the second model.
Step S35: and recalculating the model parameters by combining the optimal model order of the second model and the first model.
Wherein the risk detection means incorporates the optimal model order of the second model
Re-estimating model parameters of a first model
And checking whether the model residual error item conforms to the white noise process again. If not, the residual error item is judged to have the related information which is not extracted, and the selection is carried out
，
Sequentially determining the optimal model order for the second model according to the AIC criterion and the BIC criterion
。
Further, the risk detection device calculates the next physiological characteristic of the tested person in sequence
Time series of
And session stimulus matrix
To obtain corresponding model parameters
Step S36: the result matrix is defined using the recalculated model parameters.
Wherein the risk detection means defineb×zResult matrixR，zDetermining statistical significance level for the number of all physiological characteristics of the tested person, for example, selecting significance level of 1%, 5%, 10%
Is at a significance level ofiQuestion or answer pairjInfluence coefficient of physiological characteristic change
If the significance level is not satisfied
Is set to 0.
Step S15: and respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by using the result matrix to obtain a multi-dimensional risk detection result.
And the risk detection device performs evaluation scale analysis based on the result matrix, namely completes environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics, individual characteristic sensitivity detection and multi-sample detection.
Specifically, the first step, ambient scene pressure detection. Based on the result matrix of the individual testeesRCalculating a matrixRLine mean of
If, if
And
the maximum value indicates that the pressure of the tested person is too large in the current scene, and the tested person has great influence on other evaluation results.
And secondly, detecting question pressure of the global physiological characteristics. For result matrixRIs prepared by extractinglQuestion and calculate the mean value of each line
，
By measuring question-answer attribute matrix through heartBFind firstiThe attribute of each question is related to CQT or key problem of CIT, if
The fact that the tested person has psychological stress abnormity when the question is reported shows that the tested person meets the statistical significance level; get
The maximum question is the related or key question with the maximum pressure of the tested person
。
The third stepAnd detecting the answer pressure of the global physiological characteristic. For result matrixRIs prepared by extractinglAnswers and calculates the mean value of each line
，
Which is
The tested person has abnormal psychological pressure when answering the current question and meets the statistical significance level; get
Maximum answer, i.e. answer of maximum pressure of tested person
Their corresponding question sentence
If the question pressure detection of the global physiological characteristics also satisfies the significance level in the second step, the question and the answer have larger risk points.
And fourthly, detecting individual feature sensitivity. Based on the result matrix of the individual testeesRCalculating the column mean of the matrix
，
If, if
Or close to 0, indicating that the tested person is the first to be tested in the current scenejThe physiological characteristics are not sensitive to the question of each question in statistical sense, i.e. the first person to be testedjThe individual physiological characteristic indicators were not valid in this evaluation.
And fifthly, detecting multiple samples. In the same scene, calculating the result matrix of each tested person
And repeating the first step to the fourth step.
The risk detection device of the embodiment of the application carries out classification and vectorization processing on the CQT or CIT test question, introduces scene and scene pressure variables, and constructs related questionnaires according to different scenes; secondly, constructing a time sequence based on a session state for physiological characteristic signals which directly use a bottom layer continuous type and have different frequencies, then constructing a time sequence model, estimating model parameters, and calculating the variation intensity of physiological characteristics with different dimensions when the testee answers each question; and finally, obtaining an evaluation scale matrix, and effectively and automatically detecting and identifying the abnormal psychological or emotional pressure of the testee when answering the specific key questions.
The above embodiments are only one of the common cases of the present application and do not limit the technical scope of the present application, so that any minor modifications, equivalent changes or modifications made to the above contents according to the essence of the present application still fall within the technical scope of the present application.
With continuing reference to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of the risk detection apparatus based on multi-modal hidden information test provided in the present application. The risk detection device 600 includes a question questionnaire processing module 61, a time series processing module 62, a model parameter estimation model 63, and a risk detection model 64, wherein:
the questionnaire processing module 61 is used for constructing a structured questionnaire and a scene questionnaire feedback matrix.
The time sequence processing module 62 is configured to sequentially play an initial scene question, a question bank question and a smooth scene question in the question questionnaire, and acquire audio and video data of a tested person in an interaction process; and the method is also used for acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence.
The model parameter estimation model 63 is configured to construct a first model and a second model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, and optimize model parameters of the first model according to the second model to obtain a result matrix.
The risk detection model 64 is configured to perform environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics, and/or individual characteristic sensitivity detection by using the result matrix, respectively, to obtain a multi-dimensional risk detection result.
Please refer to fig. 7, which is a schematic structural diagram of another embodiment of the risk detection apparatus based on the multi-modal hidden information test according to the present application. The risk detection means comprises a memory 72 and a processor 71 connected to each other.
The memory 72 is for storing program instructions for implementing the risk detection method of any of the above.
The processor 71 may also be referred to as a CPU (Central Processing Unit). The processor 71 may be an integrated circuit chip having signal processing capabilities. The processor 71 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 72 may be a memory bank, a TF card, etc. and may store all the information in the string matching prediction apparatus, including the input raw data, the computer program, the intermediate operation results, and the final operation results. It stores and retrieves information based on the location specified by the controller. With the memory, the string matching prediction device has a memory function, and normal operation can be guaranteed. The memory of the string matching prediction device can be classified into a main memory (internal memory) and an auxiliary memory (external memory) according to the use, and also into an external memory and an internal memory. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a system server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application.
Please refer to fig. 8, which is a schematic structural diagram of a computer-readable storage medium according to the present application. The storage medium of the present application stores a program file 81 capable of implementing all the above risk detection methods, where the program file 81 may be stored in the storage medium in the form of a software product, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the methods according to the embodiments of the present application. The aforementioned storage device includes: various media capable of storing program codes, such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices, such as a computer, a server, a mobile phone, and a tablet.
The above embodiments are merely examples and are not intended to limit the scope of the present disclosure, and all modifications, equivalents, and flow charts using the contents of the specification and drawings of the present disclosure or those directly or indirectly applied to other related technical fields are intended to be included in the scope of the present disclosure.
Claims (9)
1. A risk detection method based on a multi-mode hidden information test is characterized by comprising the following steps:
constructing a structured questionnaire and a scene questionnaire feedback matrix;
sequentially playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire, and acquiring audio and video data of a tested person in an interactive process;
acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by using the normalized time sequence;
constructing a first model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, wherein model parameters of the first model comprise model residual errors and parameters to be estimated;
carrying out autocorrelation and heteroscedasticity test on the model residual error, and if the heteroscedasticity exists in a residual error sequence corresponding to the model residual error, establishing a second model based on the residual error sequence of the model residual error;
obtaining the optimal model order of the second model, and recalculating model parameters by combining the optimal model order of the second model and the first model;
defining a result matrix by using the recalculated model parameters;
and respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by using the result matrix to obtain multi-aspect risk detection results.
2. The risk detection method of claim 1, further comprising:
defining a dictionary matrix, traversing words in the dictionary matrix, and defining a corresponding dictionary matrix;
defining a question library and an answer library by using the dictionary matrix, wherein a plurality of question libraries are defined in the question library, and a plurality of answer library questions are defined in the answer library;
defining the initial scene question and a plurality of smooth scene questions by using the dictionary matrix;
defining a scene questionnaire feedback matrix by using the sentence query library, the answer library, the initial scene question and the smooth scene question;
and defining the attribute of the heart measuring question-answer matrix by using the attribute classification of the question library.
3. The risk detection method according to claim 2,
the sequentially playing of the initial scene question, the question bank question and the smooth scene question in the question questionnaire includes:
when the interaction starts, the initial scene question sentence is played;
circularly executing the following steps: and sequentially acquiring and playing the question and question library question, the answer library question and the smooth scene question.
4. The risk detection method according to claim 2,
the structured question questionnaire comprises:
normalizing the question text of the question questionnaire, and increasing and deleting the number of words of the question text so as to fix the length of the preset text.
5. The risk detection method according to claim 1,
before the constructing a session stimulation matrix by using the normalized time series, the risk detection method further includes:
performing stationarity test on the normalized time sequence by using unit root test;
and if the normalized time sequence is a non-stationary time sequence, performing differential processing on the normalized time sequence to generate a stationary time sequence with the same time length as the normalized time sequence.
6. The risk detection method according to claim 1,
before the self-correlation and heteroscedasticity test is performed on the model residual, the risk detection method comprises the following steps:
obtaining an optimal model order of the first model;
and performing least square estimation on the model parameters based on the optimal model order of the first model.
7. A risk detection device based on a multi-mode hidden information test is characterized by comprising a question questionnaire processing module, a time sequence processing module, a model parameter estimation model and a risk detection model, wherein,
the questionnaire processing module is used for constructing a structured questionnaire;
the time sequence processing module is used for playing an initial scene question, a question bank question and a smooth scene question in the question questionnaire in sequence and acquiring audio and video data of a tested person in an interactive process; the device is also used for acquiring physiological characteristic data and a normalized time sequence thereof based on the audio and video data, and constructing a session stimulation matrix by utilizing the normalized time sequence;
the model parameter estimation model is used for constructing a first model according to the normalized time sequence of the physiological characteristic data and the session stimulation matrix, the model parameters of the first model comprise model residual errors and parameters to be estimated, the model residual errors are subjected to autocorrelation and heteroscedasticity test, if the residual error sequence corresponding to the model residual errors has heteroscedasticity, a second model is established based on the residual error sequence of the model residual errors, the optimal model order of the second model is obtained, the model parameters are recalculated by combining the optimal model order of the second model and the first model, and a result matrix is defined by the recalculated model parameters;
the risk detection model is used for respectively carrying out environmental scene pressure detection, question pressure detection of global physiological characteristics, answer pressure detection of global physiological characteristics and/or individual characteristic sensitivity detection by utilizing the result matrix to obtain multi-aspect risk detection results.
8. A risk detection device based on a multi-mode hidden information test is characterized by comprising a processor and a memory connected with the processor, wherein,
the memory stores program instructions;
the processor is configured to execute the program instructions stored in the memory to implement the risk detection method of any of claims 1-6.
9. A computer readable storage medium, characterized in that the storage medium stores program instructions that, when executed, implement the risk detection method of any of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653116.5A CN113243918B (en) | 2021-06-11 | 2021-06-11 | Risk detection method and device based on multi-mode hidden information test |
| PCT/CN2022/089328 WO2022257630A1 (en) | 2021-06-11 | 2022-04-26 | Risk detection method and apparatus based on multi-modal concealed information test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653116.5A CN113243918B (en) | 2021-06-11 | 2021-06-11 | Risk detection method and device based on multi-mode hidden information test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113243918A CN113243918A (en) | 2021-08-13 |
| CN113243918B true CN113243918B (en) | 2021-11-30 |
Family
ID=77187611
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110653116.5A Active CN113243918B (en) | 2021-06-11 | 2021-06-11 | Risk detection method and device based on multi-mode hidden information test |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113243918B (en) |
| WO (1) | WO2022257630A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113243918B (en) * | 2021-06-11 | 2021-11-30 | 深圳般若计算机系统股份有限公司 | Risk detection method and device based on multi-mode hidden information test |
| CN116058841B (en) * | 2023-01-04 | 2023-09-19 | 北京火神永创科技有限公司 | Psychological test automatic scoring method based on skin electric wave |
| CN117792800B (en) * | 2024-02-28 | 2024-05-03 | 四川合佳科技有限公司 | Information verification method and system based on Internet of things security evaluation system |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140019383A1 (en) * | 2012-07-16 | 2014-01-16 | Dennis F. Mahoney | Means by which to examine and clarify responsibilities between counter parties in a fiduciary relationship |
| EP2973282A4 (en) * | 2013-03-13 | 2016-11-16 | Guardian Analytics Inc | Fraud detection and analysis |
| CN105991593B (en) * | 2015-02-15 | 2019-08-30 | 阿里巴巴集团控股有限公司 | A kind of method and device identifying consumer's risk |
| US20180140241A1 (en) * | 2015-05-04 | 2018-05-24 | Kontigo Care Ab | Method and device for estimating a risk of relapse of addictive behaviour |
| CN108125686B (en) * | 2017-07-13 | 2021-02-12 | 广东网金控股股份有限公司 | Anti-fraud method and system |
| US11033216B2 (en) * | 2017-10-12 | 2021-06-15 | International Business Machines Corporation | Augmenting questionnaires |
| CN108537160A (en) * | 2018-03-30 | 2018-09-14 | 平安科技（深圳）有限公司 | Risk Identification Method, device, equipment based on micro- expression and medium |
| CN109302338B (en) * | 2018-08-31 | 2022-04-19 | 南昌努比亚技术有限公司 | Intelligent risk prompting method, mobile terminal and computer readable storage medium |
| CN109461073A (en) * | 2018-12-14 | 2019-03-12 | 深圳壹账通智能科技有限公司 | Risk management method, device, computer equipment and the storage medium of intelligent recognition |
| CN109919426A (en) * | 2019-01-24 | 2019-06-21 | 平安科技（深圳）有限公司 | Check interview lie detecting method, server and computer readable storage medium |
| CN110223158B (en) * | 2019-05-21 | 2023-08-18 | 平安银行股份有限公司 | Risk user identification method and device, storage medium and server |
| CN110427803A (en) * | 2019-06-18 | 2019-11-08 | 平安科技（深圳）有限公司 | Lie detecting method, device, electronic equipment and storage medium based on video analysis |
| CN110507334A (en) * | 2019-08-21 | 2019-11-29 | 珠海学之渔心理咨询有限公司 | A kind of adult's psychological assessment method |
| CN112704499B (en) * | 2019-10-25 | 2023-11-07 | 苏州心吧人工智能技术研发有限公司 | Intelligent psychological assessment and intervention system and method based on independent space |
| CN112022172B (en) * | 2020-09-03 | 2023-03-28 | 北京中科心研科技有限公司 | Pressure detection method and device based on multi-modal physiological data |
| CN112331230A (en) * | 2020-11-17 | 2021-02-05 | 平安科技（深圳）有限公司 | Method and device for identifying fraudulent conduct, computer equipment and storage medium |
| CN113243918B (en) * | 2021-06-11 | 2021-11-30 | 深圳般若计算机系统股份有限公司 | Risk detection method and device based on multi-mode hidden information test |
-
2021
- 2021-06-11 CN CN202110653116.5A patent/CN113243918B/en active Active
-
2022
- 2022-04-26 WO PCT/CN2022/089328 patent/WO2022257630A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022257630A1 (en) | 2022-12-15 |
| CN113243918A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Song et al. | Spectral representation of behaviour primitives for depression analysis | |
| CN113243918B (en) | Risk detection method and device based on multi-mode hidden information test | |
| US11545173B2 (en) | Automatic speech-based longitudinal emotion and mood recognition for mental health treatment | |
| CN110457432B (en) | Interview scoring method, interview scoring device, interview scoring equipment and interview scoring storage medium | |
| CN111461176B (en) | Multi-mode fusion method, device, medium and equipment based on normalized mutual information | |
| Wu et al. | Automatic speech emotion recognition using modulation spectral features | |
| Pollreisz et al. | A simple algorithm for emotion recognition, using physiological signals of a smart watch | |
| Moro-Velázquez et al. | Modulation spectra morphological parameters: A new method to assess voice pathologies according to the grbas scale | |
| Fontes et al. | Classification system of pathological voices using correntropy | |
| WO2022127042A1 (en) | Examination cheating recognition method and apparatus based on speech recognition, and computer device | |
| CN115862868A (en) | Psychological assessment system, psychological assessment platform, electronic device and storage medium | |
| CN114138960A (en) | User intention identification method, device, equipment and medium | |
| CN114595692A (en) | Emotion recognition method, system and terminal equipment | |
| Sultana et al. | A non-hierarchical approach of speech emotion recognition based on enhanced wavelet coefficients and K-means clustering | |
| KR20220005232A (en) | Method, apparatur, computer program and computer readable recording medium for providing telemedicine service based on speech recognition | |
| KR102422844B1 (en) | Method of managing language risk of video content based on artificial intelligence | |
| Schafer et al. | Noise-robust speech recognition through auditory feature detection and spike sequence decoding | |
| CN115512692A (en) | Voice recognition method, device, equipment and storage medium | |
| CN109389493A (en) | Customized test question mesh input method, system and equipment based on speech recognition | |
| CN113836273A (en) | Legal consultation method based on complex context and related equipment | |
| CN114881668A (en) | Multi-mode-based deception detection method | |
| Kao et al. | Voice Response Questionnaire System for Speaker Recognition Using Biometric Authentication Interface. | |
| van Bemmel et al. | Automatic Selection of the Most Characterizing Features for Detecting COPD in Speech | |
| Emerich et al. | Bimodal approach in emotion recognition using speech and facial expressions | |
| CN116312486B (en) | Emotion recognition method, emotion recognition device, computer-readable storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |