TWM630105U - System for detecting a target lies - Google Patents

Abstract

According to one embodiment of the invention, a system for detecting cyber criminal emotion of a target is disclosed. The system comprises a storing device, a connecting device and a processing device. The storing device provides a data storage space. The processing device electrically connects to the storing device through the connecting device to access the data storage space. The processing device comprises a feature extraction module, a micro-expression recognition module, a voice emotion recognition module and a deep learning model. The feature extraction module low-level analyzes an image to extract multi-dimensional feature. The micro-expression recognition module high-level analyzes the multi-dimensional feature to identify at least one micro-expression action. The voice emotion recognition module generates a spectrum from a voice signal and analyzes the spectrum to obtain an emotion feature. The deep learning model generates a probability that the target lies according to the multi-dimensional feature, the micro-expression action and the emotion feature.

Description

Lie detection system

This work is related to a lie detection system that identifies whether a target person is lying, especially a lie detection system that assists in checking financial transactions or preventing computer crime.

Since the advent of lie detectors more than 100 years ago, the development still relies on various sensing devices to sense changes in subjects such as heartbeat, blood pressure, skin conductivity, etc., so as to determine whether the subjects lie. Despite so much information, the accuracy of lie detectors has been questioned. Moreover, with the rapidly changing landscape of financial transactions, polygraph machines, which require in-person testing by subjects, are unsuitable for checking financial transactions or preventing computer crime. Therefore, there is an urgent need to develop a new, easy-to-use and accurate lie detection technology.

One of the purposes of this creation is to provide a lie detection system that can combine the analysis results of image data and audio signals to generate a probability of a target person lying, thereby checking whether financial transactions are abnormal or even preventing illegal computer crime incidents, Preferably, artificial intelligence technology with higher accuracy can be introduced, such as: using deep learning models for analysis, without setting up various sensing devices for sensing human physiological characteristics such as heartbeat, blood pressure, skin conductivity, etc., so as to facilitate use. .

According to another aspect of the present invention, a lie identification system is provided for identifying whether a target person is lying, including a storage device, a connection device and a processing device. The storage device provides a Data storage space. The processing device is electrically connected to the storage device through the connection device to use the data storage space, and includes a feature extraction module, a micro-expression recognition module, a speech emotion recognition module and a deep learning model. The feature extraction module performs low-level analysis on an image data to extract multi-dimensional features; the micro-expression recognition module performs high-level analysis on the multi-dimensional features to recognize at least one micro-expression action; the speech emotion recognition module is based on a sound signal generating a spectrogram data, analyzing the spectrogram data to obtain an emotional feature; and the deep learning model generates a probability of the target person lying according to the aforementioned multi-dimensional features, micro-expression movements and emotional features.

1: Lie detection system

10: Processing device

15: Connection device

20: Storage device

30: Feature extraction module

31: CNN unit

40: Micro-expression recognition module

41: Heuristic filtering unit

42: GCN unit

50: Speech emotion recognition module

51: Spectrum conversion unit

52: CNN unit

60: Deep Learning Models

61: Feature Hidden Space

62: Multi-mode judgment module

63: Identifying Regression Classification Sets

S100, S200, S300, S400: Steps

FIG. 1 shows a system architecture diagram of a lie detection system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a method for detecting lies according to an embodiment of the present invention.

FIG. 3 shows a functional block diagram of a lie detection system according to another embodiment of the present invention.

FIG. 4 shows a functional block diagram of a deep learning model according to another embodiment of the present invention.

To further illustrate the various embodiments and their advantages, the present invention provides the following description in conjunction with the drawings. These drawings are a part of the disclosure content of the present invention, which are mainly used to illustrate the embodiments, and can be combined with the relevant descriptions in the specification to explain the operation principles of the embodiments. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementations and the advantages of the present invention. Elements in the figures are not drawn to scale and similar reference numerals are generally used to designate similar elements. As disclosed herein, "embodiments", "examples" and "this embodiment" do not specifically refer to a single embodiment, but may be implemented according to different combinations of the present invention without departing from the spirit and scope of the present invention. vocabulary used here The specific embodiments are only used to illustrate the principles of this creation, and should not be limited to this creation. Therefore, if "in" can include "within" and "on", "one" and "the" can include singular or plural; "borrow" can mean "from", "if" can mean "when" or "if". "Once" is displayed in the text before and after. Also, "and/or" can include any possible combination of the associated elements.

This specification discloses several examples of lie detection methods and systems that can identify whether a target person is lying, which can combine the analysis results of image data and audio signals respectively to generate a probability that a target person is lying, thereby checking whether a financial transaction is not. Abnormal or even to prevent illegal computer crime events, preferably, artificial intelligence technology with higher accuracy can be introduced, such as: deep learning (Deep Learning) model for analysis, without setting sensors such as heartbeat, blood pressure, skin Various sensing devices for human physiological characteristics such as electrical conductivity for ease of use. Please refer to FIG. 1 to show a lie detection system according to an embodiment of the present invention. The lie detection system 1 includes a processing device 10 , a connecting device 15 and a storage device 20 . The processing device 10 is electrically connected to and controls the operation of the storage device 20 through the connection device 15 to use the data storage space of the storage device 20 and access the data in the data storage space. The storage device 20 can form a communication link with an external database (not shown in the figure) or an image recording device, such as a camera, through a communication link device (not shown in the figure), thereby receiving a video of a target person, For example, a video of a target person's face and audio recordings is captured, and when the target person performs financial transactions, the video can be processed and divided into image data and audio signals before or after being input to the storage device 20 . In this example, the processing device 10 can be a processor, such as: a central processing unit (CPU), a graphics processing unit (GPU), etc., which preferably can perform matrix parallel operations; the connecting device 15 can be a signal and data transmission device. Channel, such as: bus bar or motherboard, etc.; the storage device 20 can be a memory, a hard disk or a database, etc.; the communication connection device can be a wireless communication connection device, a wired communication connection device, etc.; however, please note This creation is not limited to this. In this embodiment, a central processing unit is used to implement the lie detection system 1, and a The memory implements the storage device 20, and the communication connection device is implemented as a wireless communication connection device, however in other embodiments, the lie detection system may be implemented with other components.

In this embodiment, the processing device 10 is configured to execute the lie identification method shown in FIG. 2 , which mainly includes four steps S100 to S400 to analyze the image data and the audio signal respectively, and then comprehensively determine the probability of the target person lying. First, in step S100, the processing device 10 performs low-level analysis on the image data to extract multi-dimensional features. Next, in step S200, the processing device 10 performs high-level analysis on the multi-dimensional feature to identify at least one micro-expression action. Next, in step S300, the processing device 10 generates a spectrogram data according to a sound signal, and analyzes the spectrogram data to obtain an emotion feature. Finally, in step S200, the processing device 10 generates the probability of the target person lying according to the multi-dimensional features, micro-expression actions and emotional features. Please note that in other embodiments, any one of the above steps may be refined to include sub-steps therein, or other steps may be added before, between and/or after any two steps, but not limited thereto.

Please refer to FIG. 1 and FIG. 3 together. FIG. 3 shows a functional block diagram of a lie detection system in an embodiment, wherein each functional block diagram can be configured in the lie detection system 1 in the form of hardware or software. The lie detection system 1 of this embodiment can implement a lie detection method. The lie recognition system 1 includes a processing device 10 , a connection device 15 and a storage device 20 . The processing device 10 includes a feature extraction module 30 , a micro-expression recognition module 40 , a speech emotion recognition module 50 and a depth Learning Model 60. The feature extraction module 30 includes a Convolution Neural Networks (CNN) unit 31 or an encoder, and the CNN unit 31 is taken as an example here. It can be downloaded and/or set from a host through an initial step, such as a remote server, or not required, depending on the needs of various applications, for example, when the processing device 10 is applied to an endpoint host or handheld The CNN unit 31 can be downloaded and/or set to the remote server in real time; when the processing device 10 is applied to a server and accepts data from an endpoint host or a bank When the host makes a transaction request, the CNN unit 31 can be used to assist in determining whether the target person is lying in real time and directly.

The first step of analyzing the image data is that the processing device 10 can input the image data D _I in the video of the face of the target person and the recorded video into the feature extraction module 30, and the feature extraction module 30 will The CNN unit 31 performs low-level analysis to extract multi-dimensional features. Preferably, the image data D _I here is a frame of pictures, and can be input to the CNN unit 31 frame by frame; the multi-dimensional features can be a plurality of feature points without limitation or generated by an encoder. The feature vector of , here is an example of 68 main feature points of the face, for example, the amount of data of 68×2-dimensional floating point numbers can be extracted for each frame of picture. Preferably, the encoder 31 performs low-level analysis on the image data D _I to extract meaningful data types from complex face pixels, which is beneficial for the processing device 10 to obtain a classification function that is easy to analyze the correlation. The multi-dimensional features are then output to the micro-expression recognition module 40 and the deep learning model 60 .

The micro-expression recognition module 40 can find out whether there are predetermined micro-expression actions, and these micro-expression actions may include frowning, pursing the mouth, blinking, raising the corner of the mouth, and floating the viewing angle, etc., but not limited thereto. The micro-expression recognition module 40 may include a heuristic filtering unit 41 and a plurality of Graph Convolution Networks (GCN) units 42 or any one of a plurality of CNN units. The number of GCN units 42 is taken as an example, and the number thereof is not limited to the example in the figure. In detail, the heuristic filtering unit 41 can analyze the multi-dimensional features according to a plurality of preset micro-expression recognition rules when receiving the multi-dimensional features, so as to filter out the micro-expression features or the micro-expression features that are more in line with these micro-expression recognition rules among the multi-dimensional features. Micro-expression pictures, such as the judgment of eye opening and closing, can be judged by the Euclidean distance of the feature points of the upper and lower eyes. This method can save the time of collecting model training data sets, and can also be used to quickly create data sets in this way. labels to train the model. Alternatively, the filtered micro-expression features or micro-expression pictures can be input to the GCN unit 42 or the CNN unit, and the GCN unit 42 or the CNN unit can further identify the micro-expressions love action. Preferably, the micro-expression actions can be an n×1-dimensional integer data amount, where n=the number of micro-expression actions. Then, the micro-expression actions can be output to the deep learning model 60 .

The analysis of the voice signal _DV is first performed by the voice emotion recognition module 50 . The voice emotion recognition module 50 includes a spectrum conversion unit 51 and a CNN unit 52 . The spectrum conversion unit 51 can convert the received sound signal _DV into spectrogram data, where the spectrogram data is a Mel Scale Spectrogram in this example. The spectrogram data is then analyzed by the CNN unit 52 using transfer learning to obtain emotional features. Specifically, the CNN unit 52 may include multiple layers of convolutional layers (Convolutional layers), and the emotion feature is the output of the penultimate convolutional layer of these convolutional layers, and the emotion feature is 32×32 two-dimensional floating point numbers in this example. amount of data. The emotion features can then be output from the speech emotion recognition module 50 to the deep learning model 60 .

It can be known from the foregoing that the deep learning model 60 receives the above-mentioned multi-dimensional features, micro-expression movements and emotional features, and therefore can comprehensively judge the probability of the target person lying by combining the analysis results of the image data and the sound signal respectively. The multi-dimensional features, micro-expression actions and emotional features can be first integrated into an integrated feature by a feature latent space 61, where the multi-dimensional features, micro-expression actions and emotional features are exemplarily compressed into one-dimensional and The connected data can be deduced from the above example as 136+n+1024=1160+n-dimensional/each-frame-dimensional floating-point data volume. If the original image data has t frames of pictures, there will be t×( The data volume of 1160+n) dimensions is input into the deep learning model 60 . The deep learning model includes the aforementioned feature hidden space 61, a multi-modal judgment module (Multi-Modal Recognition Module) 62 and a recognition regression classification set (Regression Model) 63, and the multi-modal judgment module 62 is connected to the feature hidden space 61 and Identify between regression classification sets 63 . The multi-modal judgment module 62 may include any one of a Long Short-Term Memory (LSTM) convolutional neural network and a Recurrent Neural Network (RNN). Please refer to its functional block diagram. Figure 4. The feature hidden space 61 includes a plurality of hidden features for comparison. The multi-modal determination module 62 can determine the probability that the integrated feature conforms to the hidden feature in the hidden space 61 , and this probability is exemplarily represented by multi-dimensional data, such as 512-dimensional data. Next, after the identification and regression classification set 63 receives the output of the multi-modal judgment module 62, the probability that these integrated features conform to the hidden features is subjected to regression classification to finally obtain the probability of the target person lying, which is used as the judgment result. The probability of the target person lying is exemplarily presented here as one-dimensional data between 0 and 1. Please note that when the deep learning model is pre-trained with a certain amount of training data input, its accuracy in judging whether to lie or not will be improved.

Afterwards, the processing device 10 can be configured to issue an alarm when it is judged that the target person is lying according to the aforementioned judgment result. Preferably, the processing device 10 can establish different alarm generating paths for abnormal remittance requests according to different applications. For example, when the processing device 10 is applied to an endpoint host or a handheld device, an alarm message is sent back to the end that issued the transaction request. A peer host and/or a corresponding bank host that performs remittance; when the processing device 10 is applied to the host, an alert message is sent back to the endpoint host or bank host that issued the transaction request.

The above description is based on multiple different embodiments of the present invention, wherein each feature can be implemented in a single or different combination. Therefore, the disclosure of the embodiments of the present creation is a specific example to illustrate the principle of the present creation, and the present creation should not be limited to the disclosed embodiments. Further, the foregoing description and the accompanying drawings are only used for the demonstration of this creation, and are not limited thereto. Changes or combinations of other elements are possible without departing from the spirit and scope of this creation.

10: Processing device

30: Feature extraction module

31: CNN unit

40: Micro-expression recognition module

41: Heuristic filtering unit

42: GCN unit

50: Speech emotion recognition module

51: Spectrum conversion unit

52: CNN unit

60: Deep Learning Models

Claims (10)

A lie detection system for identifying whether a target person is lying, including: a storage device, providing a data storage space; a connecting device; A processing device electrically connected to the storage device through the connection device to use the data storage space, including: a feature extraction module, which performs low-level analysis on an image data to extract multi-dimensional features; A micro-expression recognition module for performing high-level analysis on the multi-dimensional feature to recognize at least one micro-expression action; a speech emotion recognition module that generates a spectrogram data according to a sound signal, and analyzes the spectrogram data to obtain an emotion feature; and A deep learning model generates a probability of the target person lying according to the multi-dimensional feature, the at least one micro-expression action and the emotional feature. The lie recognition system of claim 1, wherein the feature extraction module includes a CNN unit or an encoder, and the CNN unit or the encoder receives and analyzes the image data in a low-level manner to extract the multi-dimensional feature. The lie recognition system of claim 1, wherein the multi-dimensional feature is any one of a plurality of main feature points and feature vectors of the face image. The lie recognition system of claim 1, wherein the micro-expression recognition module comprises a heuristic filtering unit and either a GCN unit or a CNN unit, the heuristic filtering unit receives and analyzes the multi-dimensional feature , to filter out the micro-expression features or micro-expression pictures that are more in line with multiple micro-expression recognition rules in this multi-dimensional feature, and the micro-expression features or micro-expression pictures are input to this GCN unit or this CNN unit by this GCN unit or this The CNN unit recognizes the micro-expression action. The lie recognition system according to claim 1, wherein the micro-expression actions include frowning, pursing the mouth, blinking, raising the corner of the mouth, and moving the angle of view upward. The lie recognition system of claim 1, wherein the speech emotion recognition module includes a spectrum conversion unit and a CNN unit, the spectrum conversion unit converts the received sound signal into the spectrogram data, the spectrogram data And then the CNN unit uses transfer learning analysis to obtain the emotion feature. The lie recognition system of claim 6, wherein the CNN unit comprises multiple convolutional layers, and the emotion feature is the output of the penultimate convolutional layer of the convolutional layers. The lie recognition system of claim 1, wherein: The deep learning model receives the multi-dimensional feature, the at least one micro-expression action, and the emotional feature, and includes a feature hidden space, a multi-modal judgment module, and an identification regression classification set, and the multi-modal judgment module is connected to the feature Between the hidden space and the identification regression classification set, the feature hidden space includes a plurality of hidden features and integrates the multi-dimensional feature, the at least one micro-expression action and the emotional feature into an integrated feature; The multi-modal determination module determines the probability that the integrated feature conforms to the hidden features; and The identification regression classification set regresses and classifies the probability that the integrated feature matches the hidden features to obtain the probability that the target person lies. The lie detection system of claim 8, wherein the probability that the integrated feature matches the hidden features includes multi-dimensional data. The lie recognition system of claim 8, wherein the probability of the target person lying is one-dimensional data between 0 and 1.

Images