CN114461069A - Quantum CNN-LSTM-based emotion recognition method - Google Patents
Quantum CNN-LSTM-based emotion recognition method Download PDFInfo
- Publication number
- CN114461069A CN114461069A CN202210116529.4A CN202210116529A CN114461069A CN 114461069 A CN114461069 A CN 114461069A CN 202210116529 A CN202210116529 A CN 202210116529A CN 114461069 A CN114461069 A CN 114461069A
- Authority
- CN
- China
- Prior art keywords
- quantum
- state
- lstm
- cnn
- electroencephalogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 18
- 210000004556 brain Anatomy 0.000 claims abstract description 42
- 239000011159 matrix material Substances 0.000 claims abstract description 21
- 238000005259 measurement Methods 0.000 claims abstract description 18
- 230000008451 emotion Effects 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000011176 pooling Methods 0.000 claims abstract description 13
- 230000004913 activation Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 3
- 210000004027 cell Anatomy 0.000 description 22
- 238000010586 diagram Methods 0.000 description 15
- 238000013527 convolutional neural network Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 239000000047 product Substances 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000013507 mapping Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000002096 quantum dot Substances 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- PIZHFBODNLEQBL-UHFFFAOYSA-N 2,2-diethoxy-1-phenylethanone Chemical compound CCOC(OCC)C(=O)C1=CC=CC=C1 PIZHFBODNLEQBL-UHFFFAOYSA-N 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 210000003710 cerebral cortex Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 108020001507 fusion proteins Proteins 0.000 description 1
- 102000037865 fusion proteins Human genes 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000002610 neuroimaging Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Human Computer Interaction (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a quantum CNN-LSTM-based emotion recognition method, and belongs to the technical field of quantum computing. The method encodes brain wave data into brain wave encoding quantum states, inputs the brain wave encoding quantum states into quantum CNN-LSTM, and outputs recognized emotion classification information. Because the method inputs brain wave coding quantum states into the quantum CNN, the brain wave processing matrix extracted after convolution and pooling is obtained. And then extracting time sequence characteristics in the electroencephalogram processing matrix through the quantum LSTM, and outputting recognized emotion classification information after measurement, so that the electroencephalogram characteristics can be accurately extracted based on the quantum CNN-LSTM compared with the classical CNN-LSTM, the problems that operation of a certain module in the CNN-LSTM has high parallelism and consumes a large amount of computing resources are solved, the parameter quantity is reduced, and the quantum chip and the electronic chip can well work in a cooperative mode.
Description
Technical Field
The invention relates to the technical field of quantum computing, in particular to a quantum CNN-LSTM-based emotion recognition method.
Background
How to effectively acquire emotion signals and objectively and accurately interpret personal emotion states enables an intelligent information system to have better perception and decision-making capacity, and is a hot problem in intelligent information processing. The nature of human emotional changes is the high-level neural activity on the cerebral cortex. In recent years, the development of modern neuroimaging technology has established a bridge between the subjective world and the objective world. Among them, electroencephalogram (EEG) has become a mainstream tool for studying brain functions and designing brain-computer interfaces due to its high time resolution and its portability. The research of the emotion recognition method based on the electroencephalogram has good theoretical and application values in the fields of disease treatment, brain-computer interface, information evaluation and the like.
The classic CNN-LSTM algorithm is a research hotspot in the field of artificial intelligence and is applied to various application scenes such as biomedicine, materials, neuroscience and the like, but the operation of the models needs to consume a large amount of computing resources. In the past, the computational resources for algorithm operation were mainly provided by chips fabricated from electronic integrated circuits, and as the electron tunneling effect has constrained the fabrication process to the nanometer limit, the computational effort has been difficult to increase continuously. The quantum computing chip is a supplement to the electronic chip computing mode, however, the operation of the classic CNN-LSTM algorithm on the quantum chip cannot be handled in the way it is on the electronic chip.
Although the emotion recognition problem obtains good performance based on the CNN-LSTM model, operation of a certain module in the CNN-LSTM algorithm has high parallelism and consumes a large amount of computing resources, so that a quantum chip and an electronic chip cannot work together well.
Disclosure of Invention
The present invention has been made to solve the above problems, and an object of the present invention is to provide a quantum CNN-LSTM-based emotion recognition method.
The invention provides a quantum CNN-LSTM-based emotion recognition method, which is characterized by comprising the following steps: preprocessing brain wave data, and coding to obtain a corresponding brain wave coding quantum state; and inputting the brain wave coding quantum state into the quantum CNN-LSTM to obtain the recognized emotion classification information.
In the method provided by the invention, the method also has the following characteristics: the quantum CNN is used for extracting electroencephalogram features in brain wave coding quantum states, and therefore an electroencephalogram processing matrix with the features extracted is obtained; (ii) a The quantum LSTM is used for extracting the time sequence characteristics of the electroencephalogram processing matrix, and the emotion classification information is obtained after measurement.
In the method provided by the invention, the method also has the following characteristics: the quantum CNN comprises a quantum convolution module and a quantum pooling module. The quantum convolution module comprises a parameterized Pauli revolving gate; the quantum pooling module includes a parameterized Paglie rotary gate and a controlled gate.
In the method provided by the invention, the method also has the following characteristics: the quantum LSTM comprises a first quantum line, a second quantum line, a third quantum line and a fourth quantum line, wherein the first quantum line is used for forgetting the previous electroencephalogram characteristics of the cell state; the second quantum circuit is used for inputting the current electroencephalogram characteristics; the third quantum circuit is used for updating all current electroencephalogram characteristics of the cell state; the fourth quantum wire is used for determining the electroencephalogram characteristics needing to be preserved in the previous hidden state.
In the method provided by the invention, the method also has the following characteristics: wherein the expression of the first quantum wire is as follows:
ft*Ct-1＝σ'(VQC1(xt+ht-1))*Ct-1
ftto forget the door, Ct-1For the previous cell state, σ' is the quantum nonlinear activation function, VQC1Is a first variational quantum wire, xtIs the electroencephalogram characteristic at the time t, ht-1And generating a hidden state for the electroencephalogram characteristics at the time t-1.
The method provided by the invention also has the following characteristics: wherein the expression of the second quantum wire is as follows:
it*Ct'＝σ'(VQC2(xt+ht-1))*Ct'
itto the input gate, Ct' as cell candidate gate, ' sigma ' as quantum nonlinear activation function, VQC2A second variational quantum wire.
In the method provided by the invention, the method also has the following characteristics: wherein the expression of the third quantum wire is as follows:
Ct＝ft*Ct-1+it*Ct'
Ctis the current cell state.
In the method provided by the invention, the method also has the following characteristics: wherein the expression of the fourth quantum wire is as follows:
ot＝σ'(VQC3(xt+ht-1))
ht＝ot*tanh'(Ct)
otto the output gate, htThe electroencephalogram characteristics needed to be reserved in the previous hidden state.
In the method provided by the invention, the method also has the following characteristics: the initial state of the nonlinear output line corresponding to the sigma 'and the tanh' is obtained by measuring the probability amplitude of the output of the variational quantum line.
In the method provided by the invention, the method also has the following characteristics: wherein, the preset times of measuring the probability density amplitude is set as k, and |0 is obtained by measurement>Number of times of (1) is ko，|1>Number of times of (k)1，
If k iso>k1Then the initial state of the nonlinear output line is prepared at |0>In the state of and through controlling the revolving door
Then, the quantum state is changed from the initial state |0>Evolves to an output state
If k iso<k1Then the initial state of the nonlinear output line is prepared at |1>In a stationary state and controlled by a revolving door
Then, the quantum state is changed from the initial state |1>Evolves to an output state
If k iso＝k1By controlling the phase of the rotary y-gate to shift by pi/2 additionally, the initial state of the nonlinear output line can be prepared as |0>State or |1>State, the output state of the non-linear output line is |0>Or |1>And the output state is |0>Or |1>Are equally probable.
Action and effects of the invention
According to the emotion recognition method based on the quantum CNN-LSTM, brain wave data are encoded into brain wave encoding quantum states, the brain wave encoding quantum states are input into the quantum CNN-LSTM, and recognized emotion classification information is output. Because the method inputs brain wave coding quantum states into the quantum CNN, the brain wave processing matrix extracted after convolution and pooling is obtained. And then extracting time sequence characteristics in the electroencephalogram processing matrix through the quantum LSTM, and outputting recognized emotion classification information after measurement, so that the electroencephalogram characteristics can be accurately extracted based on the quantum CNN-LSTM compared with the classical CNN-LSTM, the problems that operation of a certain module in the CNN-LSTM has high parallelism and consumes a large amount of computing resources are solved, the parameter quantity is reduced, and the quantum chip and the electronic chip can well work in a cooperative mode.
Drawings
FIG. 1 is a general flow diagram of a quantum CNN-LSTM based emotion recognition method in an embodiment of the present invention;
FIG. 2 is a flow chart of acquiring brain wave encoding quantum states in an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a quantum convolution module of quantum CNN in an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a quantum pooling module of quantum CNN in an embodiment of the present invention;
FIG. 5 is a schematic diagram of the overall structure of a quantum LSTM in an embodiment of the invention;
FIG. 6 is a schematic diagram of the structure of the mapping weights and data quantum states in quantum LSTM in an embodiment of the present invention;
FIG. 7 is a schematic diagram of the structure of mapping nonlinear activation functions in quantum LSTM in an embodiment of the present invention;
FIG. 8 is a schematic diagram of the structure of the multiplication of the mapped data quantum states in quantum LSTM in an embodiment of the present invention;
FIG. 9 is a structural diagram illustrating the addition of the mapped data quantum states in quantum LSTM in an embodiment of the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the emotion recognition method based on quantum CNN-LSTM of the invention is specifically described below with reference to the embodiments and the drawings.
< example >
Fig. 1 is a general flowchart of a quantum CNN-LSTM based emotion recognition method in an embodiment of the present invention.
As shown in fig. 1, the emotion recognition method based on quantum CNN-LSTM provided by the embodiment of the present invention includes the following steps:
and step S1, preprocessing the brain wave data, and coding to obtain the corresponding brain wave coding quantum state.
In the present embodiment, the brain wave data is from a DEAP data set that records brain wave signals and peripheral physiological signals of 32 participants when viewing 40 pieces of music video, and that contains 32-channel electroencephalographic signals and 8-channel peripheral physiological signals. In this embodiment, electroencephalogram signals are used for emotion recognition and elimination of peripheral physiological signals, and the electroencephalogram signals are sampled at 512Hz and down-sampled to 128 Hz.
Fig. 2 is a flow chart of acquiring brain wave encoding quantum states in an embodiment of the present invention.
As shown in fig. 2, the process of encoding brain wave signals into corresponding brain wave encoding quantum states includes the following steps:
step S1-1, segmenting brain wave data with the length of n, wherein each segment is data with the length of m;
step S1-2, converting the data with length m into m × m (where m is 2)qubitAnd qubit is the number of bits of a qtoroch quantum wire) of the feature matrix U1；
Step S1-3, converting the matrix U1Is multiplied by U1(i.e. the
) Obtaining a Gram semi-positive definite matrix of m multiplied by m;
and step S1-4, substituting the Gram semi-positive definite matrix into an encoding function (encoding can convert the Gram semi-positive definite matrix of m multiplied by m into a quantum state density matrix of m multiplied by m) to obtain the quantum state data corresponding to the brain wave sequence, namely the brain wave encoding quantum state.
And step S2, inputting the brain wave coding quantum state obtained in the step S1 into the constructed quantum CNN for electroencephalogram feature extraction, and obtaining an electroencephalogram processing matrix with extracted features.
In this embodiment, the quantum CNN includes a quantum convolution module and a quantum pooling module, and the quantum CNN is configured to perform feature extraction on an input brain wave encoding quantum state (mxm) to obtain a brain wave processing matrix.
Fig. 3 is a schematic structural diagram of a quantum convolution module of quantum CNN in an embodiment of the present invention; fig. 4 is a schematic structural diagram of a quantum pooling module of quantum CNN in an embodiment of the present invention.
As shown in fig. 3-4, the quantum convolution module includes a parameterized dolby rotation gate and the quantum pooling module includes a parameterized dolby rotation gate and a controlled gate. The parameterized Paglie revolving door is used as a learnable parameter of the neural network, and the controlled door is used for quantum state error correction, so that electroencephalogram characteristics are extracted. The parameterized quantum circuit (quantum CNN) part can adopt a quantum convolution kernel and a quantum pooling kernel, namely the quantum CNN can be a full quantum or a quantum-classical mixture, and is also provided with a quantum convolution module and a quantum pooling module.
As shown in FIG. 3, Rx, Ry, Rz respectively represent an x-revolution gate, a y-revolution gate, and a z-revolution gate corresponding to the Pagli revolution gate, and θ represents a rotation angle,
The tensor product operator. In the quantum convolution module, the input is brain wave coding quantum state, and the output is brain wave quantum state after the quantum convolution operation.
As shown in fig. 4, the input of the quantized electroencephalogram quanta after the quantum convolution operation is input to the quantization module, and the quantized electroencephalogram quanta after the quantization operation (electroencephalogram processing matrix) is output after the single-bit and multi-bit pauli rotations.
And step S3, inputting the electroencephalogram processing matrix output in the step S2 into quantum LSTM to extract electroencephalogram time sequence characteristics, and obtaining recognized emotion classification information after measurement.
Fig. 5 is a schematic diagram of the overall structure of quantum LSTM in an embodiment of the invention.
As shown in fig. 5, the quantum LSTM in the embodiment of the present invention includes a first quantum wire, a second quantum wire, a third quantum wire, and a fourth quantum wire.
Wherein, the first quantum circuit is used for forgetting the previous brain wave characteristics of the cell state, and the expression of the first quantum circuit is as follows:
ft*Ct-1＝σ'(VQC1(xt+ht-1))*Ct-1
ftto forget the door, Ct-1For the previous cell state, σ' is the quantum nonlinear activation function, VQC1Is a first variational quantum wire, xtIs the brain wave characteristic at time t, ht-1A hidden state generated for brain wave features at time t-1. The input state can be written as a direct product of states of all qubits corresponding to the input state, specifically as follows:
FIG. 6 is a structural diagram of mapping weights and data quantum states in quantum LSTM according to an embodiment of the present invention.
As shown in FIG. 6, with VQC1One input state | x>For example, mapping weights to quantum states | x>Multiplication.
In this embodiment, for one input state | x>vQC of the place1(input line, lower common VQC)2To VQC4) Adding a corresponding weight output line whose initial quantum state is prepared as |0>And the weighted output line is controlled by the input line. Multiplication of the weights by a controlled rotation Y (CR)y(theta)) the gate is operated, CRyThe matrix of (θ) can be expressed as
Referring to fig. 5, the input control line is | x in this embodimentt>And | ht-1>In the fused state, the initial state | xt>And | ht-1>Respectively corresponding to the EEG characteristics x at the time ttAnd hidden state | h generated at time t-1t-1>. Wherein (x)t、ht-1E {0, 1}), when xt、ht-1When 0, the rotating y-gate of the weight output line does not perform, i.e. |0>→|0>. When x ist、ht-1When 1, the rotating y-gate pair |0 of the weight output line>Rotating at-2W, and multiplying by CRyAfter (-2W), the quantum state evolution process includes: i0>→cosW|0>+sinW|1>。
Referring to fig. 6, a rotary gate R is additionally provided on the weight output liney(-2b) so that the output state of the weighted output line can be represented as cosWx |0>+sinWx|1>Or cosWh |0>+sinWh|1>I.e. by
So that the slave CRyQuantum state of (-2W) output
Passing through a revolving door RyRear output quantum state | psi (-2b)>I.e. quantum state cosWx |0>+sinWx|1>Passing through a revolving door RyAfter (-2b), it becomes cos (Wx + b) |0>+sin(Wx+b)|1>Thereby enabling the addition of bias in the quantum wires.
Fig. 7 is a schematic structural diagram of mapping a nonlinear activation function in quantum LSTM in an embodiment of the invention.
As shown in FIG. 7, a nonlinear output quantum line is added after the weight output line, and nonlinear operation is performed on the probability amplitude that the quantum state is in the |0> state or the |1> state, so as to construct a nonlinear activation function. The added nonlinear output quantum circuit is used for carrying out nonlinear probability transformation on a quantum state | psi > containing weight information, and the weight output quantum circuit is used as a control circuit to carry out control rotation y gate operation on the nonlinear output quantum circuit.
In this embodiment, the quantum state | ψ of the weight output quantum line>Multiple measurements, and | 0's obtained from the multiple measurements>State or |1>The number of states selects the initial state and the rotational phase of the nonlinear output quantum line. Before measurement, the rotation phase of the rotating y gate is controlled to be initialized to 0, the measurement frequency is set to be k, and the measurement result is |0>Number of times of (k)oThe measurement result is |1>Number of times of (k)1When the measurement result is |0>At the time of state, the phase of the rotating y-gate is shifted by 2 π/k, and when the measurement is |1>In the phase, the rotating y-gate is shifted by-2 π/k. K obtained from measurementoAnd k is1The evolution of the quantum state of the size and nonlinear output line comprises the following three conditions:
if k iso>k1(k0And k1Associated with weight W), indicating a weight state |0>Probability to amplitude ratio of state |1>High state, the initial state of the nonlinear output line is prepared to be |0>And passing through the control revolving door
Then, the quantum state is changed from the initial state |0>Evolves to an output state
If k iso<k1Then the initial state of the nonlinear output line is prepared at |1>In the state of and through controlling the revolving door
Then, the quantum state is changed from the initial state |1>Evolves to an output state
If k iso＝k1When the phase of the rotary y gate is controlled to shift by pi/2 additionally, the initial state of the nonlinear output circuit is prepared to be |0>State or |1>State, the output state of the non-linear output line is |0>Or |1>And the output state is |0>Or |1>And the probabilities of the nonlinear output lines are equal, and the output state of the nonlinear output line is subjected to quantum gate operation.
In this embodiment, the forgetting door ftWith the previous cell state Ct-1Multiply, leaving behind the previous brain electrical features in some cell states.
FIG. 8 is a schematic diagram of the structure of the multiplication of the mapped data quantum states in quantum LSTM in an embodiment of the present invention.
As shown in FIG. 8, the addition of an auxiliary quantum line controls the output state | f of the nonlinear output linet>(i.e. | ψ)>) And previous sequence cell output state | Ct-1>(i.e. | Φ)>) And constructing | f by using swap-testt>And | Ct-1>Inner product of (2)<ft|Ct-1>The method comprises the following steps:
the initial state of each auxiliary quantum circuit is prepared at |0>And the state is characterized in that an Hadamard gate is respectively applied to each auxiliary quantum line, the result of controlling the switching gate (C-swap gate) is jointly realized under the control action of the corresponding auxiliary line, and the Hadamard gate is respectively applied to each auxiliary line after the result passes through the C-swap gate to obtain the output state of each auxiliary line and the state related to the direct product of the corresponding two control line states
In this embodiment, the two control lines are respectively nonlinear output lines ftAnd pre-sequence cell Ct-1The respective initial states of the two control lines are | psi>And output state | Φ>Referring to fig. 8, it can be seen that the output state of each auxiliary line is:
in this embodiment, the auxiliary quantum line and the pre-sequence cell Ct-1And a nonlinear output sub-line ftHas the same number of qubit lines and non-linear output quantum line ftIs equal to the input quantum state | xt>And | ht-1>Sum of the number of qubits.
In this embodiment, the second quantum circuit is used to input the current electroencephalogram characteristic, and its expression is as follows:
it*Ct'＝σ'(VQC2(xt+ht-1))*Ct'
itto the input gate, Ct' as cell candidate gate, ' sigma ' as quantum nonlinear activation function, VQC2A second variational quantum wire.
In the embodiment, the method for constructing the | i by using the swap-test method is used by using the same construction mode as the forgetting gatet>And | Ct'>Inner product<it|Ct'>State of relevance
Will state | ct1>And state | ct2>Information fusion update cell state | Ct>Then, add cell output circuit and output from state | ct1>And state | ct2>And the circuit controls, respectively applies C-not gate operation on the cell output circuit, and the cell state is used as the cell input of the next neuron.
In this embodiment, the third quantum line is used to update all current electroencephalogram characteristics of the cell state, and its expression is as follows:
Ct＝ft*Ct-1+it*Ct'
Ctas the current cell state, CtFrom state | ct1>And state | ct2>And (4) fusing to obtain the fusion protein.
In this embodiment, the fourth quantum line is used to determine an electroencephalogram feature that needs to be preserved in a previous hidden state, and its expression is as follows:
ot＝σ'(VQC3(xt+ht-1))
ht＝ot*tanh'(Ct)
otto the output gate, htAnd all the electroencephalogram characteristics which need to be reserved in the hidden state at the moment t.
In this embodiment, the state | C is output to the cell by using the nonlinear operation of the state probabilities of the f, i, C, o gatest>Performing a non-linear probability transformation tanh' (C)t) Obtain state | ct3>And outputs the nonlinear output state | o of the o layert>And state | ct3>The same method as that of a forgetting gate is adopted to obtain the state | ot>And state | ct3>Inner product related state | ht>，|ht>Can be used as the hidden layer input of the next neuron. Will | ht>Through C-not operation, the state can be used as the electroencephalogram characteristic of the whole cell output.
FIG. 9 is a structural diagram illustrating the addition of the mapped data quantum states in quantum LSTM in an embodiment of the present invention.
As shown in fig. 9, controlled-NOT is a controlled NOT gate, the quantum line where a is located is a control line, and the quantum line where B is located is a controlled line, for implementing the summation of two quantum states, see the controlled NOT gate appearing in fig. 5.
State | h to be outputt>After multiple measurements, recognized emotion classification information yout is obtained, the yout is input into a loss function for calculation and training, and parameters in quantum CNN and quantum LSTM are optimized by gradient descent. In this embodiment, the loss function used is an average cross entropy loss function.
Effects and effects of the embodiments
According to the emotion recognition method based on quantum CNN-LSTM according to the present embodiment, the method encodes brain wave data into brain wave encoding quantum states, and inputs the brain wave encoding quantum states into quantum CNN-LSTM, thereby outputting recognized emotion classification information. Because the method inputs brain wave coding quantum states into the quantum CNN, the brain wave processing matrix extracted after convolution and pooling is obtained. And then, the time sequence characteristics in the electroencephalogram processing matrix are extracted through the quantum LSTM, and the emotion classification information of recognition is output after measurement, so that the electroencephalogram characteristics can be accurately extracted based on the quantum CNN-LSTM compared with the classical CNN-LSTM, the problems that operation of a certain module in the CNN-LSTM has high parallelism and consumes a large amount of computing resources are solved, the parameter quantity is reduced, and the quantum chip and the electronic chip can well work in a cooperative mode.
Further, for a storage medium storing a computer program, the computer program may be configured to execute the quantum CNN-LSTM-based emotion recognition method provided by the present embodiment when executed.
Further, for an electronic product comprising a memory and a processor, the memory stores a computer program therein, and the processor is configured to run the stored computer program to execute the quantum CNN-LSTM-based emotion recognition method provided by the present embodiment.
The above embodiments are preferred examples of the present invention, and are not intended to limit the scope of the present invention.
Claims (10)
1. A emotion recognition method based on quantum CNN-LSTM is characterized by comprising the following steps:
preprocessing brain wave data, and coding to obtain a corresponding brain wave coding quantum state;
and inputting the brain wave coding quantum state into a quantum CNN-LSTM to obtain recognized emotion classification information.
2. The method of claim 1, wherein:
the quantum CNN is used for extracting electroencephalogram features in the electroencephalogram coding quantum states, so that an electroencephalogram processing matrix with the features extracted is obtained;
the quantum LSTM is used for extracting time sequence characteristics of the electroencephalogram processing matrix and obtaining recognized emotion classification information after measurement.
3. The method of claim 2, wherein:
wherein the quantum CNN comprises a quantum convolution module and a quantum pooling module.
The quantum convolution module comprises a parameterized Paglie rotation gate;
the quantum pooling module comprises a parameterized Paglie rotary gate and a controlled gate.
4. The method of claim 2, wherein:
wherein the quantum LSTM comprises a first quantum wire, a second quantum wire, a third quantum wire and a fourth quantum wire,
the first quantum circuit is used for forgetting the previous electroencephalogram characteristics of the cell state;
the second quantum circuit is used for inputting current electroencephalogram characteristics;
the third quantum circuit is used for updating all current electroencephalogram characteristics of the cell state;
the fourth quantum circuit is used for determining the electroencephalogram characteristics needing to be preserved in the previous hidden state.
5. The method of claim 4, wherein:
wherein an expression of the first quantum wire is as follows:
ft*Ct-1＝σ'(VQC1(xt+ht-1))*Ct-1
ftto forget the door, Ct-1For the previous cell state, σ' is the quantum nonlinear activation function, VQC1Is a first variational quantum wire, xtIs the electroencephalogram characteristic at the time t, ht-1And generating a hidden state for the electroencephalogram characteristics at the time t-1.
6. The method of claim 4, wherein:
wherein an expression of the second quantum wire is as follows:
it*Ct'＝σ'(VQC2(xt+ht-1))*Ct'
itto the input gate, Ct' as cell candidate gate, ' sigma ' as quantum nonlinear activation function, VQC2A second variational quantum wire.
7. The method of claim 4, wherein:
wherein an expression of the third quantum wire is as follows:
Ct＝ft*Ct-1+it*Ct'
Ctis the current cell state.
8. The method of claim 7, wherein:
wherein an expression of the fourth quantum wire is as follows:
ot＝σ'(VQC3(xt+ht-1))
ht＝ot*tanh'(Ct)
otto the output gate, htThe electroencephalogram characteristics needed to be reserved in the previous hidden state.
9. The method according to any one of claims 5-8, wherein:
and the initial state of the nonlinear output line corresponding to the sigma 'and the tanh' is obtained by measuring the probability amplitude of the output of the variational quantum line.
10. The method of claim 9, wherein:
wherein, the preset times for measuring the probability density amplitude is set as k, and |0 is obtained by measurement>Number of times of (k)o，|1>Number of times of (k)1，
If k iso>k1Then the initial state of the nonlinear output line is prepared at |0>In the state of and through controlling the revolving door
Then, the quantum state is changed from the initial state |0>Evolves to an output state
If k iso<k1Then the initial state of the nonlinear output line is prepared at |1>In the state of and through controlling the revolving door
Then, the quantum state is changed from the initial state |1>Evolves to an output state
If k iso＝k1By controlling the phase of the rotary y-gate to shift by pi/2 additionally, the initial state of the nonlinear output line can be prepared as |0>State or |1>State, the output state of the nonlinear output line is |0>Or |1>And the output state is |0>Or |1>Are equally probable.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116529.4A CN114461069A (en) | 2022-02-07 | 2022-02-07 | Quantum CNN-LSTM-based emotion recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116529.4A CN114461069A (en) | 2022-02-07 | 2022-02-07 | Quantum CNN-LSTM-based emotion recognition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114461069A true CN114461069A (en) | 2022-05-10 |
Family
ID=81410825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210116529.4A Pending CN114461069A (en) | 2022-02-07 | 2022-02-07 | Quantum CNN-LSTM-based emotion recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114461069A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116400430A (en) * | 2023-03-23 | 2023-07-07 | 本源量子计算科技（合肥）股份有限公司 | Meteorological data prediction method and device, storage medium and electronic device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106955112A (en) * | 2017-03-17 | 2017-07-18 | 泉州装备制造研究所 | Brain wave Emotion recognition method based on Quantum wavelet neural networks model |
| CN107961007A (en) * | 2018-01-05 | 2018-04-27 | 重庆邮电大学 | A kind of electroencephalogramrecognition recognition method of combination convolutional neural networks and long memory network in short-term |
| US20180150761A1 (en) * | 2016-11-29 | 2018-05-31 | The Trustees Of Dartmouth College | Emulation of quantum and quantum-inspired spectrum analysis and superposition with classical transconductor-capacitor circuits |
| CN109820525A (en) * | 2019-01-23 | 2019-05-31 | 五邑大学 | A kind of driving fatigue recognition methods based on CNN-LSTM deep learning model |
| KR20190130808A (en) * | 2018-05-15 | 2019-11-25 | 연세대학교 산학협력단 | Emotion Classification Device and Method using Convergence of Features of EEG and Face |
| CN110610168A (en) * | 2019-09-20 | 2019-12-24 | 合肥工业大学 | Electroencephalogram emotion recognition method based on attention mechanism |
| CN112244873A (en) * | 2020-09-29 | 2021-01-22 | 陕西科技大学 | Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network |
| CN112863695A (en) * | 2021-02-22 | 2021-05-28 | 西京学院 | Bidirectional long-short term memory prediction model based on quantum attention mechanism and extraction method |
| WO2021242691A2 (en) * | 2020-05-24 | 2021-12-02 | The Platypus Institute, Inc. | Measuring and strengthening physiological/neurophysiologial states predictive of superior performance |
| CN114967911A (en) * | 2022-04-18 | 2022-08-30 | 上海图灵智算量子科技有限公司 | Sleep segmentation method based on quantum CNN-LSTM |
| CN114947740A (en) * | 2022-04-18 | 2022-08-30 | 上海图灵智算量子科技有限公司 | Method for predicting epileptic seizure based on quantum CNN-GRU |
-
2022
- 2022-02-07 CN CN202210116529.4A patent/CN114461069A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180150761A1 (en) * | 2016-11-29 | 2018-05-31 | The Trustees Of Dartmouth College | Emulation of quantum and quantum-inspired spectrum analysis and superposition with classical transconductor-capacitor circuits |
| CN106955112A (en) * | 2017-03-17 | 2017-07-18 | 泉州装备制造研究所 | Brain wave Emotion recognition method based on Quantum wavelet neural networks model |
| CN107961007A (en) * | 2018-01-05 | 2018-04-27 | 重庆邮电大学 | A kind of electroencephalogramrecognition recognition method of combination convolutional neural networks and long memory network in short-term |
| KR20190130808A (en) * | 2018-05-15 | 2019-11-25 | 연세대학교 산학협력단 | Emotion Classification Device and Method using Convergence of Features of EEG and Face |
| CN109820525A (en) * | 2019-01-23 | 2019-05-31 | 五邑大学 | A kind of driving fatigue recognition methods based on CNN-LSTM deep learning model |
| CN110610168A (en) * | 2019-09-20 | 2019-12-24 | 合肥工业大学 | Electroencephalogram emotion recognition method based on attention mechanism |
| WO2021242691A2 (en) * | 2020-05-24 | 2021-12-02 | The Platypus Institute, Inc. | Measuring and strengthening physiological/neurophysiologial states predictive of superior performance |
| CN112244873A (en) * | 2020-09-29 | 2021-01-22 | 陕西科技大学 | Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network |
| CN112863695A (en) * | 2021-02-22 | 2021-05-28 | 西京学院 | Bidirectional long-short term memory prediction model based on quantum attention mechanism and extraction method |
| CN114967911A (en) * | 2022-04-18 | 2022-08-30 | 上海图灵智算量子科技有限公司 | Sleep segmentation method based on quantum CNN-LSTM |
| CN114947740A (en) * | 2022-04-18 | 2022-08-30 | 上海图灵智算量子科技有限公司 | Method for predicting epileptic seizure based on quantum CNN-GRU |
Non-Patent Citations (2)
| Title |
|---|
| 刘帅;王磊;丁旭涛;: "基于Bi-LSTM的脑电情绪识别", 山东大学学报(工学版), no. 04, 30 April 2020 (2020-04-30) * |
| 罗天健;周昌乐;: "重叠特征策略与参数优化的运动想象脑电模式识别", 模式识别与人工智能, no. 08, 15 August 2020 (2020-08-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116400430A (en) * | 2023-03-23 | 2023-07-07 | 本源量子计算科技（合肥）股份有限公司 | Meteorological data prediction method and device, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Cheng et al. | Cv-CapsNet: Complex-valued capsule network | |
| Wang et al. | Deepvid: Deep visual interpretation and diagnosis for image classifiers via knowledge distillation | |
| Oyallon et al. | Scattering networks for hybrid representation learning | |
| Yang et al. | Skeletonnet: A hybrid network with a skeleton-embedding process for multi-view image representation learning | |
| WO2019005186A1 (en) | Topology aware graph neural nets | |
| Hassan et al. | Hyper-dimensional computing challenges and opportunities for AI applications | |
| Li et al. | Deep hybrid 2-D–3-D CNN based on dual second-order attention with camera spectral sensitivity prior for spectral super-resolution | |
| Feng et al. | An efficient EEGNet processor design for portable EEG-Based BCIs | |
| CN117216546A (en) | Model training method, device, electronic equipment, storage medium and program product | |
| Jinliang et al. | EEG emotion recognition based on granger causality and capsnet neural network | |
| Shi et al. | Learning robust auto-encoders with regularizer for linearity and sparsity | |
| Xu et al. | Feature data processing: Making medical data fit deep neural networks | |
| Garau et al. | CapsulePose: A variational CapsNet for real-time end-to-end 3D human pose estimation | |
| CN114461069A (en) | Quantum CNN-LSTM-based emotion recognition method | |
| Abreu et al. | A framework for designing power-efficient inference accelerators in tree-based learning applications | |
| Liu et al. | Quantum capsule networks | |
| Padha et al. | QCLR: Quantum-LSTM contrastive learning framework for continuous mental health monitoring | |
| Xia et al. | Pluralistic image completion with gaussian mixture models | |
| CN114967911A (en) | Sleep segmentation method based on quantum CNN-LSTM | |
| Chen et al. | A unified framework for generative data augmentation: A comprehensive survey | |
| Feng et al. | Study on neural network integration method based on morphological associative memory framework | |
| Sang et al. | Image recognition based on multiscale pooling deep convolution neural networks | |
| CN114947740A (en) | Method for predicting epileptic seizure based on quantum CNN-GRU | |
| Du et al. | Efficient spatial pyramid of dilated convolution and bottleneck network for zero-shot super resolution | |
| Singh et al. | A neural architecture search for automated multimodal learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |