CN115840890B - Emotion recognition method and device based on non-contact physiological signals - Google Patents
Emotion recognition method and device based on non-contact physiological signals Download PDFInfo
- Publication number
- CN115840890B CN115840890B CN202310158469.7A CN202310158469A CN115840890B CN 115840890 B CN115840890 B CN 115840890B CN 202310158469 A CN202310158469 A CN 202310158469A CN 115840890 B CN115840890 B CN 115840890B
- Authority
- CN
- China
- Prior art keywords
- contact
- emotion
- stress
- pulse wave
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 29
- 230000008451 emotion Effects 0.000 claims abstract description 177
- 230000008447 perception Effects 0.000 claims abstract description 91
- 238000001514 detection method Methods 0.000 claims abstract description 47
- 230000006698 induction Effects 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims description 35
- 230000001939 inductive effect Effects 0.000 claims description 34
- 210000005259 peripheral blood Anatomy 0.000 claims description 18
- 239000011886 peripheral blood Substances 0.000 claims description 18
- 230000024883 vasodilation Effects 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 16
- 230000008602 contraction Effects 0.000 claims description 16
- 230000000004 hemodynamic effect Effects 0.000 claims description 16
- 230000002093 peripheral effect Effects 0.000 claims description 15
- 238000004458 analytical method Methods 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 14
- 238000003384 imaging method Methods 0.000 claims description 14
- 230000000007 visual effect Effects 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 10
- 230000001149 cognitive effect Effects 0.000 claims description 9
- 230000005053 stress perception Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000003340 mental effect Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 4
- 238000012030 stroop test Methods 0.000 claims description 3
- 230000002996 emotional effect Effects 0.000 claims 2
- 230000008901 benefit Effects 0.000 abstract description 10
- 230000014509 gene expression Effects 0.000 abstract description 9
- 238000002474 experimental method Methods 0.000 abstract description 6
- 230000019771 cognition Effects 0.000 abstract description 5
- 230000007246 mechanism Effects 0.000 abstract description 5
- 238000012795 verification Methods 0.000 abstract description 4
- 238000013461 design Methods 0.000 abstract description 3
- 238000004364 calculation method Methods 0.000 description 8
- 230000015654 memory Effects 0.000 description 6
- 238000011160 research Methods 0.000 description 5
- 230000002123 temporal effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 210000003403 autonomic nervous system Anatomy 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- JYGXADMDTFJGBT-VWUMJDOOSA-N hydrocortisone Chemical compound O=C1CC[C@]2(C)[C@H]3[C@@H](O)C[C@](C)([C@@](CC4)(O)C(=O)CO)[C@@H]4[C@@H]3CCC2=C1 JYGXADMDTFJGBT-VWUMJDOOSA-N 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000002792 vascular Effects 0.000 description 2
- 206010010144 Completed suicide Diseases 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 230000037424 autonomic function Effects 0.000 description 1
- 210000000467 autonomic pathway Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010195 expression analysis Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 229960000890 hydrocortisone Drugs 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008035 nerve activity Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009323 psychological health Effects 0.000 description 1
- 230000035485 pulse pressure Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003938 response to stress Effects 0.000 description 1
- 201000000980 schizophrenia Diseases 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Landscapes
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention discloses a non-contact physiological signal-based emotion recognition method and device, and relates to the technical field of intelligent recognition. Comprising the following steps: acquiring non-contact emotion perception data to be identified; inputting emotion perception data into a constructed non-contact physiological signal detection and stress emotion perception model; according to emotion perception data and a non-contact physiological signal detection and stress emotion perception model, emotion recognition based on the non-contact physiological signal is achieved. The invention designs a cognition pressure and stress tension emotion induction experiment oriented to non-contact emotion perception verification, establishes a non-contact physiological signal detection and stress emotion recognition model by analyzing a correlation mechanism between non-contact emotion characteristics and stress emotion, realizes emotion perception based on non-contact physiological signals, and has the advantage of non-contact compared with the traditional physiological signal emotion perception method; compared with the emotion perception method based on the expression voice, the method has the advantages that physiological signals are difficult to control autonomously, and true emotion is expected to be revealed.
Description
Technical Field
The invention relates to the technical field of intelligent recognition, in particular to a non-contact physiological signal-based emotion recognition method and device.
Background
Emotion is the basis of human experience and affects multiple daily tasks such as cognition, perception and the like in human life. In the research of artificial intelligence, the ability to recognize, analyze, understand and express emotion is one of the essential intelligence. The figure winning master Yann LeCun also emphasizes: "it is impossible to have intelligence without emotion". The concept of "emotion calculation" was first proposed by the rosalin Picard professor of the university of ma multimedia laboratory and defined emotion calculation as "emotion-related, emotion-derived, or emotion-capable calculation that can exert an influence on emotion". The emotion calculation has wide application prospects in the aspects of medical health, public safety and the like, and the application of the emotion calculation in medical psychological health such as schizophrenia and suicide screening is also reviewed in the Nature article in 2020 by the university of Stanford Li Fei. At present, compared with research in other artificial intelligence fields such as computer vision, natural language understanding and the like, research on emotion calculation is in a relatively immature stage, namely, acquisition of emotion information and emotion recognition are preconditions of emotion calculation research, and conditions are provided for further analysis and understanding.
The emotion perception channel is multidimensional, for example, non-contact emotion perception can be realized based on expressions and voices, the expression recognition can be realized by combining a computer vision algorithm after imaging through a camera, and the voices can also be realized by combining a voice signal processing method after collecting through a microphone. In addition, the fluctuation of emotion can be regulated by the autonomic nervous system of the human body to cause the change of physiological signals, so that the perception of emotion change can be realized based on the analysis of the physiological signals. Most of changes in physiological signals cannot be subjectively controlled and hidden, and compared with the dimensions of expression, voice and the like, the real emotion state can be reflected well.
However, the acquisition of the existing physiological emotion characteristics mainly faces the bottleneck of contact type signal acquisition and limitation of application scenes. The physiological signal acquisition is realized on the basis of a contact sensor, and a tested person perceives or introduces extra emotion to the signal acquisition device, so that a research result is influenced, and preparation work such as electrode patch and the like is usually required to be arranged in advance, so that the application scene is limited. The non-contact type non-sensing measurement has great application value in the emotion calculation field. Autonomic nerve activity during stress reflects an individual's ability to cope with sudden situations. In high-pressure working environments (such as aviation and command), the perception of the tension emotion and the pressure level of key post staff is important to the improvement of safety through a non-contact method. However, the non-contact signal faces the problem of low signal-to-noise ratio, which is a core difficulty faced by emotion sensing based on the non-contact signal, and noise challenges caused by movement, illumination and the like need to be overcome. At present, a reliable and effective non-contact emotion sensing method is not available, characterization and enhancement of weak telemetry photoelectric volume pulse signals are needed, and a robust extraction method of physiological emotion characteristics is provided, so that key technical support is provided for non-contact emotion sensing.
Disclosure of Invention
The invention aims at the problems that the acquisition of the existing physiological emotion characteristics mainly faces the bottleneck of contact type signal acquisition and application scene limitation and the non-contact type signal faces the problem of low signal to noise ratio.
In order to solve the technical problems, the invention provides the following technical scheme:
in one aspect, the present invention provides a method for emotion recognition based on a non-contact physiological signal, the method being implemented by an electronic device, the method comprising:
s1, acquiring non-contact emotion perception data to be identified.
S2, inputting emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model.
And S3, according to the emotion perception data and the non-contact physiological signal detection and stress emotion perception model, realizing emotion recognition based on the non-contact physiological signal.
Optionally, the constructing process of the non-contact physiological signal detection and stress perception model in S2 includes:
s21, obtaining stress data of the testee by completing a stress inducing task.
S22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal.
S23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features.
S24, according to stress emotion data and non-contact emotion characteristics, the non-contact physiological signal detection and the stress emotion perception model construction are completed.
Optionally, the stress inducing tasks in S21 include a first stage cognitive stress inducing task and a second stage stress inducing stress task.
Stress inducing tasks also include pressure sources; the stress sources include social assessment threat tasks, temporal stress tasks, and loud voice feedback tasks.
Stress data includes visual imaging data, physiological data, and staged subjective self-reports.
Optionally, the non-contact physiological signal detection is performed on the corresponding stress emotion data in S22 to obtain a non-contact pulse wave signal, which includes:
the method comprises the steps of obtaining an original signal of visual imaging data in stress emotion data through a built space-time characteristic representation learning model, projecting the original signal to a plane orthogonal to the original signal, and learning a plurality of color space transformation weights through a plurality of two-dimensional convolution weights of a first layer in the space-time characteristic representation learning model to obtain a non-contact pulse wave signal.
Optionally, feature extraction of the non-contact pulse wave signal in S23 includes:
extracting heart rate variability features of the non-contact pulse wave signals and extracting peripheral hemodynamic information features of the non-contact pulse wave signals.
Optionally, extracting heart rate variability features of the non-contact pulse wave signal includes:
s231, modeling the non-contact pulse wave signal with a quasi-periodic signal, wherein the frequency of the non-contact pulse wave signal comprises an average heart rate and a change part caused by heart rate variability.
S232, constructing the instantaneous frequency of the non-contact pulse wave signal, wherein the instantaneous frequency of the non-contact pulse wave signal comprises an average heart rate and an instantaneous frequency change part caused by heart rate variability.
S233, solving the instantaneous frequency caused by the heart rate variability by adopting an instantaneous frequency extraction method based on pulse frequency demodulation and a discrete energy analysis algorithm, and completing the heart rate variability feature extraction of the non-contact pulse wave signals.
Optionally, the instantaneous frequency extraction method based on pulse frequency demodulation includes:
using a centre frequency as
High cut-off frequency of->
Low cut-off frequency of->
Extracting a first harmonic of the non-contact pulse wave signal; wherein BW is the bandwidth of the first harmonic of the non-contact pulse wave signal, BW is driven by heart rate variability information of the fundamental sidebands.
Optionally, extracting the peripheral hemodynamic information feature of the non-contact pulse wave signal includes:
extracting envelope features of the peripheral blood volume pulse waveform of the non-contact pulse wave signal, and extracting vasodilation and contraction movement features of the non-contact pulse wave signal.
The method comprises the steps of extracting the envelope characteristics of the pulse waveform of the peripheral blood volume of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.7Hz and a low cutoff frequency of 3Hz, and completing the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal.
The method comprises the steps of extracting vasodilation and contraction movement characteristics of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.009Hz and a low cutoff frequency of 0.2Hz, and completing the extraction of the vasodilation and contraction movement characteristics of the non-contact pulse wave signal.
Optionally, in S24, according to the stress data and the non-contact emotion feature, the non-contact physiological signal detection and the stress perception model construction are completed, including:
s241, recording the staged subjective self-report in the stress data by using a Liktet table, and enabling the subjective self-report estimation scores to be three-level perception stress scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report.
S242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
In another aspect, the present invention provides a non-contact physiological signal-based emotion recognition device, which is applied to implement a non-contact physiological signal-based emotion recognition method, the device including:
and the acquisition module is used for acquiring the non-contact emotion perception data to be identified.
And the input module is used for inputting emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model.
And the output module is used for detecting and stressing the emotion perception model according to the emotion perception data and the non-contact physiological signals so as to realize emotion recognition based on the non-contact physiological signals.
Optionally, the input module is further configured to:
s21, obtaining stress data of the testee by completing a stress inducing task.
S22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal.
S23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features.
S24, according to stress emotion data and non-contact emotion characteristics, the non-contact physiological signal detection and the stress emotion perception model construction are completed.
Optionally, the stress-inducing tasks include a first stage cognitive stress-inducing task and a second stage stress-inducing stress-emotional task.
Stress inducing tasks also include pressure sources; the stress sources include social assessment threat tasks, temporal stress tasks, and loud voice feedback tasks.
Stress data includes visual imaging data, physiological data, and staged subjective self-reports.
Optionally, the input module is further configured to:
the method comprises the steps of obtaining an original signal of visual imaging data in stress emotion data through a built space-time characteristic representation learning model, projecting the original signal to a plane orthogonal to the original signal, and learning a plurality of color space transformation weights through a plurality of two-dimensional convolution weights of a first layer in the space-time characteristic representation learning model to obtain a non-contact pulse wave signal.
Optionally, the input module is further configured to:
extracting heart rate variability features of the non-contact pulse wave signals and extracting peripheral hemodynamic information features of the non-contact pulse wave signals.
Optionally, the input module is further configured to:
s231, modeling the non-contact pulse wave signal with a quasi-periodic signal, wherein the frequency of the non-contact pulse wave signal comprises an average heart rate and a change part caused by heart rate variability.
S232, constructing the instantaneous frequency of the non-contact pulse wave signal, wherein the instantaneous frequency of the non-contact pulse wave signal comprises an average heart rate and an instantaneous frequency change part caused by heart rate variability.
S233, solving the instantaneous frequency caused by the heart rate variability by adopting an instantaneous frequency extraction method based on pulse frequency demodulation and a discrete energy analysis algorithm, and completing the heart rate variability feature extraction of the non-contact pulse wave signals.
Optionally, the input module is further configured to:
using a centre frequency as
High cut-off frequency of->
Low cut-off frequency of->
Extracting a first harmonic of the non-contact pulse wave signal; wherein BW is the bandwidth of the first harmonic of the non-contact pulse wave signal, BW is driven by heart rate variability information of the fundamental sidebands.
Optionally, the input module is further configured to:
extracting envelope features of the peripheral blood volume pulse waveform of the non-contact pulse wave signal, and extracting vasodilation and contraction movement features of the non-contact pulse wave signal.
The method comprises the steps of extracting the envelope characteristics of the pulse waveform of the peripheral blood volume of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.7Hz and a low cutoff frequency of 3Hz, and completing the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal.
The method comprises the steps of extracting vasodilation and contraction movement characteristics of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.009Hz and a low cutoff frequency of 0.2Hz, and completing the extraction of the vasodilation and contraction movement characteristics of the non-contact pulse wave signal.
Optionally, the input module is further configured to:
s241, recording the staged subjective self-report in the stress data by using a Liktet table, and enabling the subjective self-report estimation scores to be three-level perception stress scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report.
S242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
In one aspect, an electronic device is provided, the electronic device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction loaded and executed by the processor to implement the above-described non-contact physiological signal-based emotion recognition method.
In one aspect, a computer-readable storage medium having stored therein at least one instruction loaded and executed by a processor to implement the above-described non-contact physiological signal based emotion recognition method is provided.
Compared with the prior art, the technical scheme has at least the following beneficial effects:
according to the scheme, the cognition pressure and stress tension emotion induction experiment oriented to non-contact emotion perception verification is designed, the non-contact physiological signal detection and stress emotion perception model is established by analyzing the association mechanism between the non-contact emotion characteristics and the stress emotion, so that emotion perception based on the non-contact physiological signal is realized, and compared with a traditional physiological signal emotion perception method, the method has the advantage of non-contact; compared with the emotion perception method based on the expression voice, the method has the advantages that physiological signals are difficult to control autonomously, and true emotion is expected to be revealed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a method for identifying emotion based on a non-contact physiological signal according to an embodiment of the present invention;
FIG. 2 is a main flow chart of a non-contact physiological signal-based emotion recognition method provided by an embodiment of the invention;
FIG. 3 is a block diagram of a non-contact physiological signal based emotion recognition device provided by an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention fall within the protection scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for emotion recognition based on a non-contact physiological signal, which may be implemented by an electronic device. As shown in fig. 1, a flowchart of a method for emotion recognition based on a non-contact physiological signal, the process flow of the method may include the following steps:
s1, acquiring non-contact emotion perception data to be identified.
In a possible embodiment, the non-contact emotion perception data may be expression data obtained by imaging with a camera, voice data collected by a microphone, and so on.
S2, inputting emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model.
Optionally, the process of constructing the non-contact physiological signal detection and stress perception model in S2 includes S21-S24:
s21, obtaining tested stress data by completing a stress inducing task.
Optionally, the stress inducing tasks in S21 include a first stage cognitive stress inducing task and a second stage stress inducing stress task.
Stress inducing tasks also include pressure sources; the stress sources include social assessment threat tasks, temporal stress tasks, and loud voice feedback tasks.
Stress data includes visual imaging data, physiological data, and staged subjective self-reports.
In one possible embodiment, as shown in fig. 2, two types of stress-inducing tasks may be employed, the first stage being cognitive stress induction, such as the straop color word test and mental arithmetic test, and the second stage being induction of stress tension, such as the public interview lecture, the two stages avoiding sequential effects with a latin square design for different subjects. Both of the above two-stage stress induction experiments were confirmed by related studies to be effective in inducing stress emotion and observable changes in physiological signals and cortisol. Although the Stroop and mental tasks have been shown to lead to stress responses, to avoid experimentally induced stress that is not strong enough that a distinguishable non-contact physiological signal cannot be detected. Therefore, on the basis of previous studies, it is also planned to introduce more pressure sources, including: (1) Social assessment threats, namely, closely observing and assessing the performance of a person, wherein three main test experimenters are positioned in the opposite directions of the tested person in the experiment, and inform the tested person that the main test experimenters are to evaluate the main test experimenters; (2) Time pressure, time limits will be set in the struop and mental tasks, and (3) loud audible feedback, in particular, a jerky high-pitched audible feedback when an answer is wrong.
S22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal.
Optionally, the step S22 may include:
the method comprises the steps of obtaining an original signal of visual imaging data in stress emotion data through a built space-time characteristic representation learning model, projecting the original signal to a plane orthogonal to the original signal, and learning a plurality of color space transformation weights through a plurality of two-dimensional convolution weights of a first layer in the space-time characteristic representation learning model to obtain a non-contact pulse wave signal.
In a possible embodiment, the non-contact pulse wave recovery is to extract subtle continuous color changes of the face in the sequence of continuous video frames due to the facial absorption-reflection characteristics. The essence is thus the problem of feature analysis in a sequence of video frames, which can be modeled by spatio-temporal feature characterization learning. In the network structure design, in order to eliminate the light intensity change in the skin tone direction, signals can be projected to a plane orthogonal to the plane, so that a plurality of two-dimensional convolution weights of a first layer in the network structure are learned through space-time characteristic representation, a plurality of color space transformation weights are learned, the original signals are projected to the plane orthogonal to the plane so as to weaken the influence of the ambient light change, and further, the non-contact pulse wave with stronger signal to noise ratio is extracted.
S23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features.
Optionally, the step S23 may include: and extracting heart rate variability characteristics of the non-contact pulse wave signals and extracting peripheral hemodynamic information characteristics of the non-contact pulse wave signals.
Optionally, the extracting the heart rate variability feature of the non-contact pulse wave signal includes:
s231, modeling the non-contact pulse wave signal with a quasi-periodic signal, wherein the frequency of the non-contact pulse wave signal comprises an average heart rate and a change part caused by heart rate variability.
S232, constructing the instantaneous frequency of the non-contact pulse wave signal, wherein the instantaneous frequency of the non-contact pulse wave signal comprises an average heart rate and an instantaneous frequency change part caused by heart rate variability.
S233, solving the instantaneous frequency caused by the heart rate variability by adopting an instantaneous frequency extraction method based on pulse frequency demodulation and a discrete energy analysis algorithm, and completing the heart rate variability feature extraction of the non-contact pulse wave signals.
In a possible implementation manner, although the accuracy of the recovered pulse wave signal can be improved in step S22, the signal-to-noise ratio of the pulse wave signal obtained based on the telemetry is still relatively low, so that the heartbeat interval extraction based on the peak point positioning is susceptible to significant interference, and therefore, the invention provides a robust extraction method of the heart rate variability characteristic in the non-contact pulse wave signal under the condition of low signal-to-noise ratio. The heart rate variability describes the change of the heart beat interval, the time period provided by the non-contact emotion perception requirement is generally short, and the corresponding short-time heart rate variability analysis statistical characteristic mainly comprises the standard deviation of the heart beat interval and the root mean square difference between the continuous heart beat intervals, and the short-time heart rate variability analysis statistical characteristic respectively corresponds to the low-frequency and high-frequency components of the heart rate variability. The pulse wave signal is modeled as a quasi-periodic signal whose frequency is made up of the average heart rate and the fraction of the variation caused by the heart rate variability. Specifically expressed as the following formula (1):
wherein,,
representing a non-contact pulse wave signal; />
Is indicated at->
Time->
Amplitude of subharmonic;
is indicated at->
Time->
Instantaneous phase of subharmonic.
Further, the method comprises the steps of,
is +.>
I.e. comprising the average heart rate and the instantaneous frequency variation caused by heart rate variability, as follows equation (2):
wherein,,
represents the average heart rate; />
Representing the instantaneous frequency change caused by heart rate variability.
Further, the frequency energy of the pulse wave signal is mainly concentrated
The first harmonic of (c) may be represented by the following formula (3):
wherein,,
and->
Respectively correspond to amplitude adjustment and frequency adjustment components.
Further, inspired by an algorithm of the contact type pulse wave signal processing, a pulse frequency domain demodulation method is to be adopted to estimate the instantaneous frequency of the base harmonic signal. Assuming the first harmonic
Is described as BW. BW is driven by heart rate variability information of the fundamental sidebands. With a centre frequency +.>
The high and low cut-off frequencies are respectively->
And->
The band-pass filter of (2) can separate out the harmonic components, wherein +.>
Represents the current analysis->
Signal fragments. Its instantaneous frequency extraction is a typical frequency demodulation problem. To utilize a discrete energy analysis algorithm that is not dependent on a time period, the original signal may be represented as a discrete sequence, where m corresponds to a discrete sample of the signal, and the following formula describes the specific steps:
and by filtering
Remove by->
The high frequency component generated is represented by the following formulas (5) (6):
further, to avoid spurious estimates due to abrupt discontinuities in the signal caused by abrupt sparse noise artifacts, a threshold is set as in equation (5)
. Thus the instantaneous frequency of heart rate variability can be determined by +.>
And (5) calculating.
Optionally, the extracting of the peripheral hemodynamic information feature of the non-contact pulse wave signal includes:
extracting envelope characteristics of peripheral blood volume pulse waveforms of the non-contact pulse wave signals and extracting characteristics of vasodilation and contraction movements of the non-contact pulse wave signals.
The method comprises the steps of extracting the envelope characteristics of the pulse waveform of the peripheral blood volume of a non-contact pulse wave signal, wherein the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal comprises the step of carrying out Butterworth filtering with the high cutoff frequency of 0.7Hz and the low cutoff frequency of 3Hz, and the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal is completed.
The vasodilation and contraction movement characteristic extraction of the non-contact pulse wave signal comprises the steps of carrying out Butterworth filtering with the high cutoff frequency of 0.009Hz and the low cutoff frequency of 0.2Hz, and completing the vasodilation and contraction movement characteristic extraction of the non-contact pulse wave signal.
In a possible embodiment, the recovered pulse wave also contains peripheral hemodynamic information, including the envelope of the peripheral blood volume pulse waveform, vasodilation exercise, and the like. The specific extraction method comprises the following steps: for heart rate variability analysis, high frequency components (0.15-0.4 Hz) and low frequency components (0.04-0.15) are extracted, and the LF/HF (low frequency/high frequency) ratio is calculated as the ratio of power in the range of 0.01-0.15Hz to power in the range of 0.15-0.4 Hz. Normalized power is calculated by dividing the low and high frequency power by the total power of 0.04-0.4 Hz. The LF/HF ratio is dimensionless and therefore not affected by normalization. And (3) performing Butterworth filtering with high cutoff frequency and low cutoff frequency of 0.7Hz and 3Hz respectively on the envelope characteristics of the peripheral blood volume pulse waveform, and then calculating the integral of the power spectrum of the signal in the same frequency band to obtain the measurement of the signal amplitude. For the vasodilation and contraction motion description, butterworth filtering is also applied to the signals (the cut-off frequency is 0.009 and 0.2 Hz), the signals generated by the Butterworth filtering are vasodilation estimated values, and then the integral of the power spectrum of the signals in the same frequency band is calculated, so that the extraction of the time-frequency domain variability characteristics of the pulse wave is realized.
S24, according to stress emotion data and non-contact emotion characteristics, the non-contact physiological signal detection and the stress emotion perception model construction are completed.
Optionally, the step S24 may include S241-S242:
s241, recording the staged subjective self-report in the stress data by using a Liktet table, and enabling the subjective self-report estimation scores to be three-level perception stress scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report.
S242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
In a possible implementation manner, the subjective report of the test is collected through a questionnaire scale after step S21, and is recorded by using a licker scale. The self-reported perceived psychological stress often has large individual differences, and therefore, a standardized clustering method is adopted to calibrate subjective evaluation. The subjective evaluation scores of the participants are correspondingly three-level perception stress scores through a clustering algorithm, so that relevant truth labels are obtained, and a stress estimation model is established by combining the extracted pulse waves and hemodynamic physiological indexes through a data driving method. After the model is built, the first step is not generated in actual use, and the non-contact physiological emotion characteristics output in the second and third steps are directly input into a pre-built model to directly output a non-contact emotion perception result.
And S3, according to the emotion perception data and the non-contact physiological signal detection and stress emotion perception model, realizing emotion recognition based on the non-contact physiological signal.
Based on emotion perception of physiological signal sensing, the application scene is limited due to the perception of the acquisition device or the introduction of additional emotion; the emotion perception method based on video expression analysis is difficult to reveal true emotion because macroscopic expressions can mask the emotion; in contrast, the non-contact pulse wave extraction method benefits from the non-contact, rapid and universal autonomic nervous system response capability, and has application potential for stress emotion assessment in a natural open scene. Pulse wave signals contain not only heart rate information, but the propagation of pulse pressure in the arteries causes changes in blood volume, thereby changing pulse shape and temporal characteristics, which can provide a large amount of information about the vascular system, including autonomic functions and vascular characteristics. Based on the method, the invention provides a non-contact physiological signal-based emotion perception method, firstly, visual imaging data, physiological data and staged subjective self-report related to the psychological stress level are collected through stress emotion induction, then, emotion feature extraction is carried out by combining non-contact physiological signal detection, and a non-contact physiological signal detection and stress emotion perception model is established through correlation mechanism analysis between emotion features and stress emotion, so that non-contact stress emotion perception based on physiological signals is realized. In addition, an instantaneous frequency extraction method based on pulse frequency demodulation is provided for the problem of high peak positioning error in HRV analysis with low signal-to-noise ratio.
In the embodiment of the invention, a cognition pressure and stress tension emotion induction experiment oriented to non-contact emotion perception verification is designed, a non-contact physiological signal detection and stress emotion perception model is established by analyzing a correlation mechanism between non-contact emotion characteristics and stress emotion, so that emotion perception based on non-contact physiological signals is realized, and compared with a traditional physiological signal emotion perception method, the method has the advantage of non-contact; compared with the emotion perception method based on the expression voice, the method has the advantages that physiological signals are difficult to control autonomously, and true emotion is expected to be revealed.
As shown in fig. 3, an embodiment of the present invention provides a non-contact physiological signal-based emotion recognition device 300, where the device 300 is applied to implement a non-contact physiological signal-based emotion recognition method, and the device 300 includes:
the acquiring module 310 is configured to acquire non-contact emotion perception data to be identified.
The input module 320 is configured to input emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model.
The output module 330 is configured to implement emotion recognition based on the non-contact physiological signal according to the emotion sensing data and the non-contact physiological signal detection and stress emotion sensing model.
Optionally, the input module 320 is further configured to:
s21, obtaining stress data of the testee by completing a stress inducing task.
S22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal.
S23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features.
S24, according to stress emotion data and non-contact emotion characteristics, the non-contact physiological signal detection and the stress emotion perception model construction are completed.
Optionally, the stress-inducing tasks include a first stage cognitive stress-inducing task and a second stage stress-inducing stress-emotional task.
Stress inducing tasks also include pressure sources; the stress sources include social assessment threat tasks, temporal stress tasks, and loud voice feedback tasks.
Stress data includes visual imaging data, physiological data, and staged subjective self-reports.
Optionally, the input module 320 is further configured to:
the method comprises the steps of obtaining an original signal of visual imaging data in stress emotion data through a built space-time characteristic representation learning model, projecting the original signal to a plane orthogonal to the original signal, and learning a plurality of color space transformation weights through a plurality of two-dimensional convolution weights of a first layer in the space-time characteristic representation learning model to obtain a non-contact pulse wave signal.
Optionally, the input module 320 is further configured to:
extracting heart rate variability features of the non-contact pulse wave signals and extracting peripheral hemodynamic information features of the non-contact pulse wave signals.
Optionally, the input module 320 is further configured to:
s231, modeling the non-contact pulse wave signal with a quasi-periodic signal, wherein the frequency of the non-contact pulse wave signal comprises an average heart rate and a change part caused by heart rate variability.
S232, constructing the instantaneous frequency of the non-contact pulse wave signal, wherein the instantaneous frequency of the non-contact pulse wave signal comprises an average heart rate and an instantaneous frequency change part caused by heart rate variability.
S233, solving the instantaneous frequency caused by the heart rate variability by adopting an instantaneous frequency extraction method based on pulse frequency demodulation and a discrete energy analysis algorithm, and completing the heart rate variability feature extraction of the non-contact pulse wave signals.
Optionally, the input module 320 is further configured to:
using a centre frequency as
High cut-off frequency of->
Low cut-off frequency of->
Extracting a first harmonic of the non-contact pulse wave signal; wherein BW is the bandwidth of the first harmonic of the non-contact pulse wave signal, BW is driven by heart rate variability information of the fundamental sidebands.
Optionally, the input module 320 is further configured to:
extracting envelope features of the peripheral blood volume pulse waveform of the non-contact pulse wave signal, and extracting vasodilation and contraction movement features of the non-contact pulse wave signal.
The method comprises the steps of extracting the envelope characteristics of the pulse waveform of the peripheral blood volume of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.7Hz and a low cutoff frequency of 3Hz, and completing the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal.
The method comprises the steps of extracting vasodilation and contraction movement characteristics of a non-contact pulse wave signal, including Butterworth filtering with a high cutoff frequency of 0.009Hz and a low cutoff frequency of 0.2Hz, and completing the extraction of the vasodilation and contraction movement characteristics of the non-contact pulse wave signal.
Optionally, the input module 320 is further configured to:
s241, recording the staged subjective self-report in the stress data by using a Liktet table, and enabling the subjective self-report estimation scores to be three-level perception stress scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report.
S242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
In the embodiment of the invention, a cognition pressure and stress tension emotion induction experiment oriented to non-contact emotion perception verification is designed, a non-contact physiological signal detection and stress emotion perception model is established by analyzing a correlation mechanism between non-contact emotion characteristics and stress emotion, so that emotion perception based on non-contact physiological signals is realized, and compared with a traditional physiological signal emotion perception method, the method has the advantage of non-contact; compared with the emotion perception method based on the expression voice, the method has the advantages that physiological signals are difficult to control autonomously, and true emotion is expected to be revealed.
Fig. 4 is a schematic structural diagram of an electronic device 400 according to an embodiment of the present invention, where the electronic device 400 may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 401 and one or more memories 402, where at least one instruction is stored in the memories 402, and the at least one instruction is loaded and executed by the processor 401 to implement the following emotion recognition method based on non-contact physiological signals:
s1, acquiring non-contact emotion perception data to be identified.
S2, inputting emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model.
And S3, according to the emotion perception data and the non-contact physiological signal detection and stress emotion perception model, realizing emotion recognition based on the non-contact physiological signal.
In an exemplary embodiment, a computer readable storage medium, e.g. a memory comprising instructions executable by a processor in a terminal to perform the above-described non-contact physiological signal based emotion recognition method is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (7)
1. A method of emotion recognition based on non-contact physiological signals, the method comprising:
s1, acquiring non-contact emotion perception data to be identified;
s2, inputting the emotion perception data into a constructed non-contact physiological signal detection and stress emotion perception model;
s3, according to the emotion perception data and the non-contact physiological signal detection and stress emotion perception model, emotion recognition based on the non-contact physiological signal is achieved;
the constructing process of the non-contact physiological signal detection and stress emotion perception model in the S2 comprises the following steps:
s21, obtaining stress emotion data of a tested person by completing a stress emotion inducing task;
the stress inducing tasks in the step S21 comprise a first-stage cognitive stress inducing task and a second-stage stress inducing stress emotion task;
the first-stage cognitive stress induction tasks comprise a Stroop color word test and a mental arithmetic test;
the second stage stress-inducing stress emotional task includes a public interview lecture;
the stress inducing task further comprises a pressure source; the pressure source comprises a social evaluation threat task, a time pressure task and a loud sound feedback task;
the stress data comprises visual imaging data, physiological data and staged subjective self-report;
s22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal;
s23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features;
s24, according to the stress emotion data and the non-contact emotion characteristics, finishing non-contact physiological signal detection and construction of a stress emotion perception model;
in S24, according to the stress data and the non-contact emotion feature, the method completes non-contact physiological signal detection and construction of a stress perception model, including:
s241, recording the staged subjective self-report in the stress emotion data by using a Likter scale, and enabling subjective self-report estimation scores to be three-level perception stress emotion scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report;
s242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
2. The method according to claim 1, wherein the step S22 of performing non-contact physiological signal detection on the stress data to obtain a non-contact pulse wave signal includes:
and acquiring an original signal of visual imaging data in the stress emotion data through the constructed space-time characteristic representation learning model, projecting the original signal to a plane orthogonal to the original signal, and learning a plurality of color space transformation weights through a plurality of two-dimensional convolution weights of a first layer in the space-time characteristic representation learning model to obtain a non-contact pulse wave signal.
3. The method according to claim 1, wherein the feature extraction of the non-contact pulse wave signal in S23 includes:
extracting heart rate variability features of the non-contact pulse wave signals and extracting peripheral hemodynamic information features of the non-contact pulse wave signals.
4. A method according to claim 3, wherein the extracting of heart rate variability features of the non-contact pulse wave signal comprises:
s231, modeling the non-contact pulse wave signal with a quasi-periodic signal, wherein the frequency of the non-contact pulse wave signal comprises an average heart rate and a change part caused by heart rate variability;
s232, constructing the instantaneous frequency of the non-contact pulse wave signal, wherein the instantaneous frequency of the non-contact pulse wave signal comprises an average heart rate and an instantaneous frequency change part caused by heart rate variability;
s233, solving the instantaneous frequency caused by the heart rate variability by adopting an instantaneous frequency extraction method based on pulse frequency demodulation and a discrete energy analysis algorithm, and completing the heart rate variability feature extraction of the non-contact pulse wave signals.
5. The method of claim 4, wherein the instantaneous frequency extraction method based on pulse frequency demodulation comprises:
using a centre frequency as
High cut-off frequency of->
Low cut-off frequency of->
Extracting a first harmonic of the non-contact pulse wave signal; wherein BW is the bandwidth of the first harmonic of the non-contact pulse wave signal, said BW being driven by heart rate variability information of the fundamental sidebands.
6. A method according to claim 3, wherein said extracting peripheral hemodynamic information features of the non-contact pulse wave signal comprises:
extracting peripheral blood volume pulse waveform envelope characteristics of the non-contact pulse wave signal, and extracting vasodilation and contraction movement characteristics of the non-contact pulse wave signal;
extracting the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal, wherein the extracting comprises the steps of carrying out Butterworth filtering with the high cutoff frequency of 0.7Hz and the low cutoff frequency of 3Hz, and completing the extraction of the envelope characteristics of the pulse waveform of the peripheral blood volume of the non-contact pulse wave signal;
extracting the vasodilation and contraction movement characteristics of the non-contact pulse wave signals, including performing Butterworth filtering with a high cutoff frequency of 0.009Hz and a low cutoff frequency of 0.2Hz, so as to finish the extraction of the vasodilation and contraction movement characteristics of the non-contact pulse wave signals.
7. A non-contact physiological signal based emotion recognition device, the device comprising:
the acquisition module is used for acquiring non-contact emotion perception data to be identified;
the input module is used for inputting the emotion perception data into the constructed non-contact physiological signal detection and stress emotion perception model;
the output module is used for detecting and stress emotion perception models according to the emotion perception data and the non-contact physiological signals and realizing emotion recognition based on the non-contact physiological signals;
the non-contact physiological signal detection and stress perception model construction process comprises the following steps:
s21, obtaining stress emotion data of a tested person by completing a stress emotion inducing task;
the stress inducing tasks in the step S21 comprise a first-stage cognitive stress inducing task and a second-stage stress inducing stress emotion task;
the first-stage cognitive stress induction tasks comprise a Stroop color word test and a mental arithmetic test;
the second stage stress-inducing stress emotional task includes a public interview lecture;
the stress inducing task further comprises a pressure source; the pressure source comprises a social evaluation threat task, a time pressure task and a loud sound feedback task;
the stress data comprises visual imaging data, physiological data and staged subjective self-report;
s22, carrying out non-contact physiological signal detection on the stress emotion data to obtain a non-contact pulse wave signal;
s23, extracting features of the non-contact pulse wave signals to obtain non-contact emotion features;
s24, according to the stress emotion data and the non-contact emotion characteristics, finishing non-contact physiological signal detection and construction of a stress emotion perception model;
in S24, according to the stress data and the non-contact emotion feature, the method completes non-contact physiological signal detection and construction of a stress perception model, including:
s241, recording the staged subjective self-report in the stress emotion data by using a Likter scale, and enabling subjective self-report estimation scores to be three-level perception stress emotion scores through a clustering algorithm to obtain a truth value label of the staged subjective self-report;
s242, a non-contact physiological signal detection and stress emotion perception model is established by adopting a data driving method according to the staged subjective self-report, the truth value label of the staged subjective self-report, the heart rate variability characteristic of the non-contact pulse wave signal and the peripheral hemodynamic information characteristic of the non-contact pulse wave signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310158469.7A CN115840890B (en) | 2023-02-24 | 2023-02-24 | Emotion recognition method and device based on non-contact physiological signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310158469.7A CN115840890B (en) | 2023-02-24 | 2023-02-24 | Emotion recognition method and device based on non-contact physiological signals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115840890A CN115840890A (en) | 2023-03-24 |
| CN115840890B true CN115840890B (en) | 2023-05-19 |
Family
ID=85580100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310158469.7A Active CN115840890B (en) | 2023-02-24 | 2023-02-24 | Emotion recognition method and device based on non-contact physiological signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115840890B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116758619B (en) * | 2023-08-17 | 2023-11-24 | 山东大学 | Facial video-based emotion classification method, system, storage medium and equipment |
| CN118177805A (en) * | 2024-05-15 | 2024-06-14 | 深圳市爱都科技有限公司 | Emotion recognition method and device, electronic equipment and readable storage medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11103140B2 (en) * | 2015-06-14 | 2021-08-31 | Facense Ltd. | Monitoring blood sugar level with a comfortable head-mounted device |
| CN107212896A (en) * | 2017-04-05 | 2017-09-29 | 天津大学 | A kind of emotional stability overall evaluation system and information processing method |
| CN107220591A (en) * | 2017-04-28 | 2017-09-29 | 哈尔滨工业大学深圳研究生院 | Multi-modal intelligent mood sensing system |
| US20220015654A1 (en) * | 2018-08-29 | 2022-01-20 | Sleep Advice Technologies S.R.L. | Photoplethysmography based detection of transitions between awake, drowsiness, and sleep phases of a subject |
| US11545173B2 (en) * | 2018-08-31 | 2023-01-03 | The Regents Of The University Of Michigan | Automatic speech-based longitudinal emotion and mood recognition for mental health treatment |
| US11790586B2 (en) * | 2020-06-19 | 2023-10-17 | Microsoft Technology Licensing, Llc | Generating physio-realistic avatars for training non-contact models to recover physiological characteristics |
| RU2750745C1 (en) * | 2020-09-23 | 2021-07-02 | Общество с ограниченной ответственностью «Оптические медицинские диагностические системы» (ООО «ОДС-мед») | Method and device for measuring the pulse wave propagation velocity when measuring blood pressure by oscillometric method |
| CN114305325A (en) * | 2020-09-30 | 2022-04-12 | 华为云计算技术有限公司 | Emotion detection method and device |
| CN113017630B (en) * | 2021-03-02 | 2022-06-24 | 贵阳像树岭科技有限公司 | Visual perception emotion recognition method |
| CN113229790A (en) * | 2021-05-17 | 2021-08-10 | 浙江大学 | Non-contact mental stress assessment system |
| CN115624322B (en) * | 2022-11-17 | 2023-04-25 | 北京科技大学 | Non-contact physiological signal detection method and system based on efficient space-time modeling |
-
2023
- 2023-02-24 CN CN202310158469.7A patent/CN115840890B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115840890A (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115840890B (en) | Emotion recognition method and device based on non-contact physiological signals | |
| Hwang et al. | Deep ECGNet: An optimal deep learning framework for monitoring mental stress using ultra short-term ECG signals | |
| Von Rosenberg et al. | Resolving ambiguities in the LF/HF ratio: LF-HF scatter plots for the categorization of mental and physical stress from HRV | |
| Zhao et al. | EmotionSense: Emotion recognition based on wearable wristband | |
| Sološenko et al. | Modeling of the photoplethysmogram during atrial fibrillation | |
| Agrafioti et al. | ECG pattern analysis for emotion detection | |
| Gravina et al. | Automatic methods for the detection of accelerative cardiac defense response | |
| CN106407733A (en) | Depression risk screening system and method based on virtual reality scene electroencephalogram signal | |
| CN108366732A (en) | Sleepy starting detection | |
| Jerritta et al. | Electrocardiogram-based emotion recognition system using empirical mode decomposition and discrete Fourier transform. | |
| RU2708807C2 (en) | Algorithm of integrated remote contactless multichannel analysis of psychoemotional and physiological state of object based on audio and video content | |
| CN114781465B (en) | rPPG-based non-contact fatigue detection system and method | |
| CN111407243B (en) | Pulse signal pressure identification method based on deep learning | |
| Hajeb‐M et al. | Deep neural network approach for continuous ECG‐based automated external defibrillator shock advisory system during cardiopulmonary resuscitation | |
| CN113197579A (en) | Intelligent psychological assessment method and system based on multi-mode information fusion | |
| CN111920420A (en) | Patient behavior multi-modal analysis and prediction system based on statistical learning | |
| Nandi et al. | Extracting new information from old waveforms: Symmetric projection attractor reconstruction: Where maths meets medicine | |
| Li et al. | Multi-modal emotion recognition based on deep learning of EEG and audio signals | |
| Immanuel et al. | Recognition of emotion with deep learning using EEG signals-the next big wave for stress management in this covid-19 outbreak | |
| Ren et al. | Comparison of the use of pupil diameter and galvanic skin response signals for affective assessment of computer users | |
| Cosentino et al. | Human-humanoid robot social interaction: Laughter | |
| Hossain et al. | Cognitive load measurement using galvanic skin response for listening tasks | |
| Athaya et al. | An efficient fingertip photoplethysmographic signal artifact detection method: A machine learning approach | |
| Vettivel et al. | System for detecting student attention pertaining and alerting | |
| Kim et al. | Development of person-independent emotion recognition system based on multiple physiological signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |