CN115952460A - Emotion recognition method and device, computer equipment and storage medium - Google Patents
Emotion recognition method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115952460A CN115952460A CN202310237595.1A CN202310237595A CN115952460A CN 115952460 A CN115952460 A CN 115952460A CN 202310237595 A CN202310237595 A CN 202310237595A CN 115952460 A CN115952460 A CN 115952460A
- Authority
- CN
- China
- Prior art keywords
- emotion
- classification model
- physiological signal
- physiological data
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 42
- 230000008451 emotion Effects 0.000 claims abstract description 91
- 238000013145 classification model Methods 0.000 claims abstract description 86
- 238000000605 extraction Methods 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 24
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 28
- 239000008280 blood Substances 0.000 claims description 28
- 210000004369 blood Anatomy 0.000 claims description 28
- 229910052760 oxygen Inorganic materials 0.000 claims description 28
- 239000001301 oxygen Substances 0.000 claims description 28
- 238000004590 computer program Methods 0.000 claims description 24
- 230000004044 response Effects 0.000 claims description 19
- 230000036760 body temperature Effects 0.000 claims description 17
- 238000007637 random forest analysis Methods 0.000 claims description 14
- 238000000513 principal component analysis Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 abstract description 3
- 230000008859 change Effects 0.000 description 8
- 238000005259 measurement Methods 0.000 description 7
- 210000000707 wrist Anatomy 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 239000013078 crystal Substances 0.000 description 5
- 238000000354 decomposition reaction Methods 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 102000001554 Hemoglobins Human genes 0.000 description 4
- 108010054147 Hemoglobins Proteins 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000031700 light absorption Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000010521 absorption reaction Methods 0.000 description 3
- 230000003321 amplification Effects 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000010355 oscillation Effects 0.000 description 3
- 210000001519 tissue Anatomy 0.000 description 3
- 108010076504 Protein Sorting Signals Proteins 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000013186 photoplethysmography Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 208000031636 Body Temperature Changes Diseases 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000017531 blood circulation Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 230000006397 emotional response Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007510 mood change Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000009323 psychological health Effects 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 231100000430 skin reaction Toxicity 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 210000000106 sweat gland Anatomy 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention provides an emotion recognition method, an emotion recognition device, computer equipment and a storage medium, wherein the method comprises the steps of acquiring various sample physiological data of a user; inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features; generating a set of physiological signal features based on the physiological signal features; and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user. After various sample physiological data of the body of the user are acquired, the characteristic extraction model and the classification model are utilized to construct the double-layer structure robust emotion analysis classification model, and compared with the traditional emotion recognition method, the emotion of the user can be recognized more effectively and accurately.
Description
Technical Field
The invention relates to the technical field of emotion recognition, in particular to an emotion recognition method and device, computer equipment and a storage medium.
Background
With the development of science and technology, emotion recognition gradually draws a great deal of attention. On one hand, emotion recognition is becoming an important tool for analyzing human-computer interaction, and the development of artificial intelligence is promoted; on the other hand, under the nervous social rhythm, the emotion recognition is beneficial to understanding the psychological trend and guaranteeing the psychological health of the user.
The currently common emotion classification model is the wheel of emotion proposed by Plutchik, which classifies the emotion into eight categories of joy, trust, fear, surprise, sadness, disgust, anger, and expectation. The traditional emotion recognition technology generally judges the emotion of a user through the expression of the user by using an image or video method, however, the facial expression is deceptive and cannot reflect the real emotion, and shooting equipment is not easy to carry and has space limitation.
Meanwhile, the emotion changes can cause various physiological signals to change, so that accurate emotion recognition can be realized by measuring real physiological signals. Considering factors such as wearability and portability, the emotion recognition wristband takes place at the same time. At present, a photoelectric sensor and a skin electric sensor are mostly carried on the emotion recognition wrist strap to record heart rate signals, blood oxygen signals and skin electric signals, and emotion is recognized through a classification algorithm.
The influence of emotion on physiological signals is complex and the individualized differences are large. The information of the photoelectric sensor and the skin electric sensor is limited, and the emotion cannot be accurately identified. The prior art can only implement two classifications (positive/negative) of emotions or identify specific ones.
Disclosure of Invention
In view of the above, it is necessary to provide a method, an apparatus, a computer device and a storage medium for emotion recognition.
A method of emotion recognition, comprising:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In one embodiment, the step of training a classification model using the physiological signal feature set to obtain an emotion classification model for analyzing an emotion of the user further includes:
and carrying out hyper-parameter optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In one embodiment, the classification model is a random forest based classification model.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In one embodiment, the step of inputting the various types of sample physiological data into a feature extraction model, extracting features of the various types of sample physiological data, and obtaining physiological signal features includes:
and inputting the various types of sample physiological data into a feature extraction model, and extracting the features of which the contribution rate is greater than a preset contribution rate from the various types of sample physiological data to obtain physiological signal features.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
An emotion recognition apparatus comprising:
the sample physiological data acquisition module is used for acquiring various sample physiological data of the user;
the physiological signal characteristic acquisition module is used for inputting various types of sample physiological data into a characteristic extraction model, and extracting the characteristics of various types of sample physiological data to obtain physiological signal characteristics;
the physiological signal characteristic set generating module is used for generating a physiological signal characteristic set based on the physiological signal characteristics;
and the classification model training module is used for training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the steps of:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting the features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
A computer program which when executed by a processor performs the steps of:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
According to the emotion recognition method, the emotion recognition device, the computer equipment and the storage medium, after various sample physiological data of the body of the user are acquired, the characteristic extraction model and the classification model are used for constructing the double-layer structure robust emotion analysis classification model, and compared with a traditional emotion recognition method, the emotion recognition method can effectively and accurately recognize the emotion of the user.
Drawings
FIG. 1 is a diagram illustrating an application scenario of an emotion recognition method in an embodiment;
FIG. 2 is a flow diagram of a method of emotion recognition in one embodiment;
FIG. 3 is a block diagram of the structure of an emotion recognition apparatus in one embodiment;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment;
fig. 5 is a data processing diagram of an emotion recognition method in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.
Example one
The emotion recognition method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The terminal 102 may be, but not limited to, various personal computers, servers, laptops, smartphones, tablets and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers. In this embodiment, the terminal 102 is a bracelet or a wristband, the terminal 102 acquires various types of sample physiological data of a user and sends the sample physiological data to the server 104, the server 104 inputs the various types of sample physiological data to the feature extraction model, and the features of the various types of sample physiological data are extracted to obtain physiological signal features; generating a set of physiological signal features based on the physiological signal features; and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
Example two
In this embodiment, as shown in fig. 2, an emotion recognition method is provided, which includes:
In this embodiment, physiological data of human bodies of different users is acquired through various sensors as sample physiological data, which is used as a sample of a model. In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
In this embodiment, the heart rate refers to the number of beats per minute of the heart of a human body, the blood oxygen content is also referred to as a blood oxygen value, the blood oxygen content refers to the number of milliliters of oxygen contained in 100 milliliters of blood, and the skin conductance level and the skin conductance response are parameters of the skin conductance response, wherein the skin conductance level is used for indicating a skin conductance baseline and changes slowly, and the skin conductance response is used for indicating the amplitude and the speed of the skin conductance change. The posture information is the motion and posture of the human body of the user, and the sound information is the information of the sound of speaking and breathing of the user.
It should be understood that the change of emotion affects various physiological signals of the human body, such as the heart rate, the blood oxygen saturation and the body temperature are increased under the excited emotion, the skin conductance level is reduced, and meanwhile, the action frequency is increased, the amplitude is increased, the speaking speed is high, and the tone is high. At present, the traditional emotion recognition bracelet is only based on a photoelectric sensor and a skin electric sensor, the collected information is limited, and only two categories (positive/negative) of emotions or specific emotion recognition can be realized. In the embodiment, the emotion can be identified more accurately and classified more finely by comprehensively analyzing the heart rate, the blood oxygen content, the skin conductance level, the skin conductance response, the posture information, the sound information and the body temperature.
In this embodiment, after signals such as heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, body temperature and the like are collected, signal sequences such as heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, body temperature and the like are made into a data set, so that sample physiological data is obtained.
And 220, inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features.
In this embodiment, the feature extraction model is a first layer model. And through a feature extraction model, effective data in the sample physiological data are reserved, and effective features are extracted, so that the physiological signal features are obtained.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In this embodiment, the feature extraction model based on principal component analysis may perform effective dimension reduction on the data set, and retain data features as much as possible.
A set of physiological signal features is generated based on the physiological signal features, step 230.
In this embodiment, after a plurality of physiological signal features are extracted, a physiological signal feature set is produced accordingly.
And 240, training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In this embodiment, the classification model is a second layer model. And inputting the physiological characteristic signal characteristic set into a classification model for training, and training the obtained classification model so as to analyze and predict the emotion of the user.
In one embodiment, the classification model is a random forest based classification model. In this embodiment, the classification model based on the random forest algorithm may predict the emotion according to the physiological signal feature set extracted by the first layer model, so as to accurately identify the emotion of the user.
And step 250, acquiring the current human physiological data of the user.
In this embodiment, after the feature extraction model of the definition layer and the classification model of the second layer are constructed, the human physiological data of the user may be collected through the smart wristband worn by the user. It is worth mentioning that a plurality of sensors are arranged on the intelligent wrist strap, and various human physiological data of the user can be collected. The following embodiment further illustrates an emotion recognition method implemented based on the smart wristband.
In the embodiment, the emotion classification model can identify nine emotion states, a user can acquire real-time human physiological data of the user by wearing an intelligent wrist strap, the two-layer model structure of the emotion classification model is used for processing and predicting, the feature extraction model of principal component analysis of the first layer is firstly adopted to effectively reduce the dimension of the human physiological data, data features are kept as far as possible, and then the classification model of the second layer based on the random forest algorithm can predict the emotion according to the features extracted from the first layer model, so that the current emotion of the user can be accurately obtained.
In the embodiment, after various sample physiological data of the body of the user are acquired, the robust emotion analysis classification model with the double-layer structure is constructed by using the feature extraction model and the classification model, and compared with the traditional emotion recognition method, the emotion of the user can be recognized more effectively and accurately.
In one embodiment, the step of training a classification model using the physiological signal feature set to obtain an emotion classification model for analyzing an emotion of the user further includes: and carrying out hyperparametric optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In the embodiment, the physiological signal feature set is divided into a training set and a testing set according to the proportion of 7, a ten-fold cross validation method is adopted to train the model through the training set, grid search is adopted to optimize the hyper-parameters, and the selected hyper-parameters are the number of decision trees, the minimum sample size of internal nodes and the minimum sample size of leaf nodes, so that the optimized emotion classification model is obtained, the emotion classification model can analyze data more accurately, and the emotion of a user can be recognized more accurately.
In one embodiment, the step of inputting the various types of sample physiological data into a feature extraction model, extracting features of the various types of sample physiological data, and obtaining physiological signal features includes: and inputting various types of sample physiological data into a feature extraction model, and extracting features of which the contribution rate is greater than a preset contribution rate from various types of sample physiological data to obtain physiological signal features.
In this embodiment, the contribution rate refers to data that can effectively reflect human body characteristics, and in an embodiment, the preset contribution rate is 95%, and after various types of sample physiological data are input to the feature extraction model, features with a contribution rate greater than 95% in the various types of sample physiological data are extracted by the feature extraction model, so as to obtain physiological signal characteristics.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
In this embodiment, a method for implementing emotion recognition based on the smart wristband will be further described.
The intelligent wrist strap of the embodiment acquires pose and motion signals of a user through the nine-axis inertia measurement unit, acquires sound signals through the MEMS microphone, and integrates the temperature sensor, the skin electric sensor and the photoelectric sensor on the back of the wrist strap to monitor body temperature, skin electric signal intensity, heart rate and blood oxygen information respectively. The collected signals are classified through a double-layer structure model based on principal component analysis and random forests, and accurate recognition of nine states of eight emotions and no emotion can be achieved.
In this embodiment, a plurality of sensor modules are disposed on the smart wristband, each sensor module includes a photoelectric sensing module, a skin electric sensing module, an Inertial Measurement Unit (IMU) sensing module, a temperature sensing module, and a sound sensing module, and the smart wristband further includes a power supply, a charging module, a microcontroller, and a storage module. The microcontroller is respectively connected with the power supply and the storage module, and is also connected with each sensor module,
the control logic is as follows: the microcontroller controls the memory module and the five sensor modules. And at each sampling moment, the microcontroller polls and reads the data acquired by each sensor and integrates the data packet for storage. Before the next sampling moment comes, the microcontroller and the sensor module both enter a dormant state to reduce power consumption. When the timer interruption comes, each module recovers to normal work and repeats the sampling process.
The detection principle of each sensor module is as follows:
1. photoelectric sensor module
Different emotions can cause changes in heart rate and arterial blood oxygen saturation. The photoelectric sensor collects heart rate and blood oxygen information by utilizing a photoplethysmography (PPG), and the principle is that when LED light is emitted to skin, the light reflected back through skin tissues is received by the photosensitive sensor, received optical signals are converted into electric signals by the photosensitive sensor, and then the electric signals are converted into digital signals by AD. The LED light is attenuated after reflection, and when there is no large movement during measurement, the light absorption of the tissues such as muscle and bone remains substantially constant and is contained only in the direct current portion of the electrical signal, while the light absorption of the artery changes with the blood flow, constituting the alternating current portion of the electrical signal. Therefore, extracting the AC signal reflects the heart rate. The content of oxygenated hemoglobin HbO2 and hemoglobin Hb in blood has a certain proportion, and the percentage of oxygen and hemoglobin in the total hemoglobin is the blood oxygen. The light absorption coefficient of Hb to 600-800nm wavelength is higher, and the light absorption coefficient of HB02 to 800-1000 wavelength is higher. Therefore, the concentration of HbO2 and Hb can be detected by red light (600 to 800 nm) and infrared light (800 to 1000 nm), respectively, and then the blood oxygen value can be obtained according to the formula (1).
The photoelectric sensor mainly comprises three modulatable LEDs, a photosensitive sensor, a filter, a signal amplification circuit and an analog-to-digital conversion circuit. The three LEDs emit green light, red light and infrared light respectively, and the light wavelength can be modulated according to the skin and blood vessel characteristics of a user. The light signal reflected by the blood vessel is received by the photosensitive sensor and converted into an electric signal, and after filtering and amplification, the electric signal is converted into a digital signal through the analog-digital conversion circuit and then is output.
The heart rate signal represented by the green light is periodically changed from wave crest to wave trough, each pulse beat corresponds to the sudden change of the wave form, and the heart rate can be calculated by counting the wave crests. The algorithm is as follows: let S be the original signal and IMF be the eigenmode function. Performing empirical mode decomposition on the heart rate signal, wherein in a decomposition result: IMF (1) represents high-frequency noise and needs to be filtered; the frequency of the IMF (3) is close to the heart rate, and the IMF frequency is added to the original data to improve the power of the signal; IMF (4) represents low frequency motion artifacts that need to be filtered out. Thus, the signal ultimately used for peak extraction is S-IMF (1) -IMF (4) + IMF (3). Using a peak extraction algorithm, local peak points of the signal used for peak extraction are found, and the two peak points are separated by at least more than 0.3s (corresponding to a 180 bpm heart rate). And (4) taking the time interval average value of all peak events in the event sequence, and obtaining the heart rate according to the sampling rate.
The blood oxygen content is calculated according to Beer-Lambert's law, and assuming that the incident light intensity is I _0, the dc component in the reflected light intensity can be expressed as:
wherein the content of the first and second substances,
represents a non-arterial tissue absorption coefficient, < >>
Indicates its concentration, <' > or>
、/>
、/>
、/>
Respectively represent
And &>
Absorption coefficient and concentration, L denotes an optical path length. The alternating current component can be represented as:
wherein
Indicating an increased optical path length of the arterial wave.
Assuming that the wavelengths of red and infrared light are
，/>
The calculation formula for calculating the blood oxygen is as follows:
wherein A and B are
And &>
The constants relating to the absorption coefficients for red and infrared light can be determined experimentally.
2. Skin electric sensing module
Galvanic skin response refers to the change in electrical conduction through the skin when the body is stimulated. The principle is that when the body is stimulated by the outside or the emotional state is changed, the activity of the vegetative nervous system can cause the changes of the relaxation and the contraction of blood vessels in the skin, the secretion of sweat glands and the like, so that the skin resistance is changed. The skin electric sensor consists of two sensing electrodes and a signal processing circuit. The electrode collects skin electric signals of the wrist and transmits the skin electric signals to the signal processing circuit, artifacts and power frequency noise are filtered, and signal combing is achieved. The processed signals are divided into skin conductance levels (baseline skin conductance, slow change) and skin conductance responses (higher amplitude and faster speed of change) by empirical mode decomposition. In this embodiment, the empirical mode decomposition may decompose the skin electrical signal into a plurality of eigen-mode functions, different eigen-mode functions represent different vibration modes, and the vibration modes of the skin conductance level and the skin conductance response have a large difference and are represented by different eigen-mode functions, so that the empirical mode decomposition may be used to decompose the visible skin signal into the skin conductance level and the skin conductance response.
3. Inertial Measurement Unit (IMU) sensing module
The change in mood is also reflected in the actions and gestures. The nine-axis IMU comprises a three-axis gyroscope, a three-axis accelerometer and a three-axis magnetometer and is nine-axis motion tracking equipment. And carrying out ellipsoid calibration on the accelerometer and the magnetometer, and carrying out zero drift calibration on the gyroscope. And setting the sampling rate to be 10Hz, and designing a third-order Kalman filter to filter the acquired signals.
4. Temperature sensing module
During the emotional response, the body temperature changes slightly. The body temperature sensor consists of a low temperature coefficient crystal oscillator, a high temperature coefficient crystal oscillator and a storage unit, wherein the oscillation frequency of the low temperature coefficient crystal oscillator is slightly influenced by temperature and is used for generating a pulse signal with fixed frequency, and the oscillation frequency of the high temperature coefficient crystal oscillator is obviously changed along with the temperature change. The storage unit and the counter convert the oscillation frequency of the crystal oscillator into a temperature digital signal.
5. Sound sensing module
The frequency of speech and the amplitude of sound vary when people are in different emotions. Thus, emotion recognition can be aided by detecting speech frequency and amplitude. The sound sensing module comprises a MEMS microphone, a signal processing circuit and an analog-to-digital conversion circuit. The microphone collects human voice signals and converts the human voice signals into electric signals, the electric signals are transmitted to the signal processing circuit to be subjected to noise reduction and amplification, and the analog-to-digital conversion circuit converts analog signals into digital signals to be output.
The intelligent wrist strap is susceptible to environmental noise and physiological artifacts, electromagnetic interference exists among sensors, and the sensors all affect sensing data. And designing a differential mode filter circuit, a common mode filter circuit and a 50Hz notch filter for filtering. For the filtered signals, detecting abnormal points by adopting a self-adaptive threshold method, and replacing the values of the abnormal points with threshold values to finish the preprocessing of data
The emotion recognition wristband of the embodiment is provided with a photoelectric sensor, a skin electric sensor, an Inertial Measurement Unit (IMU), a temperature sensor and a sound sensor, records and processes blood oxygen, heart rate, skin electric, pose, body temperature and sound signals of a user, and can realize accurate classification of nine states of eight emotions and no emotion by classifying through a double-layer structure model based on principal component analysis and random forests.
EXAMPLE III
In this embodiment, please refer to fig. 5, the smart wristband carries a photoelectric sensor, a skin electric sensor, an inertial measurement unit, a temperature sensor, and a sound sensor (MEMS microphone), and collects the blood oxygen, heart rate, skin conductance level, skin conductance response, nine-axis posture information, body temperature, and sound signals of the user.
Because the physiological signal data volume is large, the emotion is classified and predicted by adopting a double-layer structure model based on principal component analysis and random forests. The feature extraction model based on principal component analysis can effectively reduce the dimension of the data set, and retain the data features as much as possible, and the classification model based on the random forest algorithm can predict the emotion according to the features extracted from the first layer model.
In the first layer of model, namely the feature extraction model, the processed heart rate, blood oxygen, skin conductance level, skin conductance response, nine-axis posture data, body temperature and sound signal sequence are made into a data set, the feature with the contribution rate of 95% in each physiological signal is extracted by using a principal component analysis method, and the feature data set is made into a physiological signal feature data set. And then constructing a second layer model, training the multi-classification random forest model by using the characteristic data set, carrying out hyper-parameter optimization on the model by adopting a grid search and cross validation method, and finally generating a classification model for recognizing nine emotions.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.
Example four
In this embodiment, as shown in fig. 3, there is provided an emotion recognition apparatus including:
a sample physiological data obtaining module 310, configured to obtain various types of sample physiological data of a user;
a physiological signal characteristic obtaining module 320, configured to input various types of sample physiological data into a characteristic extraction model, and extract characteristics of various types of sample physiological data to obtain physiological signal characteristics;
a physiological signal feature set generating module 330 for generating a physiological signal feature set based on the physiological signal features;
and the classification model training module 340 is configured to train a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In one embodiment, the emotion recognition apparatus further includes:
and the optimization module is used for carrying out hyper-parameter optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In one embodiment, the classification model is a random forest based classification model.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In an embodiment, the physiological signal feature obtaining module is further configured to input various types of the sample physiological data to a feature extraction model, and extract features of which contribution rates are greater than a preset contribution rate in various types of the sample physiological data to obtain physiological signal features.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
For the specific definition of the emotion recognition device, reference may be made to the above definition of the emotion recognition method, which is not described herein again. The respective units in the emotion recognition apparatus described above may be implemented in whole or in part by software, hardware, and a combination thereof. The units can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the units.
EXAMPLE five
In this embodiment, a computer device is provided, and the computer device may be a portable wearable device, and in this embodiment, the computer device is an intelligent wristband. The internal structure thereof may be as shown in fig. 4. The computer device comprises a processor, a memory, a network interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores a computer program, and the non-volatile storage medium is used for storing human physiological data of a user. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with other computer devices, such as connecting with a server, and the network interface is a wireless network interface which can access a mobile communication network and is connected with the server. The computer program is executed by a processor to implement a method of emotion recognition. The display of the computer device may be a liquid crystal display or an electronic ink display, and in some embodiments, the input device includes a touch sensor, and the input device includes a photoelectric sensor, a skin-electric sensor, an Inertial Measurement Unit (IMU), a temperature sensor, and a sound sensor.
Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, there is provided a computer device comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and carrying out hyper-parameter optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In one embodiment, the classification model is a random forest based classification model.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
and inputting various types of sample physiological data into a feature extraction model, and extracting features of which the contribution rate is greater than a preset contribution rate from various types of sample physiological data to obtain physiological signal features.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
EXAMPLE six
In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, the computer program realizing the following steps when executed by a processor:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting the features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and carrying out hyper-parameter optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In one embodiment, the classification model is a random forest based classification model.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and inputting various types of sample physiological data into a feature extraction model, and extracting features of which the contribution rate is greater than a preset contribution rate from various types of sample physiological data to obtain physiological signal features.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
EXAMPLE seven
In this embodiment, a computer program is provided, which when executed by a processor, implements the steps of:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and carrying out hyperparametric optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
In one embodiment, the classification model is a random forest based classification model.
In one embodiment, the feature extraction model is a principal component analysis-based feature extraction model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
and inputting the various types of sample physiological data into a feature extraction model, and extracting the features of which the contribution rate is greater than a preset contribution rate from the various types of sample physiological data to obtain physiological signal features.
In one embodiment, the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of emotion recognition, comprising:
acquiring various types of sample physiological data of a user;
inputting various types of sample physiological data into a feature extraction model, and extracting features of various types of sample physiological data to obtain physiological signal features;
generating a set of physiological signal features based on the physiological signal features;
and training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
2. The method of claim 1, wherein the step of training a classification model using the set of physiological signal features to obtain an emotion classification model for analyzing the emotion of the user further comprises:
and carrying out hyper-parameter optimization on the emotion classification model by adopting a grid search and cross verification method to obtain the optimized emotion classification model.
3. The method of claim 1, wherein the classification model is a random forest based classification model.
4. The method of claim 1, wherein the feature extraction model is a principal component analysis-based feature extraction model.
5. The method of claim 1, wherein the step of inputting the various types of the sample physiological data into a feature extraction model to extract features of the various types of the sample physiological data to obtain physiological signal features comprises:
and inputting the various types of sample physiological data into a feature extraction model, and extracting the features of which the contribution rate is greater than a preset contribution rate from the various types of sample physiological data to obtain physiological signal features.
6. The method of claim 1, wherein the sample physiological data includes heart rate, blood oxygen content, skin conductance level, skin conductance response, posture information, sound information, and body temperature.
7. An emotion recognition apparatus, comprising:
the sample physiological data acquisition module is used for acquiring various sample physiological data of the user;
the physiological signal characteristic acquisition module is used for inputting various types of sample physiological data into the characteristic extraction model, extracting the characteristics of various types of sample physiological data and obtaining physiological signal characteristics;
the physiological signal characteristic set generating module is used for generating a physiological signal characteristic set based on the physiological signal characteristics;
and the classification model training module is used for training a classification model by using the physiological signal feature set to obtain an emotion classification model for analyzing the emotion of the user.
8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
10. A computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310237595.1A CN115952460A (en) | 2023-03-14 | 2023-03-14 | Emotion recognition method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310237595.1A CN115952460A (en) | 2023-03-14 | 2023-03-14 | Emotion recognition method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115952460A true CN115952460A (en) | 2023-04-11 |
Family
ID=87282817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310237595.1A Pending CN115952460A (en) | 2023-03-14 | 2023-03-14 | Emotion recognition method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115952460A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118177805A (en) * | 2024-05-15 | 2024-06-14 | 深圳市爱都科技有限公司 | Emotion recognition method and device, electronic equipment and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130262352A1 (en) * | 2012-03-29 | 2013-10-03 | Postech Academy-Industry Foundation | Apparatus and method for recognizing user activity |
| CN110101398A (en) * | 2018-11-29 | 2019-08-09 | 华南理工大学 | A kind of method and system detecting mood |
| CN115211858A (en) * | 2022-08-26 | 2022-10-21 | 浙大宁波理工学院 | Emotion recognition method and system based on deep learning and storable medium |
-
2023
- 2023-03-14 CN CN202310237595.1A patent/CN115952460A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130262352A1 (en) * | 2012-03-29 | 2013-10-03 | Postech Academy-Industry Foundation | Apparatus and method for recognizing user activity |
| CN110101398A (en) * | 2018-11-29 | 2019-08-09 | 华南理工大学 | A kind of method and system detecting mood |
| CN115211858A (en) * | 2022-08-26 | 2022-10-21 | 浙大宁波理工学院 | Emotion recognition method and system based on deep learning and storable medium |
Non-Patent Citations (3)
| Title |
|---|
| 张迪 等: "基于生理信号的情绪识别研究进展", 生物医学工程学杂志, pages 229 * |
| 施苑英: "Python机器学习算法及典型案例实战", 西北工业大学出版社, pages: 228 * |
| 李发权 等: "基于PCA-SVM 多生理信息融合的情绪识别方", 山东大学学报（工学版）, pages 71 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118177805A (en) * | 2024-05-15 | 2024-06-14 | 深圳市爱都科技有限公司 | Emotion recognition method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Panwar et al. | PP-Net: A deep learning framework for PPG-based blood pressure and heart rate estimation | |
| Biswas et al. | CorNET: Deep learning framework for PPG-based heart rate estimation and biometric identification in ambulant environment | |
| Khan et al. | Biometric systems utilising health data from wearable devices: applications and future challenges in computer security | |
| Liaqat et al. | WearBreathing: Real world respiratory rate monitoring using smartwatches | |
| US20190133468A1 (en) | Method and apparatus for high accuracy photoplethysmogram based atrial fibrillation detection using wearable device | |
| Zenonos et al. | HealthyOffice: Mood recognition at work using smartphones and wearable sensors | |
| Wieclaw et al. | Biometrie identification from raw ECG signal using deep learning techniques | |
| Boukhechba et al. | ActiPPG: Using deep neural networks for activity recognition from wrist-worn photoplethysmography (PPG) sensors | |
| CN103038772B (en) | Predict the system and device of the viability of patient | |
| US20200093386A1 (en) | Method of generating a model for heart rate estimation from a photoplethysmography signal and a method and a device for heart rate estimation | |
| Senyurek et al. | A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors | |
| Tazarv et al. | A deep learning approach to predict blood pressure from ppg signals | |
| Scrugli et al. | An adaptive cognitive sensor node for ECG monitoring in the Internet of Medical Things | |
| US20220031181A1 (en) | Pulse shape analysis | |
| Ahmed et al. | IoMT-based biomedical measurement systems for healthcare monitoring: A review | |
| CN115952460A (en) | Emotion recognition method and device, computer equipment and storage medium | |
| Maiorana | A survey on biometric recognition using wearable devices | |
| Pinge et al. | A comparative study between ECG-based and PPG-based heart rate monitors for stress detection | |
| Alam et al. | Autocognisys: Iot assisted context-aware automatic cognitive health assessment | |
| Liakopoulos et al. | CNN-based stress and emotion recognition in ambulatory settings | |
| Zhang et al. | Video based cocktail causal container for blood pressure classification and blood glucose prediction | |
| Gautam et al. | An smartphone-based algorithm to measure and model quantity of sleep | |
| Sundaravadivel et al. | i-sad: An edge-intelligent iot-based wearable for substance abuse detection | |
| Premalatha et al. | Design and implementation of intelligent patient in-house monitoring system based on efficient XGBoost-CNN approach | |
| Hossain et al. | Sleep well: a sound sleep monitoring framework for community scaling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230411 |