CN113974625A - 一种基于脑机跨模态迁移的情绪识别方法 - Google Patents
一种基于脑机跨模态迁移的情绪识别方法 Download PDFInfo
- Publication number
- CN113974625A CN113974625A CN202111210470.7A CN202111210470A CN113974625A CN 113974625 A CN113974625 A CN 113974625A CN 202111210470 A CN202111210470 A CN 202111210470A CN 113974625 A CN113974625 A CN 113974625A
- Authority
- CN
- China
- Prior art keywords
- image
- modal
- electroencephalogram
- cross
- emotion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000008909 emotion recognition Effects 0.000 title claims abstract description 36
- 238000013508 migration Methods 0.000 title claims abstract description 30
- 230000005012 migration Effects 0.000 title claims abstract description 30
- 230000008451 emotion Effects 0.000 claims abstract description 43
- 230000008569 process Effects 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 19
- 230000002996 emotional effect Effects 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 230000000007 visual effect Effects 0.000 claims description 11
- 235000014653 Carica parviflora Nutrition 0.000 claims description 8
- 241000243321 Cnidaria Species 0.000 claims description 8
- 238000009826 distribution Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000001537 neural effect Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 210000004556 brain Anatomy 0.000 abstract description 24
- 230000001149 cognitive effect Effects 0.000 abstract description 15
- 238000010801 machine learning Methods 0.000 abstract description 7
- 230000003930 cognitive ability Effects 0.000 abstract description 5
- 238000013459 approach Methods 0.000 abstract description 4
- 238000013461 design Methods 0.000 abstract description 2
- 238000012512 characterization method Methods 0.000 description 15
- 238000002474 experimental method Methods 0.000 description 13
- 238000013527 convolutional neural network Methods 0.000 description 7
- 230000001815 facial effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000001617 migratory effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000013401 experimental design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/377—Electroencephalography [EEG] using evoked responses
- A61B5/378—Visual stimuli
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Psychiatry (AREA)
- Animal Behavior & Ethology (AREA)
- Evolutionary Computation (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Physiology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Psychology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Educational Technology (AREA)
- Hospice & Palliative Care (AREA)
- Social Psychology (AREA)
Abstract
Description
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111210470.7A CN113974625B (zh) | 2021-10-18 | 2021-10-18 | 一种基于脑机跨模态迁移的情绪识别方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111210470.7A CN113974625B (zh) | 2021-10-18 | 2021-10-18 | 一种基于脑机跨模态迁移的情绪识别方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113974625A true CN113974625A (zh) | 2022-01-28 |
CN113974625B CN113974625B (zh) | 2024-05-03 |
Family
ID=79739174
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111210470.7A Active CN113974625B (zh) | 2021-10-18 | 2021-10-18 | 一种基于脑机跨模态迁移的情绪识别方法 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113974625B (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114140814A (zh) * | 2022-02-07 | 2022-03-04 | 北京无疆脑智科技有限公司 | 情绪识别能力的训练方法、装置及电子设备 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110110584A (zh) * | 2019-03-14 | 2019-08-09 | 杭州电子科技大学 | 一种基于cnn的情绪特征分类方法 |
CN110781945A (zh) * | 2019-10-22 | 2020-02-11 | 太原理工大学 | 一种融合多种特征的脑电信号情感识别方法及系统 |
WO2020085581A1 (ko) * | 2018-10-24 | 2020-04-30 | 서경대학교 산학협력단 | 영상평가시스템 및 영상평가방법 |
CN111523601A (zh) * | 2020-04-26 | 2020-08-11 | 道和安邦(天津)安防科技有限公司 | 一种基于知识引导和生成对抗学习的潜在情绪识别方法 |
CN111616721A (zh) * | 2020-05-31 | 2020-09-04 | 天津大学 | 基于深度学习和脑机接口的情绪识别系统及应用 |
CN112690793A (zh) * | 2020-12-28 | 2021-04-23 | 中国人民解放军战略支援部队信息工程大学 | 情绪脑电迁移模型训练方法、系统及情绪识别方法和设备 |
CN113378650A (zh) * | 2021-05-19 | 2021-09-10 | 重庆邮电大学 | 一种基于脑电源成像和正则化共空间模式的情绪识别方法 |
-
2021
- 2021-10-18 CN CN202111210470.7A patent/CN113974625B/zh active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020085581A1 (ko) * | 2018-10-24 | 2020-04-30 | 서경대학교 산학협력단 | 영상평가시스템 및 영상평가방법 |
CN110110584A (zh) * | 2019-03-14 | 2019-08-09 | 杭州电子科技大学 | 一种基于cnn的情绪特征分类方法 |
CN110781945A (zh) * | 2019-10-22 | 2020-02-11 | 太原理工大学 | 一种融合多种特征的脑电信号情感识别方法及系统 |
CN111523601A (zh) * | 2020-04-26 | 2020-08-11 | 道和安邦(天津)安防科技有限公司 | 一种基于知识引导和生成对抗学习的潜在情绪识别方法 |
CN111616721A (zh) * | 2020-05-31 | 2020-09-04 | 天津大学 | 基于深度学习和脑机接口的情绪识别系统及应用 |
CN112690793A (zh) * | 2020-12-28 | 2021-04-23 | 中国人民解放军战略支援部队信息工程大学 | 情绪脑电迁移模型训练方法、系统及情绪识别方法和设备 |
CN113378650A (zh) * | 2021-05-19 | 2021-09-10 | 重庆邮电大学 | 一种基于脑电源成像和正则化共空间模式的情绪识别方法 |
Non-Patent Citations (1)
Title |
---|
WENFEN LING 等: "Facial Emotion Recognition Based on Brain and Machine Collaborative Intelligence", 2019 IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND VIRTUAL ENVIRONMENTS FOR MEASUREMENT SYSTEMS AND APPLICATIONS (CIVEMSA), 20 April 2020 (2020-04-20) * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114140814A (zh) * | 2022-02-07 | 2022-03-04 | 北京无疆脑智科技有限公司 | 情绪识别能力的训练方法、装置及电子设备 |
Also Published As
Publication number | Publication date |
---|---|
CN113974625B (zh) | 2024-05-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111616721B (zh) | 基于深度学习和脑机接口的情绪识别系统及应用 | |
CN110507335B (zh) | 基于多模态信息的服刑人员心理健康状态评估方法及系统 | |
CN106682616B (zh) | 基于双通道特征深度学习的新生儿疼痛表情识别方法 | |
CN111709267B (zh) | 深度卷积神经网络的脑电信号情感识别方法 | |
CN112381008B (zh) | 一种基于并行序列通道映射网络的脑电情感识别方法 | |
CN111134666A (zh) | 一种多通道脑电数据的情绪识别方法及电子装置 | |
CN112766173B (zh) | 一种基于ai深度学习的多模态情感分析方法及其系统 | |
CN112244873A (zh) | 一种基于混合神经网络的脑电时空特征学习与情感分类方法 | |
CN112800998B (zh) | 融合注意力机制和dmcca的多模态情感识别方法及系统 | |
Chen et al. | Smg: A micro-gesture dataset towards spontaneous body gestures for emotional stress state analysis | |
CN113974627B (zh) | 一种基于脑机生成对抗的情绪识别方法 | |
Wang et al. | Maximum weight multi-modal information fusion algorithm of electroencephalographs and face images for emotion recognition | |
CN111920420A (zh) | 一种基于统计学习的患者行为多模态分析与预测系统 | |
CN116230234A (zh) | 多模态特征一致性心理健康异常识别方法及系统 | |
CN113974625B (zh) | 一种基于脑机跨模态迁移的情绪识别方法 | |
Murugappan et al. | Facial expression classification using KNN and decision tree classifiers | |
CN117198468A (zh) | 基于行为识别和数据分析的干预方案智慧化管理系统 | |
CN115909438A (zh) | 基于深度时空域卷积神经网络的疼痛表情识别系统 | |
CN113974628B (zh) | 一种基于脑机模态共空间的情绪识别方法 | |
Zhao et al. | GTSception: a deep learning eeg emotion recognition model based on fusion of global, time domain and frequency domain feature extraction | |
CN114186591A (zh) | 一种情绪识别系统泛化能力的提高方法 | |
Cowen et al. | Facial movements have over twenty dimensions of perceived meaning that are only partially captured with traditional methods | |
CN115429272B (zh) | 基于多模态生理信号的心理健康状态评估方法及系统 | |
Gupta et al. | EEG Signal Based Multi Class Emotion Recognition using Hybrid 1D-CNN and GRU | |
Tahira et al. | EEG based Mental Stress Detection using Deep Learning Techniques |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB03 | Change of inventor or designer information |
Inventor after: Kong Wanzeng Inventor after: Cao Zeyang Inventor after: Jin Xuanyu Inventor after: Zhang Hangkui Inventor after: Cui Qiquan Inventor after: Liu Dongjun Inventor after: Bai Yun Inventor before: Kong Wanzeng Inventor before: Liu Dongjun Inventor before: Jin Xuanyu Inventor before: Zhang Hangkui Inventor before: Cui Qiquan Inventor before: Cao Zeyang Inventor before: Bai Yun |
|
CB03 | Change of inventor or designer information | ||
GR01 | Patent grant | ||
GR01 | Patent grant |