CN115211854A - Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves - Google Patents

Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves Download PDF

Info

Publication number
CN115211854A
CN115211854A CN202110123481.5A CN202110123481A CN115211854A CN 115211854 A CN115211854 A CN 115211854A CN 202110123481 A CN202110123481 A CN 202110123481A CN 115211854 A CN115211854 A CN 115211854A
Authority
CN
China
Prior art keywords
driver
emotion
brain wave
brain
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110123481.5A
Other languages
Chinese (zh)
Inventor
杨巨成
魏峰
吴宗亮
温刚
孙笑
邢迪
吴超
于洋
王嫄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University of Science and Technology
Original Assignee
Tianjin University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University of Science and Technology filed Critical Tianjin University of Science and Technology
Priority to CN202110123481.5A priority Critical patent/CN115211854A/en
Publication of CN115211854A publication Critical patent/CN115211854A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165Evaluating the state of mind, e.g. depression, anxiety
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813Specially adapted to be attached to a specific body part
    • A61B5/6814Head
    • A61B5/6815Ear
    • A61B5/6816Ear lobe
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2503/00Evaluating a particular growth phase or type of persons or animals
    • A61B2503/20Workers
    • A61B2503/22Motor vehicles operators, e.g. drivers, pilots, captains

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Otolaryngology (AREA)
  • Physiology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention provides an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves. With the vigorous development of artificial intelligence technology and internet technology in the field of automobiles, the degree of vehicle intelligence is greatly improved, but the psychological condition of a driver is rarely concerned about, and behavior runaway caused by unstable emotion of the driver often occurs. For brain wave detection, the 'wet electrode' method is high in precision but inconvenient to wear, and the 'dry electrode' method is convenient but only has weak signals and is high in noise interference. Therefore, the invention pays attention to the psychological state of a driver, utilizes the dry electrode with limited precision, the single-channel portable brain wave detection equipment and the integrated learning to extract the time sequence characteristics, can reduce noise interference, reduce the variance of the model, establish a set of methods for analyzing the emotion of the driver in real time and simultaneously realize instant feedback and early warning so as to reduce the incentive problem of accident occurrence, integrally reduce the traffic safety risk and improve the driving comfort in the cabin.

Description

Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves
Technical Field
The invention belongs to the field of artificial intelligence and Internet of things, and relates to an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves.
Background
With the increase of vehicle intelligent degree, the vigorous development of artificial intelligence technology and internet technology in the automobile field, the aspects of intelligent human-computer interaction and the like are also changed greatly. In recent years, bus accidents still remain high, one of the reasons is that the psychological conditions of drivers are not concerned, and the behavior of drivers is out of control due to emotional problems.
In medicine, brain wave detection is complex, ointment needs to be applied to the head to increase electrode conductivity, and the method is called as a 'wet electrode' method, and although the brain wave precision is high, the method cannot be used in daily life. The consumption-level single-channel brain wave sensor dry electrode can be directly worn, is convenient to use, obtains limited signals, is weak in signals and high in noise, and cannot form accurate electroencephalograms for effective analysis, so that the consumption-level single-channel brain wave sensor dry electrode is mostly used for the fields of education and entertainment. Machine learning which has emerged in recent years can extract high-dimensional features in signals and can classify limited brain wave features.
In the supervised learning algorithm of machine learning, although a model with good performance in all aspects is expected to be obtained, the actual situation is not ideal, and sometimes only a plurality of preferred models can be obtained. If naive Bayes is suitable for a model with small correlation among different dimensions, a random forest is suitable for data with relatively low data dimensions. Ensemble learning (Ensemble Methods), however, is a "group decision" concept. I.e. multiple models are trained to predict the same problem. Three methods are mainly included. The overall error rate of the Voting method will have a lower error rate than each of the independent models. The Bootstrap aggregation method randomly extracts a training set and trains a plurality of weak classifiers for the basis. And then determining a final classification result by taking an average or voting mode. Transition fitting can be avoided to some extent. The Boosting method converts a weak learner into a strong learner through an algorithm set, and a series of weak learners are trained to combine a plurality of weak supervision models so as to obtain a better and more comprehensive strong supervision model. The potential idea of the integrated learning is that even if one weak classifier obtains wrong prediction, other weak classifiers can correct the errors, and several machine learning technologies are combined into a meta-algorithm of a prediction model, so that the effect of reducing the variance and deviation of the model or improving the prediction is achieved, and the model is more stable.
Therefore, the invention focuses on the psychological state of a driver, establishes a set of methods for analyzing the emotion of the driver in real time and realizing instant feedback and early warning by utilizing the dry electrode with limited precision, the single-channel portable brain wave detection equipment and the integrated learning, so as to reduce the incentive problem of accident occurrence, integrally reduce the traffic safety risk and improve the driving comfort in the cabin.
Disclosure of Invention
The invention aims to provide an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves by using a dry electrode with limited precision and a single-channel portable brain wave detection device.
The invention provides an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves, which comprises the following three modules:
the brain wave signal detection and preprocessing module is used for detecting brain waves, preprocessing data and extracting electroencephalogram signal time sequence characteristics;
the emotion classification module is used for classifying the brain wave signals to obtain the emotion of the driver within a period of time (which can be adjusted according to the requirement and has the range of 5 seconds to 1 minute);
and the feedback module is used for enabling the brain waves to be audible and feeding back the driver by using voice and music to adjust the emotion of the driver.
Further, the brain wave signal detection and preprocessing module acquires brain wave signals in a specific manner that: the driver electroencephalogram signals are collected by using a dry electrode with limited precision and a single-channel electroencephalogram detection module (comprising a single-channel signal processing module, three electrodes, two electrodes at earlobes and one electrode at forehead). Each time of collecting electroencephalogram signals comprises 8 waves of delta, theta, low alpha, high alpha, low beta, high beta, low gamma and high gamma,
further, the data preprocessing in the brain wave signal detection and preprocessing module specifically comprises: after data are cleaned, signal time sequence characteristics including mean, variance, summation, median, standard deviation, deviation value, kurtosis value, maximum value, minimum value and array length are extracted by using an sfresh tool.
Further, the emotion classification module specifically comprises: the Voting method of ensemble learning is used, the Voting mode is hard Voting, and 8 models of K neighbor, decision tree, random forest, adaBoosting, gradient boosting (GradientBoosting), continuous naive Bayes (GaussianNB), linear discriminant analysis and extreme gradient boosting (XGboost) are integrated. Dividing the predicted values into 5 types, corresponding to the following steps: very negative, neutral, positive, exciting.
The feedback module: judging the emotion category, firstly carrying out voice prompt to make the driver know the psychological state, audilizing the emotion, and then playing music to intervene in negative or over-excited emotion.
The invention has the advantages and positive effects that:
the time sequence characteristics of the brain wave signals are extracted, the regularity and periodicity of continuous change of the brain wave signals can be reflected, the deviation caused by extreme values and error values is statistically reduced and used as the input of machine learning, and the stability and classification accuracy of the model are improved.
By using the Voting in the integrated learning model instead of a single machine learning model, the learning accuracy is higher. A plurality of classifiers based on Boosting (such as Adaboosting, gradientboosting and XGBOSTING) and Bootstrap aggregation (such as random forest) methods and random sampling during training are used, so that the influence of noise samples on an integral model is reduced, and the variance of the model is reduced. And as the number of models increases, the variance decreases. The method is suitable for the situations of brain waves, such as large signal change, much noise and instability.
The consumption-level brain wave module is used for measurement, conductive paste does not need to be smeared, discomfort does not exist when the brain wave module is worn, and the brain wave module is convenient to use and suitable for popularization.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1: brain wave-based intelligent self-adaptive driver human-computer interaction emotion adjusting method structure diagram.
FIG. 2 is a drawing: an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves is a flow chart.
FIG. 3 is a drawing: brain wave collection method.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It should be emphasized that the embodiments described herein are illustrative rather than restrictive, and thus the present invention is not limited to the embodiments described in the detailed description, but other embodiments derived from the technical solutions of the present invention by those skilled in the art are also within the scope of the present invention.
The invention provides an intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves.
For example, the structure diagram of the brain wave-based intelligent self-adaptive driver human-computer interaction emotion regulation method (fig. 1):
the method comprises the steps of collecting brain wave signals in the driving process of a driver, transmitting the brain wave signals into an electroencephalogram processing module in real time, collecting a certain number of brain wave signals, then classifying, and finally feeding back the driver according to a classification result.
Brain wave-based intelligent self-adaptive driver man-machine interaction emotion adjusting method flow chart (figure 2):
the brain wave signal detection and preprocessing module collects brain waves by using a consumption-level single-channel brain wave module, preprocesses signal data and extracts time sequence characteristics.
Firstly, the brain wave collection mode is as shown in figure 3, two electrode clamps are arranged at the ear lobe, one electrode is arranged at the forehead, and the sampling frequency is 512Hz. UART output data, baud rate 57600,8bit data, 1 stop bit is used. 8 brain wave types are collected, a sequence of 8 unsigned integers of 3 bytes is output, and the serial port communication protocol is ch340. The 8 types of brainwaves are: delta (0.5-2.75 Hz), theta (3.5-6.75 Hz), low alpha (7.5-9.25 Hz), high alpha (10-11.75 Hz), low beta (13-16.75 Hz), high beta (18-29.75 Hz), low gamma (31-39.75 Hz), and high gamma (41-49.75 Hz). The collection was done every 1 second, 8 times (8 seconds) for a small group and 40 times (40 seconds) for a batch, forming an 8 × 8 × 5 matrix, which was pre-processed as a whole.
And secondly, removing null data and abnormal values, extracting numerical characteristics (including 10 characteristics of mean value, variance, summation, median, standard deviation, deviation value, kurtosis value, maximum value, minimum value and number row length) from the time series data by using Tsfresh to form a 10 multiplied by 5 characteristic matrix, and inputting the characteristic matrix into a classifier.
The emotion classification module uses integrated learning Voting and a method, the Voting mode is hard Voting, and 8 models of K neighbor, decision tree, random forest, adaBoosting, gradient boosting (GradientBoosting), continuous naive Bayes (GaussianB), linear discriminant analysis and extreme gradient boosting (XGBoosing) are integrated. Dividing the predicted values into 5 classes, which respectively correspond to the following steps: very negative, neutral, positive, exciting.
The method comprises the steps of firstly, defining a classifier, loading a pre-trained 8-machine learning model, loading a Sciket-lean ensemble learning Voting ensemble classifier, and defining the type as 'hard Voting' (hard Voting).
Second, load 10 × 5 feature matrix, and generate an emotion index e for each batch (10 feature values) through each classifier i,t (0<e i,t < 1), integrating a total of 8 models in the classifier, and forming a final predicted value e through hard voting t . And 5 batches in total, finally forming a 1 × 5 voting matrix, and taking the average value as a final predicted value.
And thirdly, setting a threshold value and classifying the predicted values according to the threshold value.
The feedback module synthesizes feedback prompt tones by using a voice synthesis technology and selects different music to play. The positive and pleasant songs are automatically played when the driver is in a very negative, negative mood, the soothing and soft songs are played when the driver is in an excited state, and the rest states are not fed back by music.

Claims (4)

1. An intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves is characterized by comprising the following three modules:
the brain wave signal detection and preprocessing module is used for detecting brain waves, preprocessing data and extracting electroencephalogram signal time sequence characteristics;
the emotion classification module is used for classifying the brain wave signals to obtain the emotion of the driver within a period of time (which can be adjusted according to the requirement and has the range of 5 seconds to 1 minute);
and the feedback module is used for making the brainwaves audible and feeding back the driver by using voice and music to adjust the emotion of the driver.
2. The brain wave signal detecting and preprocessing module according to claim 1, wherein a single-channel portable brain wave detecting device using precision-limited dry electrodes is used, and the time-series characteristics of brain waves are extracted.
3. The emotion classification module of claim 1, wherein a learning-by-wire Voting method is used, and the predicted values are classified into 5 classes, which correspond to very negative, neutral, positive and excited states respectively; the classification voting mode is 8 models including hard voting, K neighbor integration, decision tree integration, random forest integration, adaBoosting, gradient boosting (GradientBoosting), continuous naive Bayes (GaussianB), linear discriminant analysis and extreme gradient boosting (XGBososting).
4. The feedback module of claim 1, wherein the feedback prompt tone is synthesized by using an off-line speech synthesis technology on the internet of things terminal, different music is selected for playing according to different emotions, a positive and pleasant song is automatically played when the driver has a negative emotion, a soft and gentle song is played when the driver is excited, and music feedback is not required in other states.
CN202110123481.5A 2021-04-19 2021-04-19 Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves Pending CN115211854A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110123481.5A CN115211854A (en) 2021-04-19 2021-04-19 Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110123481.5A CN115211854A (en) 2021-04-19 2021-04-19 Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves

Publications (1)

Publication Number Publication Date
CN115211854A true CN115211854A (en) 2022-10-21

Family

ID=83605569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110123481.5A Pending CN115211854A (en) 2021-04-19 2021-04-19 Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves

Country Status (1)

Country Link
CN (1) CN115211854A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104571533A (en) * 2015-02-10 2015-04-29 北京理工大学 Device and method based on brain-computer interface technology
CN106529421A (en) * 2016-10-21 2017-03-22 燕山大学 Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology
CN110403617A (en) * 2018-04-26 2019-11-05 广州汽车集团股份有限公司 It is a kind of for monitoring the driving assistance method and system of driver's state of mind
US20190387998A1 (en) * 2014-04-22 2019-12-26 Interaxon Inc System and method for associating music with brain-state data
CN110947076A (en) * 2019-11-27 2020-04-03 华南理工大学 Intelligent brain wave music wearable device capable of adjusting mental state
CN111466931A (en) * 2020-04-24 2020-07-31 云南大学 Emotion recognition method based on EEG and food picture data set
CN112596273A (en) * 2020-12-30 2021-04-02 华南理工大学 Intelligent color-changing glasses with electroencephalogram emotion assessment and adjustment functions and control method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190387998A1 (en) * 2014-04-22 2019-12-26 Interaxon Inc System and method for associating music with brain-state data
CN104571533A (en) * 2015-02-10 2015-04-29 北京理工大学 Device and method based on brain-computer interface technology
CN106529421A (en) * 2016-10-21 2017-03-22 燕山大学 Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology
CN110403617A (en) * 2018-04-26 2019-11-05 广州汽车集团股份有限公司 It is a kind of for monitoring the driving assistance method and system of driver's state of mind
CN110947076A (en) * 2019-11-27 2020-04-03 华南理工大学 Intelligent brain wave music wearable device capable of adjusting mental state
CN111466931A (en) * 2020-04-24 2020-07-31 云南大学 Emotion recognition method based on EEG and food picture data set
CN112596273A (en) * 2020-12-30 2021-04-02 华南理工大学 Intelligent color-changing glasses with electroencephalogram emotion assessment and adjustment functions and control method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
魏琛,等: "基于集成卷积神经网络的脑电情感识别", 华东理工大学学报(自然科学版), vol. 45, no. 4, 31 August 2019 (2019-08-31), pages 614 - 622 *

Similar Documents

Publication Publication Date Title
CN110604565A (en) Brain health screening method based on portable EEG equipment
CN108847279B (en) Sleep breathing state automatic discrimination method and system based on pulse wave data
Hu et al. Epileptic signal classification based on synthetic minority oversampling and blending algorithm
CN108478216A (en) A kind of epileptic seizure intelligent Forecasting early period based on convolutional neural networks
Mallikarjun et al. Depression level prediction using EEG signal processing
CN112001305B (en) Feature optimization SSVEP asynchronous recognition method based on gradient lifting decision tree
CN111354375A (en) Cry classification method, device, server and readable storage medium
Barajas-Montiel et al. Identifying pain and hunger in infant cry with classifiers ensembles
CN111310783B (en) Speech state detection method based on electroencephalogram micro-state features and neural network model
Balam et al. Statistical channel selection method for detecting drowsiness through single-channel EEG-based BCI system
CN111743513B (en) S1 and REM sleep state detection method and system based on mixed channel EEG signal
Gao et al. EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks
CN115211854A (en) Intelligent self-adaptive driver man-machine interaction emotion adjusting method based on brain waves
CN112466284B (en) Mask voice identification method
Akinci et al. Comparison of machine learning algorithms for recognizing drowsiness in drivers using electroencephalogram (EEG) signals
Reyes-Galaviz et al. Classification of infant crying to identify pathologies in recently born babies with ANFIS
CN113768517A (en) Intelligent early warning system and early warning method for heart health quality
CN115227246A (en) Driver voice emotion recognition method for intelligent driving
CN114997229A (en) Electroencephalogram signal-based motor imagery classification method and system
CN114186591A (en) Method for improving generalization capability of emotion recognition system
Mamta et al. EEG feature extraction using time domain analysis for classifying insomnia
Nissimagoudar et al. Driver alertness detection using CNN-BiLSTM and implementation on ARM-based SBC
Gao K-Means Algorithm to Identify the Elderly Psychological Stress Analysis Algorithm
Barajas-Montiel et al. Improving baby caring with automatic infant cry recognition
EskandariNasab et al. A GRU–CNN model for auditory attention detection using microstate and recurrence quantification analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination