CN114504330A - Fatigue state monitoring system based on portable electroencephalogram acquisition head ring - Google Patents

Fatigue state monitoring system based on portable electroencephalogram acquisition head ring Download PDF

Info

Publication number
CN114504330A
CN114504330A CN202210114286.0A CN202210114286A CN114504330A CN 114504330 A CN114504330 A CN 114504330A CN 202210114286 A CN202210114286 A CN 202210114286A CN 114504330 A CN114504330 A CN 114504330A
Authority
CN
China
Prior art keywords
layer
eeg signal
output
head ring
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210114286.0A
Other languages
Chinese (zh)
Inventor
马超
张萌
高忠科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN202210114286.0A priority Critical patent/CN114504330A/en
Publication of CN114504330A publication Critical patent/CN114504330A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/251—Means for maintaining electrode contact with the body
    • A61B5/256—Wearable electrodes, e.g. having straps or bands
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7225—Details of analog processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/043—Architecture, e.g. interconnection topology based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2560/00—Constructional details of operational features of apparatus; Accessories for medical measuring apparatus
    • A61B2560/04—Constructional details of apparatus
    • A61B2560/0431—Portable apparatus, e.g. comprising a handle or case
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Psychiatry (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Psychology (AREA)
  • Physiology (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)

Abstract

A fatigue state monitoring system based on a portable EEG signal acquisition head ring comprises an EEG signal acquisition head ring and EEG signal analysis software which is connected with the EEG signal acquisition head ring through Bluetooth and is arranged at a mobile phone end, wherein the EEG signal acquisition head ring is used for acquiring an EEG signal of a user and transmitting the acquired signal to the EEG signal analysis software on the mobile phone in a wireless manner; the EEG signal analysis software processes and analyzes the received EEG signal and displays the fatigue state of the user in real time through the mobile phone. The invention can realize accurate acquisition, effective identification and correct classification of EEG electroencephalogram signals, can identify the electroencephalogram signals of a user by analyzing electroencephalogram data, obtain the brain fatigue degree of the user, classify the fatigue state of the user, provide reference for the user to know the fatigue health state of the user, and prompt the user to pay attention to rest, thereby avoiding accidents caused by fatigue operation.

Description

Fatigue state monitoring system based on portable electroencephalogram acquisition head ring
Technical Field
The invention relates to a fatigue state monitoring system. In particular to a fatigue state monitoring system based on a portable electroencephalogram acquisition head ring.
Background
According to the records of accidents in China, more than 5w of construction site safety accidents occur every year, wherein 40% of the safety accidents are caused by improper construction operations, and more than 80% of the safety accidents are caused by inattention caused by fatigue. Not only domestically, a statistical study aiming at 87134 accidents in thailand shows that the proportion of fatigue, distraction and accidents is obviously related, and in a traffic accident occurrence probability model, the influence degree even exceeds that of drunk driving.
Compared with the traditional electroencephalogram acquisition instrument of a medical institution, the portable electroencephalogram acquisition system has extremely small volume and quality under the conditions that the acquisition precision is ensured and the acquisition speed meets the requirements, has low requirements on working conditions, greatly improves the portability, and is widely applied to brain-computer interface equipment. Particularly, under the condition that the user needs to carry out daily activities, the portable electroencephalogram acquisition system can provide the condition for carrying out electroencephalogram monitoring for the user, and provides convenience for some physiological states needing to continuously monitor electroencephalogram changes. The core chip of the portable electroencephalogram acquisition system generally comprises a control chip and an A/D conversion chip. The ESP32 chip is a product of lexin corporation and can be used as a control chip. Advantages of the ESP32 chip: 1) extremely high performance: low power xtensia LX 632 bit dual core. 2) The reasonable peripheral equipment, the reasonable power consumption and the reasonable price are enriched. 3) Powerful software support: a rich software package. 4) Comprehensive and rich technical documents. 5) The types of the chips are various, and the coverage area is wide. The ESP32 meets the requirements of the electroencephalogram signal acquisition system on high precision, high stability and high computing capability, and on-chip hardware resources are enough to realize the functions required by the real-time acquisition of the electroencephalogram signals.
The electroencephalogram data is large in data volume and difficult to find out characteristics visually. It is difficult to perform manual analysis after data collection. In recent years, with the development of computer technology, more and more researchers favor to use a computer to analyze and process electroencephalogram data by using some analysis algorithms. Among many methods for analyzing electroencephalogram signals, converting time domain signals into frequency domain signals and then analyzing the frequency domain signals is an effective method. Frequency domain signals tend to contain more hidden features. And deep learning is used as an artificial intelligence technology and is widely applied to the field of electroencephalogram data analysis.
The technical problem to be solved by the invention is as follows: provides a fatigue state monitoring system based on a portable electroencephalogram acquisition head ring.
Disclosure of Invention
The invention aims to solve the technical problem of providing a fatigue state monitoring system based on a portable electroencephalogram acquisition head ring, which can acquire fatigue of a human body by acquiring electroencephalogram signals and analyzing data so as to remind a user of rest.
The technical scheme adopted by the invention is as follows: a fatigue state monitoring system based on a portable EEG signal acquisition head ring comprises an EEG signal acquisition head ring and EEG signal analysis software which is connected with the EEG signal acquisition head ring through Bluetooth and is arranged at a mobile phone end, wherein the EEG signal acquisition head ring is used for acquiring an EEG signal of a user and transmitting the acquired signal to the EEG signal analysis software on the mobile phone in a wireless manner; the EEG signal analysis software processes and analyzes the received EEG signal and displays the fatigue state of the user in real time through the mobile phone.
The EEG signal acquisition head ring comprises: the EEG signal acquisition device comprises an electrode plate and a connecting device which are sequentially connected, a bioelectric signal acquisition module for amplifying and converting EEG signals, an ESP32 processor for controlling acquisition modes and parameters of the bioelectric signal acquisition module, a Bluetooth wireless transmission circuit for transmitting EEG signals, and a power circuit respectively connected with the bioelectric signal acquisition module and the ESP32 processor, wherein the ESP32 processor is also used for controlling the transmission modes and transmission speeds of the Bluetooth wireless transmission circuit; wherein the content of the first and second substances,
the electrode plate and the connecting device are used for collecting an EEG signal of a forehead region of a user, are connected with the bioelectricity signal collecting module through a flexible flat cable and are used for collecting and transmitting bioelectricity signals;
the bioelectrical signal acquisition module is composed of a bioelectrical signal acquisition chip, the bioelectrical signal acquisition chip is integrated with a high common mode rejection ratio analog input module for receiving voltage signals acquired by an electrode plate, a low-noise programmable gain amplifier for amplifying brain voltage signals and a high-resolution synchronous sampling analog-to-digital converter for converting analog signals into digital signals, the bioelectrical signal acquisition chip adopts differential input, and the common mode rejection ratio is 110 dB;
the Bluetooth wireless transmission circuit obtains the EEG signal output by the bioelectrical signal acquisition module through the ESP32 processor, and transmits the EEG signal data to EEG signal analysis software in a wireless manner.
The input voltage of the power supply circuit is 5V, a lithium battery is used for supplying power, and different working voltages required by the system are provided through the voltage conversion module; or directly using the USB data line for power supply.
The EEG signal acquisition head ring acquires EEG signals of two electrodes including FP1 and FP2 corresponding to a 10-20 system electrode placement method of a user.
The EEG analysis method used by the EEG signal analysis software is a CNN-based convolution recursion fuzzy network deep learning analysis method aiming at fatigue state monitoring, and comprises data preprocessing, CNN and a fuzzy neural network which are sequentially carried out.
The data preprocessing process is as follows:
processing the raw EEG signal using a sixth order Butterworth bandpass filter with a cutoff frequency of 0.1-45 Hz; then, the processed EEG signal is down-sampled from 500Hz to 100 Hz; the down-sampled EEG signal is then segmented into time series segments of 30s duration.
The CNN comprises the following components in series:
(1) a first convolution layer, wherein the convolution kernel size Fs of the first convolution layer is N/100,64 convolution kernels and a moving step size Sd is N/1000; n is the number of samples included in each EEG signal time series segment, i.e., N-30 x 100-3000;
(2) the first batch processing normalization layer is used for normalizing the numerical values of the neurons, so that the data distribution meets the conditions that the mean value is 0 and the variance is 1 and is used for accelerating network convergence;
(3) a second convolutional layer, the number of convolutional kernels of the second convolutional layer is 128, the size of the convolutional kernels is 1 x 1, and the step size is 2;
(4) and the second batch processing normalization layer is used for normalizing the numerical values of the neurons, so that the distribution of the data meets the conditions that the mean value is 0 and the variance is 1. For accelerating network convergence;
(5) a first maximum pooling layer, the pool size being 2 x 2, for reducing the size of the data matrix;
(6) a third convolution layer, the convolution kernel size is 3 x 3, the step size is 2, and the number of convolution kernels is 64;
(7) and a third batch processing normalization layer, which normalizes the numerical values of the neurons to ensure that the distribution of the data meets the conditions that the mean value is 0 and the variance is 1. For accelerating network convergence;
(8) the second largest pooling layer, the pool size 2 x 2, is used to reduce the data matrix size.
The fuzzy neural network structure is as follows:
(1) an input layer: the first layer does not participate in the operation of neurons, only takes the output of the CNN as input, and the features of the EEG signal output by the CNN are described as:
Figure BDA0003495741240000031
the output of the input layer is represented as:
Figure BDA0003495741240000032
wherein x isiThe ith feature vector extracted by the CNN; k is the total number of the characteristic vectors extracted by CNN received by the input layer;
(2) blurring layer: also known as a membership function layer, a gaussian membership function is used to calculate the membership value of the received data output from the input layer, and the membership value of the fuzzy layer output is expressed as:
Figure BDA0003495741240000033
wherein m isijIs the mean of the gaussian membership functions of the jth hidden class of the ith eigenvector input,
Figure BDA0003495741240000034
the variance of the Gaussian membership function of the jth hidden class of the ith input;
(3) a space-active layer: the layer calculates the space activation intensity of each node and provides a membership value for fuzzy processing by utilizing a fuzzy layer to calculate the space activation intensity; the invention uses the improved space activation function to calculate the space activation intensity, and the calculation formula is as follows:
Figure BDA0003495741240000035
wherein the content of the first and second substances,
Figure BDA0003495741240000036
is the spatial activation intensity; fjIs a spatial activation function;
Figure BDA0003495741240000037
membership value output by the fuzzy layer;
(4) a closed-loop layer: the output of the closed-loop layer forms the input of the next layer and also forms the feedback value which is fed back to the input of the closed-loop layer, the output of the closed-loop layer
Figure BDA0003495741240000038
The calculation formula of (a) is as follows:
Figure BDA0003495741240000039
wherein, t represents the time of day,
Figure BDA00034957412400000310
for periodic feedback values, R is the total number of obfuscating rules, wijIn order to be the weight coefficient,
Figure BDA00034957412400000311
for the output of each time step, Fj(t) spatial activation strength for the current time;
(5) a forward layer: nodes in the forward layer are called forward nodes, and the input of the input layer and the output of the closed-loop layer are combined for calculation, wherein the formula is as follows:
Figure BDA00034957412400000312
wherein, wiIs the weight corresponding to the i-th input feature vector, b is the offset value, wjIs the weight corresponding to the jth closed loop layer output, h is the period feedback value;
(6) an output layer: using weighted average deblurring, the output is represented as follows:
Figure BDA00034957412400000313
wherein y is the output of the fuzzy neural network, and also constitutes the output of the CNN-based convolution recursive fuzzy network deep learning analysis method aiming at fatigue state monitoring.
The fatigue state monitoring system based on the portable electroencephalogram acquisition head ring can accurately acquire, effectively identify and correctly classify EEG electroencephalograms, identifies the electroencephalograms of a user by analyzing electroencephalogram data, obtains the brain fatigue degree of the user, classifies the fatigue state of the user, provides reference for the user to know the fatigue health state of the user, prompts the user to pay attention to rest, and avoids accidents caused by fatigue operation.
Drawings
FIG. 1 is a block diagram of the system structure of the fatigue monitoring system based on the portable electroencephalogram acquisition head ring;
FIG. 2 is a block diagram of a portable EEG signal acquisition headring configuration of the present invention;
FIG. 3 is a block diagram of the structure of a Convolutional Neural Network (CNN) for feature extraction in the present invention;
FIG. 4 is a block diagram of the fuzzy neural network with feedback for classification in the present invention.
Detailed Description
The fatigue state monitoring system based on the portable electroencephalogram acquisition head ring is described in detail below with reference to embodiments and accompanying drawings.
As shown in fig. 1, the fatigue state monitoring system based on the portable electroencephalogram acquisition head ring comprises an EEG signal acquisition head ring 1 and EEG signal analysis software 2 connected with the EEG signal acquisition head ring 1 through bluetooth and arranged at a mobile phone end, wherein the EEG signal acquisition head ring 1 is used for acquiring EEG signals of a user and transmitting the acquired signals to the EEG signal analysis software 2 on the mobile phone through wireless; the EEG signal analysis software 2 processes and analyzes the received EEG signal and displays the fatigue state of the user in real time through a mobile phone.
As shown in fig. 2, the EEG signal collecting head ring 1 includes: the EEG signal acquisition device comprises an electrode plate and a connecting device 11 which are sequentially connected, a bioelectric signal acquisition module 12 for amplifying and converting EEG signals, an ESP32 processor 13 for controlling acquisition modes and parameters of the bioelectric signal acquisition module 12, a Bluetooth wireless transmission circuit 14 for transmitting EEG signals, and a power circuit 15 respectively connected with the bioelectric signal acquisition module 12 and the ESP32 processor 13, wherein the ESP32 processor 13 is also used for controlling the transmission modes and transmission speeds of the Bluetooth wireless transmission circuit 14; wherein, the first and the second end of the pipe are connected with each other,
the electrode plate and the connecting device 11 are used for collecting an EEG signal of a forehead region of a user, are connected with the bioelectric signal collecting module 12 through a flexible flat cable and are used for collecting and transmitting bioelectric signals;
the bioelectrical signal acquisition module 12 is formed by adopting an ADS129X series bioelectrical signal acquisition chip, the bioelectrical signal acquisition chip integrates a high common mode rejection ratio analog input module for receiving voltage signals acquired by an electrode plate, a low noise Programmable Gain Amplifier (PGA) for amplifying brain electrical voltage signals and a high resolution synchronous sampling analog-to-digital converter (ADC) for converting analog signals into digital signals, the bioelectrical signal acquisition chip adopts differential input, and the Common Mode Rejection Ratio (CMRR) is 110 dB;
the bluetooth wireless transmission circuit 14 obtains the output EEG signal of the bioelectric signal acquisition module 12 through the ESP32 processor 13, and wirelessly transmits the EEG signal data to the EEG signal analysis software 2.
The input voltage of the power circuit 15 is 5V, a lithium battery is used for supplying power, and different working voltages required by the system are provided through the voltage conversion module; or directly using the USB data line for power supply.
The EEG signal acquisition head ring 1 acquires EEG signals of two electrodes including FP1 and FP2 corresponding to a 10-20 system electrode placement method of a user.
The electroencephalogram analysis method used by the EEG signal analysis software 2 is a CNN-based convolution recursion fuzzy network deep learning analysis method for fatigue state monitoring, and comprises data preprocessing, CNN and a Fuzzy Neural Network (FNN) which are sequentially performed. Wherein the content of the first and second substances,
1) the data preprocessing process is as follows:
processing the raw EEG signal using a sixth order Butterworth bandpass filter with a cutoff frequency of 0.1-45 Hz; then, the processed EEG signal is down-sampled from 500Hz to 100 Hz; the down-sampled EEG signal is then segmented into time series segments of 30s duration.
2) The CNN is shown in fig. 3, and includes, in series, in sequence:
(1) a first convolution layer, wherein the convolution kernel size Fs of the first convolution layer is N/100,64 convolution kernels and a moving step size Sd is N/1000; n is the number of samples included in each EEG signal time series segment, i.e., N-30 x 100-3000;
(2) a first batch normalization layer for accelerating network convergence;
(3) the second convolutional layer, since we used 64 convolutional kernels in the first convolutional layer, the features input to the encoder have 64 feature maps. The number of convolution kernels of the second convolution layer is 128, the size of the convolution kernels is 1 x 1, and the step length is 2;
(4) a second batch normalization layer for accelerating network convergence;
(5) a first maximum pooling layer, the pool size being 2 x 2, for reducing the size of the data matrix;
(6) a third convolution layer, the convolution kernel size is 3 x 3, the step size is 2, and the number of convolution kernels is 64;
(7) a third batch normalization layer for accelerating network convergence;
(8) the second largest pooling layer, pool size 2 x 2, is used to reduce the data matrix size.
3) The fuzzy neural network structure is shown in fig. 4, and includes:
(1) an input layer: the first layer does not participate in the operation of neurons, only takes the output of the CNN as input, and the features of the EEG signal output by the CNN are described as:
Figure BDA0003495741240000051
the output of the input layer is represented as:
Figure BDA0003495741240000052
wherein x isiThe ith feature vector extracted by the CNN; k is the total number of the characteristic vectors extracted by CNN received by the input layer;
(2) blurring layer: also known as a membership function layer, a gaussian membership function is used to calculate the membership value of the received data output from the input layer, and the membership value of the fuzzy layer output is expressed as:
Figure BDA0003495741240000053
wherein m isijIs the mean of the gaussian membership functions of the jth hidden class of the ith eigenvector input,
Figure BDA0003495741240000054
the variance of the Gaussian membership function of the jth hidden class of the ith input;
(3) a space-active layer: the layer calculates the space activation intensity of each node and provides a membership value for fuzzy processing by utilizing a fuzzy layer to calculate the space activation intensity; in a conventional FNN, the spatial activation function uses a continuous cumulative multiplication as a blurring operator, which is expressed as follows:
Figure BDA0003495741240000061
wherein, FjIs a spatial activation function. This conventional approach has drawbacks in processing EEG signals,
Figure BDA0003495741240000062
when n is large, FjThe distribution of a few features tends to deviate greatly from the overall distribution due to the complexity of the brain wave signals even if n is as small as possible, which is undesirable because of the tendency to approach 0 and the small number of features should be used to avoid this situation
To avoid this problem, the present invention uses an improved spatial activation function to calculate the spatial activation strength, the calculation formula is as follows:
Figure BDA0003495741240000063
wherein the content of the first and second substances,
Figure BDA0003495741240000064
is the spatial activation intensity; fjIs a spatial activation function;
Figure BDA0003495741240000065
membership value output by the fuzzy layer;
(4) a closed-loop layer: EEG signals are time-varying and need to take into account not only the state of the brain at a particular moment, but also the persistent effects of previous states, and are therefore introduced in this layer. The output of the closed-loop layer forms the input of the next layer and also forms the feedback value which is fed back to the input of the closed-loop layer, the output of the closed-loop layer
Figure BDA0003495741240000066
The calculation formula of (a) is as follows:
Figure BDA0003495741240000067
wherein, t represents the time of day,
Figure BDA0003495741240000068
is a periodic feedback value, where R is the total number of fuzzification rules, wijIn order to be the weight coefficient,
Figure BDA0003495741240000069
for the output of each time step, Fj(t) spatial activation strength for the current time;
(5) a forward layer: the nodes in the forward layer are called forward nodes, and the input of the input layer and the output of the closed-loop layer are combined and calculated according to the following formula:
Figure BDA00034957412400000610
wherein, wiIs the weight corresponding to the i-th input feature vector, b is the offset value, wjIs the weight corresponding to the jth closed loop layer output, h is the period feedback value;
(6) and (3) an output layer: using weighted average deblurring, the output is represented as follows:
Figure BDA00034957412400000611
wherein y is the output of the fuzzy neural network, and also constitutes the output of the CNN-based convolution recursive fuzzy network deep learning analysis method aiming at fatigue state monitoring.

Claims (7)

1. A fatigue state monitoring system based on a portable EEG signal acquisition head ring is characterized by comprising an EEG signal acquisition head ring (1) and EEG signal analysis software (2) which is connected with the EEG signal acquisition head ring (1) through Bluetooth and is arranged at a mobile phone end, wherein the EEG signal acquisition head ring (1) is used for acquiring an EEG signal of a user and transmitting the acquired signal to the EEG signal analysis software (2) on the mobile phone in a wireless manner; the EEG signal analysis software (2) processes and analyzes the received EEG signal and displays the fatigue state of the user in real time through a mobile phone.
2. The portable electroencephalogram acquisition head ring-based fatigue state monitoring system according to claim 1, characterized in that the EEG signal acquisition head ring (1) comprises: the EEG signal acquisition system comprises an electrode plate, a connecting device (11), a bioelectric signal acquisition module (12), an ESP32 processor (13), a Bluetooth wireless transmission circuit (14) and a power circuit (15), wherein the electrode plate and the connecting device are used for acquiring EEG signals, the bioelectric signal acquisition module (12) is used for amplifying and converting the EEG signals, the ESP32 processor (13) is used for controlling acquisition modes and parameters of the bioelectric signal acquisition module (12), the Bluetooth wireless transmission circuit (14) is used for transmitting the EEG signals, and the power circuit (15) is respectively connected with the bioelectric signal acquisition module (12) and the ESP32 processor (13), and the ESP32 processor (13) is also used for controlling transmission modes and transmission speed of the Bluetooth wireless transmission circuit (14); wherein the content of the first and second substances,
the electrode plate and the connecting device (11) are used for collecting an EEG signal of the forehead region of a user, are connected with the bioelectricity signal collecting module (12) through a flexible flat cable and are used for collecting and transmitting bioelectricity signals;
the bioelectrical signal acquisition module (12) is composed of a bioelectrical signal acquisition chip, the bioelectrical signal acquisition chip is integrated with a high common mode rejection ratio analog input module for receiving voltage signals acquired by an electrode plate, a low-noise programmable gain amplifier for amplifying brain electrical voltage signals and a high-resolution synchronous sampling analog-to-digital converter for converting analog signals into digital signals, the bioelectrical signal acquisition chip adopts differential input, and the common mode rejection ratio is 110 dB;
the Bluetooth wireless transmission circuit (14) obtains the EEG signal output by the bioelectrical signal acquisition module (12) through the ESP32 processor (13), and wirelessly transmits the EEG signal data to the EEG signal analysis software (2).
The input voltage of the power circuit (15) is 5V, a lithium battery is used for supplying power, and different working voltages required by the system are provided through the voltage conversion module; or directly using the USB data line for power supply.
3. The portable electroencephalogram acquisition head ring-based fatigue state monitoring system according to claim 1, characterized in that the EEG signal acquisition head ring (1) acquires EEG signals of two electrodes including FP1 and FP2 corresponding to a 10-20 system electrode placement method of a user.
4. The system for monitoring the fatigue state based on the portable electroencephalogram acquisition head ring according to claim 1, characterized in that the electroencephalogram analysis method used by the EEG signal analysis software (2), namely the CNN-based convolution recursive fuzzy network deep learning analysis method for fatigue state monitoring, comprises data preprocessing, CNN and a fuzzy neural network which are sequentially carried out.
5. The portable electroencephalogram acquisition head ring-based fatigue state monitoring system according to claim 4, wherein the data preprocessing process is as follows:
processing the raw EEG signal using a sixth order Butterworth bandpass filter with a cutoff frequency of 0.1-45 Hz; then, the processed EEG signal is down-sampled from 500Hz to 100 Hz; the down-sampled EEG signal is then segmented into time series segments of 30s duration.
6. The portable electroencephalogram acquisition head ring-based fatigue state monitoring system according to claim 4, wherein the CNN comprises the following components in series:
(1) a first convolution layer, wherein the convolution kernel size Fs of the first convolution layer is N/100,64 convolution kernels and a moving step size Sd is N/1000; n is the number of samples included in each EEG signal time series segment, i.e., N-30 x 100-3000;
(2) the first batch processing normalization layer is used for normalizing the numerical values of the neurons, so that the distribution of data meets the conditions that the mean value is 0 and the variance is 1, and is used for accelerating network convergence;
(3) a second convolutional layer, the number of convolutional kernels of the second convolutional layer is 128, the size of the convolutional kernels is 1 x 1, and the step size is 2;
(4) and the second batch processing normalization layer is used for normalizing the numerical values of the neurons, so that the distribution of the data meets the conditions that the mean value is 0 and the variance is 1. For accelerating network convergence;
(5) a first maximum pooling layer, the pool size being 2 x 2, for reducing the size of the data matrix;
(6) a third convolution layer, the convolution kernel size is 3 x 3, the step size is 2, and the number of convolution kernels is 64;
(7) and a third batch processing normalization layer, which normalizes the numerical values of the neurons to ensure that the distribution of the data meets the conditions that the mean value is 0 and the variance is 1. For accelerating network convergence;
(8) the second largest pooling layer, the pool size 2 x 2, is used to reduce the data matrix size.
7. The portable electroencephalogram acquisition head ring-based fatigue state monitoring system according to claim 4, wherein the fuzzy neural network has the following structure:
(1) an input layer: the first layer does not participate in the operation of neurons, only takes the output of the CNN as input, and the features of the EEG signal output by the CNN are described as:
Figure FDA0003495741230000021
the output of the input layer is represented as:
Figure FDA0003495741230000022
wherein x isiThe ith feature vector extracted by the CNN; k is the total number of the characteristic vectors extracted by CNN received by the input layer;
(2) blurring layer: also known as a membership function layer, a gaussian membership function is used to calculate the membership value of the received data output from the input layer, and the membership value of the fuzzy layer output is expressed as:
Figure FDA0003495741230000023
wherein m isijIs the mean of the gaussian membership functions of the jth hidden class of the ith eigenvector input,
Figure FDA0003495741230000024
a variance of a gaussian membership function of a jth hidden class for an ith input;
(3) a space-active layer: the layer calculates the space activation intensity of each node and provides a membership value for fuzzy processing by utilizing a fuzzy layer to calculate the space activation intensity; the invention uses an improved space activation function to calculate the space activation strength, and the calculation formula is as follows:
Figure FDA0003495741230000025
wherein the content of the first and second substances,
Figure FDA0003495741230000026
is the spatial activation intensity; fjIs a spatial activation function;
Figure FDA0003495741230000027
membership value output by the fuzzy layer;
(4) a closed-loop layer: the output of the closed-loop layer forms the input of the next layer and also forms the feedback value which is fed back to the input of the closed-loop layer, the output of the closed-loop layer
Figure FDA0003495741230000031
The calculation formula of (a) is as follows:
Figure FDA0003495741230000032
wherein, t represents the time of day,
Figure FDA0003495741230000033
for periodic feedback values, R is the total number of obfuscating rules, wijIn order to be the weight coefficient,
Figure FDA0003495741230000034
for the output of each time step, Fj(t) spatial activation strength for the current time;
(5) a forward layer: the nodes in the forward layer are called forward nodes, and the input of the input layer and the output of the closed-loop layer are combined and calculated according to the following formula:
Figure FDA0003495741230000035
wherein, wiIs the weight corresponding to the i-th input feature vector, b is the offset value, wjIs the weight corresponding to the jth closed loop layer output, h is the period feedback value;
(6) an output layer: using weighted average deblurring, the output is represented as follows:
Figure FDA0003495741230000036
wherein y is the output of the fuzzy neural network, and also constitutes the output of the CNN-based convolution recursive fuzzy network deep learning analysis method aiming at fatigue state monitoring.
CN202210114286.0A 2022-01-30 2022-01-30 Fatigue state monitoring system based on portable electroencephalogram acquisition head ring Pending CN114504330A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210114286.0A CN114504330A (en) 2022-01-30 2022-01-30 Fatigue state monitoring system based on portable electroencephalogram acquisition head ring

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210114286.0A CN114504330A (en) 2022-01-30 2022-01-30 Fatigue state monitoring system based on portable electroencephalogram acquisition head ring

Publications (1)

Publication Number Publication Date
CN114504330A true CN114504330A (en) 2022-05-17

Family

ID=81551884

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210114286.0A Pending CN114504330A (en) 2022-01-30 2022-01-30 Fatigue state monitoring system based on portable electroencephalogram acquisition head ring

Country Status (1)

Country Link
CN (1) CN114504330A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115137370A (en) * 2022-06-01 2022-10-04 天津工业大学 Electroencephalogram signal-based fatigue monitoring method and system
CN116628425A (en) * 2023-06-01 2023-08-22 常州易宝网络服务有限公司 Big data real-time monitoring system and method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103767703A (en) * 2014-03-04 2014-05-07 上海海事大学 Portable wireless electroencephalogram data real-time acquisition system and running method thereof
CN205405810U (en) * 2016-02-25 2016-07-27 湖南人文科技学院 Driver fatigue detection alarm system based on brain wave
CN111461206A (en) * 2020-03-30 2020-07-28 华南理工大学 Electroencephalogram-based fatigue detection method with steering wheel embedded with electroencephalogram sensor
CN111543982A (en) * 2020-04-01 2020-08-18 五邑大学 Fatigue driving detection method and device and storage medium
CN111839508A (en) * 2020-07-30 2020-10-30 陕西科技大学 Vehicle safe driving system based on mental state detection and control
US20210081754A1 (en) * 2018-01-08 2021-03-18 Eyesight Mobile Technologies Ltd. Error correction in convolutional neural networks

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103767703A (en) * 2014-03-04 2014-05-07 上海海事大学 Portable wireless electroencephalogram data real-time acquisition system and running method thereof
CN205405810U (en) * 2016-02-25 2016-07-27 湖南人文科技学院 Driver fatigue detection alarm system based on brain wave
US20210081754A1 (en) * 2018-01-08 2021-03-18 Eyesight Mobile Technologies Ltd. Error correction in convolutional neural networks
CN111461206A (en) * 2020-03-30 2020-07-28 华南理工大学 Electroencephalogram-based fatigue detection method with steering wheel embedded with electroencephalogram sensor
CN111543982A (en) * 2020-04-01 2020-08-18 五邑大学 Fatigue driving detection method and device and storage medium
CN111839508A (en) * 2020-07-30 2020-10-30 陕西科技大学 Vehicle safe driving system based on mental state detection and control

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115137370A (en) * 2022-06-01 2022-10-04 天津工业大学 Electroencephalogram signal-based fatigue monitoring method and system
CN116628425A (en) * 2023-06-01 2023-08-22 常州易宝网络服务有限公司 Big data real-time monitoring system and method

Similar Documents

Publication Publication Date Title
CN111616682B (en) Epileptic seizure early warning system based on portable electroencephalogram acquisition equipment and application
CN111631907B (en) Cerebral apoplexy patient hand rehabilitation system based on brain-computer interaction hybrid intelligence
CN111513735B (en) Major depressive disorder identification system based on brain-computer interface and deep learning and application
Jiang et al. IoT-based remote facial expression monitoring system with sEMG signal
CN114504330A (en) Fatigue state monitoring system based on portable electroencephalogram acquisition head ring
CN111616681B (en) Anesthesia state monitoring system based on portable electroencephalogram acquisition equipment and deep learning
CN111631848A (en) Mind control artificial limb system based on brain-computer hybrid intelligence
CN111513991B (en) Active hand full-finger rehabilitation equipment based on artificial intelligence technology
CN113208593A (en) Multi-modal physiological signal emotion classification method based on correlation dynamic fusion
Ahmadi et al. Brain-computer interface signal processing algorithms: A computational cost vs. accuracy analysis for wearable computers
Wang et al. Classification of EEG signal using convolutional neural networks
Gorur et al. Glossokinetic potential based tongue–machine interface for 1-D extraction
CN111493836B (en) Postoperative acute pain prediction system based on brain-computer interface and deep learning and application
CN113871028A (en) Interactive rehabilitation system based on myoelectric intelligent wearing
Lin et al. Design of smart EEG cap
Saini et al. DSCNN-CAU: deep-learning-based mental activity classification for IoT implementation toward portable BCI
ZHANG et al. Classification of imagined speech EEG signals with DWT and SVM
CN114587380A (en) Sleep state detection method, device and system based on electroencephalogram acquisition head ring
CN114242202A (en) Cerebral apoplexy hand rehabilitation training method, device and system
CN111543983B (en) Electroencephalogram signal channel selection method based on neural network
CN114504333B (en) Wearable vestibule monitoring system based on myoelectricity and application
CN113729723B (en) Electrocardiogram signal quality analysis method and device based on ResNet-50 and transfer learning
Zhang et al. The design of seven-lead electrocardiograph monitoring system based on Wi-Fi
CN114504730A (en) Portable brain-controlled hand electrical stimulation rehabilitation system based on deep learning
Bandara et al. Differentiation of signals generated by eye blinks and mouth clenching in a portable brain computer interface system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination