CN113208632A

CN113208632A - Attention detection method and system based on convolutional neural network

Info

Publication number: CN113208632A
Application number: CN202110372656.6A
Authority: CN
Inventors: 马鹏程; 卢树强; 王晓岸
Original assignee: Beijing Brain Up Technology Co ltd
Current assignee: Beijing Brain Up Technology Co ltd
Priority date: 2021-04-07
Filing date: 2021-04-07
Publication date: 2021-08-06

Abstract

The invention discloses an attention detection method and system based on a convolutional neural network, wherein the method comprises the steps that EEG signal acquisition equipment acquires an EEG signal of a user, and the EEG signal is transmitted to a data analysis system after being subjected to filtering pretreatment; the data analysis system extracts the features of the EEG signal according to task requirements and transmits the features to the judgment and identification system; the judgment and identification system identifies the attention state of the user through a convolutional neural network algorithm. The method accurately detects whether attention is concentrated or not through the convolutional neural network algorithm, improves the analysis efficiency and accuracy, and widens the application scene of detection.

Description

Attention detection method and system based on convolutional neural network

Technical Field

The invention relates to the technical field of EEG signal identification, in particular to an attention detection method and system based on a convolutional neural network.

Background

The attention detection method commonly used in the market at present is a complex experiment, such as making a large number of mathematic questions and watching movie and television videos by a testee, then performing questionnaire survey and statistical analysis, and analyzing whether attention is focused or not according to results, so that certain subjectivity exists. The method for detecting attention is complex to operate, low in efficiency, large in error and limited in application scene.

Therefore, how to improve the identification accuracy is an urgent problem to be solved.

Disclosure of Invention

The invention aims to provide an attention detection method and system based on a convolutional neural network, so as to accurately detect whether attention is concentrated or not, improve the detection efficiency and accuracy and widen the application scene of detection.

In order to solve the above technical problem, the present invention provides an attention detection method based on a convolutional neural network, including:

the EEG signal acquisition equipment acquires an EEG signal of a user, and the EEG signal is transmitted to the data analysis system after being subjected to filtering pretreatment;

the data analysis system extracts the features of the EEG signal according to task requirements and transmits the features to the judgment and identification system;

the judgment and identification system identifies the attention state of the user through a convolutional neural network algorithm.

Preferably, the filtering pretreatment comprises power frequency filtering and band-pass filtering.

Preferably, after the preprocessing step, the method further includes performing data normalization operation on the signals to ensure consistency of the signals.

Preferably, the features of the EEG signal comprise time domain features, frequency domain features, time-frequency features of the EEG signal.

Preferably, the time domain features include mean, variance and first order difference features.

Preferably, the frequency domain features are power energy values of the EEG signal at different frequency bands.

Preferably, the time-frequency features of the EEG signal are calculated using short-time fourier transform, STFT, and wavelet transform methods.

The invention also provides an attention detection system based on the convolutional neural network, which is used for realizing the method and comprises the following steps:

the EEG signal acquisition equipment is used for acquiring an EEG signal of a user, filtering and preprocessing the EEG signal and transmitting the EEG signal to the data analysis system;

the data analysis system is used for extracting the features of the EEG signals according to task requirements and transmitting the features to the judgment and identification system;

and the judgment and identification system is used for identifying the attention state of the user through a convolutional neural network algorithm.

According to the attention detection method and system based on the convolutional neural network, the EEG electroencephalogram signals of a person are obtained in real time, the frequency spectrum energy is used as the characteristic, and whether the attention is concentrated or not is accurately detected by combining the convolutional neural network algorithm, so that the analysis efficiency and accuracy are improved, and the application scene of detection is widened.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

FIG. 1 is a flow chart of an attention detection method based on a convolutional neural network according to the present invention;

FIG. 2 is an EEG waveform in a state of concentration;

FIG. 3 is a waveform of an EEG under inattentive conditions

FIG. 4 is a schematic diagram of the EEG signal frequency energy distribution in the state of concentration provided by the present invention;

FIG. 5 is a schematic diagram of the frequency energy distribution of an EEG signal in an inattentive condition provided by the present invention;

fig. 6 is a schematic structural diagram of an attention detection system based on a convolutional neural network according to the present invention.

Detailed Description

The core of the invention is to provide an attention detection method and system based on a convolutional neural network, so as to realize intelligent analysis of electroencephalogram signals, judge attention states, improve recognition accuracy and reduce recognition time.

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, fig. 1 is a flowchart illustrating an attention detection method based on a convolutional neural network according to the present invention. With reference to fig. 1, the attention detection method adopted in this embodiment specifically includes the following steps:

the method comprises the following steps that 1, EEG signal acquisition equipment acquires an EEG signal of a user, and the EEG signal is transmitted to a data analysis system after being subjected to filtering pretreatment;

in the step of acquiring EEG signals, the device adopted in this embodiment is preferably a portable BCI device, belongs to a few-channel high-precision EEG acquisition device, and acquires high-quality EEG signals from the forehead through electrodes on the premise of satisfying the quality of the acquired signals. The equipment has the characteristics of portability, signal quality stability and the like.

Electroencephalographic signals are specific electrical discharge activity of the human brain, and EEG electroencephalography is a commonly used analytical method. EEG has the advantages of convenience in acquisition mode, stability of signals, lower cost and the like. EEG signals may be divided into delta, theta, alpha, beta, and gamma band signals according to frequency distribution. When the attention of a person is very focused, the phenomenon of activity enhancement in the EEG signal at intermediate frequencies (alpha and beta waves) can be observed. By using the portable EEG acquisition equipment, the EEG signals of a person can be acquired in real time, and the power values of the EEG signals in different frequency bands and the ratios of different frequency band energies are calculated. And (4) accurately detecting whether attention is concentrated or not by using the spectral energy as a characteristic through a machine learning algorithm. FIG. 2 is an EEG waveform in a state of concentration; FIG. 3 is a waveform of an EEG in an inattentive condition.

In the preprocessing step, power frequency filtering and band-pass filtering are carried out on the original signal. Because a large amount of power frequency interference noise exists in the original EEG signal, a 50Hz notch filter can be designed in the embodiment to remove power frequency interference, the signal is subjected to 0.5-40Hz band-pass filtering, and the eye electrical artifact is removed by using independent component analysis. By the above pre-processing method irrelevant useless information in the EEG signal is removed, leaving a clean signal relevant to the experimental task.

Because the EEG signals have individual differences, further, the present embodiment may further increase the data normalization operation to ensure the consistency of the signals.

2. The data analysis system extracts the features of the EEG signal according to task requirements and transmits the features to the judgment and identification system;

the EEG feature extraction method comprises the following steps: (1) time domain characteristics, (2) frequency domain characteristics, (3) time-frequency characteristics, and the like. According to the embodiment, the EEG signal characteristics are extracted by selecting a suitable method according to task requirements.

The time domain characteristics of the signals adopt characteristics such as mean, variance and first-order difference; calculating the frequency domain characteristics of the signal, and calculating the power energy values of the signal in delta (0.5-4Hz), theta (4-8Hz), alpha (8-14Hz), beta (14-30Hz) and gamma (30-50Hz) frequency bands by using a Fast Fourier Transform (FFT) algorithm. The time-frequency characteristics of the calculated signals are calculated by adopting a short-time Fourier transform (STFT) algorithm and a wavelet transform method. According to the characteristics of the attention task, the power energy values of the signals in different frequency bands are calculated.

According to the characteristics of the computed electroencephalogram signals on the frequency domain, it can be found that when people concentrate on attention, the power energy of delta and theta low frequency bands is low, and correspondingly, the power energy of beta, gamma and other high frequency bands is high, as shown in fig. 4; while the person is inattentive, the delta and theta low bands of power energy increase and correspondingly the beta and gamma high bands of power energy decrease, as shown in FIG. 5.

Therefore, the frequency domain characteristics of the electroencephalogram signals are obviously distinguished, and the attention focusing state and the attention non-focusing state can be judged through subsequent further processing.

3. The judgment and identification system identifies the attention state of the user through a convolutional neural network algorithm.

Among them, neural networks (CNNs) are part of the field of artificial intelligence research, and the most popular neural network currently is deep Convolutional Neural Network (CNNs). CNNs are currently enjoying tremendous success in many areas of research, such as speech recognition, image segmentation, natural language processing, etc. CNNs can automatically learn features from (usually large-scale) data and generalize the results to the same type of unknown data.

The convolutional neural network is composed of three structures of convolution (convolution), activation (activation), and pooling (displacement). The result of the CNN output is a specific feature space for each image. When processing an image classification task, the feature space output by the CNN is used as an input of a fully connected layer or a fully connected neural network (FCN), and the fully connected layer is used to complete mapping from the input image to the tag set, i.e., classification, to obtain a result of determining whether the user is tired or not. This embodiment outputs two determination results of "concentration" and "non-concentration". The method comprises the steps of preprocessing an original EEG signal by artifact removal, normalizing data, calculating characteristics, and then directly sending the characteristics into a CNN network for classification.

The most important of the above steps is how to iteratively adjust the network weights by the training data, i.e. the back propagation algorithm. In addition to the CNN algorithm adopted in this embodiment, the present mainstream Convolutional Neural Networks (CNNs), such as VGG, ResNet, etc., may also be applied, and these algorithms are all adjusted and combined by simple CNN.

According to the method, the EEG electroencephalogram signals of the human are acquired in real time, the spectrum energy is used as the characteristic, and the convolutional neural network algorithm is combined to accurately detect whether the attention is concentrated or not, so that the analysis efficiency and accuracy are improved, and the application scene of detection is widened. The method uses a special portable few-channel forehead electrode electroencephalogram acquisition device, judges whether attention is concentrated or not by calculating power values of different frequency bands, can acquire the attention state of a person in real time, and timely makes judgment and early warning, is suitable for various learning, working and living scenes, ensures the real-time performance and safety of data transmission, improves the accuracy of signal analysis, ensures the accuracy of classification results by using an efficient machine learning algorithm, and has diversity in application scenes.

Referring to fig. 6, fig. 6 is a schematic structural diagram of an attention detection system based on a convolutional neural network according to the present invention, the system is used for implementing the above method, and includes:

the EEG signal acquisition equipment 101 is used for acquiring an EEG signal of a user, filtering and preprocessing the EEG signal and transmitting the processed EEG signal to a data analysis system;

the data analysis system 102 is used for extracting the features of the EEG signals according to task requirements and transmitting the features to the judgment and identification system;

and the judgment and identification system 103 is used for identifying the attention state of the user through a convolutional neural network algorithm.

Therefore, the system accurately detects whether attention is concentrated or not by acquiring the electroencephalogram signals of a person in real time, taking the frequency spectrum energy as the characteristic and combining the convolutional neural network algorithm, improves the analysis efficiency and accuracy, and widens the application scene of detection.

For the introduction of the attention detection system based on the convolutional neural network provided by the present invention, please refer to the aforementioned embodiment of the attention detection method based on the convolutional neural network, and the embodiments of the present invention are not described herein again. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The attention detection method and system based on the convolutional neural network provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.

Claims

1. An attention detection method based on a convolutional neural network, comprising:

2. The convolutional neural network-based attention detection method of claim 1, wherein said filtering preprocessing comprises power frequency filtering and band pass filtering.

3. The convolutional neural network-based attention detection method of claim 1, wherein said preprocessing step is followed by a data normalization operation on the signal to ensure signal consistency.

4. The convolutional neural network-based attention detection method of claim 1, wherein the features of the EEG signal comprise time domain features, frequency domain features, time-frequency features of the EEG signal.

5. The convolutional neural network-based attention detection method of claim 4, wherein the time-domain features include mean, variance, and first-order difference features.

6. The convolutional neural network-based attention detection method of claim 4, wherein said frequency domain feature is the power energy value of EEG signal at different frequency bands.

7. The convolutional neural network-based attention detection method of claim 4, wherein the time-frequency features of the EEG signal are calculated using a short-time Fourier transform (STFT) and a wavelet transform method.

8. An attention detection system based on a convolutional neural network for implementing the method of any one of claims 1 to 7, comprising: