WO2021184599A1 - 一种基于ms-cnn的p300信号识别方法、装置及存储介质 - Google Patents

一种基于ms-cnn的p300信号识别方法、装置及存储介质 Download PDF

Info

Publication number
WO2021184599A1
WO2021184599A1 PCT/CN2020/100343 CN2020100343W WO2021184599A1 WO 2021184599 A1 WO2021184599 A1 WO 2021184599A1 CN 2020100343 W CN2020100343 W CN 2020100343W WO 2021184599 A1 WO2021184599 A1 WO 2021184599A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
cnn
layer
convolution
convolutional layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/100343
Other languages
English (en)
French (fr)
Inventor
王洪涛
裴子安
许林峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuyi University Fujian
Original Assignee
Wuyi University Fujian
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuyi University Fujian filed Critical Wuyi University Fujian
Publication of WO2021184599A1 publication Critical patent/WO2021184599A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • the present invention relates to the field of signal recognition, in particular to a P300 signal recognition method, device and storage medium based on MS-CNN.
  • the brain-computer interface provides non-musculoskeletal control and communication by directly converting brain activities into computer or external equipment information signals. Since the first study proved the feasibility of BCI in using electroencephalogram (EEG) to move graphical objects on computer screens, great efforts have been made to promote the application of this technology in real life, with the ultimate goal of Improve the daily life of users with movement disorders.
  • EEG electroencephalogram
  • the brain-computer interface based on event-related potentials (ERP) is a non-invasive brain-computer interface, which is widely used due to its high reliability.
  • P300 is a decision-related positive waveform about 300ms after receiving a stimulus (visual, auditory, tactile, etc.). It has been repeatedly used in the development of ERP-based BCI system and has proven its usefulness in TV control, virtual keyboard design and BCI Feasibility in speller.
  • the present invention aims to solve at least one of the technical problems existing in the prior art. For this reason, the present invention proposes a P300 signal recognition method based on MS-CNN. Compared with the traditional manual extraction of features, it can obtain features that better characterize general data without excessively relying on training data.
  • the present invention also provides an MS-CNN-based P300 signal recognition device that applies the above-mentioned MS-CNN-based P300 signal recognition method.
  • the present invention also proposes a readable storage medium of a P300 signal recognition device based on MS-CNN using the above-mentioned P300 signal recognition method based on MS-CNN.
  • the MS-CNN-based P300 signal recognition method includes:
  • the MS-CNN network receives cross-subject data and performs feature extraction and classification to establish a cross-subject model
  • the MS-CNN network receives specific subject data and establishes a specific subject model.
  • the MS-CNN-based P300 signal identification method has at least the following beneficial effects: in the process of identifying the P300 signal, first collect the P300 signal, and then perform denoising processing on the collected P300 signal to remove the P300 signal
  • the MS-CNN network is established.
  • the MS-CNN network is a multi-scale convolutional neural network.
  • the convolutional neural network has a strong advantage in processing data and is performing features When extracting, it directly acts on the original data, and automatically performs feature learning layer by layer. Compared with traditional manual extraction of features, it can get features that better characterize general data, and it will not rely too much on training data and use cross-subject data.
  • the cross-subject model has higher generalization and robustness; and on the basis of the established cross-subject model, combined with migration Learning technology can obtain a specific subject model, which can identify target characters based on small samples.
  • performing denoising processing on the collected P300 signal includes:
  • the P300 signal after de-averaging preprocessing is superimposed and averaged.
  • the MS-CNN network includes:
  • Input layer used to load data
  • the first convolutional layer is composed of multiple convolution kernels to remove redundant spatial information and improve the signal-to-noise ratio of the signal;
  • the second convolutional layer is composed of three convolutional layers arranged in parallel. Each convolutional layer contains the same number of convolution kernels, but the size of each convolution kernel is inconsistent, which is used to extract features and increase the complexity of features Spend;
  • the first connection layer is used to superimpose the feature information obtained by the second convolution layer
  • the maximum pooling layer is used to reduce network parameters, speed up calculations, and prevent overfitting of a small number of training samples
  • the third convolution layer is used to perform convolution filtering processing on the features processed by the maximum pooling layer
  • the second connection layer is used to reshape the information processed by the third convolutional layer into a vector.
  • the P300 signal preprocessed by de-averaging is superimposed and averaged, wherein the calculation formula of the superimposed average can be expressed as:
  • x i (t) is the detection signal
  • si (t) is the noise signal
  • n i (t) is the original signal
  • N is the number of times of superposition and averaging.
  • the first convolutional layer is composed of multiple convolution kernels, which are used to remove redundant spatial information and improve the signal-to-noise ratio of the signal.
  • the calculation used by the first convolutional layer The formula can be expressed as:
  • f is the activation function using the corrected linear unit
  • I represents the input data
  • k represents the convolution kernel matrix
  • b represents the additive deviation
  • M j represents the selection of the input mapping.
  • the second convolutional layer is composed of three convolutional layers arranged in parallel, and each convolutional layer contains the same number of convolution kernels, but the size of each convolution kernel is inconsistent , Used to extract features and increase the complexity of features, where the calculation formula of the second convolutional layer using three different scale convolution kernels can be expressed as:
  • the third convolutional layer is used to perform convolution filtering processing on the features processed by the maximum pooling layer, where the calculation formula used by the third convolutional layer can be expressed as:
  • x 5 is the output of the maximum pooling layer
  • x 6 is the output of the third convolutional layer.
  • the MS-CNN-based P300 signal recognition method according to the above-mentioned first aspect of the present invention can be applied.
  • the P300 signal recognition device based on MS-CNN includes:
  • Acquisition unit used to acquire P300 signal
  • Denoising unit used to denoise the collected P300 signal
  • the network establishment unit is used to establish the MS-CNN network and set its network parameters
  • the processing and identification unit is used to control the MS-CNN network to receive cross-subject data and perform feature extraction and classification to establish a cross-subject model; and can control all subjects based on the transfer learning technology and the cross-subject model
  • the MS-CNN network receives specific subject data and establishes a specific subject model.
  • the MS-CNN-based P300 signal recognition device has at least the following beneficial effects: Through the above-mentioned MS-CNN-based P300 signal recognition method, compared with the traditional manual extraction of features, it can get a better characterization of general The characteristics of the data without being overly dependent on the training data.
  • the denoising unit includes:
  • the filter unit is used to perform band-pass filter processing on the collected P300 signal
  • the preprocessing unit is used to perform de-averaging preprocessing on the P300 signal that has undergone band-pass filtering processing;
  • the superposition unit is used to superimpose and average the P300 signal that has been pre-processed by de-averaging.
  • the MS-CNN-based P300 signal identification storage medium of the embodiment of the third aspect of the present invention According to the MS-CNN-based P300 signal identification storage medium of the embodiment of the third aspect of the present invention, the MS-CNN-based P300 signal identification method according to the above-mentioned first aspect of the present invention can be applied.
  • the MS-CNN-based P300 signal recognition storage medium has at least the following beneficial effects: through the above-mentioned MS-CNN-based P300 signal recognition method, it can be better characterized than the traditional manual extraction of features. The characteristics of general data will not be overly dependent on training data.
  • Fig. 1 is a flowchart of a method for identifying P300 signals based on MS-CNN in the first embodiment of the present invention
  • FIG. 2 is a working flow chart of denoising processing in the MS-CNN-based P300 signal recognition method according to the first embodiment of the present invention
  • FIG. 3 is a schematic diagram of the MS-CNN network structure in the P300 signal recognition method based on MS-CNN in the first embodiment of the present invention
  • FIG. 4 is an experimental data diagram of the information transmission rate of the P300 signal recognition method based on MS-CNN in the first embodiment of the present invention
  • Fig. 5 is a schematic structural diagram of a P300 signal recognition device based on MS-CNN according to the second embodiment of the present invention.
  • the stimulation interface in order to excite the P300 potential, is composed of 6 ⁇ 6 characters. All rows and columns of the matrix are flashed continuously and randomly for 175ms. Two of the 12 rows or columns blinking contain the target character (ie a combination of a specific row and a specific column). The response induced by the target rare stimulus is different from the non-target stimulus that does not contain the characteristics of P300.
  • Neusen W device is used to collect scalp EEG signals.
  • EEG recordings come from 64 AgCl electrodes.
  • the EEG reference electrode is Cpz, and the sampling rate is set to 250 Hz.
  • the impedance of all electrodes is kept below 10k ⁇ . Taking into account the needs of migration, 57 channels were selected for further processing, and channels for the public data set were provided.
  • the first embodiment of the present invention provides a P300 signal recognition method based on MS-CNN.
  • One of the embodiments includes but is not limited to the following steps:
  • step S100 the P300 signal is collected.
  • this step first collects the P300 signal, and prepares for the subsequent P300 signal; in this embodiment, a wet motor EEG acquisition device can be used to collect the EEG signal during the P300 experiment, where , EEG data includes P300 and non-P300. In this embodiment, all rows and columns will flash once in each experiment, and the row and column containing the target character will flash once, for a total of two flashes.
  • P300 is 1000 and non-P300 is 5000; for neural networks, classification accuracy has a lot to do with the amount of training data; in order to solve the imbalance problem, we extract P300 at five repetitions to increase the P300 sample In this way, after synthesis, the data sets of P300 and non-P300 are equal, and the total number reaches 10000 (that is, P300 and non-P300 are respectively 5000), which solves the problem of sample imbalance well, for the subsequent training of the MS-CNN neural network be prepared.
  • Step S200 Perform denoising processing on the collected P300 signal.
  • this step performs denoising processing on the collected P300 signal, and removes the interference signal in the P300 signal. EMG and power frequency noise, so it is necessary to remove the collected original P300 signal, thereby improving the signal-to-noise ratio of the signal, in order to be more accurate for subsequent identification.
  • step S300 the MS-CNN network is established and its network parameters are set.
  • the MS-CNN network is established in this step, and its network parameters are set, and multiple convolution kernels of different scales are used to extract features, and the information is diversified in different time periods, which increases the number of distinguishing features. Complexity, while maintaining classification accuracy, it can overcome the problem of low efficiency of model information transmission in the past. And the CNN network has a strong advantage in processing data, and when performing feature extraction, it directly acts on the original data, and automatically performs feature learning layer by layer. Compared with traditional manual feature extraction, it can get a better representation of general data. Features without being overly dependent on training data.
  • step S400 the MS-CNN network receives cross-subject data and performs feature extraction and classification to establish a cross-subject model.
  • this step transmits the cross-subject data to the MS-CNN network, and then uses the MS-CNN network to perform feature extraction and classification processing on the cross-subject data, and convert the recognition result into the corresponding target Characters, feedback results, and the establishment of a cross-subject model is actually the use of public data sets to build a general non-specific subject model, which is more generalized and robust.
  • Step S500 based on the transfer learning technology and the cross-subject model, the MS-CNN network receives specific subject data and establishes a specific subject model.
  • this step uses transfer learning technology and the cross-subject model obtained above to establish a specific subject model on the basis of the cross-subject model obtained; training a deep neural network requires a large number of bands.
  • Labeled data in many cases, the amount of data is not enough to train a complete network.
  • a small amount of labeled data can be used to achieve satisfactory accuracy, which is the principle of transfer learning.
  • transfer learning can be used to adjust existing training networks to solve problems that need to be solved.
  • a common approach is to first train a network on a large data set, then adjust the trained network, and finally apply the adjusted network to actual needs. Fine tuning is usually used to adjust the parameters of a deep network.
  • the migration learning strategy proposed in this embodiment is a fine-tuning strategy based on the general MS-CNN model. Keep the network structure and network parameters, and fine-tune the output layer using the data set of a specific subject. In particular, the parameters of the output layer are initialized with new random values. The back-propagation algorithm was used for 30,000 iterations, and the adaptive moment estimation was used to optimize the network parameters. Through fine-tuning, the powerful generalization ability of the deep neural network helps to avoid complex model design and time-consuming training. The established P300 recognition model of a specific participant can recognize target characters based on a small sample and then give feedback.
  • step S200 of this embodiment may include but is not limited to the following steps:
  • step S210 band-pass filtering is performed on the collected P300 signal.
  • the collected P300 signal is subjected to band-pass filtering to remove interference signals and improve the quality of the brain electrical signals, while avoiding the influence of power frequency interference.
  • Step S220 Perform averaging preprocessing on the P300 signal that has undergone band-pass filtering.
  • this step performs averaging preprocessing on the P300 signal that has undergone band-pass filtering, which also has the effect of removing interference signals and improving the accuracy of signal collection.
  • step S230 the P300 signal that has been pre-processed by de-averaging is superimposed and averaged.
  • the P300 signal that has been pre-processed by de-averaging is superimposed and averaged, thereby improving the signal-to-noise ratio of the P300 signal, and preparing for the subsequent MS-CNN network training, recognition and classification.
  • the MS-CNN network in this embodiment includes: an input layer, used to load the P300 signal to be recognized; a first convolution layer, composed of multiple convolution kernels, used to remove redundant spatial information, and
  • the traditional signal statistical processing methods such as weighted superposition averaging and co-space filtering are similar. This method effectively improves the signal-to-noise ratio of the signal while removing redundant spatial information;
  • the second convolutional layer consists of three parallel convolutions. Layered composition.
  • the number of convolution kernels in each convolutional layer is the same, and the size of each kernel is different. For the same input, convolution kernels of different scales extract different information and increase the complexity of features.
  • the signal in the first convolutional layer is time-filtered, and data features are extracted at different time periods to maximize information;
  • the first connection layer is to map the features extracted from the second convolutional layer with different filter scales Overlay, used to fuse the extracted features;
  • the maximum pooling layer this pooling operation helps to reduce the parameters of the network, thereby speeding up the calculation and preventing overfitting of a small number of training samples;
  • the third convolutional layer it It is a standard general convolutional layer. It uses 10 convolution kernels with a size of 5 to continue to perform convolution filtering operations on the features obtained by the largest pooling layer to extract more abstract, deeper, and more conducive to classification features. At the same time, this method reduces the network parameters of the last complete connection layer; the second connection layer is used to reshape the information processed by the third convolutional layer into a vector.
  • the P300 signal preprocessed by de-averaging is superimposed and averaged, and the calculation formula of superimposed average can be expressed as:
  • x i (t) is the detection signal
  • si (t) is the noise signal
  • n i (t) is the original signal
  • N is the number of times of superimposition and averaging.
  • the signal-to-noise ratio of the signal is improved by the algorithm.
  • the first convolutional layer is composed of multiple convolution kernels to remove redundant spatial information and improve the signal-to-noise ratio of the signal.
  • the calculation formula used by the first convolutional layer can be expressed as :
  • f is the activation function using the corrected linear unit
  • I represents the input data
  • k represents the convolution kernel matrix
  • b represents the additive deviation
  • M j represents the selection of the input mapping.
  • the second convolutional layer is composed of three convolutional layers arranged in parallel, and each convolutional layer contains the same number of convolution kernels, but the size of each convolution kernel is inconsistent, which is used to extract Features and increase the complexity of features.
  • the calculation formula of the second convolution layer using three different scale convolution kernels can be expressed as:
  • the third convolutional layer is used to perform convolution filtering processing on the features processed by the maximum pooling layer, where the calculation formula used by the third convolutional layer can be expressed as:
  • x 5 is the output of the maximum pooling layer
  • x 6 is the output of the third convolutional layer.
  • T refers to the time required for character recognition, and it is directly affected by the number of repetitions.
  • the information obtained from the third convolutional layer is reshaped into a vector x, and the output value of the neuron h w, b (x) can be expressed as:
  • w T represents the weight vector.
  • the output of each row and each column is obtained in the form of probability by the softmax function.
  • the only row and the only column should contain P300, otherwise it will be an incorrect prediction of the target character.
  • the decision strategy of this article is to find the maximum probability of the row and column of P300, as shown in the following formula:
  • r and c represent rows and columns
  • P r and P c represent the probability of P300 forming rows and columns
  • m represents the number of rows and columns.
  • the cross-entropy loss function is used to measure the classification error of the network.
  • the regularization method is used for the first convolutional layer to reduce the risk of overfitting, and the coefficient is set to 0.04.
  • the initial learning rate is 0.01
  • the decay rate is 0.9995
  • the maximum number of iterations is 30,000.
  • the P300 signal is first collected, and then the collected P300 signal is denoised, the interference signal in the P300 signal is removed, and the signal-to-noise ratio of the signal is improved; then the MS- The CNN network and the MS-CNN network are multi-scale convolutional neural networks.
  • Convolutional neural networks have strong advantages in processing data, and when performing feature extraction, they directly act on the original data and automatically perform feature learning layer by layer. Compared with traditional manual extraction of features, it can get features that better characterize general data, and it will not rely too much on training data.
  • cross-subject data to establish a general cross-subject model, that is, non-specific subjects Model, the cross-subject model has higher generalization and robustness; and based on the established cross-subject model, combined with transfer learning technology, a specific subject model can be obtained, which can be based on a small sample Recognize the target character.
  • the second embodiment of the present invention provides a P300 signal recognition device 1000 based on MS-CNN, including:
  • the acquisition unit 1100 is used to acquire P300 signals
  • the denoising unit 1200 is used to denoise the collected P300 signal
  • the network establishment unit 1300 is used to establish the MS-CNN network and set its network parameters
  • the processing recognition unit 1400 is used to control the MS-CNN network to receive cross-subject data and perform feature extraction and classification to establish a cross-subject model; and can control the cross-subject model based on the transfer learning technology and the cross-subject model
  • the MS-CNN network receives specific subject data and establishes a specific subject model.
  • the denoising unit 1200 includes:
  • the filtering unit 1210 is used to perform band-pass filtering processing on the collected P300 signal
  • the preprocessing unit 1220 is configured to perform de-averaging preprocessing on the P300 signal that has undergone band-pass filtering processing;
  • the superimposing unit 1230 is used for superimposing and averaging the P300 signal pre-processed by de-averaging.
  • the processing identification unit 14000 includes:
  • the extraction unit 1410 is configured to perform feature extraction processing on the received data
  • the classification unit 1420 is used to classify the data after feature extraction
  • the model establishment unit 1430 is configured to establish a model based on the classification result. In this embodiment, not only a cross-subject model, but also a specific subject model needs to be established.
  • the acquisition unit 1100 collects the P300 signal, and the denoising unit 1200 denoises the collected P300 signal to remove the interference signal, and then establishes the MS-CNN network through the network establishment unit 1300, and then transmits the data to
  • the processing and recognition unit 1400 performs feature extraction, then classifies, establishes a cross-subject model and a specific subject model respectively, and finally recognizes target characters and gives feedback. Compared with traditional manual feature extraction, it can get a better representation of general data. Features without being overly dependent on training data.
  • the third embodiment of the present invention also provides a P300 signal recognition storage medium based on MS-CNN.
  • the P300 signal recognition storage medium based on MS-CNN stores executable instructions of the P300 signal recognition device based on MS-CNN.
  • the executable instructions of the MS-CNN P300 signal recognition device are executed by one or more control processors, which can cause the above one or more control processors to execute the MS-CNN-based P300 signal recognition method in the first embodiment of the above method, for example , Execute the steps S100 to S500 of the method in FIG. 1 described above to realize the functions of the units 1100-1400 in FIG. 5.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

一种基于MS-CNN的P300信号识别方法、装置及存储介质，包括步骤：采集P300信号(S100)；对采集的P300信号进行去噪处理(S200)；建立MS-CNN网络，并设置其网络参数(S300)；所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型(S400)；基于迁移学习技术和所述跨受试者模型，所述MS-CNN网络接收特定受试者数据，建立特定受试者模型(S500)，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。

Description

一种基于MS-CNN的P300信号识别方法、装置及存储介质 技术领域
本发明涉及信号识别领域，特别涉及一种基于MS-CNN的P300信号识别方法、装置及存储介质。
背景技术
脑机接口(BCI)通过将大脑活动直接转换为计算机或外部设备的信息信号，提供非肌肉骨骼控制和通信。自从第一个研究证明BCI在使用脑电图(EEG)移动计算机屏幕上的图形对象方面的可行性以来，已经做出了很大的努力来推动该技术在现实生活中的应用，最终目的是改善运动障碍用户的日常生活。在不同的脑机接口模式中，基于事件相关电位(ERP)的脑机接口是一种非侵入性脑机接口，以其高可靠性而被广泛应用。特别地，P300是在接收到刺激(视觉、听觉、触觉等)后约300ms的决策相关正波形，已被反复用于基于ERP的BCI系统开发，并证明其在电视控制、虚拟键盘设计和BCI拼写器中的可行性。
在建立P300识别模型时，大多数研究者需要用到大量的数据进行训练，来得到较好的模型，在现实生活中，所得到的训练数据往往是小样本，并不适用于这些大样本模型。基于P300的BCI系统要应用于实际中，不单单只为少数几个人服务，所以跨受试者模型的研究当为重中之重。
发明内容
本发明旨在至少解决现有技术中存在的技术问题之一。为此，本发明提出一种基于MS-CNN的P300信号识别方法，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。
本发明还提出一种应用上述基于MS-CNN的P300信号识别方法的基于MS-CNN的P300信号识别装置。
本发明还提出一种应用上述基于MS-CNN的P300信号识别方法的基于MS-CNN的P300信号识别装置可读存储介质。
根据本发明第一方面实施例的基于MS-CNN的P300信号识别方法，包括：
采集P300信号；
对采集的P300信号进行去噪处理；
建立MS-CNN网络，并设置其网络参数；
所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型；
基于迁移学习技术和所述跨受试者模型，所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
根据本发明实施例的基于MS-CNN的P300信号识别方法，至少具有如下有益效果：在对P300信号进行识别的过程中，首先采集P300信号，然后对采集P300信号进行去噪处理，去除P300信号中的干扰信号，提高信号的信噪比；然后建立MS-CNN网络，MS-CNN网络即为多尺度卷积神经网络，卷积神经网络在处理数据方面有很强的优势，而且在进行特征提取时，直接作用于原始数据，自动逐层进行特征学习，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而且不会过分依赖于训练数据，利用跨受试者数据建立一个通用的跨受试者模型，即非特定受试者模型，跨受试者模型具有更高的泛化性和鲁棒性；并且在建立的跨受试者模型的基础上，结合迁移学习技术，能够得到特定受试者模型，从而能够基于小样本识别目标字符。
根据本发明的一些实施例，对采集的P300信号进行去噪处理，包括：
对采集的P300信号进行带通滤波处理；
对经过带通滤波处理的P300信号进行去均值预处理；
对经过去均值预处理的P300信号进行叠加平均。
根据本发明的一些实施例，所述MS-CNN网络包括：
输入层，用于加载数据；
第一卷积层，由多个卷积核组成，用于去除冗余空间信息，提高信号的信噪比；
第二卷积层，由三个平行排列的卷积层组成，每个卷积层包含的卷积核的数目相同，然而每个卷积核的大小不一致，用于提取特征并且增加特征的复杂度；
第一连接层，用于将由第二卷积层得出的特征信息进行叠加；
最大池化层，用于减少网络参数，加快计算速度，防止少量训练样本的过度拟合；
第三卷积层，用于对经过最大池化层处理的特征进行卷积滤波处理；
第二连接层，用于将经过第三卷积层处理的信息重塑为向量。
根据本发明的一些实施例，所述对经过去均值预处理的P300信号进行叠加平均，其中叠加平均的计算公式可表示为：
Figure PCTCN2020100343-appb-000001
其中，x i(t)为检测信号，s i(t)为噪声信号，n i(t)为原始信号，N为叠加平均的次数。
根据本发明的一些实施例，所述第一卷积层，由多个卷积核组成，用于去除冗余空间信息，提高信号的信噪比，其中，第一卷积层利用到的计算公式可表示为：
Figure PCTCN2020100343-appb-000002
其中，
Figure PCTCN2020100343-appb-000003
代表第一卷积层的第j个特征图，f是使用校正线性单位的激活函数，I代表输入数据，k代表卷积核矩阵，b代表加性偏差，M j代表输入映射的选择。
根据本发明的一些实施例，所述第二卷积层，由三个平行排列的卷积层组成，每个卷积层包含的卷积核的数目相同，然而每个卷积核的大小不一致，用于提取特征并且增加特征的复杂度，其中，第二卷积层利用到三个不同尺度卷积核的计算公式可表示为：
Figure PCTCN2020100343-appb-000004
Figure PCTCN2020100343-appb-000005
Figure PCTCN2020100343-appb-000006
其中，
Figure PCTCN2020100343-appb-000007
和
Figure PCTCN2020100343-appb-000008
表示第二卷积层中不同卷积核的输出映射。
根据本发明的一些实施例，所述第三卷积层，用于对经过最大池化层处理的特征进行卷积滤波处理，其中，第三卷积层利用到的计算公式可表示为：
Figure PCTCN2020100343-appb-000009
其中，x 5为经过最大池化层的输出，x 6为第三卷积层输出。
根据本发明第二方面实施例的基于MS-CNN的P300信号识别装置，能够应用根据本发明上述第一方面实施例的基于MS-CNN的P300信号识别方法。
基于MS-CNN的P300信号识别装置包括：
采集单元，用于采集P300信号；
去噪单元，用于对采集的P300信号进行去噪处理；
网络建立单元，用于建立MS-CNN网络，并设置其网络参数；
处理识别单元，用于控制所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型；并且能够基于迁移学习技术和所述跨受试者模型，控制所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
根据本发明实施例的基于MS-CNN的P300信号识别装置，至少具有如下有益效果：通过上述的基于MS-CNN的P300信号识别方法，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。
根据本发明的一些实施例，所述去噪单元包括：
滤波单元，用于对采集的P300信号进行带通滤波处理；
预处理单元，用于对经过带通滤波处理的P300信号进行去均值预处理；
叠加单元，用于对经过去均值预处理的P300信号进行叠加平均。
根据本发明第三方面实施例的基于MS-CNN的P300信号识别存储介质，能够应用根据本发明上述第一方面实施例的基于MS-CNN的P300信号识别方法。
根据本发明实施例的基于MS-CNN的P300信号识别存储介质，至少具有如下有益效果：通过上述的基于MS-CNN的P300信号识别方法，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。
本发明的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本发明的实践了解到。
附图说明
本发明的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1为本发明实施例一的基于MS-CNN的P300信号识别方法的流程图；
图2为本发明实施例一的基于MS-CNN的P300信号识别方法中的去噪处理的工作流程图；
图3为本发明实施例一的基于MS-CNN的P300信号识别方法中的MS-CNN网络结构示意图；
图4为本发明实施例一的基于MS-CNN的P300信号识别方法的信息传输率的实验数据图；
图5为本发明实施例二的基于MS-CNN的P300信号识别装置的结构示意图。
具体实施方式
下面详细描述本发明的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。
本发明的描述中，除非另有明确的限定，设置、连接等词语应做广义理 解，所属技术领域技术人员可以结合技术方案的具体内容合理确定上述词语在本发明中的具体含义。
实施例一
在本实施例中，为了激发P300电位，刺激界面由6×6个字符组成。该矩阵的所有行和列被连续和随机地闪烁175ms。12行或列中有2行闪烁包含目标字符(即一个特定行和一个特定列的组合)。靶罕见刺激诱发的反应不同于不包含P300特征的非靶刺激。
在数据采集方面，采用Neusen W装置对头皮脑电信号进行采集。根据国际10-20系统，脑电记录来自64个AgCl电极。脑电图参考电极为Cpz，采样率设置为250Hz。所有电极的阻抗都保持在10kΩ以下。考虑到迁移的需要，选择了57个通道进行进一步的处理，并提供的公共数据集的通道。
参照图1，本发明实施例一提供了一种基于MS-CNN的P300信号识别方法，其中的一种实施例包括但不限于以下步骤：
步骤S100，采集P300信号。
在本实施例中，本步骤首先采集P300信号，为了后续的P300信号做好前期准备；在本实施例中，可以利用湿电机脑电采集设备采集受试着P300实验时的脑电信号，其中，脑电数据包括P300和非P300，在本实施例中，每次实验中所有的行和列都会闪烁一次，而包含目标字符的行和列各闪一次，一共闪烁两次。在本实施例中P300为1000和非P300为5000；对于神经网络来说，分类精度与训练数据量有很大关系；为了解决不平衡问题，我们在五个重复次数下提取P300来增加P300样本，这样，合成后P300和非P300的数据集相等，总数达到10000(即P300和非P300分别为5000)，从而很好地解决了样本不平衡的问题，为了后续的MS-CNN神经网络的训练做好准备。
步骤S200，对采集的P300信号进行去噪处理。
在本实施例中，本步骤对采集的P300信号进行去噪处理，将P300信号中的干扰信号进行去除，例如脑电信号在提取时及其容易受到信号的干扰，例如 眼点、心电、肌电和工频噪声，所以需要将采集到的原始的P300信号进行去除，从而提高信号的信噪比，为了后续的识别能够更加准确。
步骤S300，建立MS-CNN网络，并设置其网络参数。
在本实施例中，本步骤建立MS-CNN网络，并且设置其网络参数，采用不同尺度的多个卷积核来提取特征，并在不同的时间段内使信息多样化，增加了判别特征的复杂性，在保持分类精度的同时，可以克服以往模型信息传输效率低的问题。并且CNN网络在处理数据方面有很强的优势，而且在进行特征提取时，直接作用于原始数据，自动逐层进行特征学习，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。
步骤S400，MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型。
在本实施例中，本步骤将跨受试者数据传输到MS-CNN网络中，然后利用MS-CNN网络对跨受试者数据进行特征提取和分类处理，并且将识别结果转化为对应的目标字符，反馈结果，跨受试者模型的建立其实是利用公共数据集建立一个通用的非特定受试者模型，更具泛化性和鲁棒性。
步骤S500，基于迁移学习技术和跨受试者模型，所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
在本实施例中，本步骤利用迁移学习技术和基于上述所得的跨受试者模型，在得到的跨受试者模型的基础上建立一个特定受试者模型；训练一个深度神经网络需要大量带标签的数据，在许多情况下，数据量不足以训练一个完整的网络。然而，当要解决的问题与现有训练网络已经解决的问题相似时，可以使用少量带标签的数据来达到令人满意的精度，这就是迁移学习的原则。启发式地，迁移学习可以用来调整现有的训练网络，以解决需要解决的问题。一种常见的做法是先在一个大数据集上训练一个网络，然后对训练好的网络进行调整，最后将调整后的网络应用到实际需求中。微调通常用于调整深度网络的参数。在本实施例中提出的迁移学习策略是基于通用MS-CNN模型的微调策略。保 留网络结构和网络参数，并使用特定受试者的数据集微调输出层。特别地，用新的随机值初始化输出层的参数。用反向传播算法进行了30000次迭代，使用自适应矩估计优化网络参数，通过微调，深度神经网络强大的泛化能力有助于避免复杂的模型设计和耗时的训练。建立的特定试者的P300识别模型，可以基于小样本识别目标字符，然后进行反馈。
参照图2，本实施例的步骤S200中，可以包括但不限于以下步骤：
步骤S210，对采集的P300信号进行带通滤波处理。
在本实施例中，本步骤将采集到的P300信号进行带通滤波处理，去除干扰信号，提高脑电信号的质量，同时避免了工频干扰的影响。
步骤S220，对经过带通滤波处理的P300信号进行去均值预处理。
在本实施例中，本步骤对经过带通滤波处理的P300信号进行去均值预处理，同样具备去除干扰信号的作用，提高信号采集的精度。
步骤S230，对经过去均值预处理的P300信号进行叠加平均。
在本实施例中，本步骤对经过去均值预处理的P300信号进行叠加平均，从而提高P300信号的信噪比，为后续的MS-CNN网络训练识别分类做好准备。
参照图3，在本实施例的MS-CNN网络包括：输入层，用于加载要识别的P300信号；第一卷积层，由多个卷积核组成，用于去除冗余空间信息，与传统的加权叠加平均、共空间滤波等信号统计处理方法类似，该方法在去除冗余空间信息的同时，有效地提高了信号的信噪比；第二卷积层，由三个平行排列的卷积层组成。每个卷积层的卷积核数目相同，而每个核的大小不同，对于同一输入，不同尺度的卷积核提取不同的信息，增加特征的复杂度，在本实施例中，在不同的时间尺度上对第一卷积层中的信号进行时间滤波，在不同的时间段提取数据特征，使信息最大化；第一连接层，将从第二卷积层不同滤波尺度提取的特征映射进行叠加，用于对提取的特征进行融合；最大池化层，这种池化操作有助于减少网络的参数，从而加快计算速度，防止少量训练样本的过度拟合；第三卷积层，它是一个标准的通用卷积层，利用大小为5的10个卷积核，继续对最大池化层得到的特征进 行卷积滤波运算，提取出更抽象、更深入、更有利于分类的特征，同时，该方法减少了最后一个完整连接层的网络参数；第二连接层，用于将经过第三卷积层处理的信息重塑为向量。
在本实施例中，对经过去均值预处理的P300信号进行叠加平均，其中叠加平均的计算公式可表示为：
Figure PCTCN2020100343-appb-000010
其中，x i(t)为检测信号，s i(t)为噪声信号，n i(t)为原始信号，N为叠加平均的次数，通过算法提高信号的信噪比。
在本实施例中，第一卷积层，由多个卷积核组成，用于去除冗余空间信息，提高信号的信噪比，其中，第一卷积层利用到的计算公式可表示为：
Figure PCTCN2020100343-appb-000011
其中，
Figure PCTCN2020100343-appb-000012
代表第一卷积层的第j个特征图，f是使用校正线性单位的激活函数，I代表输入数据，k代表卷积核矩阵，b代表加性偏差，M j代表输入映射的选择。
在本实施例中，第二卷积层，由三个平行排列的卷积层组成，每个卷积层包含的卷积核的数目相同，然而每个卷积核的大小不一致，用于提取特征并且增加特征的复杂度，其中，第二卷积层利用到三个不同尺度卷积核的计算公式可表示为：
Figure PCTCN2020100343-appb-000013
Figure PCTCN2020100343-appb-000014
Figure PCTCN2020100343-appb-000015
其中，
Figure PCTCN2020100343-appb-000016
和
Figure PCTCN2020100343-appb-000017
表示第二卷积层中不同卷积核的输出映射。
在本实施例中，第三卷积层，用于对经过最大池化层处理的特征进行卷积滤波处理，其中，第三卷积层利用到的计算公式可表示为：
Figure PCTCN2020100343-appb-000018
其中，x 5为经过最大池化层的输出，x 6为第三卷积层输出。
参照图4，在本实施例中，为了评估MS-CNN算法的有效性，需要测量信息传输率，即为ITR，可以运用以下公式：
Figure PCTCN2020100343-appb-000019
其中Q代表目标的个数。P是字符的识别精度。T是指字符识别所需的时间，它直接受重复次数的影响。
其中，在第二连接层将第三卷积层所得的信息重塑为向量x，神经元h w,b(x)的输出值可表示为：
h w,b(x)＝f(w Tx+b)
其中w T代表权重向量。每一行和每一列的输出由softmax函数以概率的形式获得。在本实施例中，在每一轮重复中，所有行和列只闪烁一次，12次闪烁中有2次包含P300。更准确地说，唯一行和唯一列应该包含P300，否则将是对目标字符的错误预测。本文的决策策略是分别求出P300的行和列的最大概率，如下式所示:
r＝argmaxP r(m)(1≤m≤6)
c＝argmaxP c(m)(7≤m≤12)
其中r和c代表行和列，P r和P c代表P300构成行和列的概率，m表示行和 列的数目。一旦确定了包含P300的行和列，就可以正确地预测目标字符。
其中，在本实施例中，利用交叉熵损失函数来衡量网络的分类误差。对第一卷积层采用正则化方法，以降低过拟合的风险，系数设为0.04。用梯度下降优化器训练权值初始学习率为0.01，衰减率为0.9995，最大迭代次数为30000次。
通过上述技术方案可知，在对P300信号进行识别的过程中，首先采集P300信号，然后对采集P300信号进行去噪处理，去除P300信号中的干扰信号，提高信号的信噪比；然后建立MS-CNN网络，MS-CNN网络即为多尺度卷积神经网络，卷积神经网络在处理数据方面有很强的优势，而且在进行特征提取时，直接作用于原始数据，自动逐层进行特征学习，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而且不会过分依赖于训练数据，利用跨受试者数据建立一个通用的跨受试者模型，即非特定受试者模型，跨受试者模型具有更高的泛化性和鲁棒性；并且在建立的跨受试者模型的基础上，结合迁移学习技术，能够得到特定受试者模型，从而能够基于小样本识别目标字符。
实施例二
参照图5，本发明实施例二提供了一种基于MS-CNN的P300信号识别装置1000，包括：
采集单元1100，用于采集P300信号；
去噪单元1200，用于对采集的P300信号进行去噪处理；
网络建立单元1300，用于建立MS-CNN网络，并设置其网络参数；
处理识别单元1400，用于控制所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型；并且能够基于迁移学习技术和所述跨受试者模型，控制所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
需要说明的是，由于本实施例中的基于MS-CNN的P300信号识别装置与上述实施例一中的基于MS-CNN的P300信号识别方法基于相同的发明构思，因此，方法实施例一中的相应内容同样适用于本装置实施例，此处不再详述。
在本实施例中，去噪单元1200包括：
滤波单元1210，用于对采集的P300信号进行带通滤波处理；
预处理单元1220，用于对经过带通滤波处理的P300信号进行去均值预处理；
叠加单元1230，用于对经过去均值预处理的P300信号进行叠加平均。
在本实施例中，处理识别单元14000包括：
提取单元1410，用于对接收到的数据进行特征提取处理；
分类单元1420，用于对进行特征提取后的数据进行分类处理；
模型建立单元1430，用于根据分类结果进行模型建立，本实施例中不仅要建立跨受试者模型，还需要建立特定受试者模型。
通过上述方案可知，采集单元1100对P300信号进行采集，去噪单元1200对采集到的P300信号进行去噪处理，去除干扰信号，然后通过网络建立单元1300建立MS-CNN网络，然后将数据传输到处理识别单元1400进行特征提取，然后分类，分别建立跨受试者模型和特定受试者模型，最后识别目标字符并且进行反馈，比起传统的人工提取特征，可以得到更好地表征一般数据的特征，而不会过分依赖于训练数据。
实施例三
本发明实施例三还提供了一种基于MS-CNN的P300信号识别存储介质，所述基于MS-CNN的P300信号识别存储介质存储有基于MS-CNN的P300信号识别装置可执行指令，该基于MS-CNN的P300信号识别装置可执行指令被一个或多个控制处理器执行，可使得上述一个或多个控制处理器执行上述方法实施例一中的基于MS-CNN的P300信号识别方法，例如，执行以上描述的图1中的方法步骤S100至S500，实现图5中的单元1100-1400的功能。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示意性实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多 个实施例或示例中以合适的方式结合。
尽管已经示出和描述了本发明的实施例，本领域的普通技术人员可以理解：在不脱离本发明的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由权利要求及其等同物限定。

Claims (10)

  1. 一种基于MS-CNN的P300信号识别方法，其特征在于，包括：
    采集P300信号；
    对采集的P300信号进行去噪处理；
    建立MS-CNN网络，并设置其网络参数；
    所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型；
    基于迁移学习技术和所述跨受试者模型，所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
  2. 根据权利要求1所述的一种基于MS-CNN的P300信号识别方法，其特征在于：对采集的P300信号进行去噪处理，包括：
    对采集的P300信号进行带通滤波处理；
    对经过带通滤波处理的P300信号进行去均值预处理；
    对经过去均值预处理的P300信号进行叠加平均。
  3. 根据权利要求1所述的一种基于MS-CNN的P300信号识别方法，其特征在于：所述MS-CNN网络包括：
    输入层，用于加载数据；
    第一卷积层，由多个卷积核组成，用于去除冗余空间信息，提高信号的信噪比；
    第二卷积层，由三个平行排列的卷积层组成，每个卷积层包含的卷积核的数目相同，然而每个卷积核的大小不一致，用于提取特征并且增加特征的复杂度；
    第一连接层，用于将由第二卷积层得出的特征信息进行叠加；
    最大池化层，用于减少网络参数，加快计算速度，防止少量训练样本的过度拟合；
    第三卷积层，用于对经过最大池化层处理的特征进行卷积滤波处理；
    第二连接层，用于将经过第三卷积层处理的信息重塑为向量。
  4. 根据权利要求2所述的一种基于MS-CNN的P300信号识别方法，其特征在于：所述对经过去均值预处理的P300信号进行叠加平均，其中叠加平均的计算 公式可表示为：
    Figure PCTCN2020100343-appb-100001
    其中，x i(t)为检测信号，s i(t)为噪声信号，n i(t)为原始信号，N为叠加平均的次数。
  5. 根据权利要求3所述的一种基于MS-CNN的P300信号识别方法，其特征在于：所述第一卷积层，由多个卷积核组成，用于去除冗余空间信息，提高信号的信噪比，其中，第一卷积层利用到的计算公式可表示为：
    Figure PCTCN2020100343-appb-100002
    其中，
    Figure PCTCN2020100343-appb-100003
    代表第一卷积层的第j个特征图，f是使用校正线性单位的激活函数，I代表输入数据，k代表卷积核矩阵，b代表加性偏差，M j代表输入映射的选择。
  6. 根据权利要求5所述的一种基于MS-CNN的P300信号识别方法，其特征在于：所述第二卷积层，由三个平行排列的卷积层组成，每个卷积层包含的卷积核的数目相同，然而每个卷积核的大小不一致，用于提取特征并且增加特征的复杂度，其中，第二卷积层利用到三个不同尺度卷积核的计算公式可表示为：
    Figure PCTCN2020100343-appb-100004
    Figure PCTCN2020100343-appb-100005
    Figure PCTCN2020100343-appb-100006
    其中，
    Figure PCTCN2020100343-appb-100007
    和
    Figure PCTCN2020100343-appb-100008
    表示第二卷积层中不同卷积核的输出映射。
  7. 根据权利要求6所述的一种基于MS-CNN的P300信号识别方法，其特征在 于：所述第三卷积层，用于对经过最大池化层处理的特征进行卷积滤波处理，其中，第三卷积层利用到的计算公式可表示为：
    Figure PCTCN2020100343-appb-100009
    其中，x 5为经过最大池化层的输出，x 6为第三卷积层输出。
  8. 一种基于MS-CNN的P300信号识别装置，其特征在于：包括：
    采集单元，用于采集P300信号；
    去噪单元，用于对采集的P300信号进行去噪处理；
    网络建立单元，用于建立MS-CNN网络，并设置其网络参数；
    处理识别单元，用于控制所述MS-CNN网络接收跨受试者数据并进行特征提取和分类，建立跨受试者模型；并且能够基于迁移学习技术和所述跨受试者模型，控制所述MS-CNN网络接收特定受试者数据，建立特定受试者模型。
  9. 根据权利要求8所述的一种基于MS-CNN的P300信号识别装置，其特征在于：所述去噪单元包括：
    滤波单元，用于对采集的P300信号进行带通滤波处理；
    预处理单元，用于对经过带通滤波处理的P300信号进行去均值预处理；
    叠加单元，用于对经过去均值预处理的P300信号进行叠加平均。
  10. 一种基于MS-CNN的P300信号识别存储介质，其特征在于：所述基于MS-CNN的P300信号识别存储介质存储有基于MS-CNN的P300信号识别装置可执行指令，基于MS-CNN的P300信号识别装置可执行指令用于使基于MS-CNN的P300信号识别装置执行如权利要求1至7任一所述的基于MS-CNN的P300信号识别方法。
PCT/CN2020/100343 2020-03-18 2020-07-06 一种基于ms-cnn的p300信号识别方法、装置及存储介质 Ceased WO2021184599A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010190967.6 2020-03-18
CN202010190967.6A CN111428601B (zh) 2020-03-18 2020-03-18 一种基于ms-cnn的p300信号识别方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2021184599A1 true WO2021184599A1 (zh) 2021-09-23

Family

ID=71553663

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/100343 Ceased WO2021184599A1 (zh) 2020-03-18 2020-07-06 一种基于ms-cnn的p300信号识别方法、装置及存储介质

Country Status (3)

Country Link
US (1) US20210295135A1 (zh)
CN (1) CN111428601B (zh)
WO (1) WO2021184599A1 (zh)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11507785B2 (en) * 2020-04-30 2022-11-22 Bae Systems Information And Electronic Systems Integration Inc. Anomaly detection system using multi-layer support vector machines and method thereof
CN113017645B (zh) * 2021-02-25 2022-03-29 华南理工大学 一种基于空洞卷积神经网络的p300信号检测方法
CN113647905A (zh) * 2021-07-13 2021-11-16 广东省科学院健康医学研究所 深度神经网络预测血压的训练方法、计算机装置和存储介质
CN114063773B (zh) * 2021-10-25 2025-02-11 华南理工大学 基于卷积神经网络和脑电大数据的跨个体脑机接口系统
CN114492560A (zh) * 2021-12-06 2022-05-13 陕西师范大学 一种基于迁移学习的脑电情绪分类方法
CN114358057A (zh) * 2021-12-16 2022-04-15 华南理工大学 一种跨个体脑电情感识别方法、系统、装置及介质
CN114822520A (zh) * 2022-04-14 2022-07-29 荃豆数字科技有限公司 一种基于语音识别生成中药材订单的方法、装置和设备
CN116340825A (zh) * 2023-03-28 2023-06-27 北京工业大学 一种基于迁移学习的跨被试rsvp脑电信号的分类方法
CN119440250A (zh) * 2024-10-29 2025-02-14 华南师范大学 一种融合眼动追踪与脑电图的意识障碍辅助交流系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160021106A1 (en) * 2007-08-29 2016-01-21 International Business Machines Corporation User authentication via evoked potential in electroencephalographic signals
CN106845401A (zh) * 2017-01-20 2017-06-13 中国科学院合肥物质科学研究院 一种基于多空间卷积神经网络的害虫图像识别方法
CN109247917A (zh) * 2018-11-21 2019-01-22 广州大学 一种空间听觉诱发p300脑电信号识别方法及装置
CN109389059A (zh) * 2018-09-26 2019-02-26 华南理工大学 一种基于cnn-lstm网络的p300检测方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120123232A1 (en) * 2008-12-16 2012-05-17 Kayvan Najarian Method and apparatus for determining heart rate variability using wavelet transformation
CN108960182B (zh) * 2018-07-19 2021-11-05 大连理工大学 一种基于深度学习的p300事件相关电位分类识别方法
CN110059565A (zh) * 2019-03-20 2019-07-26 杭州电子科技大学 一种基于改进卷积神经网络的p300脑电信号识别方法
US10646156B1 (en) * 2019-06-14 2020-05-12 Cycle Clarity, LLC Adaptive image processing in assisted reproductive imaging modalities

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160021106A1 (en) * 2007-08-29 2016-01-21 International Business Machines Corporation User authentication via evoked potential in electroencephalographic signals
CN106845401A (zh) * 2017-01-20 2017-06-13 中国科学院合肥物质科学研究院 一种基于多空间卷积神经网络的害虫图像识别方法
CN109389059A (zh) * 2018-09-26 2019-02-26 华南理工大学 一种基于cnn-lstm网络的p300检测方法
CN109247917A (zh) * 2018-11-21 2019-01-22 广州大学 一种空间听觉诱发p300脑电信号识别方法及装置

Also Published As

Publication number Publication date
CN111428601A (zh) 2020-07-17
CN111428601B (zh) 2023-08-18
US20210295135A1 (en) 2021-09-23

Similar Documents

Publication Publication Date Title
WO2021184599A1 (zh) 一种基于ms-cnn的p300信号识别方法、装置及存储介质
CN114266276B (zh) 一种基于通道注意力和多尺度时域卷积的运动想象脑电信号分类方法
CN113180692B (zh) 一种基于特征融合和注意力机制的脑电信号分类识别方法
CN109389059B (zh) 一种基于cnn-lstm网络的p300检测方法
CN110876626B (zh) 基于多导联脑电最优导联选择的抑郁检测系统
CN110069958B (zh) 一种密集深度卷积神经网络的脑电信号快速识别方法
CN108304917B (zh) 一种基于lstm网络的p300信号检测方法
CN113017645B (zh) 一种基于空洞卷积神经网络的p300信号检测方法
EP3933551A1 (en) Motor imagery electroencephalogram signal processing method, device, and storage medium
CN108960182B (zh) 一种基于深度学习的p300事件相关电位分类识别方法
CN110070105B (zh) 基于元学习实例快速筛选的脑电情绪识别方法、系统
CN110059565A (zh) 一种基于改进卷积神经网络的p300脑电信号识别方法
CN111584029B (zh) 基于判别式对抗网络的脑电自适应模型及在康复中的应用
CN110353702A (zh) 一种基于浅层卷积神经网络的情感识别方法及系统
CN113065526B (zh) 一种基于改进深度残差分组卷积网络的脑电信号分类方法
CN113128552A (zh) 一种基于深度可分离因果图卷积网络的脑电情绪识别方法
CN116522106A (zh) 一种基于迁移学习并行多尺度滤波器组时域卷积的运动想象脑电信号分类方法
Wan et al. EEG fading data classification based on improved manifold learning with adaptive neighborhood selection
CN116127286B (zh) 基于轻量级残差卷积胶囊网络的脑电情感识别检测方法及其系统
CN114139573B (zh) 一种基于脑电信号类多光谱图像序列的身份识别方法
CN111091074A (zh) 最优区域共空间模式的运动想象脑电信号分类方法
CN116524380A (zh) 一种基于脑-机信号融合的目标检测方法
CN114997315B (zh) 一种基于多通道脑电图集成的错误相关电位分类方法
CN113143261A (zh) 基于肌电信号的身份识别系统、方法及设备
CN114118176A (zh) 基于解耦表示学习的连续快速视觉演示脑电信号分类方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20926303

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20926303

Country of ref document: EP

Kind code of ref document: A1