CN116400800B - ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm - Google Patents
ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm Download PDFInfo
- Publication number
- CN116400800B CN116400800B CN202310236983.8A CN202310236983A CN116400800B CN 116400800 B CN116400800 B CN 116400800B CN 202310236983 A CN202310236983 A CN 202310236983A CN 116400800 B CN116400800 B CN 116400800B
- Authority
- CN
- China
- Prior art keywords
- brain
- signal
- ssvep
- computer
- bci
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 77
- 206010002026 amyotrophic lateral sclerosis Diseases 0.000 title claims abstract description 36
- 230000003993 interaction Effects 0.000 title claims abstract description 31
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000002452 interceptive effect Effects 0.000 claims abstract description 34
- 210000004556 brain Anatomy 0.000 claims abstract description 32
- 238000012549 training Methods 0.000 claims abstract description 22
- 210000005036 nerve Anatomy 0.000 claims abstract description 14
- 238000007405 data analysis Methods 0.000 claims abstract description 7
- 230000000638 stimulation Effects 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 10
- 230000008054 signal transmission Effects 0.000 claims description 9
- 238000013136 deep learning model Methods 0.000 claims description 7
- 230000006698 induction Effects 0.000 claims description 6
- 210000001061 forehead Anatomy 0.000 claims description 5
- 238000010801 machine learning Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 238000013101 initial test Methods 0.000 claims description 4
- 238000012821 model calculation Methods 0.000 claims description 4
- 210000000869 occipital lobe Anatomy 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 238000010183 spectrum analysis Methods 0.000 claims description 4
- 210000003478 temporal lobe Anatomy 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000007635 classification algorithm Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 claims description 3
- 210000001652 frontal lobe Anatomy 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000005065 mining Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 210000001152 parietal lobe Anatomy 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 208000002173 dizziness Diseases 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000000763 evoking effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention relates to an ALS patient human-computer interaction system and method based on a brain-computer interface and an artificial intelligence algorithm, wherein the system comprises brain-computer interface BCI signal acquisition equipment for acquiring nerve brain electrical signals of an ALS patient; the effective channel signal selection module is used for digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment to an ALS patient; the self-adaptive scale adjustment coding module is used for adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users; and the AI recognition algorithm module is used for performing model training to form an SSVEP signal recognition artificial intelligent algorithm model by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set. The invention is based on portable BCI equipment, combines a signal data analysis algorithm and an artificial intelligence algorithm to automatically extract and identify the brain signal characteristics of the user, efficiently carries out interactive signal coding, improves the interactive efficiency, and widens the interactive application scene of the brain-computer interface.
Description
Technical Field
The invention relates to the technical field of intelligent interaction of brain science EEG, brain neuroscience and artificial intelligent algorithm and man-machine interaction technology, in particular to an ALS patient man-machine interaction system and method based on a brain-machine interface and an artificial intelligent algorithm.
Background
The existing partial interaction technology is based on an electroencephalogram signal SSVEP (visual steady-state evoked signal) as a display marker. The signal stimulation quality is low, the frequency coding is unstable, dizziness is easy to cause, and the signal can not be used for a long time, so that the design can effectively solve the problems;
the signals through the SSVEP are associated with the corresponding letters after being subjected to artificial feature extraction and coding, so that simple typing display is realized. At present, the existing research and development scheme is used for setting the artificial characteristics of brain signals obtained by an SSVEP (single-phase-contrast) model, so that the workload is complex, the characteristic indexes are not uniform, the effectiveness is unstable, and the recognition efficiency is high. The existing SSVEP induced typing system needs to perform long-time data acquisition and scheme correction on a single individual, and greatly improves the use difficulty.
Disclosure of Invention
In view of the above, the invention aims to overcome the defects of the prior art, and provides an ALS patient human-computer interaction system and method based on a brain-computer interface and an artificial intelligence algorithm, which are based on portable BCI equipment, automatically extract and identify the brain signal characteristics of a user by combining a signal data analysis algorithm and the artificial intelligence algorithm, efficiently perform interaction signal coding, improve interaction efficiency and widen application scenes of interaction of the brain-computer interface.
To achieve the above object, a first aspect of the present invention provides an ALS patient human-computer interaction system based on a brain-computer interface and an artificial intelligence algorithm, comprising at least
The brain-computer interface BCI signal acquisition equipment is used for acquiring nerve brain electrical signals of an ALS patient;
the effective channel signal selection module is used for digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment to an ALS patient;
the self-adaptive scale adjustment coding module is used for adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users;
the AI recognition algorithm module is used for performing model training to form an SSVEP signal recognition artificial intelligent algorithm model by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set;
and the interactive display system is provided with an SSVEP signal recognition artificial intelligent algorithm model and is used for performing visual stimulation, signal transmission and instruction display.
The second aspect of the invention provides an ALS patient man-machine interaction method based on a brain-computer interface and an artificial intelligence algorithm, wherein the interaction method adopts the interaction system and comprises the following steps:
s1, acquiring nerve brain-brain electrical signals of an ALS patient by using brain-computer interface (BCI) signal acquisition equipment;
s2, digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment on an ALS patient;
s3, adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users;
s4, performing model training by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set to form an SSVEP signal identification artificial intelligent algorithm model;
s5, carrying an SSVEP signal recognition artificial intelligent algorithm model to perform visual stimulation and signal transmission and instruction display.
Further, in step S1, the brain-computer interface BCI signal acquisition device acquires nerve brain electrical signals of (0-1, a-z) under different expression states of the user from the forehead lobe 7 channel, the temporal lobe 6 channel, the occipital lobe 7 channel, and the parietal lobe 6 channel.
Further, in step S2, by mining the brain nerve electrical signal frequency domain and time signal characteristic analysis, an effective contact channel of the brain-computer interface BCI signal acquisition device to the user is mined, the acquired brain electrical signal is amplified and digital-to-analog converted and encoded, and transmitted to the data analysis system in a wired or wireless connection manner.
Further, after channel selection is based on concurrent signal acquisition, an instruction induction experiment paradigm is set, effective stimulation related event signals are acquired and observed, the acquisition quality, precision and consistency of the signals of all channels are counted, and optimal communication calculation channels are selected; and amplifying the effective channel signal and transmitting codes at the sensor circuit end.
Further, in step S3, an additional SSEVP paradigm is set, the command signal is collected, and the data of the collected signal is enhanced, so as to obtain an initial-test-induced associated command signal; and carrying out uniform scale calculation on various instructions corresponding to each channel to finish self-adaptive scale adjustment coding.
Further, based on the optimal SSVEP coding scheme and BCI effective channel signal correlation for individuals, acquiring electroencephalogram signal data corresponding to different letters A-Z, and storing and classifying for label and data spectrum analysis;
setting SSVEP schemes in different forms, carrying out a single signal source stimulation output test, analyzing response states under the different SSVEP schemes, and selecting an optimal SSVEP scheme;
and (3) collecting the optimal channel and the optimal SSVEP paradigm, constructing a data instruction set, and completing the collection of different instruction data sets of 0-9 and A-Z groups.
Further, in step S4, the method further comprises
After the acquisition of the data set is completed, signal preprocessing calculation is carried out, and the method mainly comprises the following steps: filtering the original data by adopting a filtering algorithm, and filtering high-frequency and low-frequency artifacts, power frequency interference, electrooculogram and other noises in the original data to obtain pure electroencephalogram signals; fourier transform or wavelet transform is carried out on frontal lobe brain signals, energy value relative ratios of different frequencies are calculated, and effective channel signals are identified;
sending the preprocessed electroencephalogram characteristic values into an algorithm model to perform artificial intelligence model to perform multi-classification algorithm training, and detecting semantics of different instruction states of a user; in this step, the algorithm model employed may be a conventional machine learning Model (ML) and a Deep Learning (DL) model; after model training is completed, a large-scale test scene is constructed for evaluation, and interactive configuration application can be performed when the blind test accuracy reaches more than 90%.
Further, in step S5, based on the physical characteristics of the ALS patient, an interactive display and control interface is set to ensure that visual stimulation, signal transmission and instruction display can be effectively performed;
1) Carrying an interactive interface in a region where the vision of a user can be reached, wherein the interactive interface mainly presents instruction output display and monitoring the change state of the brain signals of an individual;
2) The brain instruction signal induction and enhancement stimulator is carried in an auxiliary mode, the signal intensity of the user idea instruction is improved, and therefore model calculation robustness is provided;
3) And carrying the trained SSVEP signal recognition artificial intelligent algorithm model on an interactive display system to realize intelligent idea interactive application of the whole flow.
Further, the trained model is deployed on the whole interactive system, the data instruction features of the user are transmitted back to the background database and are incorporated into the algorithm model for automatic correction, and the accuracy of the algorithm model for identifying the individual instructions is continuously improved based on the reinforcement learning mechanism.
The invention adopts the technical proposal and has at least the following beneficial effects:
1. the invention can automatically detect the signal characteristics of the user in real time through an artificial intelligent algorithm, and efficiently detect the signals and calculate the characteristics;
2. the invention can excavate the effective signal channel position based on the deep learning model based on the portable brain-computer interface equipment, automatically correct the scheme by prefabricating the trained recognition model, and is convenient for users to use;
3. the invention is based on portable BCI equipment, combines a signal data analysis algorithm and an artificial intelligence algorithm to automatically extract and identify the brain signal characteristics of the user, efficiently carries out interactive signal coding, improves the interactive efficiency, and widens the interactive application scene of the brain-computer interface.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the structure of a brain-computer interface BCI signal acquisition device of the present invention;
FIG. 2 is a flow chart of the effective channel data mining analysis of the individual BCI devices of the present invention;
FIG. 3 is a schematic diagram of the data processing of the acquired data in accordance with the present invention;
FIG. 4 is a schematic diagram of the collection of interaction instruction data sets and AI model training of the large-scale different user BCI equipment of the invention;
fig. 5 is a schematic diagram of an ALS patient utilizing an interactive display system.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, based on the examples herein, which are within the scope of the invention as defined by the claims, will be within the scope of the invention as defined by the claims.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
The embodiment provides an ALS patient human-computer interaction system based on a brain-computer interface and an artificial intelligence algorithm, which at least comprises
The brain-computer interface BCI signal acquisition equipment is used for acquiring nerve brain electrical signals of an ALS patient;
the effective channel signal selection module is used for digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment to an ALS patient;
the self-adaptive scale adjustment coding module is used for adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users;
the AI recognition algorithm module is used for performing model training to form an SSVEP signal recognition artificial intelligent algorithm model by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set;
and the interactive display system is provided with an SSVEP signal recognition artificial intelligent algorithm model and is used for performing visual stimulation, signal transmission and instruction display.
As shown in fig. 1, the BCI signal acquisition device is a schematic diagram of the BCI signal acquisition device with a multi-channel (forehead lobe 7 channel, temporal lobe 6 channel, occipital lobe 7 channel, top lobe 6 channel) and high-precision wearable portable BCI (brain-computer interface) for acquiring nerve brain electrical signals of (0-1, a-z) under different expression states of a user.
Brain-computer sensing device features: the BCI equipment electrodes are dry electrodes, are mainly distributed in the whole brain region, are symmetrically distributed in a plurality of electrode point positions, and have a single channel electrode 1K-20K sampling rate, so that the accurate force sampling and signal recognition calculation of brain electrical signals can be met.
The second aspect of the invention provides an ALS patient man-machine interaction method based on a brain-computer interface and an artificial intelligence algorithm, wherein the interaction method adopts the interaction system and comprises the following steps:
s1, acquiring nerve brain-brain electrical signals of an ALS patient by using brain-computer interface (BCI) signal acquisition equipment;
s2, digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment on an ALS patient;
s3, adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users;
s4, performing model training by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set to form an SSVEP signal identification artificial intelligent algorithm model;
s5, carrying an SSVEP signal recognition artificial intelligent algorithm model to perform visual stimulation and signal transmission and instruction display.
In step S1 in this embodiment, the brain-computer interface BCI signal acquisition device acquires nerve brain electrical signals of (0-1, a-z) in different expression states of the user through the forehead lobe 7 channel, the temporal lobe 6 channel, the occipital lobe 7 channel, and the parietal lobe 6 channel.
As shown in fig. 2, in step S2, by referring to a complex system modeling algorithm and analysis and mining of frequency domain and time signal characteristics of an EEG (brain nerve electrical signal), an effective contact channel of a brain-computer interface BCI signal acquisition device for a user is mined, effective stimulus related event signals (signal source stimulus) are acquired and observed as shown in fig. 2, signal detection is performed on each channel (channel 1-channel 25), effective channel screening is performed, and the acquired brain electrical signal is amplified and digital-analog converted and encoded and transmitted to a data analysis system in a wired or wireless connection manner.
In the embodiment, after channel selection is based on concurrent signal acquisition, an instruction induction experiment paradigm is set, effective stimulation related event signals are acquired and observed, the acquisition quality, precision and consistency of the signals of each channel are counted, and the optimal communication calculation channels are selected; and amplifying the effective channel signal and transmitting codes at the sensor circuit end. The data transmission mode can be any wireless connection mode, is not limited to Bluetooth, data traffic and WiFi, and preferably adopts a wired mode to transmit data.
In step S3 in this embodiment, an additional ssavp paradigm is set, and an instruction signal is acquired, and data enhancement is performed on the acquired signal, so as to obtain an initial-test-induced associated instruction signal; and carrying out uniform scale calculation on various instructions corresponding to each channel to finish self-adaptive scale adjustment coding.
As shown in fig. 3, in step S4, after the acquisition of the data set is completed, signal preprocessing calculation is performed, which mainly includes: filtering the original data by adopting a filtering algorithm, and filtering high-frequency and low-frequency artifacts, power frequency interference, electrooculogram and other noises in the original data to obtain pure electroencephalogram signals; fourier transform or wavelet transform is carried out on frontal lobe brain signals, energy value relative ratios of different frequencies are calculated, and effective channel signals are identified;
sending the preprocessed electroencephalogram characteristic values into an algorithm model to perform artificial intelligence model to perform multi-classification algorithm training, and detecting semantics of different instruction states of a user; in this step, the algorithm model employed may be a conventional machine learning Model (ML) and a Deep Learning (DL) model; after model training is completed, a large-scale test scene is constructed for evaluation, and interactive configuration application can be performed when the blind test accuracy reaches more than 90%.
The embodiment further comprises the following steps: after the basic brain machine acquisition is completed, different acquisition instructions and acquisition paradigms can be designed, a corresponding brain nerve signal control instruction data set is constructed based on different movement intents, and the corresponding instructions are identified by respectively training a classification model through machine learning and deep learning algorithm models:
noise reduction and pseudo removal: filtering the acquired data by adopting a filtering algorithm, and filtering high-frequency and low-frequency artifacts, power frequency interference noise and electro-oculogram noise in the raw data to obtain stably distributed electroencephalogram signals;
frequency instrument conversion: performing Fourier transform or wavelet transform on the electroencephalogram signals, and calculating the energy distribution densities of waves with different frequencies and different frequency bands;
extracting time distribution characteristic values: calculating main characteristic wave band signal data, calculating the distribution trend of characteristics and data trend, and selecting a reasonable interval as a characteristic value under a corresponding time period to be used as a model calculation input value;
model identification and classification: and sending the characteristic values into an algorithm model to perform model identification and classification, and detecting an action instruction of a user at the moment. Algorithm models for a conventional machine learning algorithm and a deep learning model are several conventional classical models: SVM, decision tree, KNN, random forest, naive bayes classification, least squares, logistic regression, CNN and RNN etc. deep learning models, etc.
The brain-computer interface BCI signal acquisition equipment is utilized to adopt original EEG data, then a filter is adopted to filter the witness artifacts, frequency instrument analysis is carried out on the brain electricity at two sides, the asymmetry index of the forehead She Bo is calculated, and the characteristic value is extracted to carry out model classification and identification.
As shown in fig. 4, in this embodiment, based on the SSVEP coding scheme and BCI effective channel signal correlation optimal for the individual, electroencephalogram data corresponding to different letters a-Z are collected, and classification is stored for label and data spectrum analysis;
setting SSVEP schemes in different forms, carrying out a single signal source stimulation output test, analyzing response states under the different SSVEP schemes, and selecting an optimal SSVEP scheme;
and collecting an optimal channel and an optimal SSVEP paradigm, utilizing brain-computer interface (BCI) signal acquisition equipment to perform data acquisition, performing data instruction set construction, and completing acquisition of different interaction instruction signal sets of 0-9 and A-Z across crowds.
As shown in fig. 5, as a preferred embodiment, in step S5, based on the physical characteristics of the ALS patient, an interactive display and control interface setting is performed to ensure that visual stimulus and signal transmission and instruction display can be effectively performed;
1) Carrying an interactive interface in a region where the vision of a user can be reached, wherein the interactive interface mainly presents instruction output display and monitoring the change state of the brain signals of an individual;
2) The brain instruction signal induction and enhancement stimulator is carried in an auxiliary mode, the signal intensity of the user idea instruction is improved, and therefore model calculation robustness is provided;
3) And carrying the trained SSVEP signal recognition artificial intelligent algorithm model on an interactive display system to realize intelligent idea interactive application of the whole flow. For example, ALS patients interact with normal people through trained SSVEP signal recognition artificial intelligent algorithm models, such as daily speaking communication like your.
In this embodiment, a trained model is deployed on the whole interactive system, the data instruction features of the user are transmitted back to the background database and are incorporated into the algorithm model to perform automatic correction, and the accuracy of the algorithm model for identifying the individual instructions is continuously improved based on the reinforcement learning mechanism.
The invention can automatically detect the signal characteristics of the user in real time through an artificial intelligent algorithm, and efficiently detect the signals and calculate the characteristics; the invention can excavate the effective signal channel position based on the deep learning model based on the portable brain-computer interface equipment, automatically correct the scheme by prefabricating the trained recognition model, and is convenient for users to use; the invention is based on portable BCI equipment, combines a signal data analysis algorithm and an artificial intelligence algorithm to automatically extract and identify the brain signal characteristics of the user, efficiently carries out interactive signal coding, improves the interactive efficiency, and widens the interactive application scene of the brain-computer interface.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention.
Claims (8)
1. An ALS patient man-machine interaction system based on a brain-machine interface and an artificial intelligence algorithm is characterized in that: at least comprises
The brain-computer interface BCI signal acquisition equipment is used for acquiring nerve brain electrical signals of an ALS patient;
the effective channel signal selection module is used for digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment to an ALS patient;
the self-adaptive scale adjustment coding module is used for adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users; setting an additional SSEVP paradigm, collecting instruction signals, and carrying out data enhancement on the collected signals to obtain initial-test induced correlation instruction signals; carrying out uniform scale calculation on various instructions corresponding to each channel to finish self-adaptive scale adjustment coding; based on the optimal SSVEP coding scheme and BCI effective channel signal correlation for individuals, acquiring electroencephalogram signal data corresponding to different letters A-Z, and storing and classifying for label and data spectrum analysis; setting SSVEP schemes in different forms, carrying out a single signal source stimulation output test, analyzing response states under the different SSVEP schemes, and selecting an optimal SSVEP scheme; the optimal channel and the optimal SSVEP paradigm are collected, data instruction set construction is carried out, and acquisition of different instruction data sets of 0-9, A-Z across groups of people is completed;
the AI recognition algorithm module is used for performing model training to form an SSVEP signal recognition artificial intelligent algorithm model by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set;
and the interactive display system is provided with an SSVEP signal recognition artificial intelligent algorithm model and is used for performing visual stimulation, signal transmission and instruction display.
2. An ALS patient human-computer interaction method based on a brain-computer interface and an artificial intelligence algorithm is characterized by comprising the following steps of: the interaction method adopts the interaction system of claim 1, and comprises the following steps:
s1, acquiring nerve brain-brain electrical signals of an ALS patient by using brain-computer interface (BCI) signal acquisition equipment;
s2, digging out an effective contact channel of the brain-computer interface BCI signal acquisition equipment on an ALS patient;
s3, adapting continuous and effective SSVEP coding frequency bands through a self-adaptive algorithm to realize intelligent generation and configuration of different coding schemes for different users; in step S3, an additional SSEVP paradigm is set, acquisition of command signals is carried out, and data enhancement is carried out on the acquired signals, so that initial-test induced correlation command signals are obtained; carrying out uniform scale calculation on various instructions corresponding to each channel to finish self-adaptive scale adjustment coding;
based on the optimal SSVEP coding scheme and BCI effective channel signal correlation for individuals, acquiring electroencephalogram signal data corresponding to different letters A-Z, and storing and classifying for label and data spectrum analysis;
setting SSVEP schemes in different forms, carrying out a single signal source stimulation output test, analyzing response states under the different SSVEP schemes, and selecting an optimal SSVEP scheme;
the optimal channel and the optimal SSVEP paradigm are collected, data instruction set construction is carried out, and acquisition of different instruction data sets of 0-9, A-Z across groups of people is completed;
s4, performing model training by constructing a large-scale pre-training brain-computer BCI brain-computer (BCI) brain-computer data set to form an SSVEP signal identification artificial intelligent algorithm model;
s5, carrying an SSVEP signal recognition artificial intelligent algorithm model to perform visual stimulation and signal transmission and instruction display.
3. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 2, wherein: in step S1, the nerve brain-computer signal of 0-9,a-z under different expression states of a user is collected by a forehead lobe 7 channel, a temporal lobe 6 channel, a occipital lobe 7 channel and a parietal lobe 6 channel in brain-computer interface BCI signal collection equipment.
4. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 3, wherein: in step S2, by analyzing and mining the frequency domain and time domain signal characteristics of the brain-computer interface BCI signal acquisition equipment, an effective contact channel of the user is mined, the acquired brain-computer signal is amplified and digital-to-analog converted and encoded, and transmitted to a data analysis system in a wired or wireless connection manner.
5. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 4, wherein: after channel selection is based on concurrent signal acquisition, setting an instruction induction experiment paradigm, acquiring and observing effective stimulus related event signals, counting the acquisition quality, precision and consistency of signals of each channel, and selecting an optimal communication calculation channel; and amplifying the effective channel signal and transmitting codes at the sensor circuit end.
6. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 4, wherein: in step S4, it further comprises
After the acquisition of the data set is completed, signal preprocessing calculation is carried out, and the method mainly comprises the following steps: filtering the original data by adopting a filtering algorithm, and filtering high-frequency and low-frequency artifacts, power frequency interference and electro-oculogram noise in the original data to obtain pure electroencephalogram signals; fourier transform or wavelet transform is carried out on frontal lobe brain signals, energy value relative ratios of different frequencies are calculated, and effective channel signals are identified;
sending the preprocessed electroencephalogram characteristic values into an algorithm model to perform artificial intelligence model to perform multi-classification algorithm training, and detecting semantics of different instruction states of a user; in this step, the algorithm model employed may be a conventional machine learning Model (ML) and a Deep Learning (DL) model; after model training is completed, a large-scale test scene is constructed for evaluation, and interactive configuration application can be performed when the blind test accuracy reaches more than 90%.
7. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 4, wherein: in step S5, based on the physical characteristics of the ALS patient, an interactive display and control interface is set to ensure that visual stimulation, signal transmission and instruction display can be effectively performed;
1) Carrying an interactive interface in a region where the vision of a user can be reached, wherein the interactive interface mainly presents instruction output display and monitoring the change state of the brain signals of an individual;
2) The brain instruction signal induction and enhancement stimulator is carried in an auxiliary mode, the signal intensity of the user idea instruction is improved, and therefore model calculation robustness is provided;
3) And carrying the trained SSVEP signal recognition artificial intelligent algorithm model on an interactive display system to realize intelligent idea interactive application of the whole flow.
8. The ALS patient human-machine interaction method based on the brain-machine interface and the artificial intelligence algorithm according to claim 7, wherein: the trained model is deployed on the whole interactive system, the data instruction features of the user are returned to the background database and are incorporated into the algorithm model to be automatically corrected, and the accuracy of the algorithm model to individual instruction recognition is continuously improved based on the reinforcement learning mechanism.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310236983.8A CN116400800B (en) | 2023-03-13 | 2023-03-13 | ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310236983.8A CN116400800B (en) | 2023-03-13 | 2023-03-13 | ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116400800A CN116400800A (en) | 2023-07-07 |
| CN116400800B true CN116400800B (en) | 2024-01-02 |
Family
ID=87015052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310236983.8A Active CN116400800B (en) | 2023-03-13 | 2023-03-13 | ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116400800B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116880700A (en) * | 2023-09-07 | 2023-10-13 | 华南理工大学 | Raspberry group intelligent trolley control method and system based on wearable brain-computer interface |
| CN117159385A (en) * | 2023-09-19 | 2023-12-05 | 广东实验中学 | Intelligent medicine feeding system for disabled people fusing brain-computer interface technology |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| MX2015009497A (en) * | 2015-07-23 | 2017-01-23 | Inst Politecnico Nacional | Multi-frequency encoding method for a brain-computer interface based on steady state visually evoked potential. |
| KR101720766B1 (en) * | 2015-12-01 | 2017-03-28 | 고려대학교 산학협력단 | Apparatus and method for analysis of brain wave using ssvep |
| CN107329571A (en) * | 2017-06-29 | 2017-11-07 | 华南理工大学 | A kind of multi-channel adaptive brain-machine interaction method of Virtual practical application |
| KR20180036503A (en) * | 2016-09-30 | 2018-04-09 | 고려대학교 산학협력단 | Apparatus and method of brain-computer interface for device controlling based on brain signal |
| CN110007769A (en) * | 2019-04-16 | 2019-07-12 | 山东建筑大学 | A kind of AC system and method based on asynchronous brain-computer interface |
| CN113967022A (en) * | 2021-11-16 | 2022-01-25 | 常州大学 | Motor imagery electroencephalogram characteristic characterization method based on individual self-adaption |
| KR20220048567A (en) * | 2020-10-13 | 2022-04-20 | 한양대학교 산학협력단 | Method for stimulation frequency of ssvep signal |
| CN114415833A (en) * | 2022-01-10 | 2022-04-29 | 太原理工大学 | Electroencephalogram asynchronous control software design method based on time-space frequency conversion SSVEP |
| CN114916945A (en) * | 2022-05-20 | 2022-08-19 | 上海前瞻创新研究院有限公司 | Brain-computer interface detection method based on electroencephalogram signals, medium and electronic equipment |
| CN115414051A (en) * | 2021-05-12 | 2022-12-02 | 西安邮电大学 | Emotion classification and recognition method of electroencephalogram signal self-adaptive window |
| CN116700495A (en) * | 2023-06-13 | 2023-09-05 | 上海大学 | Brain-computer interaction method and equipment based on steady-state visual evoked potential and motor imagery |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014142962A1 (en) * | 2013-03-15 | 2014-09-18 | Intel Corporation | Brain computer interface (bci) system based on gathered temporal and spatial patterns of biophysical signals |
| GB2621265A (en) * | 2021-03-19 | 2024-02-07 | Mindportal Inc | Systems and methods involving devices that detect mind/brain activity |
-
2023
- 2023-03-13 CN CN202310236983.8A patent/CN116400800B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| MX2015009497A (en) * | 2015-07-23 | 2017-01-23 | Inst Politecnico Nacional | Multi-frequency encoding method for a brain-computer interface based on steady state visually evoked potential. |
| KR101720766B1 (en) * | 2015-12-01 | 2017-03-28 | 고려대학교 산학협력단 | Apparatus and method for analysis of brain wave using ssvep |
| KR20180036503A (en) * | 2016-09-30 | 2018-04-09 | 고려대학교 산학협력단 | Apparatus and method of brain-computer interface for device controlling based on brain signal |
| CN107329571A (en) * | 2017-06-29 | 2017-11-07 | 华南理工大学 | A kind of multi-channel adaptive brain-machine interaction method of Virtual practical application |
| CN110007769A (en) * | 2019-04-16 | 2019-07-12 | 山东建筑大学 | A kind of AC system and method based on asynchronous brain-computer interface |
| KR20220048567A (en) * | 2020-10-13 | 2022-04-20 | 한양대학교 산학협력단 | Method for stimulation frequency of ssvep signal |
| CN115414051A (en) * | 2021-05-12 | 2022-12-02 | 西安邮电大学 | Emotion classification and recognition method of electroencephalogram signal self-adaptive window |
| CN113967022A (en) * | 2021-11-16 | 2022-01-25 | 常州大学 | Motor imagery electroencephalogram characteristic characterization method based on individual self-adaption |
| CN114415833A (en) * | 2022-01-10 | 2022-04-29 | 太原理工大学 | Electroencephalogram asynchronous control software design method based on time-space frequency conversion SSVEP |
| CN114916945A (en) * | 2022-05-20 | 2022-08-19 | 上海前瞻创新研究院有限公司 | Brain-computer interface detection method based on electroencephalogram signals, medium and electronic equipment |
| CN116700495A (en) * | 2023-06-13 | 2023-09-05 | 上海大学 | Brain-computer interaction method and equipment based on steady-state visual evoked potential and motor imagery |
Non-Patent Citations (2)
| Title |
|---|
| Brain structural and perfusion changes in amyotrophic lateral sclerosis-frontotemporal dementia patients with cognitive and motor onset: a preliminary study;Yang, Xunzhe;《BRAIN IMAGING AND BEHAVIOR》;全文 * |
| 临床神经电生理在肌萎缩侧索硬化中的应用;崔丽英;《中华神经科杂志》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116400800A (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116400800B (en) | ALS patient human-computer interaction system and method based on brain-computer interface and artificial intelligence algorithm | |
| CN110969108B (en) | Limb action recognition method based on autonomic motor imagery electroencephalogram | |
| CN103793058A (en) | Method and device for classifying active brain-computer interaction system motor imagery tasks | |
| Islam et al. | Wavelet analysis based classification of emotion from EEG signal | |
| CN109620262A (en) | A kind of Emotion identification system and method based on wearable bracelet | |
| CN100594858C (en) | Electric artificial hand combined controlled by brain electricity and muscle electricity and control method | |
| CN109934089A (en) | Multistage epileptic EEG Signal automatic identifying method based on supervision gradient lifter | |
| CN110604565A (en) | Brain health screening method based on portable EEG equipment | |
| CN112244873A (en) | Electroencephalogram time-space feature learning and emotion classification method based on hybrid neural network | |
| CN106618562A (en) | Wearable epilepsy brain-electricity seizure brain area positioning device and method | |
| CN103699226A (en) | Tri-modal serial brain-computer interface method based on multi-information fusion | |
| CN114533086B (en) | Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation | |
| CN102521505A (en) | Brain electric and eye electric signal decision fusion method for identifying control intention | |
| CN104978035A (en) | Brain computer interface system evoking P300 based on somatosensory electrical stimulation and implementation method thereof | |
| CN110974219A (en) | Human brain idea recognition system based on invasive BCI | |
| CN112515685A (en) | Multi-channel electroencephalogram signal channel selection method based on time-frequency co-fusion | |
| CN111544256A (en) | Brain-controlled intelligent full limb rehabilitation method based on graph convolution and transfer learning | |
| CN108958486A (en) | Mood brain machine interface system based on recognition of face brain electricity N170 ingredient | |
| Ahmadi et al. | Brain-computer interface signal processing algorithms: A computational cost vs. accuracy analysis for wearable computers | |
| CN115770044B (en) | Emotion recognition method and device based on electroencephalogram phase amplitude coupling network | |
| CN109144238A (en) | A kind of man-machine interactive system and its exchange method based on eye electricity coding | |
| CN113558644B (en) | Emotion classification method, medium and equipment for 3D matrix and multidimensional convolution network | |
| CN114652328A (en) | Bidirectional closed-loop brain-computer interaction electro-acupuncture parameter intelligent matching system for insomnia | |
| CN108491792B (en) | Office scene human-computer interaction behavior recognition method based on electro-oculogram signals | |
| CN106843509B (en) | Brain-computer interface system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |