CN114146283A - Attention training system and method based on target detection and SSVEP - Google Patents
Attention training system and method based on target detection and SSVEP Download PDFInfo
- Publication number
- CN114146283A CN114146283A CN202110985594.6A CN202110985594A CN114146283A CN 114146283 A CN114146283 A CN 114146283A CN 202110985594 A CN202110985594 A CN 202110985594A CN 114146283 A CN114146283 A CN 114146283A
- Authority
- CN
- China
- Prior art keywords
- ssvep
- target
- target detection
- attention training
- electroencephalogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 145
- 238000001514 detection method Methods 0.000 title claims abstract description 115
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 55
- 238000007781 pre-processing Methods 0.000 claims abstract description 25
- 238000010219 correlation analysis Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 16
- 230000000638 stimulation Effects 0.000 claims description 15
- 230000003044 adaptive effect Effects 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 10
- 210000004556 brain Anatomy 0.000 claims description 8
- 238000009432 framing Methods 0.000 claims description 3
- 230000005611 electricity Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 abstract description 9
- 230000000763 evoking effect Effects 0.000 abstract description 3
- 230000008131 children development Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000003993 interaction Effects 0.000 abstract description 2
- 230000003935 attention Effects 0.000 abstract 6
- 230000010332 selective attention Effects 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 8
- 208000006096 Attention Deficit Disorder with Hyperactivity Diseases 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 4
- 230000004438 eyesight Effects 0.000 description 4
- 208000036864 Attention deficit/hyperactivity disease Diseases 0.000 description 3
- 208000015802 attention deficit-hyperactivity disease Diseases 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004936 stimulating effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 208000020358 Learning disease Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 238000002651 drug therapy Methods 0.000 description 1
- 208000013403 hyperactivity Diseases 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 201000003723 learning disability Diseases 0.000 description 1
- 231100000957 no side effect Toxicity 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 210000000857 visual cortex Anatomy 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/377—Electroencephalography [EEG] using evoked responses
- A61B5/378—Visual stimuli
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
- A61B5/386—Accessories or supplementary instruments therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
- A61M2021/0044—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense
- A61M2021/005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense images, e.g. video
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Heart & Thoracic Surgery (AREA)
- Biomedical Technology (AREA)
- Surgery (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Psychiatry (AREA)
- Pathology (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Psychology (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Acoustics & Sound (AREA)
- Anesthesiology (AREA)
- Hematology (AREA)
Abstract
The invention discloses an attention training system and method based on target detection and SSVEP (steady state visual evoked potential), which comprises a camera, an electroencephalogram cap, a computer and a mechanical arm. The camera is used for collecting a video stream which is sent to the target detection module for target detection; the electroencephalogram cap is used for collecting and transmitting SSVEP electroencephalogram signals; the computer comprises an attention training interface, a target detection unit and an SSVEP decoding unit, wherein the attention training interface draws a target frame and generates a flash block corresponding to an article in real time, the SSVEP decoding unit decodes the electroencephalogram signal through preprocessing and an FBCCA algorithm to enable the flash block watched by a user, the decoding result is converted into a control instruction to be sent to the mechanical arm, and the mechanical arm captures the corresponding article according to the instruction. The invention can carry out attention training through natural human-computer interaction, effectively improve selective attention and continuous attention level, provide reference for attention training methods and means, and show good application prospects of brain-computer interface technology in the fields of child development and the like.
Description
Technical Field
The invention designs an Attention training system and method based on target detection and SSVEP for ADHD (Attention Deficit Hyperactivity Disorder) and children and teenagers in the school age. The system comprises a camera, an electroencephalogram cap, a computer, a mechanical arm, an attention training interface, a target detection unit and an SSVEP decoding unit, and combines an electroencephalogram biological signal and a computer software training method to help a user to carry out attention training.
Background
Attention problems in children are now quite common and are diagnosed as Attention Deficit Hyperactivity Disorder (ADHD) when the clinical manifestations of Attention problems become significant. The data show that the prevalence rate of ADHD in Chinese school-age children is about 1.5% -10%, the typical symptoms of the disease comprise three types, namely inattention, hyperactivity and impulsion, and the attention problem is also the primary factor causing learning disorder of children. At present, a plurality of methods for treating attention defects at home and abroad exist, but each method has limitations, and the exploration of comprehensive treatment of different methods becomes a trend and a hot spot of research in recent years. Drug therapy is a mode which is adopted by most medical institutions, is the fastest effective for children with serious attention deficit, but has serious side effects, and once the drug is stopped, symptoms can appear repeatedly. The general computer software training is to transplant the traditional attention training task to a computer, but many software only simply uses the computer to replace paper and pen operation, the training element is single, and the task design is not innovative. The electroencephalogram biofeedback is accepted by experts and parents as a training method which has no stimulation and no side effect and can keep the training result for a long time. Meanwhile, in the learning process, the students receive the stimulation mainly in the aspects of vision and hearing, and the learning effect can be better improved by training the attention of the students in the aspects of vision and hearing.
Considering the current domestic training method, most of the existing systems are focused on one-to-one artificial training, and no systematic artificial intelligence product is designed aiming at the attention of children, but the artificial intelligence product has the characteristics of repeatability, feedback, interestingness and the like, so that an attention training system based on target detection and SSVEP is designed by combining electroencephalogram biological signals, wherein SSVEP (steady state visual evoked potential) means that when a visual stimulus with fixed frequency is received, a continuous response related to the stimulus frequency (at the fundamental frequency or the frequency multiplication position of the stimulus frequency) is generated by the visual cortex of a human brain, a voice instruction is given to the children through the system, the children are attracted to watch a flicker block with corresponding frequency so as to control a mechanical arm to grab articles, so that the artificial intelligence training mode is beneficial to keeping the interest of the children in continuing training and the enthusiasm of completing tasks in games, the children can also obtain achievement feeling by finishing corresponding instructions. Wherein, the more concentrated the children are in training, the more concentrated the eyesight is, the better the SSVEP effect is, thereby achieving the effect of training the attention in the visual and auditory aspects in the process of continuously executing the voice commands.
Disclosure of Invention
In order to solve the problems of the prior art, the invention aims to overcome the defects in the prior art, and provides an attention training system and method based on target detection and SSVEP (steady state visual evoked potential), which combines electroencephalogram biofeedback training and computer software training to help a user to carry out attention training. The basic principle is that an attention training interface generates flashing blocks corresponding to articles one by one according to the detection result of an SSD target detection algorithm, a user watches the corresponding flashing blocks according to a voice instruction, an electroencephalogram cap collects electroencephalogram signals of the user at the moment and transmits the electroencephalogram signals to an SSVEP decoding unit at a computer end, the signals are decoded through preprocessing and an FBCCA algorithm, the flashing blocks watched by the user are decoded, the decoding result is compared with instruction information, and then an instruction is sent to a mechanical arm to control the mechanical arm to grab the articles.
In order to achieve the purpose of the invention, the invention adopts the following technical scheme:
an attention training system and method based on target detection and SSVEP comprises the following steps: the system comprises a camera, an electroencephalogram cap, a computer, a mechanical arm, an attention training interface, a target detection unit and an SSVEP decoding unit, wherein the electroencephalogram cap is connected with the computer in a wireless mode; the computer comprises an attention training interface, a target detection unit and an SSVEP decoding unit, and the camera is connected with the computer in a wired mode;
the camera is used for acquiring video streams and transmitting the video streams to the attention training interface and the target detection unit of the computer end;
the electroencephalogram cap is used for collecting SSVEP electroencephalogram signals output by a user during attention training and transmitting the electroencephalogram signals to the computer terminal;
the computer comprises a target detection module, an SSD target detection algorithm is used for carrying out target detection on video streams acquired by the camera and sending target detection results to an attention training interface, the attention training interface generates flashing blocks corresponding to articles one by one in real time according to the detection results, and meanwhile, an SSVEP decoding unit is used for decoding electroencephalogram signals through preprocessing and an adaptive FBCCA algorithm to obtain the frequency of the electroencephalogram signals, outputting decoding results according to the comparison condition of the frequency of the electroencephalogram signals and stimulation signals corresponding to instructions, and sending control instructions to the mechanical arm according to the decoding results;
and the mechanical arm executes the control command and grabs the corresponding article according to the planned path.
Preferably, the process of performing the object detection by the SSD object detection and identification algorithm includes two processes of off-line training and on-line identification, wherein the off-line training step includes:
a-1, constructing a target detection training set;
a-2, inputting the training set into an SSD target detection network for training;
a-3, generating an SSD target detection model;
the online identification process comprises the following steps:
b-1, reading a video stream collected by a camera;
b-2, loading the SSD target detection model which is trained off line;
b-3, preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
b-4, adopting non-maximum value to inhibit and remove the target frame information with overhigh repetition rate;
and b-5, outputting the target information.
Preferably, the SSVEP decoding unit decodes the electroencephalogram signal through a preprocessing and adaptive filter bank typical correlation analysis FBCCA algorithm, and the specific steps are as follows:
c-1, reading the electroencephalogram data of the user according to a set period;
c-2, decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
c-3, performing correlation analysis on each sub-band component obtained by filtering and a standard sine and cosine reference signal;
c-4, calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and c-5, wherein the frequency corresponding to the maximum correlation is the recognition result.
The invention relates to an attention training method based on target detection and SSVEP, which is operated by adopting an attention training system based on target detection and SSVEP and comprises the following operation steps:
(1) the user wears the electroencephalogram cap and is connected with the camera;
(2) opening an attention training interface, clicking a button for opening a camera, and detecting the attention training interface according to an SSD target detection algorithm;
(3) clicking a training start button;
(4) when the attention training interface has stimulating flicker, a user watches the corresponding flicker block according to the voice prompt, and the electroencephalogram cap collects electroencephalogram signals and transmits the electroencephalogram signals to the SSVEP decoding unit at the computer end;
(5) the SSVEP decoding unit decodes the electroencephalogram signals through preprocessing and a self-adaptive FBCCA algorithm;
(6) after the SSVEP decoding unit decodes through the FBCCA algorithm, a control command is sent to the mechanical arm according to a decoding result;
(7) the robot arm picks the corresponding item.
Preferably, in the step (2), after the camera button is clicked to open, the step of detecting, by the attention training interface according to the SSD object detection algorithm is as follows:
(2-1) reading a video stream captured by a camera;
(2-2) loading an SSD target detection model which is trained offline;
(2-3) preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
(2-4) suppressing and removing the target frame information with the excessively high repetition rate by adopting a non-maximum value;
and (2-5) drawing a target frame around each target in the video stream in real time according to the target detection result by the attention training interface, and simultaneously numbering the drawing of the targets.
Further preferably, in the step (2-3), the detected target information includes a target position and a target name, wherein the target position includes a target frame height, a target frame width and a target frame center coordinate for framing the target.
Preferably, in the step (3), after the training start button is clicked, a transparent interface is generated above the video stream block in the attention training interface, the position information of the upper left corner of the target frame is taken as the position of the lower left corner of the flicker block, the transparent interface adds n flicker blocks in real time according to the detection result, and flicker frequencies of the flicker blocks of different targets are different.
Preferably, in the step (4), the process of drawing the stimulus flicker by the attention training interface includes:
(4-1) the camera acquires a video stream and transmits video stream information to the target recognition module and the attention training interface;
(4-2) identifying the target information in the video stream in real time by the target identification module by using an SSD algorithm;
(4-3) drawing a matched target frame around each target in real time in a video stream by the attention training interface according to the target information, selecting n targets from the frames, and marking the n targets with numbers;
and (4-4) generating a transparent interface above a video stream block in the attention training interface, taking the position information of the upper left corner of the target frame as the position of the lower left corner of the flicker block, adding n flicker blocks in real time by the transparent interface according to the detection result, and enabling the flicker blocks of different targets to have different flicker frequencies. A user watches the flicker block of the corresponding target on the test interface according to the voice instruction, and meanwhile, the electroencephalogram signals are transmitted to the SSVEP decoding unit at the computer end in a wireless mode to be processed.
Preferably, in the step (5), the SSVEP decoding unit decodes the electroencephalogram signal through a preprocessing and adaptive FBCCA algorithm, and the specific steps are as follows:
(5-1) reading the electroencephalogram data of the user according to a set period;
(5-2) decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
(5-3) carrying out correlation analysis on each subband component obtained by filtering and a standard sine and cosine reference signal;
(5-4) calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and (5-5) wherein the frequency corresponding to the maximum correlation is the recognition result. The SSVEP decoding unit carries out preprocessing, namely filtering, on electroencephalogram signals of a user, typical correlation analysis is carried out on the FBCCA filter bank, and a flicker block watched by the user is decoded and identified.
Preferably, in the step (5), the SSVEP decoding unit decodes the electroencephalogram signal through preprocessing and an adaptive FBCCA algorithm as follows:
(5-1) decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
(5-2) carrying out correlation analysis on each subband component obtained by filtering and a standard sine and cosine reference signal;
(5-3) calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and (5-4) wherein the frequency corresponding to the maximum correlation is the recognition result.
Compared with the prior art, the invention has the following obvious and prominent substantive characteristics and remarkable advantages:
1. the system adopts a target detection and SSVEP method, a channel is established between the brain of a user and a computer or other electronic equipment, the control of the brain on external equipment is realized, and the user watches a corresponding scintillation block after listening to a voice command, so that the grabbing of a mechanical arm is controlled;
2. the more concentrated the attention of the user of the system, the better the SSVEP effect, and the faster the instruction can be completed, thereby improving the attention of the user in the visual and auditory aspects, and also providing reference for designing an artificial intelligence system aiming at the attention of children in future research.
3. The method is simple and easy to implement, low in cost and suitable for popularization and application.
Drawings
Fig. 1 is a block diagram of the system architecture of the preferred embodiment of the present invention.
Fig. 2 is a flow chart of the overall experiment of the preferred embodiment of the present invention.
FIG. 3 is a flowchart of a computer program according to a preferred embodiment of the present invention.
Fig. 4 is a flow chart of attention training stimulation interface rendering according to a preferred embodiment of the present invention.
FIG. 5 is an attention training interface of a preferred embodiment of the present invention.
Detailed Description
The above-described scheme is further illustrated below with reference to specific embodiments, which are detailed below:
the first embodiment is as follows:
in this embodiment, referring to fig. 1, an attention training system based on target detection and SSVEP includes a camera, an electroencephalogram cap, a computer, a mechanical arm, an attention training interface, a target detection unit, and an SSVEP decoding unit, and is characterized in that: the electroencephalogram cap is connected with a computer in a wireless mode, the computer comprises an attention training interface, a target detection unit and an SSVEP decoding unit, and the camera is connected with the computer in a wired mode;
the camera: the attention training interface and the target detection unit are used for acquiring video streams and transmitting the video streams to the computer end;
the electroencephalogram cap comprises: the device is used for collecting the SSVEP electroencephalogram signals output by a user and transmitting the electroencephalogram signals to the computer terminal;
the computer: the system comprises a camera, an attention training interface, an SSVEP decoding unit, an FBCCA algorithm, an adaptive filter bank typical correlation analysis (FBCCA) module, a signal processing module and a control module, wherein the SSVEP decoding unit is used for decoding an electroencephalogram signal through a preprocessing and adaptive filter bank typical correlation analysis (FBCCA) algorithm, acquiring the frequency of the electroencephalogram signal, outputting a decoding result according to the frequency of the electroencephalogram signal and the contrast condition of a stimulation signal corresponding to a command, and sending a control command to a mechanical arm according to the decoding result;
the mechanical arm is: and executing the control command, and grabbing the corresponding article according to the planned path.
The system of the embodiment combines the electroencephalogram biofeedback training with the computer software training to help the user to carry out attention training. The attention training interface can generate flash blocks corresponding to the articles one by one according to the detection result of the SSD target detection algorithm, a user watches the corresponding flash blocks according to a voice instruction, the electroencephalogram cap collects electroencephalogram signals of the user at the moment and transmits the electroencephalogram signals to the SSVEP decoding unit at the computer end, the signals are decoded through preprocessing and the FBCCA algorithm, the flash blocks watched by the user are decoded, the decoding result is compared with instruction information, and then an instruction is sent to the mechanical arm to control the mechanical arm to grab the articles. Thereby achieving the effect of training the attention in the visual and auditory aspects in the process of continuously executing the voice commands.
Example two:
this embodiment is substantially the same as the first embodiment, and is characterized in that:
in this embodiment, referring to fig. 1-2, the process of performing target detection by the SSD target detection and identification algorithm includes two processes, namely, offline training and online identification, wherein the offline training step includes:
a-1, constructing a target detection training set;
a-2, inputting the training set into an SSD target detection network for training;
a-3, generating an SSD target detection model;
the online identification process comprises the following steps:
b-1, reading a video stream collected by a camera;
b-2, loading the SSD target detection model which is trained off line;
b-3, preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
b-4, adopting non-maximum value to inhibit and remove the target frame information with overhigh repetition rate;
and b-5, outputting the target information.
In this embodiment, the SSVEP decoding unit decodes the electroencephalogram signal through a preprocessing and an adaptive filter bank typical correlation analysis FBCCA algorithm, and the specific steps are as follows:
c-1, reading the electroencephalogram data of the user according to a set period;
c-2, decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
c-3, performing correlation analysis on each sub-band component obtained by filtering and a standard sine and cosine reference signal;
c-4, calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and c-5, wherein the frequency corresponding to the maximum correlation is the recognition result.
The system of the embodiment adopts a target detection and SSVEP method, a channel is established between the brain of a user and a computer or other electronic equipment, the control of the brain on external equipment is realized, and the user watches a corresponding flashing block after listening to a voice command, so that the grabbing of a mechanical arm is controlled; the more focused the user is, the better the SSVEP effect is, and the faster the instruction can be completed, thereby improving the attention of the user in visual and auditory aspects.
Example three:
this embodiment is substantially the same as the above embodiment, and is characterized in that:
in this embodiment, an attention training method based on target detection and SSVEP according to the above embodiment is operated by using the attention training system based on target detection and SSVEP, and the operation steps are as follows:
(1) the user wears the electroencephalogram cap and is connected with the camera;
(2) opening an attention training interface, clicking a button for opening a camera, and detecting the attention training interface according to an SSD target detection algorithm;
(3) clicking a training start button;
(4) when the attention training interface has stimulus flicker, a user watches the corresponding flicker block according to voice prompt, the attention training interface draws the stimulus flicker, and meanwhile, the electroencephalogram cap collects electroencephalogram signals and transmits the electroencephalogram signals to the SSVEP decoding unit at the computer end;
(5) the SSVEP decoding unit decodes the electroencephalogram signals through preprocessing and a self-adaptive FBCCA algorithm;
(6) after the SSVEP decoding unit decodes the electroencephalogram signals, a control instruction is sent to the mechanical arm according to the decoding result;
(7) the robot arm picks the corresponding item.
The attention training interface of the embodiment can generate the flashing blocks corresponding to the articles one by one according to the detection result of the SSD target detection algorithm, a user watches the corresponding flashing blocks according to a voice instruction, the electroencephalogram cap collects electroencephalogram signals of the user at the moment and transmits the electroencephalogram signals to the SSVEP decoding unit at the computer end, the signals are decoded through preprocessing and the FBCCA algorithm, the flashing blocks watched by the user are decoded, the decoding result is compared with instruction information, and then the instruction is sent to the mechanical arm to control the mechanical arm to grab the articles.
Example four:
this embodiment is substantially the same as the above embodiment, and is characterized in that:
in this embodiment, in the step (2), after the camera button is clicked and opened, the step of detecting, by the attention training interface according to the SSD object detection algorithm, is as follows:
(2-1) reading a video stream captured by a camera;
(2-2) loading an SSD target detection model which is trained offline;
(2-3) preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
(2-4) suppressing and removing the target frame information with the excessively high repetition rate by adopting a non-maximum value;
and (2-5) drawing a target frame around each target in the video stream in real time according to the target detection result by the attention training interface, and simultaneously numbering the drawing of the targets.
In this embodiment, in the step (2-3), the detected target information includes a target position and a target name, where the target position includes a target frame height, a target frame width, and a center coordinate of the target frame for framing the target.
In this embodiment, in the step (3), after the training start button is clicked, a transparent interface is generated above the video stream block in the attention training interface, the position information of the upper left corner of the target frame is taken as the position of the lower left corner of the flicker block, the transparent interface adds n flicker blocks in real time according to the detection result, and the flicker frequencies of the flicker blocks of different targets are different.
In this embodiment, in step (4), the process of drawing the stimulus flicker by the attention training interface includes:
(4-1) the camera acquires a video stream and transmits video stream information to the target recognition module and the attention training interface;
(4-2) identifying the target information in the video stream in real time by the target identification module by using an SSD algorithm;
(4-3) drawing a matched target frame around each target in real time in a video stream by the attention training interface according to the target information, selecting n targets from the frames, and marking the n targets with numbers;
and (4-4) generating a transparent interface above a video stream block in the attention training interface, taking the position information of the upper left corner of the target frame as the position of the lower left corner of the flicker block, adding n flicker blocks in real time by the transparent interface according to the detection result, and enabling the flicker blocks of different targets to have different flicker frequencies.
In this embodiment, in the step (5), the SSVEP decoding unit decodes the electroencephalogram signal through the preprocessing and adaptive FBCCA algorithm as follows:
(5-1) decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
(5-2) carrying out correlation analysis on each subband component obtained by filtering and a standard sine and cosine reference signal;
(5-3) calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and (5-4) wherein the frequency corresponding to the maximum correlation is the recognition result.
The attention training interface of the embodiment generates the scintillation blocks corresponding to the articles in real time according to the detection result, and meanwhile, the attention training interface comprises an SSVEP decoding unit which decodes the electroencephalogram signal through preprocessing and an adaptive FBCCA algorithm to obtain the frequency of the electroencephalogram signal, outputs the decoding result according to the comparison condition of the electroencephalogram signal frequency and the stimulation signal corresponding to the command, and sends the control command to the mechanical arm according to the decoding result. The more focused the user is, the better the SSVEP effect is, and the faster the instruction can be completed, thereby improving the attention of the user in visual and auditory aspects.
Example five:
this embodiment is substantially the same as the above embodiment, and is characterized in that:
in this embodiment, as shown in fig. 1, a system and method for attention training based on target detection and SSVEP includes: the device comprises a camera, an electroencephalogram cap, a computer, a mechanical arm, an attention training interface, a target detection unit and an SSVEP decoding unit. By adopting the blue sensing 8-lead dry electrode electroencephalogram cap, the user scalp can collect the electroencephalogram signal without applying electroencephalogram paste. The dry electrode cap amplifies the acquired microvolt level electroencephalogram signals, converts the electroencephalogram analog signals into digital signals through A/D (analog to digital) conversion, the sampling rate is 1000Hz, and the electroencephalogram cap and a computer realize wireless connection between the devices through a router; the computer comprises an attention training interface, a target detection unit and an SSVEP decoding unit; the attention training interface is used for generating an SSVEP stimulation interface; the target detection unit adopts an SSD target detection module to carry out target detection on a video stream acquired by a camera, the SSVEP decoding unit filters an electroencephalogram signal of a user and adopts a self-adaptive FBCCA algorithm to decode a flash block watched by the user, and a control instruction is sent to the mechanical arm according to a decoding result to control the mechanical arm to grab a corresponding article.
In this embodiment, as shown in fig. 2, an experimental procedure of an attention training system and method based on target detection and SSVEP is as follows: 1) firstly, a user wears an electroencephalogram cap to connect a camera, 2) secondly, an attention training interface at a computer end is opened, the camera on the interface is sequentially clicked to be opened and a test is started, when the attention training interface appears stimulating and flickering, 3) the user watches a corresponding flickering block according to voice prompt, 4) the electroencephalogram cap collects electroencephalogram signals of the user at the moment, 5) an SSVEP decoding unit transmitted to the computer end decodes the signals through a self-adaptive FBCCA algorithm, 6) a control signal is sent to a mechanical arm, and 7) the mechanical arm is controlled to grab corresponding articles according to a decoding result.
In the embodiment, as shown in fig. 3, a computer-side program flow of the attention training system and method based on target detection and SSVEP is divided into two parts, namely, an SSVEP decoding unit and an attention training interface, where the SSVEP decoding unit serves as a processor and the attention training interface serves as a stimulator.
The electroencephalogram signals are collected by the electroencephalogram cap and then sent to the SSVEP decoding unit of the computer end, and the program flow is as follows:
1) initializing each electroencephalogram parameter;
2) initializing a mechanical arm communication interface, and establishing communication with a mechanical arm;
3) TCP/IP communication is established between the stimulator and the stimulation device;
4) receiving electroencephalogram cap electroencephalogram signals, performing filtering and adaptive FBCCA filter bank typical correlation analysis, and decoding a flash block watched by a user;
5) sending a control instruction to the mechanical arm;
6) the mechanical arm grabs the article.
In this embodiment, the stimulator is formed by generating a flashing block by the attention training interface according to the detection result of the SSD target detection algorithm, and the program flow is as follows:
stimulator program flow:
1) TCP/IP of the connection processor program;
2) receiving target information detected by a target detection module;
3) according to the target position information, a target frame matched with the target and a flicker block added with the corresponding target are drawn in real time, namely a stimulation interface for attention training is generated;
4) the stimulation interface flicker block flickers at different frequencies so as to generate stimulation signals;
5) the receive processor processes the resulting data.
In this embodiment, the processor and the stimulator are coordinated, that is, the attention training interface generates the stimulator, the user generates corresponding electroencephalogram signals through the stimulation paradigm of the attention training interface, the electroencephalogram signals are collected and transmitted to the processor through the electroencephalogram cap, the SSVEP decoding unit in the processor decodes and analyzes the electroencephalogram signals, the analysis result is converted into a control instruction, and the mechanical arm is controlled to grasp the control instruction.
In this embodiment, as shown in fig. 4, an attention training stimulation interface drawing process based on target detection and SSVEP attention training system and method includes that a target detection unit performs real-time detection on a video stream acquired by a camera through an SSD algorithm to acquire target information, a test interface receives the target information, a matched target frame is drawn in the video stream for each target in real time, the targets are simultaneously drawn with labels in the target frames, a transparent interface is generated above a video stream block in the attention training interface, the position of the upper left corner of the target frame is used as the position of the lower left corner of a flicker block, the transparent interface adds n flicker blocks in real time according to detection results, and flicker frequencies of the flicker blocks of different targets are different. According to the SSVEP steady-state vision induction principle, a user watches different scintillation blocks, namely watching different frequency signals, the brain generates different electroencephalogram signals, and the scintillation blocks watched by the user can be known by decoding the electroencephalogram signals.
In the present embodiment, as shown in fig. 5, an attention training interface of an attention training system and method based on object detection and SSVEP includes a camera and start detection button and a video stream block, the interface draws an object frame at the video stream block according to the result of object detection and adds a flash block, and the flash frequency of different flash blocks is different.
In this embodiment, the target detection adopts an SSD (Single Shot multi box Detector) algorithm, and compared with other target detection algorithms, the SSD removes a full connection layer in the YOLO algorithm, adopts full convolution instead of the prior frames with different scales and aspect ratios, and compared with the fast R-CNN algorithm, the SSD can complete detection in one step by obtaining a candidate frame through the CNN and then performing classification and regression, so that the SSD algorithm improves the detection speed while maintaining the accuracy. The algorithm structure is as follows:
1) performing feature extraction by adopting a VGG16 model, and simultaneously sending the output of feature maps of different convolution layers to the next link for detection during feature extraction;
2) predicting the class and the coordinates of the object by adopting a series of small convolution modules;
3) performing a loss calculation, the objective loss function being a weighted sum of the position loss and the confidence loss, the objective function being expressed as
Reducing prediction error by loss minimization; in the formula, x represents a training sample, c represents a category confidence degree predicted value, L represents a position predicted value of a boundary box corresponding to a prior frame, g represents a position parameter of a ground channel, N represents the number of positive samples of the prior frame, alpha represents a weight coefficient, and L represents the weight coefficientconf(x, c) represents confidence error, Lloc(x, l, g) represents a position error;
4) the region with the highest confidence and the target is screened out by NMS (Non maximum suppression).
The electroencephalogram decoding adopts a self-adaptive FBCCA (Filter bank canonical correlation analysis) algorithm, and the method combines Bayesian estimation to obtain the optimal data length for prediction on the basis of the traditional FBCCA algorithm, so that the traditional FBCCA algorithm no longer fixes decoding time, the problem of data redundancy caused by personal difference is solved, and the working efficiency of the algorithm is improved. And (3) algorithm decoding:
1) constructing N band-pass filters for filtering;
2) reading EEG data according to periods, and respectively introducing the data into N different band-pass filters to obtain N groups of data after passing through the band-pass filters;
3) performing correlation analysis on each group of data and a reference signal formed by standard sine and cosine by using a CCA algorithm, thereby obtaining N correlation coefficients;
4) according to the formula w (n) ═ n-b+d，n∈[1,N]Solving the harmonic weight and substituting the result into the formula
Solving for rhokAs the frequency f to be foundkNormalizing the maximum correlation coefficient of the EEG signal X, and estimating the correctly predicted posterior probability according to Bayes; in the formula, b and d are constants, w (n) represents corresponding subband weight, n represents subband number, rhokRepresenting the correlation coefficient of the stimulation frequency with the kth target frequency,
representing the corresponding weight n subband coefficients;
5) if the posterior probability reaches the threshold value, the SSVEP prediction frequency is output, if the posterior probability does not reach the threshold value, the data is continuously read until the length of the data segment reaches the preset maximum value, and the SSVEP prediction frequency is forcibly output.
In summary, a system and method for attention training based on target detection and SSVEP includes: camera, brain electricity cap, computer, arm. A camera: the video stream is used for collecting and sending the video stream to the target detection module for target detection; an electroencephalogram cap: the device is used for collecting the SSVEP electroencephalogram signals output by a user during attention training and transmitting the electroencephalogram signals to a computer terminal; a computer: the system comprises an attention training interface, a target detection unit and an SSVEP decoding unit, wherein the target detection unit performs target detection on a video stream acquired by a camera through an SSD target detection algorithm and sends a target detection result to the attention training interface, the attention training interface draws a target frame according to the detection result and generates a flash block corresponding to an article in real time, the SSVEP decoding unit decodes an electroencephalogram signal through preprocessing and an FBCCA algorithm, decodes the flash block watched by a user, converts the decoding result into a control instruction and sends the control instruction to a mechanical arm, and the mechanical arm captures the corresponding article according to a planned path according to the received control instruction. The embodiment can keep the enthusiasm of the children for attention training through natural human-computer interaction, effectively improve the attention level of the children, provide certain practical value and reference significance for future attention training methods and means, and show the good application prospect of the brain-computer interface technology in the fields of child development and the like.
The embodiments of the present invention have been described with reference to the accompanying drawings, but the present invention is not limited to the embodiments, and various changes and modifications can be made according to the purpose of the invention, and any changes, modifications, substitutions, combinations or simplifications made according to the spirit and principle of the technical solution of the present invention shall be equivalent substitutions, as long as the purpose of the present invention is met, and the present invention shall fall within the protection scope of the present invention without departing from the technical principle and inventive concept of the present invention.
Claims (9)
1. The utility model provides an attention training system based on target detection and SSVEP, includes camera, brain electricity cap, computer, arm, attention training interface, target detection unit and SSVEP decoding unit, its characterized in that: the electroencephalogram cap is connected with a computer in a wireless mode, the computer comprises an attention training interface, a target detection unit and an SSVEP decoding unit, and the camera is connected with the computer in a wired mode;
the camera: the attention training interface and the target detection unit are used for acquiring video streams and transmitting the video streams to the computer end;
the electroencephalogram cap comprises: the device is used for collecting the SSVEP electroencephalogram signals output by a user and transmitting the electroencephalogram signals to the computer terminal;
the computer: the system comprises a camera, an attention training interface, an SSVEP decoding unit, an FBCCA algorithm, an adaptive filter bank typical correlation analysis (FBCCA) module, a signal processing module and a control module, wherein the SSVEP decoding unit is used for decoding an electroencephalogram signal through a preprocessing and adaptive filter bank typical correlation analysis (FBCCA) algorithm, acquiring the frequency of the electroencephalogram signal, outputting a decoding result according to the frequency of the electroencephalogram signal and the contrast condition of a stimulation signal corresponding to a command, and sending a control command to a mechanical arm according to the decoding result;
the mechanical arm is: and executing the control command, and grabbing the corresponding article according to the planned path.
2. The target detection and SSVEP-based attention training system of claim 1, wherein: the SSD target detection and identification algorithm comprises two processes of off-line training and on-line identification, wherein the off-line training comprises the following steps:
a-1, constructing a target detection training set;
a-2, inputting the training set into an SSD target detection network for training;
a-3, generating an SSD target detection model;
the online identification process comprises the following steps:
b-1, reading a video stream collected by a camera;
b-2, loading the SSD target detection model which is trained off line;
b-3, preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
b-4, adopting non-maximum value to inhibit and remove the target frame information with overhigh repetition rate;
and b-5, outputting the target information.
3. The target detection and SSVEP-based attention training system of claim 1, wherein: the SSVEP decoding unit decodes the electroencephalogram signal through a pretreatment and adaptive filter bank typical correlation analysis (FBCCA) algorithm, and the specific steps are as follows:
c-1, reading the electroencephalogram data of the user according to a set period;
c-2, decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
c-3, performing correlation analysis on each sub-band component obtained by filtering and a standard sine and cosine reference signal;
c-4, calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and c-5, wherein the frequency corresponding to the maximum correlation is the recognition result.
4. An attention training method based on target detection and SSVEP, which is operated by the attention training system based on target detection and SSVEP as claimed in claim 1, and is characterized in that the operation steps are as follows:
(1) the user wears the electroencephalogram cap and is connected with the camera;
(2) opening an attention training interface, clicking a button for opening a camera, and detecting the attention training interface according to an SSD target detection algorithm;
(3) clicking a training start button;
(4) when the attention training interface has stimulus flicker, a user watches the corresponding flicker block according to voice prompt, the attention training interface draws the stimulus flicker, and meanwhile, the electroencephalogram cap collects electroencephalogram signals and transmits the electroencephalogram signals to the SSVEP decoding unit at the computer end;
(5) the SSVEP decoding unit decodes the electroencephalogram signals through preprocessing and a self-adaptive FBCCA algorithm;
(6) after the SSVEP decoding unit decodes the electroencephalogram signals, a control instruction is sent to the mechanical arm according to the decoding result;
(7) the robot arm picks the corresponding item.
5. The target detection and SSVEP-based attention training method of claim 4, wherein: in the step (2), after the camera button is clicked and opened, the step of detecting the attention training interface according to the SSD target detection algorithm is as follows:
(2-1) reading a video stream captured by a camera;
(2-2) loading an SSD target detection model which is trained offline;
(2-3) preprocessing the picture, and transmitting the preprocessed picture into a network structure to obtain target information;
(2-4) suppressing and removing the target frame information with the excessively high repetition rate by adopting a non-maximum value;
and (2-5) drawing a target frame around each target in the video stream in real time according to the target detection result by the attention training interface, and simultaneously numbering the drawing of the targets.
6. The system and method for attention training based on target detection and SSVEP of claim 5, wherein: in the step (2-3), the detected target information includes a target position and a target name, wherein the target position includes a target frame height, a target frame width, and a center coordinate of the target frame for framing the target.
7. The target detection and SSVEP-based attention training method of claim 4, wherein: in the step (3), after the training start button is clicked, a transparent interface is generated above the video stream block in the attention training interface, the position information of the upper left corner of the target frame is taken as the position of the lower left corner of the flicker block, the transparent interface adds n flicker blocks in real time according to the detection result, and flicker block flicker frequencies of different targets are different.
8. The target detection and SSVEP-based attention training method of claim 4, wherein: in the step (4), the process of drawing the stimulus flicker by the attention training interface includes:
(4-1) the camera acquires a video stream and transmits video stream information to the target recognition module and the attention training interface;
(4-2) identifying the target information in the video stream in real time by the target identification module by using an SSD algorithm;
(4-3) drawing a matched target frame around each target in real time in a video stream by the attention training interface according to the target information, selecting n targets from the frames, and marking the n targets with numbers;
and (4-4) generating a transparent interface above a video stream block in the attention training interface, taking the position information of the upper left corner of the target frame as the position of the lower left corner of the flicker block, adding n flicker blocks in real time by the transparent interface according to the detection result, and enabling the flicker blocks of different targets to have different flicker frequencies.
9. The system and method for attention training based on target detection and SSVEP of claim 4, wherein: in the step (5), the SSVEP decoding unit decodes the electroencephalogram signal through preprocessing and the adaptive FBCCA algorithm as follows:
(5-1) decomposing the SSVEP electroencephalogram signals through a plurality of different pass bands of the filter to obtain sub-band signals passing through each sub-band of the filter;
(5-2) carrying out correlation analysis on each subband component obtained by filtering and a standard sine and cosine reference signal;
(5-3) calculating the maximum correlation coefficient and the sub-maximum correlation coefficient at the corresponding moment to obtain a result estimated by using Bayes;
and (5-4) wherein the frequency corresponding to the maximum correlation is the recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110985594.6A CN114146283A (en) | 2021-08-26 | 2021-08-26 | Attention training system and method based on target detection and SSVEP |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110985594.6A CN114146283A (en) | 2021-08-26 | 2021-08-26 | Attention training system and method based on target detection and SSVEP |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114146283A true CN114146283A (en) | 2022-03-08 |
Family
ID=80460732
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110985594.6A Pending CN114146283A (en) | 2021-08-26 | 2021-08-26 | Attention training system and method based on target detection and SSVEP |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114146283A (en) |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955270A (en) * | 2014-04-14 | 2014-07-30 | 华南理工大学 | Character high-speed input method of brain-computer interface system based on P300 |
| CN104965584A (en) * | 2015-05-19 | 2015-10-07 | 西安交通大学 | Mixing method for brain-computer interface based on SSVEP and OSP |
| US20160287157A1 (en) * | 2013-07-01 | 2016-10-06 | Gregory V. Simpson | Systems and Methods for Assessing and Improving Sustained Attention |
| CN107656612A (en) * | 2017-09-06 | 2018-02-02 | 天津大学 | Big instruction set brain-machine interface method based on P300 SSVEP |
| CN108710913A (en) * | 2018-05-21 | 2018-10-26 | 国网上海市电力公司 | A kind of switchgear presentation switch state automatic identification method based on deep learning |
| CN109366508A (en) * | 2018-09-25 | 2019-02-22 | 中国医学科学院生物医学工程研究所 | A kind of advanced machine arm control system and its implementation based on BCI |
| CN109977774A (en) * | 2019-02-25 | 2019-07-05 | 中国科学技术大学 | A kind of fast target detection method based on adaptive convolution |
| CN110084165A (en) * | 2019-04-19 | 2019-08-02 | 山东大学 | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations |
| CN110399905A (en) * | 2019-07-03 | 2019-11-01 | 常州大学 | The detection and description method of safety cap wear condition in scene of constructing |
| CN111091101A (en) * | 2019-12-23 | 2020-05-01 | 中国科学院自动化研究所 | High-precision pedestrian detection method, system and device based on one-step method |
| CN111571587A (en) * | 2020-05-13 | 2020-08-25 | 南京邮电大学 | Brain-controlled mechanical arm dining assisting system and method |
| CN111571619A (en) * | 2020-04-17 | 2020-08-25 | 上海大学 | Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing |
| CN111738070A (en) * | 2020-05-14 | 2020-10-02 | 华南理工大学 | Automatic accurate detection method for multiple small targets |
| CN111930238A (en) * | 2020-08-27 | 2020-11-13 | 北京理工大学 | Brain-computer interface system implementation method and device based on dynamic SSVEP (secure Shell-and-Play) paradigm |
| CN112008725A (en) * | 2020-08-27 | 2020-12-01 | 北京理工大学 | Human-computer fusion brain-controlled robot system |
| CN112223288A (en) * | 2020-10-09 | 2021-01-15 | 南开大学 | Visual fusion service robot control method |
| CN112287839A (en) * | 2020-10-29 | 2021-01-29 | 广西科技大学 | SSD infrared image pedestrian detection method based on transfer learning |
| CN112307955A (en) * | 2020-10-29 | 2021-02-02 | 广西科技大学 | Optimization method based on SSD infrared image pedestrian detection |
| JP2021083654A (en) * | 2019-11-27 | 2021-06-03 | 学校法人立命館 | Attention function training system, attention function training method, training processing apparatus, and computer program |
| CN113222064A (en) * | 2021-05-31 | 2021-08-06 | 苏州晗林信息技术发展有限公司 | Image target object real-time detection method, system, terminal and storage medium |
| CN113274032A (en) * | 2021-04-29 | 2021-08-20 | 上海大学 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
-
2021
- 2021-08-26 CN CN202110985594.6A patent/CN114146283A/en active Pending
Patent Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160287157A1 (en) * | 2013-07-01 | 2016-10-06 | Gregory V. Simpson | Systems and Methods for Assessing and Improving Sustained Attention |
| CN103955270A (en) * | 2014-04-14 | 2014-07-30 | 华南理工大学 | Character high-speed input method of brain-computer interface system based on P300 |
| CN104965584A (en) * | 2015-05-19 | 2015-10-07 | 西安交通大学 | Mixing method for brain-computer interface based on SSVEP and OSP |
| CN107656612A (en) * | 2017-09-06 | 2018-02-02 | 天津大学 | Big instruction set brain-machine interface method based on P300 SSVEP |
| CN108710913A (en) * | 2018-05-21 | 2018-10-26 | 国网上海市电力公司 | A kind of switchgear presentation switch state automatic identification method based on deep learning |
| CN109366508A (en) * | 2018-09-25 | 2019-02-22 | 中国医学科学院生物医学工程研究所 | A kind of advanced machine arm control system and its implementation based on BCI |
| CN109977774A (en) * | 2019-02-25 | 2019-07-05 | 中国科学技术大学 | A kind of fast target detection method based on adaptive convolution |
| CN110084165A (en) * | 2019-04-19 | 2019-08-02 | 山东大学 | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations |
| CN110399905A (en) * | 2019-07-03 | 2019-11-01 | 常州大学 | The detection and description method of safety cap wear condition in scene of constructing |
| JP2021083654A (en) * | 2019-11-27 | 2021-06-03 | 学校法人立命館 | Attention function training system, attention function training method, training processing apparatus, and computer program |
| CN111091101A (en) * | 2019-12-23 | 2020-05-01 | 中国科学院自动化研究所 | High-precision pedestrian detection method, system and device based on one-step method |
| CN111571619A (en) * | 2020-04-17 | 2020-08-25 | 上海大学 | Life assisting system and method based on SSVEP brain-controlled mechanical arm grabbing |
| CN111571587A (en) * | 2020-05-13 | 2020-08-25 | 南京邮电大学 | Brain-controlled mechanical arm dining assisting system and method |
| CN111738070A (en) * | 2020-05-14 | 2020-10-02 | 华南理工大学 | Automatic accurate detection method for multiple small targets |
| CN111930238A (en) * | 2020-08-27 | 2020-11-13 | 北京理工大学 | Brain-computer interface system implementation method and device based on dynamic SSVEP (secure Shell-and-Play) paradigm |
| CN112008725A (en) * | 2020-08-27 | 2020-12-01 | 北京理工大学 | Human-computer fusion brain-controlled robot system |
| CN112223288A (en) * | 2020-10-09 | 2021-01-15 | 南开大学 | Visual fusion service robot control method |
| CN112287839A (en) * | 2020-10-29 | 2021-01-29 | 广西科技大学 | SSD infrared image pedestrian detection method based on transfer learning |
| CN112307955A (en) * | 2020-10-29 | 2021-02-02 | 广西科技大学 | Optimization method based on SSD infrared image pedestrian detection |
| CN113274032A (en) * | 2021-04-29 | 2021-08-20 | 上海大学 | Cerebral apoplexy rehabilitation training system and method based on SSVEP + MI brain-computer interface |
| CN113222064A (en) * | 2021-05-31 | 2021-08-06 | 苏州晗林信息技术发展有限公司 | Image target object real-time detection method, system, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108304917B (en) | P300 signal detection method based on LSTM network | |
| CN111553307B (en) | Gesture recognition system fusing bioelectrical impedance information and myoelectric information | |
| CN110353702A (en) | A kind of emotion identification method and system based on shallow-layer convolutional neural networks | |
| WO2017084416A1 (en) | Feedback system based on motor imagery brain-computer interface | |
| CN106726030A (en) | Brain machine interface system and its application based on Clinical EEG Signals control machinery hands movement | |
| CN110974212A (en) | Electrocardio and myoelectric characteristic fused rehabilitation training motion state monitoring method and system | |
| CN111930238B (en) | Brain-computer interface system implementation method and device based on dynamic SSVEP (secure Shell-and-Play) paradigm | |
| CN110534180A (en) | The man-machine coadaptation Mental imagery brain machine interface system of deep learning and training method | |
| CN107822629A (en) | The detection method of extremity surface myoelectricity axle | |
| CN107212883A (en) | A kind of mechanical arm writing device and control method based on brain electric control | |
| CN109034015B (en) | FSK-SSVEP demodulation system and demodulation algorithm | |
| CN113598759A (en) | Lower limb action recognition method and system based on myoelectric feature optimization | |
| Li et al. | Learning space-time-frequency representation with two-stream attention based 3D network for motor imagery classification | |
| CN110472595B (en) | Electroencephalogram recognition model construction method and device and recognition method and device | |
| WO2018103495A1 (en) | Eog-based method and apparatus for asynchronous character input | |
| CN116225222A (en) | Brain-computer interaction intention recognition method and system based on lightweight gradient lifting decision tree | |
| CN113974627B (en) | Emotion recognition method based on brain-computer generated confrontation | |
| CN113128353B (en) | Emotion perception method and system oriented to natural man-machine interaction | |
| CN114145744A (en) | Cross-device forehead electroencephalogram emotion recognition method and system | |
| CN114146283A (en) | Attention training system and method based on target detection and SSVEP | |
| Jia | Neural network in the application of EEG signal classification method | |
| CN114098768B (en) | Cross-individual surface electromyographic signal gesture recognition method based on dynamic threshold and EasyTL | |
| CN116019467A (en) | Feature extraction method of facial nerve electromyographic signals | |
| CN115374831A (en) | Dynamic and static combination velocity imagery classification method for multi-modal registration and space-time feature attention | |
| CN114983446A (en) | Finger multi-joint continuous motion estimation method based on electromyographic signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |