CN107168346A - A kind of asynchronous system brain control UAS based on wearable display - Google Patents
A kind of asynchronous system brain control UAS based on wearable display Download PDFInfo
- Publication number
- CN107168346A CN107168346A CN201710298938.XA CN201710298938A CN107168346A CN 107168346 A CN107168346 A CN 107168346A CN 201710298938 A CN201710298938 A CN 201710298938A CN 107168346 A CN107168346 A CN 107168346A
- Authority
- CN
- China
- Prior art keywords
- wearable display
- subelement
- unmanned plane
- eeg
- asynchronous system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004556 brain Anatomy 0.000 title claims abstract description 35
- 230000000007 visual effect Effects 0.000 claims abstract description 58
- 230000004936 stimulating effect Effects 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims description 25
- 230000004438 eyesight Effects 0.000 claims description 24
- 238000000034 method Methods 0.000 claims description 22
- 238000010219 correlation analysis Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 230000000763 evoking effect Effects 0.000 abstract description 14
- 238000002474 experimental method Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 230000000638 stimulation Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 241000251730 Chondrichthyes Species 0.000 description 2
- 210000003710 cerebral cortex Anatomy 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- VZCCETWTMQHEPK-QNEBEIHSSA-N gamma-linolenic acid Chemical compound CCCCC\C=C/C\C=C/C\C=C/CCCCC(O)=O VZCCETWTMQHEPK-QNEBEIHSSA-N 0.000 description 2
- 230000009931 harmful effect Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 230000036624 brainpower Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000000869 occipital lobe Anatomy 0.000 description 1
- 230000001769 paralizing effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000012266 salt solution Substances 0.000 description 1
- 210000004761 scalp Anatomy 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 235000013311 vegetables Nutrition 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 238000009736 wetting Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Theoretical Computer Science (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Dermatology (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a kind of asynchronous system brain control UAS based on wearable display, one group of visual stimulus unit flashed with different frequency is presented using wearable display, the a different set of operational order of unmanned plane is corresponded to respectively, user watches different visual stimulus units attentively, resulting EEG signals are processed and identified the stimulating unit watched attentively by user and its corresponding operational order, recognition result, which is sent toward unmanned generator terminal, to be performed, the real time scene picture of unmanned plane is transferred back to ground, it is superimposed with visual stimulus unit, it is shown in wearable display, the portability of Steady State Visual Evoked Potential brain machine interface system can be improved by providing first person feedback to user.The present invention also makes full use of Idle state to decode, and can largely reduce and control the operating burden and sense of fatigue of flight by giving user more free flight experience using brain-computer interface.
Description
Technical field
The invention belongs to biomedical engineering and interleaving techniques field is automatically controlled, in particular it relates to a kind of stable state vision
Evoked ptential presentation mode and the brain-computer interface asynchronous system control unmanned plane during flying system based on this, it is especially a kind of to be based on wearing
Wear the asynchronous system brain control UAS of display.
Background technology
Brain-computer interface is a kind of technology interacted independent of the i.e. achievable people of externus muscle and the external world.Inconvenient row can be helped
Dynamic, paralysis, paralytic are re-established to be contacted with the external world, is expressed the wish of oneself, is aided in daily life, asynchronous system brain
Machine connects password system and follows the wish of patient to express, rather than patient's system for tracking order, with very big use value and
Development prospect.Conventional brain power mode has Mental imagery, Steady State Visual Evoked Potential, P300 etc..
Steady State Visual Evoked Potential is that, when human eye receives the visual stimulus of certain frequency, brain occipital lobe area is induced to produce
Response signal identical with frequency of stimulation or into frequency multiplication relation, the signal can be identified and translated to corresponding control instruction and use
To control external equipment.Brain machine interface system based on Steady State Visual Evoked Potential due to its stability, few training, can carry
The advantages of control instruction of confession is more and only needs simple data processing method that higher rate of information transmission can be achieved are extensive
Research, the existing brain machine interface system based on Steady State Visual Evoked Potential generally using LED board/LCD/CRT screens as regarding
Feel the presentation means stimulated, cause total system to seem very too fat to move, not readily portable, the operating experience to user is also inadequate
Immerse, the practicality to system causes harmful effect.
Unmanned plane is widely studied as a study hotspot instantly, and huge potentiality in practicality, especially
When some external equipments of UAV flight are such as mechanical arm, camera, all kinds of manipulation devices, it is by for handicapped crowd
Very big help is brought, many remote-operated life requirements of handicapped crowd is met, for example, gos out and explore the way, buy vegetables, is transported
Thing.But existing unmanned plane is typically by the way of remote control or flight ground software, for the handicapped people of opponent's pin
It is not easy-to-use, and they often have more urgent demand than abled person.
" the unmanned plane control method based on Steady State Visual Evoked Potential " of Southeast China University and " the one of the China Measures Institute
Flying vehicles control device of the kind based on Steady State Visual Evoked Potential " is proposed respectively controls nobody using Steady State Visual Evoked Potential
The method and apparatus of machine flight, but wherein visual stimulus unit is still located at a certain distance from user front, be have impact on and is used
The sense of participation of person；In addition, user control a direction flight course in, it is necessary to the square persistently be watched attentively, because stable state is regarded
Feel induce stimulating unit sense of fatigue can be brought to user in itself, watch attentively for a long time by user bring very strong flickering and
Sense of fatigue, reduce system ease for use and user to the degree of recognition of system, in the course of time, user will increasingly be unwilling
Use system.
The content of the invention
For above shortcomings in the prior art, the present invention is intended to provide a kind of based on the asynchronous of wearable display
Formula Steady State Visual Evoked Potential brain control UAS, to effectively improve the portability of system, and reduces the fatigue of user
Sense and operating burden, finally to improve the practicality of system.
It is of the invention specific using following technical scheme solution above-mentioned technical problem.
A kind of asynchronous system brain control UAS based on wearable display, including wearable display and be equipped on
Stable state vision inducting stimulating unit, eeg signal acquisition unit and EEG Processing unit in wearable display；Its
In：
The stable state vision inducting stimulating unit induces generation EEG signals by visual stimulus；
The eeg signal acquisition unit is used to gather EEG signals and send to EEG Processing unit to carry out in real time
Processing is set, and recognition result is sent to unmanned plane.
Preferably, the stable state vision inducting stimulating unit includes multiple visual stimulus son lists with different flicker frequencies
Member.
Preferably, multiple visual stimulus subelements are respectively：Subelement, upward subelement, downward subelement, left-hand rotation forward
Subelement and right-hand rotation subelement, correspondingly, each visual stimulus subelement correspond respectively to forward, upwards, downwards, turn left,
Right-hand rotation control instruction.
Preferably, backward control instruction by continuously forward, upwards, downwards, turn left and/or right-hand rotation control instruction is real
It is existing.
Preferably, the subelement forward, left-hand rotation subelement, right-hand rotation subelement, upward subelement and downward subelement
Flicker frequency is respectively 10.7Hz, 9.37Hz, 8.33Hz, 12.5Hz and 7.5Hz.
Preferably, the eeg signal acquisition unit includes portable brain electric collecting device and wireless transport module；Wherein：
The portable brain electric collecting device is any passage for the EEG signals that can gather the generation of cerebral cortex occipital region
Several brain wave acquisition equipment, for gathering EEG signals；
The wireless transport module is used to send the EEG signals collected to EEG Processing unit.
Preferably, the EEG Processing unit includes classification processing module and Classification and Identification module；Wherein：
The classification processing module is used to carry out feature extraction and classification to the EEG signals received；
The Classification and Identification module is used to sorted EEG signals are identified, and obtained recognition result is sent to nothing
It is man-machine.
Preferably, the classification processing module carries out feature extraction and classification using Canonical Correlation Analysis.
Preferably, classification processing module is to eeg signal classification：Idle state and operating conditions；Wherein：
When user watches stable state vision inducting stimulating unit attentively, its EEG signals is classified as operating conditions；
When user from stable state vision inducting stimulating unit removes sight, its EEG signals will be classified as Idle state.
Preferably, the Classification and Identification module is electric to sorted brain using the Idle state recognition methods based on threshold test
Signal is identified, and when the ratio between maximum correlation coefficient and time big coefficient correlation are less than or equal to given threshold, is judged as Idle state,
Otherwise it is operating conditions；Heading under operating conditions corresponding to the corresponding flicker frequency of maximum correlation coefficient is defeated as recognition result
Go out to unmanned plane, the flight side that the recognition result of unmanned plane is still translated under the last operating conditions is sent under Idle state
To.
Preferably, the asynchronous system brain control UAS based on wearable display, in addition to real time scene collection
And delivery unit, the real time scene is gathered and delivery unit is equipped on unmanned plane, and the real-time imaging for gathering unmanned plane is drawn
Face is simultaneously sent to wearable display；The real-time imaging picture is superimposed with stable state vision inducting stimulating unit, and being shown in wear
Wear display.
Preferably, the real time scene collection and delivery unit include camera and video frequency collection card；Wherein：
The camera is gathered for unmanned plane during flying video pictures；
The video frequency collection card is used to the unmanned plane during flying video pictures collected being wirelessly transmitted to wearable display.
Compared with prior art, the invention has the advantages that：
1st, traditional brain machine interface system based on Steady State Visual Evoked Potential is generally made using LED board/LCD/CRT screens
For the presentation means of visual stimulus, total system is caused to seem very too fat to move, not readily portable, the operating experience to user
Not enough immerse, the practicality to system causes harmful effect.Display means proposed by the present invention, stimulating unit and vision is anti-
Feedback picture is shown on wearable display, and one group has the visual stimulus unit external rings that different frequency flashes around feedback picture
Face, does not block the feedback picture visual field.User wears the display, you can manipulate unmanned plane immersion, understands current flight
Environment.
2nd, in traditional brain-computer interface control UAS, user in control a direction flight course, it is necessary to
Persistently watch the square attentively, because stable state vision inducting stimulating unit can bring sense of fatigue to user in itself, watch attentively for a long time by
Bring very strong flickering and sense of fatigue to user, reduce system ease for use and user to the degree of recognition of system, long
And long it, user will increasingly be unwilling to use system.In manipulation normal form proposed by the present invention, user need to only fly in switching
Watch visual stimulus unit during line direction attentively, unmanned plane during flying state can be kept without persistently watching visual stimulus unit attentively, reduce
The operating burden and sense of fatigue of user, and user also Z can be continuously maintained the grasp to unmanned plane during flying state, Bu Huiyin
Have no time to pay close attention to flying scene feedback persistently to watch stimulating unit attentively, and also can more be absorbed in when being look at stimulating unit, no
Need persistently to take sb's mind off sth into flying scene, be conducive to the lifting of systematic entirety energy.
3rd, traditional unmanned plane control method is, it is necessary to the operation of both hands, control method proposed by the present invention, it is only necessary to use
Person carries out the switching in spatial attention, the method that control unmanned plane is provided for the disabled, can help their life,
Make the life better quality.
Brief description of the drawings
By reading the detailed description made with reference to the following drawings to non-limiting example, further feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the system schematic of the preferred embodiment of the present invention；
Fig. 2 is that stable state vision inducting stimulating unit feeds back display signal of the picture in wearable display with real-time scene
Figure；
Placement location figures of the Fig. 3 for the electrode in portable brain electric collecting device on scalp；
Fig. 4 is asynchronous controlling normal form principle flow chart.
Embodiment
Embodiments of the invention are elaborated below：The present embodiment is carried out lower premised on technical solution of the present invention
Implement, give detailed embodiment and specific operating process.It should be pointed out that to one of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention
Protect scope.
Embodiment
The asynchronous system brain control UAS based on wearable display that the present embodiment is provided, using stable state vision inducting
Stimulating unit (current potential), one group of visual stimulus subelement flashed with different frequency is presented using wearable display, right respectively
The a different set of operational order of unmanned plane is answered, user watches different visual stimulus subelements, resulting brain telecommunications attentively
Number processed and identify the stimulation subelement and its corresponding operational order watched attentively by user, recognition result is sent toward nothing
People's generator terminal is performed, and the real time scene picture of unmanned plane is transferred back to ground, is superimposed, is shown in stable state vision inducting stimulating unit
Wearable display, first person feedback is provided to user；The normal form of flight is also controlled by designing a kind of asynchronous system
To reduce using brain-computer interface control flight operating burden and sense of fatigue by.
Further, the stable state vision inducting stimulating unit and visual feedback picture are shown in wearable display
On, described one group has the visual stimulus subelement external rings that different frequency flashes around feedback picture not influence user couple
The grasp of real-time flight scene and state, user wears the display, you can manipulate unmanned plane immersion, understands current fly
Row environment.
Further, it is the higher discrimination of guarantee and stability, and ensures that user can be to visual feedback picture
Have in complete viewing experience, the present embodiment, visual stimulus subelement there are five kinds：Forward subelement, upward subelement, downwards son
Unit, left-hand rotation subelement, right-hand rotation subelement, corresponding to five kinds of conventional control instructions：Forward control instruction, upward control instruction,
Downward control instruction, left-hand rotation control instruction, right-hand rotation control instruction, can realize the control to no-manned plane three-dimensional space flight.Such as
Backward, it can for example turn left and realize twice by continuous five kinds of control instruction solid lines.
Further, it is the portability of raising UAS, eeg signal acquisition unit is set using portable collection
Standby (portable brain electric collecting device).
Preferably, EEG Processing unit carries out feature extraction when being classified using Canonical Correlation Analysis
And classification, when Idle state and operating conditions are identified, using the very useful Idle state recognition methods based on threshold test,
When the ratio between maximum correlation coefficient and time big coefficient correlation are less than or equal to certain threshold value, Idle state is judged to, is otherwise operating conditions, is worked
Heading under state corresponding to the corresponding frequency of stimulation of maximum correlation coefficient is classification results.
Further, for reduce user operating burden and sense of fatigue by the UAS, user only needs
Watch visual stimulus unit attentively when switching heading, unmanned plane during flying shape can be kept without persistently watching visual stimulus unit attentively
State.(1) after user watches certain stimulating unit attentively and is identified, aircraft flies towards this direction；(2) user can be from stimulating unit
Sight is removed, visual feedback picture is turned to, state of flight is held, unmanned plane will persistently keep flying to this direction during being somebody's turn to do；
(3) after, user can watch other stimulating units attentively according to wish combination state of flight, and sight can be removed after being identified, weight
Multiple above-mentioned (1) (2) process.Meanwhile, user is not limited to this process, and user wants to realize when being switched fast between direction,
(2) process can still be omitted.The asynchronous system control normal form implementation method is that user watches stimulating unit attentively, its EEG signals quilt
It is categorized as the corresponding heading of the unit under working condition；User removes sight from some visual stimulus unit and turns to vision
Feed back after picture, its EEG signals will be classified as idle condition, but be sent to the order at aircraft end and be still translated into nearest one
Heading under task state.When next EEG signals are resolved as other headings, the instruction of aircraft is sent to
It is changed into other headings.
Further, unmanned plane during flying video pictures are gathered by the camera being mounted on unmanned plane, via video acquisition
Card is wirelessly transferred back wearable display.
The present invention is more fully described hereinafter with reference to accompanying drawing, various performances are shown in the drawings.
The various problems present in unmanned plane are controlled for existing Steady State Visual Evoked Potential, as shown in figure 1, the present invention
Resolving ideas is to control unmanned plane during flying using wearable display and the new control normal form of design, is specially：Use
One group of visual stimulus subelement flashed with different frequency is presented in wearable display, and a different set of of unmanned plane is corresponded to respectively
Operational order, user watches different visual stimulus units attentively, and resulting EEG signals are processed and identified and used
Stimulating unit and its corresponding operational order that person watches attentively, recognition result by send toward unmanned generator terminal perform, unmanned plane it is real-time
Scene picture is transferred back to ground, is superimposed with visual stimulus subelement, is shown in wearable display, and first is provided to user
People claims visual angle to feed back；A kind of normal form of asynchronous system control flight is have also been devised simultaneously, makes full use of Idle state to decode, to reduce
The operating burden and sense of fatigue of flight are controlled using brain-computer interface.
Embodiments of the invention and its detailed process are as follows：
As shown in Fig. 2 the wearable display that the present embodiment is used can be Fat Shark FSV1074 HD2, it
Eyeglass be the lcd screen of two 800 × 600, it is allowed to visual is developed in glasses, external equipment can also be received
The image transmitted, support is connected projection with computer HDMI wire.The present embodiment occupation mode can be drawn using OPEN GL storehouses
Visual stimulus unit, and the picture that unmanned plane is sent back is plotted in wearable display together.
As shown in Fig. 2 in the present embodiment, five visual stimulus units are from peripheral ring around real time scene interactive picture, five
Visual stimulus frequency (flicker frequency) can be 10.7Hz, 9.37Hz, 8.33Hz, 12.5Hz, 7.5Hz respectively, correspondence forward, it is left
Turn, turn right, upward, downward five headings, operating personnel can watch attentively according to the intention of oneself selection corresponding stimulates single
Member.
Use of the present invention can gather the brain wave acquisition equipment of any port number of the signal of cerebral cortex occipital region generation.This
Embodiment can use Portable acquiring equipment Emotiv Epoc, and it is the wireless EEG collections of a reference channel of 14 passage+2
Device, 12h is up to during using wireless blue tooth technology and continuation of the journey, chargeable, the sensor of salt solution or clear water wetting, without stickiness
Conductive paste.As shown in figure 3, electrode mainly chooses P3, P4,01,02 position in embodiment.
In the present embodiment, in data transfer, control instruction to aircraft end can be led to using serial ports+XBee wireless data sendings mode
Letter, real time scene picture is returned to wearable display and communicated using wireless image transmission module+UVC video frequency collection card modes.
In the present embodiment, EEG Processing module can be a Win32 application program write with Visual C++,
With data receiver, preserve, online processing instructs sending function.When handling EEG signals, 5-64Hz band logical filters are first carried out
Ripple, then long with 1s windows, the computing speed of 0.125s sliding window length carries out canonical correlation analysis to EEG signals, so as to complete spy
Extraction is levied, wherein in canonical correlation analysis algorithm, reference signal is as the harmonious wave component of the fundamental wave corresponding to frequency of stimulation, often
One frequency of stimulation and EEG signals can all export maximum correlation coefficient by correlation analysis, and the present embodiment, which has five, stimulates frequency
Rate (flicker frequency), therefore produce five maximum correlation coefficients.As shown in figure 4, the once differentiation of state available free first and operating conditions,
In five maximum correlation coefficients, maximum one and time big ratio of one are then considered as Idle state if less than certain threshold value,
Under Idle state, output order is mapped as the instruction of the last working condition, then can realize that holding original direction continues to fly
OK.Operating conditions is such as determined as, then exports a corresponding flight directive maximum in five maximum correlation coefficients.
In the present embodiment, to ensure the stability of unmanned plane during flying, refer to when EEG Processing module detects operating conditions
During order switching, unmanned plane will be flown in the 1s that instruction has just switched with slower speed along the direction, and be ignored in this 1s
It is transmitted through after other instructions come, 1s, no matter user continues faster to fly or watch attentively towards this direction with Idle state manipulation unmanned plane
Other stimulate switching heading, and this 1s buffer time contributes to unmanned plane to keep stable when switching between classification.
In the present embodiment, ARDrone unmanned planes can be used, aircraft termination receives control instruction, according to control instruction certainly
Dynamic adjustment posture realizes the flight of correspondence direction.Meanwhile, the camera carried on machine is by wireless image transmission module by real-time pictures
Wearable display is sent back, wearable display screen receives picture and drawn.
In the present embodiment, the specific implementation process manipulated using the system to unmanned plane is as follows：
(1) five healthy subjects, label S1 to S5 are chosen.
(2) preliminary experiment：Each subject receives 1 preliminary experiment altogether, and every time in experiment, subject first wears brain wave acquisition and set
Standby Emotiv Epoc and wearable display Fat shark FSV1074 HD2, then watch different regard attentively successively according to prompting
Feeling stimulates square 5s, and each square is pointed out at random, in addition, can also be prompted Idle state, now subject should be watched attentively in picture
Centre, feedback picture is replaced with a still image in this experiment, every time the common 240s of experiment.
(3) during preliminary experiment, EEG Processing module is under Win32 application programs preserve this Experiment Data Records
Come, after experiment terminates, the data of preservation are filtered and classified offline by the module, and automatic searching is most closed to every subject
Suitable Idle state detection threshold value.
(4) unmanned plane manipulation experiment：Subject manipulates unmanned plane during flying according to the wish of oneself, now EEG Processing
Module is according to the online treatment classification subject EEG signals of the optimal threshold obtained in (3), and aircraft end is arrived in output control instruction,
Real-time Feedback picture is passed back to subject in aircraft end.Subject can first watch square forward attentively, then aircraft starts slowly to fly forward
1s, if subject thinks：A. continue to keep flight forward, then can not watch any stimulation attentively, observation feedback picture, unmanned plane will with compared with
Fast speed keeps advancing；B. heading is changed, such as turns right, then watches right-hand rotation square attentively, aircraft will be with slower speed right-hand rotation 1s；c.
By that analogy.
In the present embodiment：
The present embodiment provides a kind of asynchronous system Steady State Visual Evoked Potential based on wearable display and controls nobody
Machine system, including：One group of stable state vision inducting stimulating unit is presented on wearable display with first person feedback picture
On device, user wears the display and selection is watched the corresponding visual stimulus unit of a certain heading attentively and can induced accordingly
EEG signals, the signal handles and recognized by signal processing module, and recognition result is transferred into unmanned plane, and unmanned plane can be by
Wish according to user is flown towards corresponding heading, and the real time scene picture of unmanned plane is transferred back to ground, with vision
Stimulating unit is superimposed, and is shown in wearable display.The also a kind of asynchronous system for making full use of Idle state to decode control flight
Normal form.
Stable state vision inducting stimulating unit and visual feedback picture are shown on wearable display, and user wears this
Display, you can manipulate unmanned plane, understands current flight environment immersion.
Described one group has the visual stimulus subelement external rings that different frequency flashes around feedback picture, different visions
Stimulate subelement：Forward subelement, upward subelement, downward subelement, left-hand rotation subelement, right-hand rotation subelement arrow it is right respectively
Answer unmanned plane forward, upwards, downwards, turn left, right-hand rotation heading.
The EEG signals are obtained by Portable acquiring electrode (Portable acquiring equipment) and radio to signal transacting
Resume module, the EEG Processing process mainly includes：(1) bandpass filtering；(2) the classification side based on canonical correlation analysis
Method；And the Idle state recognition methods of (3) based on threshold test, when the ratio between maximum correlation coefficient and time big coefficient correlation are less than etc.
When certain threshold value, Idle state is judged to, is otherwise operating conditions, under operating conditions corresponding to the corresponding frequency of stimulation of maximum correlation coefficient
Heading is classification results, and the result is wirelessly communicated to unmanned plane.
In the UAS, user only need to watch visual stimulus unit attentively when switching heading, without continuing
Unmanned plane during flying state can be kept by watching visual stimulus unit attentively.(1) after user watches certain stimulating unit attentively and is identified, aircraft
Towards the flight of this direction；(2) user can remove sight from stimulating unit, turn to visual feedback picture, hold state of flight, should
During unmanned plane will persistently keep to this direction fly；(3) after, user can watch it attentively according to wish combination state of flight
His stimulating unit, it is identified after can remove sight, repeat above-mentioned (1) (2) process.Meanwhile, user is not limited to this mistake
Journey, user wants to realize when being switched fast between direction, can still omit (2) process.
The asynchronous system control normal form implementation method is Idle state detection method as described above, and it is single that user watches stimulation attentively
Member, its EEG signals are classified as the corresponding heading of the unit under operating conditions；User moves from some visual stimulus unit
Open sight to turn to after visual feedback picture, its EEG signals will be classified as Idle state, but be sent to the order at aircraft end still by
It is translated as the heading under the last operating conditions.When next EEG signals are resolved as other headings, it is sent to
The instruction of aircraft is changed into other headings.
The unmanned plane during flying video pictures are gathered by the camera being mounted on unmanned plane, wireless via video frequency collection card
It is transmitted back to wearable display.
The present embodiment provides first person feedback to user can improve Steady State Visual Evoked Potential brain-computer interface
The portability of system, also makes full use of Idle state to decode, and can largely reduce the behaviour that flight is controlled using brain-computer interface
Make burden and sense of fatigue by giving user more free flight experience.
The specific embodiment of the present invention is described above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make various deformations or amendments within the scope of the claims, this not shadow
Ring the substantive content of the present invention.
Claims (10)
1. a kind of asynchronous system brain control UAS based on wearable display, it is characterised in that including wearable display
And it is equipped on stable state vision inducting stimulating unit, eeg signal acquisition unit and EEG Processing in wearable display
Unit；Wherein：
The stable state vision inducting stimulating unit induces generation EEG signals by visual stimulus；
The eeg signal acquisition unit is used to gather EEG signals and send to EEG Processing unit to be handled in real time
Set, and recognition result is sent to unmanned plane.
2. the asynchronous system brain control UAS according to claim 1 based on wearable display, it is characterised in that institute
Stating stable state vision inducting stimulating unit includes multiple visual stimulus subelements with different flicker frequencies.
3. the asynchronous system brain control UAS according to claim 2 based on wearable display, it is characterised in that many
Individual visual stimulus subelement is respectively：Subelement, upward subelement, downward subelement, left-hand rotation subelement and right rotor list forward
Member, correspondingly, each visual stimulus subelement correspond respectively to forward, upwards, downwards, turn left, right-hand rotation control instruction；
Also include control instruction backward, the control instruction backward by continuously forward, upwards, downwards, turn left and/or turn right
Control instruction is realized.
4. the asynchronous system brain control UAS according to claim 2 based on wearable display, it is characterised in that institute
The flicker frequency for stating subelement, left-hand rotation subelement, right-hand rotation subelement, upward subelement and downward subelement forward is respectively
10.7Hz, 9.37Hz, 8.33Hz, 12.5Hz and 7.5Hz.
5. the asynchronous system brain control UAS according to claim 1 based on wearable display, it is characterised in that institute
Stating eeg signal acquisition unit includes portable brain electric collecting device and wireless transport module；Wherein：
The portable brain electric collecting device is used to gather EEG signals；
The wireless transport module is used to send the EEG signals collected to EEG Processing unit.
6. the asynchronous system brain control UAS according to claim 1 based on wearable display, it is characterised in that institute
Stating EEG Processing unit includes classification processing module and Classification and Identification module；Wherein：
The classification processing module is used to carry out feature extraction and classification to the EEG signals received；
The Classification and Identification module is used to sorted EEG signals are identified, and obtained recognition result is sent to nobody
Machine.
7. the asynchronous system brain control UAS according to claim 6 based on wearable display, it is characterised in that institute
State classification processing module and feature extraction and classification are carried out using Canonical Correlation Analysis；
It is described classification processing module be to eeg signal classification：Idle state and operating conditions；Wherein：
When user watches stable state vision inducting stimulating unit attentively, its EEG signals is classified as operating conditions；
When user from stable state vision inducting stimulating unit removes sight, its EEG signals will be classified as Idle state.
8. the asynchronous system brain control UAS according to claim 1 based on wearable display, it is characterised in that institute
State Classification and Identification module sorted EEG signals are identified using the Idle state recognition methods based on threshold test, when most
When the ratio between big coefficient correlation and time big coefficient correlation are less than or equal to given threshold, it is judged as Idle state, is otherwise operating conditions；Work
Heading under state corresponding to the corresponding flicker frequency of maximum correlation coefficient is exported to unmanned plane, Idle state as recognition result
Under be sent to the heading that the recognition result of unmanned plane is still translated under the last operating conditions.
9. the asynchronous system brain control UAS according to any one of claim 1 to 8 based on wearable display, its
It is characterised by, in addition to real time scene collection and delivery unit, the real time scene is gathered and delivery unit is equipped on unmanned plane,
For gathering the real-time imaging picture of unmanned plane and being sent to wearable display；The real-time imaging picture is lured with stable state vision
Stimulating unit superposition is sent out, wearable display is shown in.
10. the asynchronous system brain control UAS according to claim 9 based on wearable display, it is characterised in that
The real time scene collection and delivery unit include camera and video frequency collection card；Wherein：
The camera is gathered for unmanned plane during flying video pictures；
The video frequency collection card is used to the unmanned plane during flying video pictures collected being wirelessly transmitted to wearable display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710298938.XA CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710298938.XA CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107168346A true CN107168346A (en) | 2017-09-15 |
Family
ID=59812424
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710298938.XA Pending CN107168346A (en) | 2017-04-28 | 2017-04-28 | A kind of asynchronous system brain control UAS based on wearable display |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107168346A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108196566A (en) * | 2018-03-16 | 2018-06-22 | 西安科技大学 | A kind of small drone cloud brain control system and its method |
| CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
| CN108897418A (en) * | 2018-05-15 | 2018-11-27 | 天津大学 | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method |
| CN109009887A (en) * | 2018-07-17 | 2018-12-18 | 东北大学 | A kind of man-machine interactive navigation system and method based on brain-computer interface |
| CN109521784A (en) * | 2018-12-13 | 2019-03-26 | 华南农业大学 | A kind of wearable upper limb ectoskeleton unmanned aerial vehicle control system of tactilely-perceptible formula and method |
| CN110502101A (en) * | 2019-05-29 | 2019-11-26 | 中国人民解放军军事科学院军事医学研究院 | Virtual reality exchange method and device based on eeg signal acquisition |
| CN110597201A (en) * | 2019-09-05 | 2019-12-20 | 佛山市云米电器科技有限公司 | Intelligent household system controlled by AR equipment and control method thereof |
| CN110658810A (en) * | 2019-08-27 | 2020-01-07 | 北京航空航天大学 | Individual combat unmanned weapon control system based on SSVEP brain-computer interface |
| CN111487988A (en) * | 2020-03-03 | 2020-08-04 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
| CN112034979A (en) * | 2020-07-31 | 2020-12-04 | 西安交通大学 | Wearable flight sensation feedback system based on force feedback |
| CN112114662A (en) * | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
| CN113009931A (en) * | 2021-03-08 | 2021-06-22 | 北京邮电大学 | Man-machine and unmanned-machine mixed formation cooperative control device and method |
| WO2023173323A1 (en) * | 2022-03-16 | 2023-09-21 | 深圳市大疆创新科技有限公司 | Power consumption control method, apparatus and system for unmanned aerial vehicle, and storage medium |
| CN117908674A (en) * | 2024-01-11 | 2024-04-19 | 北京理工大学 | Brain-computer interface and brain-computer control method for brain-controlled air-ground cooperative unmanned system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955269A (en) * | 2014-04-09 | 2014-07-30 | 天津大学 | Intelligent glass brain-computer interface method based on virtual real environment |
| CN105204517A (en) * | 2015-09-24 | 2015-12-30 | 杨珊珊 | Personal service method and system for small and mini-type unmanned aerial vehicles |
| CN106371451A (en) * | 2016-11-07 | 2017-02-01 | 东南大学 | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential |
| WO2017020763A1 (en) * | 2015-07-31 | 2017-02-09 | 广州极飞科技有限公司 | Unmanned aerial vehicle |
-
2017
- 2017-04-28 CN CN201710298938.XA patent/CN107168346A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955269A (en) * | 2014-04-09 | 2014-07-30 | 天津大学 | Intelligent glass brain-computer interface method based on virtual real environment |
| WO2017020763A1 (en) * | 2015-07-31 | 2017-02-09 | 广州极飞科技有限公司 | Unmanned aerial vehicle |
| CN105204517A (en) * | 2015-09-24 | 2015-12-30 | 杨珊珊 | Personal service method and system for small and mini-type unmanned aerial vehicles |
| CN106371451A (en) * | 2016-11-07 | 2017-02-01 | 东南大学 | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential |
Non-Patent Citations (1)
| Title |
|---|
| 王行愚 等: "脑控:基于脑-机接口的人机融合控制", 《自动化学报》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
| CN108196566A (en) * | 2018-03-16 | 2018-06-22 | 西安科技大学 | A kind of small drone cloud brain control system and its method |
| CN108897418A (en) * | 2018-05-15 | 2018-11-27 | 天津大学 | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method |
| CN109009887A (en) * | 2018-07-17 | 2018-12-18 | 东北大学 | A kind of man-machine interactive navigation system and method based on brain-computer interface |
| CN109521784B (en) * | 2018-12-13 | 2021-05-11 | 华南农业大学 | Touch sensing type wearable upper limb exoskeleton unmanned aerial vehicle control system and method |
| CN109521784A (en) * | 2018-12-13 | 2019-03-26 | 华南农业大学 | A kind of wearable upper limb ectoskeleton unmanned aerial vehicle control system of tactilely-perceptible formula and method |
| CN110502101A (en) * | 2019-05-29 | 2019-11-26 | 中国人民解放军军事科学院军事医学研究院 | Virtual reality exchange method and device based on eeg signal acquisition |
| CN110658810A (en) * | 2019-08-27 | 2020-01-07 | 北京航空航天大学 | Individual combat unmanned weapon control system based on SSVEP brain-computer interface |
| CN110597201A (en) * | 2019-09-05 | 2019-12-20 | 佛山市云米电器科技有限公司 | Intelligent household system controlled by AR equipment and control method thereof |
| CN111487988A (en) * | 2020-03-03 | 2020-08-04 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
| CN111487988B (en) * | 2020-03-03 | 2022-04-15 | 天津大学 | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface |
| CN112034979A (en) * | 2020-07-31 | 2020-12-04 | 西安交通大学 | Wearable flight sensation feedback system based on force feedback |
| CN112114662A (en) * | 2020-08-03 | 2020-12-22 | 西安交通大学 | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method |
| CN113009931A (en) * | 2021-03-08 | 2021-06-22 | 北京邮电大学 | Man-machine and unmanned-machine mixed formation cooperative control device and method |
| CN113009931B (en) * | 2021-03-08 | 2022-11-08 | 北京邮电大学 | Man-machine and unmanned-machine mixed formation cooperative control device and method |
| WO2023173323A1 (en) * | 2022-03-16 | 2023-09-21 | 深圳市大疆创新科技有限公司 | Power consumption control method, apparatus and system for unmanned aerial vehicle, and storage medium |
| CN117908674A (en) * | 2024-01-11 | 2024-04-19 | 北京理工大学 | Brain-computer interface and brain-computer control method for brain-controlled air-ground cooperative unmanned system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107168346A (en) | A kind of asynchronous system brain control UAS based on wearable display | |
| CN106292705B (en) | Multi-rotor unmanned aerial vehicle idea remote control system and operating method based on Bluetooth brain wave earphone | |
| CN113398422B (en) | Rehabilitation training system and method based on motor imagery-brain-computer interface and virtual reality | |
| CN104997581B (en) | Artificial hand control method and apparatus for driving EEG signals on the basis of facial expressions | |
| CN109366508A (en) | A kind of advanced machine arm control system and its implementation based on BCI | |
| CN108762303A (en) | A kind of portable brain control UAV system and control method based on Mental imagery | |
| CN106371451A (en) | Unmanned aerial vehicle manipulation method and device based on steady state visual evoked potential | |
| CN106671084A (en) | Mechanical arm self-directed auxiliary system and method based on brain-computer interface | |
| CN111110982A (en) | Hand rehabilitation training method based on motor imagery | |
| CN104799984A (en) | Assistance system for disabled people based on brain control mobile eye and control method for assistance system | |
| CN110262658B (en) | Brain-computer interface character input system based on enhanced attention and implementation method | |
| CN111584031B (en) | Brain-controlled intelligent limb rehabilitation system based on portable electroencephalogram acquisition equipment and application | |
| CN106491251B (en) | Non-invasive brain-computer interface-based robot arm control system and control method thereof | |
| CN103294192A (en) | LED lamp switch control device and control method thereof based on motor imagery | |
| CN110716578A (en) | Aircraft control system based on hybrid brain-computer interface and control method thereof | |
| CN109009887A (en) | A kind of man-machine interactive navigation system and method based on brain-computer interface | |
| CN108646915A (en) | The method and system of object is captured in conjunction with three-dimensional eye tracking and brain-computer interface control machinery arm | |
| CN107480716A (en) | Method and system for identifying saccade signal by combining EOG and video | |
| CN112114662A (en) | Reality-augmented self-adaptive dynamic multi-scene evoked brain control method | |
| CN106267557A (en) | A kind of brain control based on wavelet transformation and support vector machine identification actively upper limb medical rehabilitation training system | |
| CN111930238A (en) | Brain-computer interface system implementation method and device based on dynamic SSVEP (secure Shell-and-Play) paradigm | |
| CN108897418A (en) | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method | |
| CN204631770U (en) | The efficient user's trainer of a kind of brain-computer interface for Mental imagery brain electricity | |
| Li et al. | An adaptive P300 model for controlling a humanoid robot with mind | |
| CN113359991A (en) | Intelligent brain-controlled mechanical arm auxiliary feeding system and method for disabled people |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170915 |