CN113625769B - Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals - Google Patents
Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals Download PDFInfo
- Publication number
- CN113625769B CN113625769B CN202111044955.3A CN202111044955A CN113625769B CN 113625769 B CN113625769 B CN 113625769B CN 202111044955 A CN202111044955 A CN 202111044955A CN 113625769 B CN113625769 B CN 113625769B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- control
- electroencephalogram
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000015572 biosynthetic process Effects 0.000 title claims abstract description 41
- 210000004556 brain Anatomy 0.000 claims abstract description 32
- 230000000007 visual effect Effects 0.000 claims abstract description 32
- 230000000638 stimulation Effects 0.000 claims abstract description 10
- 238000005070 sampling Methods 0.000 claims description 8
- 239000013598 vector Substances 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000012706 support-vector machine Methods 0.000 claims description 5
- 230000000763 evoking effect Effects 0.000 claims description 4
- 230000001939 inductive effect Effects 0.000 claims description 4
- 238000000513 principal component analysis Methods 0.000 claims description 4
- 210000001652 frontal lobe Anatomy 0.000 claims description 3
- 230000003287 optical effect Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 2
- 238000007906 compression Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000000034 method Methods 0.000 description 4
- 238000010219 correlation analysis Methods 0.000 description 3
- 241000287181 Sturnus vulgaris Species 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000007858 starting material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
- G05D1/104—Simultaneous control of position or course in three dimensions specially adapted for aircraft involving a plurality of aircrafts, e.g. formation flying
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an unmanned aerial vehicle formation multi-mode control system based on brain electrical signals. The unmanned aerial vehicle comprises an unmanned aerial vehicle LED flicker module, a visual stimulation display module 1, a visual stimulation display module 2, an electroencephalogram signal acquisition module, an electroencephalogram signal analysis module 1, an electroencephalogram signal analysis module 2, an electroencephalogram signal analysis module 3, an electroencephalogram signal analysis module 4, a lower computer and a positioning module. According to the invention, the hand-free operation of flexible control and adjustment of the whole and part of the unmanned aerial vehicle formation is realized for the first time, the hand-free switching of the group control mode and the single machine control mode is realized, the number of control instructions reaches 18, and the basic requirements of flexible control and adjustment of the whole and part of the unmanned aerial vehicle formation can be met. In addition, when unmanned aerial vehicle formation exempts from hand control, other tasks can be carried out to both hands, can realize unmanned aerial vehicle control personnel's multitasking operation, improve and control efficiency.
Description
Technical Field
The invention belongs to the technical field of unmanned aerial vehicles, and particularly relates to an unmanned aerial vehicle formation multi-mode control system based on brain electrical signals.
Background
In the aspect of brain-controlled unmanned aerial vehicle formation, at present, no brain-controlled unmanned aerial vehicle group formation control system exists in the patent; the literature contains a brain-controlled unmanned aerial vehicle group system control literature. The control of unmanned aerial vehicle formation (3 frames) is realized in literature 1(Karavas G K,Larsson D T,Artemiadis P.A hybrid BMI for control of robotic swarms:Preliminary results[C]//2017IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2017:5065-5075.), the lifting of unmanned aerial vehicle is controlled by adopting hands, and then the scaling of unmanned aerial vehicle is controlled by adopting motor imagery electroencephalogram signals. The control mainly faces to the overall control of the unmanned aerial vehicle group, and the overall zoom-in and zoom-out functions of the unmanned aerial vehicle group in the horizontal direction are realized.
At present, no brain-controlled unmanned aerial vehicle formation multi-mode control method exists, and no research on a relevant brain-controlled unmanned aerial vehicle formation multi-mode control method exists.
Disclosure of Invention
The invention aims to fill in the blank of a multi-mode control technology of brain-controlled unmanned aerial vehicle formation, and provides a novel brain-controlled unmanned aerial vehicle formation method, which can completely realize the hands-free operation control of unmanned aerial vehicle formation on one hand and realize the flexible control and adjustment of the whole and part of unmanned aerial vehicle formation on the other hand, thereby realizing the efficient control of unmanned aerial vehicle formation.
An unmanned aerial vehicle formation multi-mode control system based on brain electrical signals comprises an unmanned aerial vehicle LED flickering module, a visual stimulation display module 1, a visual stimulation display module 2, an brain electrical signal acquisition module, a brain electrical signal analysis module 1, a brain electrical signal analysis module 2, a brain electrical signal analysis module 3, a brain electrical signal analysis module 4, a lower computer and a positioning module.
The unmanned aerial vehicle LED flickering module comprises an LED lamp which flicker at fixed frequency and is used for inducing steady-state visual evoked potentials of control personnel.
The visual stimulus display module 1 comprises 9 flicker blocks, and each block performs black and white flicker at a frequency and a phase corresponding to each block.
The visual stimulus display module 2 comprises 7 flashing blocks, each of which flashes in black and white with its own corresponding frequency and phase.
The electroencephalogram signal acquisition module comprises an electroencephalogram amplifier, brain electrodes and an electroencephalogram cap, and the electroencephalogram signal sampling frequency is fs=1000 Hz.
The electroencephalogram signal analysis module 1 is used for identifying 3 continuous and rapid blinks of a control person, so that the switching of a cluster control mode and a single machine control mode is realized.
The electroencephalogram signal analysis module 2 is used for identifying control intention of a control person on the whole machine group.
The electroencephalogram signal analysis module 3 is used for identifying a single unmanned plane target of a control person.
The electroencephalogram signal analysis module 4 is used for identifying control intention of a control person on the single unmanned aerial vehicle.
The lower computer comprises an unmanned aerial vehicle position acquisition module, a control personnel intention acquisition module, an unmanned aerial vehicle control module and an unmanned aerial vehicle formation position, speed and gesture control algorithm, and finally outputs speed and angle instructions of the unmanned aerial vehicle, and meanwhile, the speed and angle instructions are converted into unmanned aerial vehicle bottom layer mechanism execution instructions and transmitted to the unmanned aerial vehicle, so that the intention execution of the personnel is realized; the positioning module consists of an optical positioning intelligent camera and a camera cradle head and is used for recording the position of the unmanned aerial vehicle in real time.
The invention has the beneficial effects that: the invention adopts the electroencephalogram signals to realize the switching of the group control mode and the single machine control mode, and simultaneously realizes the flexible control and adjustment of the whole and part of the unmanned aerial vehicle formation. In the group control mode, the unmanned plane group integrally executes control instructions (1 take-off, 2 landing, 3 ascending, 4 descending, 5 advancing, 6 backing, 7 left shifting, 8 right shifting, 9 hovering, 10 holding); in the stand-alone control mode, the selected unmanned aerial vehicle stand-alone execution control instructions (11 up, 12 down, 13 forward, 14 backward, 15 left shift, 16 right shift, 17 hover, 18 hold). According to the invention, the hand-free operation of flexible control and adjustment of the whole and part of the unmanned aerial vehicle formation is realized for the first time, the hand-free switching of the group control mode and the single machine control mode is realized, the number of control instructions reaches 18, and the basic requirement of flexible control and adjustment of the whole and part of the unmanned aerial vehicle formation can be met. In addition, when unmanned aerial vehicle formation exempts from hand control, other tasks can be carried out to both hands, can realize unmanned aerial vehicle control personnel's multitasking operation, improve and control efficiency.
Drawings
Fig. 1 is a block diagram of an unmanned aerial vehicle formation multi-mode control system based on brain electrical signals.
Fig. 2 is a schematic diagram of a visual stimulus display module 1;
In the figure, 9 blocks are provided, each block blinks at its frequency and phase, and the unmanned aerial vehicle formation control personnel will induce a corresponding electroencephalogram pattern when looking at a particular block.
FIG. 3 is a schematic diagram of a visual stimulus display module 2;
in the figure, 7 blocks are provided, each block blinks at its frequency and phase, and the unmanned aerial vehicle formation control personnel will induce a corresponding electroencephalogram pattern when looking at a particular block.
Fig. 4 shows channels (black marks) used for electroencephalogram signal acquisition.
Fig. 5 is a flowchart of the algorithm of the electroencephalogram signal analysis module 2.
Fig. 6 is a flowchart of the algorithm of the electroencephalogram signal analysis module 3.
Fig. 7 is a flowchart of the algorithm of the electroencephalogram signal analysis module 4.
Detailed Description
The present invention will be described more fully hereinafter in order to facilitate an understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
The unmanned aerial vehicle formation multi-mode control system based on the electroencephalogram signals comprises an unmanned aerial vehicle LED flickering module, a visual stimulation display module 1, a visual stimulation display module 2, an electroencephalogram signal acquisition module, an electroencephalogram signal analysis module 1, an electroencephalogram signal analysis module 2, an electroencephalogram signal analysis module 3, an electroencephalogram signal analysis module 4, a lower computer, a positioning system and the like. The system block diagram is shown in fig. 1. The number of unmanned aerial vehicles contained in the unmanned aerial vehicle formation is uncertain, and 10 frames are taken as an example for illustration for convenience of description.
The unmanned plane LED flickering module, the visual stimulus display module 1 and the visual stimulus display module 2 realize the display of visual stimulus. The electroencephalogram signal acquisition module is used for acquiring electroencephalogram signals. The electroencephalogram signal analysis module 1 realizes the identification of 3 continuous and rapid blinks of a control person by analyzing the electroencephalogram signal, and further realizes the switching of a single mode and a cluster mode. And the electroencephalogram signal analysis module 2 is used for analyzing control instructions of the unmanned aerial vehicle group in the group mode. The electroencephalogram signal analysis module 3 realizes the selection of an object single machine in a single machine mode by analyzing the electroencephalogram signal of a control person. And the electroencephalogram signal analysis module 4 analyzes the electroencephalogram signal of the control personnel to realize the analysis of the control instruction of the target single machine in the single machine mode. The positioning system acquires the position information of the unmanned aerial vehicle in real time. The lower computer receives the position information of the unmanned aerial vehicle, converts the unmanned aerial vehicle formation control instruction or the single machine control instruction into an unmanned aerial vehicle bottom mechanism execution instruction, transmits the instruction to the unmanned aerial vehicle, and realizes the execution of the intention of a control person.
The unmanned aerial vehicle LED flicker module is characterized in that an LED lamp is installed on the upper portion of each unmanned aerial vehicle body, and the LED lamp flicker at a fixed frequency and is used for inducing steady-state visual evoked potential of control personnel. The flicker frequency and the flicker phase of 10 unmanned aerial vehicles are gamma i/phi i (i=1, 2, …, 10) respectively, and are used for inducing 10 brain electrical modes. The 10 brain electrical modes respectively correspond to 10 unmanned aerial vehicle single machines.
The visual stimulus display module 1 is shown in fig. 2, and operates in a system starter group control mode, each block flashing black and white at its own corresponding frequency and phase. fk/Φk (k=1, 2, …, 9) represents the frequency and phase of the corresponding block flicker, respectively, and corresponds to the control command k in the cluster control mode. The visual display module 1 has 9 scintillation blocks corresponding to 9 different brain electrical modes and corresponding to instructions 1-9. When the controller does not look at the interface, the 10 th brain electrical mode is induced and corresponds to the instruction 10.
The visual stimulus display module 2 is shown in fig. 3, and operates when the system starts the stand-alone control mode, and each block blinks in black and white at its own corresponding frequency and phase. fk/Φk (k=11, 12, …, 17) represents the frequency and phase of the corresponding block flicker, respectively, and corresponds to the control command k in the stand-alone control mode. The visual display module 2 has 7 flashing blocks corresponding to 7 different brain electrical modes and corresponding to instructions 11-17. When the controller does not look at the interface, the 18 th brain electrical mode is induced and corresponds to the instruction 18.
The electroencephalogram signal acquisition module comprises an electroencephalogram amplifier, an electroencephalogram electrode, an electroencephalogram cap and electroencephalogram signal acquisition software, and the electroencephalogram signal sampling frequency is fs=1000 Hz. This patent uses 23 channels, of which 15 channels (Pz, P1, P2, P3, P4, P5, P6, POz, PO3, PO4, PO7, PO8, oz, O1, O2) are distributed in the brain vision region and nearby for identifying steady-state visual evoked potentials, and the other 8 channels (FPz, FP1, FP2, AFz, AF3, AF4, AF7, AF 8) are distributed in the frontal lobe region for identifying 3 consecutive fast blink events, the specific channels are shown in fig. 4.
The electroencephalogram signal analysis module 1 aims at identifying 3 continuous and rapid blinks of a control person, so that the switching of a cluster control mode and a single machine control mode is realized. After the electroencephalogram signals with the time length T are collected, a support vector machine is adopted to identify whether 3 continuous and rapid blinks exist or not. When 3 continuous and rapid blinks are detected, switching control modes; otherwise, the switching is not performed. The electroencephalogram signal analysis module 1 adopts electroencephalogram signals of 8 channels in a frontal lobe area. The brain electrical signal corresponding to the time length T can be expressed asN=t×fs represents a sampling point in the time dimension, and l=8 represents a channel. And converting the electroencephalogram signal into a vector X= [ X 1,x2,...,xNL ], then carrying out principal component analysis, and carrying out feature extraction on the electroencephalogram data. The extracted features can be expressed as Θ=h·x
Representing the principal component analysis transformation matrix, q=50 represents the feature dimension after compression, containing 95% of the information of the original features. Then classifying by using a support vector machine, wherein the support vector machine model can be expressed as
N represents the number of support vectors, w i represents the weight of the ith support vector, Θ i represents the ith support vector of the classifier, δ represents the hyper-parameter, and ε represents the bias of the classifier. Control instructions
Wherein S 1 =0 represents the non-switching control mode, and S 1 =1 represents the switching control mode.
The electroencephalogram signal analysis module 2 corresponds to the visual stimulus display module 1. The purpose is to identify the control intention of a control person on the whole cluster when the system is switched to the cluster control mode. After acquiring the electroencephalogram signals of the time length T, 10 electroencephalogram modes are matched by adopting a typical correlation analysis algorithm, and the control instruction corresponding to the matched electroencephalogram modes is the control intention of a control person on the whole cluster. The electroencephalogram signal analysis module 2 adopts electroencephalogram signals of 15 channels of a visual area. Fig. 5 is a flowchart of the algorithm of the electroencephalogram signal analysis module 2. The brain electrical signal corresponding to the time length T can be expressed asN=t×fs represents a sampling point in the time dimension, and l=15 represents a channel. The electroencephalogram signal analysis module 2 has 9 templates, which can be expressed as
Where k=1, 2, 9,D＝60,
Calculating typical correlation coefficient rho k＝canoncorr(x,Mk of x and M k) and controlling instruction
Where k=9, η 2 is a threshold value set in advance.
The electroencephalogram signal analysis module 3 corresponds to the unmanned aerial vehicle LED flickering module. The purpose is to identify the unmanned aerial vehicle single-machine target of the control personnel when the system is switched to the single-machine control mode. The 10 brain electrical modes respectively correspond to 10 unmanned aerial vehicle single machines. After acquiring the electroencephalogram signals of the time length T, 10 electroencephalogram modes are matched by adopting a typical correlation analysis algorithm, and the corresponding unmanned plane single machine serial number of the matched electroencephalogram modes is the unmanned plane single machine to be controlled by a control person. The electroencephalogram signal analysis module 3 adopts electroencephalogram signals of 15 channels of the visual area. Fig. 6 is a flowchart of the algorithm of the electroencephalogram signal analysis module 3. The brain electrical signal corresponding to the time length T can be expressed asN=t×fs represents a sampling point in the time dimension, and l=15 represents a channel. The electroencephalogram signal analysis module 2 comprises 10 templates. The ith template may be expressed as gamma i/phi
Where i=1, 2, 10,B＝120,
Calculating typical correlation coefficient rho i＝canoncorr(x,Yi of x and Y i) and controlling instruction
S3＝argmax{ρ1,ρ2,...ρi,...,ρI-1,ρI}
Wherein i=10.
The electroencephalogram signal analysis module 4 corresponds to the visual stimulus display module 2. The purpose is to identify the control intention of the control personnel on the single unmanned plane after selecting the target single machine in the single machine control mode. After acquiring the electroencephalogram signals with the time length T, 8 electroencephalogram modes are matched by adopting a typical correlation analysis algorithm, and the matched electroencephalogram modes are the corresponding control instructions of the control personnel. The electroencephalogram signal analysis module 4 adopts electroencephalogram signals of 15 channels of the visual area. Fig. 7 is a flowchart of the algorithm of the electroencephalogram signal analysis module 4. The brain electrical signal corresponding to the time length T can be expressed asN=t×fs represents a sampling point in the time dimension, and l=15 represents a channel. The electroencephalogram signal analysis module 2 has 7 templates, which can be expressed as
Where k=11, 12,..17,D＝60,
Calculating typical correlation coefficient rho k＝canoncorr(x,Gk of x and G k) and controlling instruction
Where k=17, η 4 is a threshold value set in advance.
The lower computer comprises an unmanned aerial vehicle position acquisition module, a control personnel intention acquisition module, an unmanned aerial vehicle control module, an unmanned aerial vehicle formation position, speed and gesture control algorithm, and finally outputs the speed, angle and other instructions of the unmanned aerial vehicle, and meanwhile, the speed, angle and other instructions are converted into unmanned aerial vehicle bottom layer mechanism execution instructions and transmitted to the unmanned aerial vehicle, so that the intention execution of the personnel is realized.
The positioning system consists of an optical positioning intelligent camera, a camera holder and matched software and is used for recording the position of the unmanned aerial vehicle in real time.
The working process comprises the following steps: and the unmanned aerial vehicle formation control personnel make a decision according to the current unmanned aerial vehicle formation state. When the control mode is required to be switched, the control personnel execute 3 continuous and rapid blinking actions. The system then performs control mode switching. When the machine group control mode is switched to, the visual stimulus display module 1 starts to work, and a control person stares at a flickering square corresponding to the decision instruction for a duration T to induce a corresponding brain electrical signal; then, the electroencephalogram analysis module 2 analyzes the acquired electroencephalogram signals to obtain an electroencephalogram mode corresponding to the current time T, and further obtains the control intention of a control person at the moment; in the group control mode, when unmanned aerial vehicle formation control personnel do not need to output a group control instruction, a flicker square of a stimulation module is not needed to be stared at, a 'hold' instruction is output corresponding to the 10 th electroencephalogram mode at the moment, the unmanned aerial vehicle group keeps the instruction at the last moment, and the instruction at the last moment is executed; until the drone swarm receives instructions 1-9, the drone control instructions will remain unchanged. When switching to a single machine control mode, the unmanned aerial vehicle LED flickering module starts to work, a controller firstly selects a target single machine and looks at the LED flickering module of the target single machine for a duration T, and a corresponding electroencephalogram signal is induced; then, the electroencephalogram analysis module 3 analyzes the acquired electroencephalogram signals to obtain an electroencephalogram mode corresponding to the current time T, and further obtains a target single machine selected by a control person at the moment; after the single target is selected, the visual stimulus display module 2 starts to work, and the control personnel stares at the flashing square corresponding to the decision instruction for a duration T to induce the corresponding brain electrical signal; then, the electroencephalogram analysis module 4 analyzes the acquired electroencephalogram signals to obtain an electroencephalogram mode corresponding to the current time T, and further obtains the control intention of a control person at the moment; in the single machine control mode, when a control person does not need to output a single machine control instruction, the person does not need to stare at a flickering square of the stimulation module, an 'hold' instruction is output corresponding to the 18 th electroencephalogram mode, and the single machine of the unmanned aerial vehicle keeps the instruction at the last moment and executes the instruction at the last moment; until the unmanned aerial vehicle receives the instructions 11-17, otherwise the unmanned aerial vehicle single machine control instruction will remain unchanged. Then the lower computer obtains the instructions of the position, the speed, the angle and the like which are required to be output to the unmanned aerial vehicle at the current moment according to the intention of the control personnel and the current position, the speed and the gesture information of the unmanned aerial vehicle formation by a control module and a control algorithm in the lower computer; and finally, converting the instruction into an execution instruction of the unmanned aerial vehicle bottom mechanism, and transmitting the instruction to the unmanned aerial vehicle, so that the intention execution of the person is realized.
The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (6)
1. An unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals is characterized by comprising an unmanned aerial vehicle LED flickering module, a visual stimulation display module 1, a visual stimulation display module 2, an electroencephalogram signal acquisition module, an electroencephalogram signal analysis module 1, an electroencephalogram signal analysis module 2, an electroencephalogram signal analysis module 3, an electroencephalogram signal analysis module 4, a lower computer and a positioning module;
the electroencephalogram signal analysis module 1 is used for identifying 3 continuous and rapid blinks of a control person, so that the switching of a cluster control mode and a single machine control mode is realized; electroencephalogram signals of 8 channels of frontal lobe areas are adopted; the brain electrical signal corresponding to the time length T can be expressed as N=t×fs represents a sampling point in a time dimension, l=8 represents a channel, an electroencephalogram signal is converted into a vector x= [ X 1,x2,...,xNL ], then principal component analysis is performed, and feature extraction is performed on electroencephalogram data; the extracted features may be expressed as Θ=hann·x;
representing a principal component analysis transformation matrix, q=50 representing the feature dimension after compression, containing 95% of the original feature information; then, a support vector machine is applied to classify, and the support vector machine model can be expressed as:
n represents the number of support vectors, w i represents the weight of the ith support vector, Θ i represents the ith support vector of the classifier, δ represents a hyper-parameter, and ε represents the bias of the classifier; control instructions:
Wherein S 1 =0 represents a non-switching control mode, and S 1 =1 represents a switching control mode;
The electroencephalogram signal analysis module 2 is used for identifying control intention of a control person on the whole machine group; the electroencephalogram signal analysis module 2 has 9 templates, which can be expressed as:
where k=1, 2, 9,
Calculating a typical correlation coefficient ρ k＝canoncorr(x,Mk of x and M k), and controlling the instruction:
Wherein, k=9, η 2 is a threshold value set in advance;
the electroencephalogram signal analysis module 3 is used for identifying a single unmanned plane target of a control person; the electroencephalogram signal analysis module 3 comprises 10 templates; the ith template may be expressed as:
Where i=1, 2, 10,
Calculating a typical correlation coefficient ρ i＝canoncorr(x,Yi of x and Y i), control instructions:
S3＝argmax{ρ1,ρ2,...ρi,...,ρI-1,ρI}
wherein i=10;
the electroencephalogram signal analysis module 4 is used for identifying the control intention of a control person on the single unmanned aerial vehicle; the electroencephalogram signal analysis module 4 has 17 templates, which can be expressed as follows:
where k=11, 12,..17,
Calculating a typical correlation coefficient ρ k＝canoncorr(x,Gk of x and G k), control instructions:
where k=17, η 4 is a threshold value set in advance.
2. The unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals according to claim 1, wherein the unmanned aerial vehicle LED flashing module comprises LED lamps flashing at a fixed frequency for inducing steady-state visual evoked potentials for control personnel.
3. The unmanned aerial vehicle formation multi-mode control system based on the electroencephalogram signals according to claim 1, wherein the visual stimulus display module 1 comprises 9 flashing blocks, each of which flashes black and white with its own corresponding frequency and phase.
4. The unmanned aerial vehicle formation multi-mode control system based on the electroencephalogram signals according to claim 1, wherein the visual stimulus display module 2 comprises 7 flashing blocks, each of which flashes black and white with its own corresponding frequency and phase.
5. The unmanned aerial vehicle formation multi-mode control system based on the electroencephalogram signals according to claim 1, wherein the electroencephalogram signal acquisition module comprises an electroencephalogram amplifier, an electroencephalogram electrode and an electroencephalogram cap, and the sampling frequency of the electroencephalogram signals is fs=1000 Hz.
6. The unmanned aerial vehicle formation multi-mode control system based on the electroencephalogram signals according to claim 1 is characterized in that the lower computer comprises an unmanned aerial vehicle position acquisition module, a control personnel intention acquisition module, an unmanned aerial vehicle control module and an unmanned aerial vehicle formation position, speed and gesture control algorithm, and finally outputs speed and angle instructions of the unmanned aerial vehicle, and meanwhile the speed and angle instructions are converted into unmanned aerial vehicle bottom mechanism execution instructions and transmitted to the unmanned aerial vehicle, so that the intention execution of a person is realized; the positioning module consists of an optical positioning intelligent camera and a camera cradle head and is used for recording the position of the unmanned aerial vehicle in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044955.3A CN113625769B (en) | 2021-09-07 | 2021-09-07 | Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044955.3A CN113625769B (en) | 2021-09-07 | 2021-09-07 | Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113625769A CN113625769A (en) | 2021-11-09 |
| CN113625769B true CN113625769B (en) | 2024-05-10 |
Family
ID=78389307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111044955.3A Active CN113625769B (en) | 2021-09-07 | 2021-09-07 | Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113625769B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114237385B (en) * | 2021-11-22 | 2024-01-16 | 中国人民解放军军事科学院军事医学研究院 | Man-machine brain control interaction system based on non-invasive brain electrical signals |
| CN114305429B (en) * | 2022-01-10 | 2023-07-28 | 中国人民解放军军事科学院军事医学研究院 | Electrode cap for synchronizing transcranial direct current stimulation signal and brain electrical signal and testing method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170013092A (en) * | 2015-07-27 | 2017-02-06 | 고려대학교 산학협력단 | Eeg-based brain-machine interface apparatus and method for recognizing human intention using line stimuli on the display |
| CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
| CN108784693A (en) * | 2018-06-15 | 2018-11-13 | 北京理工大学 | P300 single-trial extraction technologies based on independent component analysis and Kalman smoothing |
| CN110134243A (en) * | 2019-05-20 | 2019-08-16 | 中国医学科学院生物医学工程研究所 | A kind of brain control mechanical arm shared control system and its method based on augmented reality |
| CN110163180A (en) * | 2019-05-29 | 2019-08-23 | 长春思帕德科技有限公司 | Mental imagery eeg data classification method and system |
| CN110531791A (en) * | 2019-08-25 | 2019-12-03 | 西北工业大学 | The machine integrated target detection unmanned vehicle of multiple instruction set hypencephalon |
| CN111638724A (en) * | 2020-05-07 | 2020-09-08 | 西北工业大学 | Novel cooperative intelligent control method for unmanned aerial vehicle group computer |
| CN112051780A (en) * | 2020-09-16 | 2020-12-08 | 北京理工大学 | Brain-computer interface-based mobile robot formation control system and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0906029D0 (en) * | 2009-04-07 | 2009-05-20 | Nat Univ Ireland Cork | A method of analysing an electroencephalogram (EEG) signal |
| WO2019068200A1 (en) * | 2017-10-06 | 2019-04-11 | Holland Bloorview Kids Rehabilitation Hospital | Brain-computer interface platform and process for classification of covert speech |
-
2021
- 2021-09-07 CN CN202111044955.3A patent/CN113625769B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170013092A (en) * | 2015-07-27 | 2017-02-06 | 고려대학교 산학협력단 | Eeg-based brain-machine interface apparatus and method for recognizing human intention using line stimuli on the display |
| CN108415565A (en) * | 2018-02-25 | 2018-08-17 | 西北工业大学 | The machine integrated intelligent control method of unmanned plane brain and technology |
| CN108784693A (en) * | 2018-06-15 | 2018-11-13 | 北京理工大学 | P300 single-trial extraction technologies based on independent component analysis and Kalman smoothing |
| CN110134243A (en) * | 2019-05-20 | 2019-08-16 | 中国医学科学院生物医学工程研究所 | A kind of brain control mechanical arm shared control system and its method based on augmented reality |
| CN110163180A (en) * | 2019-05-29 | 2019-08-23 | 长春思帕德科技有限公司 | Mental imagery eeg data classification method and system |
| CN110531791A (en) * | 2019-08-25 | 2019-12-03 | 西北工业大学 | The machine integrated target detection unmanned vehicle of multiple instruction set hypencephalon |
| CN111638724A (en) * | 2020-05-07 | 2020-09-08 | 西北工业大学 | Novel cooperative intelligent control method for unmanned aerial vehicle group computer |
| CN112051780A (en) * | 2020-09-16 | 2020-12-08 | 北京理工大学 | Brain-computer interface-based mobile robot formation control system and method |
Non-Patent Citations (2)
| Title |
|---|
| HUANG ,WB.Ergonomics analysis based on intention inference.《JOURNAL OF INTELLIGENT & FUZZY SYSTEMS》.2021,全文. * |
| 基于深度学习的头皮脑电信息解码研究进展;刘政;何峰;汤佳贝;万柏坤;明东;;中国生物医学工程学报;20200420(02);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113625769A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113625769B (en) | Unmanned aerial vehicle formation multi-mode control system based on electroencephalogram signals | |
| CN111890389B (en) | Multi-mobile robot cooperative control system based on multi-modal interactive interface | |
| CN105528593B (en) | Forward vehicle driver driving behavior prediction system and prediction method | |
| CN107957783B (en) | Multi-mode intelligent control system and method based on electroencephalogram and electromyogram information | |
| CN111055281A (en) | ROS-based autonomous mobile grabbing system and method | |
| CN105825713A (en) | Vehicular-mounted unmanned aerial vehicle auxiliary driving system and operation mode | |
| CN104015722A (en) | Vehicle driving auxiliary system supporting comfort degree adjusting | |
| CN112051780B (en) | Brain-computer interface-based mobile robot formation control system and method | |
| GB2525840A (en) | Autonomous driving system and method for same | |
| CN111487988B (en) | Brain-controlled unmanned aerial vehicle method based on steady-state visual evoked potential brain-computer interface | |
| CN209258112U (en) | Vehicle DAS (Driver Assistant System) and vehicle | |
| CN110134240A (en) | Robot wheel chair control system based on brain electricity Yu head appearance hybrid interface | |
| CN102331782A (en) | Automatic vehicle controlling method of multi-mode brain-computer interface | |
| CN110716578A (en) | Aircraft control system based on hybrid brain-computer interface and control method thereof | |
| DE102020215667A1 (en) | SYSTEM AND METHOD FOR MONITORING A COGNITIVE CONDITION OF A DRIVER OF A VEHICLE | |
| CN111399652A (en) | Multi-robot hybrid system based on layered SSVEP and visual assistance | |
| CN108989653A (en) | The fatigue driving early-warning device of vehicular adaptive environment light and head pose | |
| CN104691603A (en) | Automatic adjustment method of driving platform | |
| CN203902538U (en) | Vehicle driving assistance system capable of supporting comfort level adjustment | |
| CN109814720B (en) | Brain control method and system of equipment | |
| CN116279572B (en) | Vehicle safety situation assessment and steady driving mode switching method and system | |
| CN215954144U (en) | Unmanned aerial vehicle controls helmet and flight equipment | |
| CN112936259B (en) | Man-machine cooperation method suitable for underwater robot | |
| CN111605479A (en) | Emotion-sensing and self-adjusting interior atmosphere lamp system and control method | |
| CN114073625B (en) | Autonomous navigation's electronic wheelchair of brain electricity control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |