CN114489097B - Unmanned aerial vehicle flight attitude brain control method based on precise motion gesture - Google Patents

Abstract

The invention discloses a brain control method for the flight attitude of a drone based on precise motion gestures, specifically: the EEG signal acquisition module collects the EEG signals of the subject's body movement area when performing different precise motion gestures; the EEG signals The processing module preprocesses the EEG signal samples, constructs a deep convolutional neural network model, and then optimizes the hyperparameters of the constructed deep convolutional neural network model, and then calls the trained deep convolutional neural network model. The deep convolutional neural network model recognizes the EEG signals during the execution of different precise motion gestures, converts the results recognized by the EEG signal processing module into the flight attitude control instructions of the UAV, and sends them to the UAV control module for unmanned control. The flight attitude of the aircraft is controlled. The invention solves the problems of few control instructions and low decoding accuracy of brain electrical signals in the prior art of brain-controlled drones.

Description

Brain control method of UAV flight attitude based on precise motion gestures

technical field

The invention belongs to the technical field of UAV flight attitude control methods, and relates to a UAV flight attitude brain control method based on precision motion gestures.

Background technique

With the development and wide application of artificial intelligence technology, unmanned aerial vehicles (UAVs) present broad application scenarios in various industries due to their advantages of portability, flexibility, safety and controllability. At present, drones on the market are generally controlled by joystick remote operation, so people's hands have not been completely freed. In 2017, with the explosive development of the artificial intelligence industry, China became one of the first countries to publish a white paper on the artificial intelligence industry. Intelligent, drones, as one of the consumer terminals of artificial intelligence products, have been widely used in agriculture, military, fire protection, entertainment, etc. However, most of the existing UAVs use the traditional control mode to execute the flight attitude, and cannot perform autonomous perception, reasoning and decision-making capabilities in dynamic and complex working environments.

Therefore, accurate perception based on the operator's brain-control intention has become one of the key technologies for the intelligent research of drones. As an emerging artificial intelligence technology in recent years, brain control technology establishes a direct bridge between the brain and peripheral devices through EEG signal decoding technology, and has been widely used by scholars at home and abroad. Among them, the brain-computer interface system based on motor imagery and motor execution, as a spontaneous brain-computer interface system that does not rely on external stimulators, has attracted extensive attention from scholars at home and abroad. However, the traditional motor imagery brain-computer interface system has the problems of insufficient brain control instructions, long training time, and low decoding accuracy of EEG signals. Therefore, at this stage, it is urgent to propose a new brain control paradigm that can realize the accurate perception of the operator's motion intention.

Contents of the invention

The purpose of the present invention is to provide a brain control method for flying posture of UAV based on precise motion gestures, which solves the problem of less control instructions and low decoding accuracy of EEG signals in the existing UAV control technology. question.

The technical solution adopted in the present invention is a brain control method for the flying attitude of the drone based on precise motion gestures, including wearing an EEG signal acquisition module on the head of the subject, and the EEG signal acquisition module is connected to the EEG through the WiFi transmission module a The signal processing module and the EEG signal processing module are connected to the UAV control module through the WiFi transmission module b, and are specifically implemented according to the following steps:

Step 1, the subject makes six pre-set precision motor gestures;

Step 2, the EEG signal acquisition module collects the EEG signals in the body motor area of the subject when performing different precise motion gestures, and obtains the EEG signal samples performed by different precise motion gestures;

Step 3, the EEG signal processing module preprocesses the EEG signal samples;

Step 4, the EEG signal processing module constructs a deep convolutional neural network model, and uses the constructed deep convolutional neural network model for decoding the EEG signal;

Step 5. The EEG signal processing module optimizes the hyperparameters of the constructed deep convolutional neural network model. During the optimization process, the preprocessed EEG signal samples in step 3 are used as training data. The model is trained, and the deep convolutional neural network model constructed in step 4 is optimized according to the optimal hyperparameters of the deep convolutional neural network obtained through optimization, and saved;

Step 6, the EEG signal processing module invokes the trained deep convolutional neural network model to identify EEG signals when performing different precise motion gestures;

Step 7, convert the result identified by the EEG signal processing module into a flight attitude control command of the UAV, and send it to the UAV control module to control the flight attitude of the UAV.

The present invention is also characterized in that,

The EEG signal acquisition module specifically collects EEG signals at the electrode positions of FC3, FCz, FC4, C3, Cz, C4, CP3, and CP4 under the international standard 10/20.

Six precision movement gestures are performed: five finger closing movement, thumb single finger stretching movement, five finger opening movement, index finger single finger stretching movement, thumb and index finger double finger stretching movement, index finger and middle finger double finger stretching movement, six precision movements The gestures correspond to the drone's takeoff, landing, forward, backward, left turn, and right turn respectively.

In step 3, the EEG signal processing module preprocesses the EEG signal samples specifically as follows: sequentially perform Butterworth band-pass filtering and trend removal preprocessing on the collected EEG signal samples.

In step 5, the EEG signal processing module optimizes the hyperparameters of the constructed deep convolutional neural network model and uses the genetic algorithm to optimize the hyperparameters. number of neurons.

The beneficial effects of the present invention are:

The present invention adopts the precise movement gesture brain control method to control the flight posture of the UAV, and can realize the brain control instructions of 6 movement gestures. The convolutional neural network model is used to optimize the hyperparameters, and the time consumption is small. Then, the optimized deep convolutional neural network model is used to classify and extract the EEG signals, and the decoding accuracy of the EEG signals is high.

Description of drawings

Fig. 1 is the flow chart of the present invention based on the UAV flight attitude brain control method of precision motion gesture;

Fig. 2 is the connection diagram of each module in the UAV flight attitude brain control method based on precise motion gestures of the present invention;

3 is a schematic diagram of the layout of the EEG signal acquisition module of the present invention;

Fig. 4 is a schematic diagram of six precision movement gestures in the brain control method of drone flight attitude based on precision movement gestures in the present invention.

In the figure, 310. EEG signal acquisition module, 320. WiFi transmission module a, 330. EEG signal processing module, 340. WiFi transmission module b, 350. UAV control module.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

The brain control method of UAV flight attitude based on precision motion gestures of the present invention, as shown in Figure 1-2, includes wearing an EEG signal acquisition module 310 on the subject's head, and the EEG signal acquisition module 310 transmits the module a320 through WiFi Connected with the EEG signal processing module 330, the EEG signal processing module 330 is connected with the UAV control module 350 through the WiFi transmission module b340, as shown in Figure 3, the EEG signal acquisition module 310 specifically collects FC3 under the international standard 10/20 , FCz, FC4, C3, Cz, C4, CP3, CP4 electrode position of the EEG signal, specifically according to the following steps:

In step 1, the subject made six pre-set precise movement gestures; as shown in Figure 4, the six precision movement gestures were: five-finger closing movement, thumb-single-finger stretching movement, five-finger opening movement, index finger single-finger movement Finger stretching, thumb and index finger stretching, index finger and middle finger stretching, six precision motion gestures correspond to drone takeoff, landing, forward, backward, left turn, right turn;

Step 2, the EEG signal acquisition module 310 collects the EEG signals in the body motor area of the subject when performing different precise motor gestures, and obtains the EEG signal samples when performing different precise motor gestures;

Step 3, the EEG signal processing module 330 sequentially performs preprocessing on the EEG signal samples with Butterworth bandpass filtering and removal of trend items;

Step 4, the EEG signal processing module 330 constructs a deep convolutional neural network model, and uses the constructed deep convolutional neural network model for decoding of EEG signals;

Step 5, the EEG signal processing module 330 uses the genetic algorithm to optimize the hyperparameters of the constructed deep convolutional neural network model. During the optimization process, the EEG signal samples preprocessed in Step 3 are used as training data. The convolutional neural network model is trained, and the optimized hyperparameters are the number of convolution kernels in the convolutional layer and the number of neurons in the fully connected layer. According to the optimal hyperparameters of the neural network obtained through optimization, optimize the depth constructed in step 4. Convolutional neural network model and save it;

Step 6, the EEG signal processing module 330 calls the trained deep convolutional neural network model to identify the EEG signals when performing different precise motor gestures;

Step 7: Convert the result recognized by the EEG signal processing module 330 into a flight attitude control instruction of the drone, and send it to the drone control module 350 to control the flight attitude of the drone.

Example

In this embodiment, a portable NeuSen W64 channel EEG acquisition device is used, and the electrode positions of FC3, FCz, FC4, C3, Cz, C4, CP3, and CP4 under the international standard 10/20 are selected for EEG signal acquisition. The EEG signal collection module 310 amplifies the EEG signal after collecting the EEG signal, and then transmits the signal to the EEG signal processing module 330 through the WiFi transmission module a320. The EEG signal processing module 330 is responsible for preprocessing the EEG signal, feature extraction and pattern recognition, and converting the EEG signal recognition result into a control command of the drone and sending it to the drone control module 350 through the WiFi transmission module b340, thereby Control the UAV to perform different flight attitudes.

The brain control method of UAV flight attitude based on precise motion gestures of the present invention, when the subject starts to perform precise gesture movements, the EEG signal acquisition module 310 collects the EEG signals of the cerebral cortex of the subject, amplifies them and transmits them to the EEG signals The processing module 330 performs preprocessing, feature extraction and pattern recognition, and finally transmits the EEG signal recognition result to the UAV control module 350 to realize the control of different flight attitudes of the UAV. The specific implementation includes the following steps:

Step 1. The subjects made six corresponding precise gestures, collected the EEG signals at the electrode positions of the subjects FC3, FCz, FC4, C3, Cz, C4, CP3, and CP4, and obtained the EEG signals performed by different precise motor gestures The sample set, the six precision motion gestures in this embodiment are shown in Figure 4.

Step 2, preprocessing the collected EEG signals. In this embodiment, the collected EEG signals are subjected to 0.5-45 Hz Butterworth band-pass filtering and preprocessing for removing trend items.

Step 3, build a deep convolutional neural network structure model. In this embodiment, a two-dimensional convolutional layer is used to extract EEG signal features, and a fully connected layer is used to classify the extracted features.

Step 4, automatically optimize the hyperparameters of the deep convolutional neural network. In this embodiment, a genetic algorithm is used to optimize hyperparameters. During the optimization process, the preprocessed EEG signal samples in step 3 are used as training data to train the deep convolutional neural network model, and the deep convolutional neural network structure The model is defined as a function to be optimized, the number of convolution kernels in the convolutional layer and the number of neurons in the fully connected layer are defined as variables to be optimized, and the classification accuracy of EEG signals is defined as the function value as the evaluation of network performance standard.

Step 5, automatically iteratively optimize through the genetic algorithm to obtain the optimal hyperparameters, thereby obtaining the deep convolutional neural network structure model with the best performance, and then save the trained deep convolutional neural network structure model.

Step 6, call the deep convolutional neural network structure model trained and saved in step 5 to recognize the EEG signals of different precise motion gestures in real time.

Step 7, the EEG signal recognition results are converted into control instructions and transmitted to the UAV. In this embodiment, the corresponding relationship between the EEG signal recognition result and the control instruction is shown in Table 1.

Table 1

Claims (5)

1. The brain control method of flying attitude of unmanned aerial vehicle based on precision motion gestures, it is characterized in that, comprise wearing EEG signal acquisition module (310) to experimenter's head, described EEG signal acquisition module (310) transmits by WiFi Module a (320) is connected with an EEG signal processing module (330), and the EEG signal processing module (330) is connected with the UAV control module (350) through the WiFi transmission module b (340), specifically implemented according to the following steps : Step 1, the subject makes six pre-set precision motor gestures; Step 2, the EEG signal collection module (310) collects the EEG signals in the body motor area of the subject when performing different precise motion gestures, and obtains the EEG signal samples performed by different precise motion gestures; Step 3, the EEG signal processing module (330) preprocesses the EEG signal samples; Step 4, the EEG signal processing module (330) constructs a deep convolutional neural network model, and uses the constructed deep convolutional neural network model for decoding of EEG signals; Step 5, the EEG signal processing module (330) optimizes the hyperparameters of the constructed deep convolutional neural network model. During the optimization process, the preprocessed EEG signal samples in step 3 are used as training data, The convolutional neural network model is trained, and the deep convolutional neural network model constructed in step 4 is optimized according to the optimal hyperparameters of the deep convolutional neural network obtained through optimization, and saved; Step 6, the EEG signal processing module (330) invokes the trained deep convolutional neural network model to identify EEG signals when performing different precise motion gestures; Step 7, converting the result identified by the EEG signal processing module (330) into a flight attitude control instruction of the UAV, and sending it to the UAV control module (350) to control the flight attitude of the UAV. 2. the UAV flying attitude brain control method based on precision motion gestures according to claim 1, is characterized in that, described EEG signal acquisition module (310) specifically collects FC3, FCz, FC4 under international standard 10/20 , C3, Cz, C4, CP3, CP4 electrode position of the EEG signal. 3. The brain control method for flying attitude of unmanned aerial vehicles based on precise motion gestures according to claim 2, characterized in that, the execution actions of the six precision motion gestures are respectively: five-finger closing motion, thumb single-finger stretching motion, five-finger Open movement, single index finger stretching movement, thumb and index finger double-finger stretching movement, index finger and middle finger double-finger stretching movement, six precision movement gestures correspond to drone takeoff, landing, forward, backward, left turn, right turn respectively. 4. The method for brain control of UAV flight attitude based on precision motion gestures according to claim 3, characterized in that, in the step 3, the EEG signal processing module (330) performs preprocessing on the EEG signal samples specifically as follows: : The collected EEG signal samples are sequentially preprocessed by Butterworth band-pass filtering and trend removal. 5. the UAV flight attitude brain control method based on precision motion gestures according to claim 4, is characterized in that, in described step 5, EEG signal processing module (330) is to the depth convolutional neural network model that has been built For hyperparameter optimization, the genetic algorithm is used for hyperparameter optimization. The optimal hyperparameters are the number of convolution kernels in the convolutional layer and the number of neurons in the fully connected layer.

Images