WO2022062169A1

WO2022062169A1 - Sharing control method for electroencephalogram mobile robot in unknown environment

Info

Publication number: WO2022062169A1
Application number: PCT/CN2020/132580
Authority: WO
Inventors: 徐宝国; 刘德平; 王勇; 张坤; 宋爱国; 赵国普
Original assignee: 东南大学
Priority date: 2020-09-24
Filing date: 2020-11-30
Publication date: 2022-03-31
Also published as: CN112148011A; CN112148011B

Abstract

A sharing control method for an electroencephalogram mobile robot in an unknown environment. The sharing control method comprises: first collecting motor imagery electroencephalogram signals of a user, preprocessing the signals and performing feature extraction, performing self-adaptive weight linear summation on the electroencephalogram signals on the left side and the right side, and obtaining electroencephalogram-speed control signals; the mobile robot obtaining autonomous obstacle avoidance-speed control signals according to autonomous path planning; and the mobile robot being subjected to sharing control of the electroencephalogram-speed control signals and the autonomous obstacle avoidance-speed control signals and travelling in the unknown environment. According to the sharing control method for the electroencephalogram mobile robot in the unknown environment, a linear speed of the mobile robot can be controlled by means of the electroencephalogram-speed control signals, the direction of the linear speed of the mobile robot is controlled by means of the autonomous obstacle avoidance-speed control signals, and the traveling process is more stable by means of continuous sharing control.

Description

A shared control method for EEG mobile robots in unknown environments

technical field

The invention belongs to the technical field of robots, and in particular relates to a shared control method of an EEG mobile robot in an unknown environment.

Background technique

Brain-Computer Interface (BCI) technology is a technology that can use brain physiological signals to output control signals to external devices without going through the neuro-muscle pathway. Brain-computer interface technology obtains EEG control signals by analyzing the collected EEG signals, thereby establishing a new control method between the brain and the mobile robot.

The control tasks of mobile robots can be divided into control tasks of known environment and control tasks of unknown environment according to the cognition of the mobile robot to the working environment. The control task of the known environment refers to controlling the robot to complete tasks such as obstacle avoidance and path planning when the environment map or the location of obstacles in the environment is known. The control task of the unknown environment means that the environmental information such as the environmental map or the location of the obstacles is unknown, and the robot needs to complete tasks such as obstacle avoidance and path planning without prior knowledge of the environment. Mobile robot control in an unknown environment needs to combine multiple sensors to reasonably perceive the environment, and then perform global path planning and local path planning to complete the task.

In the existing research, the shared control of human-mobile robots in unknown environments is divided into discrete shared control and continuous shared control. Discrete shared control divides the control process into two stages: EEG control and automatic control. When no obstacles are detected, EEG control is completely used. When obstacles are detected, the robot is autonomously controlled. The robot uses depth cameras, inertial sensors, Lidar and ultrasonic sensors perceive pose and environmental information, rely on the distance information from obstacles, and combine artificial potential field method, A* algorithm, RRT algorithm and D* algorithm for path planning and navigation. This method poses a great challenge to the robot's autonomous perception and autonomous decision-making capabilities. The human-computer interaction experience is poor, and the efficiency in the process of perception and decision-making is often low. Continuous shared control uses a certain strategy to combine the user's EEG control with the robot's autonomous control. For example, the EEG control signal is regarded as a gravitational force, and the autonomous control signal is regarded as a repulsive force, and the resultant force is used to control the movement of the mobile robot. This sharing method is more complicated and is easily affected by the unevenness of the EEG signal. Others use EEG signals to control the direction of the robot's linear velocity, and the linear velocity of the robot is set to a fixed value or autonomously controlled. This sharing method is not stable enough, the forward trajectory is easy to shake, and the practicability is not high.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention discloses a shared control method of an EEG mobile robot in an unknown environment. The EEG signal is used to control the magnitude of the linear velocity of the mobile robot, and the autonomous obstacle avoidance signal is used to control the direction of the linear velocity of the mobile robot. Robotic obstacle avoidance and path planning of mobile robots to improve the efficiency of mobile robot control and human-computer interaction capabilities.

For achieving the above object, technical scheme of the present invention is as follows:

A shared control method for an EEG mobile robot in an unknown environment, comprising the following steps:

Step 1. Collect the user's motor imagery EEG signal and perform preprocessing and feature extraction;

Step 2. Perform an adaptive weight linear summation on the left and right EEG signals to obtain an EEG-speed control signal;

Step 3, the mobile robot obtains the autonomous obstacle avoidance-speed control signal according to the autonomous path planning;

Step 4. The mobile robot is under the shared control of the EEG-speed control signal and the autonomous obstacle avoidance-speed control signal, and drives in an unknown environment.

Further, the step 1 specifically includes:

Step 1a, using the first lead electrode to collect the motor imagery EEG signal of the left motor sensory cortex on the top of the user's head;

Step 1b, using the second lead electrode to collect the motor imagery EEG signal of the right motor sensory cortex on the top of the user's head;

Step 1c, use bandpass filtering and Laplace reference filtering for preprocessing:

The Laplacian reference filtering method is calculated as the mean value of the signal of this lead minus the original signal of the adjacent lead, using the formula:

Among them, d _ij is the distance between electrode i and electrode j, S _i is the electrode set within a certain distance with the i-th electrode as the center, w _ij is the calculated weight of the j-th electrode, and n is the total number of lead electrodes number, V _i is the original signal collected by a single lead,

is the lead signal of V _i after Laplace filtering;

Step 1d, use the autoregressive AR model to extract the features of the EEG signal:

The autoregressive AR model is used to extract the features of the motor imagery EEG signals of the preset rhythm of the electrodes of each channel of the user, using the formula:

in,

is the estimated voltage signal amplitude at time t, a _k is the kth AR model parameter, m is the AR model order,

is the voltage signal amplitude at time tk, ε is the estimation error, and n is the total number of lead electrodes.

Further, the step 2 specifically includes:

Multiply the amplitude L of the preset frequency band of the EEG signal on the left side of the head and the amplitude R of the preset frequency band of the EEG signal on the right side of the head by the weights w _L and w _R respectively, and the weighted sum is obtained to obtain the EEG-speed control signal C _Vvalue , using the formula:

M _Vvalue =w _L L+w _R R+b

Among them, w _L and w _R are the weights of the left and right motor imagery EEG signals, b is a constant offset term, L and R are the left and right motor imagery EEG signals, M _Vvalue is the EEG control coefficient, (M _Vvalue ) _max is the maximum value of the EEG control coefficient, (M _Vvalue ) _min is the minimum value of the EEG control coefficient, and C _Vvalue is the numerical variation coefficient of the linear velocity of the mobile robot. When C _Vvalue is greater than the preset constant value, the numerical value of the linear velocity of the robot increases. , when C _Vvalue is less than the preset constant value, the value of the robot linear velocity decreases.

Further, in the step 2, the initial values of w _L , w _R , and b are all +1.0, and the normalized minimum mean square error NLMS algorithm is used to adjust the weights adaptively.

Further, the step 3 specifically includes:

Step 3a, scan the environmental information.

Step 3b, using the Hector SLAM algorithm for synchronous positioning and mapping:

The grid size is defined, and the grid cell has three existence states: free, occupied and unknown. The bilinear filtering algorithm is used to estimate the probability that the environmental point cloud information occupies the grid, and the robot pose information is located.

Step 3c, using the A* algorithm for path planning:

Taking the position of the robot (x _n , y _n ) as the starting point, and taking the point with the shortest Euclidean distance between the free and unknown grids and the position of the robot as the target point (x _G , y _G ), the heuristic function h(n) adopts the formula:

Among them, (x _n , y _n ) represents the horizontal and vertical coordinates of the map of the current point, and (x _G , y _G ) represents the horizontal and vertical coordinates of the map of the target point.

Step 3d: Obtain the autonomous obstacle avoidance-speed control signal according to the planned path

The signal is a unit direction vector, the direction is that each parent node in the optimal path node set generated by the path planning points to the next child node, autonomous obstacle avoidance-speed control signal

Satisfy the formula:

Further, the step 4 specifically includes:

The control decision of the linear speed in the moving process of the mobile robot adopts the formula:

in,

Indicates the linear speed of the mobile robot during the forward process, and V _max represents the preset maximum linear speed;

During the forward process of the mobile robot, when it is located at each node of the planned path in the step 4a, according to the autonomous obstacle avoidance-speed control signal

angle compensation is performed in the direction of the

The beneficial effects of the present invention are:

1. The present invention can combine human brain intention control with robot autonomous control in the case of unknown environmental information, thereby improving the control efficiency and human-computer interaction ability in the unknown environment.

2. The present invention uses EEG to control the magnitude of the linear velocity of the mobile robot, uses the path independently planned by the mobile robot to control the direction of the linear velocity of the mobile robot, and enables the robot to complete obstacle avoidance and path planning in an unknown environment through the shared control method, which improves the performance of the mobile robot. Obstacle avoidance efficiency of mobile robots in unknown environments.

Description of drawings

1 is a system block diagram of a shared control method for an EEG mobile robot in an unknown environment provided by the present invention;

Fig. 2 is the action schematic diagram of the user's motor imagery task in the present invention;

FIG. 3 is a schematic diagram of an EEG signal collection position in the present invention;

4 is a schematic diagram of a bilinear filtering algorithm in the present invention;

FIG. 5 is a flowchart of the A* path planning algorithm in the present invention.

detailed description

The present invention will be further clarified below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

This embodiment provides a shared control method for an EEG mobile robot in an unknown environment. The electrical signals are linearly summed with adaptive weights to obtain the EEG-speed control signal; (3) The mobile robot obtains the autonomous obstacle avoidance-speed control signal according to the autonomous path planning; (4) The mobile robot is controlled by the EEG-speed Shared control of signals and autonomous obstacle avoidance - speed control signals, driving in unknown environments.

1. Collect the user's motor imagery EEG signals;

The video played on the screen that the user observes guides the user to generate motor imagery EEG signals. When the user imagines the movement of his hands and rests, the first lead electrode is used to collect the motor imagery EEG signal of the left motor sensory cortex of the user, and the second lead electrode is used to detect the movement of the right side of the user's head. Motor imagery EEG signals of the sensory cortex were collected.

2. Preprocessing and feature extraction of the user's EEG signals

2.1 Preprocessing using bandpass filtering and Laplace reference filtering

In this embodiment, an 8-32 Hz band-pass filter is used for the collected EEG signal to filter out the noise in the collection process and improve the signal-to-noise ratio of the signal. The bandpass filtered signal is then filtered by the Laplacian reference method. The Laplacian reference filtering method is calculated as the mean value of the signal of this lead minus the original signal of the adjacent lead, using the formula:

is the lead signal of V _i after Laplace filtering;

2.2 Feature extraction of EEG signals using autoregressive AR model

in,

is the voltage signal amplitude at time tk, ε is the estimation error, and n is the total number of lead electrodes;

3. Find the EEG-speed control signal

M _Vvalue =w _L L+w _R R+b

The initial values of w _L and w _R used in this embodiment are +1.0, and the normalized minimum mean square error NLMS algorithm is used to adjust the weights adaptively.

Input the filtered signal into the NLMS adaptive filter, given the weight w, and calculate the output value:

y(k)=w(k) ^T V(k)

Calculate the estimated error e(k):

e(k)=d(k)-y(k)

Weight update:

Among them, k is the number of updates, V(k) is the kth input signal, α and μ are the adjustment factors, and d(k) is the reference signal.

4. Autonomous path planning for mobile robots

4.1 Scan environmental information

Use two-dimensional lidar to scan the surrounding environment to obtain environmental information. The angle of each frame of the two-dimensional lidar is related to the resolution. The resolution is set to 360, and each degree corresponds to a value. The two-dimensional lidar obtains environmental information by scanning in the form of a circle on a two-dimensional plane. The two-dimensional lidar information refers to the distance of the detected obstacle, which represents the distance value of the real obstacle.

4.2 Simultaneous positioning and mapping using Hector SLAM algorithm

Define the grid size and discretize the actual environment map. The basic principle is: take the coordinates of the robot's location as the center, spread the size of the unit grid around until the virtual grid covers the plane, and then put the two-dimensional lidar information into the ground. into the corresponding grid, and all the two-dimensional lidar information falling into the grid is represented by the occupancy probability value of the grid. The bilinear filtering algorithm is used to estimate the probability that the environmental point cloud information occupies the grid. The grid cells have three existence states: free, occupied and unknown.

Given a continuous map coordinate P _m , occupying the value M(P _m ), for the gradient

After approximating with the point P _ij (i,j=0,1), perform linear interpolation along the axis and the axis,

Partial derivatives can be approximated as:

After building the raster map, align the raster map with the 2D lidar information. Using the Gauss-Newton method:

For robot pose

ξ=(P _x ,P _y ,ψ) ^T

Solve to satisfy the formula

At this point the laser scan is optimally aligned with the map. where S _i (ξ) is the global coordinate of the scanning endpoint S _i =(S _i,x ,S _i,y ) ^T . There is a S _i (ξ) that can represent the coordinates of the robot in the global coordinate system:

The equation M(S _i (ξ)) returns the map value at Si ₍ ξ). For a given initial estimate of ξ, to estimate ξ, the measurement error is optimized according to the following equation:

Do the first-order Taylor expansion of M(S _i (ξ+Δξ)):

The following equation is minimized by setting the partial derivative of Δξ to 0:

Solving for Δξ leads to the minimization problem of the Gauss-Newton equation:

in:

is the partial derivative of the occupancy grid:

4.3 Using A* algorithm for path planning

Then the cost of moving the robot to reach each node:

Among them, f(n) is the total cost of the node (x _n , y _n ), when selecting the next node to be traversed, select the node with the lowest total cost, that is, the node with the highest priority. g(n) is the cost of the node (x _n , y _n ) from the starting point (x ₀ , y ₀ ).

The flow of the A* algorithm is as follows:

1) Initialize open_set and close_set;

2) Add the starting point to open_set, and set the cost value to 0 (the highest priority);

3) If open_set is not empty, select node n with the highest priority from open_set, delete node n from open_set, and add it to close_set;

4) If the node n is the end point, the parent node is gradually traced from the end point until the start point is reached, and the found result path is returned, and the algorithm ends;

If node n is not the end point, judge whether node n has adjacent node m;

5) If the adjacent node m does not exist, then jump to step 3);

If the adjacent node m exists, add the node m to the open_set, and jump to step 3).

4.4 Seek autonomous obstacle avoidance-speed control signal

The mobile robot changes its travel direction according to the autonomously planned path, and this signal controls the direction of the linear velocity. The signal is a unit direction vector, the direction is that each parent node in the optimal path node set generated by the path planning points to the next child node, autonomous obstacle avoidance-speed control signal

Satisfy the formula:

5. Shared control of mobile robots by EEG-speed control signals and autonomous obstacle avoidance-speed control signals

in,

During the forward process of the mobile robot, when it is located at each node in the path planned by the mobile robot autonomously, according to the autonomous obstacle avoidance-speed control signal

angle compensation is performed in the direction of the

The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features.

Claims

A shared control method for an EEG mobile robot in an unknown environment, comprising the following steps:

Step 1. Collect the user's motor imagery EEG signal and perform preprocessing and feature extraction;

Step 2. Perform an adaptive weight linear summation on the left and right EEG signals to obtain an EEG-speed control signal;

Step 3, the mobile robot obtains the autonomous obstacle avoidance-speed control signal according to the autonomous path planning;

Step 4. The mobile robot is under the shared control of the EEG-speed control signal and the autonomous obstacle avoidance-speed control signal, and drives in an unknown environment.
The shared control method for an EEG mobile robot in an unknown environment according to claim 1, characterized in that: in the step 1, the motor imagery EEG signal of the user is collected and subjected to preprocessing and feature extraction, specifically:

Step 1a, using the first lead electrode to collect the motor imagery EEG signal of the left motor sensory cortex on the top of the user's head;

Step 1b, using the second lead electrode to collect the motor imagery EEG signal of the right motor sensory cortex on the top of the user's head;

Step 1c, use bandpass filtering and Laplace reference filtering for preprocessing:

The Laplacian reference filtering method is calculated as the mean value of the signal of this lead minus the original signal of the adjacent lead, using the formula:

Among them, d ij is the distance between electrode i and electrode j, S i is the electrode set within a certain distance with the i-th electrode as the center, w ij is the calculated weight of the j-th electrode, and n is the total number of lead electrodes V i is the original signal collected by a single lead, and V i LAP is the lead signal of V i after Laplace filtering;

Step 1d, use the autoregressive AR model to extract the features of the EEG signal:

The autoregressive AR model is used to extract the features of the motor imagery EEG signals of the preset rhythm of the electrodes of each channel of the user, using the formula:

Among them, Vi LAP (t) is the estimated voltage signal amplitude at time t, a k is the kth AR model parameter, m is the AR model order, Vi LAP (tk) is the voltage signal amplitude at time tk, and ε is Estimation error, n is the total number of lead electrodes.
The shared control method for an EEG mobile robot in an unknown environment according to claim 1, wherein in the step 2, the EEG signals on the left and right sides are linearly summed with adaptive weights to obtain an EEG-speed control signal. for:

Multiply the amplitude L of the preset frequency band of the EEG signal on the left side of the head and the amplitude R of the preset frequency band of the EEG signal on the right side of the head by the weights w L and w R respectively, and the weighted sum is obtained to obtain the EEG-speed control signal C Vvalue , using the formula:

M Vvalue =w L L+w R R+b

Among them, w L and w R are the weights of the left and right motor imagery EEG signals, b is a constant offset term, L and R are the left and right motor imagery EEG signals, M Vvalue is the EEG control coefficient, (M Vvalue ) max is the maximum value of the EEG control coefficient, (M Vvalue ) min is the minimum value of the EEG control coefficient, and C Vvalue is the numerical variation coefficient of the linear velocity of the mobile robot. When C Vvalue is greater than the preset constant value, the numerical value of the linear velocity of the robot increases. , when C Vvalue is less than the preset constant value, the value of the robot linear velocity decreases.
The shared control method for an EEG mobile robot in an unknown environment according to claim 3, wherein the initial values of w L and w R are +1.0, and the normalized minimum mean square error NLMS algorithm is used to adjust the weights adaptively.
The shared control method for an EEG mobile robot in an unknown environment according to claim 1, wherein in the step 3, the mobile robot obtains an autonomous obstacle avoidance-speed control signal according to an autonomous path planning, specifically:

Step 3a, scan the environmental information;

Step 3b, using the Hector SLAM algorithm for synchronous positioning and mapping:

Define the size of the grid, the grid cell has three existence states of free, occupied and unknown, and use the bilinear filtering algorithm to estimate the probability that the environmental point cloud information occupies the grid, and locate the robot pose information;

Step 3c, using the A* algorithm for path planning:

Taking the position of the robot (x n , y n ) as the starting point, and taking the point with the shortest Euclidean distance between the free and unknown grids and the position of the robot as the target point (x G , y G ), the heuristic function h(n) adopts the formula:

Among them, (x n , y n ) represents the horizontal and vertical coordinates of the map of the current point, and (x G , y G ) represents the horizontal and vertical coordinates of the map of the target point;

Step 3d: Obtain the autonomous obstacle avoidance-speed control signal according to the planned path

The signal is a unit direction vector, the direction is that each parent node in the optimal path node set generated by the path planning points to the next child node, autonomous obstacle avoidance-speed control signal
Satisfy the formula:
The shared control method for an EEG mobile robot in an unknown environment according to claim 1, wherein:

In the step 4, the mobile robot is under the shared control of the EEG-speed control signal and the autonomous obstacle avoidance-speed control signal, and drives in an unknown environment, specifically:

The control decision of the linear speed in the moving process of the mobile robot adopts the formula:

in,
Represents the linear velocity in the forward process of the mobile robot, and V max represents the preset maximum linear velocity; during the forward process of the mobile robot, when it is located at each node of the planned path in the step 4a, according to the autonomous obstacle avoidance-speed control signal
angle compensation is performed in the direction of the