CN113002558B - Intelligent driving assisting system and method for disabled people based on electroencephalogram signals - Google Patents
Intelligent driving assisting system and method for disabled people based on electroencephalogram signals Download PDFInfo
- Publication number
- CN113002558B CN113002558B CN202110343762.1A CN202110343762A CN113002558B CN 113002558 B CN113002558 B CN 113002558B CN 202110343762 A CN202110343762 A CN 202110343762A CN 113002558 B CN113002558 B CN 113002558B
- Authority
- CN
- China
- Prior art keywords
- metal strip
- signals
- electroencephalogram
- processing module
- control module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 17
- 210000004556 brain Anatomy 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 47
- 238000013528 artificial neural network Methods 0.000 claims abstract description 14
- 238000004891 communication Methods 0.000 claims abstract description 11
- 239000002184 metal Substances 0.000 claims description 65
- 238000005868 electrolysis reaction Methods 0.000 claims description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 230000007787 long-term memory Effects 0.000 claims description 2
- 230000005611 electricity Effects 0.000 abstract description 12
- 210000001503 joint Anatomy 0.000 abstract 1
- 230000009471 action Effects 0.000 description 14
- 230000033001 locomotion Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 241001513109 Chrysocephalum apiculatum Species 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000000192 social effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/20—Workers
- A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Pathology (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Mechanical Engineering (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physiology (AREA)
- Psychiatry (AREA)
- Transportation (AREA)
- Human Computer Interaction (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention relates to an electroencephalogram signal-based disabled person auxiliary intelligent driving system, which comprises an electroencephalogram signal acquisition cap, a signal processing module and a vehicle control module, wherein the signal processing module is respectively in wireless communication connection with the electroencephalogram signal acquisition cap and the vehicle control module, the electroencephalogram signal acquisition cap is worn on the head of a disabled person driver, and the vehicle control module is in communication connection with a vehicle; the brain electricity information acquisition cap acquires motor imagery brain electricity signals of six channels of a disabled driver, the signal processing module decodes the motor imagery brain electricity signals by utilizing at least two convolution neural networks, outputs brain electricity code signals and sends the brain electricity code signals to the vehicle control module, and the vehicle control module carries out interface butt joint on the brain electricity code signals and driving vehicle operation instructions to drive the vehicle to operate so as to realize auxiliary intelligent driving.
Description
Technical Field
The invention relates to the field of autonomous driving of disabled persons, in particular to an electroencephalogram signal-based disabled person auxiliary intelligent driving system and method.
Background
Disabled persons are a disadvantaged population of social life, and their lives face a wide variety of difficulties and problems due to physical reasons. The automobile is an indispensable transportation means in modern life of people, along with the development of economy and science technology, the driving conditions of disabled people are gradually relaxed, and at present, according to national regulations, the disabled people apply for drivers license to meet certain requirements, and according to different parts of physical disabilities, corresponding drivers license can be applied, so that the life and working efficiency of the social weakness group are greatly improved. However, no matter which part of the limb is disabled, certain difficulty and limitation are caused in driving, and for people with serious limb disabilities, drivers cannot apply for drivers' license at present, and autonomous driving cannot be realized.
In the prior art, the disabled person meeting driving requirements needs to use a special automobile for the disabled person when driving, the production and sales of the special automobile for the disabled person are not realized in China, manual driving is finished by mainly adding auxiliary devices to the automobile to help disabled persons, for example, a fixed support is arranged at a plastic shell at the lower part of an automobile steering wheel, a manual rod is arranged at the right side of a cab to assist driving, but the space of the cab is occupied to cause inconvenience to the driver, and in addition, the installation of a plurality of auxiliary driving devices easily causes confusion of an accelerator and a brake, and the devices of the auxiliary devices have great damage to the original automobile form, the original layout of the automobile is damaged, the overall scientificity is not great, the special automobile steering system is not suitable for drivers of various types and different disabled types, and meanwhile, certain potential safety hazards exist for operating driving of the driver.
In summary, the prior art has the following disadvantages: ① The adding device has great damage to the original automobile form device, the overall transformation is also not scientific, and the driving risk exists; ② The installation of the auxiliary driving instrument occupies the space of the cab, which causes inconvenience to the driver; ③ For the population with serious physical disabilities, even if using the auxiliary driving device, autonomous driving cannot be realized; ④ The auxiliary manual rods for the operation of a plurality of automobiles are all placed on the hands, so that confusion is easily caused, and potential safety hazards for driving exist. ⑤ To customize the driving assistance apparatus privately according to the disabled condition of the driver, the apparatus does not have uniform applicability.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide an electroencephalogram signal-based auxiliary intelligent driving system and an electroencephalogram signal-based auxiliary intelligent driving method with wide applicability, so that people with different disabled degrees can realize autonomous driving without making specific driving device instruments according to different disabled parts, and patients with different disabled parts can use the same electroencephalogram signal-based auxiliary driving system, thereby saving resources and improving the quality of life and working efficiency of disabled people.
The aim of the invention can be achieved by the following technical scheme:
The system comprises a brain electrical signal-based disabled person auxiliary intelligent driving system, a brain electrical signal acquisition cap, a signal processing module and a vehicle control module, wherein the signal processing module is respectively in wireless communication connection with the brain electrical signal acquisition cap and the vehicle control module, the brain electrical signal acquisition cap is worn on the head of a disabled person driver and is used for acquiring motor imagery brain electrical signals of the disabled person driver, and the vehicle control module is in communication connection with the vehicle and is used for driving the vehicle to run;
The brain electricity information acquisition cap acquires motor imagery brain electrical signals of six channels of a disabled person driver, the motor imagery brain electrical signals are amplified and then sent to the signal processing module, the signal processing module decodes the motor imagery brain electrical signals by utilizing at least two convolution neural networks, brain electrolysis code signals are output and sent to the vehicle control module, and the vehicle control module interfaces the brain electrolysis code signals with a driving vehicle running instruction to drive the vehicle to run so as to realize auxiliary intelligent driving.
Further, the brain electricity information acquisition cap comprises a support frame formed by assembling a plurality of arc-shaped metal strips, and channel electrodes, signal amplifiers and switches which are respectively arranged on the support shell, wherein the number of the channel electrodes is six and the channel electrodes are respectively used for acquiring motor imagery brain electrical signals of six channels, and the signal amplifiers are respectively connected with the channel electrodes and the signal processing module and are used for amplifying the motor imagery brain electrical signals acquired by the channel electrodes and sending the motor imagery brain electrical signals to the signal processing module.
Further, when the support frame is worn on the top of the head, six channel electrodes respectively correspond to OZ, O, O, C, cz and C six sites in the international 10-20 system, and the electroencephalogram signals of the six sites are respectively collected.
Furthermore, the electroencephalogram information acquisition cap further comprises a signal lamp, wherein the signal lamp is arranged on the support frame and is connected with the channel electrode, and the signal lamp is used for indicating whether the channel electrode successfully acquires the motor imagery electroencephalogram signals.
Further, the support frame comprises a first metal strip, a second metal strip and a third metal strip, wherein the first metal strip and the second metal strip are arranged in parallel, and the third metal strip is respectively connected with the midpoints of the first metal strip and the second metal strip to form a soil-shaped structure and is matched with the frame in a head top shape;
the three channel electrodes are a group and are respectively arranged on the first metal strip and the second metal strip, the signal amplifier is arranged on the third metal strip and is positioned between the first metal strip and the second metal strip, and the switch is arranged at one end of the second metal strip.
Further, one of the three channel electrodes on the first metal strip is arranged in the center of the first metal strip, and the other two channel electrodes are symmetrically arranged on two sides of the channel electrode; one of the three channel electrodes on the second metal strip is arranged in the center of the second metal strip, the other two channel electrodes are symmetrically arranged on two sides of the channel electrode, and the channel electrode is a dry electrode.
Further, the signal processing module is respectively connected with the brain electricity information acquisition cap and the vehicle control module through Bluetooth communication.
Further, the signal processing module comprises a Bluetooth module and a processing module, and the processing module is used for preprocessing and decoding the amplified signal sent by the electroencephalogram information acquisition cap and outputting an electroencephalogram code signal to the vehicle control module; the vehicle control module interfaces the brain electrolysis code signals with the driving vehicle running instructions, and forms a one-to-one mapping relation between the brain electrolysis code signals and the driving vehicle running instructions, and converts the brain electrolysis code signals into the vehicle running instructions in real time, so that auxiliary intelligent driving is realized.
Still further, the preprocessing includes artifact removal, common average referencing, calibration and bandpass filtering, and the decoding includes long and short term memory neural networks, impulse neural networks and EEGNet neural networks.
The working method of the electroencephalogram signal-based disabled person auxiliary intelligent driving system comprises the following steps of:
S1: the brain electricity information acquisition cap is worn on the head of a disabled person driver;
S2: the electroencephalogram information acquisition cap acquires motor imagery electroencephalogram signals of six channels of the disabled person driver and sends the motor imagery electroencephalogram signals to the signal processing module;
S3: the signal processing module sequentially performs preprocessing operation on the received motor imagery electroencephalogram signals;
s4: inputting the preprocessed motor imagery electroencephalogram signals into at least two convolution neural networks respectively for classification decoding;
S5: comparing whether the results of the classification decoding of different convolutional neural networks are consistent, if so, executing the step S6, and if not, returning to the step S2;
s6: transmitting the classification decoding result to a vehicle control module;
S7: the vehicle control module is matched with the driving vehicle running instruction according to the received classification decoding result to obtain a corresponding vehicle running instruction;
s8: and carrying out corresponding operation on the driving vehicle according to the vehicle running instruction, so as to realize auxiliary intelligent driving.
Compared with the prior art, the invention has the following advantages:
1) According to the invention, by arranging the electroencephalogram information acquisition cap, motor imagery electroencephalogram signals in the process of driving a vehicle can be decoded, so that various disabled people can intelligently drive by utilizing the electroencephalogram signals, and the life quality and level of disabled people, especially severe disabled groups, are greatly improved;
2) The method maps the decoded brain electrical signals and driving operation instructions one by one, and operates the brain electrical signals and the driving operation instructions through interface information transmission, and does not add and change equipment and instruments to the original design of the automobile, so that potential safety hazards caused by unreasonable transformation are greatly reduced;
3) Compared with auxiliary driving equipment such as a handle, the system has higher applicability and uniformity, and even if the parts with physical disabilities are different, no targeted electroencephalogram signal processing is needed, so that the system has higher applicability;
4) The brain electrical signal decoding in the invention uses two different neural networks, and the decoded signals have higher accuracy and stronger reliability, so that the safety and reliability coefficients of the obtained driving instruction are more stable;
5) The intelligent driving system can widely assist various disabled people in intelligent driving, can greatly improve the living standard and efficiency of the disabled people after being popularized and used, and has good economic benefit and social effect.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic structural view of the brain electrical information collection cap of the present invention;
FIG. 3 is a schematic diagram of a signal processing module performing signal processing and decoding;
FIG. 4 is a flow chart of a first method for controlling the speed of a driving maneuver according to the present disclosure;
fig. 5 is a flowchart of a second speed control method for driving actions according to the embodiment.
1. The brain electrical information acquisition cap, 11, a support frame, 111, a first metal strip, 112, a second metal strip, 113, a third metal strip, 12, a channel electrode, 13, a signal amplifier, 14, a switch, 15, a signal lamp, 2, a signal processing module, 3 and a vehicle control module.
Detailed Description
The invention will now be described in detail with reference to the drawings and specific examples. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Examples
The invention discloses an electroencephalogram signal-based auxiliary intelligent driving system and method for disabled people, which are one of brain-computer interface technologies, wherein mechanical control is performed by using an electroencephalogram signal, decoding of motor imagery electroencephalogram signals can realize that a limb disabled person can control a mechanical arm by using the electroencephalogram signal, and the disabled person can be assisted in completing a certain range of actions. The brain electric signal is utilized to assist the disabled to carry out intelligent driving without adding automobile auxiliary devices and instruments, meanwhile, patients with different degrees of disabilities can be considered, no matter what kind of physical disabilities are, a set of brain electric assisting driving equipment can be shared, private customization is not required according to the condition of the disabled patients, the brain electric signal assisting driving device has good usability, and the life quality and the life level of the disabled patients are improved well.
As shown in fig. 1, the system comprises a electroencephalogram information acquisition cap 1, a signal processing module 2 and a vehicle control module 3, wherein the signal processing module 2 is respectively in wireless communication connection with the electroencephalogram information acquisition cap 1 and the vehicle control module 3, the electroencephalogram information acquisition cap 1 is worn on the head of a disabled person driver and is used for acquiring motor imagery electroencephalogram signals of the disabled person driver, the vehicle control module 3 is in communication connection with the vehicle and is used for driving the vehicle to run, and in the embodiment, the signal processing module 2 is respectively in communication connection with the electroencephalogram information acquisition cap 1 and the vehicle control module 3 through Bluetooth.
As shown in fig. 2, the electroencephalogram information acquisition cap 1 includes a supporting frame 11 formed by assembling a plurality of arc-shaped metal strips, and a channel electrode 12, a signal amplifier 13 and a switch 14 which are respectively arranged on a supporting shell, and in this embodiment, the electroencephalogram information acquisition cap 1 further includes a signal lamp 15.
The support frame 11 comprises a first metal strip 111, a second metal strip 112 and a third metal strip 113, wherein the first metal strip 111 and the second metal strip 112 are arranged in parallel, the third metal strip 113 is respectively connected with the midpoints of the first metal strip 111 and the second metal strip 112 to form a soil-shaped structure, and the frame is matched with the head top in a matching manner.
The six channel electrodes 12 are respectively used for acquiring motor imagery electroencephalogram signals of the six channels, when the support frame 11 is worn on the top of the head, the six channel electrodes 12 respectively correspond to six sites of OZ, O1, O2, C3, cz and C4 in the international 10-20 system, and the electroencephalogram signals of the six sites are respectively acquired. Specifically, three channel electrodes 12 are arranged on the first metal strip 111 and the second metal strip 112 respectively, one of the three channel electrodes 12 on the first metal strip 111 is arranged in the center of the first metal strip 111, and the other two channel electrodes 12 are symmetrically arranged on two sides of the channel electrode 12; one of the three channel electrodes 12 on the second metal strip 112 is disposed in the center of the second metal strip 112, and the other two channel electrodes 12 are symmetrically disposed on two sides of the channel electrode 12, and the channel electrode 12 is a dry electrode.
The signal amplifier 13 is respectively connected with the channel electrode 12 and the signal processing module 2, and is used for amplifying the motor imagery electroencephalogram signal acquired by the channel electrode 12 and sending the motor imagery electroencephalogram signal to the signal processing module 2, specifically, the signal amplifier 13 is arranged on the third metal strip 113, is located between the first metal strip 111 and the second metal strip 112, and the switch 14 is arranged at one end of the second metal strip 112.
The signal lamps 15 are disposed on the support frame 11 and are connected with the channel electrode 12, so as to indicate whether the channel electrode 12 successfully collects the motor imagery electroencephalogram signals, specifically, the signal lamps 15 are respectively disposed at two ends of the first metal strip 111.
In this embodiment, the supporting frame 11 is made of metal, and can be adjusted and adapted to tightness, so that a user can adapt to head types of different sizes when wearing the device, a channel for collecting signals is closely corresponding to a head channel of a driver, dislocation is avoided, a waterproof fabric is arranged on one side, which is in contact with the head, of the driver, the device is closely contacted with the head of the driver, the head is worn for a long time, six channel electrodes 12 are arranged on the electroencephalogram information collecting cap 1 for collecting brain motion image electroencephalogram signals, and on the basis of an international 10-20 system, electroencephalogram signals of OZ, O1, O2, C3, cz and C4 channels are respectively selected, and the types of the channel electrodes 12 are dry electrodes.
The left side of the second metal strip 112 is provided with a switch button, two sides of the first metal strip 111 are provided with signal lamps 15, when the brain electric signals can be acquired and worn correctly, the signal lamps 15 are on, otherwise, the signal lamps 15 continuously flash.
There is wireless signal amplifier 13 below third metal strip 113, amplifies the signal of gathering, and yellow button is switch 14, and when switch 14 was opened and wireless connection was normal, yellow switch 14 was yellow lamp and is bright, and yellow switch 14 button scintillation when the connection was problematic, and wireless connection mode is bluetooth connection, is equipped with signal amplification, digital to analog conversion and bluetooth module inside, transmits brain electricity data to signal processing and decoding module through bluetooth.
The signal processing module 2 comprises a Bluetooth module and a processing module, and the processing module is used for preprocessing and decoding the amplified signal sent by the electroencephalogram information acquisition cap 1 and outputting an electroencephalogram code signal to the vehicle control module 3; the vehicle control module 3 interfaces the brain electrolysis code signals with the driving vehicle running instructions, forms a one-to-one mapping relation between the brain electrolysis code signals and the driving vehicle running instructions, converts the brain electrolysis code signals into the vehicle running instructions in real time, and realizes auxiliary intelligent driving, wherein preprocessing comprises artifact removal, common average reference, calibration and band-pass filtering, decoding processing adopts at least two different convolutional neural networks, whether decoding is accurate or not is judged by comparing different classification results of the two convolutional neural networks, and the convolutional neural networks can be obtained by acquiring and classifying and training motor imagery brain signals of each driver by adopting existing convolutional neural networks such as a long-short-term memory neural network, a pulse neural network, a EEGNet and the like.
The working principle of the invention is as follows:
The electroencephalogram acquisition cap 1 is worn by a disabled person driver, acquires a motor imagery electroencephalogram signal which is generated by the driver in a imagery way for driving corresponding operation actions, wherein the motor imagery electroencephalogram signal is an electroencephalogram signal when the driver imagines corresponding driving actions, and the acquired motor imagery electroencephalogram signal is amplified by the signal amplifier 13 and then transmitted to the signal processing module 2 through Bluetooth wireless transmission;
the signal processing module 2 performs signal processing, respectively utilizes two kinds of neural networks to perform identification decoding, and transmits the identified brain electrolysis code information to the vehicle control module 3 through Bluetooth;
the vehicle control module 3 interfaces the decoded signals with the running instructions of the driving vehicle, realizes one-to-one mapping between brain electrolysis code information and the running instructions of the driving vehicle, performs corresponding operation on the driving vehicle, and realizes intelligent driving by accurately utilizing the decoded brain electrical signals in real time.
The electroencephalogram signal is collected and is a motor imagery electroencephalogram signal of a driver, the motor imagery is the imagery of the driver on driving actions when the driver performs driving operation, the imagery comprises hand rotation of a steering wheel, foot stepping of a brake, accelerator and the like, for example, the driver can imagine actions of two hands rotating the steering wheel in the brain, and the actions comprise speed in the rotating process. When a driver brakes or steps on a throttle, the driver can imagine the corresponding motions of feet in the brain, all the motions involved in the driving process can acquire brain electrical signals of the corresponding motor imagination motions before the driver drives, and after preprocessing the acquired signals, at least two convolutional neural networks are used for performing the pre-training of classification decoding.
Besides the driving actions involved in the driving process, the motor imagery action brain signals of the foot and hand movements are further acquired, because the driver needs hand and foot movements in the driving process, such as the conversion between accelerator and brake, etc.
The vehicle control module 3 is provided with a small display, in which a virtual driver and virtual display of his hands and feet in the cab are displayed, and positions of the virtual driver and the feet in the cab are adjusted in real time according to changes of the motor imagery electroencephalogram signals of the driver, for example, the feet are at the throttle position and the hands are at the steering wheel position, and the display is provided with a real-time one-to-one correspondence between the virtual driver and the real driver, and when the actions or body parts of the driver are changed, the positions corresponding to the virtual driver in the screen are also changed in real time. The main function is to make the driver look over own hand and foot position in real time to prevent the position disorder, when driving starts, the initial position of driver in the display is that both hands are put to the steering wheel, and the foot is put to the throttle position.
Control of the driving action speed can be achieved using two schemes:
scheme one is shown in fig. 4:
The driver performs motor imagery on the same driving action, and the speed is mainly different in the imagery process and is divided into three grades of slow, medium and fast. After the driving action motor imagery electroencephalogram signal is decoded, the speed is further identified, and the speed is input to an interface after being identified, and driving operation is performed.
Scheme two is shown in fig. 5:
The speed module is added at the side of the small display, a driver only needs to carry out motor imagery of driving actions, the collected motor imagery does not relate to further identification of speed, and the speed can be selected on the display and is classified into three grades of slow, medium and fast. After the driver selects the corresponding speed on the display, the brain electricity can be driven only by driving motor imagination.
As shown in fig. 3, the working method of the electroencephalogram signal-based disabled person auxiliary intelligent driving system comprises the following steps:
s1: the electroencephalogram information acquisition cap 1 is worn on the head of a disabled person;
S2: the electroencephalogram information acquisition cap 1 acquires motor imagery electroencephalogram signals of six channels of a disabled person driver and sends the motor imagery electroencephalogram signals to the signal processing module 2;
S3: the signal processing module 2 sequentially performs preprocessing operation on the received motor imagery electroencephalogram signals;
s4: respectively inputting the preprocessed motor imagery electroencephalogram signals into at least two different convolutional neural networks for classification decoding;
S5: comparing whether the results of the classification decoding of different convolutional neural networks are consistent, if so, executing the step S6, and if not, returning to the step S2;
S6: transmitting the result of the classification decoding to the vehicle control module 3;
S7: the vehicle control module 3 is matched with the driving vehicle running instruction according to the received classification decoding result to obtain a corresponding vehicle running instruction;
s8: and carrying out corresponding operation on the driving vehicle according to the vehicle running instruction, so as to realize auxiliary intelligent driving.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions may be made without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (3)
1. The intelligent driving assisting system for the disabled based on the electroencephalogram signals is characterized by comprising an electroencephalogram information acquisition cap (1), a signal processing module (2) and a vehicle control module (3), wherein the signal processing module (2) is respectively in wireless communication connection with the electroencephalogram information acquisition cap (1) and the vehicle control module (3), the electroencephalogram information acquisition cap (1) is worn on the head of a disabled driver and is used for acquiring motor imagery electroencephalogram signals of the disabled driver, and the vehicle control module (3) is in communication connection with the vehicle and is used for driving the vehicle to run;
The electroencephalogram information acquisition cap (1) acquires motor imagery electroencephalogram signals of six channels of a disabled driver, the motor imagery electroencephalogram signals are amplified by signals and then are sent to the signal processing module (2), the signal processing module (2) decodes the motor imagery electroencephalogram signals by utilizing at least two convolution neural networks, brain electrolysis code signals are output and sent to the vehicle control module (3), and the vehicle control module (3) interfaces the brain electrolysis code signals with driving vehicle running instructions to drive the vehicle to run so as to realize auxiliary intelligent driving;
The electroencephalogram information acquisition cap (1) comprises a support frame (11) formed by assembling a plurality of arc-shaped metal strips, channel electrodes (12), signal amplifiers (13) and a switch (14) which are respectively arranged on a support shell, wherein the number of the channel electrodes (12) is six and are respectively used for acquiring motor imagery electroencephalogram signals of six channels, and the signal amplifiers (13) are respectively connected with the channel electrodes (12) and the signal processing module (2) and are used for amplifying the motor imagery electroencephalogram signals acquired by the channel electrodes (12) and sending the motor imagery signals to the signal processing module (2);
the electroencephalogram information acquisition cap (1) further comprises a signal lamp (15), wherein the signal lamp (15) is arranged on the support frame (11) and is connected with the channel electrode (12) to indicate whether the channel electrode (12) successfully acquires motor imagery electroencephalogram signals or not;
the support frame (11) comprises a first metal strip (111), a second metal strip (112) and a third metal strip (113), wherein the first metal strip (111) and the second metal strip (112) are arranged in parallel, and the third metal strip (113) is respectively connected with the midpoints of the first metal strip (111) and the second metal strip (112) to form a soil-shaped structure and is matched with a frame in a head top shape;
The three channel electrodes (12) are arranged on the first metal strip (111) and the second metal strip (112) respectively, the signal amplifier (13) is arranged on the third metal strip (113) and is positioned between the first metal strip (111) and the second metal strip (112), and the switch (14) is arranged at one end of the second metal strip (112);
one of the three channel electrodes (12) on the first metal strip (111) is arranged in the center of the first metal strip (111), and the other two channel electrodes are symmetrically arranged on two sides of the channel electrode (12); one of the three channel electrodes (12) on the second metal strip (112) is arranged in the center of the second metal strip (112), the other two channel electrodes are symmetrically arranged on two sides of the channel electrode (12), and the channel electrode (12) is a dry electrode;
The signal processing module (2) comprises a Bluetooth module and a processing module, and the processing module is used for preprocessing and decoding the amplified signal sent by the electroencephalogram information acquisition cap (1) and outputting an electroencephalogram code signal to the vehicle control module (3); the vehicle control module (3) interfaces the brain electrolysis code signals with the driving vehicle running instructions, and forms a one-to-one mapping relation between the brain electrolysis code signals and the driving vehicle running instructions, and converts the brain electrolysis code signals into the vehicle running instructions in real time, so that auxiliary intelligent driving is realized;
The preprocessing comprises artifact removal, common average reference, calibration and band-pass filtering, and the convolutional neural network adopted in the decoding processing comprises a long-term memory neural network, a pulse neural network and a EEGNet neural network.
2. The brain-electric signal-based disabled person auxiliary intelligent driving system according to claim 1, wherein the signal processing module (2) is respectively connected with the brain-electric signal acquisition cap (1) and the vehicle control module (3) through Bluetooth communication.
3. A method of operating an electroencephalogram signal based disabled person-assisted intelligent driving system according to any one of claims 1-2, comprising the steps of:
S1: the electroencephalogram information acquisition cap (1) is worn on the head of a disabled person driver;
S2: the electroencephalogram information acquisition cap (1) acquires motor imagery electroencephalogram signals of six channels of a disabled person driver and sends the motor imagery electroencephalogram signals to the signal processing module (2);
s3: the signal processing module (2) sequentially performs preprocessing operation on the received motor imagery electroencephalogram signals;
s4: respectively inputting the preprocessed motor imagery electroencephalogram signals into at least two different convolutional neural networks for classification decoding;
S5: comparing whether the results of the classification decoding of different convolutional neural networks are consistent, if so, executing the step S6, and if not, returning to the step S2;
s6: transmitting the result of the classification decoding to a vehicle control module (3);
S7: the vehicle control module (3) is matched with the driving vehicle running instruction according to the received classification decoding result to obtain a corresponding vehicle running instruction;
s8: and carrying out corresponding operation on the driving vehicle according to the vehicle running instruction, so as to realize auxiliary intelligent driving.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110343762.1A CN113002558B (en) | 2021-03-30 | 2021-03-30 | Intelligent driving assisting system and method for disabled people based on electroencephalogram signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110343762.1A CN113002558B (en) | 2021-03-30 | 2021-03-30 | Intelligent driving assisting system and method for disabled people based on electroencephalogram signals |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113002558A CN113002558A (en) | 2021-06-22 |
| CN113002558B true CN113002558B (en) | 2024-06-28 |
Family
ID=76409479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110343762.1A Active CN113002558B (en) | 2021-03-30 | 2021-03-30 | Intelligent driving assisting system and method for disabled people based on electroencephalogram signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113002558B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN216139976U (en) * | 2021-03-30 | 2022-03-29 | 复旦大学 | Intelligent driving system is assisted to disabled person based on brain electrical signal |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN204515387U (en) * | 2015-04-03 | 2015-07-29 | 乐卡汽车智能科技（北京）有限公司 | Electroencephalogramsignal signal collection equipment and car-mounted terminal control system |
| CN205144580U (en) * | 2015-11-05 | 2016-04-13 | 西南交通大学 | Wearable driver EEG signal mining head area |
| CN106529421A (en) * | 2016-10-21 | 2017-03-22 | 燕山大学 | Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology |
| CN107065850A (en) * | 2016-12-12 | 2017-08-18 | 东南大学 | The four motorized wheels intelligent car systems and its control method of a kind of brain wave control |
| CN107168313A (en) * | 2017-05-17 | 2017-09-15 | 北京汽车集团有限公司 | Control the method and device of vehicle drive |
| CN108446020B (en) * | 2018-02-28 | 2021-01-08 | 天津大学 | Motor imagery idea control method fusing visual effect and deep learning and application |
| CN109743656B (en) * | 2019-03-15 | 2023-04-11 | 南京邮电大学 | Intelligent motion earphone based on electroencephalogram idea and implementation method and system thereof |
| CN111711661A (en) * | 2020-05-25 | 2020-09-25 | 五邑大学 | Intelligent vehicle monitoring method and system |
-
2021
- 2021-03-30 CN CN202110343762.1A patent/CN113002558B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN216139976U (en) * | 2021-03-30 | 2022-03-29 | 复旦大学 | Intelligent driving system is assisted to disabled person based on brain electrical signal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113002558A (en) | 2021-06-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106529421A (en) | Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology | |
| CN102874113A (en) | Safety protection system used in sudden disease of driver | |
| CN204989895U (en) | Electronic wheelchair intelligence control system | |
| CN103263324A (en) | Intelligent wheelchair system based on SSVEP (steady-state visual evoked potential) | |
| CN112141118A (en) | Intelligent driving system and control method | |
| CN209634401U (en) | A kind of vehicle steering wheel with health monitoring function | |
| CN102715988A (en) | Human-computer interactive intelligent wheelchair | |
| WO2003073931A1 (en) | Electrocardiography chart apparatus and method thereof | |
| CN216139976U (en) | Intelligent driving system is assisted to disabled person based on brain electrical signal | |
| CN210990244U (en) | Vehicle-mounted human body multi-parameter monitoring terminal | |
| CN113002558B (en) | Intelligent driving assisting system and method for disabled people based on electroencephalogram signals | |
| CN104783817A (en) | Double-mode detection platform for driving state of automobile driver | |
| CN112061135A (en) | Vehicle control method and system based on steering wheel and vehicle | |
| CN212631191U (en) | Language interaction blind guiding stick device | |
| CN203902538U (en) | Vehicle driving assistance system capable of supporting comfort level adjustment | |
| CN107432793A (en) | Safety-type Two-wheeled wheelchair | |
| CN200951161Y (en) | Intelligent interactive device based on head-acting for wheel chair | |
| CN206623798U (en) | A kind of control system | |
| CN203016945U (en) | Vehicle-mounted electrocardio monitoring device based on steering wheel | |
| CN205948002U (en) | Lead blind robot | |
| CN211985408U (en) | Vehicle driver health diagnosis system | |
| CN104181819A (en) | Human brain attention assessment system in simulated driving environment, and vehicle model driving environment thereof | |
| CN204915960U (en) | Recreation and body -building bicycle | |
| CN211022740U (en) | Wireless heart sound signal acquisition device | |
| CN215779268U (en) | Wheelchair driven by facial expression electroencephalogram signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |