CN107007407B - Wheelchair control system based on eye electricity - Google Patents
Wheelchair control system based on eye electricity Download PDFInfo
- Publication number
- CN107007407B CN107007407B CN201710235251.1A CN201710235251A CN107007407B CN 107007407 B CN107007407 B CN 107007407B CN 201710235251 A CN201710235251 A CN 201710235251A CN 107007407 B CN107007407 B CN 107007407B
- Authority
- CN
- China
- Prior art keywords
- key
- blink
- wheelchair
- electro
- detection unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005611 electricity Effects 0.000 title abstract description 4
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 230000004397 blinking Effects 0.000 claims abstract description 21
- 230000000875 corresponding effect Effects 0.000 claims abstract description 12
- 238000001914 filtration Methods 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims abstract description 5
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000035484 reaction time Effects 0.000 claims description 8
- 238000002474 experimental method Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000004424 eye movement Effects 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 9
- 208000003443 Unconsciousness Diseases 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000000034 method Methods 0.000 description 5
- 210000004556 brain Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 206010033799 Paralysis Diseases 0.000 description 2
- 206010002026 amyotrophic lateral sclerosis Diseases 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002570 electrooculography Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 206010015037 epilepsy Diseases 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 208000020431 spinal cord injury Diseases 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 210000001364 upper extremity Anatomy 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G5/00—Chairs or personal conveyances specially adapted for patients or disabled persons, e.g. wheelchairs
- A61G5/10—Parts, details or accessories
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1103—Detecting eye twinkling
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/398—Electrooculography [EOG], e.g. detecting nystagmus; Electroretinography [ERG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2203/00—General characteristics of devices
- A61G2203/10—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering
- A61G2203/18—General characteristics of devices characterised by specific control means, e.g. for adjustment or steering by patient's head, eyes, facial muscles or voice
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61G—TRANSPORT, PERSONAL CONVEYANCES, OR ACCOMMODATION SPECIALLY ADAPTED FOR PATIENTS OR DISABLED PERSONS; OPERATING TABLES OR CHAIRS; CHAIRS FOR DENTISTRY; FUNERAL DEVICES
- A61G2220/00—Adaptations of particular transporting means
- A61G2220/14—Cars
- A61G2220/145—Cars driven by a patient sitting in a wheelchair
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Surgery (AREA)
- Ophthalmology & Optometry (AREA)
- Dentistry (AREA)
- Physiology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Eye Examination Apparatus (AREA)
- Electrotherapy Devices (AREA)
Abstract
The invention discloses a kind of wheelchair control systems based on eye electricity, including electro-ocular signal acquisition module, blink detection unit, decision package and wheelchair execution unit；Electro-ocular signal acquisition module is for signal acquisition and is amplified filtering, then sends collected electro-ocular signal to blink detection unit；Blink detection unit extracts waveform feature parameter from electro-ocular signal, checks whether waveform feature parameter meets threshold condition, to judge whether to blink；Decision package is according to blink detection unit as a result, output order gives wheelchair execution unit；Wheelchair execution unit is according to the instruction results of identification, the corresponding action of controling wheelchair execution；The present invention can rapidly detect signal of blinking and is accurately positioned on some specific button, to realize only be used only a kind of eye motion can safely, fast and accurately and fully provide for wheelchair control instruction.
Description
Technical Field
The invention relates to an electric wheelchair control system, in particular to an electric wheelchair control system based on eye electricity.
Background
Wheelchairs are one of the common handicapped-helping devices and are used for helping people with leg movement function deficiency to regain walking ability. A Human Machine Interface (HMI) translates human intentions into control commands, and establishes a direct communication and control channel between the human and external devices. Thus, human interface technology is widely used for wheelchair control, including joysticks and keyboards. However, existing human-machine interfaces for wheelchair control are mostly manual, requiring the user to have good upper limb movement capabilities. However, some diseases (such as amyotrophic lateral sclerosis ALS, spinal cord injury SCI, etc.) may cause deep paralysis and loss of the four-limb movement function of the patient, so it is difficult to operate the wheelchair through the conventional manual man-machine interface, and it is necessary to provide a non-manual man-machine interaction mode for them to output control commands (such as forward, backward, steering, accelerating, decelerating, stopping, etc.) quickly, accurately and sufficiently. In general, the brain and eyes of deeply paralyzed patients still maintain normal functions, so researchers have proposed wheelchair control systems based on electroencephalogram (EEG) or Electrooculography (EOG). EEG and EOG are derived from brain and eye generated bioelectric signals, respectively, both of which are non-invasive signals.
At present, electroencephalogram signals based human-computer interfaces for wheelchair control mainly use electroencephalogram modes such as motor imagery, P300 and SSVEP, wherein the response time of the motor imagery (the time for generating a command) is shortest, but the number of control instructions capable of being provided is limited (usually only 2 or 3), and therefore the human-computer interfaces are mainly used for controlling the steering of a wheelchair. P300 and SSVEP can provide rich control commands, but have long response time, low accuracy, easy fatigue and unsuitability for long-time control. Furthermore, the motor imagery effects are highly variable from individual to individual, and therefore require a relatively large amount of training data, which is time consuming.
The current electro-oculogram-based human-machine interface has the following disadvantages: first, conscious and unconscious eye movements cannot be quickly and accurately identified; secondly, the accuracy of distinguishing different eye movements needs to be improved; third, the use of multiple eye movements makes the control relatively complex and tends to cause eye fatigue.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, the present invention provides a wheelchair control system that is faster, safer and more accurate.
In order to achieve the aim, the invention provides a wheelchair control system based on electro-oculogram, which comprises an electro-oculogram signal acquisition module, a blink detection unit, a decision unit and a wheelchair execution unit;
the eye electric signal acquisition module is used for acquiring signals, amplifying and filtering the signals and then transmitting the acquired eye electric signals to the blink detection unit;
the blink detection unit extracts waveform characteristic parameters from the eye electrical signals, the waveform characteristic parameters comprise an original signal maximum value, time delay of the appearance moment of the original signal maximum value relative to the key blinking moment, a difference signal maximum value and blinking duration, and whether the waveform characteristic parameters meet threshold conditions or not is checked, so that whether blinking exists or not is judged;
the decision unit outputs an instruction to the wheelchair execution unit according to the result of the blink detection unit;
the wheelchair executing unit controls the wheelchair to execute corresponding actions according to the identified instruction result;
the decision unit outputs an instruction to the wheelchair execution unit according to the result of the blink detection unit in the following mode;
the blink reaction time of each user is measured through initial calibration, namely the time delay t of the maximum occurrence time of the original signal relative to the blink time of the keypAverage value of (A) Tp(ii) a Obtaining the time delay t of each key in the wheel flashingpEach key is evaluated according to the following formula:
ei＝|tpi-Tp|；
wherein e isiIs the evaluation value of the key i, tpiIs the time delay T of the maximum value of the original signal in the electro-oculogram data segment of the key i relative to the key flashing timepThe blink reaction time of the current user; and then, selecting the key with the minimum evaluation value as the identification result of the flicker in the current round.
Preferably, the blink detection unit determines whether to blink according to the following method:
firstly, after each key flickers, extracting a segment of electro-oculogram data, wherein the segment of electro-oculogram data comprises electro-oculogram data of 100 and 500ms after the flicking moment begins; down-sampling the data segment, filtering by a low-pass filter with the cut-off frequency of 10Hz, and differentiating to obtain a differential signal; then extracting a plurality of waveform characteristic parameters including the maximum value a of the original signalmaxThe time delay t of the maximum value of the original signal relative to the key flashing timepMaximum value s of differential signalmaxAnd duration of blinking dpn；
Blink duration d in the present inventionpnIs the time interval between the maximum value and the minimum value of the differential signal; the blink detection unit detects whether the data segment contains blink waveforms according to threshold parameters set by initial calibration before each experiment; specifically, the system sets three thresholds, respectively amplitude threshold AthSpeed threshold SthAnd duration threshold [ D ]1,D2](ii) a A extracted from the electrooculogram datamaxAnd smaxRespectively exceeds an amplitude threshold and a velocity threshold, and dpnWithin a range defined by the duration threshold, it is assumed that a blink is detected in the segment of electro-ocular data.
Preferably, only when a certain key is selected at least twice in the latest three continuous flashes as a recognition result, it is determined that the user is blinking synchronously along with the flashes of the key, and then a corresponding wheelchair control instruction is output; if no key meeting the condition exists, the system enters a new flash, and no control instruction is output.
Preferably, if a certain key is selected as the recognition result in the current round of flickering, the key is directly regarded as the final result and a corresponding wheelchair control instruction is output.
The invention has the beneficial effects that: the invention can quickly detect the blink signal and accurately position the blink signal on a specific key, thereby realizing that the wheelchair control instruction can be safely, quickly, accurately and fully provided by only using one eye action. The existing other eye electric control systems can not distinguish conscious eye movements from unconscious eye movements well, so that a certain false alarm rate exists, and the problem can be solved well; the invention also has the advantages of low equipment cost, simple operation and good practicability.
Drawings
Fig. 1 is a schematic structural diagram of an embodiment of the present invention.
FIG. 2 is a schematic diagram of the decision mechanism of the present invention.
Detailed Description
The invention is further illustrated by the following examples in conjunction with the accompanying drawings:
as shown in fig. 1 and 2, an electro-oculogram-based wheelchair control system includes an electro-oculogram signal acquisition module 1, a blink detection unit 2, a decision unit 3 and a wheelchair execution unit 4;
the eye electric signal acquisition module 1 is used for acquiring signals, amplifying and filtering the signals, and then transmitting the acquired eye electric signals to the blink detection unit 2;
the blink detection unit 2 extracts waveform characteristic parameters from the eye electrical signals, wherein the waveform characteristic parameters comprise an original signal maximum value, time delay of the appearance moment of the original signal maximum value relative to the key blinking moment, a difference signal maximum value and blinking duration, and whether the waveform characteristic parameters meet a threshold condition is checked, so that whether blinking exists is judged;
the decision unit 3 outputs an instruction to the wheelchair executing unit 4 according to the result of the blink detecting unit 2;
the wheelchair executing unit 4 controls the wheelchair to execute corresponding actions according to the identified instruction result;
the decision unit 3 outputs an instruction to the wheelchair executing unit 4 according to the result of the blink detecting unit 2 in the following way;
the blink reaction time of each user is measured through initial calibration, namely the time delay t of the maximum occurrence time of the original signal relative to the blink time of the keypAverage value of (A) Tp(ii) a Obtaining the time delay t of each key in the wheel flashingpEach key is evaluated according to the following formula:
ei＝|tpi-Tp|；
wherein e isiIs the evaluation value of the key i, tpiIs the time delay T of the maximum value of the original signal in the electro-oculogram data segment of the key i relative to the key flashing timepThe blink reaction time of the current user; and then, selecting the key with the minimum evaluation value as the identification result of the flicker in the current round. The effectiveness of this evaluation method is based on experimental observations of the reaction time T of each user to flickerpIs relatively stable and will be stable within a certain range of time window (e.g. 280 ms and 320ms, the time window width is about 40 ms). The flicker interval between adjacent keys is designed to be 90ms which is far greater than TpThe width of the stability range. So that only the time delay t of the target key ispPossibly at the reaction time TpThereby being separated from the non-target keypad.
The electro-oculogram signal acquisition module 1 is realized by adopting a four-channel electro-oculogram amplifier, and mainly comprises a preamplifier, a right leg driving circuit (DRL), a band-pass filter (1-30Hz) and a post-amplifier. The overall gain of the amplifier is 2000 and the sampling frequency is 250 Hz. The four channels are respectively 'CH 1', 'CH 2', 'COM' and 'COMLEG', each channel collects signals through corresponding electrodes attached to the surface of the skin of a human body, wherein the channels 'CH 1' and 'CH 2' are data collection channels, the channel 'COM' is used as a reference, and the channel 'COMLEG' is used as an input channel of a right leg driving circuit and used for removing common-mode noise in eye electrical signals. This embodiment only requires one data channel and two reference channels, wherein the data channel ("CH 1") is attached to the brow head, and the "COM" and "COMLEG" electrodes are attached to the left and right ear roots, respectively.
The present embodiment includes a graphical user interface, which includes two levels of interfaces, namely a switch interface (1 control button) and a control interface (13 control buttons). Each control key corresponds to a specific function and can twinkle according to a certain frequency for prompting the twinkling opportunity of the user. The user selects a key by blinking in synchronization with the blinking of the following key. In the switch interface, a single key "On" flickers at a frequency of 1 Hz; in the control interface, 13 control keys flash in sequence according to a fixed sequence, and the flash interval between the front key and the rear key is 90 ms. And starting a new flash 90ms after the end of one flash (once per key flash).
The blink detection unit 2 determines whether to blink in the following manner:
firstly, after each key flickers, extracting a segment of electro-oculogram data, wherein the segment of electro-oculogram data comprises electro-oculogram data of 100 and 500ms after the flicking moment begins; down-sampling the data segment, filtering by a low-pass filter with the cut-off frequency of 10Hz, and differentiating to obtain a differential signal; then extracting a plurality of waveform characteristic parameters including the maximum value a of the original signalmaxThe time delay t of the maximum value of the original signal relative to the key flashing timepMaximum value s of differential signalmaxAnd duration of blinking dpn；
Blink duration d in the present inventionpnIs the time interval between the maximum value and the minimum value of the differential signal; the blink detection unit detects whether the data segment contains blink waveforms according to threshold parameters set by initial calibration before each experiment; specifically, the system sets three thresholds, respectively amplitude threshold AthSpeed threshold SthAnd duration threshold [ D ]1,D2](ii) a A extracted from the electrooculogram datamaxAnd smaxRespectively exceeds an amplitude threshold and a velocity threshold, and dpnWithin a range defined by the duration threshold, it is assumed that a blink is detected in the segment of electro-ocular data. I.e. a successful blink detection needs to satisfy the following inequality simultaneously:
amax≥Ath(1)
smax≥Sth(2)
D1≤dpn≤D2(3)
an initial calibration is performed to acquire threshold parameters prior to each use. Flickers 10 times at a frequency of 1 Hz. The operator blinks synchronously with the blinking, the calibration program collects the original signal of 10 blinks, and extracts a plurality of waveform characteristic parameters including the maximum value a of the original signalmaxThe time delay t of the maximum value of the original signal relative to the key flashing timepMaximum value s of differential signalmaxAnd duration of blinking dpn. Then averaging the characteristic parameters of the 10 blinksAndwherein,respectively as the amplitude of the blink detection unitThreshold value AthAnd a speed threshold Sth，For calculating a duration threshold D1,D2]，As the blink response time of the user.
A problem is common to existing electro-ocular control systems: it is difficult to distinguish between involuntary eye movements and conscious eye movements. This problem directly increases the false alarm rate of the system (the probability that the system erroneously outputs a control command when the user is in an idle state), resulting in instability of the system. Therefore, if the final output is determined by only relying on the results from one cycle of blinking, an unintended blink by the user may result in an erroneous movement of the wheelchair, which may cause a dangerous situation in actual operation. To solve this problem, the present invention proposes a "synchronous decision mechanism": only when a certain key is selected at least twice in the latest three continuous flashes as a recognition result, the system judges that the user blinks synchronously along with the flashes of the key, and then outputs a corresponding wheelchair control instruction; if no key meeting the condition exists, the system enters a new flash, and no control instruction is output. The 'synchronous decision mechanism' emphasizes the synchronism of multiple blinks of the user and multiple blinks of the keys, and random unconscious blinks are almost impossible to be synchronous with the blinks of a certain key, so that the influence of the unconscious blinks is eliminated. In addition, in order to speed up the generation of the control instruction, another "fast decision mechanism" may also be adopted: if a certain key is selected as a recognition result in the current round of flickering, the key is directly regarded as a final result and a corresponding wheelchair control instruction is output. The advantage of the "fast decision mechanism" is that the time to generate the instructions is shortened, the disadvantage is that the accuracy is not high and it is easily affected by unintentional blinking. Because the requirement on the stopping reaction speed in the actual wheelchair control is high, the 'instant stopping' is required to be realized, and the 'error stopping' cannot cause potential danger, the invention adopts a 'quick decision mechanism' for the stopping key and a 'synchronous decision mechanism' for other keys. According to the experimental result, the average time for generating a non-stop instruction is 3.56s, the time for generating a stop instruction is 1.92s, and the overall accuracy rate reaches 97.1%.
Compared with the prior art, the method has the advantages that a graphical user interface is introduced to prompt a user to blink at a specific moment, the multi-threshold blink detection method and the time delay-based evaluation method are provided, blink signals can be quickly detected and accurately positioned on a specific key, and therefore wheelchair control instructions can be safely, quickly, accurately and fully provided only through one eye movement. According to the experimental result, the average time for generating a non-stop instruction is 3.56s, the time for generating a stop instruction is 1.92s, the overall accuracy rate reaches 97.1%, the false alarm rate is 0, and up to 13 control instructions can be provided. The existing non-manual wheelchair control technology such as a motor imagery mode in brain electricity can only provide 2 or 3 control instructions generally, the accuracy rate is different from person to person, a P300 mode generally needs 4-6 seconds for generating a control command, and an SSVEP mode easily causes fatigue and epilepsy; the highest accuracy rate of the existing wheelchair control technology based on the electro-oculogram is only about 80%. The method disclosed by the invention can well solve the problems.
The invention has another advantage that a 'synchronous decision mechanism' is adopted, and the control instruction is output only when the detected blink is synchronous with the blink of a certain key, so that the accuracy is greatly improved, the influence caused by single unconscious blink is eliminated, and the false alarm rate is 0. Other existing electro-ocular control systems typically do not distinguish between conscious and unconscious eye movements well, and therefore have a certain false alarm rate. The method disclosed by the invention can well solve the problem.
The invention has the other advantages of low equipment cost, simple operation and good practicability. The hardware signal acquisition equipment of the system is a self-made electro-oculogram amplifier, the cost is low, and only three electrodes are needed to acquire signals. The user can select the corresponding instruction only by synchronously blinking along with the key blinking, so that the method is simple and clear and does not need to be trained in advance. And due to the introduction of a calibration mechanism, the influence caused by individual difference of users is overcome.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (4)
1. A wheelchair control system based on an electro-oculogram is characterized in that: the device comprises an eye electric signal acquisition module (1), a blink detection unit (2), a decision unit (3) and a wheelchair execution unit (4);
the eye electric signal acquisition module (1) is used for acquiring signals, amplifying and filtering the signals and then transmitting the acquired eye electric signals to the blink detection unit (2);
the blink detection unit (2) extracts waveform characteristic parameters from the eye electrical signals, wherein the waveform characteristic parameters comprise the maximum value of an original signal, the time delay of the occurrence moment of the maximum value of the original signal relative to the blink moment of a key, the maximum value of a differential signal and the blink duration, and whether the waveform characteristic parameters meet threshold conditions or not is checked, so that whether blinking exists or not is judged;
the decision unit (3) outputs an instruction to the wheelchair execution unit (4) according to the result of the blink detection unit (2);
the wheelchair executing unit (4) controls the wheelchair to execute corresponding actions according to the identified instruction result;
the decision unit (3) outputs an instruction to the wheelchair execution unit (4) according to the result of the blink detection unit (2) in the following way:
the blink reaction time of each user is measured through initial calibration, namely the time delay t of the maximum occurrence time of the original signal relative to the blink time of the keypAverage value of (A) Tp(ii) a Obtaining the time delay t of each key in the wheel flashingpEach key is evaluated according to the following formula:
ei＝|tpi-Tp|；
wherein e isiIs the evaluation value of the key i, tpiIs the time delay T of the maximum value of the original signal in the electro-oculogram data segment of the key i relative to the key flashing timepThe blink reaction time of the current user; and then, selecting the key with the minimum evaluation value as the identification result of the flicker in the current round.
2. The electro-ocularly based wheelchair control system of claim 1, wherein: the blink detection unit (2) determines whether to blink in the following manner:
firstly, after each key flickers, extracting a segment of electro-oculogram data, wherein the segment of electro-oculogram data comprises electro-oculogram data of 100 and 500ms after the flicking moment begins; down-sampling the data segment, filtering by a low-pass filter with the cut-off frequency of 10Hz, and differentiating to obtain a differential signal; then extracting a plurality of waveform characteristic parameters including the maximum value a of the original signalmaxThe time delay t of the maximum value of the original signal relative to the key flashing timepMaximum value s of differential signalmaxAnd duration of blinking dpn；
The blink duration dpnIs the time interval between the maximum value and the minimum value of the differential signal; the blink detection unit detects whether the data segment contains blink waveforms according to threshold parameters set by initial calibration before each experiment; specifically, the system sets three thresholds, respectively amplitude threshold AthSpeed threshold SthAnd duration threshold [ D ]1,D2](ii) a A extracted from the electrooculogram datamaxAnd smaxRespectively exceeds an amplitude threshold and a velocity threshold, and dpnWithin a range defined by the duration threshold, it is assumed that a blink is detected in the segment of electro-ocular data.
3. The electro-ocularly based wheelchair control system of claim 1 or 2, characterized in that: only when a certain key is selected at least twice in the latest three continuous flashes as a recognition result, judging that the user is blinking synchronously along with the flashes of the key, and further outputting a corresponding wheelchair control instruction; if no key meeting the condition exists, the system enters a new flash, and no control instruction is output.
4. The electro-ocularly based wheelchair control system of claim 1 or 2, characterized in that: if a certain key is selected as a recognition result in the current round of flickering, the key is directly regarded as a final result and a corresponding wheelchair control instruction is output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710235251.1A CN107007407B (en) | 2017-04-12 | 2017-04-12 | Wheelchair control system based on eye electricity |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710235251.1A CN107007407B (en) | 2017-04-12 | 2017-04-12 | Wheelchair control system based on eye electricity |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107007407A CN107007407A (en) | 2017-08-04 |
| CN107007407B true CN107007407B (en) | 2018-09-18 |
Family
ID=59445748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710235251.1A Active CN107007407B (en) | 2017-04-12 | 2017-04-12 | Wheelchair control system based on eye electricity |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107007407B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108245154B (en) * | 2018-01-24 | 2020-10-09 | 福州大学 | Method for accurately determining blink interval in electroencephalogram or electrooculogram by using abnormal value detection |
| CN108491792B (en) * | 2018-03-21 | 2022-07-12 | 安徽大学 | Office scene human-computer interaction behavior recognition method based on electro-oculogram signals |
| CN108646726A (en) * | 2018-04-03 | 2018-10-12 | 山东农业大学 | The wheelchair control system of wheelchair control method and combination voice based on brain wave |
| CN109933270B (en) * | 2019-04-15 | 2024-06-14 | 华南理工大学 | Man-machine interaction system based on finger switch and interaction method thereof |
| CN110275621A (en) * | 2019-06-26 | 2019-09-24 | 陕西科技大学 | Barrier movable platform is helped based on electro-ocular signal control |
| CN110840666B (en) * | 2019-11-19 | 2022-09-09 | 华南理工大学 | Wheelchair mechanical arm integrated system based on electro-oculogram and machine vision and control method thereof |
| CN113520740A (en) * | 2020-04-13 | 2021-10-22 | 广东博方众济医疗科技有限公司 | Wheelchair bed control method and device, electronic equipment and storage medium |
| CN112057250B (en) * | 2020-09-21 | 2022-12-06 | 山西白求恩医院(山西医学科学院) | Intelligent eye-watching wheelchair and control method thereof |
| CN112190405A (en) * | 2020-10-14 | 2021-01-08 | 合肥工业大学 | Wheelchair control system and control method thereof |
| CN112904893A (en) * | 2021-01-19 | 2021-06-04 | 华南理工大学 | Unmanned aerial vehicle control system based on portable eye electricity collection system |
| CN114488878A (en) * | 2021-12-29 | 2022-05-13 | 华南师范大学 | Intelligent device control system and control method based on eye electric control |
| CN115590695B (en) * | 2022-10-08 | 2024-06-14 | 华南脑控(广东)智能科技有限公司 | Wheelchair control system based on electrooculogram and face recognition |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1852058A1 (en) * | 2005-02-07 | 2007-11-07 | Tokyo Metropolitan Organization for Medical Research | Fetus movement monitoring system and fetus movement information collecting device |
| CN101344919A (en) * | 2008-08-05 | 2009-01-14 | 华南理工大学 | Sight tracing method and disabled assisting system using the same |
| CN102125429A (en) * | 2011-03-18 | 2011-07-20 | 上海交通大学 | Alertness detection system based on electro-oculogram signal |
| TW201345484A (en) * | 2012-05-02 | 2013-11-16 | Tzu Chi University | Detecting apparatus for head physiological parameters and method for the same and application equipment for the same |
| CN103473294A (en) * | 2013-09-03 | 2013-12-25 | 重庆邮电大学 | MSVM (multi-class support vector machine) electroencephalogram feature classification based method and intelligent wheelchair system |
| CN104363948A (en) * | 2012-06-15 | 2015-02-18 | 株式会社日立制作所 | Stimulus presentation system |
| CN104622649A (en) * | 2015-03-05 | 2015-05-20 | 中南大学 | Wheelchair system on basis of Bluetooth myoelectric collectors and method for controlling wheelchair system |
| CN105892691A (en) * | 2016-06-07 | 2016-08-24 | 京东方科技集团股份有限公司 | Method and device for controlling travel tool and travel tool system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010005033A (en) * | 2008-06-25 | 2010-01-14 | Panasonic Electric Works Co Ltd | Walking motion analyzer |
-
2017
- 2017-04-12 CN CN201710235251.1A patent/CN107007407B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1852058A1 (en) * | 2005-02-07 | 2007-11-07 | Tokyo Metropolitan Organization for Medical Research | Fetus movement monitoring system and fetus movement information collecting device |
| CN101344919A (en) * | 2008-08-05 | 2009-01-14 | 华南理工大学 | Sight tracing method and disabled assisting system using the same |
| CN102125429A (en) * | 2011-03-18 | 2011-07-20 | 上海交通大学 | Alertness detection system based on electro-oculogram signal |
| TW201345484A (en) * | 2012-05-02 | 2013-11-16 | Tzu Chi University | Detecting apparatus for head physiological parameters and method for the same and application equipment for the same |
| CN104363948A (en) * | 2012-06-15 | 2015-02-18 | 株式会社日立制作所 | Stimulus presentation system |
| CN103473294A (en) * | 2013-09-03 | 2013-12-25 | 重庆邮电大学 | MSVM (multi-class support vector machine) electroencephalogram feature classification based method and intelligent wheelchair system |
| CN104622649A (en) * | 2015-03-05 | 2015-05-20 | 中南大学 | Wheelchair system on basis of Bluetooth myoelectric collectors and method for controlling wheelchair system |
| CN105892691A (en) * | 2016-06-07 | 2016-08-24 | 京东方科技集团股份有限公司 | Method and device for controlling travel tool and travel tool system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107007407A (en) | 2017-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107007407B (en) | Wheelchair control system based on eye electricity | |
| Huang et al. | An EOG-based human–machine interface for wheelchair control | |
| Zhang et al. | An EOG-based human–machine interface to control a smart home environment for patients with severe spinal cord injuries | |
| Pan et al. | Discrimination between control and idle states in asynchronous SSVEP-based brain switches: A pseudo-key-based approach | |
| US8473045B2 (en) | Method for controlling device by using brain wave and brain wave interface system | |
| Hori et al. | Development of a communication support device controlled by eye movements and voluntary eye blink | |
| Yamagishi et al. | Development of EOG-based communication system controlled by eight-directional eye movements | |
| CN110840666B (en) | Wheelchair mechanical arm integrated system based on electro-oculogram and machine vision and control method thereof | |
| KR20190061726A (en) | System and method for adjusting drivingcontrol of vehicle | |
| RU2571891C1 (en) | Device for operator's attention control in man-machine systems | |
| CN110824979B (en) | Unmanned equipment control system and method | |
| CN110584898B (en) | Brain-controlled wheelchair automatic obstacle avoidance method based on multiple sensors | |
| US20100160808A1 (en) | Interface system utilizing musticatory electromyogram | |
| CN110134240A (en) | Robot wheel chair control system based on brain electricity Yu head appearance hybrid interface | |
| CN113156861A (en) | Intelligent wheelchair control system | |
| KR101632830B1 (en) | Apparatus for Controlling Driving of Vehicle | |
| US7162295B2 (en) | Apparatus and method for recognizing positive/negative intention using fine change of gamma wave of brainwave | |
| Peng et al. | Control of a nursing bed based on a hybrid brain-computer interface | |
| Nakanishi et al. | Wheelchair control system by using electrooculogram signal processing | |
| Liu et al. | Detection of lower-limb movement intention from EEG signals | |
| CN107168313A (en) | Control the method and device of vehicle drive | |
| CN110051351B (en) | Tooth biting signal acquisition method and control method and device of electronic equipment | |
| Islam et al. | Computer vision based eye gaze controlled virtual keyboard for people with quadriplegia | |
| Blankertz et al. | The Berlin brain-computer interface | |
| CN110018744B (en) | Surface myoelectricity man-machine interface system based on visual stimulus interface and control method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190903 Address after: 510000 Room 703, 7 Building, Huagong Hongjinyuan, 381 Wushan Road, Tianhe District, Guangzhou City, Guangdong Province Co-patentee after: Guangzhou South China University of Technology Asset Management Co., Ltd. Patentee after: Li Yuanqing Address before: 510000 Tianhe District, Guangdong, No. five road, No. 381, Patentee before: South China University of Technology |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190916 Address after: 510670 Room 201, Building 72, Nanxiang Second Road, Huangpu District, Guangzhou City, Guangdong Province Patentee after: South China Brain Control (Guangdong) Intelligent Technology Co., Ltd. Address before: 510000 Room 703, 7 Building, Huagong Hongjinyuan, 381 Wushan Road, Tianhe District, Guangzhou City, Guangdong Province Co-patentee before: Guangzhou South China University of Technology Asset Management Co., Ltd. Patentee before: Li Yuan Qing |
|
| TR01 | Transfer of patent right |