CN202161439U - Control system capable of controlling movement of upper artificial limbs through eye movement signals - Google Patents
Control system capable of controlling movement of upper artificial limbs through eye movement signals Download PDFInfo
- Publication number
- CN202161439U CN202161439U CN201120275264XU CN201120275264U CN202161439U CN 202161439 U CN202161439 U CN 202161439U CN 201120275264X U CN201120275264X U CN 201120275264XU CN 201120275264 U CN201120275264 U CN 201120275264U CN 202161439 U CN202161439 U CN 202161439U
- Authority
- CN
- China
- Prior art keywords
- module
- eye movement
- characteristic
- movement characteristics
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The utility model discloses a control system capable of controlling the movement of upper artificial limbs through eye movement signals. The control system includes an eye image acquisition module, an eye movement feature extraction module, a predefined feature storage module, a feature matching module, a command confirmation module and a driving module. Eye turning and blinking events can be combined through the above modules, so as to produce various features and further achieve more control functions.
Description
Technical field
This utility model relates to the upper extremity prosthesis technology of rehabilitation accessory technical field, especially relates to the control system of upper extremity prosthesis.
Background technology
How to let convenient for handicapped, neatly control to go up artificial limb be to go up most important content in the artificial limb research.The bionical control signal that is used at present artificial limb control in the world mainly contains electromyographic signal, EEG signals, nerve signal and the voice signal etc. of human body self.
Adopt the ultimate principle of electromyographic signal control artificial limb to be: the action potential that produces during with the deformed limb muscle contraction is drawn by skin electrode; Through sending into computer analysis after the bioelectric amplifier amplification; Extract the validity feature of reflection motion wish, the mapping relations of utilization characteristic vector and space are come the motion of driving device arm and the folding of doing evil through another person.The main bionical control signal source of going up artificial limb at present is exactly an electromyographic signal.
Although artificial limb has obtained success on the myoelectricity in practical application, too short when the deformed limb of patients with amputation, when perhaps causing amyotrophy, enough needed control informations of last artificial limb just can not be provided because of paralysis.Simultaneously, the training of the fatigue of muscle, the change of electrode position, electromyographic signal, the fluctuation of body weight all can make the eigenvalue of electromyographic signal change, and cause the control accuracy of multi-freedom degree muscle-electric artificial limb to be difficult to improve.In addition, because the decoding capability of electromyographic signal is limit, the degree of freedom that artificial limb can be controlled on the myoelectricity also is very limited.
EEG signals are neural and electrical noise that synapse is produced in central nervous system's work process in essence.Research shows, has certain dependency between the variation of EEG signals and the motion of limbs, and this dependency is explained, sets up the corresponding relation between brain electrical acti and the autonomic movement, and cortex just can carry out information exchange with extraneous so.
Utilization from the brain electrical acti of scalp record as information source, even the most serious patients with amputation also can use.Yet the brain electric process is very complicated, and the research to it at present also is only limited to the simple brain-computer interface device of exploitation, and it is can recognized patterns fewer, and discrimination is not high, and also more complicated of present brain wave acquisition device.All need the coating conductive paste at the scalp place before each this device of use, this just makes troubles to use.
Neural bioelectrical activity does not only receive the influence of fatigue level of human body, and reproducibility is high, and nerve information do not disturb when transmitting each other mutually, has splendid definition.These characteristics make neural activity serve as the very big superiority of tool aspect the control information source.Based on this, Wan etc. have proposed to be converted into the thought that control instruction is controlled artificial limb to the human upper limb nerve signal.Yet gathering accurate nerve signal need be with in silicon chip and the electrode implant into body, on technology realizes, also has very big difficulty at present, and the research of the neural decoding problem of movable information is also carried out at present.
Controlling artificial limb with other modes compares; The control function that acoustic control can be accomplished is more; More convenient, precision is also higher, and its weak point is in specific environment, to use sound to make signal source and improper; Such as should not controlling artificial limb with sound in needs such as the meeting-place occasion that Keep silence, and patient and others also possibly cause artificial limb to produce misoperation when carrying out communication.
Summary of the invention
For overcoming the deficiency of existing artificial limb control mode technology, this utility model provides a kind of control system with artificial limb motion on the eye movement signal controlling.
A kind of control system with artificial limb motion on the eye movement signal controlling, it is made up of eye image acquisition module, eye movement characteristics extraction module, predefine characteristic storage module, characteristic matching module, confirmation command module and driver module; Wherein:
1, described eye image acquisition module is made up of photographic head and transporter, and photographic head is used to gather the image of eyes, and transporter is used for picture coding and sends to the eye movement characteristics extraction module.
2, described eye movement characteristics extraction module is used for extracting eye movement characteristics and sending it to characteristic matching module from eye image.It comprises with lower unit: blink detection unit, eyeball rotation direction recognition unit and characteristic vector memory element; Wherein:
1) the blink detection unit is used to detect incident and persistent period of writing down eyes closed nictation;
2) the eyeball rotation direction recognition unit rotation direction that is used to calculate the displacement of pupil and judges eyeball;
3) the characteristic vector memory element is used to store and the corresponding characteristic vector of the action of eyes.
3, described predefine characteristic storage module is used to store the action command of artificial limb and corresponding with it eye movement characteristics.
4, the described characteristic matching module eye movement characteristics that is used for the eye movement characteristics extraction module is extracted and the characteristic of predefine characteristic storage module are mated, the action command of artificial limb in the acquisition, and action command sent to the confirmation command module.
5, described confirmation command module is used for the action command that characteristic matching module obtains is met at experimenter's affirmation; If instruct errorless; Then send it to driver module,, then also extract eye movement characteristics again through this action command of " cancellation " order deletion if wrong.
6, described driver module comprises driver and motor, when driver receives the action command that the confirmation command module sends, just controls artificial limb and accomplishes the corresponding action of setting.
Artificial limb control method based on said system comprises the steps: to gather eye image; Extract eye movement characteristics; The identification maneuver instruction; Confirm that action command moves with accomplishing to set.Above-mentioned artificial limb control system and method adopt the signal source of eye movement signal as the control artificial limb, differentiate action command through the characteristic information that extracts eye motion, and the control artificial limb is made the action of experimenter's expectation.
The control method of this utility model control system is disguised strong, and control is convenient, and the rotation of eyes and the incident of blinking are made up, and can produce multiple different character, thereby accomplishes the more control function.
Description of drawings
Fig. 1 is the module map of artificial limb control system;
Fig. 2 is the structure chart of eye movement characteristics extraction module.
Fig. 3 is an eye image.
Fig. 4 is the flow chart of artificial limb control method.
The specific embodiment
Enforcement to this utility model further specifies according to accompanying drawing below.
This utility model adopts following technical scheme, and is as shown in Figure 1: it comprises eye image acquisition module 100, eye movement characteristics extraction module 200, predefine characteristic storage module 300, characteristic matching module 400, confirmation command module 500 and driver module 600.
Described eye image acquisition module 100 is made up of photographic head and transporter, and its function is to gather the image of eyes in real time and with sending to eye movement characteristics extraction module 200 after the picture coding.
The function of described eye movement characteristics extraction module 200 is to extract eye movement characteristics.This module comprises three unit: blink detection unit 201, eyeball rotation direction recognition unit 202 and characteristic vector memory element 203.Eye image is analyzed in blink detection unit 201, and detection incident nictation and the persistent period of writing down eyes closed, the preferably settled approximately closing time that detects eyes promptly gets into eyeball rotation direction recognition unit 202 when 0.5s is between 2s.In eyeball rotation direction recognition unit 202, the displacement through the image analysis calculation pupil is also judged the rotation direction of eyeball.Preferably agreement when detecting eyeball and turn right, writes characteristic vector memory element 203 with numeral " 1 "; When upwards rotating, write numeral " 2 "; When turning left, write numeral " 3 "; When rotating, write numeral " 4 ".Eyeball whenever the experimenter forwards a specific direction to like this; Eyeball rotation direction recognition unit 202 just joins a corresponding digital in the characteristic vector; The experimenter rotates eyes successively to different directions, has just stored the numeral corresponding with these directions in the characteristic vector in order.Preferably fix on approximately in this process when blink detection unit 201 detects continuous 2 eyes closed incidents, promptly stop identification, and will construct the characteristic vector of accomplishing and send to characteristic matching module 400 the eyeball rotation direction.Preferably arrange to be limited to 4 on the length of characteristic vector, and adjacent numeral can not repeat in the vector.After eyeball rotation direction recognition unit 202 the 4th write characteristic vector with numeral, take place like this, also will stop identification and characteristic vector is sent to characteristic matching module 400 even without the incident of closing one's eyes.Can the 4+12+36+108=160 kind can be arranged the recognized action instruction according to this about fixed system.(see figure 2)
Storing the action command and the corresponding eye movement characteristics vector of artificial limb on all in the described predefine characteristic storage module 300.According to agreement before, can deposit 160 action commands and 160 corresponding with it characteristic vectors in the predefine characteristic storage module 300.
Described characteristic matching module 400 is when receiving the characteristic vector that eye movement characteristics extraction module 200 sends; Adopt look-up table that the characteristic vector of storage in characteristic vector and the predefine characteristic storage module 300 is mated; Thereby obtain corresponding action command, then action command is sent in the confirmation command module 500.
The function of described confirmation command module 500 is the action command that characteristic matching module 400 obtains to be met at the experimenter confirm; If instruct errorless; Then action command is sent to driver module 600; If instruct wrongly, the experimenter sends " cancellation " order this action command of deletion and extracts eye movement characteristics again through eyes, preferably adopts experimenter's 3 eyes that in the time of 2s, blink continuously to constitute " cancellation " and order.
Described driver module 600 comprises driver and motor, when driver receives the action command that confirming operation module 500 sends, just controls artificial limb and accomplishes the corresponding action of setting.
Shown in Figure 4 is upper extremity prosthesis control method flow chart, comprises following five steps: gather eye image S1; Extract eye movement characteristics S2; Identification maneuver instruction S3; Confirm that action command S4 sets action S5 with accomplishing.
At first, step 1 is gathered eye image S1.Preferably adopt infrared camera picked-up experimenter eye image, and image is carried out sending into computer behind the mpeg encoded.This step need be carried out always.
Step 2, characteristic information extraction S2.In this step, the eye image of gathering is analyzed, when detect nictation incident and the closing time of eyes when 0.5s is between 2s, just begin to discern the rotation direction of eyeball.The image of shown in Figure 3 is when the experimenter faces the place ahead eyes is demarcated the central point of pupil this moment, and promptly the intersection point of cross wire is an initial point, is that coordinate axes is set up image coordinate system with the cross wire.In this coordinate system, preferably agreement is to the right an x axle forward, upwards is y axle forward, and the angle of regulation x axle is 0 degree.Utilize 45 degree lines, 135 degree lines, 225 degree lines and 315 degree lines that coordinate space is divided into four zones then.In identifying, calculate the displacement of pupil center's point in real time, and make a decision: the distance of preferably settled approximately pupil center's point and zero can think that eyeball does not rotate during less than 2 millimeters; When this distance during, confirm that according to the angle θ of displacement pupil center puts residing zone, preferably settled approximately 0＜=θ＜45 or 315＜=θ＜360 o'clock greater than 2 millimeters; Think that promptly eyeball has forwarded the right to, when 45＜=θ＜135, thinks that promptly eyeball has forwarded the top to; When 135＜=θ＜225; Think that promptly eyeball has forwarded the left side to, when 225＜=θ＜315, think that promptly eyeball has forwarded bottom to.Whenever detect the rotation of an eyeball, just the numeral corresponding with rotation direction write characteristic vector.When the number of characteristic vector element reaches 4 or when having continuous 2 eyes closed incidents to take place, stop identification, and characteristic vector is sent into characteristic matching module 400. the eyeball rotation direction
Step 3, identification maneuver instruction S3.With the characteristic vector of step 2 acquisition and mating of storing in advance with action command characteristic of correspondence vector, the controlled action command of going up artificial limb.
Step 4 is confirmed action command S4.The action command that step 3 is obtained send the affirmation in the experimenter, detects incident nictation simultaneously.Preferably agreement as in 2 seconds, detect the number of times that eyes blink and be less than 3 times thinks that then instruction is errorless and carry out action, blinks as in 2 seconds, detecting continuous 3 these eyes, thinks that then instruction is wrong, deletes this instruction and extracts eye movement characteristics again.
Step 5 is accomplished and is set this step of action S5. by driver module 600 completion.
Adopt eye movement signal controlling artificial limb, the experimenter need remember and the corresponding eye movement order of the action command of last artificial limb in advance.In the control procedure; The experimenter blink in a certain order eyes with rotate eyeball, camera acquisition experimenter's eye image, the computer of making a gift to someone; Computer extracts the eye movement characteristics vector through graphical analysis; This characteristic vector and predefine characteristic vector are complementary, identify the action command of experimenter's expectation and meet at the experimenter and confirm that the control artificial limb is accomplished corresponding action after affirmation is errorless.
The above embodiment has only expressed the specific embodiment of this utility model, and it describes comparatively in detail concrete, but can not be interpreted as the restriction to this utility model claim.Should be pointed out that for the person of ordinary skill of the art under the prerequisite that does not break away from this utility model design, can also make some distortion and improvement, these all belong to the protection domain of this utility model.
Claims (1)
1. control system with artificial limb motion on the eye movement signal controlling; It is characterized in that, it by eye image acquisition module, eye movement characteristics extraction module, be used for storing the artificial limb action command and with it the characteristic of predefine characteristic storage module, the eye movement characteristics that is used for the eye movement characteristics extraction module is extracted and the predefine characteristic storage module of the corresponding eye movement characteristics characteristic matching module of mating, be used for that the action command that characteristic matching module obtains is met at confirmation command module and the driver module that the experimenter confirms and form; Wherein:
Described eye image acquisition module is made up of picture coding and the transporter that sends to the eye movement characteristics extraction module with being used for the photographic head that is used to gather eye image;
Described eye movement characteristics extraction module comprises with lower unit: be used to detect nictation incident and write down the eyes closed persistent period the blink detection unit, be used to the eyeball rotation direction recognition unit that calculates the pupil displacement and judge the eyeball rotation direction and be used to store move the characteristic vector memory element of corresponding characteristic vector with eyes;
Described driver module comprises driver and motor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201120275264XU CN202161439U (en) | 2011-07-21 | 2011-07-21 | Control system capable of controlling movement of upper artificial limbs through eye movement signals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201120275264XU CN202161439U (en) | 2011-07-21 | 2011-07-21 | Control system capable of controlling movement of upper artificial limbs through eye movement signals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN202161439U true CN202161439U (en) | 2012-03-14 |
Family
ID=45797295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201120275264XU Expired - Fee Related CN202161439U (en) | 2011-07-21 | 2011-07-21 | Control system capable of controlling movement of upper artificial limbs through eye movement signals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN202161439U (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102309366A (en) * | 2011-07-21 | 2012-01-11 | 山东科技大学 | Control system and control method for controlling upper prosthesis to move by using eye movement signals |
| CN104997582A (en) * | 2015-07-30 | 2015-10-28 | 沈阳工业大学 | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals |
| CN105739444A (en) * | 2016-04-06 | 2016-07-06 | 济南大学 | Manipulator multiparameter controlling brain-computer interface |
| CN107260420A (en) * | 2017-07-03 | 2017-10-20 | 南京邮电大学 | Intelligent wheel chair human-computer interactive control system and method based on eye motion recognition |
| CN109718544A (en) * | 2018-12-14 | 2019-05-07 | 深圳壹账通智能科技有限公司 | Game control method based on human face recognition and the electronic device using this method |
-
2011
- 2011-07-21 CN CN201120275264XU patent/CN202161439U/en not_active Expired - Fee Related
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102309366A (en) * | 2011-07-21 | 2012-01-11 | 山东科技大学 | Control system and control method for controlling upper prosthesis to move by using eye movement signals |
| CN102309366B (en) * | 2011-07-21 | 2014-09-24 | 山东科技大学 | Control system and control method for controlling upper prosthesis to move by using eye movement signals |
| CN104997582A (en) * | 2015-07-30 | 2015-10-28 | 沈阳工业大学 | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals |
| CN104997582B (en) * | 2015-07-30 | 2017-03-22 | 沈阳工业大学 | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals |
| CN105739444A (en) * | 2016-04-06 | 2016-07-06 | 济南大学 | Manipulator multiparameter controlling brain-computer interface |
| CN107260420A (en) * | 2017-07-03 | 2017-10-20 | 南京邮电大学 | Intelligent wheel chair human-computer interactive control system and method based on eye motion recognition |
| CN109718544A (en) * | 2018-12-14 | 2019-05-07 | 深圳壹账通智能科技有限公司 | Game control method based on human face recognition and the electronic device using this method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102309366B (en) | Control system and control method for controlling upper prosthesis to move by using eye movement signals | |
| WO2018113392A1 (en) | Brain-computer interface-based robotic arm self-assisting system and method | |
| CN202161439U (en) | Control system capable of controlling movement of upper artificial limbs through eye movement signals | |
| Schröer et al. | An autonomous robotic assistant for drinking | |
| CN101947152B (en) | Electroencephalogram-voice control system and working method of humanoid artificial limb | |
| CN110840666B (en) | Wheelchair mechanical arm integrated system based on electro-oculogram and machine vision and control method thereof | |
| CN202288542U (en) | Artificial limb control device | |
| CN101987048A (en) | Artificial limb control method and system thereof | |
| CN105012057B (en) | Intelligent artificial limb based on double-arm electromyogram and attitude information acquisition and motion classifying method | |
| CN113143676B (en) | Control method of external limb finger based on brain-muscle-electricity cooperation | |
| CN106491251B (en) | Non-invasive brain-computer interface-based robot arm control system and control method thereof | |
| CN102499797A (en) | Artificial limb control method and system | |
| CN109521880B (en) | Teleoperation robot system and method based on mixed bioelectricity signal driving | |
| Wang et al. | Human-centered, ergonomic wearable device with computer vision augmented intelligence for VR multimodal human-smart home object interaction | |
| CN105739444A (en) | Manipulator multiparameter controlling brain-computer interface | |
| Gauthaam et al. | EMG controlled bionic arm | |
| CN115890655B (en) | Mechanical arm control method, device and medium based on head gesture and electrooculogram | |
| CN105137830A (en) | Traditional Chinese painting mechanical hand based on visual evoking brain-machine interface, and drawing method thereof | |
| CN112732090A (en) | Muscle cooperation-based user-independent real-time gesture recognition method | |
| CN104997582B (en) | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals | |
| Ianez et al. | Multimodal human-machine interface based on a brain-computer interface and an electrooculography interface | |
| Wei et al. | Evaluating the performance of a face movement based wheelchair control interface in an indoor environment | |
| Lenhardt et al. | An augmented-reality based brain-computer interface for robot control | |
| CN112405539B (en) | Robot natural control method based on electromyographic signals and electroencephalogram error potentials | |
| CN110751060B (en) | Portable motion mode real-time identification system based on multi-source signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120314 Termination date: 20130721 |