CN102156539A - Method for identifying target object based on eye movement scanning - Google Patents
Method for identifying target object based on eye movement scanning Download PDFInfo
- Publication number
- CN102156539A CN102156539A CN 201110075485 CN201110075485A CN102156539A CN 102156539 A CN102156539 A CN 102156539A CN 201110075485 CN201110075485 CN 201110075485 CN 201110075485 A CN201110075485 A CN 201110075485A CN 102156539 A CN102156539 A CN 102156539A
- Authority
- CN
- China
- Prior art keywords
- target object
- electro
- eye
- ocular signal
- nictation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000004424 eye movement Effects 0.000 title claims abstract description 12
- 210000001508 eye Anatomy 0.000 claims abstract description 54
- 210000005252 bulbus oculi Anatomy 0.000 claims description 13
- 210000000624 ear auricle Anatomy 0.000 claims description 4
- 210000003128 head Anatomy 0.000 claims description 3
- 238000011217 control strategy Methods 0.000 abstract description 3
- 230000003993 interaction Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 9
- 230000005611 electricity Effects 0.000 description 8
- 230000001276 controlling effect Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 229910021607 Silver chloride Inorganic materials 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 210000004087 cornea Anatomy 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000000284 resting effect Effects 0.000 description 1
- HKZLPVFGJNLROG-UHFFFAOYSA-M silver monochloride Chemical compound [Cl-].[Ag+] HKZLPVFGJNLROG-UHFFFAOYSA-M 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
Images
Landscapes
- Eye Examination Apparatus (AREA)
Abstract
The invention discloses a method for identifying a target object based on eye movement scanning. The method comprises the following steps of: reestablishing the shape of the target object by using an eye electric signal acquired when a user scans the boundary of the target object by two eyes; and then calculating a length and a width of the target object, and an angle between the target object and the horizontal direction according to the reestablished target object. The information of the identified target object can be used for controlling external equipment such as artificial hands, artificial limbs and the like, so that human and machine interaction can be realized. Compared with the conventional human-machine interaction method based on eye electric direct control, the method has the advantages that: an indirect control strategy is realized, and the precise and efficient control which cannot be realized in the prior art is achieved.
Description
Technical field
The present invention relates to a kind of target object recognition methods based on eye movement scanning.
Background technology
People's eyeball is that a cornea end is the bipolarity spheroid of negative pole for anodal, retina end, after placing electrode in the suitable position of around eyes, can collect this retina resting potential, is called eye electricity (Electro-Oculogram is called for short EOG).When about the eyeball, during left-right rotation, just can collect the variation of corresponding eye electric potential respectively in vertical, horizontal channel, and under the situations of rotational angle less than 30 degree, eyeball institute rotational angle is dull linear relationship with the electric amplitude of eye.Simultaneously, because electro-ocular signal is stronger than other physiological signals (as brain electricity, electrocardio) amplitude, signal is more directly perceived, and it is convenient to gather, and since the eighties in last century, the eye electricity is widely studied as a kind of novel man-machine interface Controlling Source.Establish the application of sending out by cable based on eye and comprise the electric mouse of eye, typewriting, wheelchair control, Smart Home etc.
Traditional man-machine interface system based on the eye electricity can be divided into two kinds according to the type of output signal: the one, and the switching value that disperses is mapped as discrete command sequence with electro-ocular signal, as upper and lower, left and right, nictation etc.; The 2nd, the successive control amount is based on the relation of eyeball rotational angle with the electric amplitude linearity of eye, with the continuous displacement of rotation mapping of eyeball, as the mouse of screen.Preceding a kind of system realizes simple, control stabilization, but the application that can develop is single; A kind of system in back can realize successive control, but system's instability, the electro-ocular signal drift causes the out of true of control.As can be seen, traditional eye electric system directly extracts control signal (disperse or continuous) from electro-ocular signal itself and controls external unit.This control mode needs eyes to participate in control procedure always, causes the fatigue of eyes easily; And each details in the control procedure all needs an electricity to instruct, and can not realize controlling efficiently.
Summary of the invention
The purpose of this invention is to provide a kind of target object recognition methods based on eye movement scanning,
The present invention realizes that the technological means that above-mentioned purpose is taked is: should comprise the steps: based on the target object recognition methods of eye movement scanning
(1) post five electrodes in user's circumference of eyes, described five electrodes lay respectively at horizontal range 2.5 centimeters apart from user's right eye tail of the eye, right eye eyeball top apart from right eye eyeball center vertically distance 2 centimeters, right eye eyeball below apart from right eye eyeball center vertically distance 2 centimeters, apart from horizontal range 2.5 centimeters of the left eye tail of the eye and the ear-lobe place of one of them ear;
(2) user's sight line is fixed on the target object, under the state that keeps head still, double nictation of the first time, then scan the border of target object with eyes, double nictation for the second time again behind the end of scan is respectively in double nictation for the first time, gather the level that produced when user's eyeball rotates and the electro-ocular signal of vertical channel respectively by each described electrode in nictation and the described scanning continuously twice for the second time;
(3) identify double nictation for the first time and the electro-ocular signal that produced double nictation for the second time the electro-ocular signal of the vertical channel of being gathered from step (2) respectively;
(4) electro-ocular signal that utilizes first double nictation and produced and second electro-ocular signal that produced double nictation calibrate the electro-ocular signal that described scanning produces, and reconstruct the shape of target object according to the electro-ocular signal that described scanning produces;
(5) calculate the angle of length, width and this target object and the horizontal direction of target object according to the shape of the target object that reconstructs.
Further, among the present invention, the user is fixed on the sight line of eyes in the lower right corner of target object when double nictation for the first time; The user is by the eyes sight line is come performing step (2) described " with the border of eyes scanning target object " with constant speed by counterclockwise move a circle along the border of target object; The user is the lower right corner that the sight line of eyes is fixed on target object when double nictation for the second time.
Further, the method of step of the present invention (4) described " reconstructing the shape of target object " is: the horizontal channel that described scanning is produced and the electro-ocular signal of vertical channel be respectively as abscissa value, ordinate value, the shape of drawing described target object according to described abscissa value and ordinate value in two dimensional surface.
With respect to prior art, the present invention has the following advantages: the sweep signal estimating target object that (1) utilizes twice double blink, can reduce the influence of an electric drift to greatest extent; (2) do not need eyes to participate in the whole control process always, can reduce the fatigue of eyes, prolong the control time; (3) the inventive method is different from traditional man-machine interactive system based on the eye electricity, from being different from the another one angle of classic method, can extract the angle of length, width and this target object and the horizontal direction of target object from electro-ocular signal; The control of external units such as the information of the target object that identifies can be further used for doing evil through another person, artificial limb, realize man-machine interaction, can avoid electro-ocular signal directly to control the limitation of external unit like this, realize a kind of indirect control strategy, thereby realize accurately identification control and mutual efficiently.
Description of drawings
Fig. 1 is the FB(flow block) of the inventive method;
Fig. 2 is eye electricity arrangement of electrodes synoptic diagram of the present invention;
Fig. 3 handles block diagram for electro-ocular signal of the present invention;
Fig. 4 is vertical channel electro-ocular signal differential signal figure of the present invention, and wherein (a) is the original electro-ocular signal of vertical channel, (b) is the differentiated result of vertical channel electro-ocular signal in (a);
Fig. 5 wherein, (a) is the target object synoptic diagram, the target object of rebuilding when (b) being scanning (a) and the oval synoptic diagram of match for the calculating identification synoptic diagram of target object size and Orientation of the present invention.
Embodiment
Below in conjunction with accompanying drawing and example the present invention is described in further detail.
As shown in Figure 1, it is as follows to the present invention is based on the target object recognition methods concrete steps of eye movement scanning:
The user at first is affixed on electrode the ad-hoc location of user's circumference of eyes, and begins to gather the level of eye movement generation and the electro-ocular signal of vertical channel.Target object generally places user about 70 centimetres place in front.The user adjusts sitting posture and begins to scan target object.At first user's sight line is fixed in the lower right corner of target object, under the state that keeps head still, double nictation, follows the border with eyes scanning target object.In the present embodiment, a kind of specific practice of " with the border of eyes scanning target object " is; The user moves a circle by counter clockwise direction along the border of target object with constant speed with the eyes sight line.End of scan retracement line is got back to the lower right corner of target object, and more double nictation of the second time, whole scanning process finishes.The electro-ocular signal of record is stored in the computing machine, is used for subsequent analysis and handles.
The present embodiment electrode used therein is the Ag/AgCl disposable electrode, has five.Shown in Fig. 2, five electrodes lay respectively at horizontal range 2.5 centimeters (being positioned at the right eye right side) apart from user's right eye tail of the eye, right eye eyeball top apart from right eye eyeball center vertically distance 2 centimeters, right eye eyeball below apart from right eye eyeball center vertically distance 2 centimeters, apart from horizontal range 2.5 centimeters (being positioned at the left eye left side) of the left eye tail of the eye and the ear-lobe place of one of them ear.Wherein, the signal that two upper and lower electrodes of right eye collect is done difference and can be obtained the vertical channel electro-ocular signal, the signal that left eye left side, right eye right electrodes collect is done difference and can be obtained the horizontal channel electro-ocular signal, and the electrode at ear-lobe place is as the reference earth signal.
The electro-ocular signal of the vertical and horizontal passage of gathering is handled block diagram as shown in Figure 3.At first electro-ocular signal is the 4 rank low pass butterworth filters of 20Hz by cutoff frequency, to remove electro-ocular signal other undesired signals in addition, as myoelectricity, power frequency interference etc.Then, the vertical channel electro-ocular signal is carried out difference, the passing threshold method identifies double nictation for the first time and the electro-ocular signal that produced double nictation for the second time.
Fig. 4 is vertical channel electro-ocular signal differential signal figure, and wherein Fig. 4 (a) is the original electro-ocular signal of vertical channel, comprises two double actions nictation in the position of beginning and end; Fig. 4 (b) is the differentiated signal graph of electro-ocular signal among Fig. 4 (a), after the electro-ocular signal difference, double signal of blinking becomes two continuous crests and trough, can remove the drift composition of electro-ocular signal simultaneously, can better detect double nictation with threshold method.The horizontal line of two levels among Fig. 4 (b) is predefined two threshold values up and down, and choosing of this threshold value varies with each individual, and general choosing ± 2 gets final product.When vertical channel eye electricity differential signal during by two threshold values up and down, promptly detects double nictation with certain time sequence successively.
In the present embodiment, for the first time and for the second time be to be used for demarcating the beginning of scanning and end double nictation.Utilize electro-ocular signal that produced double nictation for the first time and the electro-ocular signal that produced double nictation for the second time, can calibrate the beginning and the end of scanning, and extract vertical channel and the horizontal channel electro-ocular signal that produces when scanning.The electro-ocular signal that produces according to described scanning reconstructs the shape of target object, calculates the angle of length, width and this target object and the horizontal direction of target object according to the shape of the target object that reconstructs at last, identifies the information of target object thus.
Figure 5 shows that the calculating identification synoptic diagram of target object size and Orientation of the present invention, wherein, Fig. 5 (a) is the synoptic diagram of target object, and this target object is a rectangle at the projection approximation of two dimensional surface, and the angle of its major axis and horizontal direction is 30 degree; The oval synoptic diagram of match when the target object shape that Fig. 5 (b) rebuilds when being this target object of scanning and the angle of the length of calculating this target object, width and this target object and horizontal direction.Here, the method of the shape of reconstructed object object is: will scan the horizontal channel of generation and the electro-ocular signal of vertical channel, the i.e. signal of twice double blink, respectively as abscissa value, ordinate value, the shape of in two dimensional surface, drawing described target object then according to abscissa value and ordinate value.As can be seen, the target object that reconstructs is similar substantially to actual body form from Fig. 5 (b), the angle that can calculate length, width and this target object and the horizontal direction of target object from the shapometer of rebuilding.
In the present embodiment, the calculating of the angle of the length of target object, width and this target object and horizontal direction is that the shape by the target object that will reconstruct is oval with direct least square fitting, and with long axis of ellipse and minor axis respectively as the length and the width of target object, the angle of transverse and horizontal direction is as the angle of target object and horizontal direction.
The information of the target object that employing the inventive method obtains can be further used for controlling external unit.For example, one of them application as the inventive method, can realize mechanical prosthetic hand control based on eye movement scanning, the angle information input mechanical prosthetic hand control module of target object length, width and this target object and the horizontal direction that the inventive method is obtained, control module is according to the information of target object, the shape of the hand when calculating about this object of grasping with towards, and control mechanical prosthetic hand finish corresponding action, realize once successful grasping.This control strategy can be a kind of indirect control method in conjunction with human eye to the accurate control to mechanical prosthetic hand of the intelligent decision of body form and mechanical module, has avoided directly actuated limitation.
Claims (3)
1. the target object recognition methods based on eye movement scanning is characterized in that comprising the steps:
(1) post five electrodes in user's circumference of eyes, described five electrodes lay respectively at horizontal range 2.5 centimeters apart from user's right eye tail of the eye, right eye eyeball top apart from right eye eyeball center vertically distance 2 centimeters, right eye eyeball below apart from right eye eyeball center vertically distance 2 centimeters, apart from horizontal range 2.5 centimeters of the left eye tail of the eye and the ear-lobe place of one of them ear;
(2) user's sight line is fixed on the target object, under the state that keeps head still, double nictation of the first time, then scan the border of target object with eyes, double nictation for the second time again behind the end of scan is respectively in double nictation for the first time, gather the level that produced when user's eyeball rotates and the electro-ocular signal of vertical channel respectively by each described electrode in nictation and the described scanning continuously twice for the second time;
(3) identify double nictation for the first time and the electro-ocular signal that produced double nictation for the second time the electro-ocular signal of the vertical channel of being gathered from step (2) respectively;
(4) electro-ocular signal that utilizes first double nictation and produced and second electro-ocular signal that produced double nictation calibrate the electro-ocular signal that described scanning produces, and reconstruct the shape of target object according to the electro-ocular signal that described scanning produces;
(5) calculate the angle of length, width and this target object and the horizontal direction of target object according to the shape of the target object that reconstructs.
2. the target object recognition methods based on eye movement scanning according to claim 1 is characterized in that:
The user is fixed on the sight line of eyes in the lower right corner of target object when double nictation for the first time;
The user is by the eyes sight line is come performing step (2) described " with the border of eyes scanning target object " with constant speed by counterclockwise move a circle along the border of target object;
The user is the lower right corner that the sight line of eyes is fixed on target object when double nictation for the second time.
3. the target object recognition methods based on eye movement scanning according to claim 1 and 2, it is characterized in that, the method of step (4) described " reconstructing the shape of target object " is: the horizontal channel that described scanning is produced and the electro-ocular signal of vertical channel be respectively as abscissa value, ordinate value, the shape of drawing described target object according to described abscissa value and ordinate value in two dimensional surface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100754857A CN102156539B (en) | 2011-03-28 | 2011-03-28 | Method for identifying target object based on eye movement scanning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100754857A CN102156539B (en) | 2011-03-28 | 2011-03-28 | Method for identifying target object based on eye movement scanning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102156539A true CN102156539A (en) | 2011-08-17 |
| CN102156539B CN102156539B (en) | 2012-05-02 |
Family
ID=44438062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100754857A Expired - Fee Related CN102156539B (en) | 2011-03-28 | 2011-03-28 | Method for identifying target object based on eye movement scanning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102156539B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN104755023A (en) * | 2012-11-02 | 2015-07-01 | 索尼公司 | Image display device and information input device |
| CN108175547A (en) * | 2018-01-30 | 2018-06-19 | 浙江省公众信息产业有限公司 | Control method, device and the computer readable storage medium of artificial prosthesis |
| CN108874148A (en) * | 2018-07-16 | 2018-11-23 | 北京七鑫易维信息技术有限公司 | A kind of image processing method and device |
| CN113359996A (en) * | 2021-08-09 | 2021-09-07 | 季华实验室 | Life auxiliary robot control system, method and device and electronic equipment |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI569210B (en) * | 2013-11-26 | 2017-02-01 | Nat Kaohsiung First Univ Of Science And Tech | Non - invasive brain wave eye movement input control device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101308400A (en) * | 2007-05-18 | 2008-11-19 | 肖斌 | Novel human-machine interaction device based on eye-motion and head motion detection |
| CN101644952A (en) * | 2009-06-04 | 2010-02-10 | 浙江大学 | Portable wireless man-machine interactive system based on electro-oculogram signal |
| TW201042482A (en) * | 2009-05-19 | 2010-12-01 | Nat Univ Chung Hsing | Three dimensional controlling device by using electroencephalogram (EEG) and electro-oculogram (EOG) and method thereof |
-
2011
- 2011-03-28 CN CN2011100754857A patent/CN102156539B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101308400A (en) * | 2007-05-18 | 2008-11-19 | 肖斌 | Novel human-machine interaction device based on eye-motion and head motion detection |
| TW201042482A (en) * | 2009-05-19 | 2010-12-01 | Nat Univ Chung Hsing | Three dimensional controlling device by using electroencephalogram (EEG) and electro-oculogram (EOG) and method thereof |
| CN101644952A (en) * | 2009-06-04 | 2010-02-10 | 浙江大学 | Portable wireless man-machine interactive system based on electro-oculogram signal |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN102830797B (en) * | 2012-07-26 | 2015-11-25 | 深圳先进技术研究院 | A kind of man-machine interaction method based on sight line judgement and system |
| CN104755023A (en) * | 2012-11-02 | 2015-07-01 | 索尼公司 | Image display device and information input device |
| CN108175547A (en) * | 2018-01-30 | 2018-06-19 | 浙江省公众信息产业有限公司 | Control method, device and the computer readable storage medium of artificial prosthesis |
| CN108874148A (en) * | 2018-07-16 | 2018-11-23 | 北京七鑫易维信息技术有限公司 | A kind of image processing method and device |
| CN113359996A (en) * | 2021-08-09 | 2021-09-07 | 季华实验室 | Life auxiliary robot control system, method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102156539B (en) | 2012-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102156539B (en) | Method for identifying target object based on eye movement scanning | |
| CN109992113B (en) | MI-BCI system based on multi-scene induction and control method thereof | |
| CN102309366B (en) | Control system and control method for controlling upper prosthesis to move by using eye movement signals | |
| CN106671084B (en) | A kind of autonomous householder method of mechanical arm based on brain-computer interface | |
| CN103212156B (en) | The autonomous instrument of hemiplegic patient based on body image theory | |
| Chen et al. | A human-robot interface based on electrooculography | |
| Hartmann et al. | Closed-loop control of myoelectric prostheses with electrotactile feedback: influence of stimulation artifact and blanking | |
| CN104382595A (en) | Upper limb rehabilitation system and method based on myoelectric signal and virtual reality interaction technology | |
| CN111096796B (en) | Full-automatic vein puncture robot multilayer control system | |
| CN103892829B (en) | Eye movement signal identification system based on common spatial mode and identification method thereof | |
| CN107875526B (en) | Accurate control method of radiotherapy instrument in self-adaptive radiotherapy of eye tumor | |
| CN109875863A (en) | Wear-type VR eyesight lifting system based on binocular vision Yu mental image training | |
| Aungsakun et al. | Robust eye movement recognition using EOG signal for human-computer interface | |
| US9907489B2 (en) | Systems and methods for hierarchical pattern recognition for simultaneous control of multiple-degree of freedom movements for prosthetics | |
| Duvinage et al. | Control of a lower limb active prosthesis with eye movement sequences | |
| CN110882150A (en) | Massage robot acupuncture point tracking system based on visual positioning | |
| CN104971433B (en) | The functional electrostimulation athletic rehabilitation system and method manually merged with autonomous control | |
| CN104997582B (en) | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals | |
| CN109508094A (en) | A kind of vision inducting brain-machine interface method of the asynchronous eye movement switch of combination | |
| Jeong et al. | Design of a brain-controlled robot arm system based on upper-limb movement imagery | |
| Nakanishi et al. | Wheelchair control system by using electrooculogram signal processing | |
| CN109568891A (en) | The main passive exercise schema control system of healing robot and method based on brain electricity | |
| Kim et al. | Reconstruction of hand movements from EEG signals based on non-linear regression | |
| CN113426007A (en) | Closed-loop epidural electrical stimulation system for upper limb functional recovery | |
| CN211583162U (en) | User sitting posture detection device of massage armchair and massage armchair |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120502 Termination date: 20210328 |