CN101344919B - Sight tracing method and disabled assisting system using the same - Google Patents
Sight tracing method and disabled assisting system using the same Download PDFInfo
- Publication number
- CN101344919B CN101344919B CN2008100300104A CN200810030010A CN101344919B CN 101344919 B CN101344919 B CN 101344919B CN 2008100300104 A CN2008100300104 A CN 2008100300104A CN 200810030010 A CN200810030010 A CN 200810030010A CN 101344919 B CN101344919 B CN 101344919B
- Authority
- CN
- China
- Prior art keywords
- spot
- gray
- pul
- admiring
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a view line tracking method and an assistant system for the handicapped by using the method. The method comprises the steps of image pretreatment, the orientation of a human eye area by utilizing Harr features, blinking recognition, the inspection and judgment of LED reflecting lightspot in pupils, the accurate position of pupil centre, the calculation of view line direction and so on. The system comprises an embedded-like platform, a camera, a display screen and four LED and wheelchair running control modules which are installed at the four corners of the display screen and arranged into a rectangular. The system is an embedded-type system, which carries out blinking inspection, the inspection of the reflecting lightspots against infrared light sources on corneas, the accurate position of the pupil centers and the judgment of view line directions by the camera and a processor, tracks the view line of human eyes and, meanwhile, carries out the recognition of the status of the human eyes by judging the view line directions of the human eyes when gazing on the screen, thus realizing the functions of wheelchair control, music play control, text reading and webpage rolling. The invention is convenient for use and has high accuracy, thereby being capable of bringing great convenience for the lives of the handicapped.
Description
Technical field
The vision track algorithm belongs to Flame Image Process and mode identification technology among the present invention, its application apparatus: disabled assisting system relates to embedded system and singlechip technology.
Background technology
People's an eye line is played an important role in man-machine interaction, and it has directly, nature and advantage such as two-way.At present eye tracking technology ground zero does not reach the practical stage, and successful practicality project seldom and cost an arm and a leg, and is high to the requirement of hardware, need two even two above cameras usually, and requirement has very high pixel.The vision track technology generally can be divided into two types, contact and contactless.The precision of contact is high, but the user must dress special utensil, and this can bring very big uncomfortable to the user.The way that contactless general employing is handled based on video image through analyst's eye portion spectral discrimination direction of visual lines, does not produce interference to the user, uses convenient.
Disabled assisting system, the main direction of research is based on speech recognition at present.But it is, also inapplicable based on the backup system of speech recognition for the severe disability people who lacks language ability.In view of the limitation of speech recognition on disabled assisting system, eye gaze information capable of using is as intermediary, accomplishes mutual between disabled person and the backup system.Through the retrieval of prior art document is found, domestic do not have as yet report based on the technological disabled assisting system of vision track.
Summary of the invention
The objective of the invention is to overcome the above-mentioned deficiency that prior art exists; Sight tracing is provided and uses the disabled assisting system of this method; The present invention is directed to the deficiency in the eye tracking technology, provide a kind of contactless, accurately and fast track algorithm.This algorithm is low to hardware requirement, only needs a common camera, has solved the restriction of multi-cam vision track to hardware, and the high calculated amount of bringing thus, has improved the real-time of eye tracking.
The present invention simultaneously is applied to disabled assisting system with the eye tracking technology, has filled up the blank of China's disabled assisting system this respect.Through camera track human eye fixation sight line; Four kinds of control informations that four kinds of different sight directions in four zones of people's eye fixation screen are sent as the user; Be used for confirming and the switching controls command functions with moving this nictation of closing one's eyes of 1～3 second simultaneously; Realize control wheelchair running method, other basic operational functions of playing meter multimedia audio file and accomplishing computing machine.
The present invention realizes through following technical scheme:
A kind of sight tracing, at people's face dead ahead structure infrared light supply, infrared light supply can form the bright spot image in its surface after the eye cornea reflection, is called the pul spot of admiring, and it is characterized in that this method comprises the steps:
(1) detects the pul spot of admiring: gather through the postradiation facial image of infrared light supply, utilize pul the admire position of spot of the geometry character detection pul of spot of admiring;
(2) accurately locate pupil center: the pul spot of admiring is carried out the gray scale correction, after the smoothing processing, the legal position of employing level and vertical Gray Projection pupil center;
(3) calculate direction of visual lines: ask for the admire position relation of spot of pupil center and pul through geometrical calculation, thus the differentiation direction of visual lines;
(4) nictation is confirmed direction of visual lines in utilization: discern human eye state, through front and back frame human eye area black picture element number relation, confirm the action of blinking.
In the said method, said infrared light supply number is four, and is positioned on four angles of same rectangle, said pul admire spot show as four the reflection bright spots, the said geometric properties of step (1) comprises:
(1.1) position is positioned at around the pupil, with pupil center distance less than 30 pixels;
(1.2) size is 5～20 pixels, gray-scale value 100 with, show as outstanding bright spot;
(1.3) in the ocular, the spot place gray-scale value sudden change of admiring of four puls is maximum;
(1.4) four puls are admired distance between the spot in 8～18 pixel coverages, and become rectangle or similar rectangle relation.
Said four puls spot of admiring is confirmed through following steps:
At first utilize sciagraphy to orient pupil center location, around the pupil center distance less than 30 pixel coverages in search satisfy the admire pixel point set of spot geometric properties of pul; With the Laplace operator they are carried out convolution again, the value that obtains with the Laplace convolution is big more, explains that the sudden change of this place's gray scale is big more, has peaked four pixels behind the Laplace operator convolution algorithm, is four puls spot of admiring.
In the said method, spot is connected to form rectangle or type rectangle in order if four puls are admired, and then confirms as the reflection spot of four said infrared light supplies.
In the said method, first employing level and vertical Gray Projection Primary Location pupil center in the step (2) are carried out the gray-scale value correction to the pul spot of admiring, and after the smoothing processing, utilize level can obtain accurate pupil center location with vertical Gray Projection method location once more.
In the said method; Pul is admired the method for spot gray-scale value correction for being the center with one of them pul spot P0 that admires; Gray-scale value to each pixel among 8 pixel A 1～A8 adjacent with P0 is revised: revising pixel with quilt is the center; Get its adjacent 8 pixels on every side, searching comprises the minimum gray value in these 9 pixels of pixel of being revised, and it is substituted the gray-scale value of being revised pixel; Again the gray-scale value of P0 is revised: get P0, the minimum gray value in these 9 pixels of A1～A8, with the gray-scale value of its alternative P0, the admire tone correcting method of spot of other three puls is identical with P0.
In the said method, in the said geometrical calculation of step (3) be: establish P0, P1, P2, P3 and be detected four puls spot of admiring, Q is a pupil center; Utilize section formula to obtain the coordinate of the diagonal line intersection point O of P0～P3, link OQ, OP0; OP1, OP2, OP3; OP0～OP3 is divided into four zones to the rectangle that is connected into by P0～P3, calculates OQ and is in which zone, just can determine corresponding direction of visual lines.
In the said method; In the step (4) human eye action 1～3 second nictation is used for confirming direction of visual lines; Said nictation, action detected through following method: with unifying threshold value Harr feature location gained human eye area is carried out binary conversion treatment, black eye area diminishes when closing one's eyes, when former frame and after in the two field picture human eye area black picture element percentage greater than setting threshold; Judge that then human eye becomes closure from opening, action nictation is arranged.
A kind of disabled assisting system of using above-mentioned sight tracing is an embedded system, four LED and the wheelchair that it is characterized in that comprising four angles on embedded platform, camera, display screen, the installation display screen and the be arranged in rectangle control module of going; Said camera is input to embedded platform with the digital picture that collects; Said display location is connected with embedded platform through the VGA interface in the dead ahead of people's face; The wheelchair control module of going is connected with embedded platform through serial ports; The said wheelchair control module of going comprises wireless transmitter module and wireless receiving module; Wireless transmitter module is connected with said serial ports through a single-chip microcomputer; Wireless receiving module is connected with driving circuit through another single-chip microcomputer, and driving circuit is connected with the drive unit of wheelchair; Said embedded platform comprises processor and storer, and the software in the storer is used to realize step (1)～(4) of said sight tracing; After identifying user's sight line information, processor is transferred to wheelchair go control module with control signal corresponding through serial ports according to user's direction of visual lines again, and the wheelchair control module of going is controlled the drive unit of wheelchair.
In the described system through the user blink the action be used for confirming sight line control information, analog computation keyboard enter key and import user's confirmation; The line at display screen center and four angles is screen divider four zones; The sight line in these four zones of people's eye fixation is represented four direction of visual lines in upper and lower, left and right; This four direction key of simulating keyboard upper and lower, left and right is as the control information of user's input.
Advantage of the present invention and good effect are:
1. adopt advanced vision track technology, realized the disabled assisting system of a perfect in shape and function, concrete function has:
Control wheelchair with eyes, completion is advanced, and turns left, control action such as turn right and stop;
With eyes control computer multimedia, as selecting music file, control music etc., satisfy the needs of disabled person's life and amusement;
Can realize the basic operation of computing machine with sight line, as open webpage, the page turning up and down of e-book etc. make disabled person's reading electronic book glibly, browsing page, input characters etc.
2. adopt right eye sorter by Modesto Castrill ó n training; The method that this sorter proposes based on Paul Viola based on weak feature detection in the gray level image; It is the Haar characteristic detection method; This method calculated amount is little, and degree of accuracy is high, is fit to very much the real-time and the accuracy requirement of disabled assisting system.
3. on the location algorithm of pupil center; The present invention improves with the method for vertical Gray Projection algorithm location human eye existing level; Four puls spot of admiring is carried out the gray-scale value correction, has improved the degree of accuracy of algorithm, simultaneously also better controlled the complexity of algorithm.
4. in human eye state identification; The image of much closing one's eyes and opening eyes has been done to find after the comparative analysis; Can see that from eye profile binary map black picture element was more when human eye was closed, black picture element is less when closing one's eyes, and the black number characteristic of black eyes can well make a distinction the two kinds of situation of opening eyes and close one's eyes like this; Can avoid considering respectively the situation of opening eyes and closing one's eyes like this, compare other algorithms better real-time property is provided.
5. adopt a kind of real-time and effective sight calculations method; This method is through calculating pupil center and the pul speckle displacement relation judgement direction of visual lines of admiring; Digital picture to camera collection is carried out a series of images Processing Algorithm, just can accomplish contactless eye tracking.Test data proves: vision track algorithm of the present invention has 96% and differentiates accuracy rate and the processing speed of 7 frame/seconds.This technology is not limited to disabled assisting system, also can be widely used in other field.
Description of drawings
Fig. 1 is the layout synoptic diagram of display screen in the embodiment of the present invention, infrared light supply and camera.
Fig. 2 is the schematic flow sheet of sight tracing in the embodiment of the present invention.
Fig. 3 asks the pul spot of admiring to constitute the diagonal line intersection point synoptic diagram of rectangle in the embodiment of the present invention.
Fig. 4 is in the embodiment of the present invention, disabled assisting system constitute block diagram.
Fig. 5 is that the wheelchair control module of going constitutes block diagram among Fig. 4.
Embodiment
Be described further below in conjunction with the accompanying drawing specific embodiments of the invention.
Like Fig. 1, image capture module comprises infrared light supply and camera.Infrared light supply is four LED lamps that are installed in four jiaos of display screens, and camera is positioned under the screen center, the digital picture input eye tracking module of camera collection.
Like Fig. 2, eye tracking algorithm practical implementation step is following:
Step 1: image pre-service
Step 2: utilize Harr feature location human eye area
Step 3: identification nictation
Step 4: the detection and the judgement of LED reflection bright spot in the pupil
Step 5: the accurate location of pupil center
Step 6: the calculating of direction of visual lines
Wherein the practical implementation step of step 1 is:
1. coloured image gray processing
The digital picture of gathering is to have R, G, and the colour picture of B component, algorithm only need gray level image, and with the coloured image gray processing, formula is following:
V＝0.259R+0.578G+0.144B；
Gray-scale value after V representes to change in the formula, R, G, B is the red of picture, and is green, blue component.
2. down-sampling
In all types of target detection method, the time complexity and the image resolution ratio of algorithm are proportional, under target detection accuracy condition with higher, and can be through image be adopted to improve arithmetic speed down.Among the present invention camera collection to image resolution ratio be 640 * 480, with the gray scale method of average it is punctured into 320 * 240 image.
Wherein the practical implementation step of step 2 is:
The right eye sorter that employing is obtained by Modesto Castrill ó n training carries out the human eye location to image.
Wherein the practical implementation step of step 3 is:
Analyze the human eye area bianry image and can know that black picture element was more when human eye was closed, black picture element is less when opening eyes, so black picture element number characteristic can well make a distinction opening eyes and closing one's eyes in the two-value ocular image.Whether the present invention adds up the number of two-value ocular black picture element in every two field picture, and compares with former frame, utilizes the relation between the interframe black picture element number, differentiate to exist human eye to transfer action nictation of closing to by opening.Detailed process is following:
If the i two field picture is F
i, human eye area is Di
1. add up F
iIn, region D
iGray-scale value is less than 150 pixel number C
i
2. add up F
I+1In, region D
iGray-scale value is less than 150 pixel number C
i';
3. if C
i/ C
i'＞0.9 is then thought the incident of closing one's eyes to occur;
4. if continuous some frames detect less than eyes after the possible incident of closing one's eyes occurring, then confirm as and close one's eyes.
Wherein the practical implementation step of step 4 is:
In the eye areas, the pul spot of admiring has following geometric properties:
1. be positioned at around the pupil, with pupil center distance less than 30 pixels
2. size is that 5～20 pixels do not wait, and gray-scale value is more than 100;
3. in eye areas, the gray-scale value at bright spot place respectively has a maximum value, and under ideal conditions, and the admire sudden change of spot place gray scale of four puls is maximum;
4. four puls are admired between the spot distance in 8～18 pixel coverages, and are approximated to the rectangle relation;
Therefore seek bright spot through following steps:
1. with the center of the legal position of horizontal Gray Projection and vertical Gray Projection pupil, with the scope of each 30 pixel up and down at this center as the region of search;
2. in the region of search, seek the gray scale extreme point, promptly seek the point set G that gray-scale value meets the following conditions:
g(x
0，y
0)≥max{g(x，y)}，g(x
0，y
0)＞100
3. with Laplace operator as follows each point among the point set G is carried out convolution with point around it, obtain the differential f at each some g place.
Because the Laplace operator is a kind of isotropic differentiating operator, its effect is the zone of stressing gray scale sudden change in the image, and Laplace convolution intermediate value is big more, explains that the sudden change of this place's gray scale is big more.Adopt 5 * 5 Laplace operator, can further avoid noise;
4. point set G is sorted by its differential value f, select four maximum some P0～P3 of f, as candidate point;
5. check P0～P3 if can form rectangle, confirms that then P0～P3 is four puls spots of admiring, otherwise abandons current frame image.
Wherein the practical implementation step of step 5 is:
If human eye area image size is m*n, by following formula it is done level and vertical Gray Projection, finding respectively has a minimal value in the vertical direction of pupil center and the Gray Projection of horizontal direction.Horizontal Gray Projection is that each gray values of pixel points along continuous straight runs accumulative total in the gray level image is projected on the image vertical axes; Vertical Gray Projection is that each gray values of pixel points in the gray level image is vertically totally projected on the image level axle.Detailed process is following:
Ph wherein
y(y), Ph
x(x) be respectively level and vertical Gray Projection, (x y) is each gray values of pixel points in the eyes image to I.
Therefore can try to achieve the Q of pupil center is:
(x
0，y
0)where?Ph
y(y
0)＝Min(Ph
y(y))?and?Ph
x(x
0)＝Min(Ph
x(x))
LED reflection bright spot drops in the pupil region, can change drop shadow curve's characteristic, influences the pupil center location.Need respectively bright spot P0～P3 to be carried out the gray scale correction, smoothly its gray-scale value improves the accuracy of pupil center location.With P0 is that the concrete smoothing method of example is following:
| A1 | A2 | A3 |
| A4 | P0 | A5 |
| A6 | A7 | A8 |
With P0 is the center, revises the gray-scale value of pixel A 1～A8.Modification method is following: with the A1 center, get its adjacent 8 pixels on every side, searching comprises the minimum gray value in these 9 pixels of A1, with the gray-scale value of its alternative A1.In like manner can revise the gray-scale value of A2～A8, calculate and revise A1～A8 minimum value afterwards, the gray-scale value of its alternative P0.
Wherein the practical implementation step of step 6 is:
According to physics and method of geometry, can confirm the one-to-one relationship of screen and images acquired, promptly four bright spots are corresponding in the LED at four angles of screen and the eye image, and direction of visual lines is corresponding with pupil center.Therefore according to this corresponding relation, just can judge direction of visual lines through the eye image of gathering, concrete grammar is following:
1. according to accompanying drawing 3, ask the diagonal line intersection point O of P0～P3 with the method for computational geometry:
Definition by leg-of-mutton area formula and cross product has:
By fixed formula, can obtain the x coordinate that O orders and be than branch:
In like manner also can obtain the y coordinate that O is ordered.
2. link OQ, OP0, OP1, OP2, OP3, then the zone at the Q of pupil center place can be obtained according to following relationship:
Area 0: OQ drops between OP0 and the OP1, corresponding direction of visual lines be " on "
Zone 1:OQ drops between OP1 and the OP2, and corresponding direction of visual lines is " right side "
Zone 2:OQ drops between OP2 and the OP3, and corresponding direction of visual lines is a D score
Zone 3:OQ drops between OP3 and the OP0, and corresponding direction of visual lines is " left side "
The embodiment of disabled assisting system of using eye tracking algorithm of the present invention is following:
Like Fig. 4; Native system adopts the embedded platform based on the Core Duo of Intel processor; Utilize contactless eye tracking as core algorithm, in conjunction with infrared light supply, one 300,000 pixel camera head; (SuSE) Linux OS and GTK+ design user interface, and with the dolly of controlled in wireless simulation disabled person electric wheelchair.This embedded platform comprises processor, storer, and VGA, USB, AC97, a series of peripheral interfaces such as serial ports.Wherein processor is a control center, and storer, peripheral interface all link to each other in handling.Camera, LCD, audio player pass through USB respectively, VGA, and the AC97 interface is connected with embedded platform.
System mid-infrared light source is four LED irradiation lights that are installed on the display screen of people's face dead ahead four angles, are arranged in rectangle, camera be positioned at screen center under.For the camera lens that makes camera is to infrared more responsive, before camera lens, add the mating plate of filtering natural light.Line according to LCDs center and four angles is screen divider four zones; The sight line in these four zones of people's eye fixation is represented four direction of visual lines in upper and lower, left and right; This four direction key of simulating keyboard upper and lower, left and right is as the control information of user's input.Simultaneously for human eye unintentionally the sight line drift come with the difference of control sight line, design system has common and two states of locking, just people's an eye line changes the variation that causes control information during normal state.Utilize user 1～3 action nictation, this two states of switched system.Be used for confirming the sight line control information through action nictation simultaneously, simulating keyboard enter key, input user confirmation.
The software of embedded system has cured in hardware system, comprises operating system and application software two parts.Native system adopts the (SuSE) Linux OS of Ubuntu8.04, and kernel is 2.6.24, has avoided the driving problems of peripheral hardware.The described sight tracing of preamble is realized with software, is the core in the native system application software.Eye tracking software earlier carries out the pul spot detection of admiring to the digital picture of input; Pupil center location is detected and is waited a series of images Processing Algorithm nictation, calculates and according to the confirming operation direction of visual lines of blinking again; Thereby accomplish the task of the user's sight line information in the images acquired that identifies; According to user's sight line information, send control information to serial ports, thereby wheelchair is controlled again.When analog direction key and affirmation (carriage return) key, use the function library Xtest of a keyboard button simulating among the X11 under the Linun.On processor, adopt GTK+ design configuration interface simultaneously, Gtk+ has well solved the problem of display image on control on the one hand, and GTK+ supports multithreading on the other hand, and it is congested to avoid a plurality of threads to cause.
Like Fig. 5, the wheelchair control module of going is made up of wireless transmit plate and the wheelchair control wireless receiving plate of holding concurrently.The wireless transmit plate is imported single-chip microcomputer with serial data through after the level conversion, and single-chip microcomputer sends data to transmitter module through the P0 mouth.The wireless receiving plate is given single-chip microcomputer with data through the P2 oral instructions, and single-chip microcomputer turning left, is turned right and stopped control information and pass to wheelchair after a chip for driving, perhaps stops to turn left right-hand rotation engine control wheelchair running method through braking.
Because we with a dolly simulation wheelchair, consider the required driving force of motor of actual dolly, we adopt L293D chip for driving.If real wheelchair needs stronger driving force, change chip for driving into L298 and get final product.
Claims (3)
1. sight tracing, this method is the structure infrared light supply in people's face dead ahead, and infrared light supply can form the bright spot image in its surface after the eye cornea reflection, be called the pul spot of admiring, and it is characterized in that this method also comprises the steps:
(1) detects the pul spot of admiring: gather through the postradiation facial image of infrared light supply; Utilize pul the admire position of spot of the geometry character detection pul of spot of admiring; Said infrared light supply number is four, and is positioned on four angles of same rectangle, said pul admire spot show as four the reflection bright spots; Said four puls spot of admiring is confirmed through following steps: at first utilize sciagraphy to orient pupil center location, around pupil center apart from pupil center less than 30 pixel coverages in search satisfy the admire pixel point set of spot geometric properties of pul; With the Laplace operator they are carried out convolution again, the value that obtains with the Laplace convolution is big more, explains that the sudden change of this place's gray scale is big more, has peaked four pixels behind the Laplace operator convolution algorithm, is four puls spot of admiring; Said geometric properties comprises:
(1.1) position is positioned at around the pupil, with pupil center distance less than 30 pixels;
(1.2) size is 5～20 pixels, and gray-scale value shows as outstanding bright spot more than 100;
(1.3) in the ocular, the spot place gray-scale value sudden change of admiring of four puls is maximum;
(1.4) four puls are admired distance between the spot in 8～18 pixel coverages, and become the rectangle relation;
(2) first employing level and vertical Gray Projection Primary Location pupil center are carried out the gray-scale value correction to the pul spot of admiring, and after the smoothing processing, utilize level can obtain accurate pupil center location with vertical Gray Projection method location once more; Said pul is admired the method for spot gray-scale value correction for being the center with one of them pul spot P0 that admires; At first the gray-scale value of each pixel among 8 pixel A 1～A8 adjacent with P0 is revised; Promptly revising pixel with quilt is the center; Get its adjacent 8 pixels on every side, searching comprises the minimum gray value in these 9 pixels of pixel of being revised, and it is substituted the gray-scale value of being revised pixel; And then the gray-scale value of P0 revised, promptly get P0, the minimum gray value in these 9 pixels of A1～A8, with the gray-scale value of its alternative P0, the admire tone correcting method of spot of other three puls is identical with P0;
(3) calculate direction of visual lines: ask for the admire position relation of spot of pupil center and pul through geometrical calculation, thus the differentiation direction of visual lines; Said geometrical calculation is: establish P0, P1, P2, P3 and be detected four puls spot of admiring, Q is a pupil center, utilizes section formula to obtain the coordinate of the diagonal line intersection point O of P0～P3; Link OQ, OP0, OP1; OP2, OP3, OP0～OP3 is divided into four zones to the rectangle that is connected into by P0～P3; Calculate OQ and be in which zone, just can determine corresponding direction of visual lines;
(4) nictation is confirmed direction of visual lines in utilization: discern human eye state, through front and back frame human eye area black picture element number relation, confirm the action of blinking.
2. method according to claim 1 is characterized in that spot is connected to form rectangle in order if four puls are admired, and then confirms as the reflection spot of four said infrared light supplies.
3. method according to claim 2; It is characterized in that in the step (4) human eye action 1～3 second nictation being used for confirming direction of visual lines; Said nictation, action detected through following method: with unifying threshold value Harr feature location gained human eye area is carried out binary conversion treatment, black eye area diminishes when closing one's eyes, when former frame and after in the two field picture human eye area black picture element percentage greater than setting threshold; Judge that then human eye becomes closure from opening, action nictation is arranged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100300104A CN101344919B (en) | 2008-08-05 | 2008-08-05 | Sight tracing method and disabled assisting system using the same |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008100300104A CN101344919B (en) | 2008-08-05 | 2008-08-05 | Sight tracing method and disabled assisting system using the same |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101344919A CN101344919A (en) | 2009-01-14 |
| CN101344919B true CN101344919B (en) | 2012-08-22 |
Family
ID=40246921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008100300104A Expired - Fee Related CN101344919B (en) | 2008-08-05 | 2008-08-05 | Sight tracing method and disabled assisting system using the same |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101344919B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9805260B2 (en) | 2014-07-09 | 2017-10-31 | Pixart Imaging Inc. | Vehicle safety system and operating method thereof |
Families Citing this family (66)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807110B (en) * | 2009-02-17 | 2012-07-04 | 由田新技股份有限公司 | Pupil positioning method and system |
| CN101788848B (en) * | 2009-09-29 | 2012-05-23 | 北京科技大学 | Eye characteristic parameter detecting method for sight line tracking system |
| CN101813976A (en) * | 2010-03-09 | 2010-08-25 | 华南理工大学 | Sighting tracking man-computer interaction method and device based on SOC (System On Chip) |
| CN101889928B (en) * | 2010-07-27 | 2012-04-18 | 北京理工大学 | Head gesture recognition technology-based wheelchair control method |
| CN101976115B (en) * | 2010-10-15 | 2011-12-28 | 华南理工大学 | Motor imagery and P300 electroencephalographic potential-based functional key selection method |
| CN102467230A (en) * | 2010-11-11 | 2012-05-23 | 由田新技股份有限公司 | Method and system for operating and controlling vernier by human body |
| CN102779502A (en) * | 2011-05-11 | 2012-11-14 | 中兴通讯股份有限公司 | Terminal screen control method and terminal |
| CN102811308B (en) * | 2011-05-31 | 2016-08-31 | 德尔福电子(苏州)有限公司 | A kind of vehicle-mounted eye movement control system |
| CN102551385B (en) * | 2011-12-29 | 2014-08-20 | 广东工业大学 | Automatically-paging reading chair based on sight tracking and control device and control method of automatically-paging reading chair |
| CN103186233B (en) * | 2011-12-31 | 2016-03-09 | 上海飞来飞去新媒体展示设计有限公司 | Panoramic interaction control method for eye location |
| CN102749991B (en) * | 2012-04-12 | 2016-04-27 | 广东百泰科技有限公司 | A kind of contactless free space sight tracing being applicable to man-machine interaction |
| CN103376554B (en) * | 2012-04-24 | 2017-12-26 | 联想(北京)有限公司 | Hand-hold electronic equipments and display methods |
| CN102727350B (en) * | 2012-06-29 | 2015-05-13 | 中航华东光电有限公司 | Electrically-powered wheelchair control device and control method thereof |
| CN103054548A (en) * | 2012-07-05 | 2013-04-24 | 东北电力大学 | Fixation point measurement device and pupil recognition method and Purkinje image recognition method |
| CN103631364B (en) * | 2012-08-20 | 2017-06-27 | 联想(北京)有限公司 | A kind of control method and electronic equipment |
| TW201416908A (en) | 2012-10-23 | 2014-05-01 | Pixart Imaging Inc | Pupil tracking device |
| CN102981736B (en) * | 2012-10-29 | 2016-11-23 | 华为终端有限公司 | The method of unlocking screen and terminal |
| CN103793045B (en) * | 2012-10-31 | 2016-12-28 | 原相科技股份有限公司 | Pupil tracing device |
| WO2014068975A1 (en) * | 2012-11-05 | 2014-05-08 | パナソニック株式会社 | Autonomous traveling device traveling-information generation device, method, and program, and autonomous traveling device |
| CN103076876B (en) * | 2012-11-22 | 2016-02-10 | 西安电子科技大学 | Based on character entry apparatus and the method for eye tracking and speech recognition |
| CN102999164B (en) * | 2012-11-30 | 2016-08-03 | 广东欧珀移动通信有限公司 | A kind of e-book flipping-over control method and intelligent terminal |
| CN103902192A (en) * | 2012-12-28 | 2014-07-02 | 腾讯科技（北京）有限公司 | Trigger control method and trigger control device for man-machine interactive operation |
| CN103941984A (en) * | 2013-01-18 | 2014-07-23 | 维沃移动通信有限公司 | Method and system for browser webpage interface intelligent sliding in mobile handheld equipment |
| US9483115B2 (en) * | 2013-05-31 | 2016-11-01 | Disney Enterprises, Inc. | Triggering control of audio for walk-around characters |
| CN103336576B (en) * | 2013-06-28 | 2016-12-28 | 广州爱九游信息技术有限公司 | A kind of moving based on eye follows the trail of the method and device carrying out browser operation |
| CN103356163B (en) * | 2013-07-08 | 2016-03-30 | 东北电力大学 | Based on fixation point measuring device and the method thereof of video image and artificial neural network |
| CN103605422A (en) * | 2013-10-23 | 2014-02-26 | 苏州安可信通信技术有限公司 | Touch screen operation method and device |
| CN104657702B (en) * | 2013-11-25 | 2019-04-12 | 原相科技股份有限公司 | Eyeball arrangement for detecting, pupil method for detecting and iris discrimination method |
| CN103885589B (en) * | 2014-03-06 | 2017-01-25 | 华为技术有限公司 | Eye movement tracking method and device |
| CN105022981B (en) * | 2014-04-18 | 2019-08-16 | 中兴通讯股份有限公司 | A kind of method, device and mobile terminal detecting human eye health state |
| CN103927014A (en) * | 2014-04-21 | 2014-07-16 | 广州杰赛科技股份有限公司 | Character input method and device |
| WO2015167471A1 (en) * | 2014-04-29 | 2015-11-05 | Hewlett-Packard Development Company, L.P. | Gaze detector using reference frames in media |
| KR101619661B1 (en) * | 2014-12-08 | 2016-05-10 | 현대자동차주식회사 | Detection method of face direction of driver |
| CN104898971B (en) * | 2015-05-11 | 2017-12-22 | 昆明理工大学 | A kind of mouse pointer control method and system based on Visual Trace Technology |
| CN104898850B (en) * | 2015-06-30 | 2018-02-06 | 上海安翰医疗技术有限公司 | A kind of multiple devices share the input switching system and method for input unit |
| CN105105938A (en) * | 2015-07-14 | 2015-12-02 | 南京邮电大学 | Intelligent wheelchair control method and system based on face orientation identification and tracking |
| US9829976B2 (en) * | 2015-08-07 | 2017-11-28 | Tobii Ab | Gaze direction mapping |
| CN106708251A (en) * | 2015-08-12 | 2017-05-24 | 天津电眼科技有限公司 | Eyeball tracking technology-based intelligent glasses control method |
| CN105128862A (en) * | 2015-08-18 | 2015-12-09 | 上海擎感智能科技有限公司 | Vehicle terminal eyeball identification control method and vehicle terminal eyeball identification control system |
| CN105159460B (en) * | 2015-09-10 | 2018-01-23 | 哈尔滨理工大学 | The control method of the intelligent domestic appliance controller based on eye-tracking |
| PL414836A1 (en) * | 2015-11-18 | 2017-05-22 | Assistech Spółka Z Ograniczona Odpowiedzialnością | Method and the system to assist the assessment of neurological condition and to assist carrying out neurological rehabilitation, preferably the cognitive and/or language functions |
| JP6638354B2 (en) * | 2015-12-01 | 2020-01-29 | 株式会社Ｊｖｃケンウッド | Eye gaze detection device and eye gaze detection method |
| CN105843376A (en) * | 2016-03-07 | 2016-08-10 | 乐视移动智能信息技术（北京）有限公司 | Operation method of mobile terminal application, and mobile terminal |
| CN105892691A (en) * | 2016-06-07 | 2016-08-24 | 京东方科技集团股份有限公司 | Method and device for controlling travel tool and travel tool system |
| CN106446822B (en) * | 2016-09-20 | 2018-07-10 | 西安科技大学 | Blink detection method based on circle fitting |
| CN106384096B (en) * | 2016-09-20 | 2018-07-10 | 西安科技大学 | A kind of fatigue driving monitoring method based on blink detection |
| CN106919933A (en) * | 2017-03-13 | 2017-07-04 | 重庆贝奥新视野医疗设备有限公司 | The method and device of Pupil diameter |
| CN106886290A (en) * | 2017-04-08 | 2017-06-23 | 闲客智能（深圳）科技有限公司 | A kind of eye flowing control method and device |
| WO2018184244A1 (en) * | 2017-04-08 | 2018-10-11 | 闲客智能（深圳）科技有限公司 | Eye movement control method and device |
| CN107007407B (en) * | 2017-04-12 | 2018-09-18 | 华南理工大学 | Wheelchair control system based on eye electricity |
| CN107506705B (en) * | 2017-08-11 | 2021-12-17 | 西安工业大学 | Pupil-purkinje spot sight line tracking and gaze extraction method |
| CN107909058A (en) * | 2017-11-30 | 2018-04-13 | 广东欧珀移动通信有限公司 | Image processing method, device, electronic equipment and computer-readable recording medium |
| CN107909057A (en) * | 2017-11-30 | 2018-04-13 | 广东欧珀移动通信有限公司 | Image processing method, device, electronic equipment and computer-readable recording medium |
| CN108090463B (en) * | 2017-12-29 | 2021-10-26 | 腾讯科技（深圳）有限公司 | Object control method, device, storage medium and computer equipment |
| CN108196676B (en) * | 2018-01-02 | 2021-04-13 | 联想(北京)有限公司 | Tracking identification method and system |
| CN108319338B (en) * | 2018-01-31 | 2022-06-14 | 努比亚技术有限公司 | Display screen determining method, mobile terminal and computer readable storage medium |
| CN109213324A (en) * | 2018-09-06 | 2019-01-15 | 京东方科技集团股份有限公司 | Display parameter regulation method and device, display device |
| US11570016B2 (en) | 2018-12-14 | 2023-01-31 | At&T Intellectual Property I, L.P. | Assistive control of network-connected devices |
| CN110658742A (en) * | 2019-09-05 | 2020-01-07 | 四川省康复辅具技术服务中心 | Multi-mode cooperative control wheelchair control system and method |
| CN110888441B (en) * | 2019-11-29 | 2023-04-07 | 广州乐比计算机有限公司 | Gyroscope-based wheelchair control system |
| CN111208905A (en) * | 2020-01-08 | 2020-05-29 | 北京未动科技有限公司 | Multi-module sight tracking method and system and sight tracking equipment |
| CN111772575A (en) * | 2020-06-02 | 2020-10-16 | 岭南师范学院 | LED (light emitting diode) -based optical nondestructive special children detector and detection method |
| CN112022641A (en) * | 2020-09-10 | 2020-12-04 | 深圳职业技术学院 | Method and system for assisting eye rotation movement |
| CN112114671A (en) * | 2020-09-22 | 2020-12-22 | 上海汽车集团股份有限公司 | Human-vehicle interaction method and device based on human eye sight and storage medium |
| CN113221630A (en) * | 2021-03-22 | 2021-08-06 | 刘鸿 | Estimation method of human eye watching lens and application of estimation method in intelligent awakening |
| CN113599096B (en) * | 2021-07-26 | 2023-04-18 | 东南大学 | Automatic wheelchair system of eye movement control based on pupil corneal reflex technique |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6152563A (en) * | 1998-02-20 | 2000-11-28 | Hutchinson; Thomas E. | Eye gaze direction tracker |
| US6578962B1 (en) * | 2001-04-27 | 2003-06-17 | International Business Machines Corporation | Calibration-free eye gaze tracking |
| JP2004188017A (en) * | 2002-12-12 | 2004-07-08 | Canon Inc | Gaze tracking system |
-
2008
- 2008-08-05 CN CN2008100300104A patent/CN101344919B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6152563A (en) * | 1998-02-20 | 2000-11-28 | Hutchinson; Thomas E. | Eye gaze direction tracker |
| US6578962B1 (en) * | 2001-04-27 | 2003-06-17 | International Business Machines Corporation | Calibration-free eye gaze tracking |
| JP2004188017A (en) * | 2002-12-12 | 2004-07-08 | Canon Inc | Gaze tracking system |
Non-Patent Citations (1)
| Title |
|---|
| 黄莹等.基于双光源的实时视线追踪系统.《中国工程科学》.2008,(第2期),86-90. * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9805260B2 (en) | 2014-07-09 | 2017-10-31 | Pixart Imaging Inc. | Vehicle safety system and operating method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101344919A (en) | 2009-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101344919B (en) | Sight tracing method and disabled assisting system using the same | |
| CN101813976A (en) | Sighting tracking man-computer interaction method and device based on SOC (System On Chip) | |
| CN103838378B (en) | A kind of wear-type eyes control system based on pupil identification positioning | |
| CN103176607B (en) | A kind of eye-controlled mouse realization method and system | |
| CN102520796B (en) | Sight tracking method based on stepwise regression analysis mapping model | |
| US11556741B2 (en) | Devices, systems and methods for predicting gaze-related parameters using a neural network | |
| EP2549914B1 (en) | System and method for tracking the point of gaze of an observer | |
| CN104133548A (en) | Method and device for determining viewpoint area and controlling screen luminance | |
| EP0596868A2 (en) | Eye tracking method using an image pickup apparatus | |
| US11393251B2 (en) | Devices, systems and methods for predicting gaze-related parameters | |
| CN102961117A (en) | Strabismus diagnosis device based on mobile platform | |
| WO2019154509A1 (en) | Devices, systems and methods for predicting gaze-related parameters | |
| CN103677270A (en) | Human-computer interaction method based on eye movement tracking | |
| CN101241548A (en) | Eyelid detection apparatus and programs thereof | |
| CN104766059A (en) | Rapid and accurate human eye positioning method and sight estimation method based on human eye positioning | |
| CN106682603A (en) | Real time driver fatigue warning system based on multi-source information fusion | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| CN101329728B (en) | LBP human face light irradiation preprocess method based on Hamming distance restriction | |
| Hyder et al. | Real-time non-intrusive eye-gaze tracking based wheelchair control for the physically challenged | |
| US8971592B2 (en) | Method for determining eye location on a frontal face digital image to validate the frontal face and determine points of reference | |
| CN105380590B (en) | A kind of equipment and its implementation with eye position detection function | |
| CN104898971A (en) | Mouse pointer control method and system based on gaze tracking technology | |
| CN112839414A (en) | Lighting device and control method thereof | |
| CN203885510U (en) | Driver fatigue detection system based on infrared detection technology | |
| CN206656973U (en) | A kind of product batch mixing intelligent detecting instrument |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120822 Termination date: 20180805 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |