CN103793045A - Pupil tracking device - Google Patents
Pupil tracking device Download PDFInfo
- Publication number
- CN103793045A CN103793045A CN201210428613.6A CN201210428613A CN103793045A CN 103793045 A CN103793045 A CN 103793045A CN 201210428613 A CN201210428613 A CN 201210428613A CN 103793045 A CN103793045 A CN 103793045A
- Authority
- CN
- China
- Prior art keywords
- pupil
- light source
- active light
- tracing device
- brightness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
A pupil tracking device comprises an active light source, an image sensor and a control processing unit. The active light source emits light to an eyeball by a first luminance and a second luminance in turn. The image sensor acquires a first luminance image corresponding to the first luminance and a second luminance image corresponding to the second luminance. The control processing unit is used for identifying brightest areas of relative positions in the first luminance image and the second luminance image as active light source images.
Description
Technical field
The invention relates to a kind of interaction systems, particularly about a kind of pupil tracing device of at least one active light source as positioning datum point that use.
Background technology
Interactive controlling mechanism can provide user more humane control mode, because being widely used in various multimedia system, especially can be applicable to the image display system that comprises display screen.
The telepilot that use can be obtained image is a kind of conventional means as interactive man-computer interface, and described telepilot can be made into various stage property, such as bat, racket, club etc.Another kind of interactive man-computer interface does not need to carry out with any hand-held device, and for example pupil tracing device can carry out interactive operation according to the variation of user's direction of visual lines.
The problem of known pupil tracing device is to be easily subject to the impact of environment light source, for example Fig. 1 shows the schematic diagram of eye image 9, it comprises pupil image Ip and environment light source image I o, when described pupil image Ip and described environment light source image I o are when overlapped, may there is correctly locating the situation of described pupil image Ip, thereby cannot correctly carry out pupil tracing.
In view of this, the present invention also proposes a kind of pupil tracing device, and it can get rid of the impact of environment light source, improves whereby the correctness of pupil tracing.
Summary of the invention
Object of the present invention is providing a kind of pupil tracing device, and it utilizes at least one active light source towards the luminous formation positioning datum point of eyeball.
Another object of the present invention is providing a kind of pupil tracing device that is applicable to wear-type accessory.
The invention provides a kind of pupil tracing device, comprise active light source, imageing sensor and controlled processing unit.Described active light source is luminous towards eyeball with the first brightness and the second brightness in turn.Described imageing sensor obtains the first luminance picture of relatively described the first brightness and the second luminance picture of relative described the second brightness.Described controlled processing unit is in order to be identified as active light source image by the brightest area of relative position in described the first luminance picture and described the second luminance picture.
The present invention also provides a kind of pupil tracing device, comprises active light source, imageing sensor and controlled processing unit.Described active light source is luminous towards eyeball with the first brightness and the second brightness in turn.Described imageing sensor obtains the first luminance picture of relatively described the first brightness and the second luminance picture of relative described the second brightness.Described controlled processing unit is in order to calculate the difference image of described the first luminance picture and described the second luminance picture and dark areas in described difference image is identified as to active light source image.
In one embodiment, described controlled processing unit is according to described the first luminance picture or described the second luminance picture identification pupil position; Wherein, when described the first brightness is during higher than described the second brightness, described controlled processing unit encloses GTG value and is identified as pupil region between multiple pixels of GTG value scope on weekly duty minimum gray scale in described the first luminance picture, and calculates the center of gravity of described pupil region or center using as described pupil position.
In one embodiment, pupil tracing device separately comprises storage unit in order to store the pupil coordinate information that described active light source image is relevant to described pupil position.Described controlled processing unit can calculate current pupil coordinate according to described pupil coordinate information.
In one embodiment, described the first luminance picture and described the second luminance picture can be realized by the time shutter and/or the circuit gain that change described imageing sensor, might not change the brightness of described active light source.
The present invention also provides a kind of pupil tracing device, and this device comprises at least two active light sources, an imageing sensor and a controlled processing unit.Described at least two active light sources are luminous with illumination eyeball in turn.Described imageing sensor obtains the first picture frame and the second picture frame that relatively different described active light sources is lighted.Described controlled processing unit is in order to calculate the difference image of described the first picture frame and described the second picture frame and dark areas and brightest area in described difference image are identified as to active light source image.
In one embodiment, described controlled processing unit is also identified the first pupil region and is identified the second pupil region according to described the second picture frame according to described the first picture frame; And be output pupil region by the connection collection zone definitions of described the first pupil region and described the second pupil region.
In one embodiment, described pupil tracing device comprises four active light sources, and described controlled processing unit is identified in described difference image four active light source images and calculated respectively the elements of a fix of relatively described four active light source images.The eyeball center that described controlled processing unit can calculate described eyeball according to described four elements of a fix.
In one embodiment, described controlled processing unit is also exported pupil region according to described the first picture frame and described the second picture frame identification, and the line direction of described eyeball center and described output pupil region is defined as to direction of visual lines.
In the pupil tracing device of the embodiment of the present invention, described active light source image refers to the reflected image of described active light source on cornea (Cornea).
In the pupil tracing device of the embodiment of the present invention, described active light source image is the positioning datum point that is used as pupil tracing, is arranged on wear-type accessory, to fix the position of active light source image described in described eyeball therefore pupil tracing device is better.
In the pupil tracing device of the embodiment of the present invention, the degree of accuracy of pupil tracing, to put as positioning datum and to pass through to calculate difference image to eliminate the impact of environment light source, can effectively be improved in the position by correct identification active light source image.
Accompanying drawing explanation
Fig. 1 shows the schematic diagram of eye image;
Fig. 2 A shows the block schematic diagram of the pupil tracing device of the embodiment of the present invention.
Fig. 2 B shows in the pupil tracing device of the embodiment of the present invention, the schematic diagram of pupil coordinate information;
Fig. 3 A shows the running schematic diagram of the pupil tracing device of one embodiment of the invention;
Fig. 3 B shows the Image Acquisition of pupil tracing device and the schematic diagram of light source igniting of one embodiment of the invention;
Fig. 3 C shows in the pupil tracing device of one embodiment of the invention, the schematic diagram of identification active light source image;
Fig. 3 D shows the Image Acquisition of pupil tracing device and the schematic diagram of light source igniting of another embodiment of the present invention;
Fig. 4 shows in the pupil tracing device of the embodiment of the present invention, the schematic diagram of identification pupil position;
Fig. 5 A shows the running schematic diagram of the pupil tracing device of another embodiment of the present invention;
Fig. 5 B shows the Image Acquisition of pupil tracing device and the schematic diagram of light source igniting of another embodiment of the present invention;
Fig. 5 C shows in the pupil tracing device of another embodiment of the present invention, the schematic diagram of identification active light source image.
Description of reference numerals
1 pupil tracing device 111-114 active light source
12 imageing sensor 13 controlled processing units
14 storage unit 2 wear-type accessories
3 image display device 9 eye images
90 eyeballs 91,91' pupil image
92,92' iris image 93,93' cornea part
9
1-9
4eye image LS
111-LS
114reflection light point
F
b1, f
b2luminance picture f1, f2 picture frame
F
b1-f
b2difference image f1-f2 difference image
I
111-I
114active light source image I
111'-I
114' active light source image
P
1minimum gray scale value P
2pupil position
Rg GTG value scope PA pupil region
A
111, A
111' brightest area I
ppupil image
Io environment light source image C
1-C
4preset reference point
I
c1-I
c4pupil image
distance.
Embodiment
In order to allow above and other object of the present invention, feature and the advantage can be more obvious, below will coordinate appended diagram, be described in detail below.In explanation of the present invention, identical member is to represent with identical symbol, should illustrate in advance at this.
Please refer to shown in Fig. 2 A, the block schematic diagram that it shows the pupil tracing device 1 of the embodiment of the present invention, comprises at least one active light source (being for example now shown as four active light source 111-114), imageing sensor 12, controlled processing unit 13 and storage unit 14.The pupil tracing device 1 of the present embodiment mainly utilizes described active light source on eyeball 90, to be projected to a few active light source image using as positioning datum point, and according to the relativeness decision pupil coordinate of pupil position and described positioning datum point (being active light source image); Wherein, described active light source image refers to the reflected image of described active light source on cornea.Described pupil tracing device 1 for example can be by wired or wireless described pupil coordinate image display device 3(Fig. 2 B that is sent to) carry out corresponding function relatively to control described image display device 3.Should be noted that, using the object of four active light sources is the eyeball centers of calculating, if but only carry out pupil tracing, can use the active light source of any amount.
Generally speaking, in the time that described eyeball 90 is looked down, upper eyelid meeting shaded portions eyeball 90, if therefore described pupil tracing device 1 is arranged on wear-type accessory 2, the setting position of described imageing sensor 12 is better for described eyeball 90, causes detecting the situation of pupil when avoiding eyeball 90 to look down (pupil down).
Described active light source 111-114 for example can be infrared light light source, and for example infrared light light-emittingdiode affects sight line when avoiding lighting.Described active light source 111-114 in order to towards described eyeball 90 luminous with throw light on described eyeball 90 and on described eyeball 90 form reflection light points (being presented in the active light source image on described eyeball 90).Should be noted that, each active light source can only comprise single light source element or be arranged and formed by multiple light sources.
Described imageing sensor 12 for example can be the optical sensor in order to sense light energy such as ccd image sensor, cmos image sensor, and it is with the picture frame that obtains described eyeball 90 of lighting of the relatively described active light source 111-114 of frame per second (frame rate).Described imageing sensor 12 can be fixed or variable time shutter and circuit gain obtains picture frame.Should be noted that, the frame per second of described imageing sensor 12 can be not equal to the glow frequency of described active light source 111-114.
Described controlled processing unit 13 is in order to control lighting and the Image Acquisition of described imageing sensor 12 of described active light source 111-114, and the picture frame obtaining according to described imageing sensor 12 identification pupil position, pupil coordinate, active light source image, eyeball center and direction of visual lines etc. (after being specified in).In other embodiment, described controlled processing unit 13 also can be formed by multiple unit, for example with active light source 111-114 described in light source control unit control independently and independently processing unit (DSP) carry out post processing of image, there is no specific limited.
Described storage unit 14 is in order to store the pupil coordinate information that described active light source image is relevant to described pupil position.Whereby, try to achieve the current relativeness of current pupil position and active light source image when described controlled processing unit 13, can try to achieve current pupil coordinate according to pre-stored described pupil coordinate information.For example, with reference to shown in Fig. 2 B, the pupil tracing device 1 of the embodiment of the present invention for example can comprise setting program (setup procedure), and it just enters can automatically enter before each operation or accept to indicate time.In described setting program, user can watch preset reference point successively attentively, such as, but not limited to, four corner C that watch successively image display device 3 attentively
1-C
4, now 12 of described imageing sensors obtain four eye images 9 successively
1-9
4, each eye image 9
1-9
4comprise respectively pupil image I
c1-I
c4with active light source image I
111; Wherein, described active light source image I
111in four eye images 9
1-9
4in be all positioned at identical (relatively) position.For example,, when user watches reference Point C attentively
1time, described imageing sensor 12 obtains described eye image 9
1; Wherein, described pupil image I
c1to described active light source image I
111vector be assumed to be
; In like manner, described controlled processing unit 13 can be obtained respectively described eye image 9
2-9
4in vector
1, pupil tracing device can be according to described vector
defining operation space is also pre-stored within described storage unit 14.Whereby, described in the time of practical operation, 13 of controlled processing units can calculate current pupil coordinate according to described operating space.Should be noted that vector described in Fig. 2 B
direction also may be contrary.
Please refer to shown in Fig. 3 A, it shows the running schematic diagram of the pupil tracing device 1 of one embodiment of the invention.Described pupil tracing device 1 comprises active light source 111, imageing sensor 12, controlled processing unit 13 and storage unit 14.Storage unit 14 is used to store the pupil coordinate information that active light source image is relevant to pupil position as previously mentioned, therefore repeat no more in this.
Shown in Fig. 3 A to 3D, Fig. 3 B shows in the pupil tracing device of Fig. 3 A, the schematic diagram of the light source igniting of the Image Acquisition of described imageing sensor 12 and described active light source 111; Fig. 3 C shows in the pupil tracing device of Fig. 3 A, described controlled processing unit 13 is identified the schematic diagram of active light source image I 111, Fig. 3 D shows in the pupil tracing device of Fig. 3 A, another schematic diagram of the light source igniting of the Image Acquisition of described imageing sensor 12 and described active light source 111.
Described active light source 111 is luminous towards eyeball 90 with the first brightness and the second brightness in turn.Described imageing sensor 12 obtains the first luminance picture f of relatively described the first brightness
b1the second luminance picture f of relatively described the second brightness
b2.For example, in Fig. 3 C, described the first luminance picture f
b1comprise brightest area A
111(being active light source image), pupil (Pupil) image 91, iris (Iris) image 92 and cornea (Cornea) part 93; Described the second luminance picture f
b2comprise brightest area A
111' (being active light source image), pupil image 91', iris image 92' and cornea part 93'.Due to better being arranged on wear-type accessory 2 of pupil tracing device 1 of the embodiment of the present invention, so that described active light source 111 fixes with the relative position of described eyeball 90, therefore described brightest area A
111, A
111although ' be arranged in different luminance pictures, be positioned at relatively (identical) position, and described active light source image is the direct reflection spot of described active light source 111 on cornea, be brightest area in image.Therefore, described controlled processing unit 13 can be by described the first luminance picture f
b1with described the second luminance picture f
b2the brightest area of middle relative position (is A
111, A
111') be identified as active light source image I
111; Wherein, described brightest area refers to that GTG value is the highest.Whereby, even if include other environment light source images in the luminance picture that described imageing sensor 12 obtains, but because the spatial relationship between described eyeball 90 and environment light source is unfixing, described in different luminance pictures, environment light source image can not be positioned at same position, therefore find the impact that the brightest area of relative position can be got rid of environment light source whereby.In other embodiment, described imageing sensor 12 can additionally obtain not luminescent image (the 3rd luminance picture) f of a described active light source 111
b3, as shown in Figure 3 D, and by described the first luminance picture f
b1with described the second luminance picture f
b2deduct respectively described the 3rd luminance picture and obtain f
b13with f
b23filtering environmental light disturbs the also active light source image of stable detection brightest area whereby.
In addition, for example, except changing the brightness (changing the size of drive current) of described active light source 111, also can fix the brightness of described active light source 111 and the sampling parameter (for example comprising time shutter and circuit gain etc.) that changes described imageing sensor 12 reaches identical object.Also, in another embodiment, described the first brightness can equal described the second brightness, and described imageing sensor 12 obtains described the first luminance picture f with the first time shutter and the first circuit gain
b1and obtain described the second luminance picture f with the second time shutter and second circuit gain
b2; Wherein, described the first time shutter is not equal to described the second time shutter and/or described the first circuit gain and is not equal to described second circuit gain.
In addition, controlled processing unit 13 described in the present embodiment separately can be according to described the first luminance picture f
b1or described the second luminance picture f
b2identification pupil position; Wherein, the better basis of described controlled processing unit 13 has the luminance picture of higher brightness and identifies described pupil position, so can identify more clearly pupil image 91.For example, with reference to shown in Fig. 4, described controlled processing unit 13 is in order to calculate described the first luminance picture f
b1minimum gray scale value P in (now supposing that described the first brightness is higher than described the second brightness)
1, and by described minimum gray scale value P
1around GTG value is identified as pupil region PA between multiple pixels of GTG value scope Rg.In the time that described active light source 11 is lighted, the brightness of described pupil image 91 is minimum, and the brightness of the highest and described iris image 92 of brightness in described cornea region 93 is conventionally between between described pupil image 91 and described cornea region 93, therefore described minimum gray scale value P
1there will be in described pupil image 91.Therefore, described minimum gray scale value P
1near pixel region can be regarded as described pupil region PA, and described minimum gray scale value P
1near pixel region for example can utilize the image mode of (image grouping) of hiving off that inter-related pixel region is regarded as to same object, for example can disclose No. 20110176733 to the United States Patent (USP) of common assignee of the present invention with reference to transference, title is disclosed in " image-recognizing method ".In addition, the setting of described GTG value scope Rg can be adjusted according to the operating environment of described pupil tracing device 1, and for example different GTG value scope Rg can be designed respectively from indoor in open air.Moreover for avoiding noise, described controlled processing unit 13 separately can judge whether to belong to environment light source image according to characteristics such as the size and dimensions of described pupil region PA, for example, when size is too little or the circular person of non-genus may be environment light source, can be got rid of.
Then, described controlled processing unit 13 can calculate the center of gravity of described pupil region PA or center using as pupil position P
2, and according to described pupil position P
2with described reference light source image I
111relative position export described pupil position P with reference to the pupil coordinate information of storing in described storage unit 14
2relevant pupil coordinate (x, y).Described controlled processing unit 13 for example can be controlled according to described pupil coordinate (x, y) action of the vernier of image display device 3 relatively.Scrutable, described pupil position P2 can be not equal to described minimum gray scale value P
1position.
Generally speaking, in the present embodiment, described controlled processing unit 13 is identified described active light source image I according to two picture frames
111, but only for example, identify described pupil position with one (brightness the higher person), get rid of whereby environment light source impact and described active light source image and can be used as the positioning datum point of pupil tracing.
In another embodiment, in order further to eliminate the impact of environment light source, described controlled processing unit 13 can further calculate described the first luminance picture f
b1with described the second luminance picture f
b2difference image (f
b1-f
b2), and by described difference image (f
b1-f
b2) in dark areas by with described the first luminance picture f
b1or described the second luminance picture f
b2relatively, to identify active light source image I
111" (as Fig. 3 C); Wherein, described dark areas refers to that GTG value is minimum, and it for example can be 0 GTG left and right, but it can be also relative value.
Specifically, due to described the first luminance picture f
b1with the second luminance picture f
b2described in active light source image I
111(be brightest area A
111and A
111') all there is maximum brightness (for example using the high-order of GTG value scope), therefore described difference image (f
b1-f
b2) described in active light source image I
111can there is minimum brightness.In one embodiment, suppose described the first image f
b1in, active light source image I
111be 255 GTGs, pupil image 91 is 0 GTG, and iris image 92 is 128 GTGs and cornea part 93 is 228 GTGs; Described the second image f
b2in, active light source image I
111' be 255 GTGs, pupil image 91' is 0 GTG, iris image 92' is 64 GTGs and cornea part 93' is 100 GTGs; So can try to achieve described difference image (f
b1-f
b2) in, active light source image I
111" being 0 GTG, pupil image 91 " is 0 GTG, and iris image 92 " be 64 GTGs and cornea part 93 " is 128 GTGs.Because the GTG value of described pupil image 91,91' is 0 GTG, therefore described difference image (f
b1-f
b2) described in pupil image 91 " can there is equally minimum brightness.Now, can pass through more described difference image (f
b1-f
b2) and described the first luminance picture f
b1or described the second luminance picture f
b2, just can identify described active light source image I
111" the position of (as Fig. 3 C); Wherein, mode relatively for example, due to described the first luminance picture f
b1or described the second luminance picture f
b2in described pupil image 91,91' all there is minimum brightness, therefore just can directly pass through more described the first luminance picture f
b1with described difference image (f
b1-f
b2) which image object is that to change to antipode by relative incandescent dark in two images, just can determine accordingly described active light source image I
111" the position of (as Fig. 3 C); Or, by more described the second luminance picture f
b2with described difference image (f
b1-f
b2) which image object is that to change to antipode by relative incandescent dark in two images, just can determine accordingly described active light source image I
111" the position of (as Fig. 3 C).In another embodiment, can be by described difference image (f
b1-f
b2) in there is minimum brightness and image object area smaller and be identified as described active light source image and will there is minimum brightness and image object area the greater is identified as pupil image.
In addition, in the present embodiment except described active light source image I
111calculating be to utilize difference image (f
b1-f
b2) carry out outside, other comprise identification pupil position, set up pupil coordinate information and realize described the first luminance picture f
b1with described the second luminance picture f
b2all identical with last embodiment etc. mode, therefore repeat no more in this.
Please refer to shown in Fig. 5 A, it shows the running schematic diagram of the pupil tracing device 1 of another embodiment of the present invention.Described pupil tracing device 1 comprises four active light source 111-114, an imageing sensor 12, a controlled processing unit 13 and a storage unit 14.Described storage unit 14 is to store the pupil coordinate information that active light source image is relevant to pupil position as previously mentioned, therefore repeat no more in this.
Shown in Fig. 5 A to 5C, Fig. 5 B shows in the pupil tracing device of Fig. 5 A, the schematic diagram of the light source igniting of the Image Acquisition of described imageing sensor 12 and described active light source 111-114; Fig. 5 C shows that in the pupil tracing device of Fig. 5 A, described controlled processing unit 13 is identified the schematic diagram of active light source image.As previously mentioned, using four active light sources is to calculate eyeball center, if only carry out pupil tracing, can use at least two active light sources can reach the object of the present embodiment.
Described active light source 111-114 is luminous with illumination eyeball 90, for form reflection light point LS on described eyeball 90 in turn
111-LS
114.Described imageing sensor 112 obtains the first picture frame f1 and the second picture frame f2 that relatively different described active light source 111-114 lights.Described controlled processing unit 13 is in order to calculate the difference image (f1-f2) of described the first picture frame f1 and described the second picture frame f2 and for example, by dark areas (I in described difference image (f1-f2)
113' and I
114') and brightest area (for example I
111' and I
112') be identified as active light source image.
For example, in Fig. 5 C, when described active light source 111 and 112 is lighted (now described active light source 113 and 114 extinguishes) described in imageing sensor 12 obtain the first picture frame f1, it comprises pupil image 91, iris image 92, cornea part 93 and reference light source image I
111and I
112; Described in when described active light source 113 and 114 is lighted (now described active light source 111 and 112 extinguishes), imageing sensor 12 obtains the second picture frame f2, and it comprises pupil image 91', iris image 92', cornea part 93' and reference light source image I
113and I
114; Wherein, the light-source brightness of relatively described the first picture frame f1 can be identical or different with the light-source brightness of relative described the second picture frame f2, there is no specific limited.In addition, in other embodiment, relatively described the second picture frame f2 of one (for example or three) that relatively described the first picture frame f1 can light in described active light source 111-114 at least wherein lights other active light sources (for example three or), is not limited to light two active light sources.In the time that described the first picture frame f1 deducts described the second picture frame f2, reference light source image I described in described difference image (f1-f2)
111and I
112become brightest area and there is high gray value and described reference light source image I
113and I
114become dark areas and there is minimum gray scale value; Whereby, described controlled processing unit 13 can identify four active light source image I in described difference image (f1-f2)
111'-I
114'.In one embodiment, suppose in described the first picture frame f1 active light source image I
111and I
112be approximately+255 GTGs, pupil image 91 is approximately 0 GTG, and iris image 92 is approximately 64 GTGs and cornea part 93 is approximately 128 GTGs; In described the second picture frame f2, active light source image I
113and I
114be approximately+255 GTGs, pupil image 91' is approximately 0 GTG, and iris image 92' is approximately 64 GTGs and cornea part 93' is approximately 128 GTGs; So can try to achieve active light source image I in described difference image (f1-f2)
111' and I
112' be approximately+127 GTGs active light source image I
113' and I
114' being approximately-127 GTGs, other parts are approximately 0 GTG; In another embodiment, overall GTG value can be adjusted and made whole GTG values all be more than or equal to zero, for example active light source image I
111' and I
112' approximately can be adjusted to+254 GTGs active light source image I
113' and I
114' approximately can be adjusted to 0 GTG, approximately can be adjusted to+127 GTGs of other parts.Should be noted that, it is only exemplary that above-mentioned GTG is worth numerical value, not in order to limit the present invention.
Because the present embodiment uses four active light source 111-114, described controlled processing unit 13 identifies four active light source image I in described difference image (f1-f2)
111'-I
114' after, can calculate respectively relatively described four active light source image I
111'-I
114' the elements of a fix, and each elements of a fix all can be three-dimensional coordinate.The 13 eyeball centers that can calculate described eyeball 90 according to described four elements of a fix of described controlled processing unit, for example, utilize spheroid equation (x-x
0)
2+ (y-y
0)
2+ (z-z
0)
2=r
2calculate.Whereby, described controlled processing unit 13 can and then be defined as direction of visual lines by described eyeball center and the line direction of output pupil region.
In the present embodiment, the deciding means of described output pupil region is as follows: first, described controlled processing unit 13 is identified the first pupil region and identified the second pupil region according to described the second picture frame f2 according to described the first picture frame f1.The mode of identification pupil region is similar to Figure 4, is also that described controlled processing unit 13 for example, by minimum gray scale value (P in described the first picture frame f1
1) around GTG value be identified as described the first pupil region (for example PA) between multiple pixels of GTG value scope Rg; Described control for example processes single 13, by minimum gray scale value (P in described the second picture frame f2
1) around GTG value be identified as described the second pupil region (for example PA) between multiple pixels of described GTG value scope Rg.After described the first pupil region and described the second pupil region identify, 13 of described controlled processing units calculate the center of gravity of described the first pupil region or center using for example, as described the first pupil position (P
2) and calculate the center of gravity of described the second pupil region or center using for example, as described the second pupil position (P
2).
Because two picture frames of the present embodiment all can be obtained pupil image, therefore described controlled processing unit 13 is using the connection collection region of described the first pupil region and described the second pupil region as output pupil region.In like manner, described controlled processing unit 13 can determine output pupil coordinate according to described output pupil region and the pupil coordinate information being pre-stored within described storage unit 14, just comprises four active light source images herein as positioning datum point.In other embodiment, described controlled processing unit 13 also can be by one wherein in described the first pupil region and described the second pupil region as described output pupil region.
In addition, the pupil tracing device 1 of various embodiments of the present invention can be arranged in pairs or groups display unit to show image, and described display unit also can be arranged on described wear-type accessory 2, for example glasses or safety goggles.
The pupil tracing device 1 of various embodiments of the present invention can also have blink detection function.For example described controlled processing unit 13 can record respectively detect with can't detect pupil during, identification nictation action whereby.
The pupil tracing device 1 of various embodiments of the present invention can also have sleepy detection and the measuring ability of diverting one's attention.For example, in the time that described pupil tracing device 1 is applied to Vehicular device, can whether there is sleepiness or be absorbed in front in order to detect to drive, and propose warning in appropriate time.Detection sleepiness for example can be opened the ratio of entire time by detection eyes and realize; The direction of visual lines that detection diverts one's attention for example can drive by detection is realized.
The pupil tracing device 1 of various embodiments of the present invention can also have frequency of wink and dry eyes measuring ability.Particularly, described controlled processing unit 13 can instead according to detected frequency of wink push away possibility and the degree that current eyes are dry and astringent, and then reminds user's nictation.
The pupil tracing device 1 of various embodiments of the present invention can also have gesture recognition (gesture recognition) function.Now described gesture recognition for example can be identification pupil change position towards predetermined direction with pre-determined number, and itself and default posture are compared, can reach whereby the running of carrying out specific function, its similar known pupil other objects in addition that utilize, the gesture identification that for example hand or finger carry out.
The pupil tracing device 1 of various embodiments of the present invention can also have electricity-saving function, and the image change amount that for example can't detect pupil or picture frame in Preset Time is too small, can enter battery saving mode.
Scrutable, in above-mentioned each diagram, the ratio of active light source image and pupil image is only exemplary.
Should be noted that, the pupil tracing device 1 of various embodiments of the present invention can directly be made into wear-type pupil tracing device, or be fixed on wear-type accessory by binding member combination, the various wearings such as such as glasses, safety goggles or cap, sheathed or be set up in the accessory of head, better active light source and the human eye of making can not relatively move, and described active light source image can be used as the positioning datum point of following the trail of pupil.
In sum, known pupil tracing device is because getting rid of the impact of environment light source, thereby may occur detecting wrong situation.The present invention separately proposes a kind of pupil tracing device (Fig. 2 A, 3A and 5B), and it utilizes the fixing active light source of relative eyeball position as positioning datum point, and the impact that can eliminate environment light source detects degree of accuracy to improve.
Although the present invention is by disclosing with previous embodiment, it is not in order to limit the present invention, any technician in the technical field of the invention with common knowledge, without departing from the spirit and scope of the present invention, when doing various changes and modification.Therefore the scope that protection scope of the present invention ought define depending on accompanying claim scope is as the criterion.
Claims (20)
1. a pupil tracing device, this pupil tracing device comprises:
Active light source, luminous towards eyeball with the first brightness and the second brightness in turn;
Imageing sensor, obtains the first luminance picture of relatively described the first brightness and the second luminance picture of relative described the second brightness; And
Controlled processing unit, in order to be identified as active light source image by the brightest area of relative position in described the first luminance picture and described the second luminance picture.
2. pupil tracing device according to claim 1, wherein said controlled processing unit is also according to described the first luminance picture or described the second luminance picture identification pupil position.
3. pupil tracing device according to claim 2, wherein said the first brightness is higher than described the second brightness, described controlled processing unit is identified as pupil region by GTG value of enclosing on weekly duty minimum gray scale in described the first luminance picture between multiple pixels of GTG value scope, and calculates the center of gravity of described pupil region or center using as described pupil position.
4. pupil tracing device according to claim 2, this pupil tracing device also comprises storage unit, in order to store the pupil coordinate information that described active light source image is relevant to described pupil position.
5. pupil tracing device according to claim 1, wherein said the first brightness equals described the second brightness, and described imageing sensor obtains described the first luminance picture and obtains described the second luminance picture with the second time shutter and second circuit gain with the first time shutter and the first circuit gain.
6. according to the pupil tracing device described in any one claim in claim 1-5, this pupil tracing device is arranged on wear-type accessory.
7. a pupil tracing device, this pupil tracing device comprises:
Active light source, luminous towards eyeball with the first brightness and the second brightness in turn;
Imageing sensor, obtains the first luminance picture of relatively described the first brightness and the second luminance picture of relative described the second brightness; And
Controlled processing unit, in order to calculate the difference image of described the first luminance picture and described the second luminance picture, and is identified as active light source image by the dark areas in described difference image.
8. pupil tracing device according to claim 7, wherein said controlled processing unit is also according to described the first luminance picture or described the second luminance picture identification pupil position.
9. pupil tracing device according to claim 8, wherein said the first brightness is higher than described the second brightness, described controlled processing unit is identified as pupil region by GTG value of enclosing on weekly duty minimum gray scale in described the first luminance picture between multiple pixels of GTG value scope, and calculates the center of gravity of described pupil region or center using as described pupil position.
10. pupil tracing device according to claim 8, this pupil tracing device also comprises storage unit, and this storage unit is in order to store the pupil coordinate information that described active light source image is relevant to described pupil position.
11. pupil tracing devices according to claim 7, wherein said the first brightness equals described the second brightness, and described imageing sensor obtains described the first luminance picture and obtains described the second luminance picture with the second time shutter and second circuit gain with the first time shutter and the first circuit gain.
12. according to the pupil tracing device described in any one claim in claim 7-11, and this pupil tracing device is arranged on wear-type accessory.
13. 1 kinds of pupil tracing devices, this pupil tracing device comprises:
At least two active light sources, luminous with illumination eyeball in turn;
Imageing sensor, obtains the first picture frame and the second picture frame that relatively different described active light sources is lighted; And
Controlled processing unit, in order to calculate the difference image of described the first picture frame and described the second picture frame, and is identified as active light source image by dark areas and brightest area in described difference image.
14. pupil tracing devices according to claim 13, wherein said controlled processing unit is also identified the first pupil region and is identified the second pupil region according to described the second picture frame according to described the first picture frame.
15. pupil tracing devices according to claim 14, wherein said controlled processing unit is identified as GTG value of enclosing on weekly duty minimum gray scale in described the first picture frame described the first pupil region and GTG value of enclosing on weekly duty minimum gray scale in described the second picture frame is identified as to described the second pupil region between multiple pixels of described GTG value scope between multiple pixels of GTG value scope; Described controlled processing unit calculate the center of gravity of described the first pupil region or center using as described the first pupil position and calculate the center of gravity of described the second pupil region or center using as described the second pupil position.
16. pupil tracing devices according to claim 14, described controlled processing unit is also output pupil region by the connection collection zone definitions of described the first pupil region and described the second pupil region.
17. pupil tracing devices according to claim 13, this pupil tracing device comprises four active light sources, and described controlled processing unit is identified four active light source images in described difference image and calculated respectively the elements of a fix of relatively described four active light source images.
18. pupil tracing devices according to claim 17, the eyeball center that wherein said controlled processing unit also calculates described eyeball according to described four elements of a fix.
19. pupil tracing devices according to claim 18, wherein said controlled processing unit is also exported pupil region according to described the first picture frame and described the second picture frame identification, and the line direction of described eyeball center and described output pupil region is defined as to direction of visual lines.
20. according to the pupil tracing device described in any one claim in claim 13-19, and this pupil tracing device is arranged on wear-type accessory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210428613.6A CN103793045B (en) | 2012-10-31 | 2012-10-31 | Pupil tracing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210428613.6A CN103793045B (en) | 2012-10-31 | 2012-10-31 | Pupil tracing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103793045A true CN103793045A (en) | 2014-05-14 |
| CN103793045B CN103793045B (en) | 2016-12-28 |
Family
ID=50668808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210428613.6A Active CN103793045B (en) | 2012-10-31 | 2012-10-31 | Pupil tracing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103793045B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607252A (en) * | 2015-09-23 | 2016-05-25 | 宇龙计算机通信科技(深圳)有限公司 | Control method of intelligent glasses and control device thereof |
| CN106504271A (en) * | 2015-09-07 | 2017-03-15 | 三星电子株式会社 | Method and apparatus for eye tracking |
| CN107991775A (en) * | 2016-10-26 | 2018-05-04 | 中国科学院深圳先进技术研究院 | It can carry out the wear-type visual device and human eye method for tracing of people's ocular pursuit |
| CN110633685A (en) * | 2019-09-20 | 2019-12-31 | Oppo广东移动通信有限公司 | Human eye detection method and related product |
| CN111616800A (en) * | 2020-06-09 | 2020-09-04 | 电子科技大学 | Ophthalmic surgery navigation system |
| JP7273720B2 (en) | 2017-03-21 | 2023-05-15 | マジック リープ， インコーポレイテッド | Method and system for eye movement tracking in conjunction with an optical scanning projector |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110051319A (en) * | 2019-04-23 | 2019-07-26 | 七鑫易维(深圳)科技有限公司 | Adjusting method, device, equipment and the storage medium of eyeball tracking sensor |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344919A (en) * | 2008-08-05 | 2009-01-14 | 华南理工大学 | Sight tracing method and disabled assisting system using the same |
| CN101788848A (en) * | 2009-09-29 | 2010-07-28 | 北京科技大学 | Eye characteristic parameter detecting method for sight line tracking system |
| US20110211056A1 (en) * | 2010-03-01 | 2011-09-01 | Eye-Com Corporation | Systems and methods for spatially controlled scene illumination |
-
2012
- 2012-10-31 CN CN201210428613.6A patent/CN103793045B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344919A (en) * | 2008-08-05 | 2009-01-14 | 华南理工大学 | Sight tracing method and disabled assisting system using the same |
| CN101788848A (en) * | 2009-09-29 | 2010-07-28 | 北京科技大学 | Eye characteristic parameter detecting method for sight line tracking system |
| US20110211056A1 (en) * | 2010-03-01 | 2011-09-01 | Eye-Com Corporation | Systems and methods for spatially controlled scene illumination |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106504271A (en) * | 2015-09-07 | 2017-03-15 | 三星电子株式会社 | Method and apparatus for eye tracking |
| CN105607252A (en) * | 2015-09-23 | 2016-05-25 | 宇龙计算机通信科技(深圳)有限公司 | Control method of intelligent glasses and control device thereof |
| CN107991775A (en) * | 2016-10-26 | 2018-05-04 | 中国科学院深圳先进技术研究院 | It can carry out the wear-type visual device and human eye method for tracing of people's ocular pursuit |
| JP7273720B2 (en) | 2017-03-21 | 2023-05-15 | マジック リープ， インコーポレイテッド | Method and system for eye movement tracking in conjunction with an optical scanning projector |
| US11838496B2 (en) | 2017-03-21 | 2023-12-05 | Magic Leap, Inc. | Method and system for tracking eye movement in conjunction with a light scanning projector |
| CN110633685A (en) * | 2019-09-20 | 2019-12-31 | Oppo广东移动通信有限公司 | Human eye detection method and related product |
| CN110633685B (en) * | 2019-09-20 | 2022-10-25 | Oppo广东移动通信有限公司 | Human eye detection method and related product |
| CN111616800A (en) * | 2020-06-09 | 2020-09-04 | 电子科技大学 | Ophthalmic surgery navigation system |
| CN111616800B (en) * | 2020-06-09 | 2023-06-09 | 电子科技大学 | Ophthalmic surgery navigation system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103793045B (en) | 2016-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI471808B (en) | Pupil detection device | |
| US10747995B2 (en) | Pupil tracking device | |
| US11344196B2 (en) | Portable eye tracking device | |
| CN103793045A (en) | Pupil tracking device | |
| JP6308940B2 (en) | System and method for identifying eye tracking scene reference position | |
| US10310597B2 (en) | Portable eye tracking device | |
| RU2729045C2 (en) | Adaptive lighting system for mirror component and method for controlling adaptive lighting system | |
| JP4783018B2 (en) | Method and apparatus for detecting and tracking the eye and its gaze direction | |
| US20130201285A1 (en) | 3-d glasses with illuminated light guide | |
| US10503248B1 (en) | Selective color sensing for motion tracking | |
| EP2609731B1 (en) | Tracking head position and orientation | |
| WO2022095605A1 (en) | Handle control tracker | |
| EP3563217A1 (en) | Portable eye tracking device | |
| CN103565399B (en) | Pupil detector | |
| JP6687195B2 (en) | Eye image processing device | |
| US10928894B2 (en) | Eye tracking | |
| JPH06323832A (en) | Interface for vehicle | |
| JP2016049258A (en) | Lighting and imaging device and visual axis detecting apparatus including the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |