CN109522887A - A kind of Eye-controlling focus method, apparatus, equipment and storage medium - Google Patents
A kind of Eye-controlling focus method, apparatus, equipment and storage medium Download PDFInfo
- Publication number
- CN109522887A CN109522887A CN201910068612.7A CN201910068612A CN109522887A CN 109522887 A CN109522887 A CN 109522887A CN 201910068612 A CN201910068612 A CN 201910068612A CN 109522887 A CN109522887 A CN 109522887A
- Authority
- CN
- China
- Prior art keywords
- pixel
- thermal imaging
- infrared thermal
- pupil
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/143—Sensing or illuminating at different wavelengths
Abstract
The embodiment of the invention discloses a kind of Eye-controlling focus method, apparatus, equipment and storage mediums.This method comprises: obtaining infrared thermal imaging figure when user watches attentively；Corresponding pupil pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure；According to the information of the pupil pixel, information is watched in determination attentively.It solves by adopting the above technical scheme and needs to be arranged the technical issues of infrared light supply carries out Eye-controlling focus in the prior art, hardware quantity needed for reducing Eye-controlling focus module reduces the space hold amount of hardware.
Description
Technical field
The present embodiments relate to field of image processing more particularly to a kind of Eye-controlling focus method, apparatus, equipment and storage
Medium.
Background technique
Eye Tracking Technique, also referred to as eyeball tracking technology mainly estimate the sight of eyes by measurement eye movement data
The blinkpunkt position and/or.With the continuous development of science and technology, Eye Tracking Technique be widely applied to human-computer interaction, psychology and
The fields such as behavioral study, medical research and driver training and evaluation.
In the prior art, usual pupil-corneal reflection method used in the prior art is admired by obtaining pupil and Poole
The coordinate mapping relations between vector and screen that spot is formed generate calibrating parameters, then calculate the blinkpunkt coordinate on screen.
However, needing to be pointed into eyes using at least one infrared light supply using the above method, Poole is formed in eye after corneal reflection
Spot by the emperor himself, so that hardware quantity needed for Eye-controlling focus module is more, hardware occupied space is big.
Summary of the invention
The present invention provides a kind of Eye-controlling focus method, apparatus, equipment and storage medium, to reduce needed for Eye-controlling focus module
Hardware quantity reduces hardware occupied space.
In a first aspect, the embodiment of the invention provides a kind of Eye-controlling focus methods, this method comprises:
Obtain infrared thermal imaging figure when user watches attentively；
Corresponding pupil pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure；
According to the information of the pupil pixel, information is watched in determination attentively.
Second aspect, the embodiment of the invention also provides a kind of Eye-controlling focus device, which includes:
Thermograph obtains module, for obtaining infrared thermal imaging figure when user watches attentively；
Pupil pixel determining module, for the temperature value according to each pixel in the infrared thermal imaging figure, determining pair
The pupil pixel answered；
Watch information determination module attentively, for the information according to the pupil pixel, information is watched in determination attentively.
The third aspect, the embodiment of the invention also provides a kind of terminal device, which includes input unit, further includes:
One or more processors；
Storage device, for storing one or more programs；
One or more of programs are executed by one or more of processors, so that one or more of processors
Realize a kind of Eye-controlling focus method provided by any embodiment of the present invention.
Fourth aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, are stored thereon with computer
Program realizes a kind of Eye-controlling focus method provided by any embodiment of the present invention when the program is executed by processor.
Infrared thermal imaging figure when the embodiment of the present invention is watched attentively by obtaining user；According to each picture in the infrared thermal imaging figure
The temperature value of vegetarian refreshments determines corresponding pupil pixel；And according to the information of pupil pixel, information is watched in determination attentively.Using above-mentioned
Technical solution, which solves, needs to be arranged in the prior art the technical issues of infrared light supply carries out Eye-controlling focus, reduces Eye-controlling focus
Hardware quantity needed for module reduces the space hold amount of hardware.
Detailed description of the invention
Fig. 1 is the flow diagram of one of embodiment of the present invention one Eye-controlling focus method；
Fig. 2A is the flow diagram of one of embodiment of the present invention two Eye-controlling focus method；
Fig. 2 B is the flow diagram of one of embodiment of the present invention three Eye-controlling focus method；
Fig. 3 is the flow diagram of one of embodiment of the present invention four Eye-controlling focus method；
Fig. 4 is the structural schematic diagram of one of embodiment of the present invention five Eye-controlling focus device；
Fig. 5 is the hardware structural diagram of one of the embodiment of the present invention six terminal device.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
Embodiment one
Fig. 1 is a kind of flow diagram for Eye-controlling focus method that the embodiment of the present invention one provides, and the present embodiment is applicable
In to watch attentively user carry out Eye-controlling focus the case where, this method can be executed by Eye-controlling focus device, the device by software and/
Or hardware is realized, and is configured in terminal device.Wherein, terminal device can be the mobile terminals such as VR glasses, mobile phone or
The fixed terminals such as computer.The Eye-controlling focus method includes:
S110, infrared thermal imaging figure of user when watching attentively is obtained.
Wherein, infrared thermal imaging figure includes the image of the heat that user's ocular is radiated or temperature.Wherein, eye area
Domain may include pupil, iris, the white of the eye, upper eyelid and palpebra inferior.
In nature, all objects for being higher than absolute zero (- 273 DEG C) can infrared radiation.Therefore, in user
Can equally be had differences between the infra-red radiation heat of background due to itself when watching display plane attentively, obtain it is infrared heat at
As figure.Reflect the different Temperature Distribution of collected target surface using different colours in infrared thermal imaging figure.That is, red
Each pixel carries temperature information in outer thermograph.Usual target surface temperature is higher, then in infrared thermal imaging figure
Color it is brighter.
Illustratively, the pickup area of image collecting device can be set as facial area, watched attentively with directly acquiring user
When facial area corresponding to infrared thermal imaging figure.It is, of course, also possible to directly set the pickup area of image collecting device only
For eye areas, to reduce the data operation quantity when operation of subsequent determining pupil pixel, while identified pupil image is improved
The accuracy of vegetarian refreshments.
When user watches the display plane of terminal device attentively, it will the image collecting device timing or real of triggering terminal equipment
When the acquisition user infrared light that ocular is radiated when watching display plane attentively, form corresponding infrared thermal imaging figure；Or eventually
End equipment is by wirelessly obtaining the infrared thermal imaging figure acquired in image collecting device；Or terminal device by far call its
Infrared thermal imaging figure in his terminal；Or terminal device is obtained pre-stored from cloud platform storage system or is adopted through image
The infrared thermal imaging figure that acquisition means upload.
S120, corresponding pupil pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure.
Wherein, the pixel of pupil is belonged in the infrared thermal imaging figure of pupil pixel expression terminal device identification.
Terminal device determines pixel vertex type according to the temperature value of pixel each in infrared thermal imaging figure.Wherein, pixel
Vertex type includes pupil and non-pupil.The pixel that pixel vertex type is pupil is determined as pupil pixel by terminal device, and
It corresponds to using the coordinate value of each pupil pixel as location information.
Wherein, terminal device determines that the mode of pixel vertex type can according to the temperature value of pixel each in infrared thermal imaging figure
By but be not limited in a manner of following two:
Mode one: determine that the pixel for meeting preset temperature threshold is pupil pixel.It illustratively, can be by a large amount of
Known each pixel of infrared thermal imaging figure temperature value and corresponding pixel vertex type, determine pupil temperature range make
For preset temperature threshold；It can also be by technical staff rule of thumb or demand sets itself.
Mode two: divided by pixel vertex type of the pupil decision model to each pixel in infrared thermal imaging figure
Class, obtaining the pixel that pixel vertex type is pupil is pupil pixel.Illustratively, pupil decision model can be according to big
Linear discriminant model constructed by the temperature value and corresponding pixel vertex type of known each pixel of infrared thermal imaging figure of amount.Its
In, linear discriminant model can be the linear discriminants such as linear discriminant analysis, offset minimum binary, principal component analysis and supporting vector
The one or more of model.
It is understood that when the infrared heat that image collecting device infrared thermal imaging figure collected is user's face region
When image, in order to reduce data operation quantity when determining pupil pixel, while identified pupil pixel is improved just
True rate can also determine infrared thermal imaging eye figure previously according to the infrared thermal imaging figure of acquisition, it is subsequent directly according to infrared heat at
As the temperature value of each pixel in eye figure determines corresponding pupil pixel.
Optionally, it according to the infrared thermal imaging figure of acquisition, determines infrared thermal imaging eye figure, can be by around eyes
The mode of the figure of label setting shape draws a circle to approve infrared thermal imaging eye figure.Illustratively, can using fixed size rectangle into
The delineation of row infrared thermal imaging eye figure.
Optionally, it according to the infrared thermal imaging figure of acquisition, determines infrared thermal imaging eye figure, can also be by acquisition
Infrared thermal imaging eye figure is determined in infrared thermal imaging figure including eyes by way of eye recognition.
Optionally, it according to the infrared thermal imaging figure of acquisition, determines infrared thermal imaging eye figure, can also be by common face
The coordinate that portion's image carries out the pixel that human eye positions determining eye areas is target area；Obtain the mesh in infrared thermal imaging figure
Mark region obtains infrared thermal imaging eye figure.Wherein, common face-image is image collecting device when acquiring infrared thermal imaging figure,
Synchronize common face-image collected.
S130, according to the information of the pupil pixel, information is watched in determination attentively.
Wherein, watching information attentively is blinkpunkt coordinate when user watches attentively by watching area.It illustratively, can by watching area
To be aforementioned display plane.
According to the location information and the infrared thermal imaging figure and quilt of pupil pixel and/or pupil center's pixel
Mapping coefficient where watching area between plane, determines and watches attentively corresponding to pupil pixel and/or pupil center's pixel
Point coordinate.Wherein, pupil center's pixel can be calculated based on the position coordinates of pupil pixel.
Wherein, blinkpunkt can be understood as the sight vector of user and be watched attentively the intersection point of object, wherein by object is watched attentively
Including actual object, dummy object, display screen etc.；Then be by watching area a circle centered on blinkpunkt or
Other shapes region.Wherein, sight vector is to rotate an obtained taper at a certain angle centered on the pupil of user
Region.
Infrared thermal imaging figure when the embodiment of the present invention is watched attentively by obtaining user；According to each picture in the infrared thermal imaging figure
The temperature value of vegetarian refreshments determines corresponding pupil pixel；And according to the information of pupil pixel, information is watched in determination attentively.Using above-mentioned
Technical solution, which solves, needs to be arranged in the prior art the technical issues of infrared light supply carries out Eye-controlling focus, reduces Eye-controlling focus
Hardware quantity needed for module reduces the space hold amount of hardware.
Embodiment two
Fig. 2A is a kind of flow diagram of Eye-controlling focus method provided by Embodiment 2 of the present invention, and the present embodiment is above-mentioned
On the basis of the technical solution of each embodiment, step " is determined according to the temperature value of each pixel in the infrared thermal imaging figure
Corresponding pupil pixel " is refined as " traversing each pixel in the infrared thermal imaging figure, and obtaining the temperature of each pixel
Value；Based on each temperature value and pupil decision model is combined, the pixel for determining that pixel vertex type belongs to pupil is target pixel points；
Based on the coordinate value of each target pixel points, determine that the centre coordinate of the target pixel points is pupil center's pixel ", with perfect
The determination mechanism of pupil pixel.
Eye-controlling focus method as shown in Figure 2 A includes:
S210, infrared thermal imaging figure of user when watching attentively is obtained.
The step is identical as aforementioned S110, and details are not described herein.
Each pixel in S220, the traversal infrared thermal imaging figure, and obtain the temperature value of each pixel.
After terminal device obtains infrared thermal imaging figure, it is corresponding each pixel can be obtained using the traversal mode of direction initialization
Temperature value.Illustratively, the vertex that can determine the upper left corner in infrared thermal imaging figure first is starting point, the top in the lower right corner
Point is terminating point.Firstly, upper right traverses starting column (starting point column) down and obtains simultaneously since starting point column
The temperature value of each pixel；Then other temperature for arranging and obtaining each pixel simultaneously are successively traversed in the same fashion to the right
Value, until end is this time traversed after traversing the terminating point of end column (terminating point column) and having obtained corresponding temperature value
Journey.It is, of course, also possible to be traversed by other rules to each pixel in infrared thermal imaging figure, details are not described herein.
S230A, based on each temperature value and pupil decision model is combined, determines that pixel vertex type belongs to the pixel of pupil and is
Target pixel points.
According to the temperature value and corresponding pixel vertex type institute for being in advance based on each pixel of a large amount of known infrared thermal imaging figures
The linear discriminant model of building, determines the pixel classification of each temperature value of traversal, and pixel classification is belonged to each of pupil
Pixel is determined as target pixel points, while obtaining the pixel coordinate and temperature value of each target pixel points.Wherein, linear discriminant
Model can be one kind of the linear discriminant models such as linear discriminant analysis, offset minimum binary, principal component analysis and supporting vector
Or it is several.It should be noted that when the quantity of target pixel points is 1, it can be directly using determining target pixel points as pupil
Pixel.When the quantity of target pixel points is greater than 1, needs to choose a certain target pixel points or pass through each target pixel points
Setting rule, which calculates, determines pupil pixel.
S240, the coordinate value based on each target pixel points determine that the centre coordinate of the target pixel points is pupil center
Pixel.
Illustratively, the formed pattern of each pixel can be determined by the pixel coordinate of each target pixel points
Central pixel point is as pupil pixel.
Illustratively, the coordinate value of each target pixel points can also be determined using the temperature value of each target pixel points as weight
Weighted average, obtain the location information of pupil center's pixel.
Specifically, according to formulaDetermine the location information of pupil pixel；
Wherein, (X, Y) is the coordinate value of pupil center's pixel；(xi, yi) be the target pixel points coordinate
Value；IiFor the temperature value of the target pixel points；I is the number of target pixel points, wherein i >=1.
S250, according to the location information of pupil center's pixel, information is watched in determination attentively.
According to the location information of pupil center's pixel known in a large amount of infrared thermal imaging figure, and by plane of fixation
Corresponding known blinkpunkt coordinate, determines infrared thermal imaging figure and by the mapping coefficient between plane of fixation, and is stored in advance
In terminal device or in cloud platform storage system.Wherein, the plane where watching area can be understood as by plane of fixation.
According to the location information of determining pupil center's pixel and predetermined infrared thermal imaging figure with watched attentively
Mapping coefficient between plane determines blinkpunkt coordinate corresponding with pupil center's pixel.It should be noted that mapping system
Number with watch user attentively, by plane of fixation size and to watch user attentively related with by the distance between plane of fixation.
It is understood that in order to improve watch attentively information determine precision, can also determine watch information attentively during,
By allowing user repeatedly to watch same target point attentively, or watch multiple target points attentively, mapping coefficient is calibrated.Wherein, mesh
The information of punctuation bit is known blinkpunkt coordinate.
The embodiment of the present invention specifically improves the scheme of blinkpunkt coordinate, solution by the determination step of refinement pupil pixel
It has determined and has needed to be arranged the technical issues of infrared light supply carries out Eye-controlling focus in the prior art, reduced hard needed for Eye-controlling focus module
Number of packages amount reduces the space hold amount of hardware.
Embodiment three
Fig. 2 B is the flow chart of one of embodiment of the present invention three Eye-controlling focus method, and the embodiment of the present invention is in embodiment
On the basis of two technical solution, S230A is replaced with into S230B, target pixel points are determined in a manner of through threshold determination, with
Improve the determination mechanism of pupil pixel.
The pixel that S230B, temperature value meet preset threshold is target pixel points.
Illustratively, the pixel that temperature value meets preset threshold is target pixel points, can be temperature value
The first pixel for meeting first threshold is target pixel points.
Wherein, first threshold can be by the temperature value of each pixel in a large amount of known infrared thermal imaging figure and each
Pupil belonging to pixel or non-pupil type determine, can also be by technical staff rule of thumb or demand sets itself.
It is understood that in order to reduce the error of determining pupil pixel the second threshold can be met with temperature value
Second pixel of value is target pixel points, to introduce the pixel around pupil in setting regions in target pixel points, with
Reconcile target pixel points determined by each first pixel.Wherein, second threshold can pass through a large amount of known infrared thermal imaging
The white of the eye belonging to the temperature value and each pixel of each pixel in figure or non-white of the eye type are determining, can also be by technical staff
Rule of thumb or demand sets itself.
Example IV
Fig. 3 is the flow chart of one of embodiment of the present invention four Eye-controlling focus method.The embodiment of the present invention is above-mentioned each
Additional optimization has been carried out on the basis of the technical solution of embodiment.
Further, additional " to be based on the infrared thermal imaging after " obtaining infrared thermal imaging figure when user watches attentively "
Figure determines the infrared thermal imaging eye figure obtained when user watches attentively ", to improve the determination mechanism of pupil pixel.
S310, infrared thermal imaging figure of user when watching attentively is obtained.
Wherein, the image of image acquisition device is the face-image formation infrared thermal imaging figure for watching user attentively.
Image collecting device is when user watches the display plane for needing to track user's sight attentively, by the infrared thermal imaging of acquisition
Face figure is sent directly to configure the equipment of the image collecting device, or is sent directly to carry out the terminal device of Eye-controlling focus,
Or it is uploaded to cloud platform server.Correspondingly, terminal device from be configured with image collecting device equipment or terminal device
Corresponding infrared thermal imaging figure is got in itself storage system or cloud platform server.
S320, the infrared thermal imaging eye figure obtained when user watches attentively is determined based on the infrared thermal imaging figure.
Illustratively, each pixel in the available infrared thermal imaging figure positioned at the first predeterminable area is formed infrared
Thermal imaging left eye figure；And/or each pixel obtained in the infrared thermal imaging figure positioned at the second predeterminable area forms infrared heat
Right eye figure is imaged.
Based on anthropological measuring relationship, the rough position of eyes is determined from the infrared thermal imaging figure got.Wherein, terminal
When equipment obtains the infrared thermal imaging eye figure of left eye, it is set as former with first pixel in the upper left corner of infrared thermal imaging figure
20% × 30% pixel of facial area is set starting point by point, while along parallel with infrared thermal imaging figure edge
Direction expands to the right the rectangle frame that size is 25% × 20%, determines that the region in the rectangle frame is infrared thermal imaging left eye
Figure.Wherein, when terminal device obtains the infrared thermal imaging eye figure of right eye, with first pixel in the upper left corner of infrared thermal imaging figure
Point is set as origin, sets starting point, while edge and infrared thermal imaging figure for 60% × 30% pixel of facial area
The parallel direction in edge expands to the right the rectangle frame that size is 25% × 20%, determines that the region in the rectangle frame is infrared heat
Right eye figure is imaged.
S330, corresponding pupil center's pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure.
Optionally, the temperature value of each pixel in infrared thermal imaging left eye figure is obtained, and combines pupil of left eye discrimination model,
The pixel for determining that pixel vertex type belongs to pupil is left target pixel points；And/or obtain each picture in infrared thermal imaging right eye figure
The temperature value of vegetarian refreshments, and pupil of right eye discrimination model is combined, determine that pixel vertex type belongs to the right target pixel points of pupil.
Or it is optional, obtain the temperature value of each pixel in infrared thermal imaging left eye figure, and temperature value meets the
The first pixel of a left side of one threshold value is left target pixel points；And/or obtain the temperature of each pixel in infrared thermal imaging right eye figure
Value, and it is right target pixel points that temperature value, which meets the first pixel of the right side of first threshold,.It is understood that in order to improve
The accuracy of identified pupil pixel can also continue to determine that the temperature value of each pixel in infrared thermal imaging left eye figure is full
The second pixel of a left side of sufficient second threshold is left target pixel points, and/or determines each pixel in infrared thermal imaging right eye figure
The second pixel of the right side that temperature value meets second threshold is right target pixel points.
Wherein, first threshold, second threshold can pass through the temperature of each pixel of a large amount of known infrared thermal imaging figure
Value and corresponding pixel vertex type determine；It can also be by technical staff rule of thumb or demand sets itself.
Using the temperature value of each left target pixel points as weight, the weighted average of the coordinate value of each left target pixel points is determined
Value, obtains the location information of left pupil center's pixel；And/or using the temperature value of each right target pixel points as weight, determine each
The weighted average of the coordinate value of right target pixel points obtains the location information of right pupil center's pixel.
Specifically, according to formulaDetermine the location information of left pupil center's pixel；
Wherein, (Xl, Yl) be left pupil center's pixel coordinate value；(xli, yli) be left target pixel points coordinate value；
IliFor the temperature value of left target pixel points；Li is the number of left target pixel points, wherein li >=1.
And/or according to formulaDetermine the location information of right pupil center's pixel；
Wherein, (Xr, Yr) be right pupil center's pixel coordinate value；(xri, yri) be right target pixel points coordinate value；
IriFor the temperature value of right target pixel points；Ri is the number of right target pixel points, wherein ri >=1.
S340, according to the location information of pupil center's pixel, information is watched in determination attentively.
According to pupil pixel known in a large amount of infrared thermal imaging figure or the location information of pupil center's pixel, and
By the corresponding known blinkpunkt coordinate of plane of fixation, infrared thermal imaging figure is determined and by the mapping system between plane of fixation
Number, and be stored in advance in terminal device or cloud platform storage system in.
Wherein, when known pupil center's pixel is left pupil center's pixel, corresponding blinkpunkt coordinate is left eye
Blinkpunkt coordinate, to determine left eye mapping coefficient；It is corresponding when known pupil center's pixel is right pupil center's pixel
Blinkpunkt coordinate is right eye blinkpunkt coordinate, to determine right eye mapping coefficient.It should be noted that mapping coefficient and watching use attentively
Family, by plane of fixation size, right and left eyes and to watch user attentively related with by the distance between plane of fixation.
According to the location information of left pupil center's pixel and predetermined left eye mapping coefficient, determination is watched attentively flat
The left eye blinkpunkt coordinate in face；And/or it is mapped according to the location information of right pupil center's pixel and predetermined right eye
Coefficient is determined by the right eye blinkpunkt coordinate of plane of fixation.
It should be noted that can determine two when left eye blinkpunkt coordinate and right eye blinkpunkt coordinate has been determined simultaneously
The average value of person is finally determining blinkpunkt coordinate, namely what is finally determined watch information attentively.
The embodiment of the present invention passes through the obtaining step of refinement infrared thermal imaging eye figure, further to improve Eye-controlling focus entirety
Scheme, so that when the infrared thermal imaging eye figure according to left eye, right eye or eyes can watch by plane of fixation user attentively
Sight is tracked.
Embodiment five
Fig. 4 is a kind of structural schematic diagram for Eye-controlling focus device that the embodiment of the present invention five provides.The embodiment of the present invention can
Suitable for the case where user carries out Eye-controlling focus is watched attentively, which includes: that thermograph obtains module 410, pupil pixel
Determining module 420 and watch information determination module 430 attentively.
Wherein, thermograph obtains module 410, for obtaining infrared thermal imaging figure when user watches attentively；
Pupil pixel determining module 420 is determined for the temperature value according to each pixel in the infrared thermal imaging figure
Corresponding pupil pixel；
Watch information determination module 430 attentively, for the information according to the pupil pixel, information is watched in determination attentively.
The embodiment of the present invention obtains infrared thermal imaging figure when module acquisition user watches attentively by thermograph；Pass through pupil
Pixel determining module determines corresponding pupil pixel according to the temperature value of each pixel in the infrared thermal imaging figure；Pass through note
Visual information determining module determines according to the information of pupil pixel and watches information attentively.Solves existing skill by adopting the above technical scheme
Need to be arranged the technical issues of infrared light supply carries out Eye-controlling focus in art, hardware quantity needed for reducing Eye-controlling focus module subtracts
The space hold amount of hardware is lacked.
On the basis of the technical solution of above-described embodiment, further, the pupil pixel determining module 420, packet
It includes:
Temperature acquiring unit for traversing each pixel in the infrared thermal imaging figure, and obtains the temperature of each pixel
Angle value；
Target pixel points determination unit, for determining pixel vertex type based on each temperature value and in conjunction with pupil decision model
The pixel for belonging to pupil is target pixel points；Pupil pixel determination unit, for the coordinate value based on each target pixel points,
The centre coordinate for determining the target pixel points is pupil center's pixel.
On the basis of the technical solution of above-described embodiment, further, target pixel points determination unit is also used to determine
The pixel that temperature value meets preset threshold is target pixel points.
On the basis of the technical solution of above-described embodiment, further, the pupil decision model is according to known red
Linear discriminant model constructed by the temperature value and corresponding pixel vertex type of outer each pixel of thermograph；Wherein, the picture
Vegetarian refreshments type includes pupil and non-pupil.
On the basis of the technical solution of above-described embodiment, further, target pixel points determination unit is executing determination
Temperature value meet preset threshold pixel be target pixel points when, be specifically used for:
The first pixel that temperature value meets first threshold is that target pixel points and/or temperature value meet the
Second pixel of two threshold values is target pixel points.
On the basis of the technical solution of above-described embodiment, further, pupil pixel determination unit, is specifically used for:
Using the temperature value of each target pixel points as weight, determines the weighted average of the coordinate value of each target pixel points, obtain
To the location information of pupil center's pixel.
On the basis of the technical solution of above-described embodiment, further, which further includes that thermal imaging eye figure determines mould
Block is used for:
Based on the infrared thermal imaging figure after infrared thermal imaging figure when the acquisition user watches attentively, the use is determined
Infrared thermal imaging eye figure when family is watched attentively；
Correspondingly, thermal imaging eye figure determining module includes:
Left eye figure acquiring unit, for obtaining each pixel dot for being located at the first predeterminable area in the infrared thermal imaging figure
At infrared thermal imaging left eye figure；And/or
Right eye figure acquiring unit, for obtaining each pixel dot for being located at the second predeterminable area in the infrared thermal imaging figure
At infrared thermal imaging right eye figure.
Further, described to watch information determination module 430 attentively, it is specifically used for:
According to the location information and the infrared thermal imaging figure and quilt of pupil pixel and/or pupil center's pixel
Mapping coefficient where watching area between plane, determines and watches attentively corresponding to pupil pixel and/or pupil center's pixel
Point coordinate.
Eye-controlling focus method provided by any embodiment of the invention can be performed in above-mentioned Eye-controlling focus device, has and executes view
The corresponding functional module of line method for tracing and beneficial effect.
Embodiment six
Fig. 5 is a kind of hardware structural diagram for terminal device that the embodiment of the present invention six provides.As shown in figure 5, the end
End equipment, including input unit 510, processor 520 and storage device 530.
Wherein, input unit 510, for obtaining infrared thermal imaging figure when user watches attentively；
One or more processors 520；
Storage device 530, for storing one or more programs.
In Fig. 5 by taking a processor 520 as an example, input unit 510 in the terminal device can by bus or other
Mode is connected with processor 520 and storage device 530, and processor 520 and storage device 530 are also by bus or its other party
Formula connects, in Fig. 5 for being connected by bus.
In the present embodiment, the infrared thermal imaging that the processor 520 in terminal device can be obtained according to input unit 510
The temperature value of each pixel in figure determines corresponding pupil pixel；It can also be according to the location information of pupil pixel, really
Surely watch information attentively.
Storage device 530 in the terminal device is used as a kind of computer readable storage medium, can be used for storing one or
Multiple programs, described program can be software program, computer executable program and module, such as sight in the embodiment of the present invention
(for example, attached thermograph shown in Fig. 4 obtains module 410, pupil pixel is true for the corresponding program instruction/module of method for tracing
Cover half block 420 and watch information determination module 430 attentively).The software that processor 520 is stored in storage device 530 by operation
Program, instruction and module, thereby executing the various function application and data processing of terminal device, i.e. the realization above method is real
Apply the Eye-controlling focus method in example.
Storage device 530 may include storing program area and storage data area, wherein storing program area can store operation system
Application program needed for system, at least one function；Storage data area can storing data etc. (infrared heat in such as above-described embodiment at
As figure, each pixel temperature value, pupil pixel and watch information attentively etc.).In addition, storage device 530 may include high speed with
Machine access memory, can also include nonvolatile memory, a for example, at least disk memory, flush memory device or its
His non-volatile solid state memory part.In some instances, storage device 530 can further comprise remote relative to processor 520
The memory of journey setting, these remote memories can pass through network connection to terminal device.The example of above-mentioned network include but
It is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
In addition, the embodiment of the present invention also provides a kind of computer readable storage medium, it is stored thereon with computer program, it should
Realize that the present invention implements the Eye-controlling focus method provided when program is executed by Eye-controlling focus device, this method comprises: obtaining user
Infrared thermal imaging figure when watching attentively；Corresponding pupil pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure
Point；According to the information of the pupil pixel, information is watched in determination attentively.
By the description above with respect to embodiment, it is apparent to those skilled in the art that, the present invention
It can be realized by software and required common hardware, naturally it is also possible to which by hardware realization, but in many cases, the former is more
Good embodiment.Based on this understanding, technical solution of the present invention substantially in other words contributes to the prior art
Part can be embodied in the form of software products, which can store in computer readable storage medium
In, floppy disk, read-only memory (Read-Only Memory, ROM), random access memory (Random such as computer
Access Memory, RAM), flash memory (FLASH), hard disk or CD etc., including some instructions are with so that a computer is set
Standby (can be personal computer, server or the network equipment etc.) executes method described in each embodiment of the present invention.
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (10)
1. a kind of Eye-controlling focus method characterized by comprising
Obtain infrared thermal imaging figure when user watches attentively；
Corresponding pupil pixel is determined according to the temperature value of each pixel in the infrared thermal imaging figure；
According to the information of the pupil pixel, information is watched in determination attentively.
2. the method according to claim 1, wherein described according to each pixel in the infrared thermal imaging figure
Temperature value determines corresponding pupil pixel, comprising:
Each pixel in the infrared thermal imaging figure is traversed, and obtains the temperature value of each pixel；
Based on each temperature value and pupil decision model is combined, the pixel for determining that pixel vertex type belongs to pupil is object pixel
Point；
Based on the coordinate value of each target pixel points, determine that the centre coordinate of the target pixel points is pupil center's pixel.
3. the method according to claim 1, wherein described according to each pixel in the infrared thermal imaging figure
Temperature value determines corresponding pupil pixel, comprising:
Each pixel in the infrared thermal imaging figure is traversed, and obtains the temperature value of each pixel；
The pixel that temperature value meets preset threshold is target pixel points；
Based on the coordinate value of each target pixel points, determine that the centre coordinate of the target pixel points is pupil center's pixel.
4. according to the method described in claim 3, it is characterized in that, the pixel that the temperature value meets preset threshold is
Target pixel points, comprising:
The first pixel that temperature value meets first threshold is target pixel points；And/or temperature value meets the second threshold
Second pixel of value is target pixel points.
5. according to the method described in claim 2, it is characterized in that, the coordinate value based on each target pixel points, determines institute
The centre coordinate for stating target pixel points is pupil center's pixel, comprising:
Using the temperature value of each target pixel points as weight, determines the weighted average of the coordinate value of each target pixel points, obtain pupil
The location information of hole central pixel point.
6. the method according to claim 1, wherein it is described acquisition user watch attentively when infrared thermal imaging figure it
Afterwards, further includes:
Based on the infrared thermal imaging figure, the infrared thermal imaging eye figure when user watches attentively is determined；
Wherein, described to be based on the infrared thermal imaging figure, determine the infrared thermal imaging eye figure when user watches attentively, comprising:
It obtains in the infrared thermal imaging figure and forms infrared thermal imaging left eye figure positioned at each pixel of the first predeterminable area；With/
Or
It obtains in the infrared thermal imaging figure and forms infrared thermal imaging right eye figure positioned at each pixel of the second predeterminable area.
7. the method according to claim 1, wherein the information according to the pixel, letter is watched in determination attentively
Breath, comprising:
According to the location information and the infrared thermal imaging figure of pupil pixel and/or pupil center's pixel with watched attentively
Mapping coefficient where region between plane determines that blinkpunkt corresponding to pupil pixel and/or pupil center's pixel is sat
Mark.
8. a kind of Eye-controlling focus device characterized by comprising
Thermograph obtains module, for obtaining infrared thermal imaging figure when user watches attentively；
Pupil pixel determining module determines corresponding for the temperature value according to each pixel in the infrared thermal imaging figure
Pupil pixel；
Watch information determination module attentively, for the location information according to the pupil pixel, information is watched in determination attentively.
9. a kind of terminal device, which is characterized in that including input unit, further includes:
One or more processors；
Storage device, for storing one or more programs；
One or more of programs are executed by one or more of processors, so that one or more of processors are realized
Such as a kind of Eye-controlling focus method of any of claims 1-7.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
A kind of such as Eye-controlling focus method of any of claims 1-7 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910068612.7A CN109522887A (en) | 2019-01-24 | 2019-01-24 | A kind of Eye-controlling focus method, apparatus, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910068612.7A CN109522887A (en) | 2019-01-24 | 2019-01-24 | A kind of Eye-controlling focus method, apparatus, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109522887A true CN109522887A (en) | 2019-03-26 |
Family
ID=65799391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910068612.7A Pending CN109522887A (en) | 2019-01-24 | 2019-01-24 | A kind of Eye-controlling focus method, apparatus, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109522887A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111443665A (en) * | 2020-03-11 | 2020-07-24 | 东南大学 | Intelligent ward control method and system based on eye movement signals |
| CN111507298A (en) * | 2020-04-24 | 2020-08-07 | 深圳数联天下智能科技有限公司 | Face detection method and device, computer equipment and storage medium |
| CN112101064A (en) * | 2019-06-17 | 2020-12-18 | 北京七鑫易维科技有限公司 | Sight tracking method, device, equipment and storage medium |
| CN113408406A (en) * | 2021-06-17 | 2021-09-17 | 杭州嘉轩信息科技有限公司 | Sight tracking method and system |
| WO2022257120A1 (en) * | 2021-06-11 | 2022-12-15 | 华为技术有限公司 | Pupil position determination method, device and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101642376A (en) * | 2008-08-04 | 2010-02-10 | 南京大学 | Device and method for detecting fatigue |
| US20150070654A1 (en) * | 2012-12-11 | 2015-03-12 | Elwha Llc | Self-aligning unobtrusive active eye interrogation |
| US20150199066A1 (en) * | 2014-01-16 | 2015-07-16 | Samsung Electronics Co., Ltd. | Display apparatus and controlling method thereof |
| US20160274762A1 (en) * | 2015-03-16 | 2016-09-22 | The Eye Tribe Aps | Device interaction in augmented reality |
| KR20180011600A (en) * | 2016-07-25 | 2018-02-02 | 연세대학교 산학협력단 | Detection system and method for sinkhole using of infrared rays camera |
| CN108881724A (en) * | 2018-07-17 | 2018-11-23 | 北京七鑫易维信息技术有限公司 | A kind of image acquiring method, device, equipment and storage medium |
-
2019
- 2019-01-24 CN CN201910068612.7A patent/CN109522887A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101642376A (en) * | 2008-08-04 | 2010-02-10 | 南京大学 | Device and method for detecting fatigue |
| US20150070654A1 (en) * | 2012-12-11 | 2015-03-12 | Elwha Llc | Self-aligning unobtrusive active eye interrogation |
| US20150199066A1 (en) * | 2014-01-16 | 2015-07-16 | Samsung Electronics Co., Ltd. | Display apparatus and controlling method thereof |
| US20160274762A1 (en) * | 2015-03-16 | 2016-09-22 | The Eye Tribe Aps | Device interaction in augmented reality |
| KR20180011600A (en) * | 2016-07-25 | 2018-02-02 | 연세대학교 산학협력단 | Detection system and method for sinkhole using of infrared rays camera |
| CN108881724A (en) * | 2018-07-17 | 2018-11-23 | 北京七鑫易维信息技术有限公司 | A kind of image acquiring method, device, equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| QUAN WANG等: "Thermographic Eye Tracking", 《PROCEEDINGS OF THE NINTH BIENNIAL ACM SYMPOSIUM ON EYE TRACKING RESEARCH & APPLICATIONS》 * |
| 郑爽等: "一种模糊红外图像人眼快速定位方法", 《红外技术》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101064A (en) * | 2019-06-17 | 2020-12-18 | 北京七鑫易维科技有限公司 | Sight tracking method, device, equipment and storage medium |
| CN111443665A (en) * | 2020-03-11 | 2020-07-24 | 东南大学 | Intelligent ward control method and system based on eye movement signals |
| CN111507298A (en) * | 2020-04-24 | 2020-08-07 | 深圳数联天下智能科技有限公司 | Face detection method and device, computer equipment and storage medium |
| CN111507298B (en) * | 2020-04-24 | 2023-12-12 | 深圳数联天下智能科技有限公司 | Face detection method, device, computer equipment and storage medium |
| WO2022257120A1 (en) * | 2021-06-11 | 2022-12-15 | 华为技术有限公司 | Pupil position determination method, device and system |
| CN113408406A (en) * | 2021-06-17 | 2021-09-17 | 杭州嘉轩信息科技有限公司 | Sight tracking method and system |
| CN113408406B (en) * | 2021-06-17 | 2023-02-28 | 杭州嘉轩信息科技有限公司 | Sight tracking method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109522887A (en) | A kind of Eye-controlling focus method, apparatus, equipment and storage medium | |
| CN108427503B (en) | Human eye tracking method and human eye tracking device | |
| CN106598221B (en) | 3D direction of visual lines estimation method based on eye critical point detection | |
| US9864430B2 (en) | Gaze tracking via eye gaze model | |
| CN106529409B (en) | A kind of eye gaze visual angle measuring method based on head pose | |
| CN104978548B (en) | A kind of gaze estimation method and device based on three-dimensional active shape model | |
| CN109271914A (en) | Detect method, apparatus, storage medium and the terminal device of sight drop point | |
| CN103870796B (en) | Eye sight evaluation method and device | |
| CN108038884B (en) | Calibration method, calibration device, storage medium and processor | |
| CN109690553A (en) | The system and method for executing eye gaze tracking | |
| JP4692526B2 (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| CN109472189B (en) | Pupil radius compensation | |
| WO2023011339A1 (en) | Line-of-sight direction tracking method and apparatus | |
| JP2016173313A (en) | Visual line direction estimation system, visual line direction estimation method and visual line direction estimation program | |
| CN108921148A (en) | Determine the method and device of positive face tilt angle | |
| CN111631744B (en) | Method, device and system for CT scanning positioning | |
| US20160203358A1 (en) | Systems and methods for using curvatures to analyze facial and body features | |
| CN109948435A (en) | Sitting posture prompting method and device | |
| CN109284778A (en) | Face face value calculating method, computing device and electronic equipment | |
| JP2019215688A (en) | Visual line measuring device, visual line measurement method and visual line measurement program for performing automatic calibration | |
| CN116051631A (en) | Light spot labeling method and system | |
| CN112232128A (en) | Eye tracking based method for identifying care needs of old disabled people | |
| WO2020237940A1 (en) | Fatigue detection method and device based on human eye state identification | |
| CN109034108B (en) | Sight estimation method, device and system | |
| CN114967128B (en) | Sight tracking system and method applied to VR glasses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190326 |
|
| RJ01 | Rejection of invention patent application after publication |