CN107545302A - A kind of united direction of visual lines computational methods of human eye right and left eyes image - Google Patents
A kind of united direction of visual lines computational methods of human eye right and left eyes image Download PDFInfo
- Publication number
- CN107545302A CN107545302A CN201710650058.4A CN201710650058A CN107545302A CN 107545302 A CN107545302 A CN 107545302A CN 201710650058 A CN201710650058 A CN 201710650058A CN 107545302 A CN107545302 A CN 107545302A
- Authority
- CN
- China
- Prior art keywords
- eye
- image
- information
- visual lines
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
Abstract
The invention provides a kind of united direction of visual lines computational methods of human eye right and left eyes image, including：The extraction model of binocular information, eye image is inputted, by dual channel model, automatically extract the information characteristics of the left eye contained in image and right eye respectively；The extraction model of human eye united information feature, inputs the binocular images of user, models coupling binocular information, extracts the united information feature of human eye；A kind of unified algorithm has been invented, by input feature vector information, three-dimensional direction of visual lines is calculated.One of application of the present invention is virtual reality and man-machine interaction, and its principle is the eyes image by shooting user, calculates user's direction of visual lines, so as to be interacted with intelligence system interface or Virtual Reality Object.The present invention can be also widely applied to the fields such as training, Entertainment, video monitoring, medical monitoring.
Description
Technical field
The present invention relates to computer vision and image processing field, specifically a kind of human eye right and left eyes image is united
Direction of visual lines computational methods.
Background technology
The dynamic tracking of Eye-controlling focus/eye understands for user behavior and efficient man-machine interaction is significant.The mankind can feel
Know that the part in information more than 80% is received by human eye, and the part wherein more than 90% is handled by vision system.Therefore, sight
It is to reflect people and the important clue of extraneous interaction.In recent years, it is rapid due to virtual reality technology and human-computer interaction technology
Development, the application value of Eye Tracking Technique gradually highlight；On the other hand, direction of visual lines is calculated in computer vision field still
It is be rich in challenge the problem of.
Current Eye Tracking Technique has fundamentally been divided into the Eye Tracking Technique based on outward appearance and regarding based on model
Two kinds of line tracer technique.In current environment, because the Eye Tracking Technique based on model often has very high accuracy,
So that most people are absorbed in the research of the Eye-controlling focus method based on model.Eye-controlling focus skill based on model
Art needs experimenter to provide many geometric properties, such as the direction of pupil, and establishes an eye model with this, passes through
Model predicts the direction of people's sight.Just because of this, the Eye Tracking Technique based on model has following a few point defects:1) need
Want expensive equipment and instrument.Eye Tracking Technique based on model is by establishing an eye model or other geometry moulds
Type predicts the direction of visual lines of participant, so it needs to extract some passes of participant by using some unique equipment
Model is established in the geometric properties of eyes, and with this.2) Eye Tracking Technique based on model is needed in strict indoor environment
Middle progress.It is usually to be surveyed by infrared ray by needing the geometric properties that experimenter provides in the Eye Tracking Technique based on model
Measure, and contained too many infrared light in some other interference sources, such as sunshine, the measurement knot that this can be to instrument
Fruit causes very serious interference, so the equipment of measurement is also required to be placed in strict indoor environment, the sun is avoided with this
The interference of other infrared lights such as light.3) Eye Tracking Technique based on model needs high-resolution image to be trained, institute
To there is confined operating distance, in general no more than 60cm.Therefore, the experimental method based on model can not generally make
For in most conventional environment.
In contrast to this, the Eye Tracking Technique based on outward appearance is directly by from human eye picture learning to various information,
The mapping relations established between picture and human eye sight, the direction of sight is obtained with this, it does not have above-mentioned based on model
A variety of limitations that Eye Tracking Technique is possessed, it only needs to go to photograph the outside drawing of human eye by a common camera
Piece.This condition causes the Eye Tracking Technique based on outward appearance to have universal applicability, also the sight based on outward appearance is chased after
Track technology has the application prospect that can not be matched in excellence or beauty.But due to based on the Eye Tracking Technique of outward appearance for sampling tool, sampling
The requirement of environment is not strict, so that the input data of model, often there is diversified such environmental effects, such as
Say：Illumination, participant, head position etc..The intensity of illumination can cause picture to become bright or dark, in very dark environment, people
Even more it is difficult to the eye image that a people is told from picture；Likewise, the difference of head position is even more adopting for human eye
Sample has very big influence, for same person, shoots its front and shines and laterally take pictures resulting eye image just
It is different.A variety of factors above so that just have very big make an uproar in the input data of the Eye Tracking Technique based on outward appearance
Acoustic intelligence, the challenge that this Eye Tracking Technique for being also based on outward appearance is faced, just because of this defect, based on outward appearance
Eye Tracking Technique, in accuracy, be nothing like the Eye-controlling focus method based on model.It is meanwhile current based on outward appearance
Eye-controlling focus method, often using the single eye images of user as input information, but in the application of reality, each time
For user profile collect be all often user's a certain moment binocular images, by binocular images of synchronization point
Input is opened, also just have ignored the correlation in binocular images on a certain moment.
Coming in recent years, various new-type models are also to arise at the historic moment, in this various model, nerve net
The performance of network is especially prominent.A classification of the convolutional neural networks as neutral net in deep learning, it is even more very burning hot.
Because the convolutional neural networks in deep learning are provided with the characteristic of local sensing so that it can be good at extracting picture
Local feature, retain the local correlation information of picture, simultaneously because the reason for weights are shared so that the convolution god in deep learning
It also not may require that the consumption substantial amounts of time goes to be trained through network, also therefore, the convolutional neural networks in deep learning exist
Showed in various image processing tasks it is especially prominent, such as：Image classification, target detection, and semantic separation etc..Meanwhile closely
Over a little years, the rapid development of hardware, the convolutional neural networks even more allowed in deep learning show more in image processing method
Remarkably.But determine that method not yet has similar information report for human eye sight direction.
The content of the invention
The technology of the present invention solves problem：Overcome the deficiencies in the prior art, there is provided a kind of human eye right and left eyes image regards in combination
Line direction calculating method, go to extract the information factor contained in image by using neutral net, and pass through adaptive side
Method adjusts neural network model, the final direction of visual lines for predicting eyes, mainly by combining the image information of eyes, so as to be used for
Solve the problems, such as that the single eye images noise that is inputted in the Eye-controlling focus method based on outward appearance is larger, it is achieved thereby that high-precision three
Tie up direction of visual lines prediction.
The technology of the present invention solution：A kind of united direction of visual lines computational methods of human eye right and left eyes image, comprising following
Step：
(1) user's face-image is shot, positions left eye or right eye region, eye image is pre-processed, realizes to head position
Amendment, and be fixed the eye image of pixel size；
(2) dual channel model is established, inputs the image information of left eye and right eye in eye image respectively, uses depth nerve
Network model extracts and exports left eye and the information characteristics of right eye respectively；
(3) one channel model is established, inputs the image information of left eye and right eye, using deep neural network model extraction simultaneously
Export right and left eyes image united information feature；
(4) method for using regression analysis, it is special with reference to the information characteristics and right and left eyes image united information of left eye and right eye
Sign, and pass through combined optimization, three-dimensional direction of visual lines corresponding to prediction eyes difference；Or the information of left eye and right eye is used alone
Feature or right and left eyes image united information feature, using the method for regression analysis, after optimization, predict corresponding to eyes difference
Three-dimensional direction of visual lines
The step (2) establishes dual channel model, inputs the image information of left eye and right eye in eye image respectively, passes through
Dual channel model extracts and exports left eye and the detailed process of the information characteristics of right eye is as follows respectively：
(21) by the left eye of revised fixed size and eye image IlAnd IrInput in dual channel model, IlAnd IrPoint
Jing Guo not a passage processing；
(22) each passage is a deep neural network model, and the model is rolled up to the eye image of input
Product, Chi Hua, full attended operation, export the characteristic vector of regular length；
(23) characteristic vector of regular length caused by each passage is that corresponding input picture passes through deep neural network
Information characteristics after extraction, information characteristics caused by two passages are connected, obtain the letter of final left eye and right eye
Cease feature.
The step (3) establishes one channel model, inputs the pictorial information of left eye and right eye, is extracted using one channel model
And the detailed process for exporting right and left eyes image united information feature is as follows：
(31) by the eye image input one channel model of revised fixed size；
(32) deep neural network model is used respectively, and convolution, Chi Hua, full connection behaviour are carried out to the image of right and left eyes respectively
Make, export the right and left eyes information characteristics after simplifying；
(33) right and left eyes information characteristics are connected, multiple full articulamentums are added after deep neural network model, using connecting entirely
Laminated and right and left eyes information characteristics are connect, finally give right and left eyes image united information feature.
The step (4) is using the method using regression analysis, with reference to left eye and the information characteristics of right eye, and left and right eye pattern
As united information feature, and pass through combined optimization, the detailed process of three-dimensional direction of visual lines is as follows corresponding to prediction eyes difference：
(41) left eye after Introduced Malaria and eye image IlAnd IR,And the true left eye direction of visual lines corresponding to image
glWith true right eye direction of visual lines gr；
(42) using the deep neural network model proposed in step (2) and step (3), extract double corresponding to image
Eye information characteristics and united information feature；
(43) information characteristics of all extractions are connected or a kind of feature is used alone as global feature, use regression analysis
Method, the left eye direction of visual lines f (I) predictedlWith the right eye direction of visual lines f (I) of predictionr；
(44) using differential seat angle as error amount, using the method for gradient decline, optimization is iterated to model so that prediction
Direction of visual lines become closer in real direction of visual lines；
(45) selection prediction direction of visual lines passes through closest to the model in Real line-of-sight direction as final model, model
Input eye image, the direction of visual lines predicted, and using the direction of visual lines as final prediction result.
Compared with the method for other Eye-controlling focus, the present invention beneficial to the characteristics of be：
(1) the information characteristics extraction model of eyes has been invented, the information characteristics of eyes can be extracted, and also can be independent
The characteristic information using eyes predict direction of visual lines, as a result still better than the simple eye sight method for tracing of in general；
(2) in view of, there is the contact of certain correlation, having invented human eye joint letter in the binocular images of synchronization
The extraction model of feature is ceased, the correlative character information of eyes can be extracted, be effectively combined the image information of eyes, simultaneously
Direction of visual lines only is predicted using only the correlative character information of eyes, as a result still better than the simple eye sight method for tracing of in general；
(3) neutral net of a multichannel is established, network, can be more by the method for recurrence by input feature vector information
Accurate prediction obtains the three-dimensional direction of visual lines of eyes, at the same can effectively solve some single eye images larger noise be present and
The problem of causing prediction result inaccuracy.
Brief description of the drawings
Fig. 1 is the network structure simplified schematic diagram of the present invention, and wherein a is the dual channel model in step (2) in the content of the invention,
B is the one channel model in step (3) in the content of the invention, and c is the sight forecast model of step (4) in the content of the invention；
Fig. 2 is the base neural network structural representation of the present invention；
Fig. 3 is the overall construction drawing of the computational methods that matching direction of visual lines is analyzed based on user's eyes of the present invention；
Fig. 4 is the model training flow chart of the present invention.
Embodiment
The specific implementation to the present invention elaborates below in conjunction with the accompanying drawings.
The invention provides a kind of united direction of visual lines computational methods of human eye right and left eyes image, input human eye information is special
Sign, the eyes direction of visual lines of people is predicted, meanwhile, single channel and binary channels deep neural network model are proposed respectively, are used for
The information characteristics used in extracting method.This method does not have additional demand to system, the people's eye pattern shot using only one camera
As input.Meanwhile the present invention can eliminate the larger error feelings of some simple eye noises by the image information of combination eyes
Condition, it is achieved thereby that comparing the more preferable robustness of other similar approach.
First, obtained for eye image, the present invention includes below scheme.Using one camera, shooting contains user's face
The image in region.Left eye or right eye region are positioned using existing human face analysis method.The eye image extracted is located in advance
Reason, obtains the eye image of fixed pixel size being corrected to head position.
Secondly, invented while extracted left eye and the binary channels deep neural network model of right eye information feature, inputted
On the premise of binocular images, left eye and eye image respectively enter a passage, and after the independent processing of each passage, respectively
Obtain the characteristic information of left eye and right eye.
Further, the single channel depth nerve for the human eye united information feature extraction for being directed to binocular images input has been invented
Network model, on the premise of binocular images are inputted, establish deep neural network model, model handles left eye and right eye simultaneously
Image information, and the image information after processing is combined, model is further handled the information of combination again, so as to obtain on left eye
With the united information feature of right eye.
Finally, by combining two network models above, it is true the united direction of visual lines of human eye right and left eyes image has been invented
Determine method.Method establishes multichannel deep neural network direction of visual lines forecast model, by merging two information characteristics above
Extraction model, connects the obtained information characteristics of eyes and the united information feature of eyes, and regression analysis obtains the prediction of eyes
Direction of visual lines, and the direction of visual lines by statistical forecast and the angular deviation in Real line-of-sight direction, to weigh "current" model
Quality.The method that model is declined using gradient is independently optimized, by using formula：
Angular deviation is calculated, wherein n represents the quantity of the image pair of input, and to reduce angular deviation as target, it is right
Model is constantly optimized, during combined optimization, whenever inputting a pair of eye images and Real line-of-sight direction, just
An iteration optimization is carried out to model, after all known image information have been inputted, optimization process terminates, and has obtained final
Model.In practical application, model directly predicts the sight side of eye image by receiving a pair of brand-new eye images
To.
, can be individually by the information characteristics or connection of human eye right and left eyes meanwhile direction of visual lines method of estimation has reduction property
Close information characteristics and be used for regression analysis, obtain the prediction direction of visual lines of eyes, and using input picture Real line-of-sight direction and
Average angle deviation between output prediction direction of visual lines carries out adaptive adjustment to forecast model as error term.Equally,
Direction of visual lines method of estimation has addition property, can after the united information feature of the information characteristics of eyes and eyes has been obtained
Directly to add some other relevant information features, and using all features to be overall, carry out regression analysis judgement.
It is described in detail again below, it is specific as follows refering to the network structure simplified schematic diagram of Fig. 1 present invention：
(a) in Fig. 1 be for and meanwhile extract the binary channels deep neural network model of left eye and right eye information feature
Structure schematic diagram TE-I.Model passes through the network based on convolutional neural networks (CNN) by inputting the images of eyes, image
Processing exports the characteristic information of left eye and right eye respectively, and characteristic information is called in eyes feature, can also be pre- by eyes feature
Measure the direction of visual lines of eyes；
(b) in Fig. 1 be for and meanwhile extract left eye and the single channel deep neural network model knot of right eye information feature
Structure schematic diagram TE-II.After inputting binocular images, image passes through the network processes based on CNN respectively first, obtains each solely
Vertical characteristic information, a full articulamentum is then passed through, each independent feature fusion has been finally given into eyes phase
Closing property information, likewise, only by the correlation information of eyes, it is also predicted that obtaining the direction of visual lines of eyes；
(c) in Fig. 1 is the structural representation of the network model of the united direction of visual lines computational methods of human eye right and left eyes image
TE-A is schemed, by combining two kinds of model structures in (b) in (a) and Fig. 1 in Fig. 1, by disposably obtaining eyes feature
With eyes correlative character, and using these information as global feature, regression analysis obtains the direction of visual lines of eyes.
In above-mentioned three kinds of models, due to consideration that when shooting human eye outward appearance from front using camera, user's head position
The difference put, the eye image that can to photograph have different deformation, although eye image is initially being carried out accordingly
Conversion influences all to eliminate to eliminate the influence of head position, so when analysis direction of visual lines is predicted,
Head position vector is also added in final feature set.
Refering to the base neural network structural representation of Fig. 2 present invention.In order to extract excellent feature from image
Information, it is contemplated that outstanding representation of the convolutional neural networks in image procossing at present, method employ CNN networks and carried as feature
The basic network taken.The input of network is the gray scale picture of one 36 × 60, exports and can oneself for x dimensional features, x concrete numerical value
Row setting.Picture first passes around one layer of convolution after input, and the size of convolution kernel is set as 5 × 5, and output channel number is set
Be set to 20, after first layer convolution, output be 20 32 × 56 sizes picture, then, picture passes through maximum pond
Change layer, by 2 × 2 Chi Huahou, the picture that output is 20 16 × 28.Then, this 20 pictures is subjected to convolution, convolution again
Core is still 5 × 5, output channel 50, exports the picture of 50 12 × 24 altogether, then this 50 pictures is passed through to 2 × 2 maximum
Pond layer, obtain the picture of 50 6 × 12.Finally, this 50 6 × 12 picture is spread out, obtains 50 × 6 × 12 numbers, led to
Full articulamentum is crossed, the x dimensional features finally wanted.
The overall construction drawing of method is determined refering to a kind of united direction of visual lines of human eye right and left eyes image of Fig. 3 present invention.
The present invention carrys out the eyes three-dimensional direction of visual lines of forecast analysis user with this by independently setting up neutral net.Method passes through defeated
Enter the human eye gray scale picture of fixed size, and head angle vector, obtain 1506 dimensional feature vectors, then divided again by returning
Analysis, obtains the eyes direction of visual lines of 6 dimensions.The general structure of the present invention equally contains step (2) in the content of the invention, (3)
Network.Structure in the network general structure such as Fig. 3 of step (2) using two human eye pictures above as input, network difference are defeated
Enter the image of left eye and right eye, convolution then is carried out to image using the CNN networks in Fig. 2, and last feature quantity x is set
It is set to 1000, has obtained the characteristic vector that length is 1000；Characteristic vector passes through a full articulamentum (FC) respectively again, respectively
To the characteristic vector of 500 dimensions；Finally, simply the two 500 characteristic vectors tieed up are connected, as Part I
Output characteristic.Structure of following two images as input, Part II are same in the network general structure such as Fig. 3 of step (3)
Using left eye and eye image as input, convolution, and final feature quantity x are carried out to image by using CNN networks
It is set as 500, has respectively obtained the characteristic vector of 500 dimensions, then, be simply connected to become this 500 characteristic vector tieed up
The vector of 1000 dimensions, and characteristic vector is merged by a full articulamentum, the characteristic vector of 500 dimensions is obtained, and with this
Output of the characteristic vector of 500 dimensions as Part II；Likewise, due to consideration that the difference of head position, it will to figure
As causing some not eliminable influences, the present invention is using head position vector as Part III input, not through processing, directly
Connect and be added in final characteristic vector.Three parts output 1000 dimensions, 500 dimensions, the characteristic vector of 6 dimensions, by these respectively
Vector is attached, and has just obtained the final feature of 1506 dimensions.
Flow chart is predicted refering to the direction of visual lines based on user's binocular images of Fig. 4 present invention, with reference to the phase being described above
Close particular technique, the specific implementation process of the direction of visual lines prediction introduced below based on user's binocular images.
First, the prediction mould is initialized using the simple random method for assigning initial value by the model to being proposed in Fig. 3
Type.Then, eye image after treatment is inputted, whenever a pair of eye image I of inputlAnd Ir, one can be obtained by network
To the three-dimensional direction of visual lines f (I) of predictionlWith f (I)r, the direction of visual lines of left eye and right eye is represented respectively.Again by with it is original
Three-dimensional direction of visual lines glAnd grIt is compared, tries to achieve the angular deviation of prediction, the method then declined by using gradient, with drop
Low angle deviation is target, constantly optimizes network, whenever inputting a pair of images, just carries out an iteration tune to network parameter
It is whole.After all images have been inputted, what is obtained is exactly final forecast model.In final forecast model, schemed by inputting
Picture, can be to predict the direction of visual lines corresponding to image.
The representative embodiment of the present invention is the foregoing is only, is done according to technical scheme any etc.
Effect conversion, all should belong to protection scope of the present invention.
Claims (4)
1. a kind of united direction of visual lines computational methods of human eye right and left eyes image, it is characterised in that comprise the steps of：
(1) user's face-image is shot, positions left eye or right eye region, eye image is pre-processed, realizes and head position is repaiied
Just, and it is fixed the eye image of pixel size；
(2) dual channel model is established, the image information of left eye and right eye in eye image is inputted respectively, uses deep neural network
Model extracts and exports left eye and the information characteristics of right eye respectively；
(3) one channel model is established, the image information of left eye and right eye is inputted, is extracted and exported using deep neural network model
Right and left eyes image united information feature；
(4) method for using regression analysis, with reference to left eye and the information characteristics and right and left eyes image united information feature of right eye, and
By combined optimization, three-dimensional direction of visual lines corresponding to prediction eyes difference；Or the information characteristics of left eye and right eye are used alone
Or right and left eyes image united information feature, it is three-dimensional corresponding to prediction eyes difference after optimization using the method for regression analysis
Direction of visual lines.
2. the united direction of visual lines computational methods of human eye right and left eyes image according to claim 1, it is characterised in that：It is described
Step (2) establishes dual channel model, inputs the image information of left eye and right eye in eye image respectively, by dual channel model point
Indescribably take and export left eye and the detailed process of the information characteristics of right eye is as follows：
(21) by the left eye of revised fixed size and eye image IlAnd IrInput in dual channel model, IlAnd IrPass through respectively
One passage processing；
(22) each passage is a deep neural network model, and the model carries out convolution, pond to the eye image of input
Change, full attended operation, export the characteristic vector of regular length；
(23) characteristic vector of regular length caused by each passage is that corresponding input picture extracts by deep neural network
Information characteristics afterwards, information characteristics caused by two passages are connected, the information for obtaining final left eye and right eye is special
Sign.
3. the united direction of visual lines computational methods of human eye right and left eyes image according to claim 1, it is characterised in that：It is described
Step (3) establishes one channel model, inputs the pictorial information of left eye and right eye, is extracted using one channel model and export right and left eyes
The detailed process of image united information feature is as follows：
(31) by the eye image input one channel model of revised fixed size；
(32) deep neural network model is used respectively, and convolution, Chi Hua, full attended operation are carried out to the image of right and left eyes respectively,
Export the right and left eyes information characteristics after simplifying；
(33) right and left eyes information characteristics are connected, multiple full articulamentums are added after deep neural network model, use full articulamentum
Merge the information characteristics of right and left eyes, finally give right and left eyes image united information feature.
4. the united direction of visual lines computational methods of human eye right and left eyes image according to claim 1, it is characterised in that：It is described
Step (4) is special with reference to left eye and the information characteristics of right eye, and right and left eyes image united information using the method using regression analysis
Sign, and pass through combined optimization, the detailed process of three-dimensional direction of visual lines is as follows corresponding to prediction eyes difference：
(41) left eye after Introduced Malaria and eye image IlAnd Ir, and the true left eye direction of visual lines g corresponding to imagelWith
True right eye direction of visual lines gr；
(42) using the deep neural network model proposed in step (2) and step (3), the eyes letter corresponding to image is extracted
Cease feature and united information feature；
(43) information characteristics of all extractions are connected or a kind of feature is used alone as global feature, use the side of regression analysis
Method, the left eye direction of visual lines f (I) predictedlWith the right eye direction of visual lines f (I) of predictionr；
(44) using differential seat angle as error amount, using the method for gradient decline, optimization is iterated to model so that prediction regards
Line direction is become closer in real direction of visual lines；
(45) for selection prediction direction of visual lines closest to the model in Real line-of-sight direction as final model, model passes through input
Eye image, the direction of visual lines predicted, and using the direction of visual lines as final prediction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710650058.4A CN107545302B (en) | 2017-08-02 | 2017-08-02 | Eye direction calculation method for combination of left eye image and right eye image of human eye |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710650058.4A CN107545302B (en) | 2017-08-02 | 2017-08-02 | Eye direction calculation method for combination of left eye image and right eye image of human eye |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107545302A true CN107545302A (en) | 2018-01-05 |
| CN107545302B CN107545302B (en) | 2020-07-07 |
Family
ID=60971290
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710650058.4A Active CN107545302B (en) | 2017-08-02 | 2017-08-02 | Eye direction calculation method for combination of left eye image and right eye image of human eye |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107545302B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109240510A (en) * | 2018-10-30 | 2019-01-18 | 东北大学 | Augmented reality human-computer interaction device and control method based on Eye-controlling focus |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
| CN109840019A (en) * | 2019-02-22 | 2019-06-04 | 网易（杭州）网络有限公司 | Control method, device and the storage medium of virtual portrait |
| WO2019154510A1 (en) * | 2018-02-09 | 2019-08-15 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| CN110321773A (en) * | 2018-03-30 | 2019-10-11 | 托比股份公司 | Use the neural metwork training for watching prediction attentively for three-dimensional (3D) of calibration parameter |
| CN111325736A (en) * | 2020-02-27 | 2020-06-23 | 成都航空职业技术学院 | Sight angle estimation method based on human eye difference image |
| CN112183200A (en) * | 2020-08-25 | 2021-01-05 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN112383765A (en) * | 2020-11-10 | 2021-02-19 | 中移雄安信息通信科技有限公司 | VR image transmission method and device |
| CN112711984A (en) * | 2020-12-09 | 2021-04-27 | 北京航空航天大学 | Fixation point positioning method and device and electronic equipment |
| CN113227878A (en) * | 2018-08-31 | 2021-08-06 | 目察科技股份有限公司 | Method and system for gaze estimation |
| US11194161B2 (en) | 2018-02-09 | 2021-12-07 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| WO2022246804A1 (en) * | 2021-05-28 | 2022-12-01 | 京东方科技集团股份有限公司 | Line of sight tracking method, apparatus, and system |
| US11537202B2 (en) | 2019-01-16 | 2022-12-27 | Pupil Labs Gmbh | Methods for generating calibration data for head-wearable devices and eye tracking system |
| US11556741B2 (en) | 2018-02-09 | 2023-01-17 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters using a neural network |
| CN116052264A (en) * | 2023-03-31 | 2023-05-02 | 广州视景医疗软件有限公司 | Sight estimation method and device based on nonlinear deviation calibration |
| US11676422B2 (en) | 2019-06-05 | 2023-06-13 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| CN117742388A (en) * | 2024-02-07 | 2024-03-22 | 广东海洋大学 | Intelligent control method and system based on reading furniture |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105303170A (en) * | 2015-10-16 | 2016-02-03 | 浙江工业大学 | Human eye feature based sight line estimation method |
| CN106909220A (en) * | 2017-02-21 | 2017-06-30 | 山东师范大学 | A kind of sight line exchange method suitable for touch-control |
-
2017
- 2017-08-02 CN CN201710650058.4A patent/CN107545302B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105303170A (en) * | 2015-10-16 | 2016-02-03 | 浙江工业大学 | Human eye feature based sight line estimation method |
| CN106909220A (en) * | 2017-02-21 | 2017-06-30 | 山东师范大学 | A kind of sight line exchange method suitable for touch-control |
Non-Patent Citations (3)
| Title |
|---|
| ANJITH GEORGE ET AL.: "Real-time Eye Gaze Direction Classification Using Convolutional Neural Network", 《2016 INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND COMMUNICATIONS》 * |
| KYLE KRAFKA ET AL.: "Eye Tracking for Everyone", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| XUCONG ZHANG ET AL.: "Appearance-Based Gaze Estimation in the Wild", 《2015 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11556741B2 (en) | 2018-02-09 | 2023-01-17 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters using a neural network |
| US11194161B2 (en) | 2018-02-09 | 2021-12-07 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| US11393251B2 (en) | 2018-02-09 | 2022-07-19 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| WO2019154510A1 (en) * | 2018-02-09 | 2019-08-15 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| US11340461B2 (en) | 2018-02-09 | 2022-05-24 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| EP3749172B1 (en) | 2018-02-09 | 2022-03-30 | Pupil Labs GmbH | Devices, systems and methods for predicting gaze-related parameters |
| CN110321773A (en) * | 2018-03-30 | 2019-10-11 | 托比股份公司 | Use the neural metwork training for watching prediction attentively for three-dimensional (3D) of calibration parameter |
| CN113227878A (en) * | 2018-08-31 | 2021-08-06 | 目察科技股份有限公司 | Method and system for gaze estimation |
| CN109376777A (en) * | 2018-10-18 | 2019-02-22 | 四川木牛流马智能科技有限公司 | Cervical cancer tissues pathological image analysis method and equipment based on deep learning |
| CN109240510A (en) * | 2018-10-30 | 2019-01-18 | 东北大学 | Augmented reality human-computer interaction device and control method based on Eye-controlling focus |
| CN109240510B (en) * | 2018-10-30 | 2023-12-26 | 东北大学 | Augmented reality man-machine interaction equipment based on sight tracking and control method |
| WO2020087919A1 (en) * | 2018-10-30 | 2020-05-07 | 东北大学 | Augmented reality human-computer interaction device and a control method based on gaze tracking |
| US11537202B2 (en) | 2019-01-16 | 2022-12-27 | Pupil Labs Gmbh | Methods for generating calibration data for head-wearable devices and eye tracking system |
| CN109840019A (en) * | 2019-02-22 | 2019-06-04 | 网易（杭州）网络有限公司 | Control method, device and the storage medium of virtual portrait |
| US11676422B2 (en) | 2019-06-05 | 2023-06-13 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| CN111325736A (en) * | 2020-02-27 | 2020-06-23 | 成都航空职业技术学院 | Sight angle estimation method based on human eye difference image |
| CN111325736B (en) * | 2020-02-27 | 2024-02-27 | 成都航空职业技术学院 | Eye differential image-based sight angle estimation method |
| CN112183200B (en) * | 2020-08-25 | 2023-10-17 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN112183200A (en) * | 2020-08-25 | 2021-01-05 | 中电海康集团有限公司 | Eye movement tracking method and system based on video image |
| CN112383765B (en) * | 2020-11-10 | 2023-04-07 | 中移雄安信息通信科技有限公司 | VR image transmission method and device |
| CN112383765A (en) * | 2020-11-10 | 2021-02-19 | 中移雄安信息通信科技有限公司 | VR image transmission method and device |
| CN112711984A (en) * | 2020-12-09 | 2021-04-27 | 北京航空航天大学 | Fixation point positioning method and device and electronic equipment |
| WO2022246804A1 (en) * | 2021-05-28 | 2022-12-01 | 京东方科技集团股份有限公司 | Line of sight tracking method, apparatus, and system |
| CN116052264A (en) * | 2023-03-31 | 2023-05-02 | 广州视景医疗软件有限公司 | Sight estimation method and device based on nonlinear deviation calibration |
| CN116052264B (en) * | 2023-03-31 | 2023-07-04 | 广州视景医疗软件有限公司 | Sight estimation method and device based on nonlinear deviation calibration |
| CN117742388A (en) * | 2024-02-07 | 2024-03-22 | 广东海洋大学 | Intelligent control method and system based on reading furniture |
| CN117742388B (en) * | 2024-02-07 | 2024-04-30 | 广东海洋大学 | Intelligent control method and system based on reading furniture |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107545302B (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107545302A (en) | A kind of united direction of visual lines computational methods of human eye right and left eyes image | |
| CN106648103B (en) | A kind of the gesture tracking method and VR helmet of VR helmet | |
| CN105069746B (en) | Video real-time face replacement method and its system based on local affine invariant and color transfer technology | |
| CN107229920B (en) | Behavior identification method based on integration depth typical time warping and related correction | |
| CN102520796B (en) | Sight tracking method based on stepwise regression analysis mapping model | |
| CN108241849A (en) | Human body interactive action recognition methods based on video | |
| CN109472198A (en) | A kind of video smiling face's recognition methods of attitude robust | |
| CN108895981A (en) | A kind of method for three-dimensional measurement, device, server and storage medium | |
| CN109671142A (en) | A kind of intelligence makeups method and intelligent makeups mirror | |
| CN110472542A (en) | A kind of infrared image pedestrian detection method and detection system based on deep learning | |
| CN108021889A (en) | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information | |
| CN104821010A (en) | Binocular-vision-based real-time extraction method and system for three-dimensional hand information | |
| CN111274921A (en) | Method for recognizing human body behaviors by utilizing attitude mask | |
| CN109800676A (en) | Gesture identification method and system based on depth information | |
| CN108470178B (en) | Depth map significance detection method combined with depth credibility evaluation factor | |
| CN109886356A (en) | A kind of target tracking method based on three branch's neural networks | |
| CN113762133A (en) | Self-weight fitness auxiliary coaching system, method and terminal based on human body posture recognition | |
| CN104038799A (en) | Three-dimensional television-oriented gesture manipulation method | |
| CN110135277B (en) | Human behavior recognition method based on convolutional neural network | |
| CN108073851A (en) | A kind of method, apparatus and electronic equipment for capturing gesture identification | |
| CN111176447A (en) | Augmented reality eye movement interaction method fusing depth network and geometric model | |
| CN114821640A (en) | Skeleton action identification method based on multi-stream multi-scale expansion space-time diagram convolution network | |
| CN109086707A (en) | A kind of expression method for tracing based on DCNNs-LSTM model | |
| Amrutha et al. | Human Body Pose Estimation and Applications | |
| CN115761791A (en) | Human body semantic prediction module based on 2D image, virtual clothes changing model and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |