CN106599994B - A kind of gaze estimation method based on depth Recurrent networks - Google Patents
A kind of gaze estimation method based on depth Recurrent networks Download PDFInfo
- Publication number
- CN106599994B CN106599994B CN201611036387.1A CN201611036387A CN106599994B CN 106599994 B CN106599994 B CN 106599994B CN 201611036387 A CN201611036387 A CN 201611036387A CN 106599994 B CN106599994 B CN 106599994B
- Authority
- CN
- China
- Prior art keywords
- depth
- indicate
- layers
- parameter
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
Abstract
This patent proposes a kind of gaze estimation method based on depth Recurrent networks, belongs to computer vision and machine learning field.The main thought of this method is the mapping relations established between input picture feature and sight by depth Recurrent networks.Firstly, extracting the gradient orientation histogram feature in eyes image region；Then, 5 layers of depth regression model is established, input picture feature is fitted and exports the mapping relations between direction of visual lines；Later, the parameter of gradient descent method optimization depth regression model is utilized；Finally, for eyes image to be estimated, direction of visual lines is estimated using the depth model succeeded in school.
Description
Technical field
The invention belongs to technical field of computer vision, are related to deep learning method, mainly solve sight estimation and sight
The visions mapping problems such as tracking can be applied to the fields such as vehicle security drive and area-of-interest detection.
Background technique
In computer vision, sight estimation refers to the face-image according to input, positions human eye area and according to eyeball
Automatically estimate direction of visual lines in position.Existing gaze estimation method includes two classes: (1) method based on geometry and (2) base
In the method for recurrence.It is detailed in bibliography: Takahiro Ishikawa, Simon Baker, Iain Matthews, and
Takeo Kanade, Passive Driver Gaze Tracking with Active Appearance Models,
tech.report CMU-RI-TR-04-08,2004。
Gaze estimation method based on geometry mainly passes through positioning pupil center, upper palpebra inferior characteristic point and canthus position
Etc. calculating direction of visual lines.The characteristic point for positioning above-mentioned ocular mainly utilizes movable appearance model (Active
Appearance Model), this is a kind of side by face overall situation appearance positioning facial feature points (canthus point, corners of the mouth point etc.)
Method.The advantages of gaze estimation method based on geometry, is that sight accuracy of estimation is high when eye feature point location is accurate, and
Disadvantage be active appearance models vulnerable to illumination, block influence with posture, cause eye feature point location inaccurate.It is detailed in ginseng
Examine document: Takahiro Ishikawa, Simon Baker, Iain Matthews, and Takeo Kanade, Passive
Driver Gaze Tracking with Active Appearance Models,tech.report CMU-RI-TR-04-
08,2004 and Iain Matthews, and Simon Baker, Active Appearance Models Revisited,
International Journal of Computer Vision,Vol.60,No.2,pp.135-164,2004.
Gaze estimation method based on recurrence mainly passes through detection human eye area, and establish eyes image feature and sight it
Between mapping relations.Existing method mainly passes through support vector regression, Gaussian process recurrence etc. and establishes mapping relations.Based on recurrence
Gaze estimation method major advantage be it is simple under the premise of the human eye area accurate positioning easily realize, and the disadvantage is that existing
Homing method is difficult accurately to describe very much the mapping relations between human eye feature and sight.It is detailed in bibliography: Zhiwei
Zhu,Qiang Ji,and Kristin P.Bennett,Nonlinear Eye Gaze Mapping Function
Estimation via Support Vector Regression,The 18th International Conference on
Pattern Recognition, Vol.1, pp.1132-1135,2006. and Oliver Williams, Andrew Blake,
and Roberto Cipolla.Sparse and Semi-supervised Visual Mapping with the
S3GP.The 2006IEEE Computer Society Conference on Computer Vision and Pattern
Recognition,Vol.1,pp.230-237,2006.
In the gaze estimation method based on homing method, sixty-four dollar question be establish it is non-from eye feature to sight
Linear regression model (LRM).In nonlinear regression model (NLRM), present depth regression model has been found mould best to solve this problem
One of type.Due to its high accuracy, flexibility and strong versatility, depth method is currently widely used.In recent years, to encode certainly
Device is the research work of core, is more and more applied in practical problem.It is detailed in bibliography: Yan LeCun, Bengio
Yoshua Bengio,and Geoffrey Hinton,Deep Learning,Nature,Vol.521,pp 436-444。
Summary of the invention
The present invention provides a kind of gaze estimation methods based on depth network.Collected eyes image is carried out first
Size normalization；Later, depth is established between input eyes image gradient orientation histogram feature and corresponding sight return mould
Type；Then, the parameter of depth regression model is initialized, and solves depth model parameter using gradient descent method；Estimate finally, treating
It counts eyes image and extracts gradient orientation histogram feature, and utilize the depth regression model estimation direction of visual lines acquired.Algorithm
Schematic diagram is referring to fig. 2.
In order to easily describe the content of present invention, some terms are defined first.
Definition 1: direction of visual lines.The angle in eyeball fixes direction is usually indicated by vector in two-dimensional space, this to
Amount is made of two elements, and first element is level angle, and second element is vertical angle.
Definition 2: gradient orientation histogram feature.Piece image is described using the directional spreding of image pixel intensities gradient or edge
In object presentation and shape Visual Feature Retrieval Process method.Its implementation, which first divides the image into, small is called pane location
Connected region；Then the gradient direction or edge orientation histogram of each pixel in pane location are acquired；It is finally that these are straight
Square figure, which combines, can be formed by Feature Descriptor.It, can also be these local histograms in image in order to improve accuracy
Bigger section (block) in degree of comparing normalization (contrast-normalized), the method is each by first calculating
Density of the histogram in this section (block), then does each pane location in section according to this density value and returns
One changes.There can be stronger robustness to illumination variation and shade by the normalization.
Definition 3: depth Recurrent networks.Refer to the neural network successively returned, each layer of output is next layer of input.
Define 4:S shape function.Sigmoid function (sigmoid function) can generally be expressed as σ (), and expression formula is
Definition 5: back-propagation algorithm.It is a kind of supervised learning algorithm, is often used to train multilayer neural network.General packet
Containing two stages: (1) the propagated forward stage, which inputs training, is sent into network to obtain exciter response；(2) back-propagation phase will
Exciter response inputs corresponding target output with training and asks poor, to obtain the response error of hidden layer and output layer.
Definition 6: gradient descent method.It is a kind of unconfined optimization method, when solving objective function minimum value, finds ladder
Direction is spent, and is searched for along gradient opposite direction, the method until reaching local minimum.
A kind of gaze estimation method based on depth Recurrent networks according to the invention, it is comprised the steps of:
Step 1: acquisition N width includes the eyes image (see Fig. 1) of different sight, and records corresponding when acquisition each image
Direction of visual linesynOne-dimensional representation horizontal direction, two-dimensional representation vertical direction, subscript n indicate the n-th width image
Corresponding direction of visual lines；
Step 2: the eyes image normalized that will be acquired in step 1, and extract gradient orientation histogram feature
(Histogramof Oriented Gradient,HOG)。
Step 3: the range of the corresponding direction of visual lines of N width image is normalized into [0,1] section, specific practice are as follows:
WhereinIndicate the component of the calibration sight jth dimension of n-th of sample, ynjNumerical value after indicating dimension normalization.
Step 4: the corresponding mapping function of projected depth regression model is to input feature vectorWherein s1Indicate special
The dimension of sign successively maps input feature vector,
Indicate the input of l+1 layers of i-th of unit,Indicate connection deep neural network
L layers of all slParameter between a unit and l+1 layers of i-th of unit.Specifically,It indicates to connect l layers
Parameter between j-th of unit and l+1 i-th of unit of layer,For bias term relevant to l+1 layers of hidden unit i, sl+1
For the number of l+1 layers of hidden unit.Whether l+1 layers of i-th of unit is activated, and is determined by the output of sigmoid function, it may be assumed that
Above formula can also indicate are as follows:
The output layer of the depth regression model of this patent design shares 2 units, uses symbolIt indicates, to estimate
The level angle and vertical angle of direction of visual lines are counted, subscript (L) indicates the label of output layer.Entire depth regression model function
hW, b(xn) indicate to input as xnWhen sight estimated value, it may be assumed that
Step 5: by gradient orientation histogram feature [x normalized in step 31..., xN] as depth regression model
Input, corresponding calibration direction of visual lines are [y1..., yN], establish the objective function of depth regression model:
WhereinL is the layer of depth Recurrent networks
Number,
Step 6: inputting to indicate to work as xnWhen, any unit i of any one layer of l is to error sum of squares The size of contribution defines error termI=1 ..., sl, l=2 ..., L.For output
Layer (L layers), the corresponding error term of each unit i are as follows:
It indicatesDerivative calculate l=2 using Back Propagation Algorithm, each node at 3, L-1 layers
The corresponding error term of j
Following objective function J (w, b) is finally obtained about parameterWithPartial derivative
WhereinWithIt indicates to work as and input as xnWhen corresponding l layers of j-th of unit output and l+1 layer i-th
The corresponding error term of a unit.Objective function J (w, b) is finally obtained about parameter vector w, the gradient of bWith
Step 7: parameter w and b in order to acquire optimal deep neural network, our first initiation parameters, parameter are initial
Change value is the reconstructed error minimum so that input signal, obtains initial value w[0]And b[0]And then it is carried out using gradient descent method
Optimization；That is:
Wherein subscript [t] and [t+1] indicate the t times and t+1 iteration.Stop iteration when w and b meet the condition of convergence.
Step 8: for new eyes image, detecting ocular and extract gradient orientation histogram feature, numerical value is returned
One change after be sent into trained depth network, obtain corresponding direction of visual lines estimated value, and numberical range is reverted to-
90 ° Dao+90 °.
It should be understood that
Further, the specific method of affiliated step 2 is that the eyes image that will be acquired in step 1 is normalized to size and is
100 × 60 pixels, during gradient orientation histogram feature calculation, the parameter of areal is set as 2 × 2, Mei Yiqu
The number parameter of elementary area is set as 4 × 4 in domain, and the number of steering column is set as 9, and it is corresponding to finally obtain any piece image
Gradient orientation histogram feature dimension be 1152, and remember the corresponding gradient orientation histogram feature of any n-th width image to
AmountAnd then to every one-dimensional progress numerical value normalization, data area is compressed to [0,1] section, for n-th of sample
This, the data x of i-th dimensionniNormalize formula
For the minimum value in all sample i-th dimensions,It is similar fixed
Justice is maximum value in all sample i-th dimensions.
The stack self-encoding encoder mentioned in step 4, each layer of number of unit are respectively s1=1440, s2=300, s3=
250 and s4=200, output layer only has 2 units, s5=2.
When seeking depth network parameter using gradient descent method in step 7, the condition of convergence be front and back twice iteration parameter not
Change again, that is, reaches local best points；Gradient descent method optimizes method are as follows:
Wherein subscript [t] and [t+1] indicate the t times and t+1 iteration.Stop iteration when w and b meet the condition of convergence.
Innovation of the invention is:
It proposes to utilize depth Recurrent networks, establishes the Nonlinear Mapping relationship between eyes image and direction of visual lines.This hair
The bright N eyes image of acquisition first is normalized to the image that size is 100*60 as training sample, and depth image, mentions simultaneously
1152 dimension gradient orientation histogram features are taken, corresponding direction of visual lines is then recorded.Later, projected depth Recurrent networks, the depth
It spends network and removes input layer and output layer, totally 3 layers of middle layer.Then, on training sample and calibration direction of visual lines data, ladder is utilized
Each layer parameter of degree decline calligraphy learning depth Recurrent networks.Finally, for the eyes image of sight to be estimated, gradient direction is extracted
Histogram feature estimates direction of visual lines according to the above-mentioned depth Recurrent networks model succeeded in school.With traditional gaze estimation method
Compare, this method can simulation input feature to direction of visual lines complex mapping relation, effectively overcome shallow Model estimation
The not high problem of accuracy.
Detailed description of the invention
Fig. 1 is that sight estimates schematic diagram；
Fig. 2 is depth Recurrent networks schematic diagram.
Specific embodiment
According to the method for the present invention, the training pattern of depth Recurrent networks is write first with Matlab or C language；It connects
The collected training sample of input and training depth Recurrent networks parameter；Then it is straight gradient direction to be extracted to acquired image
Square figure feature is input in trained depth Recurrent networks as source data and is handled；The direction of visual lines estimated.This
The method of invention can be used in the sight estimation problem of eye in natural scene.
A kind of gaze estimation method based on depth Recurrent networks, it is comprised the steps of:
Step 1: acquisition N width includes the eyes image (see Fig. 1) of different sight, and records corresponding when acquisition each image
Direction of visual linesynOne-dimensional representation horizontal direction, two-dimensional representation vertical direction, subscript n indicate the n-th width image
Corresponding direction of visual lines；
Step 2: it is 100 × 60 pixels that the eyes image acquired in step 1, which is normalized to size, and extracts gradient direction
Histogram feature (Histogram of Oriented Gradient, HOG).In the process of gradient orientation histogram feature calculation
In, the parameter of areal is set as 2 × 2, and the number parameter of elementary area is set as 4 × 4 in each region, of steering column
Number is set as 9, and the dimension for finally obtaining the corresponding gradient orientation histogram feature of any piece image is 1152, and remembers any the
The corresponding gradient orientation histogram feature vector of n width imageAnd then it to every one-dimensional progress numerical value normalization, will count
According to Ratage Coutpressioit to [0,1] section, for n-th of sample, the data x of i-th dimensionniNormalize formula
For the minimum value in all sample i-th dimensions,It is similar fixed
Justice is maximum value in all sample i-th dimensions.
Step 3: the range of the corresponding direction of visual lines of N width image is normalized into [0,1] section, specific practice are as follows:
WhereinIndicate the component of the calibration sight jth dimension of n-th of sample, ynjNumerical value after indicating dimension normalization.
Step 4: the corresponding mapping function of projected depth regression model is to input feature vectorWherein s1Indicate special
The dimension of sign successively maps input feature vector,
Indicate the input of l+1 layers of i-th of unit,Indicate connection deep neural network
L layers of all slParameter between a unit and l+1 layers of i-th of unit.Specifically,It indicates to connect l layers
Parameter between j-th of unit and l+1 i-th of unit of layer,For bias term relevant to l+1 layers of hidden unit i, Sl+1
For the number of l+1 layers of hidden unit.Whether l+1 layers of i-th of unit is activated, and is determined by the output of sigmoid function, it may be assumed that
Above formula can also indicate are as follows:
The output layer of the depth regression model of this patent design shares 2 units, uses symbolIt indicates, to estimate
The level angle and vertical angle of direction of visual lines are counted, subscript (L) indicates the label of output layer.Entire depth regression model function
hW, b(xn) indicate to input as xnWhen sight estimated value, it may be assumed that
Step 5: by gradient orientation histogram feature [x normalized in step 31..., xN] as depth regression model
Input, corresponding calibration direction of visual lines are [y1..., yN], establish the objective function of depth regression model:
WhereinL is the layer of depth Recurrent networks
Number,
Step 6: inputting to indicate to work as xnWhen, any unit i of any one layer of l is to error sum of squares The size of contribution defines error termI=1 ..., sl, l=2 ..., L.For output
Layer (L layers), the corresponding error term of each unit i are as follows:
It indicatesDerivative calculate l=2 using Back Propagation Algorithm, each node at 3, L-1 layers
The corresponding error term of j
Following objective function J (w, b) is finally obtained about parameterWithPartial derivative
WhereinWithIt indicates to work as and input as xnWhen corresponding l layers of j-th of unit output and l+1 layer i-th
The corresponding error term of a unit.Objective function J (w, b) is finally obtained about parameter vector w, the gradient of bWith
Step 7: parameter w and b in order to acquire optimal deep neural network, our first initiation parameters, parameter are initial
Change value is the reconstructed error minimum so that input signal, obtains initial value w[0]And b[0]And then it is carried out using gradient descent method
Optimization, that is:
Wherein subscript [t] and [t+1] indicate the t times and t+1 iteration.Stop iteration when w and b meet the condition of convergence.
Step 8: for new eyes image, detecting ocular and extract gradient orientation histogram feature, numerical value is returned
One change after be sent into trained depth network, obtain corresponding direction of visual lines estimated value, and numberical range is reverted to-
90 ° Dao+90 °.
It should be understood that
The stack self-encoding encoder mentioned in step 3, each layer of number of unit are respectively s1=1440, s2=300, s3=
250 and s4=200, output layer only has 2 units, it may be assumed that s5=2.
When seeking depth network parameter using gradient descent method in step 7, the condition of convergence be front and back twice iteration parameter not
Change again, that is, reaches local best points.
Claims (3)
1. a kind of gaze estimation method based on depth Recurrent networks, comprising the following steps:
Step 1: acquisition N width includes the eyes image of different sight, and records corresponding direction of visual lines when acquisition each imageynOne-dimensional representation horizontal direction, two-dimensional representation vertical direction, subscript n indicates the corresponding view of the n-th width image
Line direction；
Step 2: the eyes image normalized that will be acquired in step 1, and gradient orientation histogram feature is extracted, obtain N width
Gradient orientation histogram feature [the x of image1..., xN]；
Step 3: the range that N width image corresponds to direction of visual lines being normalized into [0,1] section, obtaining calibration direction of visual lines is
[y1..., yN], specific practice are as follows:
WhereinIndicate the component of the calibration sight jth dimension of the n-th width image, ynjNumerical value after indicating dimension normalization, wherein for
Expression facilitates yniIt is expressed as yn；
Step 4: the corresponding mapping function of projected depth regression model is to input feature vectorWherein s1Indicate feature
Dimension successively maps input feature vector,
Indicate the input of l+1 layers of i-th of unit,Indicate l layers of deep neural network of connection
All slParameter between a unit and l+1 layers of i-th of unit；Specifically,Indicate j-th of l layers of connection
Parameter between unit and l+1 i-th of unit of layer,Indicate the output of the sigmoid function of l+1 layers of i-th of unit,
For bias term relevant to l+1 layers of hidden unit i, Sl+1For the number of l+1 layers of hidden unit；L+1 layers of i-th of unit
Whether it is activated, is determined by the output of sigmoid function, it may be assumed that
The output layer of the depth regression model of design shares 2 units, uses symbolIt indicates, to estimate direction of visual lines
Level angle and vertical angle, subscript (L) indicate output layer label；Entire depth regression model function hW, b(xn) table
Show to work as and input as xnWhen sight estimated value, it may be assumed thatσ () indicates sigmoid function；
Step 5: by the normalized gradient orientation histogram feature [x of step 21..., xN] input as depth regression model,
Corresponding calibration direction of visual lines is [y1..., yN], establish the objective function of depth regression model:
WhereinL is the number of plies of depth Recurrent networks,The intensity of λ Section 2 bound term；
Step 6: inputting to indicate to work as xnWhen, any unit i of any one layer of l is to error sum of squares The size of contribution defines an error termOutput layer is used as L layers
When, the corresponding error term of each unit i are as follows:
It indicatesDerivative,It indicates to work as and input as xnWhen L i-th of node of layer input, to biography after utilization
It broadcasts algorithm, calculates l=2, each node j corresponding error term at 3, L-1 layers
Following objective function J (w, b) is finally obtained about parameterWithPartial derivative
WhereinWithIt indicates to work as and input as xnWhen corresponding l layers of j-th of unit output and i-th of l+1 layer it is single
The corresponding error term of member；Objective function J (w, b) is finally obtained about parameter vector w, the gradient of bWith
Step 7: parameter w and b in order to acquire optimal deep neural network, our first initiation parameters, parameter initialization value
It is the reconstructed error minimum so that input signal, obtains initial value w[0]And b[0]And then it is optimized using gradient descent method；
That is:
Wherein subscript [t] and [t+1] indicate the t times and t+1 iteration；Stop iteration when w and b meet the condition of convergence, α is indicated
The step-length of gradient decline；
Step 8: for new eyes image, detects ocular and extract gradient orientation histogram feature, numerical value normalization
Be sent into trained depth network later, obtain corresponding direction of visual lines estimated value, and by numberical range revert to -90 ° to+
90°。
2. a kind of gaze estimation method based on depth Recurrent networks as described in claim 1, it is characterised in that the step 2
Method particularly includes: it is 100 × 60 pixels that the eyes image acquired in step 1, which is normalized to size, in gradient direction histogram
During figure feature calculation, the parameter of areal is set as 2 × 2, the number parameter setting of elementary area in each region
It is 4 × 4, the number of steering column is set as 9, finally obtains the dimension of the corresponding gradient orientation histogram feature of any piece image
It is 1152, and remembers the corresponding gradient orientation histogram feature vector of any n-th width imageAnd then to it is every it is one-dimensional into
The normalization of line number value, is compressed to [0,1] section for data area, for the n-th width image, the data x of i-th dimensionniNormalization is public
Formula
For the minimum value in all sample i-th dimensions,It is similar to be defined as
Maximum value in all sample i-th dimensions,Indicate the n-th width image, the gradient orientation histogram feature vector of the data of i-th dimension,
To indicate to facilitate xniIt is expressed as xn。
3. a kind of gaze estimation method based on depth Recurrent networks as described in claim 1, it is characterised in that sharp in step 7
When seeking depth network parameter with gradient descent method, the condition of convergence is that the parameter of front and back iteration twice no longer changes, that is, reaches part
Optimum point；Gradient descent method optimizes method are as follows:
Wherein subscript [t] and [t+1] indicate the t times and t+1 iteration；Stop iteration when w and b meet the condition of convergence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611036387.1A CN106599994B (en) | 2016-11-23 | 2016-11-23 | A kind of gaze estimation method based on depth Recurrent networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611036387.1A CN106599994B (en) | 2016-11-23 | 2016-11-23 | A kind of gaze estimation method based on depth Recurrent networks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106599994A CN106599994A (en) | 2017-04-26 |
| CN106599994B true CN106599994B (en) | 2019-02-15 |
Family
ID=58591880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611036387.1A Expired - Fee Related CN106599994B (en) | 2016-11-23 | 2016-11-23 | A kind of gaze estimation method based on depth Recurrent networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106599994B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409173B (en) * | 2017-08-18 | 2021-06-04 | 安徽三联交通应用技术股份有限公司 | Driver state monitoring method, system, medium and equipment based on deep learning |
| CN109409172B (en) * | 2017-08-18 | 2021-08-13 | 安徽三联交通应用技术股份有限公司 | Driver sight line detection method, system, medium, and apparatus |
| CN108345843B (en) * | 2018-01-25 | 2020-12-29 | 电子科技大学 | Head posture estimation method based on mixed depth regression network |
| EP3750029A1 (en) | 2018-02-09 | 2020-12-16 | Pupil Labs GmbH | Devices, systems and methods for predicting gaze-related parameters using a neural network |
| WO2019154509A1 (en) | 2018-02-09 | 2019-08-15 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| US11393251B2 (en) | 2018-02-09 | 2022-07-19 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| CN108681699A (en) * | 2018-05-04 | 2018-10-19 | 上海像我信息科技有限公司 | A kind of gaze estimation method and line-of-sight estimation device based on deep learning |
| CN111277857B (en) * | 2018-12-04 | 2021-04-13 | 清华大学 | Streaming media scheduling method and device |
| EP3912013A1 (en) | 2019-01-16 | 2021-11-24 | Pupil Labs GmbH | Methods for generating calibration data for head-wearable devices and eye tracking system |
| US11676422B2 (en) | 2019-06-05 | 2023-06-13 | Pupil Labs Gmbh | Devices, systems and methods for predicting gaze-related parameters |
| CN110347837B (en) * | 2019-07-17 | 2022-02-18 | 电子科技大学 | Cardiovascular disease unplanned hospitalization risk prediction method |
| CN110533065A (en) * | 2019-07-18 | 2019-12-03 | 西安电子科技大学 | Based on the shield attitude prediction technique from coding characteristic and deep learning regression model |
| CN114241179A (en) * | 2021-12-06 | 2022-03-25 | 电子科技大学 | Sight estimation method based on self-learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120166083A1 (en) * | 2010-12-28 | 2012-06-28 | Aisin Aw Co., Ltd. | Navigation device, navigation method, and navigation program |
| CN102830793A (en) * | 2011-06-16 | 2012-12-19 | 北京三星通信技术研究有限公司 | Sight tracking method and sight tracking device |
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105354548A (en) * | 2015-10-30 | 2016-02-24 | 武汉大学 | Surveillance video pedestrian re-recognition method based on ImageNet retrieval |
| CN106056039A (en) * | 2016-05-18 | 2016-10-26 | 电子科技大学 | Robust mixed regression method for line of sight estimation |
-
2016
- 2016-11-23 CN CN201611036387.1A patent/CN106599994B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120166083A1 (en) * | 2010-12-28 | 2012-06-28 | Aisin Aw Co., Ltd. | Navigation device, navigation method, and navigation program |
| CN102830793A (en) * | 2011-06-16 | 2012-12-19 | 北京三星通信技术研究有限公司 | Sight tracking method and sight tracking device |
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105354548A (en) * | 2015-10-30 | 2016-02-24 | 武汉大学 | Surveillance video pedestrian re-recognition method based on ImageNet retrieval |
| CN106056039A (en) * | 2016-05-18 | 2016-10-26 | 电子科技大学 | Robust mixed regression method for line of sight estimation |
Non-Patent Citations (3)
| Title |
|---|
| 3D Road Environment Modeling Applied to Visibility Mapping: an Experimental;Jean-Philippe Tarel等;《ACM》;20121231;全文 * |
| Eye Gaze Calculation Based on nonlinear polynomial and Generalized Regression Neural Network;Chi Jian-nan等;《IEEE》;20091231;全文 * |
| 基于单目摄像头的驾驶员注视区域估计方法;冮俊冶;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160715;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106599994A (en) | 2017-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106599994B (en) | A kind of gaze estimation method based on depth Recurrent networks | |
| CN106897675B (en) | Face living body detection method combining binocular vision depth characteristic and apparent characteristic | |
| CN103942577B (en) | Based on the personal identification method for establishing sample database and composite character certainly in video monitoring | |
| CN110210551A (en) | A kind of visual target tracking method based on adaptive main body sensitivity | |
| CN108182397B (en) | Multi-pose multi-scale human face verification method | |
| CN109102547A (en) | Robot based on object identification deep learning model grabs position and orientation estimation method | |
| CN107145842A (en) | With reference to LBP characteristic patterns and the face identification method of convolutional neural networks | |
| CN108665481A (en) | Multilayer depth characteristic fusion it is adaptive resist block infrared object tracking method | |
| CN104268539A (en) | High-performance human face recognition method and system | |
| CN103971106B (en) | Various visual angles facial image gender identification method and device | |
| CN106599810B (en) | A kind of head pose estimation method encoded certainly based on stack | |
| CN109389045A (en) | Micro- expression recognition method and device based on mixing space-time convolution model | |
| Sang et al. | Pose-invariant face recognition via RGB-D images | |
| CN109255319A (en) | For the recognition of face payment information method for anti-counterfeit of still photo | |
| CN106599785A (en) | Method and device for building human body 3D feature identity information database | |
| CN108921064B (en) | Pedestrian re-identification method based on multi-feature fusion | |
| CN104036299B (en) | A kind of human eye contour tracing method based on local grain AAM | |
| Ravi et al. | Sign language recognition with multi feature fusion and ANN classifier | |
| He et al. | Human segmentation of infrared image for mobile robot search | |
| Ejbali et al. | Face recognition based on beta 2D elastic bunch graph matching | |
| CN106056039A (en) | Robust mixed regression method for line of sight estimation | |
| CN105404883B (en) | A kind of heterogeneous three-dimensional face identification method | |
| Pathak et al. | Entropy based CNN for segmentation of noisy color eye images using color, texture and brightness contour features | |
| Baqar et al. | Efficient iris recognition system based on dual boundary detection using robust variable learning rate multilayer feed forward neural network | |
| Lin et al. | An object tracking method based on CNN and optical flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190215 Termination date: 20211123 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |