CN113095274A - Sight estimation method, system, device and storage medium - Google Patents
Sight estimation method, system, device and storage medium Download PDFInfo
- Publication number
- CN113095274A CN113095274A CN202110450755.1A CN202110450755A CN113095274A CN 113095274 A CN113095274 A CN 113095274A CN 202110450755 A CN202110450755 A CN 202110450755A CN 113095274 A CN113095274 A CN 113095274A
- Authority
- CN
- China
- Prior art keywords
- vector
- eye image
- human eye
- estimation
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 239000013598 vector Substances 0.000 claims abstract description 75
- 241000282414 Homo sapiens Species 0.000 claims abstract description 69
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 210000003128 head Anatomy 0.000 claims description 61
- 239000011159 matrix material Substances 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000193 eyeblink Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a sight line estimation method, a system, a device and a storage medium, wherein the method comprises the following steps: obtaining a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector; carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector; the regularized eye images and head pose estimation vectors are input to a pre-trained CNN network, and the network output is converted to a 3D gaze direction vector. The system comprises: the device comprises an image preprocessing module, a data regularization module and a result output module. The apparatus includes a memory and a processor for performing the gaze estimation method described above. By using the invention, a high-precision sight line estimation result can be obtained. The sight line estimation method, the sight line estimation system, the sight line estimation device and the storage medium can be widely applied to the field of sight line estimation.
Description
Technical Field
The present invention relates to the field of gaze estimation, and in particular, to a gaze estimation method, system, apparatus, and storage medium.
Background
The sight line estimation technology is a technology for researching how to accurately track the visual direction and the visual attention of human beings, has wide application scenes and great application value in actual life, can be applied to the fields of cognitive science, psychology, medical research, automobile driving, entertainment, advertising, marketing research and the like, brings convenience to the life of people, comprehensively improves the social technological level, and is accompanied with the continuous improvement of an optical imaging technology and image processing capacity, particularly the development of computer vision, the sight line estimation method based on images starts to become dominant.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a gaze estimation method, system, device and storage medium, which have high accuracy, no calibration, and simple operation.
The first technical scheme adopted by the invention is as follows: a gaze estimation method, comprising the steps of:
obtaining a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
the regularized eye images and head pose estimation vectors are input to a pre-trained CNN network, and the network output is converted to a 3D gaze direction vector.
Further, the step of obtaining a human face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector specifically includes:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and a two-dimensional coordinate of a face key point corresponding to an image is obtained;
acquiring an eye image according to the eye key point position in the two-dimensional coordinates of the face key point;
acquiring a 3D face key point model;
and fitting the two-dimensional coordinates of the key points of the human face with the 3D key point model based on an EPnP algorithm to obtain a 3D head rotation vector.
Further, before regularizing the image of the eye, the method further comprises the step of detecting the eye blink and screening the image of the eye, and specifically comprises the following steps:
obtaining a horizontal line and a vertical line which pass through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is greater than a preset threshold value, determining that the human eye image is in an eye opening state, and performing sight line estimation;
and determining that the ratio is smaller than a preset threshold value, determining that the human eye image is in an eye closing state, and not performing sight line estimation.
Further, the formula for data regularization is as follows:
M＝S*R
in the above equation, R denotes an inverse matrix of the camera rotation matrix, and S denotes a scaling matrix.
Further, the step of performing data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head pose estimation vector specifically includes:
processing the image of the person eye and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
then scaling the camera coordinate system by an S scaling matrix;
and finally obtaining the regularized human eye image and the head posture estimation vector through perspective transformation.
Further, the training step of the pre-trained CNN network specifically includes:
acquiring a human eye image with a real sight angle label and a head posture estimation vector, and inputting the human eye image and the head posture estimation vector into a CNN network to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain a trained sight estimation model.
Further, the step of inputting the regularized eye images and the head pose estimation vector into a pre-trained CNN network and converting the network output into a 3D gaze direction vector specifically includes:
inputting the regularized human eye image and the head pose estimation vector into a pre-trained CNN network;
obtaining eye characteristics through convolution of the convolution layer and compression of the pooling layer;
splicing the head posture estimation vector with the extracted eye features through a full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
The second technical scheme adopted by the invention is as follows: a gaze estimation system, comprising:
the image preprocessing module is used for acquiring a face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into the pre-trained CNN network and converting the network output into a 3D sight line direction vector.
The third technical scheme adopted by the invention is as follows: a gaze estimation device, comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, the at least one program causes the at least one processor to implement a gaze estimation method as described above.
The fourth technical scheme adopted by the invention is as follows: a storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by the processor, are for implementing a gaze estimation method as described above.
The method, the system, the device and the storage medium have the advantages that: the invention firstly judges whether a human face exists, if so, the positions of a plurality of key points of the eyes are determined to carry out human eye detection, and finally the obtained eye image is input into a CNN network through clipping so as to realize the sight estimation.
Drawings
FIG. 1 is a flow chart of the steps of a gaze estimation method of the present invention;
FIG. 2 is a schematic diagram of a gaze estimation method in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of data regularization of a human eye image according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a regularizing human eye (left and right eye) image according to an embodiment of the present invention;
FIG. 5 is a diagram of 68 key points of a face according to an embodiment of the present invention;
fig. 6 is a block diagram of a sight line estimation system according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
Referring to fig. 1 and 2, the present invention provides a gaze estimation method, including the steps of:
s1, obtaining a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
s2, carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
s3, inputting the regularized human eye image and the head pose estimation vector into a pre-trained CNN network, and converting the network output into a 3D sight line direction vector.
Further, as a preferred embodiment of the method, the step of obtaining the face image and performing the key point detection and the 3D model fitting process to obtain the eye image and the 3D head rotation vector specifically includes:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and a two-dimensional coordinate of a face key point corresponding to an image is obtained;
acquiring an eye image according to the eye key point position in the two-dimensional coordinates of the face key point;
acquiring a 3D face key point model;
specifically, a 3D-FAN network is adopted to perform fine adjustment on data sets such as 300W, 300W-LP-3D and the like, and 68 personal face key point models (namely average face models) required by the text are obtained.
And fitting the two-dimensional coordinates of the key points of the human face with the 3D key point model based on an EPnP algorithm to obtain a 3D head rotation vector.
The EPnP algorithm is a weighted sum of n three-dimensional spatial points represented as 4 virtual control points. Then, the coordinates of the 4 control points in the camera coordinate system need to be estimated, and the coordinates of the control points in the camera coordinate system can be obtained by weighting and adding eigenvectors representing a 12-by-12 matrix of the coordinates and solving a small constant quadratic equation to select correct weights. And finally, according to Euclidean motion of the camera coordinate system and the world coordinate system, a translation vector and a rotation matrix of the coordinate system can be solved.
Referring to fig. 3, a head coordinate system (X) of the average face modelh，Yh，Zh) The definition mode is as follows: origin at the tip of the nose, ZhThe axis being perpendicular to the plane formed by the three midpoints of the eyes and mouth, XhThe axis being parallel to a line passing through the midpoints of the eyes, YhAxis perpendicular to ZhAxis and XhAxis, coordinate system unit is meter. And the external eye angle distance of the model is set to be 90 mm. Wherein the triangleThe shape area is a plane formed by three middle points of the eyes and the mouth; the dots are sequentially from top to bottom and from left to right: the external canthus of the left and right eyes, the tip of the nose and two key points of the mouth.
As a further preferred embodiment of the method, before regularizing the image of the human eye, the method further includes a step of performing blink detection and screening on the image of the human eye, which specifically includes:
obtaining a horizontal line and a vertical line which pass through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is greater than a preset threshold value, determining that the human eye image is in an eye opening state, and performing sight line estimation;
and determining that the ratio is smaller than a preset threshold value, determining that the human eye image is in an eye closing state, and not performing sight line estimation.
Specifically, referring to FIG. 5, based on face keypoint detection, we can determine 68 specific face keypoints, each with a specific index. Therefore, we can get several key point indexes of the left and right eyes as (36,37,38,39,40,41) and (42,43,44,45,46,47), respectively, and when the eyes are opened and closed, the length of the horizontal line is almost constant, while the vertical line is different. When the eyes are open, the vertical line length is much longer than when closed. The eyes are closed and the vertical line length is almost zero.
Further as a preferred embodiment of the method, the formula of data regularization is as follows:
M＝S*R
in the above equation, R represents the inverse of the camera rotation matrix, which makes the x-axis of the head coordinate system perpendicular to the y-axis of the camera coordinate system, the camera z-axis towards the eye position, and S represents the scaling matrix, so that the distance of the eye to the camera coordinate system remains fixed.
Further, as a preferred embodiment of the method, the step of performing data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head pose estimation vector specifically includes:
processing the image of the person eye and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
then scaling the camera coordinate system by an S scaling matrix;
and finally obtaining the regularized human eye image and the head posture estimation vector through perspective transformation.
Specifically, in order to achieve high-precision sight line estimation under different camera parameters, data regularization is required, that is, the distance between the camera and the position of human eyes is ensured to be fixed by regularizing the input image, the x axis of the head coordinate system is perpendicular to the y axis of the camera coordinate system, and the z axis of the camera faces the eyes.
Image regularization step schematic referring to fig. 3 and 4, (a) from the head coordinate system (top) and camera coordinate system (bottom) centered on the tip of the nose; (b) the camera coordinate system is rotated by a rotation matrix; (c) then scaling the camera coordinate system by an S scaling matrix; (d) and finally obtaining the regularized eye image through perspective transformation.
As a further preferred embodiment of the method, the training step of the pre-trained CNN network specifically includes:
acquiring a human eye image with a real sight angle label and a head posture estimation vector, and inputting the human eye image and the head posture estimation vector into a CNN network to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain a trained sight estimation model.
As a preferred embodiment of the method, the step of inputting the regularized eye image and the head pose estimation vector into a pre-trained CNN network and converting the network output into a 3D gaze direction vector specifically includes:
inputting the regularized human eye image and the head pose estimation vector into a pre-trained CNN network;
obtaining eye characteristics through convolution of the convolution layer and compression of the pooling layer;
specifically, the convolutional layer performs convolution operation, extracts eye features, and compresses input features and extracts main features through a pooling layer, thereby simplifying the network computation complexity.
Splicing the head posture estimation vector with the extracted eye features through a full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
As shown in fig. 6, a sight line estimation system includes:
the image preprocessing module is used for acquiring a face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into the pre-trained CNN network and converting the network output into a 3D sight line direction vector.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
A sight line estimation device:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, the at least one program causes the at least one processor to implement a gaze estimation method as described above.
The contents in the above method embodiments are all applicable to the present apparatus embodiment, the functions specifically implemented by the present apparatus embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present apparatus embodiment are also the same as those achieved by the above method embodiments.
A storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by the processor, are for implementing a gaze estimation method as described above.
The contents in the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those in the above method embodiments, and the advantageous effects achieved by the present storage medium embodiment are also the same as those achieved by the above method embodiments.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A gaze estimation method, characterized by comprising the steps of:
obtaining a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
the regularized eye images and head pose estimation vectors are input to a pre-trained CNN network, and the network output is converted to a 3D gaze direction vector.
2. The gaze estimation method of claim 1, wherein the step of obtaining a human face image and performing keypoint detection and 3D model fitting to obtain a human eye image and a 3D head rotation vector comprises:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and a two-dimensional coordinate of a face key point corresponding to an image is obtained;
acquiring an eye image according to the eye key point position in the two-dimensional coordinates of the face key point;
acquiring a 3D face key point model;
and fitting the two-dimensional coordinates of the key points of the human face with the 3D key point model based on an EPnP algorithm to obtain a 3D head rotation vector.
3. The gaze estimation method of claim 2, further comprising the step of blink detection and screening of the images before regularizing the images, in particular comprising:
obtaining a horizontal line and a vertical line which pass through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is greater than a preset threshold value, determining that the human eye image is in an eye opening state, and performing sight line estimation;
and determining that the ratio is smaller than a preset threshold value, determining that the human eye image is in an eye closing state, and not performing sight line estimation.
4. The gaze estimation method of claim 3, wherein the data regularization is formulated as follows:
M＝S*R
in the above equation, R denotes an inverse matrix of the camera rotation matrix, and S denotes a scaling matrix.
5. The gaze estimation method of claim 4, wherein the step of regularizing the images of the person's eye and the 3D head rotation vectors to obtain regularized images of the person's eye and head pose estimation vectors comprises:
processing the image of the person eye and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
then scaling the camera coordinate system by an S scaling matrix;
and finally obtaining the regularized human eye image and the head posture estimation vector through perspective transformation.
6. The gaze estimation method of claim 5, characterized in that the training step of the pre-trained CNN network specifically comprises:
acquiring a human eye image with a real sight angle label and a head posture estimation vector, and inputting the human eye image and the head posture estimation vector into a CNN network to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain a trained sight estimation model.
7. The method of claim 6, wherein the step of inputting the regularized eye images and the head pose estimation vectors into a pre-trained CNN network and converting the network output into a 3D gaze direction vector comprises:
inputting the regularized human eye image and the head pose estimation vector into a pre-trained CNN network;
obtaining eye characteristics through convolution of the convolution layer and compression of the pooling layer;
splicing the head posture estimation vector with the extracted eye features through a full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
8. A gaze estimation system, comprising:
the image preprocessing module is used for acquiring a face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into the pre-trained CNN network and converting the network output into a 3D sight line direction vector.
9. A gaze estimation device, comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement a gaze estimation method as claimed in any one of claims 1-7.
10. A storage medium having stored therein instructions executable by a processor, the storage medium comprising: the processor-executable instructions, when executed by the processor, are for implementing a gaze estimation method as claimed in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110450755.1A CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110450755.1A CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113095274A true CN113095274A (en) | 2021-07-09 |
| CN113095274B CN113095274B (en) | 2024-02-09 |
Family
ID=76680139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110450755.1A Active CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095274B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822174A (en) * | 2021-09-02 | 2021-12-21 | 北京的卢深视科技有限公司 | Gaze estimation method, electronic device, and storage medium |
| CN114967935A (en) * | 2022-06-29 | 2022-08-30 | 深圳职业技术学院 | Interaction method and device based on sight estimation, terminal equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171218A (en) * | 2018-01-29 | 2018-06-15 | 深圳市唯特视科技有限公司 | A kind of gaze estimation method for watching network attentively based on appearance of depth |
| CN108171152A (en) * | 2017-12-26 | 2018-06-15 | 深圳大学 | Deep learning human eye sight estimation method, equipment, system and readable storage medium storing program for executing |
| CN108875524A (en) * | 2018-01-02 | 2018-11-23 | 北京旷视科技有限公司 | Gaze estimation method, device, system and storage medium |
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A kind of convolutional neural networks gaze estimation method and system based on attention mechanism |
| WO2020228224A1 (en) * | 2019-05-11 | 2020-11-19 | 初速度（苏州）科技有限公司 | Face part distance measurement method and apparatus, and vehicle-mounted terminal |
| CN111985403A (en) * | 2020-08-20 | 2020-11-24 | 中再云图技术有限公司 | Distracted driving detection method based on face posture estimation and sight line deviation |
| CN112257696A (en) * | 2020-12-23 | 2021-01-22 | 北京万里红科技股份有限公司 | Sight estimation method and computing equipment |
| CN112488067A (en) * | 2020-12-18 | 2021-03-12 | 北京的卢深视科技有限公司 | Face pose estimation method and device, electronic equipment and storage medium |
-
2021
- 2021-04-26 CN CN202110450755.1A patent/CN113095274B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171152A (en) * | 2017-12-26 | 2018-06-15 | 深圳大学 | Deep learning human eye sight estimation method, equipment, system and readable storage medium storing program for executing |
| CN108875524A (en) * | 2018-01-02 | 2018-11-23 | 北京旷视科技有限公司 | Gaze estimation method, device, system and storage medium |
| CN108171218A (en) * | 2018-01-29 | 2018-06-15 | 深圳市唯特视科技有限公司 | A kind of gaze estimation method for watching network attentively based on appearance of depth |
| WO2020228224A1 (en) * | 2019-05-11 | 2020-11-19 | 初速度（苏州）科技有限公司 | Face part distance measurement method and apparatus, and vehicle-mounted terminal |
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A kind of convolutional neural networks gaze estimation method and system based on attention mechanism |
| CN111985403A (en) * | 2020-08-20 | 2020-11-24 | 中再云图技术有限公司 | Distracted driving detection method based on face posture estimation and sight line deviation |
| CN112488067A (en) * | 2020-12-18 | 2021-03-12 | 北京的卢深视科技有限公司 | Face pose estimation method and device, electronic equipment and storage medium |
| CN112257696A (en) * | 2020-12-23 | 2021-01-22 | 北京万里红科技股份有限公司 | Sight estimation method and computing equipment |
Non-Patent Citations (1)
| Title |
|---|
| 朱玉斌;延向军;申旭奇;卢兆林;: "基于级联宽度学习的疲劳驾驶检测", 计算机工程与设计, no. 02, pages 245 - 249 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822174A (en) * | 2021-09-02 | 2021-12-21 | 北京的卢深视科技有限公司 | Gaze estimation method, electronic device, and storage medium |
| CN114967935A (en) * | 2022-06-29 | 2022-08-30 | 深圳职业技术学院 | Interaction method and device based on sight estimation, terminal equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113095274B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | Unsupervised representation learning for gaze estimation | |
| CN108876879B (en) | Method and device for realizing human face animation, computer equipment and storage medium | |
| US11600013B2 (en) | Facial features tracker with advanced training for natural rendering of human faces in real-time | |
| CN106068514B (en) | System and method for identifying face in free media | |
| CN104978548B (en) | A kind of gaze estimation method and device based on three-dimensional active shape model | |
| WO2020103700A1 (en) | Image recognition method based on micro facial expressions, apparatus and related device | |
| CN109684969B (en) | Gaze position estimation method, computer device, and storage medium | |
| US20150035825A1 (en) | Method for real-time face animation based on single video camera | |
| CN112766160A (en) | Face replacement method based on multi-stage attribute encoder and attention mechanism | |
| WO2001099048A2 (en) | Non-linear morphing of faces and their dynamics | |
| CN112614213A (en) | Facial expression determination method, expression parameter determination model, medium and device | |
| CN113095274A (en) | Sight estimation method, system, device and storage medium | |
| CN111046734A (en) | Multi-modal fusion sight line estimation method based on expansion convolution | |
| CN111754637B (en) | Large-scale three-dimensional face synthesis system with suppressed sample similarity | |
| CN113610046B (en) | Behavior recognition method based on depth video linkage characteristics | |
| CN115661246A (en) | Attitude estimation method based on self-supervision learning | |
| CN111626152A (en) | Space-time sight direction estimation prototype design based on Few-shot | |
| Tomar et al. | Deep hyfeat based attention in attention model for face super-resolution | |
| Ham et al. | Learning a manifold-constrained map between image sets: applications to matching and pose estimation | |
| CN113591797B (en) | Depth video behavior recognition method | |
| CN113807251A (en) | Sight estimation method based on appearance | |
| Park | Representation learning for webcam-based gaze estimation | |
| Somepalli et al. | Implementation of single camera markerless facial motion capture using blendshapes | |
| CN111739168B (en) | Large-scale three-dimensional face synthesis method with suppressed sample similarity | |
| CN116110108A (en) | Intelligent man-machine interaction method based on viewpoint tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |