CN113095274B - Sight estimation method, system, device and storage medium - Google Patents
Sight estimation method, system, device and storage medium Download PDFInfo
- Publication number
- CN113095274B CN113095274B CN202110450755.1A CN202110450755A CN113095274B CN 113095274 B CN113095274 B CN 113095274B CN 202110450755 A CN202110450755 A CN 202110450755A CN 113095274 B CN113095274 B CN 113095274B
- Authority
- CN
- China
- Prior art keywords
- human eye
- sight
- eye image
- vector
- estimation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 241000282414 Homo sapiens Species 0.000 claims abstract description 82
- 239000013598 vector Substances 0.000 claims abstract description 73
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 210000003128 head Anatomy 0.000 claims description 58
- 239000011159 matrix material Substances 0.000 claims description 19
- 230000009466 transformation Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 4
- 238000003475 lamination Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a sight line estimation method, a sight line estimation system, a sight line estimation device and a storage medium, wherein the sight line estimation method comprises the following steps: acquiring a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector; carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector; the regularized eye image and the head pose estimation vector are input to a pre-trained CNN network and the network output is converted into a 3D gaze direction vector. The system comprises: the device comprises an image preprocessing module, a data regularization module and a result output module. The apparatus includes a memory and a processor for performing the line-of-sight estimation method described above. By using the method and the device, the high-precision sight line estimation result can be obtained. The sight line estimation method, the sight line estimation system, the sight line estimation device and the storage medium can be widely applied to the field of sight line estimation.
Description
Technical Field
The present invention relates to the field of line of sight estimation, and in particular, to a line of sight estimation method, system, apparatus, and storage medium.
Background
The visual line estimation technology is a technology for researching how to accurately track the visual direction and visual attention of human beings, has wide application scenes and huge application value in actual life, can be applied to the fields of cognition science, psychology, medical research, automobile driving, entertainment, advertisement, marketing research and the like, brings convenience to life, comprehensively improves social science and technology level, and is accompanied with continuous improvement of optical imaging technology and image processing capability, particularly development of computer vision, the visual line estimation method based on images starts to be dominant, and the current visual line estimation method comprises a model-based method and a visual line based on expression, but the existing method has the problems of low estimation precision, slower estimation speed, strong scene dependence, complex experimental process, poor user experience and the like.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a sight line estimation method, a sight line estimation system, a sight line estimation device and a sight line estimation storage medium, which have high precision, do not need calibration and are simple to operate.
The first technical scheme adopted by the invention is as follows: a line-of-sight estimation method, comprising the steps of:
acquiring a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
the regularized eye image and the head pose estimation vector are input to a pre-trained CNN network and the network output is converted into a 3D gaze direction vector.
Further, the step of obtaining a human face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector specifically includes:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and two-dimensional coordinates of the face key points corresponding to the image are obtained;
acquiring a human eye image according to the positions of eye key points in the two-dimensional coordinates of the human face key points;
acquiring a 3D face key point model;
and fitting the two-dimensional coordinates of the face key points and the 3D face key point model based on an EPnP algorithm to obtain a 3D head rotation vector.
Further, before regularization of the data of the human eye image, the method further comprises the steps of blink detection and screening of the human eye image, and specifically comprises the following steps:
obtaining a horizontal line and a vertical line passing through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is larger than a preset threshold value, determining that the human eye image is in an eye opening state, and estimating the sight line;
and judging that the ratio is smaller than a preset threshold value, determining that the human eye image is in a closed eye state, and not estimating the sight line.
Further, the formula for data regularization is as follows:
M＝S*R
in the above equation, R represents an inverse of the camera rotation matrix, and S represents a scaling matrix.
Further, the step of regularizing the data of the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector specifically includes:
processing the human eye image and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
scaling the camera coordinate system by an S scaling matrix;
and finally obtaining regularized human eye images and head gesture estimation vectors through perspective transformation.
Further, the training step of the pre-trained CNN network specifically includes:
the method comprises the steps of obtaining a human eye image with a real sight angle label and inputting a head posture estimation vector to a CNN (computer numerical network) to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain the trained sight estimation model.
Further, the step of inputting the regularized eye image and the head pose estimation vector into a pre-trained CNN network and converting the network output into a 3D gaze direction vector specifically includes:
inputting regularized eye images and head pose estimation vectors into a pre-trained CNN network;
the eye characteristics are obtained through convolution of a convolution layer and lamination of a pooling layer;
splicing the head posture estimation vector with the extracted eye characteristics through the full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
The second technical scheme adopted by the invention is as follows: a gaze estimation system, comprising:
the image preprocessing module is used for acquiring a human face image, performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into a pre-trained CNN network and converting network output into a 3D sight direction vector.
The third technical scheme adopted by the invention is as follows: a line-of-sight estimation apparatus comprising:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement a gaze estimation method as described above.
The fourth technical scheme adopted by the invention is as follows: a storage medium having stored therein instructions executable by a processor, characterized by: the processor executable instructions when executed by the processor are for implementing a gaze estimation method as described above.
The method, the system, the device and the storage medium have the beneficial effects that: the invention firstly judges whether the face exists, then carries out human eye detection by determining a plurality of key points of eyes, finally inputs the obtained eye image into a CNN network through cutting so as to realize sight estimation, screens closed-eye pictures through blink detection, and enables the sight direction estimation to be more reasonable and accurate.
Drawings
FIG. 1 is a flow chart of the steps of a line-of-sight estimation method of the present invention;
FIG. 2 is a schematic diagram of a line-of-sight estimation method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of data regularization of a human eye image in accordance with an embodiment of the invention;
FIG. 4 is a schematic illustration of regularized human eye (left and right eye) images in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of 68 face keypoints according to an embodiment of the invention;
fig. 6 is a block diagram of the structure of a line-of-sight estimating system of the present invention.
Detailed Description
The invention will now be described in further detail with reference to the drawings and to specific examples. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art.
Referring to fig. 1 and 2, the present invention provides a line of sight estimation method, comprising the steps of:
s1, acquiring a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
s2, carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
s3, inputting regularized human eye images and head posture estimation vectors into a pre-trained CNN network, and converting network output into 3D sight direction vectors.
Further as a preferred embodiment of the method, the step of obtaining a human face image and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector specifically includes:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and two-dimensional coordinates of the face key points corresponding to the image are obtained;
acquiring a human eye image according to the positions of eye key points in the two-dimensional coordinates of the human face key points;
acquiring a 3D face key point model;
specifically, a 3D-FAN network is adopted to carry out fine adjustment on data sets of 300W, 300W-LP-3D and the like, so as to obtain 68 face key point models (namely average face models) required by the method.
And fitting the two-dimensional coordinates of the face key points and the 3D face key point model based on an EPnP algorithm to obtain a 3D head rotation vector.
Wherein, the EPnP algorithm is a weighted sum of n three-dimensional space points expressed as 4 virtual control points. Then, the coordinates of the 4 control points in the camera coordinate system need to be estimated, and the coordinates of the control points in the camera coordinate system can be obtained by expressing the coordinates as a weighted sum of feature vectors of a 12 x 12 matrix and solving a small constant quadratic equation to select the correct weight. Finally, according to Euclidean motions of a camera coordinate system and a world coordinate system, a translation vector and a rotation matrix of the coordinate system can be solved.
Referring to fig. 3, a head coordinate system (X h ，Y h ，Z h ) The definition mode is as follows: origin at the tip of the nose, Z h The axis is perpendicular to the plane formed by the three midpoints of eyes and mouth, X h The axis being parallel to a line passing through the midpoint of the eyes, Y h Perpendicular to axis Z h Axis and X h Axes, coordinates are in meters. And the distance between the outer corners of the model is set to be 90mm. Wherein the triangular area is a plane formed by three midpoints of eyes and mouth; the order of the dots from top to bottom and from left to right is as follows: left and right outer corners of the eye, nose tip and two key points of the mouth.
Further as a preferred embodiment of the method, before regularizing the data of the human eye image, the method further includes a step of blink detection and screening of the human eye image, specifically including:
obtaining a horizontal line and a vertical line passing through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is larger than a preset threshold value, determining that the human eye image is in an eye opening state, and estimating the sight line;
and judging that the ratio is smaller than a preset threshold value, determining that the human eye image is in a closed eye state, and not estimating the sight line.
Specifically, referring to fig. 5, based on face key point detection, we can determine 68 specific face key points, each with a specific index. Thus, we can get several key point indices for the left and right eyes (36,37,38,39,40,41) and (42,43,44,45,46,47), respectively, with the length of the horizontal line almost unchanged and the vertical line different when the eyes are open and closed. When the eye is open, the vertical line length is much greater than when closed. The eye was closed and the vertical line length was almost zero.
Further as a preferred embodiment of the method, the formula for data regularization is as follows:
M＝S*R
in the above equation, R represents the inverse of the camera rotation matrix, which causes the x-axis of the head coordinate system to be perpendicular to the y-axis of the camera coordinate system, the camera z-axis to be oriented toward the eye position, and S represents the scaling matrix, which causes the distance of the eye from the camera coordinate system to remain fixed.
Further as a preferred embodiment of the method, the step of regularizing the data of the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head pose estimation vector specifically includes:
processing the human eye image and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
scaling the camera coordinate system by an S scaling matrix;
and finally obtaining regularized human eye images and head gesture estimation vectors through perspective transformation.
Specifically, in order to realize high-precision line-of-sight estimation under different camera parameter conditions, data regularization is needed, that is, regularization processing is performed on an input image, so that the distance from a camera to the position of human eyes is ensured to be fixed, the x-axis of a head coordinate system is perpendicular to the y-axis of the camera coordinate system, and the z-axis of the camera faces the eyes.
Image regularization step schematic views referring to fig. 3 and 4, (a) from the head coordinate system centered on the tip of the nose (up) and the camera coordinate system (down); (b) the camera coordinate system is rotated in a rotation matrix; (c) scaling the camera coordinate system by the S scaling matrix; (d) And finally obtaining the regularized eye image through perspective transformation.
Further as a preferred embodiment of the method, the training step of the pre-trained CNN network specifically includes:
the method comprises the steps of obtaining a human eye image with a real sight angle label and inputting a head posture estimation vector to a CNN (computer numerical network) to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain the trained sight estimation model.
Further as a preferred embodiment of the method, the step of inputting regularized eye images and head pose estimation vectors into a pre-trained CNN network and converting the network output into 3D gaze direction vectors specifically comprises:
inputting regularized eye images and head pose estimation vectors into a pre-trained CNN network;
the eye characteristics are obtained through convolution of a convolution layer and lamination of a pooling layer;
specifically, the convolution layer carries out convolution operation, extracts eye features, and passes through the pooling layer to compress input features and extract main features, so that network calculation complexity is simplified.
Splicing the head posture estimation vector with the extracted eye characteristics through the full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
As shown in fig. 6, a line-of-sight estimation system includes:
the image preprocessing module is used for acquiring a human face image, performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into a pre-trained CNN network and converting network output into a 3D sight direction vector.
The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
A line-of-sight estimating apparatus:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement a gaze estimation method as described above.
The content in the method embodiment is applicable to the embodiment of the device, and the functions specifically realized by the embodiment of the device are the same as those of the method embodiment, and the obtained beneficial effects are the same as those of the method embodiment.
A storage medium having stored therein instructions executable by a processor, characterized by: the processor executable instructions when executed by the processor are for implementing a gaze estimation method as described above.
The content in the method embodiment is applicable to the storage medium embodiment, and functions specifically implemented by the storage medium embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
While the preferred embodiment of the present invention has been described in detail, the invention is not limited to the embodiment, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the invention, and these modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.
Claims (8)
1. A line-of-sight estimation method, comprising the steps of:
acquiring a human face image, and performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
inputting regularized human eye images and head pose estimation vectors into a pre-trained CNN network, and converting network output into 3D gaze direction vectors;
the step of obtaining a human face image, performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector specifically comprises the following steps:
acquiring a complete face image;
2D face alignment is carried out based on dlib face detection and 68 face key point detection, and two-dimensional coordinates of the face key points corresponding to the image are obtained;
acquiring a human eye image according to the positions of eye key points in the two-dimensional coordinates of the human face key points;
acquiring a 3D face key point model;
fitting the two-dimensional coordinates of the key points of the human face and the key point model of the 3D human face based on an EPnP algorithm to obtain a 3D head rotation vector;
the EPnP algorithm is to represent n three-dimensional space points as a weighted sum of 4 virtual control points;
before regularization of the data of the human eye image, the method further comprises the steps of blink detection and screening of the human eye image, and specifically comprises the following steps:
obtaining a horizontal line and a vertical line passing through eyes according to the left eye key point information and the right eye key point information in the human eye image;
calculating the ratio of the horizontal line to the corresponding vertical line;
judging that the ratio is larger than a preset threshold value, determining that the human eye image is in an eye opening state, and estimating the sight line;
and judging that the ratio is smaller than a preset threshold value, determining that the human eye image is in a closed eye state, and not estimating the sight line.
2. The line-of-sight estimation method of claim 1, wherein the data regularization formula is as follows:
M＝S*R
in the above equation, R represents an inverse of the camera rotation matrix, and S represents a scaling matrix.
3. The line-of-sight estimating method according to claim 2, wherein the step of regularizing the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head pose estimating vector specifically comprises:
processing the human eye image and the 3D head rotation vector based on the transformation matrix;
rotating the camera coordinate system by an R rotation matrix;
scaling the camera coordinate system by an S scaling matrix;
and finally obtaining regularized human eye images and head gesture estimation vectors through perspective transformation.
4. A line-of-sight estimation method according to claim 3, wherein the training step of the pre-trained CNN network specifically comprises:
the method comprises the steps of obtaining a human eye image with a real sight angle label and inputting a head posture estimation vector to a CNN (computer numerical network) to obtain network output;
calculating the error between the network output and the real sight angle label based on the loss function of the mean square error to obtain an error result;
and adjusting network parameters according to the error result to obtain the trained sight estimation model.
5. The line-of-sight estimating method according to claim 4, wherein the step of inputting regularized eye images and head pose estimating vectors to a pre-trained CNN network and converting network outputs into 3D line-of-sight direction vectors, specifically comprises:
inputting regularized eye images and head pose estimation vectors into a pre-trained CNN network;
the eye characteristics are obtained through convolution of a convolution layer and lamination of a pooling layer;
splicing the head posture estimation vector with the extracted eye characteristics through the full connection layer, and outputting a 2D sight angle;
and geometrically converting the 2D sight angle to obtain a 3D sight direction vector.
6. A line of sight estimation system for performing the line of sight estimation method of claim 1, comprising:
the image preprocessing module is used for acquiring a human face image, performing key point detection and 3D model fitting processing to obtain a human eye image and a 3D head rotation vector;
the data regularization module is used for carrying out data regularization on the human eye image and the 3D head rotation vector to obtain a regularized human eye image and a head posture estimation vector;
and the result output module is used for inputting the regularized human eye image and the head posture estimation vector into a pre-trained CNN network and converting network output into a 3D sight direction vector.
7. A line-of-sight estimating apparatus, comprising:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement a gaze estimation method as recited in any of claims 1-5.
8. A storage medium having stored therein instructions executable by a processor, characterized by: the processor executable instructions when executed by a processor are for implementing a line of sight estimation method as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110450755.1A CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110450755.1A CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113095274A CN113095274A (en) | 2021-07-09 |
| CN113095274B true CN113095274B (en) | 2024-02-09 |
Family
ID=76680139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110450755.1A Active CN113095274B (en) | 2021-04-26 | 2021-04-26 | Sight estimation method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095274B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822174B (en) * | 2021-09-02 | 2022-12-16 | 合肥的卢深视科技有限公司 | Sight line estimation method, electronic device and storage medium |
| CN114879843B (en) * | 2022-05-12 | 2024-07-02 | 平安科技（深圳）有限公司 | Sight redirection method based on artificial intelligence and related equipment |
| CN114967935B (en) * | 2022-06-29 | 2023-04-07 | 深圳职业技术学院 | Interaction method and device based on sight estimation, terminal equipment and storage medium |
| CN118314197A (en) * | 2023-01-06 | 2024-07-09 | 京东方科技集团股份有限公司 | Human eye positioning method, device, equipment and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171152A (en) * | 2017-12-26 | 2018-06-15 | 深圳大学 | Deep learning human eye sight estimation method, equipment, system and readable storage medium storing program for executing |
| CN108171218A (en) * | 2018-01-29 | 2018-06-15 | 深圳市唯特视科技有限公司 | A kind of gaze estimation method for watching network attentively based on appearance of depth |
| CN108875524A (en) * | 2018-01-02 | 2018-11-23 | 北京旷视科技有限公司 | Gaze estimation method, device, system and storage medium |
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A kind of convolutional neural networks gaze estimation method and system based on attention mechanism |
| WO2020228224A1 (en) * | 2019-05-11 | 2020-11-19 | 初速度（苏州）科技有限公司 | Face part distance measurement method and apparatus, and vehicle-mounted terminal |
| CN111985403A (en) * | 2020-08-20 | 2020-11-24 | 中再云图技术有限公司 | Distracted driving detection method based on face posture estimation and sight line deviation |
| CN112257696A (en) * | 2020-12-23 | 2021-01-22 | 北京万里红科技股份有限公司 | Sight estimation method and computing equipment |
| CN112488067A (en) * | 2020-12-18 | 2021-03-12 | 北京的卢深视科技有限公司 | Face pose estimation method and device, electronic equipment and storage medium |
-
2021
- 2021-04-26 CN CN202110450755.1A patent/CN113095274B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171152A (en) * | 2017-12-26 | 2018-06-15 | 深圳大学 | Deep learning human eye sight estimation method, equipment, system and readable storage medium storing program for executing |
| CN108875524A (en) * | 2018-01-02 | 2018-11-23 | 北京旷视科技有限公司 | Gaze estimation method, device, system and storage medium |
| CN108171218A (en) * | 2018-01-29 | 2018-06-15 | 深圳市唯特视科技有限公司 | A kind of gaze estimation method for watching network attentively based on appearance of depth |
| WO2020228224A1 (en) * | 2019-05-11 | 2020-11-19 | 初速度（苏州）科技有限公司 | Face part distance measurement method and apparatus, and vehicle-mounted terminal |
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A kind of convolutional neural networks gaze estimation method and system based on attention mechanism |
| CN111985403A (en) * | 2020-08-20 | 2020-11-24 | 中再云图技术有限公司 | Distracted driving detection method based on face posture estimation and sight line deviation |
| CN112488067A (en) * | 2020-12-18 | 2021-03-12 | 北京的卢深视科技有限公司 | Face pose estimation method and device, electronic equipment and storage medium |
| CN112257696A (en) * | 2020-12-23 | 2021-01-22 | 北京万里红科技股份有限公司 | Sight estimation method and computing equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于级联宽度学习的疲劳驾驶检测;朱玉斌;延向军;申旭奇;卢兆林;;计算机工程与设计(02);第245-249页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113095274A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113095274B (en) | Sight estimation method, system, device and storage medium | |
| US10706577B2 (en) | Facial features tracker with advanced training for natural rendering of human faces in real-time | |
| CN111325823B (en) | Method, device and equipment for acquiring face texture image and storage medium | |
| US9798384B2 (en) | Eye gaze tracking method and apparatus and computer-readable recording medium | |
| US9361723B2 (en) | Method for real-time face animation based on single video camera | |
| CN112614213A (en) | Facial expression determination method, expression parameter determination model, medium and device | |
| CN113366491B (en) | Eyeball tracking method, device and storage medium | |
| CN108805979A (en) | A kind of dynamic model three-dimensional rebuilding method, device, equipment and storage medium | |
| CN111046734A (en) | Multi-modal fusion sight line estimation method based on expansion convolution | |
| CN111815768B (en) | Three-dimensional face reconstruction method and device | |
| CN115661246A (en) | Attitude estimation method based on self-supervision learning | |
| CN112749611A (en) | Face point cloud model generation method and device, storage medium and electronic equipment | |
| CN117218246A (en) | Training method and device for image generation model, electronic equipment and storage medium | |
| Kang et al. | Real-time eye tracking for bare and sunglasses-wearing faces for augmented reality 3D head-up displays | |
| CN114029952A (en) | Robot operation control method, device and system | |
| CN110008873B (en) | Facial expression capturing method, system and equipment | |
| CN116681579A (en) | Real-time video face replacement method, medium and system | |
| CN113822174B (en) | Sight line estimation method, electronic device and storage medium | |
| CN114120443A (en) | Classroom teaching gesture recognition method and system based on 3D human body posture estimation | |
| CN110097644B (en) | Expression migration method, device and system based on mixed reality and processor | |
| Somepalli et al. | Implementation of single camera markerless facial motion capture using blendshapes | |
| Jian et al. | Realistic face animation generation from videos | |
| Hu et al. | Semi-supervised Multitask Learning using Gaze Focus for Gaze Estimation | |
| Will et al. | An Optimized Marker Layout for 3D Facial Motion Capture. | |
| CN109671108B (en) | Single multi-view face image attitude estimation method capable of rotating randomly in plane |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |