CN108960302B - Head attitude estimation method based on random forest - Google Patents

Head attitude estimation method based on random forest Download PDF

Info

Publication number
CN108960302B
CN108960302B CN201810638061.9A CN201810638061A CN108960302B CN 108960302 B CN108960302 B CN 108960302B CN 201810638061 A CN201810638061 A CN 201810638061A CN 108960302 B CN108960302 B CN 108960302B
Authority
CN
China
Prior art keywords
data
random forest
characteristic point
feature point
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810638061.9A
Other languages
Chinese (zh)
Other versions
CN108960302A (en
Inventor
董延超
林敏静
何士波
岳继光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji University
Original Assignee
Tongji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji University filed Critical Tongji University
Priority to CN201810638061.9A priority Critical patent/CN108960302B/en
Publication of CN108960302A publication Critical patent/CN108960302A/en
Application granted granted Critical
Publication of CN108960302B publication Critical patent/CN108960302B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

The invention relates to a head posture estimation method based on a random forest, which comprises the following steps: step S1: loading a plurality of pictures, and importing feature point data and actual head posture data of each picture; step S2: dividing all pictures of known feature point data and actual head posture data into a training set and a test set; step S3: training by adopting a training set picture to obtain an optimal random forest model, wherein the optimal random forest model is used for obtaining estimated head posture data according to the feature point data; step S4: testing the trained random forest model by using the test set picture, if the test error is smaller than a threshold value, executing the step S5, and if not, returning to the step S1; step S5: and performing sight estimation on the picture to be estimated by adopting a random forest model. Compared with the prior art, the invention can carry out regression of each angle of the head posture by only using 8 feature points of the face.

Description

Head attitude estimation method based on random forest
Technical Field
The invention relates to a head posture estimation method, in particular to a head posture estimation method based on a random forest.
Background
With the progress of science and technology, people have higher and higher requirements on safety, and therefore, the requirements on the face recognition technology are also increased. Many current technologies achieve quite satisfactory effects in relatively ideal environments such as laboratories, but in practical applications, because the natural environment and human postures vary greatly, the inevitable factors all seriously affect the accuracy of face recognition. For face recognition and related problems, head pose estimation is an important preprocessing process, and robust face recognition is still difficult to perform under the condition of head pose change, so that head pose estimation is an important method for improving the technical performance of face recognition as a prerequisite for solving the problems.
In recent years, there has been a growing demand for line-of-sight tracking research. In gaze tracking, head pose estimation plays a crucial role. It is generally considered that when people want to gaze in a certain direction, the head is usually turned to the gazed target, and then the eyes are turned to place the target in the visual field, so the calculation of the sight line is based on the head as a reference coordinate system, and the estimation of the head posture is an important prerequisite for sight line tracking.
In summary, head pose estimation is an important research in the neighborhood of computer vision and pattern recognition in the twenty-first century. In recent years, head pose estimation is mainly applied to face recognition, sight direction estimation, automobile safety assistant driving and the like, and due to wide application of the head pose estimation in many aspects, researchers are attracting more and more attention.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a head posture estimation method based on random forests.
The purpose of the invention can be realized by the following technical scheme:
a head pose estimation method based on random forests comprises the following steps:
step S1: loading a plurality of pictures, and importing feature point data and actual head posture data of each picture;
step S2: dividing all pictures of known feature point data and actual head posture data into a training set and a test set;
step S3: training by adopting a training set picture to obtain an optimal random forest model, wherein the optimal random forest model is used for obtaining estimated head posture data according to the feature point data;
step S4: testing the trained random forest model by using the test set picture, if the test error is smaller than a threshold value, executing the step S5, and if not, returning to the step S1;
step S5: and performing sight estimation on the picture to be estimated by adopting a random forest model.
The feature point data includes:
two left face characteristic points are arranged, namely a 1 st characteristic point and a 2 nd characteristic point,
two right face characteristic points are arranged, are a 4 th characteristic point and a 3 rd characteristic point and are respectively symmetrical to the left face characteristic point;
the number of the vertical characteristic points is four, the vertical characteristic points are vertically distributed on the nose tip from top to bottom and are respectively a 5 th characteristic point, a 6 th characteristic point, a 7 th characteristic point and an 8 th characteristic point;
the head pose data comprises: head pitch angle, azimuth angle and roll angle.
The step S1 specifically includes: and generating a plurality of pictures of the known feature point data and the actual head posture data through three-dimensional modeling software.
The step S1 includes:
step S11: generating a plurality of three-dimensional portraits by three-dimensional modeling software;
step S12: acquiring three-dimensional portrait feature point data and actual head posture data;
step S12: and deriving a plane picture according to the three-dimensional portrait.
The method for obtaining the optimal random forest model by training the training set picture comprises the following steps:
step S31: loading feature point data and actual head posture data of a training set picture;
step S32: obtaining corresponding picture face features according to the feature point data of each training set picture;
step S33: and performing regression analysis by using the facial features of the training set picture and the actual head posture data and adopting a random forest, training to obtain an optimal random forest model, and finding out the relation of obtaining estimated head posture data according to the feature point data.
The testing process in the step S4 specifically includes:
step S41: loading feature point data and actual head posture data of the test set picture;
step S42: obtaining the facial features of the corresponding pictures according to the feature point data of the pictures of each test set;
step S43: and taking the facial features of the test set pictures as the input of the trained random forest model to obtain estimated head posture data.
The facial features include:
a transverse length ratio feature comprising:
Figure GDA0002981317290000031
chal
wherein: lijIs the distance between the ith characteristic point and the jth characteristic point, chalAverage of the first 8 length ratios;
a transverse angle ratio feature comprising:
Figure GDA0002981317290000032
cha∠
wherein: the angle value cha is an angle value which takes the jth characteristic point as a peak in a triangle formed by the ith characteristic point, the jth characteristic point and the kth characteristic point∠Is the average of the first 7 angular ratios;
a longitudinal length ratio feature comprising:
Figure GDA0002981317290000033
wherein: lijThe distance between the ith characteristic point and the jth characteristic point is obtained;
the included angle between the characteristic point vector and the coordinate axis comprises the following steps:
Figure GDA0002981317290000034
wherein:
Figure GDA0002981317290000035
is a vector lijThe value of the angle to the horizontal axis,
Figure GDA0002981317290000036
is a vector lijAngle value from the vertical axis.
Compared with the prior art, the invention has the following beneficial effects:
1) regression of each angle of the head posture can be performed by only using 8 feature points of the face.
2) And generating a plurality of pictures with known sight characteristic quantities and real sight results through three-dimensional modeling software, so that reliable data sources of a test set and a training set can be provided.
Drawings
FIG. 1 is a schematic flow chart of the main steps of the present invention;
fig. 2 is a schematic diagram of feature point selection.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
A method for estimating head pose based on random forest, as shown in fig. 1, includes:
step S1: loading a plurality of pictures, and importing feature point data and actual head posture data of each picture, wherein as shown in fig. 2, the feature point data comprises:
two left face feature points are set, which are the 1 st feature point and the 2 nd feature point respectively, in this embodiment, the left face feature point can be selected from the left vertex and the right vertex of the eye socket,
two right face feature points are set, which are the 4 th feature point and the 3 rd feature point, and are respectively symmetrical to the left face feature point, similarly, in this embodiment, the right face feature points may be selected from the right and left vertices of the eye socket;
the number of the vertical characteristic points is four, the vertical characteristic points are vertically distributed on the nose tip from top to bottom and are respectively a 5 th characteristic point, a 6 th characteristic point, a 7 th characteristic point and an 8 th characteristic point;
the head pose data includes: head pitch angle, azimuth angle and roll angle.
Step S1 specifically includes: generating a plurality of pictures of known feature point data and actual head pose data through three-dimensional modeling software, wherein the pictures comprise:
step S11: generating a plurality of three-dimensional portraits by three-dimensional modeling software;
step S12: acquiring three-dimensional portrait feature point data and actual head posture data;
step S12: and deriving a plane picture according to the three-dimensional portrait.
Step S2: dividing all pictures of known feature point data and actual head posture data into a training set and a test set;
step S3: training by adopting a training set picture to obtain an optimal random forest model, wherein the optimal random forest model is used for obtaining estimated head posture data according to the feature point data, and the method specifically comprises the following steps:
step S31: loading feature point data and actual head posture data of a training set picture;
step S32: obtaining corresponding picture face features according to the feature point data of each training set picture;
step S33: and performing regression analysis by using the facial features of the training set picture and the actual head posture data and adopting a random forest, training to obtain an optimal random forest model, and finding out the relation of obtaining estimated head posture data according to the feature point data.
Step S4: testing the trained random forest model by using the test set picture, if the test error is smaller than a threshold value, executing the step S5, if not, returning to the step S1, wherein the test process specifically comprises the following steps:
step S41: loading feature point data and actual head posture data of the test set picture;
step S42: obtaining the facial features of the corresponding pictures according to the feature point data of the pictures of each test set;
step S43: and taking the facial features of the test set pictures as the input of the trained random forest model to obtain estimated head posture data.
Step S5: and performing sight estimation on the picture to be estimated by adopting a random forest model.
In this embodiment, the facial features include:
the lateral length ratio feature, which can reflect the variation of azimuth direction yaw, extracts 10-dimensional features in total, including:
Figure GDA0002981317290000051
chal
wherein: lijIs the distance between the ith characteristic point and the jth characteristic point, chalAverage of the first 8 length ratios;
the transverse angle ratio feature, which can also reflect the change of the yaw direction, extracts 9-dimensional angle ratio features in total, and comprises the following steps:
Figure GDA0002981317290000052
cha∠
wherein: the angle value cha is an angle value which takes the jth characteristic point as a peak in a triangle formed by the ith characteristic point, the jth characteristic point and the kth characteristic point∠Is the average of the first 7 angular ratios;
a longitudinal length ratio feature that can reflect changes in the pitch direction, comprising:
Figure GDA0002981317290000053
wherein: lijThe distance between the ith characteristic point and the jth characteristic point is obtained;
the included angle between the feature point vector and the coordinate axis can reflect the change of the roll angle roll direction, and the feature includes:
Figure GDA0002981317290000054
wherein:
Figure GDA0002981317290000055
is a vector lijThe value of the angle to the horizontal axis,
Figure GDA0002981317290000056
is a vector lijAngle value from the vertical axis.
By the method for selecting the facial features, the problems of overlarge calculated amount and the like caused by redundancy of data selection can be avoided while the accuracy is achieved.

Claims (4)

1. A head pose estimation method based on a random forest is characterized by comprising the following steps:
step S1: loading a plurality of pictures, and importing feature point data and actual head posture data of each picture;
step S2: the pictures of all known feature point data and actual head pose data are divided into a training set and a test set,
step S3: training by adopting a training set picture to obtain an optimal random forest model, wherein the optimal random forest model is used for obtaining estimated head posture data according to the feature point data,
step S4: testing the trained random forest model by using the test set picture, if the test error is smaller than the threshold value, executing the step S5, if not, returning to the step S1,
step S5: carrying out sight estimation on a picture to be estimated by adopting a random forest model;
the feature point data includes:
two left face characteristic points are arranged, namely a 1 st characteristic point and a 2 nd characteristic point,
two right face characteristic points are arranged, are the 4 th characteristic point and the 3 rd characteristic point and are respectively symmetrical with the left face characteristic point,
four vertical characteristic points are vertically distributed on the nose tip from top to bottom and respectively are a 5 th characteristic point, a 6 th characteristic point, a 7 th characteristic point and an 8 th characteristic point,
the head pose data comprises: head pitch, azimuth and roll angles;
the method for obtaining the optimal random forest model by training the training set picture comprises the following steps:
step S31: loading feature point data and actual head pose data of the training set picture,
step S32: obtaining the face features of the corresponding pictures according to the feature point data of the pictures of each training set,
step S33: performing regression analysis by using the facial features of the training set picture and the actual head posture data and adopting a random forest, training to obtain an optimal random forest model, and finding out the relation of obtaining estimated head posture data according to the feature point data;
the facial features include:
a transverse length ratio feature comprising:
Figure FDA0002992676020000011
chal
wherein: lijIs the distance between the ith characteristic point and the jth characteristic point, chalThe average of the first 8 length ratios, the transverse angle ratio feature, includes:
Figure FDA0002992676020000012
cha∠
wherein: the angle value cha is an angle value which takes the jth characteristic point as a peak in a triangle formed by the ith characteristic point, the jth characteristic point and the kth characteristic point∠Is the average of the first 7 angular ratios,
a longitudinal length ratio feature comprising:
Figure FDA0002992676020000021
wherein: lijIs the distance between the ith characteristic point and the jth characteristic point,
the included angle between the characteristic point vector and the coordinate axis comprises the following steps:
Figure FDA0002992676020000022
wherein:
Figure FDA0002992676020000023
is a vector lijThe value of the angle to the horizontal axis,
Figure FDA0002992676020000024
is a vector lijAngle value from the vertical axis.
2. The method for estimating head pose based on random forest according to claim 1, wherein the step S1 specifically comprises: and generating a plurality of pictures of the known feature point data and the actual head posture data through three-dimensional modeling software.
3. A method for estimating head pose based on random forest according to claim 2, wherein the step S1 comprises:
step S11: generating a plurality of three-dimensional portraits by three-dimensional modeling software;
step S12: acquiring three-dimensional portrait feature point data and actual head posture data;
step S12: and deriving a plane picture according to the three-dimensional portrait.
4. The method for estimating head pose based on random forest as claimed in claim 1, wherein the testing process in step S4 specifically comprises:
step S41: loading feature point data and actual head posture data of the test set picture;
step S42: obtaining the facial features of the corresponding pictures according to the feature point data of the pictures of each test set;
step S43: and taking the facial features of the test set pictures as the input of the trained random forest model to obtain estimated head posture data.
CN201810638061.9A 2018-06-20 2018-06-20 Head attitude estimation method based on random forest Active CN108960302B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810638061.9A CN108960302B (en) 2018-06-20 2018-06-20 Head attitude estimation method based on random forest

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810638061.9A CN108960302B (en) 2018-06-20 2018-06-20 Head attitude estimation method based on random forest

Publications (2)

Publication Number Publication Date
CN108960302A CN108960302A (en) 2018-12-07
CN108960302B true CN108960302B (en) 2021-06-04

Family

ID=64490594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810638061.9A Active CN108960302B (en) 2018-06-20 2018-06-20 Head attitude estimation method based on random forest

Country Status (1)

Country Link
CN (1) CN108960302B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115273180B (en) * 2022-07-01 2023-08-15 南通大学 Online examination invigilating method based on random forest

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102737235A (en) * 2012-06-28 2012-10-17 中国科学院自动化研究所 Head posture estimation method based on depth information and color image
CN106529409A (en) * 2016-10-10 2017-03-22 中山大学 Eye ocular fixation visual angle measuring method based on head posture
CN106778579A (en) * 2016-12-07 2017-05-31 电子科技大学 A kind of head pose estimation method based on accumulative attribute
CN108171218A (en) * 2018-01-29 2018-06-15 深圳市唯特视科技有限公司 A kind of gaze estimation method for watching network attentively based on appearance of depth

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150313530A1 (en) * 2013-08-16 2015-11-05 Affectiva, Inc. Mental state event definition generation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102737235A (en) * 2012-06-28 2012-10-17 中国科学院自动化研究所 Head posture estimation method based on depth information and color image
CN106529409A (en) * 2016-10-10 2017-03-22 中山大学 Eye ocular fixation visual angle measuring method based on head posture
CN106778579A (en) * 2016-12-07 2017-05-31 电子科技大学 A kind of head pose estimation method based on accumulative attribute
CN108171218A (en) * 2018-01-29 2018-06-15 深圳市唯特视科技有限公司 A kind of gaze estimation method for watching network attentively based on appearance of depth

Also Published As

Publication number Publication date
CN108960302A (en) 2018-12-07

Similar Documents

Publication Publication Date Title
US10082868B2 (en) Calculation method of line-of-sight direction based on analysis and match of iris contour in human eye image
CN109359526B (en) Human face posture estimation method, device and equipment
CN102999942B (en) Three-dimensional face reconstruction method
CN109785396B (en) Writing posture monitoring method, system and device based on binocular camera
CN103824050A (en) Cascade regression-based face key point positioning method
CN104573665A (en) Continuous motion recognition method based on improved viterbi algorithm
JP2016173313A (en) Visual line direction estimation system, visual line direction estimation method and visual line direction estimation program
CN108171133A (en) A kind of dynamic gesture identification method of feature based covariance matrix
CN113642393B (en) Attention mechanism-based multi-feature fusion sight estimation method
CN111339982A (en) Multi-stage pupil center positioning technology implementation method based on features
CN112101247B (en) Face pose estimation method, device, equipment and storage medium
CN112017212B (en) Training and tracking method and system of face key point tracking model
CN112232310A (en) Face recognition system and method for expression capture
CN108960302B (en) Head attitude estimation method based on random forest
CN107422844A (en) A kind of information processing method and electronic equipment
CN104091148A (en) Facial feature point positioning method and device
US20220175240A1 (en) A device and method for evaluating a performance of a visual equipment for a visual task
CN107545242B (en) Method and device for deducing human body action posture through 2D image
CN104318264A (en) Facial feature point tracking method based on human eye preferential fitting
CN113223125B (en) Face driving method, device, equipment and medium for virtual image
CN107330934B (en) Low-dimensional cluster adjustment calculation method and system
CN112861809A (en) Classroom new line detection system based on multi-target video analysis and working method thereof
CN110598647B (en) Head posture recognition method based on image recognition
CN111222469A (en) Coarse-to-fine human face posture quantitative estimation method
CN111222448B (en) Image conversion method and related product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant