CN107945213A - A kind of position of human eye tracking determines method, apparatus, equipment and storage medium - Google Patents
A kind of position of human eye tracking determines method, apparatus, equipment and storage medium Download PDFInfo
- Publication number
- CN107945213A CN107945213A CN201711129575.3A CN201711129575A CN107945213A CN 107945213 A CN107945213 A CN 107945213A CN 201711129575 A CN201711129575 A CN 201711129575A CN 107945213 A CN107945213 A CN 107945213A
- Authority
- CN
- China
- Prior art keywords
- human eye
- field picture
- corner location
- initial pictures
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Abstract
The invention discloses a kind of position of human eye to track the method for determining, including：Read the multiple image of detection object；Determining for position of human eye is carried out to each two field picture of reading according to default human eye detection algorithm successively；The image for determining position of human eye for the first time is determined as initial pictures, detects and records each corner location of initial pictures.For each two field picture being successively read after initial pictures, when determining the position of human eye of the two field picture according to human eye detection algorithm, detect and record each corner location of the two field picture；Otherwise, using optical flow tracking algorithm, according to each corner location of the previous frame image of the two field picture, calculate and record each corner location of the two field picture with the position of human eye of the definite two field picture.The technical solution provided using the embodiment of the present invention, improves the discrimination of human eye detection.Present invention also offers a kind of position of human eye tracking determining device, equipment and storage medium, there is relevant art effect.
Description
Technical field
The present invention relates to tracing of human eye technical field, is tracked more particularly to a kind of position of human eye and determines method, apparatus, sets
Standby and storage medium.
Background technology
In recent years, tracing of human eye becomes a big hot topic of research.Such as fatigue driving, the recognition of face under specific occasion etc.
Occasion, tracing of human eye technology have application.
Tracing of human eye method relatively common at present is to carry out tracing of human eye based on AdaBoost cascade classifiers technology.
Adaboost is a kind of iterative algorithm, and accurate result is obtained by aligning the training of counter-example.Using AdaBoost
When algorithm carries out tracing of human eye, behind the candidate region for determining human eye, the threshold value of related coefficient and threshold classification function is compared
Compared with finally to determine whether candidate region is human eye area.
Discrimination is a more important index of tracing of human eye algorithm, but in the method for existing tracing of human eye,
Missing inspection, happen occasionally, it is necessary to improve the discrimination of tracing of human eye with situation about losing.
There is the discrimination that Part Methods can improve tracing of human eye in the prior art, but still suffer from various problems.For example, make
When carrying out tracing of human eye with AdaBoost algorithms, the threshold size of threshold classification function has result very big influence.If
Threshold value is reduced, discrimination can be improved, but this method also results in misclassification rate rise while discrimination is improved.Existing skill
It is using cascade classifier cascade increase series to also have a kind of processing method in art.But although this method can be reduced and known by mistake
Rate, but discrimination is also reduced at the same time, it can also increase the time of identification, while be also impossible to ad infinitum increase cascade classifier
Series.Therefore, both approaches all can not improve discrimination on the premise of ensureing that misclassification rate is relatively low.
In conclusion how when carrying out tracing of human eye, discrimination is improved, and the negative of reduction misclassification rate will not be produced
Effect, is the technical problem that current those skilled in the art are badly in need of solving.
The content of the invention
Tracked the object of the present invention is to provide a kind of position of human eye and determine method, apparatus, equipment and storage medium, improved
The discrimination of human eye detection.
In order to solve the above technical problems, the present invention provides following technical solution：
A kind of position of human eye tracking determines method, and this method includes：
Read the multiple image of detection object；
Determining for position of human eye is carried out to each two field picture of reading according to default human eye detection algorithm successively；
When determining position of human eye for the first time according to the human eye detection algorithm, position of human eye will be determined for the first time
Image is determined as initial pictures, detects and records each corner location of the initial pictures；
For each two field picture being successively read after the initial pictures, determined when according to the human eye detection algorithm
When going out the position of human eye of the two field picture, detect and record each corner location of the two field picture；
For each two field picture being successively read after the initial pictures, when not true according to the human eye detection algorithm
When making the position of human eye of the two field picture, using optical flow tracking algorithm, according to each angle point of the previous frame image of the two field picture
Position, calculates and records each corner location of the two field picture with the position of human eye of the definite two field picture.
Preferably, it is described to utilize optical flow tracking algorithm, according to each corner location of the previous frame image of the two field picture, meter
Calculate and record each corner location of the two field picture to determine the position of human eye of the two field picture, including：
Using optical flow tracking algorithm, determine each characteristic point to be tracked of the two field picture relative to the respective of previous frame image
Displacement；
According to each corner location of the previous frame image of the two field picture, based on each displacement, previous frame is calculated
Each corner location of image respective positions and records in the two field picture, to determine the position of human eye of the two field picture.
Preferably, it is described determine the two field picture each characteristic point to be tracked relative to previous frame image respective displacement
Amount, including：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked adjacent two
The not minimum point of interframe luminance difference, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
Preferably, it is described to detect and record each corner location of the initial pictures, including：
According to the brightness of image or edge variation of the initial pictures, detect and record each angle of the initial pictures
Point position.
A kind of position of human eye tracks determining device, which includes：
Image reading module, for reading the multiple image of detection object；
Position of human eye determining module, for being carried out successively to each two field picture of reading according to default human eye detection algorithm
Position of human eye determines；
Initial angle point position determination module, position of human eye is determined for working as the first time according to the human eye detection algorithm
When, the image for determining position of human eye for the first time is determined as initial pictures, detects and records each angle of the initial pictures
Point position；
The first determining module of corner location, for for each two field picture being successively read after the initial pictures,
When determining the position of human eye of the two field picture according to the human eye detection algorithm, detect and record each angle point of the two field picture
Position；
The second determining module of corner location, for for each two field picture being successively read after the initial pictures,
When being not determined by the position of human eye of the two field picture according to the human eye detection algorithm, using optical flow tracking algorithm, according to the frame
Each corner location of the previous frame image of image, calculates and records each corner location of the two field picture with the definite two field picture
Position of human eye.
Preferably, second determining module of corner location includes following submodule：
Displacement determination sub-module, for utilizing optical flow tracking algorithm, determines each characteristic point phase to be tracked of the two field picture
For the respective displacement of previous frame image；
The second determination sub-module of corner location, for each corner location of the previous frame image according to the two field picture, base
In each displacement, calculate each corner location of previous frame image and respective positions and recorded in the two field picture, with
Determine the position of human eye of the two field picture.
Preferably, the displacement determination sub-module is specifically used for：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked adjacent two
The not minimum point of interframe luminance difference, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
Preferably, the initial angle point position determination module, is specifically used for：
According to the brightness of image or edge variation of the initial pictures, detect and record each angle of the initial pictures
Point position.
A kind of position of human eye tracking determines equipment, which includes：
Memory, for storing computer program；
Processor, for perform the computer program with realize read detection object multiple image；According to default
Human eye detection algorithm carries out determining for position of human eye to each two field picture of reading successively；When according to the human eye detection algorithm
When once determining position of human eye, the image for determining position of human eye for the first time is determined as initial pictures, detects and records institute
State each corner location of initial pictures；For each two field picture being successively read after the initial pictures, when according to institute
When stating human eye detection algorithm and determining the position of human eye of the two field picture, detect and record each corner location of the two field picture；Pin
To each two field picture being successively read after the initial pictures, when being not determined by the frame figure according to the human eye detection algorithm
During the position of human eye of picture, using optical flow tracking algorithm, according to each corner location of the previous frame image of the two field picture, calculate simultaneously
Each corner location of the two field picture is recorded with the position of human eye of the definite two field picture.
A kind of computer-readable recording medium, is stored with tracing of human eye on the computer-readable recording medium and determines journey
Sequence, the tracing of human eye determine to realize the step of above-mentioned tracing of human eye determines method when program is executed by processor.
The technical solution provided using the embodiment of the present invention, reads the multiple image of detection object；According to default people
Eye detection algorithm carries out determining for position of human eye to each two field picture of reading successively；When true for the first time according to human eye detection algorithm
When making position of human eye, the image for determining position of human eye for the first time is determined as initial pictures, detects and records initial pictures
Each corner location；For each two field picture being successively read after initial pictures, determined when according to human eye detection algorithm
When going out the position of human eye of the two field picture, detect and record each corner location of the two field picture；For after initial pictures according to
Each two field picture of secondary reading, when being not determined by the position of human eye of the two field picture according to human eye detection algorithm, using light stream with
Track algorithm, according to each corner location of the previous frame image of the two field picture, calculates and records each angle point position of the two field picture
Put to determine the position of human eye of the two field picture.
Image for that can determine position of human eye according to human eye detection algorithm, detects and records each of these two field pictures
A corner location.When a certain two field picture is not determined by position of human eye according to human eye detection algorithm, pass through each of previous frame image
Corner location, using optical flow tracking algorithm, determines each corner location of the two field picture and records.Determine the angle point of the two field picture
After position, the position of human eye of the two field picture has also been determined that.That is, position of human eye is not determined for human eye detection algorithm
Image, by the corner location of previous frame image, may finally determine the position of human eye of the two field picture, also allow for the application
Scheme be not in missing inspection, the situation of loss, improves discrimination.And the application improves the scheme of discrimination, is not
By limiting the parameter of human eye detection algorithm, such as when human eye algorithm is AdaBoost algorithms, the threshold classification that does not reduce
The threshold value of function so that the scheme of the application is not in due to improving misclassification rate caused by the parameter of restriction human eye detection algorithm
Situation.That is, the scheme of the application, on the premise of misclassification rate is not influenced, improves discrimination.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is attached drawing needed in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the implementing procedure figure that a kind of position of human eye tracking determines method in the present invention；
Fig. 2 is the structure diagram that a kind of position of human eye tracks determining device in the present invention；
Fig. 3 is the structure diagram that a kind of position of human eye tracking determines equipment in the present invention.
Embodiment
The core of the present invention is to provide a kind of position of human eye tracking and determines method, improves the discrimination of human eye detection.
In order to make those skilled in the art more fully understand the present invention program, with reference to the accompanying drawings and detailed description
The present invention is described in further detail.Obviously, described embodiment is only part of the embodiment of the present invention, rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Lower all other embodiments obtained, belong to the scope of protection of the invention.
Please refer to Fig.1, determine the implementing procedure figure of method for a kind of position of human eye tracking in the present invention, this method include with
Lower step：
S101：Read the multiple image of detection object.
Image capture device can be used to carry out Image Acquisition to detection object, and by the multiframe of the detection object of acquisition
Image is sent to processing equipment, so that processing equipment carries out subsequent treatment after reading image.It is, for example, possible to use camera obtains
The multiple image of detection object, and computer is sent to, so that the corresponding program of computer reads the multiple image of detection object.
After the multiple image of detection object is read, the operation of step S102 can be carried out.
S102：Determining for position of human eye is carried out to each two field picture of reading according to default human eye detection algorithm successively.
Default human eye detection algorithm can be the AdaBoost algorithms introduced in background technology, it is of course also possible to be it
He reads the algorithm for the position of human eye being capable of determining that after the image of detection object in image.Human eye position described in this application
Put, refer to including a region of human eye.
It is pointed out that position of human eye is being carried out really to each two field picture of reading successively using AdaBoost algorithms
Periodically, can the threshold value for the threshold classification function that AdaBoost algorithms use be set higher, so that AdaBoost algorithms
Misclassification rate when carrying out tracing of human eye is relatively low.Certainly, the specific setting of threshold value, and the related coefficient of AdaBoost algorithms are set
It is fixed, it can be set and be adjusted according to actual conditions, as carried out threshold value according to the requirement in practical application to misclassification rate
Setting and adjustment.When implementing the solution of the present invention using other human eye detection algorithms, it is relatively low can usually to choose misclassification rate
Human eye detection algorithm.
S103：When determining position of human eye for the first time according to human eye detection algorithm, position of human eye will be determined for the first time
Image be determined as initial pictures, detect and record each corner location of initial pictures.
In general, read the first two field picture of detection object, it is possible to determine first according to default human eye detection algorithm
Position of human eye in two field picture, the first two field picture that will be read is as initial pictures.Of course it is not excluded read the first two field picture
The situation of position of human eye can not be determined afterwards, then human eye detection algorithm can be utilized to continue to read the next frame figure of detection object
Picture, until determining position of human eye according to human eye detection algorithm.When according to default human eye detection algorithm for the first time determine people
During eye position, for the ease of description, the two field picture for determining position of human eye can be known as initial pictures, that is to say, that will
Determine that the image of position of human eye is determined as initial pictures for the first time.
After initial pictures are determined, detect and record each corner location of initial pictures.Corner location is two dimensional image
Curvature is the position of maximum on the violent position of brightness change or image border curve.It can be detected using Corner Detection Algorithm
Each corner location of initial pictures, after each corner location of initial pictures is detected, can be recorded, such as will
The data storage detected is in the database.
After detecting and recording each corner location of initial pictures, the operation of step S104 can be carried out.
S104：For each two field picture being successively read after initial pictures, determined when according to human eye detection algorithm
During the position of human eye of the two field picture, detect and record each corner location of the two field picture.
After initial pictures are determined, for each two field picture being successively read after initial pictures, when according to human eye
When detection algorithm determines the position of human eye of the two field picture, detect and record each corner location of the two field picture.Certainly, into
During the detection of each corner location of each two field picture of row, can also as the use Corner Detection Algorithm described in step S103 into
The detection of each corner location of row.
It is usual for each two field picture being successively read after initial pictures, the position of human eye for the two field picture determined
It is a rectangular area, can be to the position of human eye determined, i.e., when carrying out the detection of each corner location of the two field picture
The detection of each corner location is carried out to the rectangular area of the position of human eye for the two field picture determined.
S105：For each two field picture being successively read after initial pictures, do not determined when according to human eye detection algorithm
When going out the position of human eye of the two field picture, using optical flow tracking algorithm, according to each angle point position of the previous frame image of the two field picture
Put, calculate and record each corner location of the two field picture with the position of human eye of the definite two field picture.
For the part two field picture of detection object, the human eye position in image possibly can not be determined according to human eye detection algorithm
Put.Such as missing inspection, with losing.For each two field picture being successively read after initial pictures, when according to people
When eye detection algorithm is not determined by the position of human eye of the two field picture, using optical flow tracking algorithm, according to the previous frame of the two field picture
Each corner location of image, calculates and records each corner location of the two field picture with the position of human eye of the definite two field picture.
That is, the corner location of the two field picture is calculated according to the corner location of previous frame image, to pass through the angle point of the two field picture
The position of human eye of the location determination two field picture, the method for also allowing for the present invention is not in missing inspection, the situation with losing, and is improved
Discrimination.
For example, 5 two field pictures have been successively read after initial pictures, can be according to human eye detection for preceding 4 two field picture
Algorithm determines the position of human eye of each two field picture, but the position of human eye of the 5th two field picture can not be determined according to human eye detection algorithm.
Optical flow tracking algorithm can be utilized, according to each corner location of the 4th two field picture, calculates and records each angle of the 5th two field picture
Point position is with the position of human eye of definite 5th two field picture.It is pointed out that the position of human eye determined in such as the 4th two field picture
Region be rectangular area, correspondingly, the region that occupies of each corner location in the 4th two field picture also just forms a rectangle region
Domain.Using optical flow tracking algorithm, each corner location of the 5th two field picture is determined according to each corner location of the 4th two field picture
When, the region that each corner location of the 5th two field picture occupies can be considered as the position of human eye of the 5th two field picture, but the 5th two field picture
The region that occupies of each corner location be just not necessarily rectangular area.
The technical solution provided using the embodiment of the present invention, reads the multiple image of detection object；According to default people
Eye detection algorithm carries out determining for position of human eye to each two field picture of reading successively；When true for the first time according to human eye detection algorithm
When making position of human eye, the image for determining position of human eye for the first time is determined as initial pictures, detects and records initial pictures
Each corner location；For each two field picture being successively read after initial pictures, determined when according to human eye detection algorithm
When going out the position of human eye of the two field picture, detect and record each corner location of the two field picture；For after initial pictures according to
Each two field picture of secondary reading, when being not determined by the position of human eye of the two field picture according to human eye detection algorithm, using light stream with
Track algorithm, according to each corner location of the previous frame image of the two field picture, calculates and records each angle point position of the two field picture
Put to determine the position of human eye of the two field picture.
Image for that can determine position of human eye according to human eye detection algorithm, detects and records each of these two field pictures
A corner location.When a certain two field picture is not determined by position of human eye according to human eye detection algorithm, pass through each of previous frame image
Corner location, using optical flow tracking algorithm, determines each corner location of the two field picture and records.Determine the angle point of the two field picture
After position, the position of human eye of the two field picture has also been determined that.That is, position of human eye is not determined for human eye detection algorithm
Image, by the corner location of previous frame image, may finally determine the position of human eye of the two field picture, also allow for the application
Scheme be not in missing inspection, the situation of loss, improves discrimination.And the application improves the scheme of discrimination, is not
By limiting the parameter of human eye detection algorithm, such as when human eye algorithm is AdaBoost algorithms, the threshold classification that does not reduce
The threshold value of function so that the scheme of the application is not in due to improving misclassification rate caused by the parameter of restriction human eye detection algorithm
Situation.That is, the scheme of the application, on the premise of misclassification rate is not influenced, improves discrimination.
In a kind of embodiment of the present invention, the utilization optical flow tracking algorithm in step S105, according to the frame figure
Each corner location of the previous frame image of picture, calculates and records each corner location of the two field picture with the definite two field picture
Position of human eye, including：
Step 1：Using optical flow tracking algorithm, determine each characteristic point to be tracked of the two field picture relative to previous frame figure
The respective displacement of picture；
Step 2：According to each corner location of the previous frame image of the two field picture, based on each displacement, upper one is calculated
Each corner location of two field picture respective positions and records in the two field picture, to determine the position of human eye of the two field picture.
For the ease of description, above-mentioned two step is merged into explanation.
For each two field picture being successively read after initial pictures, when being not determined by the frame according to human eye detection algorithm
During the position of human eye of image, using optical flow tracking algorithm, determine each characteristic point to be tracked of the two field picture relative to previous frame
The respective displacement of image.After determining each displacement, based on each displacement, according to the previous frame image of the two field picture
Each corner location, each corner location for calculating previous frame image respective positions and records in the two field picture, with true
The position of human eye of the fixed two field picture.
Still by taking the 5th two field picture as an example, when being not determined by the position of human eye of the 5th two field picture according to human eye detection algorithm, profit
With light rigid-liquid coupled system, determine each characteristic point to be tracked of the 5th two field picture relative to the respective displacement of the 4th two field picture.
Such as the displacement of No.1 characteristic point to be tracked is (1,2), in the 4th two field picture, the No.1 where No.1 characteristic point to be tracked
The coordinate of some corner location in region to be tracked is (10,10), might as well be called No.1 angle point.Then it was determined that
In 5th two field picture, the corner location coordinate of No.1 angle point is (11,12).Certainly, this sentences No.1 characteristic point to be tracked and one
Illustrated exemplified by bugle point, for each characteristic point to be tracked of the 5th two field picture, be referred to No.1 characteristic point to be tracked
Operation.After the respective displacement for determining each characteristic point to be tracked, due to the angle point of each angle point in the 4th two field picture
Position has been recorded, and based on each displacement, each corner location that can calculate the 4th two field picture is each in the 5th two field picture
From position, and recorded.After each corner location that the 5th two field picture is determined, region that these corner locations occupy
It is exactly the position of human eye of the 5th two field picture.This kind of embodiment of the present invention, angle point is determined by the displacement of characteristic point to be tracked
The change of position is more quick when calculating.
In a kind of embodiment of the present invention, above-mentioned steps one include：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked adjacent two
The not minimum point of interframe luminance difference, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
For the ease of stating still by taking No.1 characteristic point to be tracked as an example, the region where No.1 characteristic point to be tracked is known as
No.1 characteristic area to be tracked.Determine No.1 characteristic area to be tracked in the not minimum point of adjacent two interframe luminance difference, Ke Yitong
Cross and calculate No.1 characteristic area to be tracked in the minimum value of adjacent two interframe gray scale difference quadratic sum to position.
For example, in the 5th frame, the coordinate of characteristic point to be tracked can be set as (X, Y), the brightness value of characteristic point to be tracked is
I (X, Y, T+t), this o'clock the position in the 4th two field picture be (X-x, Y-y), brightness value is I (X-x, Y-y, T).(x, y) is treated
Tracking characteristics o'clock are in the 5th two field picture relative to the displacement of the 4th two field picture.Have in the case where illumination condition is constant：I (X,
Y, T+t)=I (X-x, Y-y, T).Noise can be introduced generally, due to light change, n (X, Y) can be set to, thus is had：I (X,
Y, T+t)=I (X-x, Y-y, T)+n (X, Y).N (X, Y) integrated square is gray scale difference quadratic sum, can be expressed as ε, i.e.,：
ε=∫ ∫ n2(X, Y) dXdY=∫ ∫ [I (X, Y, T+t)-I (X-x, Y-y, T)]2dXdY；
Find the unknown quantity d=(dX, dY) so that ε minimumsT。
Make X=(X, Y)T, A (X-d)=I (X-x, Y-y, T), when d is unique and with X than it is insignificant a small amount of when, to A (X-
D) carry out the expansion of first order Taylor formula and ignore high-order term and have：
A (X-d)=A (X)-[A 'x(X)-A’y(X)]d；
Wherein A 'x(X) and A 'y(X) it is respectively partial derivatives of the A in x and y directions.
G=(A' can be setx(X)-A'y(X))2.Above-mentioned Taylor expansion is brought into the formula of gray scale difference quadratic sum to be obtained：
ε=∫ ∫ [B (X)-A (X)+gTd]2dX；
Derivation, order are carried out to the formulaTry to achieve：
∫∫(ggTDX) d=∫ ∫ (B (X)-A (X)) gdX, i.e.,：
Can obtain displacement (x, y) of the characteristic point to be tracked relative to previous frame image by solving the equation, still with
Exemplified by 5th two field picture, it may be determined that go out each characteristic point to be tracked of the 5th two field picture relative to the displacement of the 4th two field picture.Again
Based on each displacement, according to each corner location of the 4th two field picture, determine each corner location of the 5th two field picture, that is, realize
Tracking to each angle point of the 4th frame.Determine each corner location of the 5th two field picture, also determined that in the 5th two field picture
Position of human eye.In this kind of embodiment of the present invention, characteristic point to be tracked is determined by the difference in brightness of adjacent two interframe
Displacement so that the calculating of displacement is more accurate.
In a kind of embodiment of the present invention, detection in step S103 simultaneously records each angle point of initial pictures
Position, including：
According to the brightness of image or edge variation of initial pictures, detect and record each corner location of initial pictures.
The method for detecting corner location can be known as Corner Detection Algorithm.Certainly, to first in the embodiment of the application
The detection of each corner location of beginning image is introduced, and for each two field picture being successively read after initial pictures, works as root
When determining the position of human eye of the two field picture according to human eye detection algorithm, the frame figure can be carried out using same Corner Detection Algorithm
The detection of each corner location of picture.
The application is illustrated by taking the Harris Corner Detection Algorithms based on brightness as an example, i.e., according to the image of initial pictures
Brightness, detects each corner location of initial pictures, after each corner location of initial pictures is detected, can be remembered
Record.
The first-order difference of image can be used to calculate the square matrices of each pixel average gradient, gone out by Eigenvalues analysis
The position of angle point.It would generally amplify noise due to differentiating, so in calculating process, Gauss be carried out to image first and is put down
It is sliding, then calculate luminance gradient value on x directions and y directions respectively again.If I (x, y) represents coordinate (x, y) pixel in image
Gray value, image gradient is calculated using Sobel operators.
Ix can be set as the gradient on x directions, Iy is the gradient on y directions, can obtain the calculation formula of Ix and Iy：
Ix (x, y)=[I (x+1, y-1)+2I (x+1, y)+I (x+1, y+1)]-[I (x-1, y-1)+2I (x-1, y)+I (x-
1, y+1)]；
Iy (x, y)=[I (x-1, y+1)+2I (x, y+1)+I (x+1, y+1)]-[I (x-1, y-1)+2I (x, y-1)+I (x+
1, y+1)].
The difference correlation matrix M of each pixel in image can be calculated in two formulas more than：
When two characteristic values of a difference correlation matrix are all very big, then the point can be considered angle point.Therefore,
The greater in the characteristic value of each pixel matrix M and keeping characteristics value can be calculated, then to all the points that are calculated
Characteristic value carries out non-maximum suppression.When carrying out non-maximum and suppressing, the local feature that can be only remained in 3 × 3 neighborhoods
It is worth maximum of points.
Corresponding to above method embodiment, the embodiment of the present invention additionally provides a kind of position of human eye tracking determining device,
Position of human eye tracking determining device described below tracks the method for determining with above-described position of human eye can correspond reference.
It is shown in Figure 2, the structure diagram of determining device, the device bag are tracked for a kind of position of human eye in the present invention
Include：
Image reading module 210, for reading the multiple image of detection object；
Position of human eye determining module 202, for according to default human eye detection algorithm successively to each two field picture of reading
Carry out determining for position of human eye；
Initial angle point position determination module 203, for when determining position of human eye for the first time according to human eye detection algorithm,
The image for determining position of human eye for the first time is determined as initial pictures, detects and records each corner location of initial pictures；
The first determining module of corner location 204, for for each two field picture being successively read after initial pictures, when
When determining the position of human eye of the two field picture according to human eye detection algorithm, detect and record each corner location of the two field picture；
The second determining module of corner location 205, for for each two field picture being successively read after initial pictures, when
When being not determined by the position of human eye of the two field picture according to human eye detection algorithm, using optical flow tracking algorithm, according to the two field picture
Each corner location of previous frame image, calculates and records each corner location of the two field picture with the human eye of the definite two field picture
Position.
In a kind of embodiment of the present invention, the second determining module of corner location 205 includes following submodule：
Displacement determination sub-module, for utilizing optical flow tracking algorithm, determines each characteristic point phase to be tracked of the two field picture
For the respective displacement of previous frame image；
The second determination sub-module of corner location, for each corner location of the previous frame image according to the two field picture, base
In each displacement, calculate each corner location of previous frame image and respective positions and recorded in the two field picture, to determine
The position of human eye of the two field picture.
In a kind of embodiment of the present invention, displacement determination sub-module is specifically used for：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked adjacent two
The not minimum point of interframe luminance difference, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
In a kind of embodiment of the present invention, initial angle point position determination module 203, is specifically used for：
According to the brightness of image or edge variation of initial pictures, detect and record each corner location of initial pictures.
Corresponding to above method and device embodiment, the embodiment of the present invention additionally provides a kind of position of human eye tracking and determines
Equipment, position of human eye tracking described below determine that equipment is tracked with above-described position of human eye and determine that method and device can phase
Mutually to should refer to.
It is shown in Figure 3, the structure diagram of equipment, the equipment bag are determined for a kind of position of human eye tracking in the present invention
Include：
Memory 301, for storing computer program；
Processor 302, for performing computer program to realize：Read the multiple image of detection object；According to default
Human eye detection algorithm carries out determining for position of human eye to each two field picture of reading successively；When according to human eye detection algorithm first time
When determining position of human eye, the image for determining position of human eye for the first time is determined as initial pictures, detects and records initial graph
Each corner location of picture；For each two field picture being successively read after initial pictures, when true according to human eye detection algorithm
When making the position of human eye of the two field picture, detect and record each corner location of the two field picture；For after initial pictures
The each two field picture being successively read, when being not determined by the position of human eye of the two field picture according to human eye detection algorithm, utilizes light stream
Track algorithm, according to each corner location of the previous frame image of the two field picture, calculates and records each angle point of the two field picture
Position is with the position of human eye of the definite two field picture.
Corresponding to above method, device and apparatus embodiments, the embodiment of the present invention additionally provides a kind of computer-readable
Storage medium, is stored with tracing of human eye on the computer-readable recording medium and determines program, which determines that program is located
Reason device realizes the step of above-mentioned tracing of human eye determines method when performing.The computer-readable recording medium and above-described human eye
Position tracking determines that method, apparatus and equipment can correspond reference.
Each embodiment is described by the way of progressive in this specification, what each embodiment stressed be with it is other
The difference of embodiment, between each embodiment same or similar part mutually referring to.For dress disclosed in embodiment
For putting, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related part is referring to method part
Explanation.
Professional further appreciates that, with reference to each exemplary unit of the embodiments described herein description
And algorithm steps, can be realized with electronic hardware, computer software or the combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software, generally describes each exemplary composition and step according to function in the above description.These
Function is performed with hardware or software mode actually, application-specific and design constraint depending on technical solution.Specialty
Technical staff can realize described function to each specific application using distinct methods, but this realization should not
Think beyond the scope of this invention.
Can directly it be held with reference to the step of method or algorithm that the embodiments described herein describes with hardware, processor
Capable software module, or the two combination are implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Specific case used herein is set forth the principle of the present invention and embodiment, and above example is said
It is bright to be only intended to help and understand technical scheme and its core concept.It should be pointed out that for the common of the art
For technical staff, without departing from the principle of the present invention, some improvement and modification can also be carried out to the present invention, these
Improve and modification is also fallen into the protection domain of the claims in the present invention.
Claims (10)
1. a kind of position of human eye tracking determines method, it is characterised in that including：
Read the multiple image of detection object；
Determining for position of human eye is carried out to each two field picture of reading according to default human eye detection algorithm successively；
When determining position of human eye for the first time according to the human eye detection algorithm, the image of position of human eye will be determined for the first time
It is determined as initial pictures, detects and record each corner location of the initial pictures；
For each two field picture being successively read after the initial pictures, when determining this according to the human eye detection algorithm
During the position of human eye of two field picture, detect and record each corner location of the two field picture；
For each two field picture being successively read after the initial pictures, it is not determined by when according to the human eye detection algorithm
During the position of human eye of the two field picture, using optical flow tracking algorithm, according to each corner location of the previous frame image of the two field picture,
Calculate and record each corner location of the two field picture with the position of human eye of the definite two field picture.
2. according to the method described in claim 1, it is characterized in that, described utilize optical flow tracking algorithm, according to the two field picture
Each corner location of previous frame image, calculates and records each corner location of the two field picture with the human eye of the definite two field picture
Position, including：
Using optical flow tracking algorithm, determine each characteristic point to be tracked of the two field picture relative to the respective position of previous frame image
Shifting amount；
According to each corner location of the previous frame image of the two field picture, based on each displacement, previous frame image is calculated
Each corner location respective positions and recorded in the two field picture, to determine the position of human eye of the two field picture.
3. the according to the method described in claim 2, it is characterized in that, each characteristic point phase to be tracked for determining the two field picture
For the respective displacement of previous frame image, including：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked in adjacent two interframe
The point of difference in brightness minimum, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
4. method according to any one of claims 1 to 3, it is characterised in that described to detect and record the initial pictures
Each corner location, including：
According to the brightness of image or edge variation of the initial pictures, detect and record each angle point position of the initial pictures
Put.
5. a kind of position of human eye tracks determining device, it is characterised in that including：
Image reading module, for reading the multiple image of detection object；
Position of human eye determining module, for carrying out human eye to each two field picture of reading successively according to default human eye detection algorithm
Position determines；
Initial angle point position determination module, for when determining position of human eye for the first time according to the human eye detection algorithm, inciting somebody to action
Determine that the image of position of human eye is determined as initial pictures for the first time, detect and record each angle point position of the initial pictures
Put；
The first determining module of corner location, for for each two field picture being successively read after the initial pictures, working as root
When determining the position of human eye of the two field picture according to the human eye detection algorithm, detect and record each angle point position of the two field picture
Put；
The second determining module of corner location, for for each two field picture being successively read after the initial pictures, working as root
When being not determined by the position of human eye of the two field picture according to the human eye detection algorithm, using optical flow tracking algorithm, according to the two field picture
Previous frame image each corner location, calculate and record each corner location of the two field picture to determine the people of the two field picture
Eye position.
6. device according to claim 5, it is characterised in that second determining module of corner location includes following submodule
Block：
Displacement determination sub-module, for utilizing optical flow tracking algorithm, determine each characteristic point to be tracked of the two field picture relative to
The respective displacement of previous frame image；
The second determination sub-module of corner location, for each corner location of the previous frame image according to the two field picture, based on each
A displacement, each corner location for calculating previous frame image respective positions and record in the two field picture, to determine
The position of human eye of the two field picture.
7. device according to claim 6, it is characterised in that the displacement determination sub-module is specifically used for：
For each characteristic point to be tracked, according to the characteristic area to be tracked where the characteristic point to be tracked in adjacent two interframe
The point of difference in brightness minimum, determines the characteristic point to be tracked of the two field picture relative to the displacement of previous frame image.
8. according to claim 5 to 7 any one of them device, it is characterised in that the initial angle point position determination module, tool
Body is used for：
According to the brightness of image or edge variation of the initial pictures, detect and record each angle point position of the initial pictures
Put.
9. a kind of position of human eye tracking determines equipment, it is characterised in that including：
Memory, for storing computer program；
Processor, for perform the computer program with realize read detection object multiple image；According to default human eye
Detection algorithm carries out determining for position of human eye to each two field picture of reading successively；When according to human eye detection algorithm first time
When determining position of human eye, the image for determining position of human eye for the first time is determined as initial pictures, detects and records described first
Each corner location of beginning image；For each two field picture being successively read after the initial pictures, when according to the people
When eye detection algorithm determines the position of human eye of the two field picture, detect and record each corner location of the two field picture；For
The each two field picture being successively read after the initial pictures, when being not determined by the two field picture according to the human eye detection algorithm
During position of human eye, using optical flow tracking algorithm, according to each corner location of the previous frame image of the two field picture, calculate and record
Each corner location of the two field picture is with the position of human eye of the definite two field picture.
A kind of 10. computer-readable recording medium, it is characterised in that be stored with the computer-readable recording medium human eye with
Track determines program, and the tracing of human eye determines to realize the human eye as described in any one of Claims 1-4 when program is executed by processor
Tracking determines the step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711129575.3A CN107945213A (en) | 2017-11-15 | 2017-11-15 | A kind of position of human eye tracking determines method, apparatus, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711129575.3A CN107945213A (en) | 2017-11-15 | 2017-11-15 | A kind of position of human eye tracking determines method, apparatus, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107945213A true CN107945213A (en) | 2018-04-20 |
Family
ID=61931233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711129575.3A Pending CN107945213A (en) | 2017-11-15 | 2017-11-15 | A kind of position of human eye tracking determines method, apparatus, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107945213A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923717A (en) * | 2009-06-10 | 2010-12-22 | 新奥特（北京）视频技术有限公司 | Method for accurately tracking characteristic points of quick movement target |
| CN104898971A (en) * | 2015-05-11 | 2015-09-09 | 昆明理工大学 | Mouse pointer control method and system based on gaze tracking technology |
| CN105120185A (en) * | 2015-08-27 | 2015-12-02 | 新奥特（北京）视频技术有限公司 | Image matting method and device for video images |
-
2017
- 2017-11-15 CN CN201711129575.3A patent/CN107945213A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923717A (en) * | 2009-06-10 | 2010-12-22 | 新奥特（北京）视频技术有限公司 | Method for accurately tracking characteristic points of quick movement target |
| CN104898971A (en) * | 2015-05-11 | 2015-09-09 | 昆明理工大学 | Mouse pointer control method and system based on gaze tracking technology |
| CN105120185A (en) * | 2015-08-27 | 2015-12-02 | 新奥特（北京）视频技术有限公司 | Image matting method and device for video images |
Non-Patent Citations (2)
| Title |
|---|
| 杨青云: ""人眼定位技术研究与应用"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 王彬 等: ""基于Lucas-Kanada光流法的人眼特征点实时跟踪方法"", 《计算机工程》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9098760B2 (en) | Face recognizing apparatus and face recognizing method | |
| US10825184B2 (en) | Maximum connected domain marking method, target tracking method, and augmented reality/virtual reality apparatus | |
| US8331619B2 (en) | Image processing apparatus and image processing method | |
| TWI459312B (en) | Face tracking method | |
| CN103577815B (en) | A kind of face alignment method and system | |
| WO2019033574A1 (en) | Electronic device, dynamic video face recognition method and system, and storage medium | |
| CN110264493B (en) | Method and device for tracking multiple target objects in motion state | |
| CN109145708B (en) | Pedestrian flow statistical method based on RGB and D information fusion | |
| US20150036879A1 (en) | Posture estimating apparatus, posture estimating method and storing medium | |
| CN109086734A (en) | The method and device that pupil image is positioned in a kind of pair of eye image | |
| CN102542552B (en) | Frontlighting and backlighting judgment method of video images and detection method of shooting time | |
| US9384400B2 (en) | Method and apparatus for identifying salient events by analyzing salient video segments identified by sensor information | |
| CN105868708A (en) | Image object identifying method and apparatus | |
| CN110084258A (en) | Face preferred method, equipment and storage medium based on video human face identification | |
| US20130243251A1 (en) | Image processing device and image processing method | |
| CN104732197A (en) | Target line detection device and method | |
| CN110555866A (en) | Infrared target tracking method for improving KCF feature descriptor | |
| CN109523573A (en) | The tracking and device of target object | |
| He et al. | Exploring reliable visual tracking via target embedding network | |
| CN107145820B (en) | Binocular positioning method based on HOG characteristics and FAST algorithm | |
| Luo et al. | Salient object detection in videos by optimal spatio-temporal path discovery | |
| CN107945213A (en) | A kind of position of human eye tracking determines method, apparatus, equipment and storage medium | |
| WO2019201029A1 (en) | Candidate box update method and apparatus | |
| Liu et al. | Deep learning-based automatic player identification and logging in american football videos | |
| CN102314592B (en) | A kind of recognition methods of smiling face's image and recognition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180420 |