CN114267080A - Non-difference blink identification method based on angle change - Google Patents
Non-difference blink identification method based on angle change Download PDFInfo
- Publication number
- CN114267080A CN114267080A CN202111655134.3A CN202111655134A CN114267080A CN 114267080 A CN114267080 A CN 114267080A CN 202111655134 A CN202111655134 A CN 202111655134A CN 114267080 A CN114267080 A CN 114267080A
- Authority
- CN
- China
- Prior art keywords
- point
- eye
- angle
- nose
- characteristic point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000008859 change Effects 0.000 title claims abstract description 9
- 239000003550 marker Substances 0.000 claims abstract description 3
- 210000000744 eyelid Anatomy 0.000 claims description 23
- 230000004397 blinking Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a non-differential blink identification method based on angle change, which comprises the following steps of firstly, detecting a human face based on a human face monitor; secondly, determining A, B, C three most suitable blink marker points in the face; eye mark point A, C moves on the axis of the eye, and nose mark point B should be as close as possible to the horizontal line when eye coordinate point A, C is closed; and finally, constructing a characteristic triangle through the detected characteristic information of the nose mark points and the eye mark points, outputting the angle represented by the current eye characteristics, and judging the eye state. The invention can effectively and accurately detect whether the blink happens or not.
Description
Technical Field
The invention relates to a blink recognition method, in particular to a nondifferential blink recognition method based on angle change.
Background
It is important to detect blinking, for example, when detecting fatigue driving of a driver, blinking detection is an important index for determining whether the driver is fatigue driving; in human-computer interaction, blinking is used as an important judgment index for judging whether a person is a real person.
The current main methods for detecting blinking are: soukupova T judges whether the eye is open or closed according to the Eye Aspect Ratio (EAR) of the feature points around the eye; however, for persons with smaller eyes, blinking thereof cannot be detected. Blinking is the process of rapid closing and opening of the human eye, with each person having a different blink pattern. This pattern varied in closing and opening speed, degree of squeezing the eye, and duration of blinking, which lasted approximately 100-. Detecting a face in a video stream by using a face detector, and positioning a nose and an eyelid outline by using a face key point detector; from the detected landmarks in the image, an eye opening angle (EA) is derived, which is used to estimate the state of open eyes.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a non-differential blink identification method based on angle change, which can effectively and accurately detect whether blinking occurs or not.
The technical scheme is as follows: the invention provides a non-difference blink identification method based on angle change, which specifically comprises the following steps:
(1) detecting a human face based on a human face monitor;
(2) determining A, B, C the three most appropriate blink marker points in the face; eye mark point A, C moves on the axis of the eye, and nose mark point B should be as close as possible to the horizontal line when eye coordinate point A, C is closed;
(3) and constructing a characteristic triangle through the detected characteristic information of the nose mark points and the eye mark points, outputting the angle represented by the current eye characteristics, and judging the eye state.
Further, the determination process of the point A, C in step (2) is as follows:
respectively taking any characteristic point A of upper eyelid of the left eye and the right eye1、C1The two points repeatedly move up and down along with the upper eyelid during the blinking process of the human eye, when the human eye is closed, the upper eyelid is superposed with the lower eyelid, the position of the characteristic point of the left eye and the right eye on the lower eyelid is the lowest point which can be reached during the blinking process of the human eye, and the characteristic point is named as a point A2、C2Taking any point B on a straight line in front of the eyebrow center and the nose tip as a nose characteristic point;
when the position of the nose characteristic point B is fixed, the characteristic point A of the eyelid on both eyes1、C1Is not determined, and the eye opening angle EA is defined by an angle A when the eyes are opened1BC1Indicates that when both eyes are closed, EA is the angle A2BC2Represents;
to make the EA vary widely, the EA should be minimal when the eye is open and maximal when the eye is closed, denoted as angle C1BC2And angle A1BA2Maximum sum of (C), angle C1BC2The degree of (A) is as follows:
wherein X represents an angle C1BC2C represents BC1B represents BC2A represents C1C2Length of (d); by using the reverseThe cosine function can be given by:
from equations (1) and (2), it can be deduced that X becomes larger as the value of the cosine function becomes smaller; the larger the value of a, the smaller the value of the cosine function, so C1And C2The points which are the highest and the lowest of the eyes respectively, namely the moving track of the characteristic point of the right eye is coincided with the central axis of the right eye; similarly, the movement locus of the feature point of the left eye should coincide with the central axis of the left eye.
Further, the determination process of the point B in the step (2) is as follows
Characteristic point A of eyelid on both eyes1、C1The position of the nose characteristic point B is not fixed, the nose characteristic point B moves on a straight line between the eyebrow center and the nose tip, wherein when the characteristic point B and the point A are in contact2、C2On the same horizontal line, the position is named point B2(ii) a Calculating the angle C1BC2Angle Y of (2); the nose characteristic point B is from the tip of the nose to the point B2When moving linearly, with B2Is shorter and shorter, i.e. the value of Y is larger and larger; nose feature point B at point B2When moving straight to the eyebrow center, the moving direction is along with B2Is longer and longer, i.e. the value of Y is smaller and smaller; the nose characteristic point should be selected from the tip of the nose to the point B2Segment and not with point B2The coincidence, i.e., nose feature point B, should be as close as possible to the horizontal line when eye coordinate point A, C is closed.
Further, the step (3) is realized as follows:
wherein (A)x,Ay)、(Bx,By)、(Cx,Cy) Respectively, are the coordinates of the index point A, B, C, AdisDenotes the distance between B and C, BdisDenotes the distance between A and C, CdisThe distance between a and B is indicated and EA is the angle of the blink detection angle.
Further, the EA is open at 2rad or more and closed at 2rad or less.
Has the advantages that: compared with the prior art, the invention has the beneficial effects that: the invention constructs a characteristic included angle through the detected characteristic information of the eye nose part, outputs the angle represented by the current eye characteristic and judges the eye state; the method can effectively and accurately detect whether the blink exists or not based on the angle change and the undifferentiated blink identification method.
Drawings
FIG. 1 is a schematic diagram of Dlib face feature point positioning;
FIG. 2 is a simplified facial feature trajectory diagram;
FIG. 3 is a schematic diagram of the fixed position of the nose landmark B and the movement of the landmark A, C on both eyes;
FIG. 4 illustrates a movement path diagram of nose landmark B for determining the movement path of landmark A, C;
fig. 5 is a graph of blink detection results using the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The invention provides a non-differential blink identification method based on angle change, which detects a human face by using a human face monitor in an Opencv library; detecting face key points by using a face feature point detector in a Dlib library; as shown in fig. 1, 0-67 key points are detected; and constructing a characteristic triangle through the detected characteristic information of the eye nose part, outputting the angle represented by the current eye characteristic, and judging the eye state. The method specifically comprises the following steps:
first, the movement trajectories of the three feature points are simplified as shown in fig. 2, where l1、l3Respectively taking any characteristic point A of the upper eyelid of the left eye and the right eye1、C1The two points repeatedly move up and down along with the upper eyelid during the blinking process of the human eye, when the human eye is closed, the upper eyelid is superposed with the lower eyelid, the position of the characteristic point of the left eye and the right eye on the lower eyelid is the lowest point which can be reached during the blinking process of the human eye, and the characteristic point is named as a point A2、C2Meanwhile, in order to construct the blink angle, any point B on a straight line in front of the eyebrow center and the nose tip needs to be taken as a nose feature point.
The position of the nose characteristic point B is fixed, and the characteristic points A of the eyelid on both eyes1、C1When both eyes are open, the eye opening angle (EA) is defined by an angle A as shown in FIG. 31BC1Showing that when both eyes are closed, the angle of opening (EA) of the eye is defined by angle A2BC2And (4) showing.
To make the eye opening angle (EA) more widely variable, the eye opening angle (EA) should be the smallest when the eye is open and the eye opening angle (EA) should be the largest when the eye is closed, which may be represented as angle C1BC2And angle A1BA2The sum of (C) is maximum, and because of the symmetry, the two angles are of equal size, so long as angle C is present1BC2Or angle A1BA2One of the angular ranges is maximized.
At an angle C1BC2As the object of the study, the following formula is available:
wherein X represents an angle C1BC2C represents BC1B represents BC2A represents C1C2Is determined by the position of the movement locus of the eye feature points a, C.
The following equation can be obtained by using the inverse cosine function:
it can be deduced from equations (1) and (2) that X becomes larger as the value of the cosine function becomes smaller.
Since C and b are fixed values, the value of the cosine function is determined by a, i.e. by C1And C2The distance therebetween. The larger the value of a, the smaller the value of the cosine function, so C1And C2The points of the eye which are the highest and the lowest points respectively, namely the movement track of the characteristic point of the right eye, are coincident with the central axis of the right eye. Similarly, the movement locus of the feature point of the left eye should coincide with the central axis of the left eye.
Characteristic point A of eyelid on both eyes1、C1The position of the nose characteristic point B is not fixed. As shown in FIG. 4, the nose feature point B moves on a straight line between the center of the eyebrow and the tip of the nose, when feature point B and point A2、C2On the same horizontal line, the position is named point B2。
At an angle C1BC2As an object of study, the smaller the value of the cosine function, the larger the value of the inverse cosine function, i.e., the larger X. From the properties of the cosine function it follows:
the nose characteristic point B is from the tip of the nose to the point B2When moving linearly, with B2The distance of (a) is shorter and shorter, the value of the cosine function is smaller and smaller, and the value of the inverse cosine function is larger and larger, that is, the value of Y is larger and larger.
Nose feature point B at point B2When moving straight to the eyebrow center, the moving direction is along with B2The distance of (a) is longer and longer, the value of the cosine function is larger and larger, and the value of the inverse cosine function is smaller and smaller, namely the value of Y is smaller and smaller.
Therefore, the nose feature point should be selected from the tip of the nose to the point B2Segment and point of no contactB2The coincidence, i.e., nose feature point B, should be as close as possible to the horizontal line when eye coordinate point A, C is closed.
Eye landmark point A, C all moves on the eye axis, and nose landmark point B should be as close as possible to the horizontal line when eye coordinate point A, C is closed.
For each frame of the input video, nose and eyelid contours are detected, and an eye opening angle (EA) is calculated:
wherein (A)x,Ay)、(Bx,By)、(Cx,Cy) Respectively, are the coordinates of the index point A, B, C, AdisDenotes the distance between B and C, BdisDenotes the distance between A and C, CdisThe distance between a and B is indicated and EA is the angle of the blink detection angle.
The eye opening angle (EA) is maintained almost constant above 2rad when the eye is open and less than 2rad when the eye is closed.
The invention is tested on a Windows development platform, is realized by using a python language, selects a human face detector and a human face characteristic point detector in an Opencv library and a Dlib library to detect and analyze the human face in the video, and inputs the video by adopting a camera mode. Fig. 5 is a diagram illustrating the effect of the present blink detection method on a video stream, and it can be seen from the diagram that the present invention can effectively and accurately detect blinks.
Claims (5)
1. A method for identifying undifferentiated blinking based on angle change is characterized by comprising the following steps:
(1) detecting a human face based on a human face monitor;
(2) determining A, B, C the three most appropriate blink marker points in the face; eye mark point A, C moves on the axis of the eye, and nose mark point B should be as close as possible to the horizontal line when eye coordinate point A, C is closed;
(3) and constructing a characteristic triangle through the detected characteristic information of the nose mark points and the eye mark points, outputting the angle represented by the current eye characteristics, and judging the eye state.
2. The method for non-differential blink recognition based on angle variation according to claim 1, wherein the determination of the point A, C in step (2) is as follows:
respectively taking any characteristic point A of upper eyelid of the left eye and the right eye1、C1The two points repeatedly move up and down along with the upper eyelid during the blinking process of the human eye, when the human eye is closed, the upper eyelid is superposed with the lower eyelid, the position of the characteristic point of the left eye and the right eye on the lower eyelid is the lowest point which can be reached during the blinking process of the human eye, and the characteristic point is named as a point A2、C2Taking any point B on a straight line in front of the eyebrow center and the nose tip as a nose characteristic point;
when the position of the nose characteristic point B is fixed, the characteristic point A of the eyelid on both eyes1、C1Is not determined, and the eye opening angle EA is defined by an angle A when the eyes are opened1BC1Indicates that when both eyes are closed, EA is the angle A2BC2Represents;
to make the EA vary widely, the EA should be minimal when the eye is open and maximal when the eye is closed, denoted as angle C1BC2And angle A1BA2Maximum sum of (C), angle C1BC2The degree of (A) is as follows:
wherein X represents an angle C1BC2C represents BC1B represents BC2A represents C1C2Length of (d); the following equation can be obtained by using the inverse cosine function:
from equations (1) and (2), it can be deduced that X becomes larger as the value of the cosine function becomes smaller; the larger the value of a, the smaller the value of the cosine function, so C1And C2The points which are the highest and the lowest of the eyes respectively, namely the moving track of the characteristic point of the right eye is coincided with the central axis of the right eye; similarly, the movement locus of the feature point of the left eye should coincide with the central axis of the left eye.
3. The method for non-differential blink recognition based on angle variation as claimed in claim 1, wherein the determination of the point B in step (2) is as follows
Characteristic point A of eyelid on both eyes1、C1The position of the nose characteristic point B is not fixed, the nose characteristic point B moves on a straight line between the eyebrow center and the nose tip, wherein when the characteristic point B and the point A are in contact2、C2On the same horizontal line, the position is named point B2(ii) a Calculating the angle C1BC2Angle Y of (2); the nose characteristic point B is from the tip of the nose to the point B2When moving linearly, with B2Is shorter and shorter, i.e. the value of Y is larger and larger; nose feature point B at point B2When moving straight to the eyebrow center, the moving direction is along with B2Is longer and longer, i.e. the value of Y is smaller and smaller; the nose characteristic point should be selected from the tip of the nose to the point B2Segment and not with point B2The coincidence, i.e., nose feature point B, should be as close as possible to the horizontal line when eye coordinate point A, C is closed.
4. The method for non-differential blink recognition based on angle variation according to claim 1, wherein the step (3) is implemented as follows:
wherein (A)x,Ay)、(Bx,By)、(Cx,Cy) Respectively, are the coordinates of the index point A, B, C, AdisDenotes the distance between B and C, BdisDenotes the distance between A and C, CdisThe distance between a and B is indicated and EA is the angle of the blink detection angle.
5. The method of claim 4, wherein the EA is open at 2rad or more and closed at 2rad or less.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111655134.3A CN114267080B (en) | 2021-12-30 | 2021-12-30 | Non-difference blink identification method based on angle change |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111655134.3A CN114267080B (en) | 2021-12-30 | 2021-12-30 | Non-difference blink identification method based on angle change |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114267080A true CN114267080A (en) | 2022-04-01 |
| CN114267080B CN114267080B (en) | 2023-03-24 |
Family
ID=80831870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111655134.3A Active CN114267080B (en) | 2021-12-30 | 2021-12-30 | Non-difference blink identification method based on angle change |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114267080B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863545A (en) * | 2022-07-05 | 2022-08-05 | 之江实验室 | Automatic blink detection method and device based on DeepLabCut |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080252745A1 (en) * | 2007-04-13 | 2008-10-16 | Fujifilm Corporation | Apparatus for detecting blinking state of eye |
| JP2009176005A (en) * | 2008-01-24 | 2009-08-06 | Toyota Motor Corp | Characteristic point detection method for face image and its device |
| CN109460704A (en) * | 2018-09-18 | 2019-03-12 | 厦门瑞为信息技术有限公司 | A kind of fatigue detection method based on deep learning, system and computer equipment |
| CN109840565A (en) * | 2019-01-31 | 2019-06-04 | 成都大学 | A kind of blink detection method based on eye contour feature point aspect ratio |
| WO2019232871A1 (en) * | 2018-06-08 | 2019-12-12 | 平安科技（深圳）有限公司 | Glasses virtual wearing method and apparatus, and computer device and storage medium |
| CN113627300A (en) * | 2021-08-02 | 2021-11-09 | 中电福富信息科技有限公司 | Face recognition and living body detection method based on deep learning |
| CN113627256A (en) * | 2021-07-09 | 2021-11-09 | 武汉大学 | Method and system for detecting counterfeit video based on blink synchronization and binocular movement detection |
-
2021
- 2021-12-30 CN CN202111655134.3A patent/CN114267080B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080252745A1 (en) * | 2007-04-13 | 2008-10-16 | Fujifilm Corporation | Apparatus for detecting blinking state of eye |
| JP2009176005A (en) * | 2008-01-24 | 2009-08-06 | Toyota Motor Corp | Characteristic point detection method for face image and its device |
| WO2019232871A1 (en) * | 2018-06-08 | 2019-12-12 | 平安科技（深圳）有限公司 | Glasses virtual wearing method and apparatus, and computer device and storage medium |
| CN109460704A (en) * | 2018-09-18 | 2019-03-12 | 厦门瑞为信息技术有限公司 | A kind of fatigue detection method based on deep learning, system and computer equipment |
| CN109840565A (en) * | 2019-01-31 | 2019-06-04 | 成都大学 | A kind of blink detection method based on eye contour feature point aspect ratio |
| CN113627256A (en) * | 2021-07-09 | 2021-11-09 | 武汉大学 | Method and system for detecting counterfeit video based on blink synchronization and binocular movement detection |
| CN113627300A (en) * | 2021-08-02 | 2021-11-09 | 中电福富信息科技有限公司 | Face recognition and living body detection method based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| SOFIA JENNIFER JOHN等: "Real time blink recognition from various head pose using single eye", 《MULTIMEDIA TOOLS & APPLICATIONS》 * |
| 郑伟成等: "基于深度学习的疲劳驾驶检测算法", 《计算机工程》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863545A (en) * | 2022-07-05 | 2022-08-05 | 之江实验室 | Automatic blink detection method and device based on DeepLabCut |
| CN114863545B (en) * | 2022-07-05 | 2022-10-21 | 之江实验室 | Automatic blink detection method and device based on deep LabCut |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114267080B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhuang et al. | Driver fatigue detection method based on eye states with pupil and iris segmentation | |
| CN104616438B (en) | A kind of motion detection method of yawning for fatigue driving detection | |
| US7043056B2 (en) | Facial image processing system | |
| CN102324025B (en) | Human face detection and tracking method based on Gaussian skin color model and feature analysis | |
| CN110334600A (en) | A kind of multiple features fusion driver exception expression recognition method | |
| CN107358152B (en) | Living body identification method and system | |
| CN109472198A (en) | A kind of video smiling face's recognition methods of attitude robust | |
| CN110221699B (en) | Eye movement behavior identification method of front-facing camera video source | |
| CN111344703B (en) | User authentication device and method based on iris recognition | |
| CN105809144A (en) | Gesture recognition system and method adopting action segmentation | |
| CN108596087B (en) | Driving fatigue degree detection regression model based on double-network result | |
| CN109145852B (en) | Driver fatigue state identification method based on eye opening and closing state | |
| KR102005150B1 (en) | Facial expression recognition system and method using machine learning | |
| CN109711239B (en) | Visual attention detection method based on improved mixed increment dynamic Bayesian network | |
| CN114267080B (en) | Non-difference blink identification method based on angle change | |
| CN116051631A (en) | Light spot labeling method and system | |
| CN115937928A (en) | Learning state monitoring method and system based on multi-vision feature fusion | |
| Gupta et al. | Hands-free PC Control” controlling of mouse cursor using eye movement | |
| CN114022514A (en) | Real-time sight line inference method integrating head posture and eyeball tracking | |
| CN111104817A (en) | Fatigue detection method based on deep learning | |
| CN107977622A (en) | Eyes detection method based on pupil feature | |
| CN112257512B (en) | Indirect eye state detection method and computer readable storage medium | |
| Richarz et al. | There you go!-estimating pointing gestures in monocular images for mobile robot instruction | |
| Handrich et al. | A robust method for human pose estimation based on geodesic distance features | |
| Ouellet et al. | Multimodal biometric identification system for mobile robots combining human metrology to face recognition and speaker identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20220401 Assignee: Jiangsu Kesheng Xuanyi Technology Co.,Ltd. Assignor: HUAIYIN INSTITUTE OF TECHNOLOGY Contract record no.: X2023980048436 Denomination of invention: A method for indiscriminate blink recognition based on angle variation Granted publication date: 20230324 License type: Common License Record date: 20231129 |