CN109635851A - A kind of smart television human fatigue detection system and method based on face multiple features fusion - Google Patents
A kind of smart television human fatigue detection system and method based on face multiple features fusion Download PDFInfo
- Publication number
- CN109635851A CN109635851A CN201811410439.6A CN201811410439A CN109635851A CN 109635851 A CN109635851 A CN 109635851A CN 201811410439 A CN201811410439 A CN 201811410439A CN 109635851 A CN109635851 A CN 109635851A
- Authority
- CN
- China
- Prior art keywords
- fatigue
- module
- eye
- value
- multiple features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Ophthalmology & Optometry (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The present invention relates to a kind of smart television human fatigue detection systems and method based on face multiple features fusion.The system is based on smart television application, specifically includes that fatigue detecting setup module, video data acquiring module, human face characteristic point extraction module, eyes aspect ratio computing module, blink detection module, head pose detection module, tired intelligent detection module.Frequency of blinking in the unit time is judged by the variation of eyes aspect ratio, the generation for judging drowsiness event by head pose gradient and duration length in the unit time, cascading judgement human fatigue state, and a kind of implementation method is proposed based on this system, more comprehensive multiple features fusion analysis and joint decision have been carried out to fatigue detecting various dimensions feature in smart television usage scenario, the accuracy of fatigue identification is improved, the probability of false triggering is reduced.
Description
Technical field
The present invention relates to Intelligent television terminal technology and field of face identification, more particularly to a kind of face multiple features that are based on to melt
The smart television human fatigue detection system and method for conjunction.
Background technique
With the development of society and the progress of science and technology, the living standard of the people is increasingly improved, and smart television is at thousand
It is popularized in ten thousand families, becomes the important composition of people's daily life amusement and recreation, it is very popular.But in life,
Prolonged viewing TV often generates tired state, situation about falling asleep unconsciously frequent occurrence, and at this time if electricity
Depending on continuing to play, the waste of resource not only will form, also will affect sleep quality, the fatigue of spectators when how to judge to watch TV
State becomes problem concerned by people.
Summary of the invention
The present invention is directed to existing issue, proposes a kind of smart television human fatigue detection based on face multiple features fusion
System and method, system and method carry out joint fatigue detecting using blink feature, the multiple features fusions such as sitting posture feature, are based on eye
The concept of eyeball aspect ratio, it is contemplated that the statistical nature of the sleep priori of user and drowsiness characteristic, one threshold value of proposition of innovation is certainly
The blink detection decision rule for adapting to adjustment accurately enumerates the standard of blink judgement.Meanwhile sentencing in above-mentioned adaptive blink
Certainly except criterion, this key feature of body posture and the detection of head inclination angle is introduced, to smart television usage scenario
Middle fatigue detecting various dimensions feature has carried out more comprehensive multiple features fusion analysis and joint decision, improves fatigue identification
Accuracy reduces the probability of false triggering.
The present invention solves above-mentioned technical problem, and one aspect of the present invention provides a kind of intelligence electricity based on face multiple features fusion
Depending on human fatigue detection system, fatigue detecting setup module, video data acquiring module are specifically included, human face characteristic point extracts mould
Block, eyes aspect ratio computing module, blink detection module, head pose detection module, tired intelligent detection module.
The fatigue detecting setup module, for fatigue detecting relevant parameter to be arranged.Particularly, in current setting without more
When new configuration, default configuration is executed automatically.
The video data acquiring module, the acquisition for vision signal.It can be generally intelligence by USB standard interface
Energy TV is equipped with camera external equipment, and the real-time video that camera acquires or data of taking pictures are passed to corresponding position by this module
Storage, provides real-time data for subsequent human face characteristic point.
The human face characteristic point extraction module is adopted for obtaining the video data provided from video data acquiring module
Facial feature points detection is carried out with mature open source software.Wherein, facial feature points detection, i.e. facial modeling, face
Alignment, be to be carried out on the basis of Face datection, Face datection include to characteristic point on the face such as corners of the mouth, canthus etc. into
Row positioning.The extraction of similar characteristics, such as the library Dlib of industry maturation are carried out using mature open source software, it is that one kind is opened
Source, the free C++ Open-Source Tools packet comprising machine learning algorithm, the library Dlib has been widely used in industry and at present
Art field, including robot, embedded device, mobile phone and large-scale high-performance computing environment.
The preferred library Dlib human face characteristic point as in extracts the main tool realized herein, its extraction module can detecte
68 key features of face.
The eyes aspect ratio computing module is chosen in multiple features for extracting in human face characteristic point extraction module
Crucial eye feature point calculates eyes aspect ratio.
The physical meaning of eyes aspect ratio indicates the ratio of longitudinal eye widths and lateral eye widths.
It is preferably calculated herein from 6 crucial eye feature points in 68 key features that the library Dlib is extracted.
The blink detection module calculates eye for obtaining the eyes aspect ratio obtained in eyes aspect ratio computing module
The numerical value of the eyeball fatigue factor, judges the generation of blink event.When eye fatigue factor value is greater than given threshold value, mark is known
It is clipped to primary blink event, and counts the number of winks occurred in period given time；When eye fatigue factor value be less than etc.
When given threshold value, blink event does not occur for mark.
The head pose detection module, it is crucial in the multiple features extracted in human face characteristic point extraction module for obtaining
6 eye feature point datas coordinate, carry out operation head inclination angle, judge the generation of sleeping position event.Work as head inclination
When angle is greater than given threshold value, the sleeping position event of a posture head inclination is recognized, statistics is greater than given threshold value
Duration；When head inclination angle is less than or equal to given threshold value, dwell time statistics carries out circulation time system according to this
Meter.
The fatigue intelligent detection module, for carrying out joint decision fatigue according to the whole features obtained with upper module
State.What number of winks and head pose detection module obtained in the period demand obtained according to blink detection module is given
Head pose dipping event duration cascading judgement in period obtains the tired alert feature value of intelligence, makes fatigue state and sentence
It is disconnected.The tired alert feature of definition intelligence is Th_tired, and fatigue detecting threshold value is SleepTh, by Th_tired and SleepTh into
Row compares, if it is greater than SleepTh, then it represents that in a state of fatigue；If it is less than equal to SleepTh, then it represents that in non-tired
Labor state.
Be using the beneficial effect of above system: this system has used for reference the concept of mainstream eyes the ratio of width to height, it is contemplated that user
Sleep priori statistical nature and drowsiness characteristic, innovate on this basis proposes a threshold adaptive adjustment blink
Decision rule is detected, the standard of blink judgement is more accurately enumerated；Meanwhile above-mentioned adaptive blink decision rule it
Outside, this key feature for introducing body posture and the detection of head inclination angle examines fatigue in smart television usage scenario
It surveys various dimensions feature and has carried out more comprehensive multiple features fusion analysis and joint decision, improve the accuracy of fatigue identification,
Reduce the probability of false triggering；Without user's interaction and operation, the complex scene that intelligentized control method fatigue detecting is related to makes
With more humanized, be capable of it is effective it is friendly tired prompting is carried out to user, improve user's comforts of use.
Another aspect of the present invention also provides a kind of smart television human fatigue detection method based on face multiple features fusion,
The following steps are included:
Step 1: opening intelligent fatigue detecting and control setup module, carry out fatigue detecting relative parameters setting；
Step 2: starting video data acquiring module carries out video data acquiring；
Step 3: the video data obtained according to step 2 carries out facial feature points detection, and stores；
Step 4: the human face characteristic point obtained according to step 3 calculates eyes aspect ratio；
Step 5: the eyes aspect ratio being calculated according to step 4 calculates the eye fatigue factor, according to the eye fatigue factor
Blink event whether occurs with the multilevel iudge of given threshold value；
When the eye fatigue factor is greater than given threshold value, then the generation of the primary blink event of mark, counts period demand
Interior number of winks；When the eye fatigue factor is less than or equal to given threshold value, then identifies and blink event does not occur.
Step 6: the coordinate by obtaining 6 key feature point datas of two eyes in left and right carries out operation acquisition head and inclines
Rake angle decides whether that sleeping position event occurs；
When head inclination angle is greater than given threshold value, the generation of a head pose dipping event is identified, statistics is given
Duration in period；When head inclination angle is less than or equal to given threshold value, mark head oblique attitude event does not occur.
Step 7: head in the period demand that number of winks and step 5 obtain in the period demand obtained according to step 5
Oblique attitude incident duration cascading judgement obtains the tired alert feature value of intelligence, makes fatigue state judgement.
By intelligent fatigue characteristic value and given fatigue detecting threshold value comparison, intelligent fatigue characteristic value is greater than given fatigue
It indicates to be currently in fatigue state when detection threshold value, intelligent fatigue characteristic value is less than or equal to indicate when given fatigue detecting threshold value
It is not in fatigue state at present.
Further, the specific method is as follows for the calculating of eyes aspect ratio in the step 4:
Definition eyes the ratio of width to height is EAR, and molecular computing is the characteristic points of eyes is being hung down in EAR (a) expression formula in following formula
The upward distance of histogram, what denominator calculated is the distance of the characteristic point of eyes in the horizontal direction.Since level point only has one group,
And vertical point has two groups, so denominator has been multiplied by 2, the weight to guarantee two groups of characteristic points is identical, shown in formula specific as follows:
F1-F6 is above-mentioned 6 crucial eye features, and each F point includes its x and y coordinates value on face coordinate system；
A characterization is currently left eye or right eye, and a value 0 indicates left eye, and value 1 indicates right eye.
Wherein,
Dist function stand be two characteristic points linear distance, we are with the ith and jth key feature points of left eye
Distance calculate for.
Wherein F (a, b, i) is human eye key feature coordinate defined in computing module S400；
1st parameter a characterization is currently left eye or right eye, and value 0 indicates left eye, and value 1 indicates right eye；2nd ginseng
Number b expression is currently x coordinate or y-coordinate, and value is respectively x and y；3rd parameter i indicates the rope of eyes main points feature
Draw, value is 1~6.
In view of countenance and local form, we are finally weighted and averaged calculating to the ratio of width to height of two eyes, obtain
To eyes aspect ratio Ehwr (EyeHeightWidthRatio):
Eyes aspect ratio Ehwr, as last output numerical value.
Further, the calculation method of the eye fatigue factor in the step 5 is specific as follows:
The definition eye fatigue factor is D_eye, and formula is as follows:
Above-mentioned S (T) be drowsiness time factor, wherein T indicate television system current time T, value range be [0,
24], at any one moment for characterizing 24 hours one day, usually indicated with a discrete piecewise function.Drowsiness time factor table
Levied the sleep priori of user statistical nature and drowsiness characteristic, in drowsiness time interval, there are high values.According to user
Work and rest rule, we can be defined on one day different periods S (T) numerical value, such as meet the psychological need of people, it is afternoon and deep
The drowsiness higher numerical value of degree, promotes the drowsiness factor of user, in this period, user is easier to sleep at the time of night
?.Therefore in identical D_eye, in morning, higher Ehwr can trigger blink detection module when afternoon.This design distinguished
In the way of fixed sentence threshold value in the prior art, it joined the new feature for meeting human body work and rest and physiological law, reach threshold
The purpose for the blink detection that value adaptively adjusts.
Further, the calculation method of the current head tilt angle in the step 6 is specific as follows:
Definition current head tilt angle is theta:
F (a, b, i) is the human eye key feature coordinate of definition, and the 1st parameter a characterization is currently left eye or right eye, is taken
Value 0 indicates left eye, and value 1 indicates right eye；2nd parameter b expression is currently x coordinate or y-coordinate, and value is respectively x and y；
3rd parameter i indicates the index of eyes main points feature, and value is 1 to 6.
The angle that above formula is equivalent to horizontal axis by calculating right and left eyes symmetrical feature coordinate on face coordinate system obtains some
The tilt angle of feature is weighted and averaged processing for all key features and obtains reliable and stable statistical nature.Calculate knot
Fruit theta can characterize face and be equivalent to the horizontal tilt angle sat up straight, as the key factor of attitude detection, when in sleep
Or lie on one's side trend when, the tilt angle of face generally will increase to 40 degree or more, to obtain posture fatigue
Adjudicate feature.
Further, the calculation method of the tired alert feature value of intelligence in the step 7 is specific as follows:
Delimiting period time decision is Tinterval, intelligence fatigue alert feature Th_tired:
Th_tired=count (AEB) * θ (Time_SleepPos))
It once blinks in the blink detection module of above-mentioned AEB (Adaptive Eye Blink) expression threshold adaptive adjustment
The successful judgement of detection, count (AEB) indicate the number of winks detected in judgement cycle time Tinterval；Time_
After SleepPos indicates that the detection of head pose detection module sleeping position is adjudicated successfully in given time decision, cumulative statistics is in
The duration of drowsiness sleeping position.
Due to fatigue blink probability of occurrence it is larger, tired sleeping position probability of occurrence is relatively low, to above two feature into
Posture feature weight of having gone processing, defines θ are as follows:
It regard sleep as weight coefficient, is mapped to [1, q] section, q parameter flexible choice is in [1.1~1.8] section.
Be using the beneficial effect of the above method: this method has used for reference the concept of mainstream eyes the ratio of width to height, it is contemplated that user
Sleep priori statistical nature and drowsiness characteristic, innovate on this basis proposes a threshold adaptive adjustment blink
Decision rule is detected, the standard of blink judgement is more accurately enumerated；Meanwhile above-mentioned adaptive blink decision rule it
Outside, this key feature for introducing body posture and the detection of head inclination angle examines fatigue in smart television usage scenario
It surveys various dimensions feature and has carried out more comprehensive multiple features fusion analysis and joint decision, improve the accuracy of fatigue identification,
Reduce the probability of false triggering；Without user's interaction and operation, the complex scene that intelligentized control method fatigue detecting is related to makes
With more humanized, be capable of it is effective it is friendly tired prompting is carried out to user, improve user's comforts of use.
Detailed description of the invention
Fig. 1 is a kind of system structure diagram of the fatigue detecting system based on smart television provided in an embodiment of the present invention；
Fig. 2 is a kind of fatigue detection method flow chart based on smart television provided in an embodiment of the present invention；
Fig. 3 is eyes 6 crucial key feature points schematic diagrames of the embodiment of the present invention.
In attached drawing, parts list represented by the reference numerals are as follows:
S100, television system nucleus module, S200, fatigue detecting setup module, S300, video data acquiring module,
S400, human face characteristic point extraction module, S500, eyes aspect ratio computing module, S600, blink detection module, S700, head appearance
State detection module, S800, tired intelligent detection module, S900, tired reminding module.
Specific embodiment
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the invention.
As shown in Figure 1, being a kind of system knot of the fatigue detecting system based on smart television provided in an embodiment of the present invention
Structure block diagram, including television system nucleus module S100, fatigue detecting setup module S200, video data acquiring module S300, people
Face characteristic point extraction module S400, eyes aspect ratio computing module S500, blink detection module S600, head pose detection module
S700, tired intelligent detection module S800 and tired reminding module S900.
The television system nucleus module S100 is mainly responsible for the device drives initialization of TV core, TV basis clothes
The starting of business, press key message response and processing, network communication, the processes such as camera device driving are TV basis and core clothes
The supplier of business.
The fatigue detecting setup module S200, for fatigue detecting relevant parameter to be arranged.Particularly, when current setting does not have
When thering is update to configure, the automatic configuration logic for executing default.
Further, the fatigue detecting setup module when system realizes prompting function simultaneously is also responsible for that tired inspection is arranged
Mode and content parameters that fatigue is reminded are executed after surveying triggering.
The video data acquiring module S300, the acquisition for vision signal.Generally can by USB standard interface,
It is equipped with camera external equipment for smart television, this module is incoming corresponding by the real-time video that camera acquires or data of taking pictures
Position storage, provides real-time video data for subsequent human face characteristic point.
The human face characteristic point extraction module S400, for obtaining the video counts provided from video data acquiring module
According to using mature open source software progress facial feature points detection.Human face characteristic point extracts in fact in the preferred library Dlib conduct herein
Existing main tool, extraction module can detecte 68 key features of face.
The eyes aspect ratio computing module S500, in multiple features for being extracted in human face characteristic point extraction module
Crucial eye feature point is chosen, eyes aspect ratio is calculated.68 preferably extracted herein from human face characteristic point extraction module S400
6 crucial eye feature points in a key feature are calculated.
The blink detection module S600 is counted for obtaining the eyes aspect ratio obtained in eyes aspect ratio computing module
The numerical value for calculating the eye fatigue factor, judges the generation of blink event.When eye fatigue factor value is greater than given threshold value, mark
Primary blink event is recognized, and counts the number of winks occurred in period given time；When the eye of threshold adaptive adjustment
When eyeball fatigue factor value is less than or equal to given threshold value, blink event does not occur for mark.
Since the size and degree of fatigue of eyes aspect ratio are inversely proportional, its inverse characterizes the degree of fatigue of user, when
When user is in a state of fatigue, longitudinal eye-level can obviously become smaller, so that the inverse of eyes aspect ratio can be bright
It is aobvious to increase.
The head pose detection module S700, for obtaining in the multiple features extracted in human face characteristic point extraction module
The coordinate of 6 crucial eye feature point datas carries out operation head inclination angle.When head inclination angle is greater than given threshold
When value, the sleeping position event of a posture head inclination is recognized, statistics is greater than the duration of given threshold value；Work as head
When tilt angle is less than or equal to given threshold value, dwell time statistics carries out circulation time statistics according to this.
The fatigue intelligent detection module S800, for carrying out joint decision according to the whole features obtained with upper module
Fatigue state.Number of winks and head pose detection module obtain in the period demand obtained according to blink detection module
Head pose dipping event duration cascading judgement in period demand obtains the tired alert feature value of intelligence, makes tired shape
State judgement.
The fatigue reminding module S900, enforces TV screen protection, and voice reminder human fatigue state needs to rest, or
Broadcasting, which is releived, music and suspends current TV content broadcasting, or takes automatic shutdown strategy.
Fig. 2 is a kind of fatigue detection method stream based on smart television provided in an embodiment of the present invention based on above system
Cheng Tu, this method step specifically include that
Step 1: hardware and bottom layer driving are supported to prepare, and are equipped with the camera hardware of a standard in the usb mouth of TV.
Step 2: opening fatigue detecting and control setup module S200, carry out fatigue detecting relative parameters setting.
Specifically, we carry out similar following setting to parameter:
The weight k=1.5 in the drowsiness section in the blink detection module S600 of threshold adaptive adjustment；
Blink judgement characteristic threshold value Dblink_th=5 in the blink detection module S600 of threshold adaptive adjustment；
Head inclination angle threshold Dhead_th=pi/2*0.8 in head pose detection module S700；
Periodic time decision Tinterval=100s in head pose detection module S700；
Sleep posture weight coefficient q=1.2 in head pose detection module S700；
Fatigue detecting threshold value SleepTh=30 in head pose detection module S700；
The strategy that fatigue is reminded in tired reminding module S900 are as follows: automatic shutdown after light music of releiving plays；
If selecting default setting, the similar default parameters of above-mentioned parameter can be provided.
Step 3: dollying head preview interface can prompt user's adjustment to take the photograph if face is not detected in starting for the first time
As head angle and direction, until viewing television area can detecte until face.
Step 4: starting video data acquiring module S300 carries out video data acquiring；
Step 5: the video data obtained according to step 4 carries out facial feature points detection using the library Dlib, and extraction detects
Face 68 key features, and store；
Step 6: the human face characteristic point obtained according to step 5 calculates eyes the ratio of width to height；
F1-F6 is above-mentioned 6 crucial eye features, and each F point includes its x and y coordinates value on face coordinate system；
A characterization is currently left eye or right eye, and a value 0 indicates left eye, and value 1 indicates right eye.
Wherein,
Dist function stand be two characteristic points linear distance, we are with the ith and jth key feature points of left eye
Distance calculate for.
Wherein F (a, b, i) is human eye key feature coordinate defined in computing module S400；
1st parameter a characterization is currently left eye or right eye, and value 0 indicates left eye, and value 1 indicates right eye；2nd ginseng
Number b expression is currently x coordinate or y-coordinate, and value is respectively x and y；3rd parameter i indicates the rope of eyes main points feature
Draw, value is 1~6.
In view of countenance and local form, we are finally weighted and averaged calculating to the ratio of width to height of two eyes, obtain
To eyes aspect ratio Ehwr (Eye Height Width Ratio):
Eyes aspect ratio Ehwr, as last output numerical value.
Step 7: the eyes aspect ratio being calculated according to step 6 calculates the adaptive eye fatigue factor, according to eyes
Whether the multilevel iudge of the tired factor and given threshold value occurs blink event；
Specifically, the eye fatigue factor is defined as follows:
Above-mentioned S (T) be drowsiness time factor, wherein T indicate television system current time T, value range be [0,
24], 24 hours one day any one moment is characterized.
Indicate that S (T) value interval, value are as follows with a discrete piecewise function:
In time zone morning [0,8], afternoon [12,15], the late into the night [21,24] these three sections for user it is drowsiness because
Element is promoted, and is easier to fall asleep in this period user, and the k in above-mentioned expression formula is the weight in drowsiness section, therefore,
We choose 1 in non-drowsiness section, can generally be finely adjusted k value between 1 to 2, and expressing drowsiness factor pair Ehwr influences
Weight.
S (T) expression formula that linear fit herein provides only is used as scene example to be described；It can for the design of S (T)
To be not limited to above-mentioned linear segmented Function Fitting, for quadratic function, log function, smoothed curve, multinomial, index equal part
Section fitting, it is all to reach the strategy and purpose that the processing of differentiation weight is carried out according to drowsiness time interval, all in this patent
Within protection scope.
When the eye fatigue factor is greater than given threshold value, then the generation of the primary blink event of mark, counts period demand
Interior number of winks；When the eye fatigue factor is less than or equal to given threshold value, then identifies and blink event does not occur.
Step 7: the coordinate by obtaining 6 key feature point datas of two eyes in left and right carries out operation acquisition head and inclines
The calculating of rake angle decides whether that sleeping position event occurs；
Head inclination angle is theta:
F (a, b, i) is the human eye key feature coordinate of definition, and the 1st parameter a characterization is currently left eye or right eye, is taken
Value 0 indicates left eye, and value 1 indicates right eye；2nd parameter b expression is currently x coordinate or y-coordinate, and value is respectively x and y；
3rd parameter i indicates the index of eyes main points feature, and value is 1 to 6.
When head inclination angle theta is greater than given threshold value, the generation of a head pose dipping event is identified, is counted
Duration in period demand；When head inclination angle theta is less than or equal to given threshold value, head oblique attitude event is identified
Do not occur.
Step 8: according to number of winks in period demand and head pose dipping event duration cascading judgement, obtaining tired
Labor alert feature value makes fatigue state judgement.
Periodical time decision is Tinterval, tired alert feature Th_tired:
Th_tired=count (AEB) * θ (Time_SleepPos))
It once blinks in the blink detection module of above-mentioned AEB (Adaptive Eye Blink) expression threshold adaptive adjustment
The successful judgement of detection, count (AEB) indicate the number of winks detected in judgement cycle time Tinterval；Time_
After SleepPos indicates that the detection of head pose detection module sleeping position is adjudicated successfully in given time decision, cumulative statistics is in
The drowsiness duration slept.
Due to fatigue blink probability of occurrence it is larger, tired sleeping position probability of occurrence is relatively low, to above two feature into
Posture feature weight of having gone processing, defines θ are as follows:
It regard sleep as weight coefficient, is mapped to [1, q] section, q parameter flexible choice is in [1.1~1.8] section.
Fatigue characteristic value Th_tired and given fatigue detecting threshold value comparison, fatigue characteristic value Th_tired are greater than and given
It indicates to be currently in fatigue state when fixed fatigue detecting threshold value, fatigue characteristic value Th_tired is less than or equal to given fatigue inspection
It indicates to be not in fatigue state at present when surveying threshold value.
Step 9: tired reminding module S800 fatigue reminding module can according to the fatigue state court verdict that step 8 obtains,
And project is set in step 1, it plays and releives music and suspend current TV content broadcasting, automatic shutdown after terminating.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (10)
1. a kind of smart television human fatigue detection system based on face multiple features fusion, including fatigue detecting setup module,
Video data acquiring module, human face characteristic point extraction module, eyes aspect ratio computing module, blink detection module, head pose
Detection module, tired intelligent detection module；
The fatigue detecting setup module, for fatigue detecting relevant parameter to be arranged；
The video data acquiring module, the acquisition for vision signal；
The human face characteristic point extraction module carries out people for obtaining the video data provided from video data acquiring module
Face characteristic point detection；
The eyes aspect ratio computing module is chosen in multiple features for extracting in human face characteristic point extraction module crucial
Eye feature point, calculate eyes aspect ratio；
It is tired to calculate eyes for obtaining the eyes aspect ratio obtained in eyes aspect ratio computing module for the blink detection module
The numerical value of the labor factor judges the generation of blink event；
The head pose detection module, for obtaining crucial eye in the multiple features extracted in human face characteristic point extraction module
The coordinate of characteristic point carries out operation and obtains head inclination angle, judges the generation of sleeping position event；
The fatigue intelligent detection module, for all spies according to blink detection module and the acquisition of head pose detection module
Sign carries out joint decision fatigue state.
2. a kind of smart television human fatigue detection system based on face multiple features fusion according to claim 1, special
Sign is: the human face characteristic point extraction module carries out facial feature points detection using the library Dlib.
3. a kind of smart television human fatigue detection system based on face multiple features fusion according to claim 2, special
Sign is: crucial eye feature point is chosen in the multiple features extracted in the eyes aspect ratio computing module extraction module,
The crucial eye feature of 6 specially chosen from 68 key features for carrying out facial feature points detection extraction based on the library Dlib
Point.
4. a kind of smart television human fatigue detection system based on face multiple features fusion according to claim 1, special
Sign is: the blink detection module, judges the generation of blink event, specially when eye fatigue factor value be greater than it is given
When threshold value, primary blink event is recognized, and counts the number of winks occurred in period given time；When eye fatigue because
When subnumber value is less than or equal to given threshold value, blink event does not occur for mark.
5. a kind of smart television human fatigue detection system based on face multiple features fusion according to claim 1, special
Sign is: judging the generation of sleeping position event in the head pose detection module, specially gives when head inclination angle is greater than
Threshold value when, recognize the sleeping position event of a posture head inclination, statistics is greater than the duration of given threshold value；Work as head
When portion's tilt angle is less than or equal to given threshold value, dwell time statistics carries out circulation time statistics according to this.
6. a kind of smart television human fatigue detection method based on face multiple features fusion, which is characterized in that include following step
It is rapid:
Step 1: opening intelligent fatigue detecting and control setup module, carry out fatigue detecting relative parameters setting；
Step 2: starting video data acquiring module carries out video data acquiring；
Step 3: the video data obtained according to step 2 carries out facial feature points detection, and stores；
Step 4: the human face characteristic point obtained according to step 3 calculates eyes aspect ratio；
Step 5: the eyes aspect ratio being calculated according to step 4, calculate the eye fatigue factor, according to the eye fatigue factor with give
Whether the multilevel iudge for determining threshold value occurs blink event, counts number of winks in period demand；
Step 6: the coordinate by obtaining 6 key feature point datas of two eyes in left and right carries out operation and obtains head inclination angle
Degree decides whether that sleeping position event, head pose dipping event duration in statistics period demand occurs；
Step 7: head pose in the period demand that number of winks and step 6 obtain in the period demand obtained according to step 5
Dipping event duration cascading judgement obtains the tired alert feature value of intelligence, makes fatigue state judgement.
7. a kind of smart television human fatigue detection method based on face multiple features fusion according to claim 6, special
Sign is: in the step 4 calculate eyes aspect ratio, definition EAR be eyes the ratio of width to height, Dist for two characteristic points straight line away from
From Ehwr is eyes aspect ratio, and calculation formula is as follows:
Wherein, F indicates 6 crucial eye feature point coordinates, and i and j indicate the index of main points eye feature point, and Fi indicates the
I crucial eye feature point, Fj indicate j-th of crucial eye feature point, and a characterization is currently left eye or right eye, 0 table of a value
Show left eye, value 1 indicates right eye.
8. a kind of smart television human fatigue detection method based on face multiple features fusion according to claim 7, special
Sign is: the eye fatigue factor is calculated in the step 5, the definition eye fatigue factor is D_eye, and calculation formula is as follows:
Wherein, S (T) indicates drowsiness time factor, is discrete piecewise function, and variable T indicates the current time of television system.
9. a kind of smart television human fatigue detection method based on face multiple features fusion according to claim 6, special
Sign is: the calculating of head inclination angle in the step 6, and definition current head tilt angle is theta, and calculation formula is such as
Under:
Wherein, F (a, b, i) is the human eye key feature coordinate of definition, and the 1st parameter a characterization is currently left eye or right eye, is taken
Value 0 indicates left eye, and value 1 indicates right eye；2nd parameter b expression is currently x coordinate or y-coordinate, and value is respectively x and y；
3rd parameter i indicates the index of eyes main points feature, and value is 1~6.
10. a kind of smart television human fatigue detection method based on face multiple features fusion according to claim 6, special
Sign is: the calculating of intelligence fatigue alert feature value in the step 7 defines the tired alert feature Th_tired of intelligence, calculates
Formula is as follows:
Th_tired=count (AEB) * θ (Time_SleepPos)),
Wherein, Tinterval is periodical time decision, and θ is posture feature weight, and q is weight coefficient, Time_SleepPos
For the duration of drowsiness sleeping position, count (AEB) indicates the number of winks detected in judgement cycle time Tinterval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811410439.6A CN109635851A (en) | 2018-11-23 | 2018-11-23 | A kind of smart television human fatigue detection system and method based on face multiple features fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811410439.6A CN109635851A (en) | 2018-11-23 | 2018-11-23 | A kind of smart television human fatigue detection system and method based on face multiple features fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109635851A true CN109635851A (en) | 2019-04-16 |
Family
ID=66069428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811410439.6A Pending CN109635851A (en) | 2018-11-23 | 2018-11-23 | A kind of smart television human fatigue detection system and method based on face multiple features fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109635851A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110675291A (en) * | 2019-09-17 | 2020-01-10 | 智教源清(北京)科技有限公司 | Method for acquiring fatigue eye time of students, server and readable storage medium |
| CN111402317A (en) * | 2020-03-26 | 2020-07-10 | 北京新氧科技有限公司 | Eye feature measuring method, device and terminal |
| CN113591635A (en) * | 2021-07-19 | 2021-11-02 | 东莞职业技术学院 | Intelligent office desk and chair, control method and storage medium |
| CN115019220A (en) * | 2022-04-19 | 2022-09-06 | 北京拙河科技有限公司 | Posture tracking method and system based on deep learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622600A (en) * | 2012-02-02 | 2012-08-01 | 西南交通大学 | High-speed train driver alertness detecting method based on face image and eye movement analysis |
| CN106372621A (en) * | 2016-09-30 | 2017-02-01 | 防城港市港口区高创信息技术有限公司 | Face recognition-based fatigue driving detection method |
| CN107679468A (en) * | 2017-09-19 | 2018-02-09 | 浙江师范大学 | A kind of embedded computer vision detects fatigue driving method and device |
-
2018
- 2018-11-23 CN CN201811410439.6A patent/CN109635851A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622600A (en) * | 2012-02-02 | 2012-08-01 | 西南交通大学 | High-speed train driver alertness detecting method based on face image and eye movement analysis |
| CN106372621A (en) * | 2016-09-30 | 2017-02-01 | 防城港市港口区高创信息技术有限公司 | Face recognition-based fatigue driving detection method |
| CN107679468A (en) * | 2017-09-19 | 2018-02-09 | 浙江师范大学 | A kind of embedded computer vision detects fatigue driving method and device |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110675291A (en) * | 2019-09-17 | 2020-01-10 | 智教源清(北京)科技有限公司 | Method for acquiring fatigue eye time of students, server and readable storage medium |
| CN111402317A (en) * | 2020-03-26 | 2020-07-10 | 北京新氧科技有限公司 | Eye feature measuring method, device and terminal |
| CN113591635A (en) * | 2021-07-19 | 2021-11-02 | 东莞职业技术学院 | Intelligent office desk and chair, control method and storage medium |
| CN115019220A (en) * | 2022-04-19 | 2022-09-06 | 北京拙河科技有限公司 | Posture tracking method and system based on deep learning |
| CN115019220B (en) * | 2022-04-19 | 2023-02-03 | 北京拙河科技有限公司 | Posture tracking method and system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109635851A (en) | A kind of smart television human fatigue detection system and method based on face multiple features fusion | |
| CN105022981B (en) | A kind of method, device and mobile terminal detecting human eye health state | |
| CN106965675B (en) | A kind of lorry swarm intelligence safety work system | |
| CN101305913B (en) | Face beauty assessment method based on video | |
| CN106203394B (en) | Fatigue driving safety monitoring method based on human eye state detection | |
| CN105227832B (en) | A kind of self-timer method, self-heterodyne system and camera terminal based on critical point detection | |
| CN108960056A (en) | A kind of fall detection method based on posture analysis and Support Vector data description | |
| CN110319545A (en) | Environmental management technique, device and air-conditioning | |
| CN101655907B (en) | Trainman driving state monitoring intelligent alarm system | |
| CN107273071A (en) | Electronic installation, screen adjustment system and method | |
| CN106372621A (en) | Face recognition-based fatigue driving detection method | |
| CN110119672A (en) | A kind of embedded fatigue state detection system and method | |
| CN106792177A (en) | A kind of TV control method and system | |
| CN109190522B (en) | Living body detection method based on infrared camera | |
| CN101593425A (en) | A kind of fatigue driving monitoring method and system based on machine vision | |
| CN103093124A (en) | Method for restricting the usage of electronic equipment and electronic equipment | |
| CN110458010A (en) | A kind of intelligent desk lamp, a kind of sitting posture detecting method and a kind of electronic equipment | |
| CN108377339A (en) | A kind of photographic method and camera arrangement | |
| CN107340849A (en) | Mobile device and its eyeshield control method | |
| CN110063736B (en) | Eye movement parameter monitoring fatigue detection and wake-up promotion system based on MOD-Net network | |
| CN105095885A (en) | Human eyes state detection method and human eyes state detection device | |
| CN109948435A (en) | Sitting posture prompting method and device | |
| CN111616718A (en) | Method and system for detecting fatigue state of driver based on attitude characteristics | |
| CN109785592A (en) | Desk lamp and its control method, computer storage medium | |
| CN107958572A (en) | A kind of baby monitoring systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190416 |
|
| RJ01 | Rejection of invention patent application after publication |