CN106203394A - Fatigue driving safety monitoring method based on human eye state detection - Google Patents
Fatigue driving safety monitoring method based on human eye state detection Download PDFInfo
- Publication number
- CN106203394A CN106203394A CN201610590537.7A CN201610590537A CN106203394A CN 106203394 A CN106203394 A CN 106203394A CN 201610590537 A CN201610590537 A CN 201610590537A CN 106203394 A CN106203394 A CN 106203394A
- Authority
- CN
- China
- Prior art keywords
- human eye
- eye
- state
- layer
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Abstract
The present invention relates to a kind of fatigue driving safety monitoring method based on human eye state detection.The present invention has taken into full account the various complicated factors in driving procedure, first eliminates the insecure situation of human eye data that accidentalia causes, is accurately positioned human eye, then judges, from human eye vision aspect, the state that human eye is current.The present invention need not the technology such as eyeball analysis and the trajectory analysis of complexity.
Description
Technical field
The invention belongs to intelligent and safe technical field, relate to a kind of fatigue driving safety monitoring based on human eye state detection
Method.
Background technology
Fatigue driving detection is the pith in safe driving, detects the fatigue of driver in traveling the most automatically
State, and remind driver safety to drive, this problem have become as one wide interesting issue.Current about fatigue detecting
There is a variety of method." driver fatigue detection " 104887253A is by gathering and vehicle yaw in time period
The data that rate is relevant, and calculate its deviation with ideal trajectory to provide the instruction of dangerous driving state.Owing to affecting yaw-rate
The factor changed is a lot, and the reliability of the yaw-rate data that this method gathers is the strongest." fatigue detecting system and method "
105718033A, by gathering the picture of eyeball image, and adds up wherein ratio shared by red pixel point.This method requirement
Eyeball picture quality is higher, and this is relatively difficult to realize during actual use.
Summary of the invention
The present invention needs the problem solved to be the approximate region obtaining human eye the most accurately, and the technology of employing is mainly people
Face detection and facial modeling.The method can be accurately positioned human eye area.On the other hand need to solve how to determine
Whether there is human eye and human eye in human eye area opens closed state, then opens, by the human eye in statistics a period of time, the ratio closed
Rate carries out the detection of fatigue state.
The inventive method comprises the following steps:
Step 1. utilizes infrared camera to gather facial image.
Step 2. is by facial image gray processing, by Face datection algorithm, detects face location.
Step 3. utilizes the face location or the face location of the believable facial feature points detection of former frame detected, will
Face normalization.
Step 4. utilizes facial modeling algorithm based on LBF, locating human face's characteristic point；Calculate respectively in right and left eyes
Heart point, to the distance of prenasale, is then chosen the eye center point close to prenasale, is obtained the position of credible human eye.
Step 5., on RGB color figure, intercepts human eye regional area picture centered by the credible human eye chosen, for working as
Front human eye regional area picture, adds up the color histogram of tri-Color Channels of RGB respectively, and carries out color histogram normalizing
Change.
Step 6. carries out statistics of histogram to current human eye regional area, calculates gray value and accounts for the ratio of whole gray-scale map
The example gray value more than 50%, if this gray value is less than the threshold value set, then it is assumed that human eye is lighttight sunglasses, and is given
Non-human eye state, and no longer carry out network judgement.
Normalized image is added in deep neural network model by step 7., the output of deep neural network model work as forefathers
The state of eye, is divided into three kinds: open eyes, close one's eyes and non-human eye.
Step 8. arranges value of feedback according to the human eye detection state of present frame, detects for next frame human eye feature point.
If non-human eye, then it is insincere for arranging current face's positioning feature point result, then next frame then needs to re-use face
The human face region that detection algorithm provides.If opening and closing eyes, then it is credible for arranging value of feedback, updates the current shape that opens and closes eyes simultaneously
The state pool of state.
Step 9. adds up the eye closing rate of the state pool that opens and closes eyes under present frame, if current eye closing rate exceedes default threshold
Value, then export alarm condition.If current state is abnormal, then provide abnormality alarm.
Beneficial effects of the present invention:
1, position of human eye can be positioned the most accurately, and the interference such as illumination and glasses is had certain robustness.
2, make use of inter-frame information, it is possible to reduce the false alarm that accidentalia is brought.
3, alarm is the highest, it is allowed to user, according to different processors, adjusts threshold value thus improves verification and measurement ratio.
Accompanying drawing explanation
Fig. 1 is the inventive method flow chart.
Fig. 2 is eyes detection flow chart.
Fig. 3 is eye status detect net network structure.
Detailed description of the invention
Below in conjunction with embodiment and accompanying drawing 1, the invention will be further described:
The equipment that the data acquisition of the present embodiment uses is infrared camera, is installed in driver's cabin.Installation requirement: photographic head occupies
Middle erection, it is desirable to photographic head focal length is about 8MM, photographic head distance face is 70-80cm.The parameter that set algorithm requires, main
Including the video frame number of statistics, the upper lower threshold value of fatigue detecting.To IR video stream, the algorithm that calling this algorithm provides connects
Mouthful, algorithm can call corresponding model provide the fatigue state of current driver's according to input picture, if fatigue will be given
Warning message.Specific embodiments is as follows, sees Fig. 1 and Fig. 2:
1. utilize infrared camera to gather facial image.
2. by facial image gray processing, utilize conventional Face datection algorithm, detect face location.
3. utilize the face location detected or the face location of the believable facial feature points detection of former frame, by face
Normalization.Facial feature points detection module combines the human eye state value of feedback of former frame, specifically arranges by described in step 8.
4. utilize facial modeling algorithm based on LBF, locating human face's characteristic point.The present invention calculates right and left eyes respectively
Central point, to the distance of prenasale, is then chosen the eye center point close to prenasale, is obtained the position of credible human eye.
5., on RGB color figure, centered by the credible human eye chosen, intercept the human eye regional area figure of 32*32 size
Sheet, for current human eye regional area picture, adds up the color histogram of tri-Color Channels of RGB respectively, and it is straight to carry out color
Side's figure normalization.
6. pair current human eye regional area carries out statistics of histogram, and calculating gray value accounts for the ratio of whole gray-scale map and surpasses
Cross the gray value of 50%, if this gray value is less than the threshold value set, then it is assumed that human eye is lighttight sunglasses, need to provide inhuman
Eye state, and no longer carry out network judgement.
7. normalized image is added in deep neural network model, as it is shown on figure 3, input picture is through one layer of core size
Being 5, pad is the convolutional layer conv1 of 2, after convolution dimension of picture keep constant, thereafter connect Batch Norm layer, scale layer with
And Pooling layer, connect activation primitive Relu afterwards.Hereafter reconnecting one layer of core size is 3, and pad is the convolutional layer conv1_ of 2
1, Batch Norm layer and relu layer, then connecting core is 5, and pad is the convolutional layer of 2, after three-layer coil is long-pending, needs to reduce and works as
The number of front parameters optimization, uses dropout strategy, reduces the probability of over-fitting.Connect full articulamentum afterwards, through dropout
The probability of rear calculating current network output.The final state exporting current human eye, is divided into three kinds: open eyes, and closes one's eyes and non-human eye.
8. value of feedback is set according to the human eye detection state of present frame, for next frame human eye feature point detection module.
If non-human eye, then it is insincere for arranging current face's positioning feature point result, then next frame then needs to re-use face
The human face region that detection algorithm provides.If opening and closing eyes, then it is credible for arranging value of feedback, updates the current shape that opens and closes eyes simultaneously
The state pool of state.
9. the eye closing rate of the state pool that opens and closes eyes under statistics present frame, if current eye closing rate exceedes default threshold value, then
Output alarm condition.If current state is abnormal, then provide abnormality alarm.
For being accurately positioned position of human eye, the present invention on the basis of feature point detection algorithm based on LBF, add based on
The initialization feature method of current face's attitude.That is, if former frame is through human eye state analysis, if it is decided that current signature spot check
Survey rationally, then utilize former frame feature point detection result, estimate the attitude of current face, then initial average shape is carried out appearance
State disturbance, the shape after disturbance is as the primary iteration shape of conventional distinguished point based detection algorithm.
It is credible human eye area that the present invention detects the input human eye area of human eye state.I.e. pick out more accurate in right and left eyes
True local human eye area.Particularly as follows: calculating right and left eyes central point is to the distance of prenasale respectively, then choose from prenasale relatively
Near eye center point, and intercept face local area image with this.This realization not only reduces operand but also strengthen people
The robustness of eye feature point detection.
Human eye state, in addition to opening eyes and closing one's eyes, is also provided with the third human eye state: non-human eye.Non-human eye is as one
Feedback mechanism, is input in facial feature points detection module.
The detection algorithm that opens and closes eyes that the present invention uses is a kind of strategy based on deep neural network.Special according to human eye local
The feature planned network structure levied, illumination effect need to be removed through histogram equalization by human eye topography.In network structure first
Be designed as core size and the picture boundary of the convolutional layer of layer expand number and are respectively set to 5 and 2, input picture after convolutional layer,
The size of picture keeps constant, and second layer convolutional layer design tactics also keeps dimension of picture constant, and human eye so can be kept special
The integrity levied.Overall network structure includes three-layer coil lamination, connects Batch Norm layer and Pooling after every layer of convolutional layer
Layer, is followed by the full articulamentum of two-layer.
Human eye state pond is set, value of feedback is set according to the human eye detection state of present frame, for next frame human eye feature point
Detection uses.If non-human eye, then it is insincere for arranging current face's positioning feature point result, then next frame then needs again
Utilize the human face region that Face datection operator provides.If opening and closing eyes, then update the state pool of current human eye state.
To sum up, the present invention has taken into full account the various complicated factors in driving procedure, first eliminates accidentalia and causes
The insecure situation of human eye data, be accurately positioned human eye, then judge, from human eye vision aspect, the state that human eye is current.This
The technology such as the bright eyeball analysis that need not complexity and trajectory analysis.
The above, only presently preferred embodiments of the present invention, it is not intended to limit protection scope of the present invention, should carry
Understanding, the present invention is not limited to implementation as described herein, and the purpose that these implementations describe is to help this area
In technical staff put into practice the present invention.
Claims (3)
1. fatigue driving safety monitoring method based on human eye state detection, it is characterised in that the method comprises the following steps:
Step 1. utilizes infrared camera to gather facial image；
Step 2. is by facial image gray processing, by Face datection algorithm, detects face location；
Step 3. utilizes the face location or the face location of the believable facial feature points detection of former frame detected, by face
Normalization；
Step 4. utilizes facial modeling algorithm based on LBF, locating human face's characteristic point；Calculate right and left eyes central point respectively
To the distance of prenasale, then choose the eye center point close to prenasale, obtain the position of credible human eye；
Step 5., on RGB color figure, intercepts human eye regional area picture, for working as forefathers centered by the credible human eye chosen
Eye regional area picture, adds up the color histogram of tri-Color Channels of RGB respectively, and carries out color histogram normalization；
Step 6. carries out statistics of histogram to current human eye regional area, and calculating gray value accounts for the ratio of whole gray-scale map and surpasses
Cross the gray value of 50%, if this gray value is less than the threshold value set, then it is assumed that human eye is lighttight sunglasses, and provides inhuman
Eye state, and no longer carry out network judgement；
Normalized image is added in deep neural network model by step 7., exports the state of current human eye, is divided into three kinds: open
Eye, eye closing and non-human eye；
Step 8. arranges value of feedback according to the human eye detection state of present frame, detects for next frame human eye feature point；If
For non-human eye, then it is insincere for arranging current face's positioning feature point result, then next frame then needs to re-use Face datection
The human face region that algorithm provides；If opening and closing eyes, then it is credible for arranging value of feedback, updates the current state that opens and closes eyes simultaneously
State pool；
Step 9. adds up the eye closing rate of the state pool that opens and closes eyes under present frame, if current eye closing rate exceedes default threshold value, then
Output alarm condition；If current state is abnormal, then provide abnormality alarm.
Fatigue driving safety monitoring method based on human eye state detection the most according to claim 1, it is characterised in that: step
The size of the human eye regional area picture in rapid 5 is 32*32.
Fatigue driving safety monitoring method based on human eye state detection the most according to claim 1, it is characterised in that: step
Input picture in rapid 7 is through one layer of core a size of 5, and pad is the convolutional layer conv1 of 2, and after convolution, dimension of picture keeps constant,
Thereafter connect Batch Norm layer, scale layer and Pooling layer, connect activation primitive Relu afterwards；Hereafter one layer is reconnected
Core size is 3, and pad is convolutional layer conv1_1, Batch Norm layer and the relu layer of 2, and then connecting core is 5, and pad is 2
Convolutional layer, after three-layer coil is long-pending, uses dropout strategy to reduce the probability of over-fitting；Connect full articulamentum, warp afterwards
The probability of current network output is calculated after dropout.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610590537.7A CN106203394B (en) | 2016-07-26 | 2016-07-26 | Fatigue driving safety monitoring method based on human eye state detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610590537.7A CN106203394B (en) | 2016-07-26 | 2016-07-26 | Fatigue driving safety monitoring method based on human eye state detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106203394A true CN106203394A (en) | 2016-12-07 |
| CN106203394B CN106203394B (en) | 2019-04-26 |
Family
ID=57495099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610590537.7A Active CN106203394B (en) | 2016-07-26 | 2016-07-26 | Fatigue driving safety monitoring method based on human eye state detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106203394B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292261A (en) * | 2017-06-16 | 2017-10-24 | 深圳天珑无线科技有限公司 | A kind of photographic method and its mobile terminal |
| CN107832721A (en) * | 2017-11-16 | 2018-03-23 | 百度在线网络技术（北京）有限公司 | Method and apparatus for output information |
| CN107977623A (en) * | 2017-11-30 | 2018-05-01 | 睿视智觉（深圳）算法技术有限公司 | A kind of robustness human eye state determination methods |
| CN108294759A (en) * | 2017-01-13 | 2018-07-20 | 天津工业大学 | A kind of Driver Fatigue Detection based on CNN Eye state recognitions |
| CN108415653A (en) * | 2018-03-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Screen locking method and device for terminal device |
| CN108701214A (en) * | 2017-12-25 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Image processing method, device and equipment |
| CN108805085A (en) * | 2018-06-14 | 2018-11-13 | 北京理工大学 | Intelligent sleep detection method based on eye recognition and system |
| CN108814630A (en) * | 2018-07-11 | 2018-11-16 | 长安大学 | A kind of driving behavior monitor detection device and method |
| CN108960071A (en) * | 2018-06-06 | 2018-12-07 | 武汉幻视智能科技有限公司 | A kind of eye opening closed-eye state detection method |
| CN109508632A (en) * | 2018-09-30 | 2019-03-22 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN109711309A (en) * | 2018-12-20 | 2019-05-03 | 北京邮电大学 | A kind of method whether automatic identification portrait picture closes one's eyes |
| CN110115796A (en) * | 2019-05-06 | 2019-08-13 | 苏州国科视清医疗科技有限公司 | The awake system of fatigue detecting and rush of eye movement parameter monitoring based on N-range image processing algorithm |
| CN110232327A (en) * | 2019-05-21 | 2019-09-13 | 浙江师范大学 | A kind of driving fatigue detection method based on trapezoidal concatenated convolutional neural network |
| CN111104817A (en) * | 2018-10-25 | 2020-05-05 | 中车株洲电力机车研究所有限公司 | Fatigue detection method based on deep learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080100857A1 (en) * | 2006-10-27 | 2008-05-01 | Seiko Epson Corporation | Image scanner printer |
| CN101377814A (en) * | 2007-08-27 | 2009-03-04 | 索尼株式会社 | Face image processing apparatus, face image processing method, and computer program |
| CN101639894A (en) * | 2009-08-31 | 2010-02-03 | 华南理工大学 | Method for detecting train driver behavior and fatigue state on line and detection system thereof |
| CN102831408A (en) * | 2012-08-29 | 2012-12-19 | 华南理工大学 | Human face recognition method |
| CN103049740A (en) * | 2012-12-13 | 2013-04-17 | 杜鹢 | Method and device for detecting fatigue state based on video image |
| CN103065121A (en) * | 2012-12-13 | 2013-04-24 | 李秋华 | Engine driver state monitoring method and device based on video face analysis |
| CN104269028A (en) * | 2014-10-23 | 2015-01-07 | 深圳大学 | Fatigue driving detection method and system |
-
2016
- 2016-07-26 CN CN201610590537.7A patent/CN106203394B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080100857A1 (en) * | 2006-10-27 | 2008-05-01 | Seiko Epson Corporation | Image scanner printer |
| CN101377814A (en) * | 2007-08-27 | 2009-03-04 | 索尼株式会社 | Face image processing apparatus, face image processing method, and computer program |
| CN101639894A (en) * | 2009-08-31 | 2010-02-03 | 华南理工大学 | Method for detecting train driver behavior and fatigue state on line and detection system thereof |
| CN102831408A (en) * | 2012-08-29 | 2012-12-19 | 华南理工大学 | Human face recognition method |
| CN103049740A (en) * | 2012-12-13 | 2013-04-17 | 杜鹢 | Method and device for detecting fatigue state based on video image |
| CN103065121A (en) * | 2012-12-13 | 2013-04-24 | 李秋华 | Engine driver state monitoring method and device based on video face analysis |
| CN104269028A (en) * | 2014-10-23 | 2015-01-07 | 深圳大学 | Fatigue driving detection method and system |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108294759A (en) * | 2017-01-13 | 2018-07-20 | 天津工业大学 | A kind of Driver Fatigue Detection based on CNN Eye state recognitions |
| CN107292261A (en) * | 2017-06-16 | 2017-10-24 | 深圳天珑无线科技有限公司 | A kind of photographic method and its mobile terminal |
| CN107832721A (en) * | 2017-11-16 | 2018-03-23 | 百度在线网络技术（北京）有限公司 | Method and apparatus for output information |
| CN107832721B (en) * | 2017-11-16 | 2021-12-07 | 百度在线网络技术（北京）有限公司 | Method and apparatus for outputting information |
| CN107977623A (en) * | 2017-11-30 | 2018-05-01 | 睿视智觉（深圳）算法技术有限公司 | A kind of robustness human eye state determination methods |
| CN108701214A (en) * | 2017-12-25 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Image processing method, device and equipment |
| CN108415653A (en) * | 2018-03-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Screen locking method and device for terminal device |
| CN108960071A (en) * | 2018-06-06 | 2018-12-07 | 武汉幻视智能科技有限公司 | A kind of eye opening closed-eye state detection method |
| CN108805085A (en) * | 2018-06-14 | 2018-11-13 | 北京理工大学 | Intelligent sleep detection method based on eye recognition and system |
| CN108814630A (en) * | 2018-07-11 | 2018-11-16 | 长安大学 | A kind of driving behavior monitor detection device and method |
| CN109508632A (en) * | 2018-09-30 | 2019-03-22 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN111104817A (en) * | 2018-10-25 | 2020-05-05 | 中车株洲电力机车研究所有限公司 | Fatigue detection method based on deep learning |
| CN109711309A (en) * | 2018-12-20 | 2019-05-03 | 北京邮电大学 | A kind of method whether automatic identification portrait picture closes one's eyes |
| CN109711309B (en) * | 2018-12-20 | 2020-11-27 | 北京邮电大学 | Method for automatically identifying whether portrait picture is eye-closed |
| CN110115796A (en) * | 2019-05-06 | 2019-08-13 | 苏州国科视清医疗科技有限公司 | The awake system of fatigue detecting and rush of eye movement parameter monitoring based on N-range image processing algorithm |
| CN110115796B (en) * | 2019-05-06 | 2021-10-22 | 苏州国科视清医疗科技有限公司 | Fatigue detection and awakening promotion system based on eye movement parameter monitoring of N-range image processing algorithm |
| CN110232327A (en) * | 2019-05-21 | 2019-09-13 | 浙江师范大学 | A kind of driving fatigue detection method based on trapezoidal concatenated convolutional neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106203394B (en) | 2019-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106203394A (en) | Fatigue driving safety monitoring method based on human eye state detection | |
| CN107491769A (en) | Method for detecting fatigue driving and system based on AdaBoost algorithms | |
| CN106965675B (en) | A kind of lorry swarm intelligence safety work system | |
| CN103440475B (en) | A kind of ATM user face visibility judge system and method | |
| CN112016457A (en) | Driver distraction and dangerous driving behavior recognition method, device and storage medium | |
| WO2016038784A1 (en) | Driver state determination apparatus | |
| CN103839379B (en) | Automobile and driver fatigue early warning detecting method and system for automobile | |
| CN109919049A (en) | Fatigue detection method based on deep learning human face modeling | |
| CN106781282A (en) | A kind of intelligent travelling crane driver fatigue early warning system | |
| CN105286802B (en) | Driver Fatigue Detection based on video information | |
| CN103888680B (en) | A kind of adjusting method of camera time for exposure | |
| CN103761516B (en) | ATM abnormal human face detection based on video monitoring | |
| CN110532976A (en) | Method for detecting fatigue driving and system based on machine learning and multiple features fusion | |
| CN104063700B (en) | The method of eye center point location in natural lighting front face image | |
| CN107229922A (en) | A kind of fatigue driving monitoring method and device | |
| CN106355838A (en) | Fatigue driving detection method and system | |
| CN106504751A (en) | Self adaptation lip reading exchange method and interactive device | |
| CN103400110A (en) | Abnormal face detection method in front of ATM (automatic teller machine) | |
| CN110728241A (en) | Driver fatigue detection method based on deep learning multi-feature fusion | |
| CN110490139A (en) | Night fatigue driving judgment method based on recognition of face | |
| CN108040997A (en) | A kind of insect pest monitoring method based on machine vision | |
| CN109740477A (en) | Study in Driver Fatigue State Surveillance System and its fatigue detection method | |
| CN103198616A (en) | Method and system for detecting fatigue driving based on head and neck movement feature recognition of driver | |
| CN106650635A (en) | Method and system for detecting rearview mirror viewing behavior of driver | |
| CN110103816A (en) | A kind of driving condition detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Fatigue driving safety monitoring method based on human eye state detection Effective date of registration: 20190821 Granted publication date: 20190426 Pledgee: Hangzhou Yuhang Small and Medium-sized Enterprise Transfer Service Co., Ltd. Pledgor: ZHEJIANG ICARE VISION TECHNOLOGY CO., LTD. Registration number: Y2019330000020 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |