WO2015192171A1 - Monitoring drowsiness - Google Patents
Monitoring drowsiness Download PDFInfo
- Publication number
- WO2015192171A1 WO2015192171A1 PCT/AU2015/000359 AU2015000359W WO2015192171A1 WO 2015192171 A1 WO2015192171 A1 WO 2015192171A1 AU 2015000359 W AU2015000359 W AU 2015000359W WO 2015192171 A1 WO2015192171 A1 WO 2015192171A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- eye movement
- drowsiness
- amplitude
- values
- Prior art date
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/14—Arrangements specially adapted for eye photography
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1103—Detecting eye twinkling
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1113—Local tracking of patients, e.g. in a hospital or private home
- A61B5/1114—Tracking parts of the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7271—Specific aspects of physiological measurement analysis
- A61B5/7278—Artificial waveform generation or derivation, e.g. synthesising signals from measured signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
Definitions
- This invention relates to methods of measuring drowsiness and in particular of using eyelid movement data from any source that has low sampling rates. Background to the invention.
- USA patents 7071831 , 7616125 and 7791491 relate to a method and algorithm for predicting the onset of potentially fatal drowsiness.
- the method in practice uses spectacle mounted I R sensors to sense eye and eyelid movement and measure amplitude and velocity of these movements for processing in an algorithm that provides a measure that is applicable to a scale of drowsiness.
- Other sensor systems such as cameras could be used to collect this data but have not been used as their sampling rates are low.
- the attraction of a camera based system is that spectacle need not be worn to collect the data and is thus less intrusive.
- USA patent 7043056 to a camera based method of determining a head pose measurement.
- USA patent 7460693 to a camera based method of locating a face within an input image including eye locations.
- USA patent 7653213 to a face tracking system utilising a Kalman filter and deriving a Jacobian.
- USA patent 8165347 to a face tracking system which includes a determination if glasses are being worn.
- USA patent 7809160 for eye tracking without camera calibration USA patent 7809160 for eye tracking without camera calibration.
- the present invention provides a method of determining drowsiness which includes the steps of
- This invention is predicated on the unexpected discovery that lower sampling rates can provide sufficient data for use in the method of USA patents 7071831 , 7616125 and 779149.
- the system of those patents uses a sample rate of 500Hz to gather amplitude measurements for eyelid movements.
- the sampling rates for various type of sensor devices may be 50Hz, 100Hz, 200Hz.
- the upper limit for the sample rate of the eye movement data source is 200Hz.
- An alternate embodiment of the invention avoids the need to adjust the sample rate of the eye movement data source (such as that obtained from a video camera). Modifications to the methods of collation and calculation of eye
- movement amplitude, velocity and durations used in the JDS algorithm may be adjusted to account for low sampling rates.
- the amplitude to velocity ratio for eyelid opening and closing are used as the main measure of drowsiness onset In the John Drowsiness Scale (JDS).
- JDS John Drowsiness Scale
- the ratio of the amplitude of to the maximum velocity (AVR) for both closing and opening phases during blinks increases with drowsiness and can be used to predict lapses in vigilance.
- the AVR for eyelid closure and reopening are different for the same amplitude. Generally eyelids close more quickly than they reopen and the two velocities are only moderately correlated. Sleep deprivation , restriction or other reasons for drowsiness increases AVR for both closing and reopening.
- the values calculated for the purposes of comparison need to be collated over a predetermined period of time and expressed in an appropriate way. These calculations are preferentially expressed as averages, standard deviations, percentiles or counts.
- the eyelid parameters measured and the values selected for calculation can be determined by conducting trials and may be any suitable combination of parameters. Preferably the velocity to amplitude ratios are calculated for each detected movement and then expressed as averages and standard deviations over a predetermined interval. Other parameters such as duration of opening, closure and closing may also be collated and included in the final calculated value. Eye movements such as saccades may be used as additional parameters if they are available from the data source. The various parameters are preferably weighted in reaching the final calculation.
- Eyelid and eye movement may be monitored using any suitable technology or sensors including video or digital camera technology to identify and measure the appropriate eye movements. Detailed description of the invention
- Figure 1A is a good quality eyelid aperture data signal derived from video showing a 30 second period of blinks
- Figure 1 B is a noisy eyelid aperture data signal derived from video showing a 24 second period of noise between blinks
- Figure 2 illustrates a graph of Johns drowsiness score derived fromvideo based signals over 120 minutes
- Figure 3 illustrates JDS score for 500Hz vs 50Hz; 50Hz to reflect the applicability of JDS for lower sampling rates;
- Figure 4 illustrates minimal error of 50Hz signal against the original 500Hz signal.
- a data set of eyelid movements obtained using a camera-based system was obtained using a third party program which supplied the data as a CSV output file with time and eye opening (aperture) value for each video sample.
- the data was then upscaled to 500 Hz and the amplitude rescaled.
- FIG. 1A is a good quality signal and figure 1 B illustrates that blinks can be identified in a noisy signal.
- Figure 3 illustrates the difference in the JDS scores over time derived from the original 500Hz data and that derived from the 50Hz data set.
- Figure 4 illustrates the error of the JDS scores derived from the 50Hz data set against the original 500Hz data set. This demonstrates that the use of a lower sampling frequency provides reliable scores.
- An alternate embodiment of the invention is possible without the need to adjust the sample rate of the eye movement data source (such as that obtained from a video camera). Modifications to the methods of collation and calculation of eye movement amplitude, velocity and durations may be adjusted to account for alternate sampling rates.
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/318,417 US20170119248A1 (en) | 2014-06-20 | 2015-06-19 | Monitoring drowsiness |
AU2015278237A AU2015278237B2 (en) | 2014-06-20 | 2015-06-19 | Monitoring drowsiness |
EP15809658.6A EP3157433B1 (en) | 2014-06-20 | 2015-06-19 | Monitoring drowsiness |
CN201580029511.9A CN106456077A (en) | 2014-06-20 | 2015-06-19 | Monitoring drowsiness |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2014902364 | 2014-06-20 | ||
AU2014902364A AU2014902364A0 (en) | 2014-06-20 | Monitoring drowsiness |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015192171A1 true WO2015192171A1 (en) | 2015-12-23 |
Family
ID=54934565
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/AU2015/000359 WO2015192171A1 (en) | 2014-06-20 | 2015-06-19 | Monitoring drowsiness |
Country Status (5)
Country | Link |
---|---|
US (1) | US20170119248A1 (en) |
EP (1) | EP3157433B1 (en) |
CN (1) | CN106456077A (en) |
AU (1) | AU2015278237B2 (en) |
WO (1) | WO2015192171A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019095117A1 (en) * | 2017-11-14 | 2019-05-23 | 华为技术有限公司 | Facial image detection method and terminal device |
CN113164155A (en) * | 2018-09-07 | 2021-07-23 | 心脏起搏器股份公司 | System and method for reconstructing heart sounds |
US20210386345A1 (en) | 2018-10-23 | 2021-12-16 | Sdip Holdings Pty Ltd | Devices and processing systems configured to enable extended monitoring and analysis of subject neurological factors via blepharometric data collection |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006092022A1 (en) * | 2005-03-04 | 2006-09-08 | Sleep Diagnostics Pty. Ltd | Measuring alertness |
EP2410501A1 (en) * | 2009-03-19 | 2012-01-25 | Aisin Seiki Kabushiki Kaisha | Drowsiness assessment device and program |
WO2014031042A1 (en) * | 2012-08-20 | 2014-02-27 | Autoliv Development Ab | Eyelid movement processing for detection of drowsiness |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AUPR872301A0 (en) * | 2001-11-08 | 2001-11-29 | Sleep Diagnostics Pty Ltd | Alertness monitor |
US8487775B2 (en) * | 2006-06-11 | 2013-07-16 | Volvo Technology Corporation | Method and apparatus for determining and analyzing a location of visual interest |
US8140149B2 (en) * | 2008-07-04 | 2012-03-20 | Toyota Jidosha Kabushiki Kaisha | Drowsiness detector |
JP4657379B1 (en) * | 2010-09-01 | 2011-03-23 | 株式会社ナックイメージテクノロジー | High speed video camera |
TWI478691B (en) * | 2012-01-06 | 2015-04-01 | Wistron Corp | Drowsy detction method and device using the same |
JP5938655B2 (en) * | 2012-01-11 | 2016-06-22 | パナソニックIpマネジメント株式会社 | Playback device, imaging device, and program |
US9035808B2 (en) * | 2013-06-05 | 2015-05-19 | Mstar Semiconductor, Inc. | Communication system and sample rate converter thereof |
JP6227996B2 (en) * | 2013-12-18 | 2017-11-08 | 浜松ホトニクス株式会社 | Measuring device and measuring method |
-
2015
- 2015-06-19 WO PCT/AU2015/000359 patent/WO2015192171A1/en active Application Filing
- 2015-06-19 AU AU2015278237A patent/AU2015278237B2/en active Active
- 2015-06-19 EP EP15809658.6A patent/EP3157433B1/en active Active
- 2015-06-19 CN CN201580029511.9A patent/CN106456077A/en active Pending
- 2015-06-19 US US15/318,417 patent/US20170119248A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006092022A1 (en) * | 2005-03-04 | 2006-09-08 | Sleep Diagnostics Pty. Ltd | Measuring alertness |
US7791491B2 (en) * | 2005-03-04 | 2010-09-07 | Sleep Diagnostics Pty., Ltd | Measuring alertness |
EP2410501A1 (en) * | 2009-03-19 | 2012-01-25 | Aisin Seiki Kabushiki Kaisha | Drowsiness assessment device and program |
WO2014031042A1 (en) * | 2012-08-20 | 2014-02-27 | Autoliv Development Ab | Eyelid movement processing for detection of drowsiness |
Non-Patent Citations (1)
Title |
---|
See also references of EP3157433A4 * |
Also Published As
Publication number | Publication date |
---|---|
EP3157433A4 (en) | 2018-03-14 |
AU2015278237A1 (en) | 2016-12-08 |
CN106456077A (en) | 2017-02-22 |
US20170119248A1 (en) | 2017-05-04 |
EP3157433B1 (en) | 2022-11-23 |
AU2015278237B2 (en) | 2018-01-18 |
EP3157433A1 (en) | 2017-04-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5078815B2 (en) | Eye opening degree estimation device | |
RU2681362C2 (en) | Method and apparatus for estimating fall risk of user | |
US9875393B2 (en) | Information processing apparatus, information processing method, and program | |
US9799194B2 (en) | Method for detecting drowning and device for detecting drowning | |
US10448829B2 (en) | Biological rhythm disturbance degree calculating device, biological rhythm disturbance degree calculating system, and biological rhythm disturbance degree calculating method | |
AU2015278237B2 (en) | Monitoring drowsiness | |
JP4845886B2 (en) | Method and apparatus for generating personal alert level indications | |
US20150342519A1 (en) | System and method for diagnosing medical condition | |
WO2016168724A1 (en) | System and method for concussion detection and quantification | |
KR101565970B1 (en) | Apparatus and method for determining stroke during the sleep | |
US20220246015A1 (en) | Fall detection method and apparatus, and wearable device | |
CN107123427B (en) | Method and device for determining noise sound quality | |
KR101276973B1 (en) | Pulse frequency measurement method and apparatus | |
US20170221503A1 (en) | Audio processing apparatus and audio processing method | |
CN106326672A (en) | Falling into sleep detecting method and system | |
JP6695817B2 (en) | Biological signal processing device, program and method for making judgment based on the degree of separation from a unit space | |
CN113091257B (en) | Control method, device, equipment and storage medium of air conditioner | |
JP7089650B2 (en) | Processing equipment, systems, processing methods, and programs | |
JP2019017555A (en) | Biological signal analysis device, biological signal analysis method and program | |
JP6936709B2 (en) | Voice detection system and voice detection method | |
JP2018130342A (en) | Wakefulness estimation device, wakefulness estimation method and wakefulness estimation system | |
US20220370009A1 (en) | Stress estimation device, stress estimation method, and recording media | |
US20210345937A1 (en) | Analysis of neurological conditions, including prediction of future seizure events and/or detection of current seizure events, based on analysis of blepharometric data | |
JP6918714B2 (en) | Subject state estimation device and subject state estimation method | |
KR101080761B1 (en) | The judgement device and method for the opening and closing of eyes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15809658 Country of ref document: EP Kind code of ref document: A1 |
|
REEP | Request for entry into the european phase |
Ref document number: 2015809658 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2015809658 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2015278237 Country of ref document: AU Date of ref document: 20150619 Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15318417 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |