KR20170073357A - Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof - Google Patents

Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof Download PDF

Info

Publication number
KR20170073357A
KR20170073357A KR1020150182161A KR20150182161A KR20170073357A KR 20170073357 A KR20170073357 A KR 20170073357A KR 1020150182161 A KR1020150182161 A KR 1020150182161A KR 20150182161 A KR20150182161 A KR 20150182161A KR 20170073357 A KR20170073357 A KR 20170073357A
Authority
KR
South Korea
Prior art keywords
eye
drowsiness
area
unit
detecting
Prior art date
Application number
KR1020150182161A
Other languages
Korean (ko)
Other versions
KR101795188B1 (en
Inventor
허남웅
전슬기
김응환
강명구
Original Assignee
현대자동차주식회사
주식회사 바이브라시스템
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 현대자동차주식회사, 주식회사 바이브라시스템 filed Critical 현대자동차주식회사
Priority to KR1020150182161A priority Critical patent/KR101795188B1/en
Publication of KR20170073357A publication Critical patent/KR20170073357A/en
Application granted granted Critical
Publication of KR101795188B1 publication Critical patent/KR101795188B1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K28/00—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
    • B60K28/02—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G06K9/00604—
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness

Landscapes

  • Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Traffic Control Systems (AREA)
  • Emergency Alarm Devices (AREA)
  • Image Analysis (AREA)

Abstract

An apparatus for detecting a drowsiness in an eye position region vibration detection apparatus according to an embodiment of the present invention includes an eye position region recognition unit for recognizing a region where an eye is located from a face image; And an eye flicker detecting unit for detecting an eye flicker through vibration detection with respect to an area where the eye is located.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for detecting drowsiness in an eye,

The present invention relates to an apparatus and method for detecting drowsiness in an eye position region, and more particularly, to a technique for detecting a vibration in an eye position region to prevent a traffic accident caused by drowsiness.

Automobiles are being used like ordinary necessities of the general public, and the time spent by ordinary people in automobiles in everyday life is increasing. As a result, driver's fatigue, boredom, or drowsiness caused by lack of oxygen in the automobile occurs, and the incidence of accidents due to drowsiness is high. Therefore, various technologies have been developed to prevent drowsiness driving in order to reduce the damage of human life and property.

In order to detect such drowsiness, it has conventionally been determined that the user closed his or her eyes, and the drowsiness was detected by measuring or determining the time of closing the eyes, the speed of closing the eyes, and the extent to which the eyelids cover the eyes.

In this method, the position of the face is first recognized through the face recognition process, and the position of the eye is accurately detected at the detected face region to determine whether the eye is closed by looking at the shape of the eye. At this time, the process of determining the shape of the eye requires a complicated and time-consuming process of applying a pattern recognition algorithm, and it is required to have a performance capable of recognizing various shapes and states of eyes in order to determine an eye shape.

An embodiment of the present invention is to provide an apparatus and method for detecting drowsiness in an eye-position region vibration detection that can detect vibration in a position region of eyes and determine whether the subject is in a drowsy state.

The technical problems of the present invention are not limited to the above-mentioned technical problems, and other technical problems which are not mentioned can be understood by those skilled in the art from the following description.

An apparatus for detecting a drowsiness in an eye position region vibration detection apparatus according to an embodiment of the present invention includes an eye position region recognition unit for recognizing a region where an eye is located from a face image; And an eye flicker detecting unit for detecting an eye flicker through vibration detection with respect to an area where the eye is located.

In addition, the present invention may further include a face image acquisition unit for acquiring a face image.

In addition, the eye position area recognizing unit may include a half of the actual eyes or a third of the eyes.

The eye flicker detecting unit may calculate an amplitude magnitude by calculating a difference between pixel values of the frame with respect to an area where the eye is located and calculate an average value of the magnitude magnitude and generate a vibration waveform by graphically representing an average value per frame have.

And a drowsiness determination unit for determining the drowsiness component of the user using the output of the eye flicker detection unit.

The drowsiness determination unit may determine drowsiness using the magnitude of vibration amplitude of the eye blinker and the time without eye blinking.

The drowsiness determination unit may determine that there is no flickering if the amplitude magnitude of the eye blink signal is greater than a predetermined magnitude.

Wherein the drowsiness determination unit determines that the eye blinking time is a warning state when the first reference time has elapsed and determines that the eye blinking time is a dangerous state when the second reference time has elapsed longer than the first reference time can do.

The method of detecting drowsiness by sensing the eye position region vibration according to an embodiment of the present invention

Recognizing an area where the eyes are located from the face image; And detecting eye flicker through vibration detection for the region in which the eye is located.

The step of recognizing the area where the eyes are located may recognize an area including half of the actual eyes or more than 1/3 of the eyes as the eye position area.

The step of recognizing the area where the eye is located may include calculating an amplitude magnitude by calculating a difference between pixel values of the frame with respect to an area in which the eye is located, calculating an average value of the magnitude magnitude, A vibration waveform can be generated.

This technique can minimize the eye detection time by recognizing the eye area through face recognition without detecting the position of the eye accurately and detecting blinking in the eye area through vibration detection.

FIG. 1 is a block diagram of a drowsiness detecting apparatus according to an embodiment of the present invention.
FIG. 2 is a view illustrating a face recognition area and an eye area by the drowsiness detection device according to an embodiment of the present invention. FIG.
FIG. 3 is a flowchart illustrating a method of detecting drowsiness by sensing eye position region vibration according to an embodiment of the present invention.
4 is a graph illustrating amplitude magnitudes calculated using difference between frames for points in an eye position area according to an exemplary embodiment of the present invention.
5 is a graph for explaining drowsiness detection according to an embodiment of the present invention.
FIG. 6 is a block diagram of a computer system to which the drowsiness detection technique based on the detection of eye-position region vibration is applied according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. Also, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 6. FIG.

FIG. 1 is a block diagram of a drowsiness detecting apparatus according to an embodiment of the present invention.

The apparatus for detecting a drowsiness in the eye position region vibration detection according to an embodiment of the present invention includes a face image obtaining unit 110, an eye position region recognizing unit 120, a blink detection unit 130, a filtering unit 140, (150).

The facial image acquiring unit 110 acquires a face image of a user and may include a camera or the like.

The eye position region recognizing unit 120 recognizes an eye position region from a face image. As shown in FIG. 2, the face is recognized in the face image, and the eye position area 20 is recognized in the recognized face recognition area 10. At this time, the eye position region 20, which is considered to have eyes, does not actually have the eye at the correct position, and can detect the flicker even if it is only about half or one third.

The eye blink detector 130 calculates an amplitude by calculating a pixel difference between frames of pixels in the eye position area 20 and adds the amplitude magnitudes of the respective pixels to obtain a total amplitude magnitude value . The eye blink detector 130 calculates an average value by dividing the total amplitude magnitude value by the number of pixels in the eye position area 20, and the calculation formula of the average value X (t) is as shown in the following equation (1).

Figure pat00001

Here, the average value X (t) becomes the amplitude magnitude value, and the vibration waveform represented by the time axis is the Blink signal using the average value in all the frames. N is the number of pixels and pixel difference is the pixel value difference.

Then, the blink detection unit 130 generates a vibration waveform as shown in FIG. 4 by displaying the calculated average value in a graph. 4 is a graph illustrating amplitude magnitudes calculated using difference between frames for points in an eye position area according to an exemplary embodiment of the present invention.

The filtering unit 140 filters the average value output from the blink detection unit 130, that is, the blink signal. At this time, the filtering unit 140 may include a low pass filter (LPF) using an FIR filter.

The sleep determination unit 150 determines whether the user is drowsy using the output of the blink detection unit 130. [ That is, the drowsiness determination unit 150 determines the drowsiness using the magnitude of the vibration amplitude of the eye blink and the time without blinking. The drowsiness determination unit 150 detects the amplitude magnitude of a signal generated every time the eye blinks, and regards it as having no flickering if it is less than a certain size (0.09, for example).

The drowsiness determination unit 150 detects an amplitude magnitude from the filtered blink signal and counts a non-flickering time with a timer to notify a warning when a non-flickering time elapses from the first reference time, When the second reference time longer than the reference time elapses, the risk is notified.

As described above, according to the present invention, when an eye flicker is detected, an area in which an eye is located is designated instead of an eye pattern recognition, and a flicker of the eye is detected by sensing a vibration at the corresponding position. It is possible to minimize the amount of computation and the time required to find the position of the memory.

Hereinafter, a method of detecting drowsiness by sensing the eye-position region vibration according to an embodiment of the present invention will be described in detail with reference to FIG.

The eye position region recognition unit 120 recognizes an eye position region in the face image acquired by the face image acquisition unit 110 (S101).

The blink detection unit 130 calculates an amplitude of a pixel value between frames for the eye position area (S102), adds the amplitude magnitudes based on the difference between the pixel values of the frames, And calculates an average value (S103).

The blink detection unit 130 generates a vibration waveform by graphically representing an average value per frame (S104).

After performing the above steps S103 to S104 for both eyes, the average value of both eyes is calculated (S105). At this time, the average value of both eyes becomes the final blink signal.

The filtering unit 140 filters the blink signal (S106).

The drowsiness determination unit 150 detects the amplitude magnitude from the filtered blink signal and determines whether the blinking of the eye is determined according to the amplitude magnitude (S107). The drowsiness determination unit 150 alerts the user when the predetermined non-flickering time has elapsed (S108).

The drowsiness judging unit 150 notifies the user of a danger when the non-flicker time elapses from a predetermined second reference time (S109).

5 is a graph for explaining drowsiness detection according to an embodiment of the present invention.

Referring to FIG. 5, a warning (A) notification is performed when the magnitude of the signal detected from the blink signal is 0.9 or more, and a risk (B) notification is performed when the amplitude of the signal is two or more.

As described above, the present invention can comprehensively designate an area where eyes are located through face recognition, detect blinking of eyes with the vibration detection technique, and detect drowsiness by flickering.

Accordingly, the present invention does not require complicated and various pattern recognition processes for eye shapes, and can detect flicker without comprehensively recognizing the positions of the eyes and precisely finding the positions of the eyes. Therefore, There is almost no probability of misrecognizing the eye position.

The embodiment of the present invention can be applied to mount an algorithm on an embedded system operated by a small microprocessor, mount the algorithm on the vehicle, and photograph an actual driver to make a drowsiness. In addition, it is possible to detect drowsiness by sensing blinking of eyes by simulating a computer that shoots a driving scene and creates a moving image and stores it in a storage device.

FIG. 6 is a block diagram of a computer system to which the drowsiness detection technique based on the detection of eye-position region vibration is applied according to an embodiment of the present invention.

6, a computing system 1000 includes at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, (1600), and a network interface (1700).

The processor 1100 may be a central processing unit (CPU) or a memory device 1300 and / or a semiconductor device that performs processing for instructions stored in the storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) and a RAM (Random Access Memory).

Thus, the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by processor 1100, or in a combination of the two. The software module may reside in a storage medium (i.e., memory 1300 and / or storage 1600) such as a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, You may.

An exemplary storage medium is coupled to the processor 1100, which can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor 1100. [ The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Drowsiness detection device
110: facial image acquiring unit
120: eye position area recognition unit
130: eye blink detector
140:
150: drowsiness judgment unit

Claims (11)

An eye position area recognition unit for recognizing an area where the eyes are located from the face image; And
An eye flicker detecting unit for detecting an eye flicker through vibration detection for an area where the eye is located;
Wherein the drowsiness detection device comprises: The method according to claim 1,
A facial image obtaining unit
Wherein the drowsiness detection unit is configured to detect the drowsiness of the eye region by vibration detection. The method according to claim 1,
The eye-position-area recognizing unit recognizes,
Wherein the region in which the eye is located comprises half of the actual eye or a third point of the eye. The method according to claim 1,
Wherein the eye blink detection unit comprises:
Calculating an amplitude magnitude by calculating a difference between pixel values of the frame with respect to an area in which the eye is located, calculating an average value of the magnitude magnitudes, and generating an oscillation waveform by graphically representing an average value per frame. Drowsiness detection device by vibration detection. The method according to claim 1,
A drowsiness judgment unit for determining the drowsiness component of the user by using the output of the eye blink detection unit,
Wherein the drowsiness detection unit is configured to detect the drowsiness of the eye region by vibration detection. The method of claim 5,
Wherein the drowsiness judging unit judges,
Wherein the drowsiness is determined by using a magnitude of vibration amplitude of the eye flicker and a time without eye flicker. The method of claim 6,
Wherein the drowsiness judging unit judges,
And determines that there is no flicker if the magnitude of the blink signal is equal to or greater than a predetermined magnitude. The method of claim 6,
Wherein the drowsiness judging unit judges,
Wherein the control unit determines that the blinking-free time is a warning state when a first predetermined reference time elapses, and determines that the blinking-free time is a dangerous state when a second reference time elapses after the first blinking time is longer than the first reference time Apparatus for detecting drowsiness by detecting vibration in the eye position region. Recognizing an area where the eyes are located from the face image; And
Detecting eye flicker through vibration detection for an area in which the eye is located;
And detecting the drowsiness of the eye region by vibration detection. The method of claim 9,
Wherein the step of recognizing the area in which the eye is located comprises:
Wherein an area including half of the actual eye or at least one third of the eye is recognized as the eye position area. The method of claim 9,
Wherein the step of recognizing the area in which the eye is located comprises:
Calculating an amplitude magnitude by calculating a difference between pixel values of the frame with respect to an area in which the eye is located, calculating an average value of the magnitude magnitudes, and generating an oscillation waveform by graphically representing an average value per frame. Detection of drowsiness by vibration detection.
KR1020150182161A 2015-12-18 2015-12-18 Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof KR101795188B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150182161A KR101795188B1 (en) 2015-12-18 2015-12-18 Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150182161A KR101795188B1 (en) 2015-12-18 2015-12-18 Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof

Publications (2)

Publication Number Publication Date
KR20170073357A true KR20170073357A (en) 2017-06-28
KR101795188B1 KR101795188B1 (en) 2017-11-07

Family

ID=59280797

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150182161A KR101795188B1 (en) 2015-12-18 2015-12-18 Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof

Country Status (1)

Country Link
KR (1) KR101795188B1 (en)

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008171108A (en) * 2007-01-10 2008-07-24 Matsushita Electric Ind Co Ltd Face condition determining device and imaging device

Also Published As

Publication number Publication date
KR101795188B1 (en) 2017-11-07

Similar Documents

Publication Publication Date Title
US20200334477A1 (en) State estimation apparatus, state estimation method, and state estimation program
US7835834B2 (en) Method of mitigating driver distraction
JP5207249B2 (en) Driver condition monitoring system
US9526448B2 (en) State estimation device and state estimation program
JP4899059B2 (en) Sleepiness detection device
US20100322507A1 (en) System and method for detecting drowsy facial expressions of vehicle drivers under changing illumination conditions
CN107209979A (en) The method and apparatus that microsleep for the driver to vehicle is identified
WO2017067399A1 (en) Method and device for early warning based on image identification
JP2015529512A (en) Processing related to eyelid movement to detect drowsiness
JP6399311B2 (en) Dozing detection device
JP6754228B2 (en) Methods and devices for supplying reference levels for eye opening
Nakamura et al. Detection of driver's drowsy facial expression
Hanafi et al. A real time deep learning based driver monitoring system
JP5139470B2 (en) Sleepiness level estimation device and sleepiness level estimation method
KR101795188B1 (en) Apparatus for detecting drowsy using vibration sensing in the eye region and Method thereof
KR20220048533A (en) Apparatus for detecting attention level of driver and method thereof
EP3000395A1 (en) System and method for monitoring a subject's eye
JP5050794B2 (en) Sleepiness detection device, sleepiness detection method
JP6281238B2 (en) Recognition degree estimation device and recognition degree estimation method
JP4973613B2 (en) Driver status detection device
JP5411653B2 (en) Sleepiness determination device
Parsai et al. Intelligent Monitoring System for Driver’s Alertness (A vision based approach)
JP2018130342A (en) Wakefulness estimation device, wakefulness estimation method and wakefulness estimation system
JP7019394B2 (en) Visual target detection device, visual target detection method, and program
JP2017076180A (en) State determination device

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
AMND Amendment
E601 Decision to refuse application
AMND Amendment
E90F Notification of reason for final refusal
AMND Amendment
X701 Decision to grant (after re-examination)
GRNT Written decision to grant