KR20180131045A - Drowsiness preventing apparatus integrally formed by multi-sensored detectionalarm means and central processing means while driving a car and control methed therefor - Google Patents

Abstract

The present invention relates to a drowsy driving preventing apparatus with an integrated multisensor-based detection and alarm means and central processing means, and a control method therefor, wherein a detection means detecting an electroencephalogram, an electromyogram of an eyeball region, an eye blinking image and a head slope to detect drowsiness by using a plurality of sensors, and an alarm means for warning drowsiness are integrated with each other to be worn on user′s face, thereby allowing drowsiness and alert signals to be directly sensed through the head and to be delivered immediately, and giving warning step by step from the initial state of drowsiness.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a drowsiness driving prevention device and a control method of a drowsiness driving prevention device and a control method in which a multi-sensor based sensing and alarming means and a central processing means are integrally formed. }

The present invention relates to a drowsiness driving prevention device and a control method in which a multi-sensor-based sensing and alarming means and a central processing means are integrated. More particularly, the present invention relates to a drowsiness prevention device and a control method, A drowsiness detecting means for detecting a tilt of the image and head and an alerting means for warning of drowsiness are integrally provided to be worn on the face so that drowsiness and an alarm are made through the face and detected from the early stage of drowsiness and warned stepwise The present invention relates to a drowsiness driving prevention device and a control method in which a multi-sensor-based sensing / alarming means and a central processing means are integrated.

Today, as the number of drivers suddenly increases, the number of driving accidents with drinking, speeding, and sleepiness continues to increase, which increases the damage to oneself and other people. Especially, in case of drowsy driving, the most serious thing is that it is difficult for the driver himself to be aware of the moments when he / she falls asleep momentarily while driving, and the accident rate due to the accident is also reported to be two times higher than general accident. Drinking and speeding are restricted by law, but in the case of drowsy driving, it can be prevented only by the driver himself or herself.

Therefore, many studies and techniques have been reported to prevent drowsiness driving at home and abroad. The classification of each type includes a method using human's biological signals such as a pulse, an electro-encephalogram (EEG), an electro-oculogram (EOG) or an electromyogram (EMG) The size of the pupil, eye flicker, or the inclination of the head.

In the case of electrons that use bio-signals, EEG signals were first invented by Hans Berger, a German medical scientist in 1920, and many studies have shown that EEG on sleep, And EEG, EOG, and EMG signals are widely used as highly accurate factors for judging drowsiness [Non-Patent Documents 1, 2, and 4]. For example, in Korean Patent No. 10-0958166, the sleeping interval is used to detect a drowsiness prediction point and the drowsiness is judged by the ratio of alpha wave and theta wave [see Patent Document 1 ].

However, in the case of using the EEG as described above, it is basically necessary to attach a plurality of EEG electrodes to the upper and lower parts of the head and eyelids, and it is difficult to accurately detect the initial state of the drowsiness and the state of entering the drowsy by only one EEG element. There are many practical and economical limitations to apply to vehicles.

In the latter case using a behavior signal, it is determined whether the pupil or the eyelid is open or closed in the eye region of the face by using a camera such as a camera, and the sleep state is directly determined by the eye image. It is known to be useful in practical terms. For example, M. Suzuki et al. Determined the drowsiness by the degree of eyelid opening and blinking time using the image of the face [Non-Patent Literature 3], Korean Patent 10-1669447 detected the face and eye region, To detect drowsiness [Patent Document 2]. In addition, Korean Patent No. 10-1407910 proposes a device for preventing drowsiness which gives a strong electrical stimulus to the body by detecting a head tilting by drowsiness using a tilt sensor [Patent Document 3].

However, in the case of using the image means as described above, the face is necessarily photographed from the front face, and a face is recognized by using a known recognition technique such as Haar-like Features, Adaboost, Projection Histogram, In order to keep track of the eye area with the camera, there are many obstacles to accurate image detection due to errors caused by driver's posture, movement, and vehicle vibration. There was a factor. In addition, the method of detecting drowsiness by head inclination has a limitation in preventing drowsiness operation because it detects a trailing signal appearing in a state in which the eyes are closed and the cognitive ability is lacked if the head is tilted. Drowsy driving is an instantaneous phenomenon. To stop a car driving at 100km / h, about 130m is required for the combined reaction distance and braking distance. When it blinks for 3 ~ 4 seconds, it is more than 100m on the general road, , It is very important to detect drowsy driving in advance because it travels about 1.5 to 3 times this distance.

As described above, conventional methods using bio-signals such as brain waves, and flicker and head tilt signals provide an academic basis for detecting drowsiness in each region. However, as mentioned above, It is difficult to distinguish whether the blinking is due to drowsiness or habit or other factors, so it is difficult to accurately detect drowsiness with only one or two detection elements. There were many limitations to doing.

[1] Korean Patent No. 10-0958166 (entitled " Drowsiness Detection Device Method and Method). [2] Korean Patent No. 10-1669447 (entitled: Image-Based Drowsy Recognition System and Recognition Method). [3] Korean Patent No. 10-1407910 (entitled "Ear-Resisting Sleepiness Prevention Device Using Acceleration Sensing and Electrical Stimulation").

[1] V. Valley and R. Broughton, "The physiological (EEG) nature of drowsiness and its relation to performance deficits in narcoleptics," Electroencephalogr. Clin. Neurophysiol., Vol. 55, pp. 243-251, 1983. [2] Kaneko, K., & Sakamoto, K. (1999). "Evaluation of three types of blinks with the use of electroculogram and electromyogram, Perceptual and Motor Skills," 88 (3), 1037-1052. [3] M. Suzuki, N. Yamamoto, O. Yamamoto, T. Nakano and S. Yamamoto, "Measurement of driver's consciousness by image processing - a method for presuming driver's drowsiness by eye-blinks coping with individual differences" Conference on Systems, Man and Cybernetics, vol. 10, pp. 2891-296, 2006. [4] Nolan, H .: Whelan, R .; Reilly, R.B. (2010), " FFASER: Fully Automated Statistical Thresholding for EEG artifact Rejection ", Journal of Neuroscience Wethods 192 (1): 152-62. [5] R. Vigario. "Extraction of ocular artifacts from EEG using Independent Component Analysis (ICA)," E lectroencep h. clin. N. et al., Proc. Soc., 103 (3), pp. 395-404, 1997. [6] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol.12, pp. I-511 to I-518, 2001 . [7] http://opencv.willowgarage.com/wiki/, Jan. 2011. [8] F. Wang, M. Zhou and B. Zhu, "A Novel Feature Based Rapid Eye State Detection Method," Proceedings of the IEEE International Conference on Robotics and Biomimetics, Guilin, China, pp. . 2009. [9] D. Dinges, "PERCOS: a valid psychophysio-logical measure of alertness as assessed by psychomotor igilance," Federal Highway Administration, Office of Motor Carriers, Technical Report, MCRT-98-006, 1998. [10] W. Zhao and R. Chellappa, "Discriminant analysis of principal for face recognition," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 366-341, 1998. [11] Peter N. Belhumeur, Joao P. Hespanha and David J. Krigman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection," Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.336-341, 1998 .

The present invention relates to a drowsiness prevention device and a control method thereof, and more particularly to a drowsiness detection device composed of an EEG and EOG detection brain wave detector, an eye area EMG detector, an eye opening / And a warning means for warning the drowsy state can be directly worn on the face so that both the drowsiness detection and the alarm transmission are made directly through the head and the warning is issued step by step from the initial state of drowsiness. A drowsiness driving prevention device and a control method in which an alarm means and a central processing means are integrally formed.

According to the drowsiness driving prevention device having the multi-sensor-based drowsiness detection / alarming means and the central processing means integrally constructed according to the present invention, the EEG, EOG, EMG, Detecting means for detecting a signal of the head slope directly through the eyes and the face region and preprocessing the signal; The sensing means is integrally formed with the sensing means, converts each sensed signal detected by the sensing means into a digital signal, and arithmetically averages the sensed signals per unit time in a time series sequential manner to compare the average value for comparison with the reference value for determining the drowsiness, Central processing means for determining an awakening state (Da), an initial state of drowsiness (Db), a drowsy state (Dc) and a drowsy state (Dd) and generating a selective control signal; An alarm means which is integrally formed with the sensing means and generates an alarm sound whose intensity is gradually increased in accordance with a selective control signal according to each determination result from the central processing means; And vehicle control means for receiving an optional control signal according to a determination result from the central processing means and controlling the blinking signal, the internal ventilation, and the deceleration of the vehicle step by step, so that drowsiness detection and warning are directly performed through the face Provided is a drowsiness driving prevention device for detecting a warning from the initial state of drowsiness.

According to an embodiment of the present invention, the sensing means, the central processing means, and the alarm means are constituted by a pair of structural members each having grooves formed in the longitudinal direction thereof, and a flexible connecting member integrally connecting these members, A drowsiness driving preventing device in which a groove is formed on the face by a leg of a pair of glasses.

According to the control method of the drowsiness driving prevention device having the multi-sensor-based drowsiness detection / alarm means and the central processing means integrally constructed according to the present invention, the EEG, EOG, EMG, eye image and head tilt signal (Vef, Vof, Vmf) corresponding to the respective signals for the awakening state Da, the drowsy initial state Db, the drowsy state Dc and the drowsy state Dd according to the drowsy state, , Vif, and Vwf); The central processing means may perform optimization analysis using the LDA to obtain the specific data values for drowsiness comparison, and determine each of the specific data values for the corresponding drowsiness determination It is determined whether the comparison result of the EEG, EOG, and EMG signals is within the reference values (Vef, Vof, and Vmf) of the drowsy initial state Db by comparing the reference values (Vef, Vof, Vmf, and Vif) Determining whether the comparison result is within the reference value Vif of the drowsy initial state Db, finally determining the drowsy initial state Db and ventilating the vehicle with the primary alarm by the alarm means; If the drowsiness initial state remains unchanged in step S2, the EOG signal is compared again to determine whether it is within the reference value Vof of the drowsy entry state Dc, and then compared with the image signal again to determine the reference value of the drowsy entry state Dc (S3) of judging whether the dormant entry state (Dc) is finalized and generating a warning signal to the following vehicle together with the secondary alarm, if it is in the vehicle (Vif); If the drowsiness state is still maintained in step S3, the image signal is compared with the image signal again to determine whether the dormancy state is within the reference value Vif of the drowsy state (Dd). Then, Determining whether the specific data value is within the drowsiness determination slope reference value (Vwf), finally determining the drowsy state (Dd), and decelerating the vehicle with a third level alarm at a high level, The present invention provides a control method of a drowsiness driving preventing device so that drowsiness detection and warning are performed step by step.

The present invention is miniaturized so that the drowsiness signal can be directly detected and alerted through the eyes and the face by making the drowsiness detection means and the alarm means integrally constitute and can be worn on the face. Especially, a plurality of living bodies and behavior signals And the drowsiness judgment rate was maximized by detecting stepwise from the initial state of drowsiness.

In particular, it is possible to directly detect the eye image from the eye area without directly extracting it from the eye area by taking the whole face as in the conventional method, thereby minimizing the measurement error with the continuous tracking and thus solving the conventional problems, Can be provided.

The EEG detection of the present invention is intended to detect only the drowsiness component rather than the elaborate measurement for medical purposes, and it is possible to measure with a comparatively simple electrode and a known analysis algorithm. Recently, a combination of a carbon nanomaterial and a bio- A highly sensitive electrode has been developed to measure ultrasmall EEG more than several times more than before, so that a more reliable drowsiness prevention device can be put to practical use.

FIG. 1 is an exploded perspective view for explaining a drowsiness driving prevention device in which a multi-sensor-based sensing means and an alarm means are integrally formed according to an embodiment of the present invention.
2 is a view for explaining the wearing state of the apparatus of the present invention and the sensing means
FIG. 3 is a diagram for explaining a characteristic of the EEG according to the frequency range of the EEG and a waveform using the power spectrum analysis method.
4 is a view for explaining EEG, EOG, EMG, and eye blink signals by the multi-sensor-based sensing means of the present invention.
5 is a graph showing the number of blinks of the eye proceeding from the awake state to the drowsy state.
6 is a diagram showing a gray level image, a binary image and a vertical projection histogram of one side of the eye region according to the present invention.
FIG. 7 is a block diagram for an overall configuration of the drowsiness driving prevention device of the present invention.
8 is a detailed block diagram of the sensing means and the alarm means according to the present invention.
9 is a detailed block diagram of the central processing means according to the present invention.
10 is a detailed block diagram of the vehicle control means according to the present invention.
Fig. 11 is a flowchart showing an embodiment of a control method for the drowsiness driving prevention apparatus of the present invention shown in Fig. 7; Fig.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is an exploded perspective view of a drowsiness driving prevention apparatus 100 according to an embodiment of the present invention. The sensing means and the warning means constructed of multiple sensors are constituted by a pair of members 100a and 100b as shown in FIG. Which are integrally connected by the connecting member 100c, and although not shown, the central processing means are also integrally formed in the member, which will be described in detail later.

First, a pair of EEG electrodes 1a1 and 1a1 for contact with the temporal part (P1 in FIG. 2B), which is the temple part of the user, for detecting the EEG and EOG signals are provided on both of the pair of the constituent members 100a and 100b. , An EMG electrode (2a) which is in close contact with a position of an eyeball around the eyeball (Fig. 2B, P2) around the eyeball responsible for the blinking muscle to detect a blinking muscle signal, A camera 3 for detecting an eye blinking image and a gyro sensor 4 for detecting the inclination of the head in the x, y and z directions (see Fig. 2A) ). The inner end of the component member 100a on which the EMG electrode 2a is provided is adjacent to the ear and has a relatively small field effect, so that the EMG ground electrode 2b is provided. Next, as shown in the figure, a reference electrode 7a and a ground electrode 7b for the EEG are detachably attached to the user's earlobe by a wire jack, as shown in the figure. Finally, a speaker 5 for auditory and tactile alert and a vibration motor 6 are incorporated in the constituent member as an alarm means.

A pair of the constituent members 100a and 100b constituted by the sensing means and the alarm means as described above are integrally connected by the flexible connecting member 100c and are provided with grooves 8a and 8b formed in the longitudinal direction inside the both constituent members And can be fastened to the eyeglasses legs 9a and 9b. Here, the connecting member 100c connecting the pair of constituent members is made of a stretchable material whose width can be adjusted because the width of glasses is different for each person, and a data line for electrically connecting the respective units to each other is built in It is in the form of a pupil. Further, the connecting member 100c further includes a pair of EEG electrodes 1c and 1d configured to be in close contact with the front head (P3 and P4 in FIG. 2B). These electrodes also detect the EEG signal, . In general, in the temporal region, signals of major EEG bands are detected to determine drowsiness such as beta waves, alpha waves, and cetaphars. The frontal part is located where the EOG signal is best detected when the element eye blinks. Is known to be preferable.

As described above, the EEG detection according to the present invention is composed of four channels of EEG electrodes 1a to 1d, two at the temporal region and at the frontal region. In addition, the electroencephalogram (EEG) and electroencephalogram (EMG) detection electrodes were prepared by using a dry electrode made of silver (Ag) or silver chloride (AgCL), and the electroencephalogram electrode was attached by a unipolar induction method Respectively. The camera 3 is an infrared ray (IR) camera for detecting a monochrome image, which is processed to block interference of visible light such as sunlight or illumination inside or outside the vehicle, which obstructs image detection of the eyes. Of course, the inclination sensor of the head can adopt a gyro sensor or other acceleration sensor. The structure of the drowsiness driving prevention device 100 of the present invention shown in FIG. 1 is not limited to the following embodiments, but may be constructed to be wearable to the glasses as in the present embodiment, Or may be integrally manufactured.

FIG. 2A is a perspective view of the wearer of the drowsiness driving prevention device 100 of FIG. 1, showing that the gyro sensor 4 detects an inclination angle of the head in the x, y, 2b show the positions of the temporal bulb P1 to which the EEG electrodes 1a and 1b are closely attached, the position of the OCS P2 to which the EMG electrode 2a for detecting the blinking muscle signal adheres and the positions of the EEG signal detecting brain electrodes 1c, 1d are in close contact with each other.

FIG. 3A shows the magnitude, characteristic, and waveform of electrical components of EEG using Power Spectrum Analysis (PSA). PSA is an FFT (Fast Fourier Transform) of autocorrelation function, Log scale power. Generally, brain waves are divided into a delta (delta) wave, a theta wave, an alpha wave, a beta wave and a beta wave in accordance with a frequency band oscillating at a frequency of 0.5 to 50 Hz and an amplitude of about 5 to 200 [ β) wave, and gamma (γ) wave component.

In FIG. 3A, only the EEG components of the four frequency bands related to drowsiness are summarized. In general, when the user is awake and drowsy, the activity of the brain gradually decreases, the brain wave frequency gradually decreases, and the amplitude gradually increases. In the EEG, the beta wave (13 ~ 30Hz, 5 ~ 20μV) appears mainly in the frontal part when tensed or concentrated, and the alpha wave (0 ~ 12Hz, 20 ~ 60μV) is emotionally calm, Is relaxed and appears mainly in the early stage of drowsiness. It is also detected in the frontal lobe, which is loosely relaxed in the open eye, and is likely to lead to drowsiness in the open eye. It is known that EEG waves (0.5 Hz to 4 Hz, 20 to 100 μV, 4 to 8 Hz, 20 to 100 μV) mainly appear in the temporal region of the sleeping surface and pass through when the attention and concentration converge suddenly. ) Appear to be submerged in the water and appear predominant in the frontal region [Non-Patent Documents 1 and 2].

As can be seen from the above, the alpha wave, theta wave and delta wave components are most closely related to drowsiness in the brain waves. Referring to FIGS. 3A and 3B, in general, the brain waves in the awake state stay in the beta wave period, In the present invention, the characteristic of the EEG is used as a factor for detecting drowsiness. In the present invention, the sleepiness of the user is detected.

Referring to FIG. 4, the top EEG EEG signals are classified into the alpha wave (FIG. 4B), theta wave (FIG. 4C) and the delta wave (FIG. 4D) The EOG waveform is a waveform that shows the waveform of each general frequency band. Below the EOG waveform is an EOG waveform included in the EEG signal when the eye blinks according to each waveform section. The bottom shows a raw state Are shown in order.

FIGS. 5A to 5D are graphs showing Eye Blink Rate (EBR) per unit time for each section during a state of falling into drowsiness in a wakefulness state as illustrated in FIG. Referring to FIGS. 5A to 5D and EOG and EMG in FIG. 4, the biopsy of blinking according to the degree of drowsiness is described. In the awakening state of FIG. 4A, the average number of EBRs per unit time is shown. As can be seen from the EEG and EMG signal waveforms in FIG. 4B, the EBR is rapidly increased due to intentional blinking of the eye several times in order to generally cause drowsiness. In the section 4c in which the user enters the drowsy state, the EBR gradually decreases, The EBR in the section of FIG. 4D is almost reduced. [Patent Document 1] FIG.

6A to 6D are gray level images (GLI: Gray Level Images) taken at the side of the eye region, and show a state in which the eyes are fully opened, which can occur in each of the sections of the brain waves shown in FIG. 4, State, a state in which a person is almost in a state of falling into drowsiness, and a state of falling into drowsiness. The Blinking Duration Rate (BDR) of the eye per unit time is inversely related to the number of eye blink EBR . Below this, the binary image (BI) of the eye region corresponding to each state and the accumulated value ( IPF ) of the pixels of the vertical projection histogram are shown in order.

In the present invention, drowsiness is detected by using various drowsiness factors and features described in FIGS. 4 to 6, and the construction and operation of the present invention will be described in more detail.

7 is a block diagram for explaining the overall configuration of the drowsiness prevention device 100 of the present invention. The dormancy prevention device 100 includes a detection and alarm means 10 for detecting and alarming drowsiness and a central processing means (20), and vehicle control means (30).

First, the sensing and alarming means 10 is composed of a sensing means 11 and an alarm means 14. The sensing means 11 senses an EEG signal, an electromyogram signal of an eye region, A drowsiness detecting device 12 for detecting a signal and a head inclination signal, and a signal preprocessing device 13 for preprocessing the detected signals according to respective signal processing standards, Respectively. And a power source 15 for supplying the above means.

The central processing unit 20 includes a digital signal processing unit 21 for converting a signal of each of the detectors detected by the detecting unit 11 into a binary coded decimal (BCD) code and processing the signal in a time series manner, A memory unit 22 for storing a BCD code-processed time-series sequential average value for each sensing signal, a reference value for drowsiness determination for each sensing signal, and an analysis algorithm for determining drowsiness, a time series sequential average value (23) for judging drowsiness by comparing the reference value stored in the memory with the reference value stored in the memory unit by the analysis algorithm, and an alarming means (14) and a vehicle control means (30) And a control signal generator 24 for outputting control signals selectively.

Finally, the vehicle control unit 30 includes a vehicle control unit 31 that receives an optional control signal from the control signal generation unit 24 in accordance with the drowsiness judgment of the operation control unit 23, And a vehicle alarm device 32 for ventilating the interior of the vehicle by the internal combustion engine, generating a warning signal to the following vehicle, and gradually decelerating the vehicle speed.

FIG. 8 is a detailed block diagram of the sensing means and the alarm means according to the present invention, and the configuration and operation of each means of the present invention described in FIG. 7 will be described in detail.

The drowsiness detection device 12 of the sensing means 11 is constituted by an electroencephalogram sensor 12a for detecting EEG and EOG signals by four electroencephalogram electrodes 1a to 1d described in FIG. And an inclination sensor 12d for detecting the inclination of the head, and the signal preprocessing device 13 is constituted by an EMG sensor 12b for detecting a signal, an eye image sensor 12c for photographing an eye region by the camera 3, An EEG component processing unit 13a, an EMG signal processing unit 13b, an eye image processing unit 13c, and a tilt signal processing unit 13d, respectively.

In the EEG component processing unit 13a, since the original raw EEG signal detected by the four EEG electrodes is very low at a frequency of 0.5 to 50 Hz and an amplitude of about 5 to 200 μV, the original EEG signal is firstly amplified by differential input pre- And the signal is amplified to a degree not to saturate at least 1,000 times or more, and is filtered by each frequency band to remove unnecessary noise and then A / D converted. The EEG signal received by the EEG electrodes 1c and 1d on the front of the 4-channel electrodes includes an EOG signal (see FIG. 4) which is larger in amplitude than the EEG signal and appears when the eye flickers. Therefore, the EEG component processing unit 13a is an element for judging drowsiness. In order to utilize the EOG signal, the EEG signal received from the 4-channel electrode is separated into an EOG signal and an EOG signal by an Independent Component Analysis (ICA) Of pure EEGs, respectively. For reference, EEG detection techniques using a fully automated ICA that remixes only a desired signal by eliminating the noise of the EEG signal and preventing mixing of some EEG signals have been reported [Non-Patent Documents 4 and 5 ].

The ICA is a technique for independently separating original signals and specific signals from signals having various signals in the fields of neural network, bio-signal measurement, statistical data analysis, feature extraction, etc. To simplify the theory, The signal detected at the EEG electrode of the invention can be regarded as a linearly calculated signal of several independent signals, and expressed mathematically as Y = ZX . Here, Y ( Y1, Y2 ... Yn ) is the EEG signal measured at the electrode, Z is the mixing matrix and X ( X1, X2 ... Xn ) are independent components extracted from Y using ICA .

U (U1, U2 ... Un) signal becomes U = WY when a W matrix satisfying WZ = I is obtained. The original signal can be estimated as X , and the original signal can be estimated from the EEG signal measured at each electrode (Equation 3), such that U = WY = WZX = X The X signal can be obtained.

On the other hand, the joint entropy for the two random variables X and Y is obtained by the following equation (1).

If X and Y are independent in Equation (1), H ( X, Y ) = H ( X ) + H ( Y ). The conditional entropy of X and Y when Y has a particular value b _k is defined as the entropy of the Conditional Probability Distribution,

. In Equation (2), when the average of Y is a value b _k , the following Equation 3 is transformed.

And the mutual information of X and Y (Mutual Information) is I (X; Y) = H (X) + H (Y) - H (X, Y) that are, in other words, X and Y each limit entropy (Marginal Entropy of ) Is defined as the sum of H ( X ) + H ( Y ) minus the joint entropy H ( X, Y ). When the joint entropy of X and Y is maximized, the amount of mutual information between X and Y is minimized. Therefore, when the portion including X and Y is minimized, the mutual information amount is minimum, and X and Y can be defined as independent signals. Therefore, if the original EEG signal Y is actually independent, the estimated EOG signal X is also accurate Signal can be detected.

As described above, since EEG and EOG signals can be separated from each other by the ICA using EEG signals detected from the four channel electrodes of the present invention, EOG can be detected by a separate channel And the EEG component processing unit 13a can be configured as a dedicated module equipped with an ICA algorithm.

Next, the EMG signal processing unit 13b lowers the EMG signal of the original state sensed by the EMG electrode 2a to a frequency of about 10 to 500 Hz and an amplitude of several mV to several tens of mV, V to several tens V, so that the signal is amplified at a frequency of at least 500 to 1,000 times with a preamplifier in the same manner as the EEG, and the signal is filtered with a frequency band filter of about 10 to 500 Hz to remove unnecessary noise and then A / D converted. The EMG signal processing unit 13b may also be configured as a dedicated module for EMG signal detection.

On the other hand, the eye image processing unit 13c generates a gray level image in which the eye region is designated as black and white contrast instead of RGB by the camera 3, converts the image into a binary image using a projection histogram, As shown in Fig. 6, when the eye opening rate is completely opened (70% or more, Fig. 6A), half-rolled (less than 70% (FIG. 6C), and a completely rolled state (20% only, FIG. 6D).

For this purpose, in the present invention, a cumulative value obtained by coding the relative brightness change of neighboring rectangular pixels in each window in binary in the gray scale image is calculated using the projection histogram proposed by Viola and Jones, LBP (Local Binary Pattern) method [6]. The software of such a face detection algorithm is provided in a known OpenCV (Non-Patent Document 7), and the eye image processing unit 12c can be configured as a custom-type module on which the LBP operation algorithm is mounted.

In order to detect the eye region by the LBP method, first, since the eye region is located in the upper half of the face region after the image of the entire face is detected by the camera, the entire face region size is divided into the window region of width (W) And the upper part is defined as an eye area. Then, in a state in which the eye is opened, the histogram curve has a minimum value in the vertical direction based on the position having the largest peak value in the horizontal projection histogram on the vertical axis The eye region must be detected again in the entire face region [Non-Patent Document 8]. However, in the present invention, as shown in Fig. 1, since the camera 3 directly detects an image in the eye region, it is unnecessary to detect the entire face region, that is, the horizontal projection histogram, The degree of eye opening / closing can be detected with only one vertical projection histogram.

If the black pixel value is 1 and the white pixel value is 0 for the binary image I , then I ( x, y ) can be detected at the point (x, y) by the LBP method The integral projection function IPF in the vertical direction in the section [y ₁ , y ₂ ] can be expressed as Equation (4).

6, the vertical projection histogram of the eye region image having the horizontal size W and the vertical size d (= d _max ) shows the distance on the horizontal axis ( x ) and the vertical axis (y) (= IPF _v x ) represents the cumulative value of the pixel, the maximum value of IPF _v ( x ) can be obtained as shown in Equation (5) below.

The N 2 in the binary image I i-th image of the maximum value of h _i tteunnun on N image for unit time when said one h a h _i a I _i h _max, h the minimum value _min, the maximum value and the minimum value of If the difference is denoted by h _dif ,

Y _{bmax ,} y _amin to be the template value.

The h _i value of the image I _i to be input later is used as a feature value of the eye region to be compared with the template value to detect the state of the eye. As shown in FIG. 6A to 6 (d) It can be seen that the accumulated values (IPF) of each pixel according to the image GI can be calculated proportionally to the size of the pupil visible.

D. Dinges proposed PERCLOS (PERCENTAGE OF EYE CLOSURE) parameter according to the degree of eye closure, and defined at least 80% of the state as sleepiness [Non-Patent Document 9]. In the present invention, it is also assumed that the eyes are relaxed by 80% or more in the present invention based on this, and it is determined that the eye is in the drowsy state. In each of the frequency band segments of the EEG shown in Figs. 4A to 4D and the eye opening ratio segments of Figs. 6A to 6D, significantly arousal state image floating snow of i _i more than 70% is input as compared to floating template value, the normal state (Da), less than 70% for more than 50% drowsiness initial state (Db), less than 50% to 20% by drowsiness , be represented by a function f (h _i) of the h _i as shown in equation (9) below to separate the film to enter the state (Dc), less than 20% in the four pattern parameters for a nap state (Dd) and the image I _{of i} The state g ( I _i ) can be expressed as Equation (10).

Thus, g ( I _i ) It is possible to calculate a data value for judging drowsiness according to the eye opening rate as shown in Fig.

Finally, the tilt signal processing unit 13d can obtain the tilt angle of the head with the sensor value detected by the gyro sensor that detects the angular velocity of the head inclined with respect to each axis of x, y and z, (AV) = rotation angle (RA) / time (T), and V * T = distance (D), AV * T = RA. Therefore, by integrating the angle of rotation RA using the angular velocity measured during the unit time ( t1, t2 ), the angle of rotation in each axis of x, y, z can be obtained as follows.

Where W is the angular velocity and the velocity in the clockwise direction on the x, y, z axis is output as a positive (+) value and the opposite direction as a negative (-) voltage. The tilt signal processing unit 13d may also be a module integrated with the gyro sensor.

As described above, the detection signals for EEG, EOG, EMG, eye image and head slope are detected by the signal preprocessing device 13 by the drowsiness detection device 12 using a plurality of sensors, Converted into a processed value, and transmitted to the central processing unit 20, and the process thereafter will be described with reference to FIG.

The detectors 12a to 12d in the drowsiness detecting device 12 of FIG. 8 may be implemented as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) Specific Integrated Circuit), but these modules are not necessarily limited to software or hardware, and may be configured in an addressable storage medium.

9 is a block diagram showing the configuration of the central processing means 20 of the present invention. The configuration of the central processing means 20 includes a digital signal processor (digital demodulator) 21a and a DMA processor 21b having one or more direct memory access A memory 22, an arithmetic control unit 23 (CPU), and first, second and third control signal generators 24a and 24b, which are constituted of a main memory unit 22a and a sub memory unit 22b, , And 24c.

The detection values from the plurality of detectors processed by the signal preprocessing device 13 of the detection means 11 are detected by the binarization device 21a as being advantageous for CPU operation For example, the decimal number "12" is converted into the data value of the binary BCD code represented by "0010010" or "459" is converted into the data value of "111001011" and outputted in a time series manner. The DMA processing unit 21b processes these time series data values by arithmetically averaging them over a unit time and temporarily stores them in the sub memory unit 22b for temporary storage. When data is accumulated, the DMA processing unit 21b It is waked up and processed. For reference, the DMA controller is configured to receive data of a plurality of sensors, and is used to increase the input / output transfer rate by reducing the load on the CPU by directly accessing the data exchange between the peripheral I / O device and the memory unit without going through the CPU . Accordingly, the fact that the data detected from the plurality of detectors is temporarily stored in the memory unit 22 by the DMA signal processing unit 21b means that the data is temporarily stored for a very short period of time for the purpose of increasing the processing speed of a plurality of data, So it can be seen almost in real time without having any effect on judging drowsiness.

A drowsiness analysis algorithm, which will be described later, is mounted in the main memory 22a in the memory 22, and the frequency and voltage magnitude of the band corresponding to the component signal of the alpha wave are quantified A reference value for a drowsiness judgment EEG reference value and EOG, EMG, eye image and head tilt signal are stored (see Table 1 to be described later). The EEG, EOG, EMG, eye image, and tilt signals detected by the respective detectors 12a to 12d from the detectors 12a to 12d are detected by the DMA processing unit 21b in the sub memory unit 22b, One value is temporarily stored, and later these values are used by the operation control unit as a specific value for drowsiness comparison. For reference, the main memory unit 23a is a ROM or a flash memory, and the sub memory unit 23b is a register, a flash memory, an EEPROM, a RAM, or the like.

Here, the reference values for the respective detection signals stored in the main memory 22a may be used as a normalized average value of a normal person, and on the other hand, particularly, EOG, EMG, Since the average number of eye flicker and the size of the pupil are different, there may be a slight error in shaping the average value, such as the EEG signal and the slope signal. Therefore, the numerical value per each drowsiness state It can also be used as a reference value for determining the drowsiness. For this purpose, a reference value for the EOG, EMG, and eye image signals may be set in an initial stage of using the apparatus of the present invention.

Next, the arithmetic and control unit 23 calculates a time series data value for each sensing signal stored in the secondary memory 22b as an average value of sequential data per unit time, and calculates each drowsiness comparison specific data value using a predetermined analysis method After the extraction, it is compared with the respective drowsiness determination reference values stored in the main memory 22a to determine drowsiness. Here, the sequential data is an ordered set of data, for example, S = S1, S2 ... Sn, and each item is numerical data.

As the above-mentioned analysis methods, known principal component analysis (PCA) and linear discriminant analysis (LDA) are known. In general, the PCA is a method for reducing data to represent the data optimally. The LDA is a method for optimally classifying input data by reducing the dimension while keeping the discriminative information value of the classes of input data as possible as possible. We calculated the EEG signal as an independent component using the ICA and minimized the dimension. Since the data to be classified is input from several detectors, we adopted the LDA analysis algorithm that minimizes the classification error between the data. LDA is basically a theory based on Fisher's Linear Discriminant (FLD) theory based on W. Zhao and Peter N. Belhumeur et al., Widely publicized in the fields of bio-signal measurement, statistical data analysis, And therefore, a detailed description thereof will be omitted (Non-Patent Documents 10 and 11).

In the present invention, the drowsiness state is divided into four states according to the intervals of the frequency band of the EEG component: an arousal state (Da), an initial state (Db), an entry state (Dc), and a drowsy state (Dd) The sleep determination operation of the operation control unit 23 will be described in detail with reference to Table 1 below. Note that the reference values in Table 1 are merely numerical examples for convenience of explanation.

As described above, the arithmetic control unit 23 arithmetically averages detected time series data values detected from the sensors 12a to 12d temporarily stored in the sub-memory unit 22b into sequential data values per unit time, and performs optimization analysis using the LDA And compares these values with the reference values for determining drowsiness in Table 1 stored in the main memory 22a to determine each drowsiness state.

First, the operation control unit 23 arithmetically averages the sequential data of the EEG signal detected from the EEG detector 12a and temporarily stored in the main memory unit 22a to obtain a specific data value optimized using the LDA And compares this value with the EEG reference values Vef of the frequency bands of the beta wave and the alpha wave components in Table 1 stored in the main memory unit 22a to determine the state of drowsiness belonging to each corresponding EEG reference value range . As mentioned in the description of the brain wave in FIG. 3, theoretically well-defined alpha-wave and drowsy-state seta are present in the human body. However, in reality, human beings have different biological characteristics, Since the boundaries are ambiguous, in the present invention, as shown in Table 1, two reference values (Vef) from only the arousal component of the arousal state to the alpha-tare component appearing from the arousal state are used as judgment variables of drowsiness. It is also important to detect drowsiness as early as possible, since it is important to detect it at an early stage.

Next, with respect to the EOG signal detected from the EEG detector 12a and independently separated by the ICA in the EEG component processing unit 13a, the operation control unit 23 obtains the mean value obtained from each sequential data (LDO), and compared with the EOG reference value (Vof) in the main memory section, the arousal state (Da), the drowsiness initial state State Db and drowsy entry state Dc, respectively. That is, when the specific data values calculated by the EOG signal are compared with the EOG reference value, when the range is within the reference value range of 5 to 18 (5? Vof? 18) (Vf? 45), it is judged to be the drowsiness initial state (Db), and when it decreases gradually to 4 or less (Vof? 4), the drowsiness state (Dc) is determined. Since the drowsiness state (Dd) of the last delta wave section falls into drowsiness and there is almost no blinking in the present invention, the present invention detects the state before drowsiness, so the reference value of this section is excluded from the judgment variable. For reference, the numerical value of the EOG reference value (Vof) in Table 1 is the number of eye flicker, and based on the research data of the Korean Ophthalmological Society (2007.11, No. 11-323), the eye flicker frequency in the normal state is 5 ~ 18 As shown in FIG.

 The EMG signal detected from the EMG sensor 12b is also compared with the reference value Vmf in the main memory unit by the specific data values calculated by the operation control unit 23 in the same manner as described above, As shown in FIG. Here, the reference value for determining the drowsiness for each state is the same as the EOG reference value, and a further explanation will be omitted. In the present invention, the blinking signal is detected using the EOG signal and the EMG signal. The number of flicker is detected by an EOG signal having a relatively larger amplitude than a general EEG signal. The EMG signal is intentionally detected The signal is used for extracting the signal of the orbicularis muscle which appears when it is blinking greatly, and since the two signals are synchronous with each other, the result of detecting the number of flicker can be obtained by using either one of them, which is complementary .

The operation control unit 23 also calculates a specific data value in the same manner as the above-described method using the image value detected from the eye image sensor 12c, compares the specific data value with the reference value Vif in the main memory, (70%) of the eyes according to the eye opening rate (%) as described above, the degree of awakening (Da) is less than 70% and 50% or more (50≤Vif <70) (20 Vif <50) to the drowsiness entry state (Dc) and less than 20% (Vif <20) to the drowsy state (Dd) in the initial drowsiness state (Db) (See Table 1).

 Finally, the arithmetic and control unit 23 also compares the reference value Vwf with the specific data value calculated in the same manner as described above with respect to the tilt signal detected from the tilt sensor 12d, and detects the state of drowsiness according to the tilt angle of the head do. In the present invention, the absolute value of the inclination reference angle of the head, for example, in the x, y, and z axes is set to a range of 15 degrees or more (15 Vwf) on the basis of 0 degrees, (Dd) when it was more than 15 °. Here, the reference angle of the inclination is arbitrarily set for the purpose of explanation, and may be changed. The reference value (Vwf) for the inclination as shown in Table 1 is applied only to one section during the last drowsy state (Dd), and the reference value Outside intervals were excluded from the drowsiness judgment variable.

As described above, the result of the determination of the degree of drowsiness by the operation control unit 23 for each state is transmitted to the control signal generator 24 composed of the first, second, and third signal generators 24a, 24b, and 24c Output. That is, when the determination result of the drowsiness of the operation control unit 23 is the initial state Db, the first signal generation unit 24a generates the first control signal, And the third signal generator 24c transmits the third control signal to the alarm means 14 and the vehicle control means 30, respectively, when the determination result is the drowsy state (Dc) . At this time, the first, second, and third control signals are output as a pulse signal having a pulse width, that is, a strong signal step by step.

The alarm means 14 is constituted by an auditory alarm 14a and a tactile alarm 14b of a speaker or a vibration motor and is controlled by the first, second and third signal generators 24a , 24b, and 24c to generate an alarm sound and a vibration sound corresponding to the drowsy state.

10 is a detailed block diagram of the vehicle control means according to the present invention. The vehicle control means 30 includes a vehicle control unit 31, a ventilator 32a, an emergency blinker 32b, 32c. The vehicle control unit 31 receives the first control signal from the first signal generating unit 24a and outputs the first control signal to the vehicle control unit 31. [ And activates the emergency blinker lamp 32b that gives a warning signal to the following vehicle when the second control signal is received from the second signal generator 24b, Finally, when it is determined that the vehicle is in the drowsy state (Dc) and the third control signal is received from the third signal generating unit 24c, the vehicle decelerator 32c decelerates the vehicle speed step by step.

The signal output from the control signal generator 24 to the vehicle control means 30 is a wireless pulse signal transmitted by the infrared transmitting and receiving means although not shown in the figure, ), ZigBee (ZigBee), etc., or may be connected to the inside of the vehicle through a wired connection.

As described above, according to the drowsiness prevention apparatus of the present invention, the sensing means 11, the central processing means 20 and the alarm means 14 are provided with grooves 8a and 8b Are formed integrally with a pair of the constituent members 100a and 100b respectively formed with the elastic members 100a and 100b and the elastic connecting members 100c connecting the members to each other and can be worn on the face by glasses.

Next, Fig. 11 is a flow chart showing an embodiment of a control method of the drowsiness driving prevention device according to the present invention, and will be described with reference to Figs. 8 to 10 and Table 1. Fig.

First, a reference numeral S001 for setting a reference value for determining a drowsiness is a reference value for the EEG, EOG, EMG, eye image, and tilt signal detected from each of the detectors 12a to 12d, as described above, The EOG, EMG, and eye image signals may be used as reference values for determining the drowsiness based on values obtained by averaging the sequential data of the EOG, EMG, and eye image signals per unit time See Table 1). The operation control unit 23 arithmetically averages the detected time series data values from the sensors 12a to 12d temporarily stored in the sub memory unit 22b to sequential data values per unit time, EOG, EMG, eye image, and tilt signal, and compares these values with the reference values for determining drowsiness stored in the main memory 22a to determine drowsiness for each state , And will be described in detail later in step S002.

In step S002, the operation control unit 23 compares the specific data value calculated from the EEG signal with the EEG reference value Vef for each brain wave frequency band stored in the main memory unit 22a, (Db), and alpha-wave band is repeated if the reference value range to determine if it is in the (Vef≤12) N (NO) to start step S002 of, for example, if (Yes) and proceeds to the next step. In other words, if the result of the comparison between the specific data value of the EEG signal and the EEG reference value Vef is in the reference value range (13? Vef) of the beta wave band, i.e., 13 or more, S002 is repeated, and when it is 12 or less (Vef &lt; / = 12) at which the alpha wave band section starts, it is determined to be the initial state (Db) and the process proceeds to the next step.

In the next step, it is determined whether there is a blinking signal from the EEG detector 12a or the EMG detector 12b. In step S003, similarly, the specific data value calculated from the EOG signal by the operation control unit and the EOG reference value comparison of the (Vof), for example, by determining whether the rapid increase in the number of times per minute blink EOG signal is drowsiness initial state (Db) of 19 or higher reference value range (19≤Vof≤45) of less than 45, again if NO Step S003 is repeated, and if yes , proceed to the next step. In step S004, it is determined whether the intentional eye blink signal EMG signal generation count calculated in the same manner as in step S003 increases sharply in the range of 19 to 45 (19 Vmf 45). If NO , If YES , the process proceeds to the next step S005.

In step S005, the eye opening rate is determined using the image signal detected from the eye image sensor 12c. The specific data value calculated from the eye image signal is compared with the reference value Vif in the main memory by the operation control unit a result, when it is determined whether or not the eye opening rate of the reference value range (50≤Vif <70) or more than 70% and 50% of eyes, the eye opening rate is still NO arousal state of the eye: a (Da 70≤Vif) is determined If the answer to the step S005 is YES , a final decision is made in the next step S006 to the drowsy initial state (Db), and the alarm means 14 generates a low-level primary alarm sound to note drowsiness at regular intervals The ventilator 32a of the vehicle is operated and the process proceeds to the next step S007. Step S007 in by the primary result, the nap state that ventilate the warning sound and the vehicle is whether the recovery is determined as normal, Yes if it returns to step S002 and restarts the drowsiness detection operation, NO is drowsiness initial state (Db) is as And proceeds to the next step S008.

In step S008, the operation control unit again performs the comparison operation to determine whether the number of eye blinks caused by the EOG signal is within the reference value range (Vof? 4) of the drowsy entry state Dc. If NO , And if Yes is less than 4, the process proceeds to the next step S009. In step S009 it is determined that the eye opening rate of the eye over a range less than 50%, 20% reference value (20≤Vif <50) calculated by the operational and control section repeats the case of step S009 again, and NO, in the case of Yes, and then step The alarming means makes a final decision on the drowsy entry state (Dc) at S010 and generates an intermediate level secondary alarm sound warning the drowsiness at regular intervals again. At the same time, the emergency flushing light 32b of the vehicle is operated, . Step S011 In the results drowsiness, while generating the second alarm whether recovery is determined as normal, and Yes is returned to step S002 and restarts the drowsiness outstanding operation, if NO yet determined that the drowsiness entering state (Dc) And proceeds to the next step S012.

Step S012 In the case of eye opening rate of the eye generated by the operation control unit is to determine whether the nap state (Dd) of the reference value range (Vif <20) of less than 20%, and if NO, repeat the above step S012, and Yes, and then The process proceeds to step S013. If the step S013 in the same manner to determine if the head tilt signal angle range of the reference value (15≤Vwf) 15 ° or more calculated by the operational and control section, which is again repeated step S013 NO, and Yes, in step S014 the state drowsiness (Dd) to generate a tertiary alarm sound of a high level enough to awaken drowsiness by the alarm means, and at the same time, decelerates the vehicle by the vehicle speed reducer (32c) and ends the operation.

As described above, according to the present invention, various factors for determining the drowsiness by integrating the drowsiness detection means and the alarm means are directly detected in the eyes and head regions by using a plurality of sensors to increase the accuracy of drowsiness detection, To warn.

For reference, the electrodes for drowsy detection, tilt sensors and detectors described in the specification and claims to be hereinafter described are expressed in combination in each embodiment for ease of explanation, but these sensors and sensors have the same meaning, And the EEG, the safety, the EOG, the EMG and the EMG have the same meaning, and do not depart from the spirit and scope of the present invention.

While the present invention has been described in detail with reference to the embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: drowsiness driving prevention device 10: detection / alarm means
11: sensing means 12: signal preprocessing device
13: Alarm means 14: Power source
20: central processing means 21: digital signal processing section
22: Memory section 23: Operation control section
24: control signal generator 30: vehicle control means

Claims (8)

A drowsiness driving prevention device comprising a dementia detection / alarming means and a central processing means integrated with each other,
EEG, safety (EOG), EMG (Electroencephalogram) of the eye area, eye area image, and head tilt signal are directly detected by eyes and face area Detecting means for detecting each of the detected signals and preprocessing the detected signals according to a signal processing standard;
And an average value calculating unit for calculating an average value for comparing the sleepiness of each signal calculated by converting the detection signals detected by the sensing unit into digital signals and arithmetically averaging the signals per unit time in a time series sequential manner, A drowsy state Dc and a drowsy state Dd in accordance with the state of drowsiness and generates a selective control signal in accordance with the determination result, Way;
An alarm means formed integrally with the sensing means for receiving an optional control signal according to each drowsiness judgment result from the central processing means and generating an alarm sound having a gradually increasing intensity; And
Vehicle control means for receiving an optional control signal according to each drowsiness determination result from the central processing means and controlling the flicker signal, the internal ventilation, and the deceleration of the vehicle step by step;
Wherein the drowsiness detection and warning is performed through the face and the drowsiness is alerted from an initial state of drowsiness, wherein the drowsiness detection and warning means and the central processing means are integrated. The method according to claim 1,
(a1) an EEG detector for detecting an EEG signal in close contact with both temporal and frontal parts of the face, an EMG sensor for detecting an EMG signal in close contact with an eyeball on one side of the eye, a gray level image signal And a slope detector for detecting a head tilt signal, and a2) amplifying and filtering the EEG signal, extracting an EOG signal included in the EEG signal as an independent component by ICA, An EMG signal processing unit for calculating a detection value by amplifying and filtering the EMG signal, an amplifying unit for amplifying and filtering the gray level image signal, and an image pixel accumulation value by an LBP using a vertical projection histogram An eye image processing unit for calculating each feature value according to the eye opening ratio, and a tilt processing unit for calculating the tilt signal value A signal preprocessing device configured;
b) the central processing means comprises: b1) a binarization apparatus for converting the calculated values received from each processing section in the signal preprocessing apparatus into binarized BCD data values, and b2) B3) a memory unit for storing a time series average value per unit time for each sensor processed by the digital signal processing unit and the drowsiness judgment reference value, b4) a memory unit for storing the time series average value and the drowsiness judgment reference value, The average values are arithmetically averaged by sequential data per unit time through optimization analysis using LDA to obtain respective data values for comparison of drowsiness comparison and the drowsiness state is determined by comparing these values with the drowsiness reference value stored in the memory And b5) generating a selective control signal according to the determination result of the operation control unit And it is provided with first, second and third signal generating portion configured control signal generating unit;
Wherein the alarm means comprises an auditory and tactile alarm for receiving an optional control signal according to the drowsy state and controlling the intensity of the alarm sound;
d) the vehicle control means comprises: d1) a vehicle control portion for receiving the control signal and controlling the vehicle; And d2) a ventilator which operates when the control signal of the first signal generator is received by the vehicle controller, an emergency flasher which operates when a control signal of the second signal generator is received, Wherein the drowsiness detection and alarm means and the central processing means are integrally provided. The method according to claim 1 or 2,
The detecting means, the central processing means, and the alarm means may be constituted by a pair of constituent members each having a groove formed therein in the longitudinal direction thereof and a flexible connecting member integrally connecting the members, Wherein the drowsiness detection and alarm means and the central processing means are integrated with each other. The method of claim 3,
Wherein the EEG sensor is composed of four EEG electrodes installed so as to be in close contact with the inner side of the pair of constituent members and the inner side of the connecting member with both the front and back of the face and the EMG sensor comprises the pair of constituent members And an EMG electrode provided so as to be in close contact with the eyeball side of the eye on one side of the eye, and the eye image sensor is constituted by a camera installed on the inside of one of the pair of the members so as to face one side of the eye, And the tilt sensor is constituted by a gyro sensor embedded in one of the pair of constituent members;
Wherein the auditory and tactile alarms of the alarm means are embedded in any one of the pair of constituent members, wherein the dazzle detection / alarm means and the central processing means are integrated. 5. The method of claim 4,
The transmission means of the control signal output from the control signal generating means to the vehicle control means may be any one of an infrared (IR) transceiver, a Bluetooth for short range communication or a ZigBee module, Wherein the drowsiness detection and alarm means and the central processing means are integrated with each other. A method for controlling a drowsiness driving preventive device comprising a sensor-based drowsiness detection / alarm unit and a central processing unit,
(Db), drowsiness state (Dc), and drowsiness state (Dc) of the EEG, EOG, eye image and head tilt signal detected from the sensing means, (S1) of setting reference values (Vef, Vof, Vmf, Vif, and Vwf) of EEG, EOG, EMG, eye image,
Time series average values from the EEG, EOG, EMG, image, and tilt signals are arithmetically averaged by sequential data per unit time through optimization analysis using LDA by the central processing means to obtain respective drowsiness comparison specific data values, EOG, EMG and the image signal are compared with the corresponding drowsiness determination reference values Vef, Vof, Vmf and Vif to determine whether the comparison result of the EEG, EOG and EMG signals corresponds to the reference value of the drowsiness initial state Db (Vf, Vof and Vmf), and if the comparison result of the image signal is within the reference value Vif of the drowsy initial state (Db), it is finally determined to be the drowsy initial state (Db) Generating a low-level primary alarm sound at a predetermined interval and ventilating the vehicle (S2);
If the initial state of the drowsiness state is maintained, the EOG signal is compared again to determine whether it is within the reference value Vof of the drowsy state Dc, If it is within the reference value Vif of the drowsy entry state (Dc), it is determined again in the drowsy entry state (Dc) to generate a secondary alarm sound at an intermediate level at a predetermined interval, Generating a flashing signal (S3); And
If the drowsiness state is still maintained, it is determined whether the drowsiness state is within the reference value Vif of the drowsy state Dd. If the drowsiness state is still maintained, When the result of comparing the drowsiness comparison specific data value of the tilt signal calculated by the central processing means with the drowsiness determination slope reference value (Vwf) is within the reference value (Vwf) of the drowsy state (Dd) (S4) of gradually decelerating the vehicle by generating a tertiary alarm sound of a high level and finally deciding the vehicle with the vehicle speed (Dd);
Wherein the drowsiness detecting and warning means and the central processing means are integrally provided, wherein the drowsiness detection and warning is performed step by step from the initial state of drowsiness. The method according to claim 6,
If the comparison result of the EEG, the EOG, the EMG and the image signal is still in the awake state (Da) in the step (S2), the comparison operation for each signal is repeated again;
If the result of the comparison between the EOG signal and the image signal is still in the drowsy initial state (Db) in the step S3, the comparison operation for each signal is repeated again;
Wherein when the result of the comparison with the image signal and the tilt signal is in the drowsy entry state (Dc) in the step (S4), the comparison operation for each signal is repeated again. Wherein the alarm means and the central processing means are integrated. The method according to claim 6 or 7,
The EEG reference value Vef for the arousal state Da and the drowsy initial state Db is in the range of 13? Vef of the beta wave frequency band of the average EEG per unit time and Vef? 12 of the start band of the alpha wave;
The EOG and EMG reference values Vof for the arousal state Da, the drowsiness initial state Db and the drowsy entry state Dc are set such that the average eye blinking times of the EOG signals per unit time are 5? Vof and Vmf? 18), 19? (Vof and Vmf)? 45, and (Vof and Vmf)? 4,
The eye image reference value Vif for the arousal state Da, the drowsy initial state Db, the drowsy state Dc and the drowsy state Dd is set such that the average eye opening rate per unit time is 70? Vif, 50? Vif < 70, 20 < Vif < 50 and Vif &lt; And
Wherein the drowsiness judgment reference head slope reference value (Vwf) of the drowsy state (Dd) has an average slope angle per unit time in the range of 15? Vwf, wherein the drowsiness detection and alarm means based on the multi sensor and the central processing means A method of controlling an operation preventing device.

Images