KR102507038B1 - Surveillant system of driver for safety - Google Patents

Abstract

The present invention provides a system for driving a device for combating drowsiness by determining levels based on specific information of brain waves and pupil states of a driver and determining a final level of drowsiness as an average value of the two levels.

Description

Driver monitoring system for safety {Surveillant system of driver for safety}

The present invention relates to a driver monitoring system for safety.

The most common cause of automobile accidents is driver drowsiness. In order to relieve drivers of drowsiness, drowsiness rest areas or truck rest areas are installed in the middle of highways, but they are not used frequently because many drivers save time. Therefore, it can be said that accidents caused by drowsy driving always exist on the road.

In order to solve this problem, many patent documents have been presented.

Patent No. 10-2055097 discloses a content that determines whether a driver is drowsy and sounds a warning sound by using a carbon dioxide sensor that measures the concentration of carbon dioxide inside the vehicle and a lane departure detection camera that photographs whether or not the vehicle has departed from the lane. Patent No. 10-1965578 extracts the characteristic elements of eye blink, eyelid size, face angle, and face shaking from a face image, uses template data digitized, and determines whether the driver is drowsy by using the driver's gripping force on the steering wheel. It discloses the contents of the judgment. Patent No. 10-1189008 measures the amplitude-to-velocity ratio for the closing and opening motions of the eyelids during driver's blinking, and measures the duration of the opening and closing motions. Based on data collected from alert and drowsy subjects, we disclose logic to add weighted average values for amplitude versus velocity for closing and opening motions to obtain a measure of drowsiness that is compared to a scale for drowsiness. .

However, in order to determine whether the driver is drowsy, the driver's EEG analysis is important and provides an accurate index. EEG is an electrical signal that measures the electrical activity of the brain using electrodes attached to the surface of the head. In general, when observing brain waves, a power spectrum analysis method that is classified according to frequency is used. In general, brain waves are artificially divided into δ waves (0.2 ~ 3.99 Hz), θ waves (4 ~ 7.99 Hz), α waves (8 ~ 12.99 Hz), β waves (13 ~ 29.99 Hz), and γ waves according to the range of vibrating frequencies. It is classified into waves (30~50 Hz), and the most frequently detected brain wave when a driver is drowsy is α wave, followed by β wave.

In addition, in order to accurately determine whether the driver is drowsy, it is necessary to supplement the determination by analyzing not only the brain waves but also the driver's eyelid movements and reflecting the correlation with the driver's drowsiness.

However, a technique for determining whether a driver is drowsy by simultaneously using brain wave analysis and eyelid patterns is not known.

Furthermore, in the prior art, when it is determined as drowsiness, it mainly relies on a method of generating a warning sound, but it can be said that emotional stimulation or refinement tailored to the frequency of the brain wave is more effective than such physical stimulation.

The inventors overcame the above limitations of the prior art to develop a novel driver monitoring system capable of accurately determining whether a driver is drowsy and effectively removing the driver's drowsiness.

Therefore, an object of the present invention is to provide a driver monitoring system for safety that accurately determines whether the driver is drowsy by analyzing both the driver's brain wave and the eyelid pattern, and can effectively track the driver's drowsiness.

In order to achieve the above object, the present invention is a driver monitoring system for safety, wherein the system measures an EEG sensor installed on a headrest that contacts the back of the driver and a pattern of the eyelids of the driver's face to measure the driver's brain waves. and a vision sensor installed in the rear mirror to sense, and the control unit of the system determines the driver's drowsiness level based on the information from the EEG sensor, determines the driver's drowsiness level based on the information from the vision sensor, and determines the level of drowsiness of the two sensors Provided is a driver monitoring system that determines the driver's meditation level based on the above and drives a drowsiness avoidance device.

The monitoring system further includes a memory, the memory includes a database for determining a level according to the degree of drowsiness based on the EEG signal measured by the EEG sensor, and a level according to the degree of drowsiness based on the eyelid pattern measured by the vision sensor. A database for determining may be stored.

The drowsiness avoidance device may include a vibration motor, a Peltier device, an audio device, and a speaker.

The vibration motor, audio device, and speaker may be installed on a headrest of a driver's seat.

The Peltier device may be installed on a steering wheel or a driver's seat.

The controller may determine the meditation level based on a simple average value of the degree of drowsiness obtained from the EEG sensor and the degree of drowsiness obtained from the vision sensor, or an average value obtained by multiplying one of the two drowsiness levels by a weight and adding the average value.

The control unit divides the driver's meditation level into 4 levels. At level 1, the audio device is driven to transmit pre-stored music, and at level 2, a vibration motor mounted on the headrest is driven to vibrate the headrest. In the level, the Peltier device installed on the steering wheel or the driver's seat is driven to cool down the temperature of the driver's fingers, hips, and head, and in the fourth level, the ADAS driving unit is operated to switch to smart cruise mode or autonomous driving mode. .

The monitoring system further includes a portable mobile device, and a dedicated app for communicating with a control unit is installed on the mobile device to display a button for driving the drowsiness avoiding device and a meditation level button, so that the driver can actively use the drowsiness avoiding device. You can drive it and adjust the meditation level.
In addition, the present invention is a driver monitoring system for determining the degree of drowsiness of a driver, wherein the system includes an EEG sensor installed on a headrest in contact with the back of the driver to measure brain waves of the driver and a pattern of the eyelids in the driver's face. It includes a vision sensor installed in the rear mirror, and the control unit of the system determines the driver's drowsiness level based on the EEG sensor information, determines the driver's drowsiness level based on the information from the vision sensor, and based on the drowsiness level of the two sensors The driver's meditation level is determined to drive the drowsiness avoidance device, and the monitoring system further includes a memory, the memory includes a database for determining the level according to the degree of drowsiness based on the brain wave signal measured by the EEG sensor, and a vision sensor. A database for determining the level according to the degree of drowsiness based on the eyelid pattern measured by the controller is stored, and the control unit uses the image information of the vision sensor to determine the movement of the pupils in the driver's pupil, the time the eyelids are closed for a certain period of time, and the eyes closed. The number of blinks and the degree of eyelid closure are analyzed, and the vision level is determined by referring to the database on the eyelid pattern recorded in the memory. By analyzing the duration, spectrum amplitude and intensity, and period for each β wave and γ wave, the brain wave pattern is determined, and the brain wave level is determined by comparing with the database stored in memory. The stronger the spectrum, the higher the grade, and the controller determines the meditation level based on the average value of the brain wave level representing the degree of drowsiness obtained from the EEG sensor and the vision level representing the degree of drowsiness obtained from the vision sensor. divides the driver's meditation level into 4 levels. In the 1st level, the audio device is driven to transmit pre-memorized music, in the 2nd level, the vibration motor mounted on the headrest is driven to vibrate the headrest, and in the 3rd level, The driver touches the driver by driving the Peltier device installed on the steering wheel or the driver's seat. coolly lowers the temperature of the fingers, hips, and head, and drives the ADAS driving unit at level 4 to switch to smart cruise mode or autonomous driving mode, and the drowsiness avoidance device includes a vibration motor, a peltier device, an audio device, and a speaker , The vibration motor, the audio device and the speaker are installed on the headrest of the driver's seat to provide a driver monitoring system.

The present invention provides an effect of providing a driver monitoring system for safety that accurately determines whether the driver is drowsy by analyzing both the driver's brain wave and the eyelid pattern, and can effectively track the driver's drowsiness.

1 is an overall configuration diagram of a driver monitoring system for safety of the present invention;
2 is a view showing the configuration of a headrest in a driver's seat of the present invention;
3 is a view showing an example of mounting a vision sensor and a Peltier vidice on the inside front of a vehicle according to the present invention;
Fig. 4 shows an operating process performed by a controller of the present invention; and
5 is a diagram showing a driver monitoring system wirelessly communicating with a mobile device, such as a portable smart phone, in a further embodiment of the present invention.

Each embodiment according to the present invention is only one example to aid understanding of the present invention, and the present invention is not limited to these embodiments. The present invention may be composed of a combination of at least one or more of individual components and individual functions included in each embodiment.

1 is an overall configuration diagram of a driver monitoring system 1 for safety according to the present invention. The monitoring system 1 of the present invention includes a control unit 2, an EEG sensor 4, a vision sensor 6, and a memory M.

The EEG (ElectroEncephaloGraphy) sensor 4 is a sensor for measuring brain waves. The basic EEG sensor is to install wires and electrodes on the human head, but in the present invention, as will be described later, it is preferable to be an exposed type or a band type that is in contact with the driver's head so that it is mounted on the headrest of the driver's seat. In the case of a band-type sensor, a metal electrode-type brain wave detection sensor with a built-in waterproof fiber and a microwave circuit is used. The vision sensor 6 is for detecting a pattern of the eyelids of the driver's face.

The information detected by the EEG sensor 4 and the vision sensor 6 is transmitted to the control unit 2 wirelessly or wired. The control unit 2 refers to the memory M based on the transmitted information and determines a meditation level according to the degree of drowsiness of the driver. In addition, when it is determined that the driver is drowsy, the vibration motor 8, the Peltier device 10, the audio device 12, and the speaker 14 are driven to warn the driver of drowsiness and to get out of the drowsy state.

In addition, the control unit 2 is connected to the ADAS driving unit 300 that performs a smart cruise function and an autonomous driving function.

A database for determining the level according to the degree of drowsiness based on the measured brain waves and a database for determining the level according to the degree of drowsiness based on the eyelid pattern are stored in the memory M. The database of the memory M may be changed in real time by user setting and updating or learning by artificial intelligence. In addition, the memory M stores notification information such as cell phone numbers of family members or acquaintances of the driver in preparation for an emergency situation in which the degree of drowsiness is high.

2 is a view showing the configuration of the headrest 100 in the driver's seat of the present invention.

An EEG sensor 4 for measuring the driver's brain waves is mounted on the headrest 100. The EEG sensor 4 includes a plurality of modules composed of an exposed terminal 4a having an electrode embedded therein, and a wire 4b connected to the exposed terminal 4a and connected to a microwave circuit on the opposite side. The exposed terminal 4a does not necessarily come into direct contact with the driver's head as long as it can measure the driver's brain waves, and may be covered with a soft cloth or mesh cover. Although the number of modules is not particularly limited, it is preferable that the number of modules sufficiently measure brain waves and evenly arranged corresponding to the points of the protruding occipital region.

The vibration motor 8 is installed to chase the drowsiness by vibrating the headrest 100 corresponding to the pillow when it is determined as drowsy. An elastic vibrating member or the like that rotates with a motor pinion may be installed on the front of the vibration motor 8 .

An audio device 12 is embedded inside the headrest 100 . The audio device 12 stores music written in advance in the ROM, and transmits selected music to the speaker 14 according to a drive signal from the control unit 2 .

The speaker 14 is for urging the driver's attention through music or a voice or warning sound when it is determined that the driver is drowsy.

The headrest 100 may further include a memory such as an SD card 16. In the SD card 16, brain wave information of the driver and driving information of the vibration motor 8 and the speaker 12 are stored. The headrest 100 of the present invention is provided with an EEG sensor 4 or the like built in in advance, and in this respect can be referred to as a smart headrest.

3 is a view showing an example of mounting the vision sensor 6 and the Peltier vidice 10 on the inside front of the vehicle according to the present invention.

The vision sensor 6 includes, for example, two pattern recognition cameras 202a and 202b installed on the left side of the rear mirror 202 of the vehicle. As shown, the face of the driver seated on the left can be optimally photographed in this position. The image data taken by the pattern recognition cameras 202a and 202b is transmitted to the controller 2, and information such as pupil movement, eyelid movement state, and degree of eye closing can be acquired.

The peltier device 10 is also called an electric refrigeration system, and forced cooling of the high-temperature part on the other side that requires low-temperature cooling by using the phenomenon that the temperature difference persists at both ends of several layers of conductive material when current flows In the present invention, the high-temperature part is mounted on the inside of the mounted member and the low-temperature part is arranged on the outside, that is, on the side facing the driver's body. The Peltier bidis 10 is preferably mounted at a position where it can come into contact with the driver's body, that is, the lower part of the seat, the steering wheel, or the headrest 10.

Next, on the basis of the above description, referring to Fig. 4, the operation process performed by the controller 2 of the present invention will be described. In the present invention, based on the information obtained from the EEG sensor 4 and the information obtained from the vision sensor 6, the level of drowsiness of the driver is determined to determine the meditation level, and an appropriate drowsiness combating process is started accordingly.

First, the controller 2 receives image information mainly focused on the driver's eyes from the vision sensor 6 (S10).

Next, the control unit 2 analyzes the pupil movement in the driver's pupil, the time the eyelids are closed for a certain period of time, the number of eye blinks, the degree to which the eyelids close the pupils, etc. from the image information, and the eyelids recorded in the memory (M). The vision level is determined by referring to the pattern database (S12). The longer the eyelids are closed and the more times the eyes blink, the higher the level. The vision level ranges from low to high levels of drowsiness, for example, 1 level, 2 levels, ..., 4 levels may be assigned, but this does not limit the scope of the present invention.

The controller 2 receives brain wave information from the EEG sensor 4 simultaneously with the above operation (S20).

Next, the controller 2 determines the driver's brain wave level with reference to the memory M (S22). Specifically, by analyzing the duration, spectrum amplitude and intensity, and period for each δ wave, α wave, β wave, and γ wave of the driver's brain wave, the brain wave pattern is determined and compared with the database stored in the memory M. EEG levels according to EEG patterns are stored in the memory M. The longer the duration of the α wave, the shorter the appearance period, and the stronger the spectrum, the higher the grade. EEG levels range from low to high levels of drowsiness, for example, 1 level, 2 levels, ..., 4 levels may be assigned, but this does not limit the scope of the present invention.

Next, the controller 2 synthesizes the two data based on the vision level and the brain wave level (S30) and determines the meditation level (S32). The meditation level can be determined by obtaining a simple average value of the two levels, or by multiplying the vision level or brain wave level by a weight and then adding another level value to obtain the average value. The meditation level determined by the control unit 2 is stored in the memory M along with vision and brain wave level information and can be used as data for artificial learning.

Next, the control unit drives a drowsiness avoidance device such as the audio device 12 according to the determined meditation level (S40) to prevent the driver's drowsiness.

For example, if 4 grades are assigned to the meditation level from low to high in order of drowsiness, at level 1, the audio device 12 is driven to transmit pre-stored music through the speaker 14. In this way, since the present invention transmits music suitable for awakening sleepiness based on brain wave signals, scientific, rational, and emotionally stable effects can be expected.

In parallel with this, a phrase warning of drowsiness or an alarm sound can be emitted through the speaker 14 . If the meditation level is level 2, the vibration motor 8 is driven along with the music to apply vibration to the headrest 100 to remove the driver's drowsiness.

If the meditation level is level 3, the Peltier device 10 is driven to cool down the temperature of the fingers, hips, and head that the driver touches so that he or she can escape from drowsiness.

If the meditation level is level 4, the control unit 2 drives the ADAS driving unit 300 to activate the smart cruise function to control the vehicle to drive at a constant speed without colliding with front and rear vehicles, or switch to autonomous driving mode to activate the smart cruise function. Of course, for example, it guides and stops the vehicle to a safe zone such as a nearby sleepy shelter or rest area. In parallel, it contacts family members or acquaintances through pre-stored notification information and induces them to directly call the driver, so that the driver's drowsiness is pursued.

The above is an example for combating drowsiness according to the present invention, and driving targets and functions according to each level can be changed in various combinations.

Figure 5 is a diagram showing a driver monitoring system 1 wirelessly communicating with a mobile device 400, such as a portable smart phone, in a further embodiment of the present invention. A dedicated app for communication with the controller 2 is installed in the mobile device 400, and a button and meditation for driving the vibration motor 8, the Peltier device 10, or the audio device 12 are displayed on the display 402. A level button is displayed. When the driver or the user presses the corresponding button, this signal is transmitted to the control unit 2, and the control unit 2 drives the corresponding drowsiness avoidance device. Therefore, according to a further embodiment of the present invention, before passively waiting for the meditation level to be detected, it is possible to actively and actively eliminate drowsiness to ensure safe driving. In addition, when the meditation level is uncomfortable or burdensome to the driver, a pleasant and stable driving atmosphere can be created by appropriately adjusting the level.

Although the embodiments of the present invention have been described above, this does not limit the present invention and various modifications and variations are possible for the present invention. It is obvious that the scope of the present invention extends to the same or equivalent scope as the claims described below.

Claims (8)

A driver monitoring system that determines the driver's drowsiness level, the system includes an EEG sensor installed on the headrest in contact with the back of the driver to measure the driver's brain waves, and a vision installed on the rear mirror to detect the eyelid pattern of the driver's face. The system includes a sensor, and the controller of the system determines the driver's drowsiness level based on the EEG sensor information, determines the driver's drowsiness level based on the vision sensor information, and determines the driver's meditation level based on the driver's drowsiness level. determined to drive the drowsiness avoidance device;
The monitoring system further includes a memory, the memory includes a database for determining a level according to the degree of drowsiness based on the EEG signal measured by the EEG sensor, and a level according to the degree of drowsiness based on the eyelid pattern measured by the vision sensor. A database for determining is stored,
The control unit analyzes the eye movement in the driver's pupil, the eyelid closed time for a certain period of time, the number of eye blinks, and the degree to which the eyelid closes the pupil from the image information of the vision sensor, and stores a database related to the eyelid pattern recorded in the memory. to determine the vision level by reference,
The controller determines the brain wave pattern by analyzing the duration, spectrum amplitude and intensity, and cycle for each δ wave, α wave, β wave, and γ wave of the driver's brain wave among the brain wave information of the EEG sensor, and determines the brain wave pattern and In contrast, the brain wave level is determined, the longer the duration of the α wave, the shorter the appearance period, and the stronger the spectrum, the higher the grade,
The control unit determines the meditation level based on the average value of the brain wave level representing the degree of drowsiness obtained from the EEG sensor and the vision level representing the degree of drowsiness obtained from the vision sensor,
The control unit divides the driver's meditation level into 4 levels. At level 1, the audio device is driven to transmit pre-stored music, and at level 2, a vibration motor mounted on the headrest is driven to vibrate the headrest. In the level, the Peltier device installed on the steering wheel or driver's seat is driven to cool down the temperature of the driver's fingers, hips, and head, and in the fourth level, the ADAS driving unit is driven to switch to smart cruise mode or autonomous driving mode,
The drowsiness avoidance device includes a vibration motor, a Peltier device, an audio device, and a speaker,
The vibration motor, the audio device and the speaker are installed on the headrest of the driver's seat. delete delete delete According to claim 1,
The Peltier device is installed on a steering wheel or a driver's seat, driver monitoring system. According to claim 1,
Wherein the controller determines the meditation level based on an average value obtained by multiplying a weighted value by one of the degree of drowsiness obtained from the EEG sensor and the degree of drowsiness obtained from the vision sensor. delete According to claim 1,
The monitoring system further includes a portable mobile device, and a dedicated app for communicating with a control unit is installed on the mobile device to display a button for driving the drowsiness avoiding device and a meditation level button, so that the driver can actively use the drowsiness avoiding device. A driver monitoring system that allows you to drive and adjust your meditation level.

Images