KR20230026571A - Surveillant system of driver for safety - Google Patents

Abstract

The purpose of the present invention is to provide a driver monitoring system for safety which can accurately determine whether a driver is drowsy by analyzing both the brain wave and the eyelid pattern of the driver, and can effectively counteract the driver's drowsiness. In order to achieve the purpose, the present invention provides a driver monitoring system for safety, wherein the system includes an EEG sensor installed on a headrest that makes contact with the back of a driver's head and a vision sensor installed on a rear mirror to detect a pattern of driver's eyelids, and a control unit of the system determines a drowsiness level of the driver based on the EEG sensor information, determines a drowsiness level of the driver based on the vision sensor information, determines the meditation level of the driver based on the driver's drowsiness levels from the two sensors, and drives a drowsiness counteracting device.

Description

Driver monitoring system for safety {Surveillant system of driver for safety}

The present invention relates to a driver monitoring system for safety.

The most common cause of automobile accidents is driver drowsiness. In order to relieve drivers of drowsiness, drowsiness rest areas or truck rest areas are installed in the middle of highways, but they are not used frequently because many drivers save time. Therefore, it can be said that accidents caused by drowsy driving always exist on the road.

In order to solve this problem, many patent documents have been presented.

Patent No. 10-2055097 discloses a content that determines whether a driver is drowsy and sounds a warning sound by using a carbon dioxide sensor that measures the concentration of carbon dioxide inside the vehicle and a lane departure detection camera that photographs whether or not the vehicle has departed from the lane. Patent No. 10-1965578 extracts the characteristic elements of eye blink, eyelid size, face angle, and face shaking from a face image, uses template data digitized, and determines whether the driver is drowsy by using the driver's gripping force on the steering wheel. It discloses the contents of judging. Patent No. 10-1189008 measures the amplitude-to-velocity ratio for the closing and opening motions of the eyelids during driver's blinking, and measures the duration of the opening and closing motions. Based on data collected from alert and drowsy subjects, we disclose logic to add weighted average values for amplitude versus velocity for closing and opening motions to obtain a measure of drowsiness that is compared to a scale for drowsiness. .

However, in order to determine whether the driver is drowsy, the driver's EEG analysis is important and provides an accurate indicator. EEG is an electrical signal that measures the electrical activity of the brain using electrodes attached to the surface of the head. In general, when observing brain waves, a power spectrum analysis method that is classified according to frequency is used. In general, brain waves are artificially divided into δ waves (0.2 ~ 3.99 Hz), θ waves (4 ~ 7.99 Hz), α waves (8 ~ 12.99 Hz), β waves (13 ~ 29.99 Hz), and γ waves according to the range of vibrating frequencies. It is classified into waves (30~50 Hz), and the most frequently detected brain wave when a driver is drowsy is α wave, followed by β wave.

In addition, in order to accurately determine whether the driver is drowsy, it is necessary to supplement the determination by analyzing not only the brain waves but also the driver's eyelid movements and reflecting the correlation with the driver's drowsiness.

However, a technique for determining whether a driver is drowsy by simultaneously using brain wave analysis and eyelid patterns is not known.

Furthermore, in the prior art, when it is determined as drowsiness, it mainly relies on a method of generating a warning sound, but it can be said that emotional stimulation or refinement tailored to the frequency of the brain wave is more effective than such physical stimulation.

The inventors overcame the above limitations of the prior art to develop a novel driver monitoring system capable of accurately determining whether a driver is drowsy and effectively removing the driver's drowsiness.

Therefore, an object of the present invention is to provide a driver monitoring system for safety that accurately determines whether the driver is drowsy by analyzing both the driver's brain wave and the eyelid pattern, and can effectively track the driver's drowsiness.

In order to achieve the above object, the present invention is a driver monitoring system for safety, wherein the system measures an EEG sensor installed on a headrest that contacts the back of the driver and a pattern of the eyelids of the driver's face to measure the driver's brain waves. and a vision sensor installed in the rear mirror to sense, and the control unit of the system determines the driver's drowsiness level based on the information from the EEG sensor, determines the driver's drowsiness level based on the information from the vision sensor, and determines the level of drowsiness of the two sensors Provided is a driver monitoring system that determines the driver's meditation level based on the above and drives a drowsiness avoidance device.

The monitoring system further includes a memory, the memory includes a database for determining a level according to the degree of drowsiness based on the EEG signal measured by the EEG sensor, and a level according to the degree of drowsiness based on the eyelid pattern measured by the vision sensor. A database for determining may be stored.

The drowsiness avoidance device may include a vibration motor, a Peltier device, an audio device, and a speaker.

The vibration motor, audio device, and speaker may be installed on a headrest of a driver's seat.

The Peltier device may be installed on a steering wheel or a driver's seat.

The controller may determine the meditation level based on a simple average value of the degree of drowsiness obtained from the EEG sensor and the degree of drowsiness obtained from the vision sensor, or an average value obtained by multiplying one of the two drowsiness levels by a weight and adding the average value.

The control unit divides the driver's meditation level into 4 levels. At level 1, the audio device is driven to transmit pre-stored music, and at level 2, a vibration motor mounted on the headrest is driven to vibrate the headrest. In the level, the Peltier device installed on the steering wheel or the driver's seat is driven to cool down the temperature of the driver's fingers, hips, and head, and in the fourth level, the ADAS driving unit is operated to switch to smart cruise mode or autonomous driving mode. .

The monitoring system further includes a portable mobile device, and a dedicated app for communicating with a control unit is installed on the mobile device to display a button for driving the drowsiness avoiding device and a meditation level button, so that the driver can actively use the drowsiness avoiding device. You can drive it and adjust the meditation level.

The present invention provides an effect of providing a driver monitoring system for safety that accurately determines whether the driver is drowsy by analyzing both the driver's brain wave and the eyelid pattern, and can effectively track the driver's drowsiness.

1 is an overall configuration diagram of a driver monitoring system for safety of the present invention;
2 is a view showing the configuration of a headrest in a driver's seat of the present invention;
3 is a view showing an example of mounting a vision sensor and a Peltier vidice on the inside front of a vehicle according to the present invention;
Fig. 4 shows an operating process performed by a controller of the present invention; and
5 is a diagram showing a driver monitoring system wirelessly communicating with a mobile device, such as a portable smart phone, in a further embodiment of the present invention.

Each embodiment according to the present invention is only one example to aid understanding of the present invention, and the present invention is not limited to these embodiments. The present invention may be composed of a combination of at least one or more of individual components and individual functions included in each embodiment.

1 is an overall configuration diagram of a driver monitoring system 1 for safety according to the present invention. The monitoring system 1 of the present invention includes a control unit 2, an EEG sensor 4, a vision sensor 6, and a memory M.

The EEG (ElectroEncephaloGraphy) sensor 4 is a sensor for measuring brain waves. The basic EEG sensor is to install wires and electrodes on the human head, but in the present invention, as will be described later, it is preferable to be an exposed type or a band type that is in contact with the driver's head so that it is mounted on the headrest of the driver's seat. In the case of a band-type sensor, a metal electrode-type brain wave detection sensor with a built-in waterproof fiber and a microwave circuit is used. The vision sensor 6 is for detecting a pattern of the eyelids of the driver's face.

The information detected by the EEG sensor 4 and the vision sensor 6 is transmitted to the control unit 2 wirelessly or wired. The control unit 2 refers to the memory M based on the transmitted information and determines a meditation level according to the degree of drowsiness of the driver. In addition, when it is determined that the driver is drowsy, the vibration motor 8, the Peltier device 10, the audio device 12, and the speaker 14 are driven to warn the driver of drowsiness and to get out of the drowsy state.

In addition, the control unit 2 is connected to the ADAS driving unit 300 that performs a smart cruise function and an autonomous driving function.

A database for determining the level according to the degree of drowsiness based on the measured brain waves and a database for determining the level according to the degree of drowsiness based on the eyelid pattern are stored in the memory M. The database of the memory M may be changed in real time by user setting and updating or learning by artificial intelligence. In addition, the memory M stores notification information such as cell phone numbers of family members or acquaintances of the driver in preparation for an emergency situation in which the degree of drowsiness is high.

2 is a view showing the configuration of the headrest 100 in the driver's seat of the present invention.

An EEG sensor 4 for measuring the driver's brain waves is mounted on the headrest 100. The EEG sensor 4 includes a plurality of modules composed of an exposed terminal 4a having an electrode embedded therein, and a wire 4b connected to the exposed terminal 4a and connected to a microwave circuit on the opposite side. The exposed terminal 4a does not necessarily come into direct contact with the driver's head as long as it can measure the driver's brain waves, and may be covered with a soft cloth or mesh cover. The number of modules is not particularly limited, but it is preferable that the number of modules sufficiently measure brain waves and evenly arranged corresponding to the points of the protruding occipital region.

The vibration motor 8 is installed to chase the drowsiness by vibrating the headrest 100 corresponding to the pillow when it is determined as drowsy. An elastic vibrating member or the like that rotates with a motor pinion may be installed on the front of the vibration motor 8 .

An audio device 12 is embedded inside the headrest 100 . The audio device 12 stores music written in advance in the ROM, and transmits selected music to the speaker 14 according to a drive signal from the control unit 2 .

The speaker 14 is for urging the driver's attention through music or a voice or warning sound when it is determined that the driver is drowsy.

The headrest 100 may further include a memory such as an SD card 16. In the SD card 16, brain wave information of the driver and driving information of the vibration motor 8 and the speaker 12 are stored. The headrest 100 of the present invention is provided with an EEG sensor 4 or the like built in in advance, and in this respect can be referred to as a smart headrest.

3 is a view showing an example of mounting the vision sensor 6 and the Peltier vidice 10 on the inside front of the vehicle according to the present invention.

The vision sensor 6 includes, for example, two pattern recognition cameras 202a and 202b installed on the left side of the rear mirror 202 of the vehicle. As shown, the face of the driver seated on the left can be optimally photographed in this position. The image data taken by the pattern recognition cameras 202a and 202b is transmitted to the controller 2, and information such as pupil movement, eyelid movement state, and degree of eye closing can be acquired.

The peltier device 10 is also called an electric refrigeration system, and forced cooling of the high-temperature part on the other side that requires low-temperature cooling by using the phenomenon that the temperature difference persists at both ends of several layers of conductive material when current flows In the present invention, the high-temperature part is arranged inside the mounted member and the low-temperature part is arranged outside, that is, on the side facing the driver's body. The Peltier bidis 10 is preferably mounted at a position where it can come in contact with the driver's body, that is, the lower part of the seat, the steering wheel, or the headrest 10.

Next, on the basis of the above description, referring to Fig. 4, the operation process performed by the controller 2 of the present invention will be described. In the present invention, based on the information obtained from the EEG sensor 4 and the information obtained from the vision sensor 6, the level of drowsiness of the driver is determined to determine the meditation level, and an appropriate drowsiness combating process is started accordingly.

First, the controller 2 receives image information mainly focused on the driver's eyes from the vision sensor 6 (S10).

Next, the control unit 2 analyzes the pupil movement in the driver's pupil, the time the eyelids are closed during the pre-determined time, the number of times the eyes blink, the degree to which the eyelids close the pupils, etc. from the image information, and the eyelids recorded in the memory (M). The vision level is determined by referring to the pattern database (S12). The longer the eyelids are closed and the more times the eyes blink, the higher the level. The vision level ranges from low to high levels of drowsiness, for example, 1 level, 2 levels, ..., 4 levels may be assigned, but this does not limit the scope of the present invention.

The controller 2 receives brain wave information from the EEG sensor 4 simultaneously with the above operation (S20).

Next, the controller 2 determines the driver's brain wave level with reference to the memory M (S22). Specifically, by analyzing the duration, spectrum amplitude and intensity, and period for each δ wave, α wave, β wave, and γ wave of the driver's brain wave, the brain wave pattern is determined and compared with the database stored in the memory M. EEG levels according to EEG patterns are stored in the memory M. The longer the duration of the α wave, the shorter the appearance period, and the stronger the spectrum, the higher the grade. EEG levels range from low to high levels of drowsiness, for example, 1 level, 2 levels, ..., 4 levels may be assigned, but this does not limit the scope of the present invention.

Next, the controller 2 synthesizes the two data based on the vision level and the brain wave level (S30) and determines the meditation level (S32). The meditation level can be determined by obtaining a simple average value of the two levels, or by multiplying the vision level or brain wave level by a weight and then adding another level value to obtain the average value. The meditation level determined by the control unit 2 is stored in the memory M along with vision and brain wave level information and can be used as data for artificial learning.

Next, the control unit drives a drowsiness avoidance device such as the audio device 12 according to the determined meditation level (S40) to prevent the driver's drowsiness.

For example, if 4 grades are assigned to the meditation level from low to high in order of drowsiness, at level 1, the audio device 12 is driven to transmit pre-stored music through the speaker 14. In this way, since the present invention transmits music suitable for awakening sleepiness based on brain wave signals, scientific, rational, and emotionally stable effects can be expected.

In parallel with this, a phrase warning of drowsiness or an alarm sound can be emitted through the speaker 14 . If the meditation level is level 2, the vibration motor 8 is driven along with the music to apply vibration to the headrest 100 to remove the driver's drowsiness.

If the meditation level is level 3, the Peltier device 10 is driven to cool down the temperature of the fingers, hips, and head that the driver touches so that he or she can escape from drowsiness.

If the meditation level is level 4, the control unit 2 drives the ADAS driving unit 300 to activate the smart cruise function to control the vehicle to drive at a constant speed without colliding with front and rear vehicles, or switch to autonomous driving mode to activate the smart cruise function. Of course, for example, it guides and stops the vehicle to a safe zone such as a nearby sleepy shelter or rest area. In parallel, it contacts family members or acquaintances through pre-stored notification information and induces them to directly call the driver, so that the driver's drowsiness is pursued.

The above is an example for combating drowsiness according to the present invention, and driving targets and functions according to each level can be changed in various combinations.

Figure 5 is a diagram showing a driver monitoring system 1 wirelessly communicating with a mobile device 400, such as a portable smart phone, in a further embodiment of the present invention. A dedicated app for communication with the controller 2 is installed in the mobile device 400, and a button and meditation for driving the vibration motor 8, the Peltier device 10, or the audio device 12 are displayed on the display 402. A level button is displayed. When the driver or the user presses the corresponding button, this signal is transmitted to the control unit 2, and the control unit 2 drives the corresponding drowsiness avoidance device. Therefore, according to a further embodiment of the present invention, before passively waiting for the meditation level to be detected, it is possible to ensure safe driving by actively and actively removing drowsiness. In addition, when the meditation level is uncomfortable or burdensome to the driver, a pleasant and stable driving atmosphere can be created by appropriately adjusting the level.

Although the embodiments of the present invention have been described above, this does not limit the present invention and various modifications and variations are possible for the present invention. It is obvious that the scope of the present invention extends to the same or equivalent scope as the claims described below.

Claims (8)

As a driver monitoring system for safety, the system includes an EEG sensor installed on a headrest in contact with the back of the driver to measure the driver's brain waves, and a vision sensor installed on a rear mirror to detect a pattern of the driver's eyelids in the driver's face, , The controller of the system determines the degree of drowsiness of the driver based on the information of the EEG sensor, determines the degree of drowsiness of the driver based on the information of the vision sensor, and determines the driver's meditation level based on the degree of drowsiness of the two sensors to avoid drowsiness. A driver monitoring system that drives the device. According to claim 1,
The monitoring system further includes a memory, the memory includes a database for determining a level according to the degree of drowsiness based on the EEG signal measured by the EEG sensor, and a level according to the degree of drowsiness based on the eyelid pattern measured by the vision sensor. A driver monitoring system in which a database for determining According to claim 2,
The driver monitoring system, wherein the drowsiness avoidance device includes a vibration motor, a Peltier device, an audio device, and a speaker. According to claim 3,
The vibration motor, the audio device and the speaker are installed on the headrest of the driver's seat. According to claim 3,
The Peltier device is installed on a steering wheel or a driver's seat, driver monitoring system. According to claim 2,
The controller determines the meditation level based on a simple average value of the degree of drowsiness obtained from the EEG sensor and the degree of drowsiness obtained from the vision sensor, or an average value obtained by multiplying one of the two drowsiness levels by a weight and then adding the average value. According to claim 6,
The control unit divides the driver's meditation level into 4 levels. At level 1, the audio device is driven to transmit pre-stored music, and at level 2, a vibration motor mounted on the headrest is driven to vibrate the headrest. Level 4 drives the Peltier device installed on the steering wheel or driver's seat to cool down the temperature of the driver's fingers, hips, and head. Level 4 drives the ADAS drive unit to switch to smart cruise mode or autonomous driving mode. surveillance system. According to claim 1,
The monitoring system further includes a portable mobile device, and a dedicated app for communicating with a control unit is installed on the mobile device to display a button for driving the drowsiness avoiding device and a meditation level button, so that the driver can actively use the drowsiness avoiding device. A driver monitoring system that allows you to drive and adjust your meditation level.

Images