KR102608299B1 - driver assistance apparatus and CONTROL METHOD - Google Patents

Abstract

According to one embodiment of the disclosed invention, a steering wheel; A front image sensor installed in a vehicle, having a front view of the vehicle, and acquiring front image data; a front non-image sensor selected from the group consisting of a radar sensor and a lidar sensor, installed in the vehicle, having a detection field of view in front of the vehicle, and acquiring front detection data; A corner non-image sensor selected from the group consisting of a radar sensor and a LiDAR sensor, installed in the vehicle, having a detection field of view on the side of the vehicle, and acquiring side detection data; a sensor unit provided on the steering wheel and measuring biosignal data of one or more of the driver's electrocardiogram, the driver's neural activity, or brain waves measured from the driver's peripheral nerves; a control unit including a processor that processes the front image data, the front sense data, the side sense data, and the bio-signal data; It includes a front object located in front of the vehicle and a front object located on a side of the vehicle in response to processing the bio-signal data, the image data, the front sensing data, and the side sensing data. Detecting side objects, the control unit provides a driver assistance system that warns the driver.

Description

Driver assistance system and control method thereof {driver assistance apparatus and CONTROL METHOD}

The present invention relates to driver assistance systems.

Recently, research has been conducted on vehicles equipped with an advanced driver assist system (ADAS) that actively provides information on vehicle status, driver status, and surrounding environment to reduce the burden on the driver and improve convenience. is actively underway.

Examples of advanced driver assistance systems mounted on vehicles include Forward Collision Avoidance (FCA), Autonomous Emergency Brake (AEB), and Driver Attention Warning (DAW). In the past, driver assistance systems such as the above generally sensed the vehicle's surrounding environment and operated based on this. However, since the driving environment is not necessarily determined only by the surrounding environment, there is a need to detect the state of the driver, who is driving the vehicle, and operate the driver assistance system.

With this problem in mind, a system that detects the driver's brain activity was developed. Conventionally, the driver's brain activity is directly detected through microcurrent signals that can be detected from the skin around the driver's head by installing an electroencephalogram (EEG sensor) on the skin around the driver's head, or by detecting eye movements in front of the driver. The main method used was to install a sensor (Electrooculogram, EOG sensor) to detect the driver's brain activity based on the driver's eye movements.

In addition, a method of detecting changes in the driver's facial state was used to prevent drowsy driving, etc., but the accuracy of the driver's state change is low and the characteristics of the facial state change are different for each person, so the approach to the changing face is well known. There is not. Despite these driver state detection technologies, fatal traffic accidents caused by drowsy driving were found to account for 15-20% of all traffic accidents.

The conventional system for detecting the driver's brain activity or detecting changes in the driver's facial condition as described above is a method of detecting the driver's biological signals by separately connecting electrodes to the driver's skin, so it is difficult for the driver to use it realistically, and the accuracy There is a problem with falling.

Therefore, there is a need to develop technology that can accurately detect the driver's condition and assist the driver's driving without attaching a separate electrode or device to the driver.

In order to solve the above problems, one aspect of the disclosed invention is a driver assistance system and its control for measuring the driver's brain waves based on the driver's peripheral nerve signals based on the steering wheel and detecting the driver's state. Provides a method.

The disclosed driver assistance system and/or control method includes a steering wheel; A front image sensor installed in a vehicle, having a front view of the vehicle, and acquiring front image data; a front non-image sensor selected from the group consisting of a radar sensor and a lidar sensor, installed in the vehicle, having a detection field of view in front of the vehicle, and acquiring front detection data; A corner non-image sensor selected from the group consisting of a radar sensor and a LiDAR sensor, installed in the vehicle, having a detection field of view on the side of the vehicle, and acquiring side detection data; a sensor unit provided on the steering wheel and measuring biosignal data of one or more of the driver's electrocardiogram, the driver's neural activity, or brain waves measured from the driver's peripheral nerves; and a control unit including a processor that processes the front image data, the front sense data, the side sense data, and the bio-signal data, wherein the control unit is configured to: Determine the drowsiness of the driver.

The controller of the disclosed driver assistance system and/or control method may calculate a threshold based on a frequency band in which the amplitude of the brain wave changes rapidly in the bio-signal data.

The controller of the disclosed driver assistance system and/or control method may compare the amplitude of the brain wave measured from the bio-signal data with the calculated threshold.

The disclosed driver assistance system and/or control method may further include a warning unit that outputs sound or vibration, and the control unit may control the warning unit based on the determination result.

The control unit of the disclosed driver assistance system and/or control method determines whether the driver is drowsy and then determines whether there is a collision with a front object and a side object based on at least one of front image data, front detection data, and side detection data; , the steering device can be controlled based on the determination result of the collision.

The disclosed driver assistance system and/or control method includes acquiring front image data of the vehicle by a camera installed in the vehicle and having a front view of the vehicle; Obtaining front radar data of the vehicle by a front radar installed in the vehicle and having a front detection field of view of the vehicle; Obtaining side radar data of the vehicle using a corner radar installed in the vehicle and having a side detection field of view of the vehicle; In response to processing the front image data, the front radar data, and the side radar data, detect a front object located in front of the vehicle and a side object located on a side of the vehicle; Measures at least one biosignal data of the driver's heart rate, the driver's neural activity, or brain waves measured from the driver's peripheral nerves through a sensor unit provided on the steering wheel; The driver's drowsy state is determined based on the brain waves included in the bio-signal data.

In the disclosed driver assistance system and/or control method, the determination may include calculating a threshold based on a frequency band in which the amplitude of the brain wave rapidly changes in the bio-signal data.

The disclosed driver assistance system and/or control method may include comparing the amplitude of the brain wave measured from the biosignal data with the calculated threshold.

The disclosed driver assistance system and/or control method may further include outputting sound or vibration by a warning unit, and controlling the warning unit based on the determination result.

The disclosed driver assistance system and/or control method determines whether the driver is drowsy and then determines whether there is a collision with a front object and a side object based on at least one of the front image data, the front radar data, and the side radar data. And, the steering device may be controlled based on the result of determining whether there is a collision.

According to one aspect of the disclosed invention, a driver assistance system is provided on a steering wheel and includes a sensor unit that measures biosignals from the driver's peripheral nerves and a control unit that determines whether the driver is drowsy based on the signals measured by the sensor unit. As a result, it is possible to measure brain waves through peripheral nerve signals, detect the driver's condition through this, and warn the driver to promote safe driving.

According to another aspect of the disclosed invention, the present invention detects the driver's state, detects the surrounding environment of the vehicle, and warns the driver based on the detected driver's state and the sensed surrounding environment information, or provides a steering and/or braking device. can be controlled.

1 shows the configuration of a vehicle according to one embodiment.
Figure 2 shows the configuration of a driver assistance system according to one embodiment.
Figure 3 shows a camera and radar included in a driver assistance system according to one embodiment.
Figure 4 shows the interior of a vehicle in which the present invention is installed according to one embodiment.
Figure 5 is a control block diagram of a case where the control unit of the present invention controls a braking device and/or a steering device according to an embodiment, and a case where the warning unit warns by making an alarm sound or vibrating the steering wheel.
Figure 6 is a diagram showing the driver's brain wave spectrum classification when the driver is in a drowsy state.
Figure 7 is a comparison diagram of brain wave spectra according to the driver's condition.

Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the disclosed invention pertains is omitted. The term 'part, module, member, block' used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'part, module, member, block' may be implemented as a single component, or It is also possible for one 'part, module, member, or block' to include multiple components.

Throughout the specification, when a part is said to be “connected” to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection through a wireless communication network. do.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Throughout the specification, when a member is said to be located “on” another member, this includes not only cases where a member is in contact with another member, but also cases where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the disclosed invention will be described with reference to the attached drawings.

1 shows the configuration of a vehicle according to one embodiment.

As shown in FIG. 1, the vehicle 1 includes an engine 10, a transmission 20, a braking device 30, and a steering device 40. The engine 10 includes a cylinder and a piston and can generate power for the vehicle 1 to run. The transmission 20 includes a plurality of gears and can transmit power generated by the engine 10 to the wheels. The braking device 30 can slow down the vehicle 1 or stop the vehicle 1 through friction with the wheels. The steering device 40 can change the driving direction of the vehicle 1.

The vehicle 1 may include a plurality of electrical components. For example, vehicle 1 includes an Engine Management System (EMS) (11), a Transmission Control Unit (TCU) (21), and an Electronic Brake Control Module ( 31), Electronic Power Steering (EPS) 41, Body Control Module (BCM), and Driver Assistance System (DAS).

The engine management system 11 may control the engine 10 in response to the driver's intention to accelerate through the accelerator pedal or a request from the driver assistance system 100. For example, engine management system 11 may control the torque of engine 10.

The transmission control unit 21 may control the transmission 20 in response to the driver's shift command through the shift lever and/or the driving speed of the vehicle 1. For example, the transmission control unit 21 can adjust the shift ratio from the engine 10 to the wheels.

The electronic brake control module 31 may control the braking device 30 in response to the driver's intention to brake through the brake pedal and/or slip of the wheels. For example, the electronic brake control module 31 may temporarily release the brakes on the wheels in response to wheel slip detected when braking the vehicle 1 (Anti-lock Braking Systems, ABS). The electronic brake control module 31 can selectively release the brakes on the wheels in response to oversteering and/or understeering detected when steering the vehicle 1 (electronic stability control, ESC). ). Additionally, the electronic braking control module 31 can temporarily brake the wheels in response to wheel slip detected when driving the vehicle 1 (Traction Control System, TCS).

The electronic steering device 41 can assist the operation of the steering device 40 so that the driver can easily operate the steering wheel 50 in response to the driver's steering intention through the steering wheel 50. For example, the electronic steering device 41 may assist the operation of the steering device 40 to reduce steering force when driving at low speeds or parking and increase steering force when driving at high speeds.

The body control module 51 can control the operation of electrical components that provide convenience to the driver or ensure the driver's safety. For example, the body control module 51 can control headlamps, wipers, clusters, multi-function switches, and turn signal lamps.

The driver assistance system 100 can detect the driver's condition through the sensor unit 103 and assist the driver in operating (driving, braking, and steering) the vehicle 1. For example, the driver assistance system 100 detects the environment around the vehicle 1 (e.g., other vehicles, pedestrians, cyclists, lanes, road signs, etc.) and detects the driver's condition. Driving and/or braking and/or steering of the vehicle 1 may be controlled in response to the environment and the detected driver's state.

The driver assistance system 100 can provide various functions to the driver. For example, the driver assistance system 100 includes Lane Departure Warning (LDW), Lane Keeping Assist (LKA), High Beam Assist (HBA), and Automatic Emergency Braking ( It can provide Autonomous Emergency Braking (AEB), Traffic Sign Recognition (TSR), Smart Cruise Control (SCC), and Blind Spot Detection (BSD).

The driver assistance system 100 includes a camera module 101 that acquires image data around the vehicle 1, a radar module 102 that acquires object data around the vehicle 1, and a sensor unit that detects the driver's condition. Includes (103).

The camera module 101 includes a camera 101a and a controller (Electronic Control Unit, ECU) 101b, and is capable of photographing the front of the vehicle 1 and recognizing other vehicles, pedestrians, cyclists, lanes, road signs, etc. You can.

The radar module 102 includes a radar 102a and a controller 102b, and is capable of acquiring the relative position, relative speed, etc. of objects (e.g., other vehicles, pedestrians, cyclists, etc.) around the vehicle 1. there is.

The sensor unit 103 measures neural activity by the interaction between the first sensor 103a that specifies the driver's brain waves, the second sensor 103b that measures the driver's electrocardiogram, and the driver's sympathetic and parasympathetic nerves. It may include a third sensor 103c.

Meanwhile, the driver assistance system 100 is not limited to what is shown in FIG. 1 and may further include a lidar that scans the surroundings of the vehicle 1 and detects objects.

The above electronic components can communicate with each other through a vehicle communication network (NT). For example, electronic components transmit data through Ethernet, MOST (Media Oriented Systems Transport), Flexray, CAN (Controller Area Network), LIN (Local Interconnect Network), etc. You can give and receive. For example, the driver assistance system 100 provides drive control signals, braking signals, and steering signals to the engine management system 11, the electronic brake control module 31, and the electronic steering device 41, respectively, through the vehicle communication network (NT). Signals can be transmitted.

Figure 2 shows a specific configuration of a driver assistance system according to an embodiment.

As shown in FIG. 2 , the vehicle 1 may include a braking system 32, a steering system 42, a driver assistance system 100, and a warning unit 52.

The braking system 32 includes the electronic braking control module 31 (see FIG. 1) and the braking device 30 (see FIG. 1) described in conjunction with FIG. 1, and the steering system 42 includes the electronic steering device 41 (see FIG. 1). 1) and a steering device 40 (see FIG. 1).

The driver assistance system 100 may include a sensor unit 103, a front camera 110, a front radar 120, a plurality of corner radars 130, and a control unit 140.

The first sensor 103a of the sensor unit 103 detects peripheral nerve signals from the driver's hand and measures the driver's brain waves based on this. Since the driver's peripheral nerve signal detected by the first sensor 103a is an electrical signal, the first sensor 103a may be made of conductive polymer or graphene.

As an example of the disclosed first sensor 103a, the first sensor 103a may use a dry electrode rather than a wet electrode. When using a wet electrode as the electrode of the first sensor 103a, a gel-type liquid must be applied to the driver's scalp and the electrode must be attached, which is inconvenient for the driver while driving. However, when a dry electrode is used as the electrode of the first sensor 103a, there is no need to apply gel-type liquid to the skin, and the driver's peripheral nerve signals detected from the driver's hands, not the driver's scalp, can be detected. It is convenient for the driver while driving.

The second sensor 103b of the sensor unit 103 can measure the driver's electrocardiogram. Since the driver's electrocardiogram detected by the second sensor 103b is also an electrical signal similar to the driver's peripheral nerve signal, the second sensor 103b may be made of conductive polymer or graphene.

The third sensor 103c of the sensor unit 103 detects the driver's peripheral nerve signals and detects changes in the driver's parasympathetic nerve activity and sympathetic nerve activity based on the driver's peripheral nerve signal. Since the driver's neural activity signal detected by the third sensor 103c is also an electrical signal, the third sensor 103c may be made of conductive polymer or graphene.

The control unit 140 includes a processor 141 and a memory 142. The control unit 140 is a separate integrated controller (101b, see FIG. 1) of the camera module (101, see FIG. 1) and/or a controller (102b, see FIG. 1) of the radar module (102, see FIG. 1). May include a controller.

The processor 141 collects the driver's bio-signals measured by the sensor unit 103 and creates an EEG spectrum database based on the collected bio-signals.

In addition, the processor 141 processes the front image data of the front camera 110, the front radar data of the front radar 120, and the corner radar data of the plurality of corner radars 130, and operates the braking system 32 and the steering system. Braking signals and steering signals for controlling (42) can be generated.

The processor 141 is an image processor that processes front image data from the front camera 110 and/or a digital signal processor that processes radar data from the radars 120 and 130, and/or generates braking signals and steering signals, It may include a micro control unit (MCU) that processes the driver's biological signals.

Processor 141 may transmit a braking signal to braking system 32 based on whether a collision with a front object is predicted. If there is no lateral object or a collision with the lateral object is not predicted, the processor 141 may transmit a steering signal to the steering system 42 to avoid collision with the front object. If a collision with a side object is predicted after steering, the processor 141 may not transmit the steering signal to the steering system 42.

The processor 141 may transmit a steering signal to the steering system 42 based on the driver's biological signals and the positions (distance and direction) and relative speeds of objects on the side of the vehicle 1.

The processor 141 calculates the Distance to Collision (DTC) based on the driver's biological signals and the relative speed of the objects ahead, and calculates the Distance to Collision (DTC) based on the comparison between the distance to the collision and the distance to the objects ahead. It is possible to warn the driver of a collision or transmit a braking signal to the braking system 32.

The processor 141 determines whether to avoid a collision with a front object by changing the driving direction of the vehicle 1 based on the driver's biological signals and the positions (distance and direction) and relative speed of objects on the side of the vehicle 1. You can judge. For example, if the driver's brain waves are determined to be those of a sleepy state, and there is no object located on the side of the vehicle 1, the control unit 140 sends a steering signal to the steering system 42 to avoid collision with the front object. ) can be transmitted.

A detailed description of the operation of the processor 141 will be described later with reference to other drawings below.

The memory 142 provides the processor 141 with programs and/or data for processing the driver's biosignal data, programs and/or data for processing image data, and programs and/or data for processing radar data. And, the processor 141 may store programs and/or data for generating braking signals and/or steering signals.

The memory 142 temporarily stores image data received from the front camera 110 and/or radar data received from the radars 120 and 130, and processes the image data and/or radar data by the processor 141. Results can be temporarily remembered.

The memory 142 includes not only volatile memories such as S-RAM and D-RAM, but also flash memory, Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), etc. It may include non-volatile memory.

Meanwhile, the control chip constituting the control unit 140 includes a dot matrix display and an on-chip debugging emulator, and can be a 'Renesas promotion board' that supports analog functions. Additionally, since the measured brain wave spectrum may be different for each vehicle driver, the software that drives the control unit 140 may include customization. The software may be software developed in the C++ language.

The warning unit 52 generates a sound or vibration to a user determined to be drowsy.

Specifically, the warning unit 52 can output a sound by controlling a speaker or navigation system provided in the vehicle 1. The warning unit 52 may output a specific alarm signal to the user.

The warning unit 52 can control a vibration device provided on the steering wheel 50. The warning unit 52 may transmit vibration to the driver through a vibration device.

The driver assistance system 100 is not limited to what is shown in FIG. 2 and may further include a lidar that scans the surroundings of the vehicle 1 and detects objects.

Figure 3 shows a camera and radar included in a driver assistance system according to one embodiment.

The front camera 110 may have a field of view 110a facing the front of the vehicle 1 as shown in FIG. 3 . The front camera 110 may be installed on the front windshield of the vehicle 1, for example.

The front camera 110 can photograph the front of the vehicle 1 and acquire image data of the front of the vehicle 1. The image data in front of the vehicle 1 may include the position of other vehicles or pedestrians or cyclists located in front of the vehicle 1 or the lane.

The front camera 110 may include a plurality of lenses and an image sensor. The image sensor may include a plurality of photo diodes that convert light into an electrical signal, and the plurality of photo diodes may be arranged in a two-dimensional matrix.

The front camera 110 may be electrically connected to the control unit 140. For example, the front camera 110 is connected to the control unit 140 through a vehicle communication network (NT), connected to the control unit 140 through a hard wire, or a printed circuit board (Printed Circuit Board, It can be connected to the control unit 140 through a PCB).

The front camera 110 can transmit image data from the front of the vehicle 1 to the control unit 140.

The front radar 120 may have a field of sensing 120a pointing toward the front of the vehicle 1 as shown in FIG. 3 . The front radar 120 may be installed, for example, on the grille or bumper of the vehicle 1.

The front radar 120 may include a transmitting antenna (or transmitting antenna array) that radiates transmitted radio waves toward the front of the vehicle 1, and a receiving antenna (or receiving antenna array) that receives reflected radio waves reflected by an object. there is. The front radar 120 can acquire front radar data from a transmitted radio wave transmitted by a transmitting antenna and a reflected radio wave received by a receiving antenna. Forward radar data may include distance information and speed information regarding other vehicles or pedestrians or cyclists located in front of the vehicle 1. The front radar 120 calculates the state distance to the object based on the phase difference (or time difference) between the transmitted radio wave and the reflected radio wave, and calculates the relative speed of the object based on the frequency difference between the transmitted radio wave and the reflected radio wave. can do.

The front radar 120 may be connected to the control unit 140, for example, through a vehicle communication network (NT) or a hard wire or printed circuit board. The front radar 120 may transmit front radar data to the control unit 140.

The plurality of corner radars 130 include a first corner radar 131 installed on the front right side of the vehicle 1, a second corner radar 132 installed on the front left side of the vehicle 1, and a second corner radar 132 installed on the front left side of the vehicle 1. ) includes a third corner radar 133 installed on the rear right side of the vehicle (1) and a fourth corner radar 134 installed on the rear left side of the vehicle (1).

The first corner radar 131 may have a detection field of view 131a facing the front right side of the vehicle 1 as shown in FIG. 3 . The front radar 120 may be installed, for example, on the right side of the front bumper of the vehicle 1. The second corner radar 132 may have a detection field of view 132a facing the front left side of the vehicle 1, and may be installed, for example, on the left side of the front bumper of the vehicle 1. The third corner radar 133 may have a detection field of view 133a facing the rear right side of the vehicle 1, and may be installed, for example, on the right side of the rear bumper of the vehicle 1. The fourth corner radar 134 may have a detection field of view 134a facing toward the rear left side of the vehicle 1, and may be installed, for example, on the left side of the rear bumper of the vehicle 1.

Each of the first, second, third, and fourth corner radars 131, 132, 133, and 134 may include a transmitting antenna and a receiving antenna. The first, second, third, and fourth corner radars 131, 132, 133, and 134 acquire first corner radar data, second corner radar data, third corner radar data, and fourth corner radar data, respectively. can do. The first corner radar data may include distance information and speed information about other vehicles, pedestrians, or cyclists (hereinafter referred to as “objects”) located on the right front of the vehicle 1. The second corner radar data may include distance information and speed of an object located on the front left of the vehicle 1. The third and fourth corner radar data may include distance information and relative speed of objects located on the rear right side of the vehicle 1 and the rear left side of the vehicle 1.

Each of the first, second, third and fourth corner radars 131, 132, 133, and 134 may be connected to the control unit 140, for example, through a vehicle communication network (NT) or a hard wire or printed circuit board. there is. The first, second, third, and fourth corner radars 131, 132, 133, and 134 may transmit first, second, third, and fourth corner radar data to the control unit 140, respectively.

Figure 4 shows the interior of a vehicle according to one embodiment. As shown in Figure 4, the vehicle 1 includes a steering wheel 50 and a sensor unit 103.

The sensor unit 103 includes a first sensor 103a that measures the user's brain waves, a second sensor 103b that measures the user's electrocardiogram, and a third sensor that measures the neural activity of the user's sympathetic or parasympathetic nerves. (103c).

Here, the sensor unit 103 may be located on the steering wheel 50, and is not necessarily limited to the position shown in FIG. 4. Additionally, the first sensor 103a, the second sensor 103b, and the third sensor 103c are not necessarily located at the same location and may be provided at different locations.

Figure 5 shows an example of a warning system of a driver assistance system according to an embodiment and an emergency steering and/or emergency braking method of a driver assistance system according to an embodiment. However, this is only a preferred embodiment for achieving the purpose of the present invention, and some steps may be added or deleted as needed.

The sensor unit 103 measures the driver's biological signals (211).

As described above, the biosignal may include at least one of brain waves, electrocardiogram, and sympathetic or parasympathetic nerve activity measured from the driver's peripheral nerves.

The control unit 140 builds a database of the driver's biometric information in real time based on the driver's measured biometric signals (212).

Here, the database includes all measured biometric signals of the driver.

Specifically, the constructed database may include the driver's brain wave information. Since the driver's brain waves are measured as electrical signals, the brain wave information can form a waveform based on the frequency band, and an brain wave spectrum database can be built based on the formed waveform.

The control unit 140 classifies the driver's brain wave spectrum and calculates a threshold value of the brain wave amplitude corresponding to a sleepy state (213).

Specifically, the control unit 140 quantitatively classifies the driver's drowsy brain waves from the generated brain wave spectrum database. For such quantitative measurement, the FFT (Fast Fourier Transform) method can be applied. Since the driver's brain waves show a waveform in which the amplitude rapidly increases in a drowsy state, the control unit 140 may calculate a specific frequency band in which the amplitude of the driver's brain waves increases as the threshold.

After the threshold is calculated, the control unit 140 compares the biological signal measured at the next time point with the assumed threshold (214).

If the measured biosignal is less than the threshold, the control unit measures the biosignal again (No)

If the measured signal is greater than the threshold, the control unit 140 generates a command value to the warning unit 52 (215) and controls the braking device and/or the steering device (225).

That is, if the control unit 140 determines that the driver's brain waves are in a drowsy state as a result of analyzing the driver's brain wave database, it can transmit a signal to the warning unit 52. Additionally, if the control unit 140 determines that the driver is drowsy, it may control the steering and/or braking system to induce the vehicle to park on the shoulder. Meanwhile, the warning unit 52 issues a warning to the driver. This could be a way to sound an alarm. The warning unit 52 may use a method of vibrating the steering wheel 50. However, the warning method of the disclosed invention is not limited to this, and other warning methods may be included.

As another example of the disclosed invention, if the driver's brain waves are determined to be those of a drowsy state and a collision with the front object is determined based on the time until the collision or the distance until the collision, the control unit 140 detects the collision with the front object. To avoid this, a steering signal can be transmitted to the steering system 42.

Figure 6 is a driver's electroencephalogram generated by the control unit 140 based on bio-signals measured by the sensor unit 103 of the present invention according to one embodiment. However, this is only an example of the driver of the present invention, and of course, the electroencephalogram may be different depending on the driver.

The X-axis in FIG. 6 represents the frequency band of the driver's brain waves, and the Y-axis represents the amplitude of the driver's brain waves that have gone through an FFT (Fast Fourier Transform) process.

As shown in FIG. 6, the driver's brain waves tend to be activated in the amplitude of a specific frequency band in a drowsy state, so the control unit 140 creates a database for biological signals that can be measured in various forms as shown in FIG. 6. , and various threshold values can be set in the constructed database.

Figure 7 is a comparison diagram of brain waves when the driver is in a normal state and when the driver is in an abnormal state such as drowsiness.

The X-axis of FIG. 7 shows the cycle of the driver's brain waves per 5 seconds, and the Y-axis shows the amplitude of the brain waves according to the frequency band of the driver's brain waves.

When the driver is in an abnormal state, such as drowsy, the period and amplitude of the driver's brain waves tend to be longer or shorter than the period and amplitude of the brain waves in the normal state. Therefore, when a change in the period and amplitude of the brain wave as shown in FIG. 7 is detected in the built database, the control unit 140 determines that the current driver is in an abnormal state and warns the user or controls the steering and braking devices. can be controlled.

The system and method disclosed through this are provided in the steering wheel 50 and determine whether the driver is drowsy based on the sensor unit 103 that measures signals from the driver's peripheral nerves and the signals measured by the sensor unit 103. It is a driver assistance system including a control unit 140 that measures brain waves through peripheral nerve signals and can detect the driver's condition through this.

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. Instructions may be stored in the form of program code, and when executed by a processor, may create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media storing instructions that can be decoded by a computer. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, and optical data storage devices.

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which the present invention pertains will understand that the present invention can be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

1: Vehicle 50: Steering wheel
100: Driver assistance system 103: Sensor unit
110: front camera 120: front radar
130: Multiple corner raiders 131: First corner raider
132: 2nd corner radar 133: 3rd corner radar
134: fourth corner radar 140: control unit

Claims (10)

steering wheel;
A front image sensor installed in a vehicle, having a front view of the vehicle, and acquiring front image data;
a front non-image sensor selected from a sensor group consisting of a radar sensor and a lidar sensor, installed in the vehicle, having a detection field of view in front of the vehicle, and acquiring front detection data;
A lateral non-image sensor selected from a sensor group consisting of a radar sensor and a LiDAR sensor, installed in the vehicle, having a detection field of view on the side of the vehicle, and acquiring side detection data;
a sensor unit provided on the steering wheel and measuring bio-signal data including the driver's electrocardiogram, the driver's neural activity, or at least one brain wave measured from the driver's peripheral nerves; and
A control unit including a processor that processes the front image data, the front sense data, the side sense data, and the bio-signal data,
The control unit,
A driver assistance system that determines the driver's drowsy state based on the brain waves included in the bio-signal data. According to clause 1,
The control unit,
A driver assistance system that calculates a threshold based on a frequency band in which the amplitude of the brain wave changes rapidly in the bio-signal data. According to clause 2,
The control unit,
A driver assistance system that compares the amplitude of the brain wave measured from the bio-signal data with the calculated threshold. According to clause 1,
It further includes a warning unit that outputs sound or vibration,
The control unit,
A driver assistance system that controls the warning unit based on the judgment result. According to clause 1,
The control unit,
After determining whether the driver is drowsy, determining whether there is a collision with a front object and a side object based on at least one of the front image data, the front detection data, and the side detection data,
A driver assistance system that controls a steering device based on the determination result of the collision. Obtaining front image data of the vehicle using a camera installed in the vehicle and having a front view of the vehicle;
Obtaining front detection data of the vehicle by a front radar sensor or LiDAR sensor installed in the vehicle and having a front detection field of view of the vehicle;
Obtaining side detection data of the vehicle by a side radar sensor or LiDAR sensor installed in the vehicle and having a side detection field of view of the vehicle;
In response to processing the front image data, the front detection data, and the side detection data, detect a front object located in front of the vehicle and a side object located on a side of the vehicle;
Measure bio-signal data including the driver's heart rate, the driver's neural activity, or at least one brain wave measured from the driver's peripheral nerves through a sensor unit provided on the steering wheel; and
A control method of a driver assistance system comprising: determining a drowsy state of the driver based on the brain waves included in the bio-signal data. According to clause 6,
The above judgment is,
A control method for a driver assistance system comprising: calculating a threshold based on a frequency band in which the amplitude of the brain wave rapidly changes in the bio-signal data. According to clause 7,
A control method of a driver assistance system that compares the amplitude of the brain wave measured from the bio-signal data with the calculated threshold. According to clause 6,
It further includes outputting a sound or vibration by a warning unit,
A control method of a driver assistance system that controls the warning unit based on the determination result. According to clause 6,
After determining whether the driver is drowsy, determining whether there is a collision with a front object and a side object based on at least one of the front image data, the front radar data, and the side radar data,
A control method of a driver assistance system that controls a steering device based on the determination result of the collision.