KR20200055171A - Apparatus and method for supporting safe driving - Google Patents

Abstract

Disclosed are an apparatus and a method of supporting safe driving. The apparatus for supporting safe driving comprises: an emotional index detection unit which detects the emotional index of a driver and passengers by using at least one of a vision index in accordance with the movements of eyes, facial expressions, and dazzling of eyes of each of the driver and passengers, voice index in accordance with characteristics of voice signals, and behavior index in accordance with movements; a driver danger index detection unit which detects the driver danger index by using driving information of the driver′s own vehicle and surrounding environment information; a driving difficulty detection unit which detects the driving difficulty in accordance with the driving position of the driver′s own vehicle; a vehicle driving unit which controls the autonomous driving of the driver′s own vehicle; and a control unit which allows the driver′s own vehicle to drive into a safe region by controlling the vehicle driving unit in accordance with whether the emotional index, the driver danger index, and the driving difficulty are included in a preset safe area or not.

Description

Safe driving support device and method {APPARATUS AND METHOD FOR SUPPORTING SAFE DRIVING}

The present invention relates to a safe driving support device and method, and more specifically, to determine whether the vehicle is currently driving safely by considering the driver's and passenger's emotional index, driver's risk index, and driving difficulty in general, and to determine whether the vehicle is safe. It relates to a safe driving support device and method for moving a vehicle to a safe area through steering and braking control to prevent accidents in advance, and to improve driving feelings by improving the emotional state of drivers and passengers.

In general, various factors, such as increased number of vehicles on the road, limited road capacity, and causes of stress associated with modern life, tend to exacerbate high stress conditions that can lead to road rage and / or other stress-causing accidents.

Conventional approaches to road safety do not mitigate these types of accidents.

Recently, the autonomous driving control system has partially disclosed a technology for controlling an autonomous vehicle by reflecting the driver's emotional characteristics, but it is insufficient to induce safe driving by comprehensively considering driving conditions or external situations.

Background art of the present invention is disclosed in 'Vehicle Voice Interface Device' of Korean Patent Publication No. 10-2017-0109758 (October 10, 2017).

The present invention was devised to improve the above-mentioned problems, and the purpose according to one aspect of the present invention is to determine whether the vehicle is currently safe by considering the driver's and passenger's emotional index, driver's risk index, and driving difficulty in a comprehensive manner. According to the judgment result, the vehicle can be moved to a safe area through steering and braking control of the vehicle to prevent accidents in advance, as well as to improve the driver's and passengers' emotional state, thereby improving driving safety. Is to provide a way.

The apparatus for supporting safe driving according to an aspect of the present invention includes at least one of a visual index according to the movement of the gaze of each driver and a passenger, a facial expression and glare, a speech index according to the characteristics of a voice signal, and a behavior index depending on the movement. An emotion index detection unit for detecting an emotion index of a driver and a passenger by using it; A driver risk index detecting unit for detecting a driver risk index using the own vehicle driving information and surrounding environment information of the host vehicle; A driving difficulty detecting unit detecting a driving difficulty according to a driving position of the host vehicle; A vehicle driving unit that controls autonomous driving of the host vehicle; And a control unit for controlling the vehicle driving unit according to whether the emotion index, the driver's risk index, and the driving difficulty are set in the set safety zone, so that the host vehicle travels to the safety zone.

The emotion index detection unit of the present invention includes: a first emotion recognition unit that detects a characteristic value of a voice signal of a driver and a passenger and detects a voice index according to the detected characteristic value; A second emotion recognition unit that detects a driver and a passenger's visual index by photographing a driver and a passenger and analyzing an image taken around the driver; A third emotion recognition unit detecting a position of the accelerator or brake pedal to detect an oscillation type of the accelerator pedal or brake pedal, a driver's operating frequency, and an action index according to the movement of a passenger; And it characterized in that it comprises an emotional index calculating unit for calculating the emotional index using the voice index, visual index and behavioral index.

The first emotion recognition unit of the present invention, a voice signal detection unit for detecting the voice signal of the driver and passengers; And a voice index detection unit that detects a characteristic value of the voice signal detected by the voice signal detection unit and detects a voice index corresponding to the detected characteristic value.

The second emotion recognition unit of the present invention includes: a driver face photographing unit for photographing a driver's eye or face; A passenger face photographing unit for photographing a passenger's eye or face; A stereo camera photographing the surroundings of the host vehicle; And detecting a gaze and a face based on the driver's eye position in an image photographed by the driver's face photographing unit, and detecting a gaze and a face based on the passenger's eye location in the image photographed by the passenger's face photographing unit, stereo The time to detect the position of the light source from the image captured by the camera, and then to detect the visual index according to the driver's and passenger's gaze movements, facial expressions, and the driver's glare by the position of the light source and the position of the eyes Characterized in that it comprises an index detection unit.

The third emotion recognition unit of the present invention includes an oscillation type detection unit that detects an oscillation type of an Excel pedal or a brake pedal; A passenger motion index detector configured to detect a passenger motion index using at least one of a passenger's arm motion size, a motion radius, and a passenger's motion frequency; A collision frequency detection unit that detects a collision operation index according to a driver's operation frequency; And an oscillation index corresponding to the oscillation type of the Excel pedal or the brake pedal, respectively detected by the oscillation type detection unit, and the cushion operation index detected by the collision frequency detection unit. And it characterized in that it comprises a behavior index detection unit for detecting the behavior index of the driver and the passenger by using the passenger motion index detected by the passenger motion index detection unit.

The present invention is characterized in that it further comprises an emotional state control unit for adjusting the emotional state of the driver and the passenger so that the emotional state of the driver and the passenger becomes a steady state.

The control unit of the present invention is characterized by controlling the emotion state control unit if the emotion index is not included in the predetermined stable region by applying the voice index, the visual index, and the behavior index to the learning-based emotion map.

The learning-based emotion map of the present invention is characterized in that the emotional state of the driver is divided into a stable region or a dangerous region according to each of the voice index, the visual index, and the behavior index.

The control unit of the present invention is characterized in that it controls the windshield transmittance adjustment unit included in the emotional state control unit according to the visual index detected by the emotion index detection unit.

The driver's risk index detection unit of the present invention includes: a self-driving travel trajectory generation unit generating a self-driving travel trajectory using self-driving travel information received from an internal sensor that senses a traveling situation of the own vehicle; A surrounding vehicle trajectory generation unit that generates a surrounding vehicle trajectory using the surrounding environment information received from an external sensor that detects a surrounding situation of the host vehicle; A trajectory load calculation unit that calculates a trajectory load indicating a comparison result between a trajectory distance, which is a difference between a surrounding vehicle trajectory and a self-driving trajectory, and a preset threshold; And it characterized in that it comprises a risk index management unit for generating a driver risk index corresponding to the calculated trajectory load.

The driving difficulty detection unit of the present invention is characterized in that it detects the driving difficulty according to the current position of the host vehicle by defining and digitizing the driving difficulty for each zone.

The control unit of the present invention is characterized in that, when the emotion index, the driver's risk index, and the driving difficulty are not included in the safety zone, the vehicle driving unit is controlled to drive the host vehicle to the safety zone.

According to another aspect of the present invention, a safe driving support method includes: an emotion index detection unit, a visual index according to the gaze movements, facial expressions, and glare of each driver and a passenger, a voice index depending on characteristics of a voice signal, and a behavior index according to movement. Detecting an emotion index of a driver and a passenger using at least one of the following; A driver risk index detecting unit detecting a driver risk index using the own vehicle driving information and the surrounding environment information; A driving difficulty detecting unit detecting a driving difficulty according to a driving position of the host vehicle; And a step in which the control unit determines whether the emotion index, the driver's risk index, and the driving difficulty are included in the predetermined safety area, and controls the vehicle driving unit according to the determination result to drive the host vehicle to the safe area.

In the step of detecting the emotional index of the present invention, a driver and a passenger's voice signal characteristic values are detected, a voice index is detected according to the detected characteristic values, a driver and a passenger are photographed, and an image photographed around the driver is analyzed. The visual index is detected according to the driver's glare by the position of the light source and the position of the light source and the position of the light source and the eyes of each passenger and the passenger, and detects the position of the accelerator or brake pedal to detect the position of the accelerator or brake pedal. , After detecting the behavioral index according to the driver's frequency of operation of the driver and the movement of the passenger, the emotion index is calculated using the voice index, the visual index, and the behavioral index.

The control unit of the present invention further includes the step of controlling the emotional state of the driver and the passenger so that the emotional state of the driver and the passenger becomes a steady state by controlling the emotional state adjusting unit according to whether the emotional index is included in the predetermined stable region. Is done.

In the present invention, the step of adjusting the emotional state of the driver and the passenger is characterized in that the control unit includes controlling the windshield transmittance adjustment unit included in the emotional state adjusting unit according to the visual index detected by the emotional index detecting unit.

The step of detecting the driver's risk index of the present invention includes: generating a vehicle driving trajectory using the vehicle driving information received from an internal sensor that detects a driving situation of the vehicle; Generating surrounding vehicle trajectories of the own vehicle by using the surrounding environment information input from an external sensor detecting the surrounding situation of the own vehicle; Calculating a trajectory load amount representing a comparison result between a trajectory distance, which is a difference between a surrounding vehicle trajectory and a self-driving trajectory, and a preset threshold; And generating a driver risk index corresponding to the calculated trajectory load.

The step of detecting the driving difficulty of the present invention is characterized in that, by defining and digitizing the driving difficulty for each zone, the driving difficulty is detected according to the current position of the host vehicle.

The step of driving the host vehicle of the present invention to a safe area is characterized by determining whether the emotion index is included in a predetermined stable region by applying a voice index, a visual index, and a behavior index to a learning-based emotion map.

The learning-based emotion map of the present invention is characterized in that the emotional state of the driver is divided into a stable region or a dangerous region according to each of the voice index, the visual index, and the behavior index.

The safe driving support apparatus and method according to an aspect of the present invention comprehensively considers the driver's and passenger's emotion index, the driver's risk index, and the driving difficulty to determine whether the vehicle is currently driving and to steer the vehicle according to the determination result. The vehicle is moved to a safe area through braking control to prevent accidents.

1 is a block diagram of a safe driving support apparatus according to an embodiment of the present invention.
2 is a block diagram of an emotional state detection unit according to an embodiment of the present invention.
3 is a view showing an emotion map showing a stable region of an emotion index according to an embodiment of the present invention.
4 is a block diagram of a driver risk index detection unit according to an embodiment of the present invention.
5 is a block diagram of a surrounding environment recognition unit of FIG. 4.
FIG. 6 is a diagram schematically showing three dangerous areas classified based on TTC in the vehicle load calculation unit shown in FIG. 5.
FIG. 7 is a diagram illustrating a configuration diagram of a sensor for detecting surrounding vehicles present in each of the dangerous areas illustrated in FIG. 6.
8 is a view showing an example of detecting a surrounding vehicle through the sensor configuration shown in FIG. 7.
9 is a view showing a safety region of an emotional index, a driver's risk index, and driving difficulty according to an embodiment of the present invention.
10 is a block diagram of an emotional state control unit according to an embodiment of the present invention.
11 is a block diagram of a vehicle driving unit according to an embodiment of the present invention.
12 is a flowchart of a method for supporting safe driving according to an embodiment of the present invention.
13 is a flowchart illustrating a driver risk index detection process according to an embodiment of the present invention.

Hereinafter, a safe driving support apparatus and method according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of the lines or the size of components shown in the drawings may be exaggerated for clarity and convenience. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to a user's or operator's intention or practice. Therefore, the definition of these terms should be made based on the contents throughout the present specification.

1 is a block diagram of a safe driving support apparatus according to an embodiment of the present invention, FIG. 2 is a block diagram of an emotional state detection unit according to an embodiment of the present invention, and FIG. 3 is an embodiment of the present invention FIG. 4 is a block diagram of a driver's risk index detection unit according to an embodiment of the present invention, and FIG. 5 is a block diagram of a surrounding environment recognition unit of FIG. 4. 6 is a diagram schematically showing three dangerous areas divided based on TTC by the surrounding vehicle load calculation unit illustrated in FIG. 5, and FIG. 7 is a view of surrounding vehicles present for each dangerous area illustrated in FIG. 6. 8 is a view showing a sensor configuration for detecting, FIG. 8 is a view showing an example of detecting a surrounding vehicle through the sensor configuration shown in FIG. 7, and FIG. 9 is an emotional index and driver risk according to an embodiment of the present invention FIG. 10 is a block diagram of an emotional state control unit according to an embodiment of the present invention, and FIG. 11 is a block diagram of a vehicle driving unit according to an embodiment of the present invention. .

Referring to Figure 1, the safe driving support apparatus according to an embodiment of the present invention the emotion index detection unit 10, the driver risk index detection unit 20, the driving difficulty detection unit, the control unit 40, the emotional state control unit 50 And a vehicle driving unit 60.

The emotion index detection unit 10 uses the driver's and passenger's eyesight, at least one of a visual index according to facial expression and glare, a voice index according to the characteristics of the voice signal, and an action index according to the movement. Emotional index is detected.

Referring to FIG. 2, the emotion index detection unit 10 includes a first emotion recognition unit 11, a second emotion recognition unit 12, a third emotion recognition unit 13, and an emotion index calculation unit 14. .

The first emotion recognition unit 11 detects a characteristic value of the voice signal of the driver or passenger and detects the characteristic value by detecting the voice index indicating the emotional state of the driver or passenger using the voice signal of the driver or passenger Voice index is detected in accordance with.

The voice index is a voice-based detection of the emotional state of the driver or passenger and the detected emotional state is numerical, and may be set to each of the characteristic values of the driver's or passenger's speech sounds.

The first emotion recognition unit 11 includes a voice signal detection unit 111 and a voice index detection unit 112.

The voice signal detection unit 111 detects the voice signal of the driver or passenger, and converts the utterance sound of the driver or passenger into an electrical voice signal.

A microphone or the like may be employed as the voice signal detection unit 111.

The voice index detection unit 112 processes the voice signal detected by the voice signal detection unit 111 to detect a characteristic value, for example, a tone or pitch of a driver or a passenger, and detects the detected voice signal. The voice index corresponding to the tone or pitch of the driver or passenger is detected.

The second emotion recognition unit 12 photographs the eyes or faces of the driver or passenger and analyzes the captured image to detect the visual index of the driver or passenger.

The visual index detects the emotional state of the driver or passenger on a visual basis and quantifies the detected emotional state, and may be set to the driver's gaze, facial expression, and glare.

The second emotion recognition unit 12 includes a driver face photographing unit 121, a passenger face photographing unit 123, a stereo camera 124, and a visual index detection unit 122.

The driver's face photographing unit 121 photographs the driver's eyes or face. The installation location of the driver's face photographing unit 121 is not particularly limited, and may be installed at various locations in the host vehicle as long as it can photograph the driver's eyes or face.

The passenger's face photographing unit 123 photographs the passenger's eyes or face. The installation position of the passenger's face photographing unit 121 is not particularly limited, and may be installed at various positions in the host vehicle as long as it can photograph the passenger's eyes or face.

The stereo camera 124 photographs the surroundings of the host vehicle to detect the location of the light source outside the host vehicle.

The visual index detection unit 122 detects the driver's and passenger's gazes and faces from the images captured by the driver's face photographing unit 121, the passenger's face photographing unit 123, and the stereo camera 124, respectively, and the detected driver and passenger In addition to detecting the visual index according to the movement or facial expression of the eye, the 3D coordinates of the light source are detected from the image captured by the stereo camera 124, and the position of the driver's eyes in the image captured by the driver's face photographing unit 121 Can be detected in 3D coordinates to detect the visual index according to glare.

The visual index detection unit 122 detects a visual index corresponding to each of the gaze or facial expression and glare that appears when the driver and the passenger's emotional state are in an excited state, tracking movement or direction of the gaze, or detecting a change in facial expression Then, after the glare detected by the light source is detected, the movement or direction of these gazes and changes in facial expressions and the driver's visual index corresponding to the glare are detected.

The third emotion recognition unit 13 detects the behavioral index according to the emotional state of the driver and the passenger.

The behavior index is based on the behavior and detects the driver's or passenger's emotional state and quantifies the detected emotional state, and is set to the type of oscillation or the movement of the passenger when the driver activates the cushion or presses the excel pedal or brake pedal. do.

The third emotion recognition unit 13 includes an oscillation type detection unit 131, a passenger motion index detection unit 133, a crash frequency detection unit 134, and a behavior index detection unit 132.

The oscillation type detection unit 131 detects the oscillation type of the excel pedal or brake pedal through the position of the excel pedal or brake pedal.

The passenger motion index detection unit 133 detects the passenger motion index according to the motion of the passenger. That is, the passenger motion index detection unit 133 photographs the passenger and detects the passenger motion index using at least one of the motion of the passenger, for example, the size and radius of the arm motion of the passenger and the frequency of motion of the passenger in the captured image. do. Here, the movement of the passenger may be photographed through a separate camera that photographs the passenger inside.

That is, the passenger motion index detecting unit 133 pre-stores the passenger motion index corresponding to the motion of the passenger, and after detecting any one of the arm motion size and motion radius of the passenger and the frequency of the motion of the passenger in the captured image, A passenger motion index corresponding to the detected motion is detected.

The collection frequency detection unit 134 detects the operation coefficient of the driver according to the operation frequency of the driver.

The behavior index detection unit 132 includes an oscillation index corresponding to the oscillation type of the excel pedal or brake pedal sensed by the oscillation type detection unit 131, the driver's collision operation index and the passenger sensed by the collision frequency detection unit 134. The driver's and passenger's action index is detected using the passenger's motion index detected by the motion index detector 133.

Here, the behavior index may be any one of an oscillation index, a friction operation index, and a passenger motion index, or a value using both an oscillation index, a friction operation index, and a passenger motion index may be employed.

For example, if the action index is the oscillation index, the kinking action index, and the passenger motion index, the value of these may be a relatively large value.

In addition, when the behavioral index uses both the oscillation index, the kinetic operation index, and the passenger motion index, the behavioral index may be an average value of the oscillation index, the kinetic operation index, and the passenger motion index, or may be calculated by reflecting weights in each of them. .

The emotion index calculating unit 14 calculates the emotion index by applying the above-described voice index, visual index, and behavior index to the learning-based emotion map.

The learning-based emotion map is a three-dimensional emotion map using a voice index, a visual index, and a behavioral index, and whether the driver's emotional state is stable or dangerous according to each of the voice index, the visual index, and the behavioral index. Make it possible to judge. That is, in the learning-based emotion map, the emotional state of the driver is divided into a stable area or a dangerous area according to each of the voice index, the visual index, and the behavior index.

In addition, the emotion index calculator 14 detects the emotion index by using a voice index, a visual index, and an action index as shown in FIG. 3.

When the emotional index is not included in the predetermined stable region by applying the voice index, the visual index, and the action index to the learning-based emotion map, the emotional state control unit 50 operates to maintain the driver's emotional state in a steady state. This will be described later.

The driver's risk index detection unit 20 analyzes the driving tendency and calculates a driver's risk index (or surrounding risk index) corresponding thereto. The calculated result is provided to the driver in various types of information, informing the driver of the danger of driving.

The driver risk index detection unit 20 detects the current driving state of the host vehicle and the driving situation around the host vehicle in real time through various sensors in the vehicle, and determines the driving risk situation of the vehicle reflecting the detected result.

Referring to FIG. 4, the driver risk index detection unit 20 includes an internal sensor 21, an external sensor 22, and a surrounding environment recognition unit 23.

The internal sensor 21 detects the driving situation of the host vehicle 24 and generates the own vehicle driving information. Here, the vehicle driving information may include vehicle speed information, yaw rate information, steering angle information, acceleration information, and wheel speed information.

Accordingly, the internal sensor 21 detects vehicle speed sensor 211 for obtaining vehicle speed information, yaw rate sensor 212 for obtaining yaw rate information, steering angle sensor 213 for obtaining steering angle information, and acceleration and direction of the vehicle It may include a G sensor 214, and a wheel speed sensor 215 for obtaining the wheel speed information.

The external sensor 22 detects the surrounding situation of the host vehicle 24 and generates surrounding environment information. Here, the surrounding environment information may include front / rear radar information, front / rear image information, lateral ultrasound information, AVM (Around View Monitoring) image information, and navigation information.

Therefore, the external sensor 22 includes a front / rear radar 221 for acquiring front / rear radar information, a front / rear camera 222 for acquiring front / rear image information, and a lateral ultrasound sensor for acquiring lateral ultrasound information ( 223), an AVM camera 224 for acquiring AVM image information, and a navigation 225 for acquiring navigation information.

The surrounding environment recognition unit 23 calculates the trajectory load amount using the own vehicle driving information provided from the internal sensor 21 and the surrounding environment information provided from the external sensor 22, and through this, the driver's risk index (or surrounding risk index) ). This will be described in detail with reference to FIG. 5 below.

Referring to FIG. 5, the surrounding environment recognition unit 23 calculates the trajectory load amount using the own vehicle driving information provided from the internal sensor 21 and the surrounding environment information provided from the external sensor 22, thereby driving the driver. Manage the index (or surrounding risk index). For more accurate driver risk index management, the surrounding environment recognition unit 23 may manage the driver risk index in consideration of the surrounding vehicle load and the road load in addition to the trajectory load. Therefore, considering the trade-off between the system load and the driver's risk index, only the trajectory load is considered, the case where only the trajectory load and the surrounding vehicle load are considered, and the case where only the trajectory load and the road load are considered are performed to perform the driver's risk index. Various system designs are possible.

In FIG. 5, it should be noted that the driver risk index management considering all of the trajectory load, the surrounding vehicle load, and the road load is an exemplary view, and the present invention is not limited thereto.

The surrounding environment recognition unit 23 includes a host vehicle driving trajectory generation unit 231, a surrounding vehicle trajectory generation unit 232, a trajectory load calculation unit 233, a surrounding vehicle load calculation unit 234, and a road load calculation unit 235. And a driver risk index management unit 236.

The vehicle driving trajectory generation unit 231 predicts (acquires or generates) the vehicle driving trajectory using vehicle speed information, steering angle information, acceleration / deceleration information, and yaw rate information included in the vehicle driving information from the internal sensor 21. Additionally, the own vehicle driving trajectory generation unit 231 corrects the own vehicle driving trajectory predicted through the navigation information included in the surrounding environment information from the external sensor 22, for example, driving road information.

The surrounding vehicle trajectory generation unit 232 predicts (acquires or generates) the surrounding vehicle driving trajectory using front / rear radar information, image information, and lateral ultrasound information included in the surrounding environment information from the external sensor 22.

The front / rear radar 221 for acquiring front / rear radar information may obtain accurate distance information (longitudinal) although the object discrimination accuracy is somewhat inferior.

On the other hand, since the cameras 222 and 224 for acquiring the image information acquire a monocular image, the accuracy of the distance information (longitudinal) decreases, but there is an advantage of obtaining high object identification and accurate lateral information.

Therefore, in order to obtain the surrounding vehicle trajectory, the longitudinal distance information in the target vehicle model formula is obtained using the front / rear radar 221, and the lateral distance information is the front / rear camera 222, the AVM camera, and the lateral ultrasound sensor 223. ).

Equation 1 below shows a vehicle model equation used by the surrounding vehicle trajectory generation unit 232 to predict the surrounding vehicle trajectory.

Here, x, Vx, y, Vy are the state variables of the target vehicle, x, y are the position of the target vehicle, and are measured from the video camera. Vx and Vy are target vehicle speeds. A is the vehicle model formula, H is the lateral tooth model model, and the state variables represent the distance, velocity, and y-axis distance and velocity in the x-axis direction. System noise and measured noise are assumed to be white Gaussian.

The trajectory load calculation unit 233 calculates a trajectory load amount W _Trj indicating a comparison result between a trajectory distance, which is a difference between a surrounding vehicle trajectory and a self-driving trajectory, and a preset threshold.

When there is a risk of collision by predicting the own vehicle trajectory and the surrounding vehicle trajectory, this may be expressed as a trajectory load (W _Trj ) because it requires high attention from the driver.

Therefore, the trajectory load amount W _Trj is calculated by using Equation 2 below using the host vehicle travel trajectory 13-1 and the surrounding vehicle trajectory 13-3.

Here, D _Trj is the own vehicle trajectory, T _Trj is the surrounding vehicle trajectory, and i is the detected surrounding vehicle 1,2, ... , n.

According to Equation 2 above, the trajectory load amount W _Trj is set to 1 when the trajectory distance is lower than the threshold value and 0 when the trajectory distance is compared by comparing the detected trajectories of the surrounding vehicle with the trajectory of the _host vehicle.

The surrounding vehicle load amount calculating unit 234 determines the number of vehicles in the front / rear / side and whether there is a lane change from the surrounding environment information, and calculates the surrounding vehicle load W _S through this. If there are vehicles around the host vehicle while driving, and the trajectories of these vehicles are severe, this can act as a load that requires attention from the driver.

The calculation of the surrounding vehicle load performed by the surrounding vehicle load calculation unit 234 is based on the time to collision (TTC), based on the host vehicle 24, as shown in FIG. 6, three danger areas (SECTION) ①, SECTION ② and SECTION ③). Here, the TTC is defined as the time spent until the target vehicle hits the target vehicle when the closing speed of the target vehicle is constant. The TTC may be obtained by using vehicle speed information and steering angle information obtained from the vehicle speed sensor 211 and the steering angle sensor 213 of the internal sensor 21, respectively.

For example, as shown in FIGS. 7 and 8, the detection areas 262 and 263 by the front / rear radar 221, the detection areas 261 by the front / rear camera 222, and the side ultrasound sensor 223 By detecting the surrounding vehicles 271,272,273,274 of the own vehicle 24 detected in the detection region 45 detected by each, and calculating the time divided by the relative speed value at the relative distance from the vehicle detected by the TTC value, the danger region (SECTION ①, ② and ③) are available.

When the three danger areas (SECTION ①, ②, and ③) are classified, the surrounding vehicle load calculation unit 234 determines the number of vehicles detected in each divided region and the presence or absence of lane change to determine the surrounding vehicle load Ws. Calculate.

The calculated surrounding vehicle load (Ws) increases as the number of vehicles detected in SECTION ① increases and the number of lane changes increases.

Conversely, if there is little or no vehicle detected in the vicinity, if the vehicle is in SECTION ③, and the trajectory of the surrounding vehicle is not severe, the surrounding vehicle load Ws decreases.

The surrounding vehicle load Ws may be represented by Equation 3 below.

Here, α and β are weighting factors, S is the position of the detected vehicle (SECTION ①, ② and ③), and L is the presence or absence of the lane change of the detected surrounding vehicle. Here, it is set to 1 when there is a change and 0 when there is no change. The detected vehicles are marked with 1 to n.

The road load calculation unit 235 calculates a road load using a road shape, a road surface state, and a traffic state included in the surrounding environment information. It is necessary to calculate these road loads because the curved roads rather than straight roads, the intersections rather than the regular roads, and the worse the traffic ahead, require the driver's attention.

The road load calculation unit 235 uses the navigation information obtained from the navigation 225 in the vehicle to grasp the shape of the road ahead to be driven, and reflects the road surface state information obtained from the front / rear cameras (paved road, The unpaved road) can be calculated by Equation 4 below.

Here, A is a value indicating the road condition. For example, A has a larger value as the curvature value of the road ahead increases, and has a large value in the case of traffic light changes, pedestrians, speed limit, and child protection zones. B reflects the pavement and unpaved as the road surface condition value, and C is the traffic volume of the road ahead, which has a larger value as the traffic volume increases. All of these A, B, and C can be normalized in the range of 0-5.

The driver risk index management unit 236 calculates the loads (W _Trj , W _S , W _R ) is provided to the driver with an integrated driver risk index (around risk index). For example, the driver risk index management unit 236 _{sums the} trajectory load W _Trj , the surrounding vehicle load W _S and the road load W _R , and generates the summed result value as the driver risk index.

The driver risk index can be expressed by Equation 5 below.

The driving difficulty detection unit 30 detects the driving difficulty according to the driving position of the host vehicle 24. That is, the driving difficulty detection unit 30 determines and determines whether the current location of the host vehicle 24 is included in the above-mentioned region by defining and digitizing the driving difficulty in each region in advance, and then responding to the current location of the host vehicle 24 Detects the driving difficulty.

Here, the current location of the host vehicle 24 is a region where an accident frequently occurs, or a driving difficulty is set relatively high in a point where it is relatively difficult to drive, such as a tunnel or a mountain road, or an actual accident or an occurrence.

The control unit 40 controls the vehicle driving unit 60 according to whether the emotion index, the driver's risk index, and the driving difficulty are included in the predetermined safety area, so that the host vehicle 24 travels to the safety area.

That is, the control unit 40 analyzes each of the emotion index, the driver's risk index, and the driving difficulty as shown in FIG. 9 to determine whether or not it is included in the preset safe area. The control unit 60 is controlled so that the host vehicle 24 can travel to a safe area. In addition, the control unit 40 warns that each of the emotional index, the driver's risk index, and the driving difficulty is not included in the safe area, or warns the user of the danger with a preset telephone number, for example, a telephone number of a driver or acquaintance.

In addition, the control unit 40 applies the voice index, the visual index and the behavior index to the learning-based emotion map, and if the emotion index is not included in the predetermined stable region, the emotion state control unit 50 is controlled to control the emotion of the driver or passenger. The emotional state of the driver or passenger is adjusted so that the state becomes a calm state.

The emotional state adjusting unit 50 adjusts the emotional state of the driver or the passenger so that the emotional state of the driver or the passenger becomes a steady state.

Referring to FIG. 10, the emotional state control unit 50 includes an image output unit 51, an audio output unit 52, an air conditioner 53, a fragrance output unit 54, a massager 55, and ambient light 56. And a windshield transmittance adjustment unit 57 may be included.

Here, the windshield transmittance adjustment unit 57 may control the transmittance of the control unit 40 according to the glare detected by the visual index detection unit 122.

That is, the control unit 40 inverts the coordinates of the windshield that passes when the light of the light source reaches the driver's eye based on the 3D coordinates of the light source and the driver's eye position detected by the visual index detector 122. By controlling the windshield transmittance adjusting unit 57 to lower the windshield transmittance around the windshield coordinates, it is possible to prevent glare from the driver.

For reference, in this embodiment, the emotional state control unit 50 is not limited to the above-described embodiment, and various devices capable of adjusting the driver's emotion may be included.

The vehicle driving unit 60 drives the vehicle to a safe area through an autonomous driving method.

Referring to FIG. 11, the vehicle driving unit 60 includes an engine control module 61 that controls the engine of the host vehicle 24, a braking control module 62 that controls braking of the host vehicle 24, and a host vehicle 24. A vehicle control module controlling the steering control module 63 for controlling steering and the engine control module 61, the braking control module 62, and the steering control module 63 to control the vehicle to move to a safe area ( 64).

Hereinafter, a method for supporting safe driving according to an embodiment of the present invention will be described in detail with reference to FIGS. 12 and 13.

12 is a flowchart of a method for supporting safe driving according to an embodiment of the present invention, and FIG. 13 is a flowchart illustrating a process of detecting a driver's risk index according to an embodiment of the present invention.

Referring to FIG. 12, first, the emotion index detecting unit 10 detects an emotion index of a driver or a passenger using at least one of a driver's or a passenger's time, voice, and behavior (S10).

In this case, the first emotion recognition unit 11 detects a characteristic value of a voice signal of a driver and a passenger and a voice index according to the detected characteristic value. The second emotion recognition unit 12 photographs the eyes or faces of the driver and passengers, detects the driver's gaze and face in the captured image, and then detects the movements, facial expressions, and glare of each of the detected drivers and passengers' gazes. It detects the time index. The third emotion recognition unit 13 detects an oscillation type according to a change in the position of an Excel pedal or a brake pedal sensed by the oscillation type detection unit 131, and then detects an oscillation index corresponding to the oscillation type, and applies a collision. After detecting the driver's kinetic operation index and the passenger motion index detected by the frequency detecting unit 134, the passenger's motion index detected by the driver 133, and then using the detected oscillation index, the friction operation index, and the passenger motion index Detects driver and passenger behavior index.

As described above, as the voice index, the visual index, and the behavior index are detected by the first emotion recognition unit 11, the second emotion recognition unit 12, and the third emotion recognition unit 13, the emotion index calculation unit (14) calculates the emotional index by applying these voice index, visual index and behavioral index to the learning-based emotion map.

In addition, the driver risk index detection unit 20 analyzes the driving propensity to detect the corresponding driver risk index (or surrounding risk index) (S20).

Referring to FIG. 13, the surrounding environment recognition unit 23 generates a driving trajectory for the host vehicle 24 (S210), and predicts the position of the host vehicle 24 through the generated vehicle driving trajectory (S220). Such a vehicle driving trajectory may be generated using vehicle speed information, steering angle information acceleration / deceleration information, and yaw rate information.

Subsequently, the surrounding environment recognition unit 23 generates a surrounding vehicle trajectory (S230) and predicts the location of the surrounding vehicle (S240). The surrounding vehicle trajectory may be generated using longitudinal distance information obtained from the radar 221 and transverse distance information obtained through a camera and an ultrasonic generator. Here, the longitudinal distance information is the longitudinal distance information of the surrounding vehicle with respect to the host vehicle 24, and the lateral distance information is the transverse distance information of the surrounding vehicle with respect to the host vehicle 24.

Subsequently, the surrounding environment recognition unit 23 calculates the trajectory load (S250). The trajectory load amount can be calculated using the own vehicle driving trajectory and the surrounding vehicle driving trajectory. For example, by comparing the detected driving trajectories of the surrounding vehicles with the own vehicle driving trajectories, a difference between them is 1 when the trajectory distance is lower than the threshold value and 1 when the trajectory distance is lower than the threshold value.

When the trajectory load amount is calculated, the surrounding environment recognition unit 23 calculates the surrounding vehicle load amount (S260). The surrounding vehicle load is calculated in consideration of the number of vehicles for each of the dangerous areas partitioned around the host vehicle 24 based on the TTC and the presence or absence of a lane change. A plurality of danger areas can be obtained by recognizing surrounding vehicles using a radar 221, a camera, and ultrasound, and calculating a time obtained by dividing a relative speed value from a relative distance from the vehicle detected by the TTC value. .

Subsequently, the surrounding environment recognition unit 23 calculates the road load (S270). Road loads may be calculated through navigation information, road surface condition information, traffic information, and the like.

Subsequently, the surrounding environment recognition unit 23 calculates a driver risk index (S280). The driving risk index can be calculated by adding up the trajectory load, the surrounding vehicle load, and the road load.

On the other hand, the driving difficulty detection unit 30 defines and quantifies the driving difficulty for each region in advance, confirms whether the current location of the host vehicle 24 is included in any of the above regions, and then responds to the current location of the host vehicle 24 The detected driving difficulty is detected (S30).

The current location of the host vehicle 24 is a region where an accident frequently occurs, or a driving difficulty is relatively high in a point where it is relatively difficult to drive, such as a tunnel or a mountain road, or an actual accident or an indication.

As described above, as the emotional index, the driver's risk index, and the driving difficulty are detected, the control unit 40 determines whether the emotional index, the driver's risk index, and the driving difficulty are included in a predetermined safety area (S40), and the determination result Depending on whether the emotion index, the driver's risk index, and the driving difficulty are included in the predetermined safety area, the driver or the family is warned, or the vehicle driving unit 60 is controlled to allow the host vehicle 24 to drive to the safe area (S50). .

In this case, the vehicle driving unit 60 allows the host vehicle 24 to travel to the safe area when the emotion index, the driver's risk index, and the driving difficulty are not included in the safe area, and the control unit 40 controls the emotion index, the driver It warns that each of the risk index and the difficulty of driving is not included in the safety zone, or warns of danger with a preset telephone number, for example, the telephone number of a driver or acquaintance.

On the other hand, the control unit 40 applies the voice index, the visual index, and the behavioral index to the learning-based emotion map, and if the emotion index is not included in the predetermined stable region, the emotion state control unit 50 is controlled to control the emotion of the driver or passenger. The emotional state of the driver or passenger may be adjusted so that the state becomes a calm state.

At this time, the control unit 40 may control the windshield transmittance adjustment unit 50 included in the emotional state control unit 50 according to the glare detected by the visual index detection unit 122.

That is, the control unit 40 inverts the coordinates of the windshield that passes when the light of the light source reaches the driver's eye based on the 3D coordinates of the light source and the driver's eye position detected by the visual index detector 122. By controlling the windshield transmittance adjustment unit 57 to lower the windshield transmittance around the windshield coordinates, it is possible to prevent glare from the driver.

As described above, the safe driving support apparatus and method according to an embodiment of the present invention comprehensively considers the driver's and passenger's emotion index, the driver's risk index, and the driving difficulty to determine whether the vehicle is currently driving and to determine whether the vehicle is safe. The vehicle is moved to a safe area through steering and braking control of the vehicle to prevent accidents.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those skilled in the art to which the technology pertains are capable of various modifications and other equivalent embodiments. Will understand. Therefore, the true technical protection scope of the present invention should be defined by the claims below.

10: emotion index detection unit 11: first emotion recognition unit
111: voice signal detection unit 112: voice index detection unit
12: second emotion recognition unit 121: driver face photographing unit
122: visual index detection unit 123: passenger face photographing unit
124: stereo camera 13: third emotion recognition unit
131: oscillation type detection unit 132: behavioral index detection unit
133: passenger motion index detection unit 134: crack frequency detection unit
14: emotional index calculation unit 20: driver risk index detection unit
21: internal sensor 211: vehicle speed sensor
212: yaw rate sensor 213: steering angle sensor
214: G sensor 215: Wheel speed sensor
22: external sensor 221: front / rear radar
222: front / rear camera 223: side ultrasonic sensor
224: AVM camera 23: surroundings recognition unit
231: own vehicle driving trajectory generating unit 232: surrounding vehicle trajectory generating unit
233: trajectory load calculation unit 234: surrounding vehicle load calculation unit
235: Road load calculation unit 236: Driver risk index management unit
30: driving difficulty detection unit 40: control unit
50: emotional state control unit 51: video output unit
52: audio output unit 53: air conditioner
54: fragrance output section 55: massager
56: ambient light 57: windshield transmittance adjustment unit
60: vehicle driving unit 61: engine control module
62: brake control module 63: steering control module
64: vehicle control module

Claims (20)

Emotion index that detects the driver's and passenger's emotion index using at least one of the driver's and passenger's gaze movement, facial expression and glare visual index, the voice signal's characteristics, and the movement's behavior index. Detection unit;
A driver risk index detecting unit for detecting a driver risk index using the own vehicle driving information and surrounding environment information of the host vehicle;
A driving difficulty detecting unit detecting a driving difficulty according to a driving position of the host vehicle;
A vehicle driving unit that controls autonomous driving of the host vehicle; And
Safe driving support device including a control unit to control the vehicle driving unit according to whether the emotion index, the driver's risk index and the driving difficulty is included in a predetermined safety area to drive the vehicle to a safe area.
The method of claim 1, wherein the emotion index detection unit,
A first emotion recognition unit that detects a characteristic value of a driver and a passenger's voice signal and detects a voice index according to the detected characteristic value;
A second emotion recognition unit that detects a driver and a passenger's visual index by photographing a driver and a passenger and analyzing an image taken around the driver;
A third emotion recognition unit detecting a position of the accelerator or brake pedal to detect an oscillation type of the accelerator pedal or brake pedal, and an action index according to the movement of a passenger; And
And an emotional index calculator configured to calculate the emotional index using the voice index, the visual index, and the action index.
The method of claim 2, wherein the first emotion recognition unit,
A voice signal detection unit for detecting a voice signal of a driver and a passenger; And
And a voice index detection unit detecting a characteristic value of the voice signal detected by the voice signal detection unit and detecting a voice index corresponding to the detected characteristic value.
The method of claim 2, wherein the second emotion recognition unit,
A driver's face photographing unit that photographs a driver's eye or face;
A passenger face photographing unit for photographing a passenger's eye or face;
A stereo camera photographing the surroundings of the host vehicle; And
The gaze and the face are detected based on the driver's eye position in the image photographed by the driver's face photographing unit, and the gaze and the face are detected based on the occupant's eye location in the image photographed by the passenger's face photographing unit, After detecting the position of the light source in the image photographed by the stereo camera, the visual index is detected according to the driver's and passenger's gaze movements, facial expressions, and the driver's glare by the position of the light source and the position of the eyes. Safe driving support device comprising a time index detection unit.
The method of claim 2, wherein the third emotion recognition unit,
An oscillation type detection unit that detects an oscillation type of an Excel pedal or a brake pedal;
A passenger motion index detector configured to detect a passenger motion index using at least one of a passenger's arm motion size, a motion radius, and a passenger's motion frequency;
A collision frequency detection unit that detects a collision operation index according to a driver's operation frequency; And
The oscillation index corresponding to the oscillation type of each of the excel pedal or brake pedal sensed by the oscillation type detection unit, and the ignition operation index detected by the crush frequency detection unit. And a behavior index detection unit configured to detect a behavior index of a driver and a passenger by using the occupant motion index detected by the occupant motion index detection unit.
The safe driving support apparatus according to claim 2, further comprising an emotional state control unit that adjusts the emotional state of the driver and the passenger so that the emotional state of the driver and the passenger becomes a steady state.
The method according to claim 6, wherein the control unit controls the emotion state control unit if the emotion index is not included in a predetermined stable region by applying the voice index, the visual index, and the behavior index to a learning-based emotion map. Features a safe driving support device.
The safe driving support apparatus according to claim 7, wherein the learning-based emotion map is divided into a stable area or a dangerous area according to the voice index, the visual index, and the behavior index.
The safe driving support apparatus according to claim 6, wherein the control unit controls a windshield transmittance adjustment unit included in the emotional state adjustment unit according to the visual index detected by the emotional index detection unit.
According to claim 1, wherein the driver risk index detection unit,
A vehicle driving trajectory generating unit generating a vehicle driving trajectory using the vehicle driving information received from an internal sensor that detects a driving situation of the vehicle;
A surrounding vehicle trajectory generation unit that generates a surrounding vehicle trajectory using ambient environment information input from an external sensor that senses the surrounding situation of the host vehicle;
A trajectory load calculation unit that calculates a trajectory load indicating a comparison result between a trajectory distance which is a difference between the surrounding vehicle trajectory and the own vehicle travel trajectory and a preset threshold; And
And a risk index management unit generating a driver risk index corresponding to the calculated trajectory load.
The safe driving support apparatus according to claim 1, wherein the driving difficulty detection unit detects the driving difficulty according to a current position of the host vehicle by defining and digitizing the driving difficulty for each zone.
The safe driving of claim 1, wherein the control unit controls the vehicle driving unit to drive the host vehicle to a safe area when the emotion index, the driver risk index, and the driving difficulty are not included in the safe area. Supported devices.
The emotional index detection unit uses the driver's and passenger's emotional index to determine the driver's and passenger's emotional index using at least one of the visual index according to the movement, facial expression and glare of each driver, the voice index according to the characteristics of the voice signal, and the behavioral index according to the movement. Detecting;
A driver risk index detecting unit detecting a driver risk index using the own vehicle driving information and the surrounding environment information;
A driving difficulty detecting unit detecting a driving difficulty according to a driving position of the host vehicle; And
Safe driving support comprising the step of controlling the vehicle driving unit according to the determination result by determining whether the control unit includes the emotion index, the driver's risk index, and the driving difficulty in a predetermined safety area, and thereby driving the vehicle to a safe area Way.
The method of claim 13, wherein the step of detecting the emotional index,
Detects the characteristic values of the driver's and passenger's voice signals, detects the voice index according to the detected characteristic values, analyzes the image of the driver and the passenger and the surroundings, and analyzes the movements, facial expressions of each driver's and passenger's gaze, And detects the visual index according to the glare of the driver by the position of the light source and the position of the eye, and detects the position of the accelerator or brake pedal to detect the oscillation type of the accelerator or brake pedal, the driver's frequency of operation and the movement of the passenger A method of supporting safe driving, after detecting the behavioral index, and calculating the emotional index using the voice index, the visual index, and the behavioral index.
14. The method of claim 13, wherein the control unit controls the emotional state control unit according to whether the emotional index is included in the predetermined stable region to adjust the emotional state of the driver and the passenger so that the emotional state of the driver and the passenger is steady. Safe driving support method further comprising a.
16. The method of claim 15, wherein the step of adjusting the emotional state of the driver and the passenger includes controlling the windshield transmittance adjusting unit included in the emotional state adjusting unit according to the visual index detected by the emotional index detecting unit. Safe driving support method, characterized in that.
The method of claim 13, wherein the step of detecting the driver's risk index,
Generating a own vehicle driving trajectory using the own vehicle driving information received from an internal sensor that detects the driving situation of the own vehicle;
Generating a surrounding vehicle trajectory of the own vehicle using the surrounding environment information received from an external sensor detecting the surrounding situation of the own vehicle;
Calculating a trajectory load amount indicating a comparison result between a trajectory distance which is a difference between the surrounding vehicle trajectory and the host vehicle trajectory and a preset threshold; And
And generating a driver risk index corresponding to the calculated trajectory load.
14. The method of claim 13, wherein detecting the driving difficulty comprises detecting and driving the driving difficulty according to the current location of the host vehicle by defining and digitizing the driving difficulty for each zone.
The method of claim 13, wherein the step of causing the host vehicle to travel to a safe area includes whether the emotion index is included in a predetermined stable area by applying the voice index, the visual index, and the behavior index to a learning-based emotion map. Safe driving support method, characterized in that to determine.
The method of claim 19, wherein the learning-based emotion map is characterized in that the driver's emotional state is divided into a stable area or a dangerous area according to each of the voice index, the visual index, and the action index.

Images