JP4948996B2 - Vehicle driving support device - Google Patents

Description

  The present invention relates to a vehicle driving support device capable of performing appropriate warning and automatic brake control in accordance with a driver's forward attention state.

  In recent years, various technologies have been proposed and put into practical use in vehicles so that the state of the driver and the driving environment can be detected and the vehicle control can be optimally executed without a sense of incongruity.

For example, in Japanese Patent Laid-Open No. 11-230322, the standard deviation of brightness and the average brightness value of the vehicle front image are detected as the driving environment of the vehicle, and the engine brake characteristics when the accelerator pedal is released are corrected according to these values. A technology of a speed change control device for a continuously variable transmission is disclosed.
JP-A-11-230322

  In general, drivers tend to be more ambiguous when driving in the suburbs than when driving in urban areas. It is necessary to call attention to the preceding vehicle etc. rather than the case where it is. However, as in the technique disclosed in Patent Document 1 described above, when the control is simply corrected using the average value or the standard deviation, the current traveling state and simple numerical values are used regardless of the characteristics of the image. Because it is only a comparison of data, while driving in an environment that is not a suburb, it is judged as a suburb from the average value and standard deviation value, and conversely, while driving in a suburb, the average value and standard deviation of There is a possibility that it is determined to be an urban area from the value, and there is a problem that it is impossible to perform a natural and appropriate control of a warning to the driver.

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a driving support device for a vehicle that can accurately detect an environment in which a driver travels and can perform natural and optimal alarms and automatic brake control. Yes.

The present invention provides image information acquisition means for acquiring image information ahead of a vehicle, driver state determination means for determining one state of a driver's sloppy state and over-attention state, and the above one determined by the driver state determination means. Reference image setting means for setting image information in front of the vehicle in a state as a reference image, correlation value calculation means for calculating a normalized correlation between the reference image and the current vehicle forward image, and the result of the normalized correlation in advance Compared with a set threshold value, determines one state of the driver's sloppy state or over-attention state, determines the environment in which the driver is traveling in the determined driver state, and results of determination of the traveling environment And at least one of a brake means and a notification means, and a variable control means for operating the variable means.

  According to the vehicle driving support apparatus of the present invention, it is possible to accurately detect the environment in which the driver travels and to perform natural and optimal alarms and automatic brake control.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1 to 8 show an embodiment of the present invention, FIG. 1 is a schematic configuration diagram of a driving support device mounted on a vehicle, FIG. 2 is a driving support program flowchart, FIG. 3 is a reference image setting program flowchart, and FIG. 4 is an explanatory diagram of the dispersion value of the gaze behavior in the forward field of view and the width of the preceding vehicle, FIG. 5 is an explanatory diagram of examples of various attention evaluation values, and FIG. FIG. 7 is an explanatory diagram of the braking start distance in the normal time and the casual time, and FIG. 8 is an explanatory diagram showing an example of an image area for calculating the normalized correlation.

  In FIG. 1, reference numeral 1 denotes a vehicle such as an automobile (host vehicle). The host vehicle 1 includes a driving support device 2 having a contact warning for a preceding vehicle and an automatic brake control function for avoiding a collision. It is installed.

  The driving support device 2 includes a stereo camera 3 that captures the front outside the vehicle, a stereo image recognition device 4 that processes a signal from the stereo camera 3, a visual field camera 5 that captures the driver's eye movement, an infrared lamp 6, a visual field camera 5, and an infrared ray. A gaze state detection device 7 that detects the gaze state of the driver using the lamp 6, a control unit 8, a monitor 9 (notification means) that displays an alarm, a sound generation device 10 (notification means) that issues an alarm, and executes automatic brake control A throttle valve control device 12 (brake means) that controls the throttle valve 11 and an automatic brake control device 13 (brake means) are mainly configured.

  In addition, the host vehicle 1 is provided with a vehicle speed sensor 21 that detects a vehicle speed, a handle angle sensor 22 that detects a handle angle, and a yaw rate sensor 23 that detects a yaw rate. The vehicle speed from the vehicle speed sensor 21 is input to the stereo image recognition device 4 and the control unit 8, and the steering wheel angle from the steering wheel angle sensor 22 and the yaw rate from the yaw rate sensor 23 are input to the stereo image recognition device 4.

  The stereo camera 3 is composed of a set of (left and right) CCD cameras using a solid-state imaging device such as a charge coupled device (CCD) as a stereo optical system. These left and right CCD cameras are each mounted at a predetermined interval in front of the ceiling in the passenger compartment, take a stereo image of an object (a three-dimensional object) outside the vehicle from different viewpoints, and output image data to the stereo image recognition device 4.

  The stereo image recognition device 4 receives image data, vehicle speed, steering wheel angle, and yaw rate signals from the stereo camera 3, and based on the image data, forward information such as three-dimensional object data, sidewall data, and white line data in front of the host vehicle 1. And the traveling path of the host vehicle 1 (the host vehicle traveling path) is estimated from the forward information and the driving state of the host vehicle 1. Then, a travel region is set based on the traveling path of the host vehicle, and a preceding vehicle ahead of the host vehicle 1 is identified and extracted according to the presence state of the three-dimensional object with respect to the travel region, and the result is output to the control unit 8. To do.

  The above-described estimation of the own vehicle traveling path is performed as follows, for example. At this time, the three-dimensional coordinate system in the real space is a coordinate system fixed to the own vehicle 1, the left and right (width) direction of the own vehicle 1 is the X coordinate, the up and down direction of the own vehicle 1 is the Y coordinate, and the front and rear of the own vehicle 1. The direction is indicated by the Z coordinate. Then, with the road surface directly below the center of the two CCD cameras constituting the stereo camera 3 as the origin, the right side of the host vehicle 1 is the + side of the X axis, the upper side of the host vehicle 1 is the + side of the Y axis, and the host vehicle 1 Is set as the positive side of the Z axis.

  a. The own vehicle traveling path estimation based on the white line ... If the white line data on both the left and right sides or the left and right sides is obtained, and the shape of the lane in which the vehicle 1 is traveling can be estimated from these white line data, the own vehicle travels The road is formed in parallel with the white line in consideration of the width of the host vehicle 1 and the position of the host vehicle 1 in the current lane.

  b. Self-vehicle travel path estimation based on side data of guardrails, curbs, etc. Side wall data on both the left and right sides or left and right sides is obtained, and the shape of the lane in which the vehicle 1 is traveling is estimated from these side wall data If possible, the own vehicle traveling path is formed in parallel with the side wall in consideration of the width of the own vehicle 1 and the position of the own vehicle 1 in the current lane.

  c. Estimating own vehicle traveling path based on preceding vehicle trajectory: Estimating the own vehicle traveling path based on the past traveling trajectory of the preceding vehicle extracted from the three-dimensional object data.

  d. Estimating own vehicle travel path based on travel path of own vehicle 1 ... Estimating the own vehicle travel path based on the driving state of the own vehicle 1. For example, assuming that the yaw rate is γ, the host vehicle speed is Vo, and the steering wheel angle is θH, the host vehicle traveling path is estimated by the following procedure.

First, it is determined whether the yaw rate sensor 23 is valid. If the yaw rate sensor 23 is valid, the current turning curvature Cua is calculated by the following equation (1).
Cua = γ / Vo (1)

On the other hand, if the yaw rate sensor 23 is invalid, it is determined whether or not the steering angle δ calculated from the steering wheel angle θH is a predetermined value (for example, 0.57 degrees) or more, and the steering angle δ is 0. When the steering is performed at 57 ° or more, the current turning curvature Cua is calculated using the steering angle δ and the host vehicle speed Vo, for example, by the following equations (2) and (3).
Re = (1 + A · V ² ) · (L / δ) (2)
Cua = 1 / Re (3)
Here, Re is a turning radius, A is a vehicle stability factor, and L is a wheelbase.

  When the steering angle δ is smaller than 0.57 degrees, the current turning curvature Cua is set to 0 (straight running state).

  In this way, the average turning curvature is calculated from the turning curvature of the past predetermined time (for example, about 0.3 seconds) to which the obtained current turning curvature Cua is added, and the own vehicle traveling path is estimated.

  Even when the yaw rate sensor 23 is valid and the current turning curvature Cua is calculated by the above-described equation (1), if the steering angle δ is smaller than 0.57 degrees, the current turning curvature is Cua may be corrected to 0 (straight running state).

  For example, a width of about 1.1 m on the left and right sides is set as the travel region of the host vehicle, with the host vehicle traveling path estimated as described above approximately as the center.

  The processing of the image data from the stereo camera 3 in the stereo image recognition device 4 is performed as follows, for example. First, a process for obtaining distance information based on the principle of triangulation is performed on the stereo image pair in front of the host vehicle 1 captured by the CCD camera of the stereo camera 3 to express a three-dimensional distance distribution. Generate a distance image. Then, based on this data, a well-known grouping process is performed and compared with frames (windows) such as three-dimensional road shape data, side wall data, and three-dimensional object data stored in advance. Sidewall data such as guardrails and curbs, and three-dimensional object data such as vehicles are extracted.

  The white line data, the side wall data, and the three-dimensional object data extracted in this way are assigned different numbers for each data. Further, regarding the three-dimensional object data, the stopped object and the order of moving in the same direction as the own vehicle 1 are determined from the relationship between the relative change in the distance from the own vehicle 1 and the vehicle speed of the own vehicle 1. It is classified and output as a moving object. Then, for example, among the forward moving objects protruding into the own vehicle traveling area, the three-dimensional object closest to the own vehicle 1 is registered as a preceding vehicle, which is detected continuously for a predetermined time.

  Further, the stereo image recognition device 4 obtains the density of the image captured by the stereo camera 3 (in this embodiment, the entire image area is targeted) by calculation processing, and outputs it to the control unit 8.

  Thus, in the present embodiment, the stereo camera 3 and the stereo image recognition device 4 constitute image information acquisition means.

  On the other hand, the gaze behavior of the driver in the present embodiment is detected by a so-called pupil / corneal reflection method. Therefore, the visual field camera 5 is a camera equipped with an infrared CCD, and the infrared lamp 6 is an LED lamp. The line-of-sight state detection device 7 detects the center of the pupil while simultaneously detecting the center of the pupil by the visual camera 5 while the virtual image by the infrared lamp 6 on the cornea is translated by the eye movement due to the difference between the rotation centers of the cornea and the eyeball. Detecting eye movements as a reference makes it possible to detect gaze behavior. Note that the detection of the gaze behavior is not limited to this detection method, and if possible, other detection methods (EOG (Electro-Oculography) method, scleral reflection method, corneal reflection method, search coil method, etc.) It may be detected. The visual field camera 5, the infrared lamp 6, and the line-of-sight state detection device 7 constitute a driver state determination unit.

  From the stereo image recognition device 4, the control unit 8 determines that the own vehicle traveling path, the travel region, the preceding vehicle information, the three-dimensional object information other than the preceding vehicle, and the density of the image captured by the stereo camera 3 are from the gaze state detection device 7. The driver's line-of-sight behavior signal (unit: angle) is input from the vehicle speed sensor 21 as the vehicle speed.

  At this time, as shown in FIG. 4, the width information of the preceding vehicle from the stereo image recognition device 4 is given in length units (W in FIG. 4), and the driver's line-of-sight behavior is given in angle units. In order to enable calculation, as shown in FIG. 5, the width W of the preceding vehicle is converted into a value α in angular units.

This conversion formula is based on the following formula (4).
α = 2 · arctan ((W / 2) / L) (4)
Here, L is the inter-vehicle distance.

Also, from the driver's line-of-sight behavior signal input to the control unit 8, a dispersion value β is calculated by the following equation (5) as a value indicating the variation in the line-of-sight behavior with respect to the preceding vehicle. That is, the point of gaze on the virtual plane is calculated based on the rotation angle of the eyeball. The horizontal component of the gazing point on the virtual plane is set to xi, a certain time span [t1, t2] (for example, 30 to 60 seconds) is set, and the horizontal dispersion value β of the gazing point in the meantime is
β = (1 / (t2−t1 + 1)) · Σ _{j = t1} ^t2 (xj ² −xa ² ) (5)
Here, xa is an average value and is obtained by the following equation (6).

xa = (1 / (t2−t1 + 1)) · Σ _{j = t1} ^t2 xj (6)

Note that the standard deviation sx may be used as a value indicating the variation in the line-of-sight behavior with respect to the preceding vehicle.
sx = ((1 / n) · Σ _{j = t1} ^t2 (xj ² −xa ² )) ^1/2 (7)

  Then, the ratio of the width α of the preceding vehicle to the variance value β of the driver's line-of-sight behavior is calculated as the attention evaluation value Sh representing the attention state (Sh = α / β), and this attention evaluation value Sh is calculated in advance. If it is smaller than the set evaluation threshold Shc (for example, 0.1) (for example, in the case of β2 in FIG. 5), the driver's attention state for the preceding vehicle is low, and the driver determines that he is in a loose state. . Then, the density of the image captured by the stereo camera 3 in this state is set as reference image data. Note that the state of β1 in FIG. 5 indicates a state in which the driver is overly careful.

In the control unit 8, the normalized correlation (correlation value) Kco between the density of the image (input image) captured by the stereo camera 3 and the density of the above-described reference image is set according to the driving support program described later, for example, (8 ) Calculate by the formula.
Kco = (A / (B · C) ^1/2 ) · 10000 (8)

Here, A is a cross-correlation between the input image and the reference image, and is calculated by the following equation (9).
A = N · Σ (I · T) −Σ (I) · Σ (T) (9)
Note that N is the area of the reference image (the entire image area in this embodiment), I is the density of the input image, and T is the density of the reference image.

B is the autocorrelation of the input image and is calculated by the following equation (10).
B = N · Σ (I) ² − (Σ (I)) ² (10)
C is the autocorrelation of the reference image and is calculated by the following equation (11).

C = N · Σ (T) ² − (Σ (T)) ² (11)

  When the correlation value Kco based on the above-described normalized correlation is larger than the preset value φ, the correlation between the input image and the reference image is high, and the driver is in a loose state and traveling in the suburbs. It judges that the possibility is high, and outputs an alarm control change command and an automatic brake control change command.

  Specifically, the change of the alarm control when the driver is in an ambiguous state is executed by increasing the frequency of the alarm control more than usual. For example, as shown in FIG. 6, the normal alarm flag is set at an interval as shown in FIG. 6 (a), but in a casual time, it is set at a narrow interval as shown in FIG. 6 (b). An alarm flag is set, and an alarm is issued from the monitor 9 and the sound generator 10. The change of the alarm control is continued for a certain period of time while the change command is issued or after the change command is issued.

  In addition, the change of automatic brake control for collision avoidance when the driver is in a sloppy state is specifically by changing the inter-vehicle distance with the preceding vehicle so that the brake control operates at a longer distance than usual. Done. For example, as shown in FIG. 7, if the distance at which the automatic brake is operated during normal times is Lal (distance according to the vehicle speed), the throttle valve control device 12 can The start timing (for example, the deceleration start timing by constant deceleration) of the automatic brake control that is output and executed to the brake control device 13 is performed earlier. The change in the automatic brake control is continued for a certain period of time while the change command is issued or after the change command is issued.

  As described above, the control unit 8 is configured to have functions as a driver state determination unit, a reference image setting unit, a correlation value calculation unit, and a variable control unit.

Next, the driving support program executed by the control unit 8 will be described with reference to the flowchart of FIG.
First, in step (hereinafter abbreviated as “S”) 101, reference image data is read. The reference image setting program will be described later with reference to the flowchart of FIG.

  In step S102, the current image data (input image data) is read.

  Next, the process proceeds to S103, and the correlation value Kco based on the normalized correlation is calculated according to the above equation (8).

  In S104, the correlation value Kco based on the normalized correlation is compared with a preset value φ. If the correlation value Kco is equal to or less than the set value φ (Kco ≦ φ), the input image and the reference image are compared. The correlation is low and it is determined that the possibility of traveling in the city is high, and the program is exited as it is (normal alarm control, automatic brake control).

  Conversely, if the correlation value Kco is greater than the set value φ (Kco> φ), it is determined that the correlation between the input image and the reference image is high and there is a high possibility that the vehicle is traveling in the suburbs. A change command is output so that the frequency of control is higher than usual. This change in alarm control is continued for a certain period of time while it is determined that the correlation between the input image and the reference image is high, or after the correlation between the input image and the reference image is determined to be high.

  Next, in S106, a change command is output so that the start of the automatic brake control is advanced, and the program is exited. This change in the automatic brake control is continued for a certain period of time while it is determined that the correlation between the input image and the reference image is high or after the correlation between the input image and the reference image is determined to be high.

Next, the reference image setting program will be described with reference to the flowchart of FIG.
First, necessary parameters are read in S201, the process proceeds to S202, the preceding vehicle is extracted by the stereo camera 3 and the stereo image recognition device 4, and the process proceeds to S203. Is converted into an angle α by the above-described equation (4).

  Thereafter, the process proceeds to S204, and the control unit 8 calculates the average value of the driver's line-of-sight behavior and the variance value β from the average value by the above-described expressions (5) and (6), and the process proceeds to S205. Then, the ratio of the width α of the preceding vehicle to the variance value β of the gaze behavior of the driver is calculated as the attention evaluation value Sh representing the attention state (Sh = α / β).

  Then, the process proceeds to S206, where the attention evaluation value Sh is compared with a preset evaluation threshold Shc, and when it is determined that the attention evaluation value Sh is smaller than the evaluation threshold Shc (Sh <Shc), S207 Then, it is determined that the attention state of driving with respect to the preceding vehicle is not strong (random state), the input image data is set as the reference image data, and the program is exited. Conversely, if it is determined that the attention evaluation value Sh is equal to or greater than the evaluation threshold value Shc (Sh ≧ Shc), it is determined that the attention state of driving with respect to the preceding vehicle is strong, and the program is directly exited.

  As described above, according to the embodiment of the present invention, the input image when it is determined to be in a random state based on the driver's gaze behavior in advance is set as the reference image, and the normalized correlation between the reference image and the input image is calculated. However, if the driver's muzzle state is detected from the correlation value of the normalized correlation, and it is determined that it is a mute state, the alarm frequency is increased and the start timing of the automatic brake control is changed to be advanced. Yes. For this reason, since it is possible to change the alarm control and the automatic brake control with high accuracy based on the front environment where the vehicle is traveling, it is possible to perform a natural and optimum alarm and automatic brake control.

  In the embodiment of the present invention, the normalization correlation is calculated for the entire image area. However, for example, as illustrated in FIG. FIG. 8A shows an example in which an image is vertically divided into three, and regions P and Q at both right and left ends thereof are subjected to normalized correlation. FIG. 8B shows an example in which the image is vertically divided into three, and regions R and S at the upper end portions of the left and right end regions are subjected to normalization correlation. In this way, the calculation amount of the normalized correlation can be reduced by setting a partial target region. In addition, since the normalized correlation can be calculated by canceling the data in the central portion that is easily affected by traffic jams, it is possible to improve the determination accuracy.

  Further, in the embodiment of the present invention, the input image when it is determined in advance as a random state based on the gaze behavior of the driver is set as the reference image, but conversely based on the gaze behavior of the driver. If the input image when it is determined that the state of excessive attention is set as the reference image, the normalized correlation between the reference image and the input image in the excessive attention state is calculated. Further, it is possible to change so as to perform the alarm control and the automatic brake control opposite to those of the above-described embodiment based on the correlation with the reference image in the excessive attention state.

  Furthermore, in the embodiment of the present invention, the reference image based on the driver's gaze behavior is set based on the driver's gaze behavior. However, the driver's handle operation or the like is detected, and the input image at that time is used as the reference image. It is good as an image.

  In the embodiment of the present invention, it is instructed to change both the alarm control and the automatic brake control. However, either one of the controls may be changed. Further, the alarm control is changed by changing the frequency, but it goes without saying that other changes such as changing the volume may be used.

Schematic configuration diagram of a driving support device mounted on a vehicle Driving assistance program flowchart Reference image setting program flowchart Explanatory diagram of variance of gaze behavior in front view and width of preceding vehicle Illustration of examples of various attention evaluation values Illustration of alarm control during normal and casual times Illustration of braking start distance in normal and casual times Explanatory drawing which shows an example of the image area | region which calculates a normalization correlation

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Own vehicle 2 Driving support device 3 Stereo camera (image information acquisition means)
4 Stereo image recognition device (image information acquisition means)
5 Field-of-view camera (driver status determination means)
6 Infrared lamp (driver status judgment means)
7. Gaze state detection device (driver state determination means)
8 Control unit (driver state determination means, reference image setting means, correlation value calculation means, variable control means)
9 Monitor (notification means)
10 Voice generator (notification means)
12 Throttle valve control device (brake means)
13 Automatic brake control device (brake means)

Claims (5)

Image information acquisition means for acquiring image information ahead of the vehicle;
Driver state determination means for determining one state of a driver's ill-mannered state and an over-attention state;
Reference image setting means for setting image information in front of the vehicle in the one state determined by the driver state determination means as a reference image;
Correlation value calculating means for calculating a normalized correlation between the reference image and the current vehicle forward image;
The result of the normalized correlation is compared with a preset threshold value to determine one state of the driver's sloppy state or over-attention state, and the driving environment of the driver is determined in the determined driver state And variable control means for operating by varying at least one of the brake means and the notification means according to the determination result of the traveling environment ,
A vehicle driving support apparatus comprising:   The correlation value calculating means calculates the normalized correlation based on the autocorrelation of the reference image, the autocorrelation of the current vehicle forward image, and the cross correlation of the reference image and the current vehicle forward image. The vehicle driving support device according to claim 1.   The vehicle driving support device according to claim 1, wherein the driver state determination unit determines one of the loose state and the over-attention state based on a gaze behavior of the driver.   4. The correlation value calculating means according to claim 1, wherein the correlation value calculating means calculates the normalized correlation with respect to image regions on both left and right end sides of the reference image and the current vehicle front image. The vehicle driving support apparatus according to one of the above.   The correlation value calculating means calculates a normalized correlation between the reference image in the driver's casual state set by the reference image setting means and the current vehicle forward image,
  When the value of the normalized correlation is larger than a preset threshold value, the variable control means determines that the driver is in a loose state and has a high possibility of traveling in the suburbs, and the brake means 5. The method according to claim 1, wherein at least one of changing the brake start timing in a direction earlier than normal and changing in a direction in which the alarm frequency by the notification means is higher than normal is performed. The vehicle driving support apparatus according to one of the above.

Images