JP4730191B2 - VEHICLE DRIVE ASSISTANCE DEVICE AND WALKING STATE DETECTION DEVICE - Google Patents

Description

  The present invention relates to a vehicle driving support device that detects a driver's looking-aside state and performs predetermined driving support control, and a side-viewing state detection device that detects a driver's looking-aside state.

Conventionally, as this type of device, a CCD camera detects a driver's side-by-side or dozing operation, and when a side-by-side or dozing operation is detected exceeding the allowable time determined from the time between the vehicle and the preceding vehicle, an alarm etc. What is performed is known (for example, refer to Patent Document 1). In this apparatus, a warning is given at an early timing during high-speed driving, or an alarm is given at an early timing in consideration of the possibility of interruption or the like even during low-speed driving by a characteristic method of setting an allowable time. In addition, the allowable time is corrected in consideration of vehicle behavior such as vehicle speed and acceleration, and vehicle surroundings such as traffic congestion and weather.
JP 2002-2129968 A

  By the way, it is considered that the inconvenience that the vehicle fluctuates from side to side due to looking aside depends on the state in which the driver holds the steering wheel. However, in the conventional apparatus, since such consideration is not made, there is a case where an alarm cannot be performed at an appropriate timing. In other words, the driver is looking aside in the so-called one-handed handle state or in the state of letting go and the alarm timing is delayed despite the high urgency of the alarm, and conversely, the steering wheel is held securely with both hands. In spite of this, there are cases where the driver feels bothered by giving an excessive warning.

  Further, the method for suppressing the inconvenience due to looking aside is not limited to the one that gives an alarm like the above-described conventional device, and other effective control methods can be considered.

  The present invention is intended to solve such a problem, and is a vehicle driving support device capable of appropriately detecting a driver's looking aside and appropriately performing predetermined control for suppressing inconvenience due to the looking aside. The main purpose is to provide

  In order to achieve the above object, a first aspect of the present invention includes: a look-a-side detection unit that detects a driver's look-aside state; and a holding state detection unit that detects a holding state of the steering wheel by the driver. When the driver's side-by-side state is detected by the side-by-side state detection means and the holding state detection unit detects that the driver is not holding the steering wheel with both hands, predetermined driving support control is performed. This is a vehicle driving support device.

  According to the first aspect of the present invention, the state of looking aside is detected, and the driver is not in a state of holding the steering wheel with both hands (that is, the state of a so-called one-handed handle or the state of letting go). Is detected, predetermined driving support control is performed to suppress inconvenience due to looking aside. Here, for example, the predetermined driving support control may be an alarm at an early timing as compared to an alarm that is issued when only a side-by-side condition is detected, or may be other control. The inconvenience that the vehicle fluctuates from side to side due to looking aside is considered to depend on the state of the steering wheel held by the driver. Therefore, such control appropriately performs the predetermined driving support control for suppressing the inconvenience due to looking aside. Can be done.

  Further, the first aspect of the present invention includes alarm means for performing an alarm operation when, for example, the driver's looking-aside state is detected by the driver's looking-aside state detecting means, and the driver's looking-aside state is detected by the looking-ahead state detecting means. And when the holding state detecting means detects that the driver is not holding the steering wheel with both hands, the driver's side-by-side state is detected by the side-by-side state detecting means and the driver is driven by the holding state detecting means. Compared to the case where it is detected that the person is holding the steering wheel with both hands, the warning means is controlled so as to issue a warning at an earlier timing. In this case, when the driver's side-by-side state is detected by the side-by-side state detection means and the driver's holding state detection unit detects that the driver is not holding the steering wheel with any hand, Compared to the case where the state detection means detects that the driver is looking aside and the holding state detection means detects that the driver is holding the steering wheel with one hand, the warning is issued at an earlier timing. The warning means may be controlled as described above.

  Further, in the first aspect of the present invention, for example, the vehicle includes lane keeping control means for outputting a steering force so that the host vehicle travels while keeping the lane, and the predetermined driving support control includes the lane keeping control means. It is the control to operate.

  In the first aspect of the present invention, it is preferable that imaging means for imaging the driver's head is provided, and the look-ahead state detection means calculates the feature amount in the vertical direction of the driver's face from the captured image of the imaging means. It is a means for detecting and detecting the driver's aside state based on the change in the feature amount. In this case, more preferably, the looking-aside state detecting unit is a unit that learns in advance a reference amount serving as a reference for detecting the looking-aside state with respect to the feature amount.

  In addition, the first aspect of the present invention preferably includes a steering steering angle detection means for detecting a steering steering angle, and when the steering steering angle detected by the steering steering angle detection means is outside a predetermined range, The predetermined driving support control is stopped.

  A second aspect of the present invention is a side-by-side state detection device that detects a driver's side-by-side state, an imaging unit that images the driver's head, and a vertical direction on the driver's face from an image captured by the imaging unit. A feature amount detecting means for detecting the feature amount of the image and a reference amount learning means for learning a reference amount that serves as a reference for detecting the aside state with respect to the feature amount detected by the feature amount detecting means. A side-by-side state detection device that detects a side-by-side state of a driver in a vertical direction by comparing a detected feature amount with a reference amount learned by a reference amount learning unit.

  ADVANTAGE OF THE INVENTION According to this invention, the driving assistance device for vehicles which can detect appropriately a driver | operator's looking aside and can perform predetermined driving assistance control for suppressing the trouble by a looking aside can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

  Hereinafter, an embodiment of a vehicle driving support apparatus according to the present invention will be described with reference to FIGS. FIG. 1 is a diagram illustrating an example of the overall configuration of a vehicle driving support apparatus 1 according to an embodiment of the present invention. The vehicle driving support device 1 includes, as main components, a vehicle interior camera 10, a steering touch sensor 20, an alarm device 30, a lane keeping control system 40, and an electronic control unit for driving support device (hereinafter referred to as driving support). 60) (referred to as device ECU). Communication between devices in the vehicle in the following description is performed using an appropriate communication protocol such as CAN (Controller Area Network).

  As shown in FIG. 2, the vehicle interior camera 10 is, for example, a CCD camera or a CMOS camera disposed near the combination meter 12 in front of the driver's seat, and the driver's head 16 passes through a hole 14A inside the steering wheel 14. Image. The hole portion 14A occupies a substantially upper half inside the steering wheel 14 when viewed from the driver in a state where the steering angle is at a reference angle (or neutral angle; the steering angle when the vehicle goes straight). Is. In the vehicle interior camera 10, the angle of view, the focus, the arrangement position, and the like are set so that the driver's head 16 fits in the hole 14 </ b> A in the captured image when the steering angle is the reference angle. The captured image of the vehicle interior camera 10 is transmitted to the driving support device ECU 60.

  The steering touch sensor 20 detects, for example, temperature change, vibration (due to heartbeat), pressure change, voltage change, and the like at a plurality of locations on the steering wheel 14 so that the steering wheel 14 can be touched by the driver's hand at several locations. It is a sensor for detecting whether it is (that is, hold | maintained). If it is detected by the sensor that the steering wheel 14 is not touching the driver's hand at all, it can be determined that the steering wheel 14 is in a so-called hand-off state, and the steering wheel 14 touches the driver's hand at one place. If it is detected that the steering wheel 14 is touching the driver's hand at two locations, the driver can hold both hands. Thus, it can be determined that the steering wheel 14 is being held. The output data of the steering touch sensor 20 is transmitted to the driving support device ECU 60.

  For example, the warning device 30 shares a route guidance speaker provided in the vehicle navigation device and issues a warning by voice. Further, the alarm is not limited to the one that emits an alarm by voice, but the alarm may be performed by causing a light to be emitted by using a display device such as a head-up display, or by attaching a vibrator to the driving seat or the steering wheel 14 by vibration. An alarm may be given. Further, a buzzer 48 described later may be shared. The alarm device 30 performs the alarm operation as described above in accordance with an instruction signal from the driving support device ECU 60.

  The lane keeping control system 40 is a control system that performs driving support control for the host vehicle to travel while maintaining the traveling lane. As shown in FIG. 3, the lane keeping control system 40 includes a lane recognition camera 42, a lane keeping control switch 44, a lane keeping control electronic control unit (hereinafter referred to as a lane keeping control ECU) 46, and a buzzer 48. And the steering device 50.

  The lane recognition camera 42 is, for example, a CCD camera or a CMOS camera disposed at the upper center of the windshield, and has an optical axis directed obliquely downward in front of the vehicle and images a road in front of the vehicle. The captured image of the lane recognition camera 42 is transmitted to the lane keeping control ECU 46. The lane keeping control switch 44 is, for example, a switch that is disposed on the side of the steering wheel 14 of the instrument panel and can switch the lane keeping control system 40 on / off.

  The lane keeping control ECU 46 is a computer unit in which, for example, a CPU, a ROM, a RAM, and the like are connected to each other via a bus (not shown), and further includes a communication port, a timer, a counter, and the like. The lane keeping control ECU 46 executes lane departure warning control and lane keeping support control when the lane keeping control switch 44 is on. Note that switching between these two controls is not described because it does not form the core of the present invention.

  What is necessary for performing the above control is the positional relationship between the traveling lane and the host vehicle. In order to obtain the positional relationship, the lane keeping control ECU 46 first recognizes road lane markings in the captured image of the lane recognition camera 42. This recognition is performed, for example, by performing binarization processing or feature point extraction processing on the captured image of the lane recognition camera 42, thereby obtaining pixels (road lane line candidate points) that are considered to correspond to road lane lines on the road. Of the selected road lane marking candidate points are recognized as road lane markings. In addition to this, it is possible to recognize the road lane markings by various methods, and the road shoulder and the center separation band may be detected by a method such as pattern matching. Then, based on the position occupied by the recognized road lane marking in the image, the displacement of the center of the host vehicle (arbitrary position on the center line in the vehicle width direction) from the center of the traveling lane, and the extending direction of the traveling lane The angle of the traveling direction of the host vehicle is derived with reference to.

  The lane departure warning control is based on the positional relationship obtained as described above, and when the vehicle is expected to depart from the driving lane after a predetermined time (for example, 0 comma number [sec] to several [sec]). Is a control for alerting the driver by instructing the steering device 50 and outputting a small auxiliary steering force for a short time. The lane keeping support function is a control that assists the steering operation of the driver so as to travel near the center of the traveling lane by instructing the steering device 50 to continuously output a small auxiliary steering force. By these controls, it is possible to suppress deviation of the host vehicle from the traveling lane.

  The steering device 50 is, for example, an electric power steering device that includes a steering angle sensor 52, a torque sensor 54, an assist motor 56, and a controller 58.

  For example, the steering angle sensor 52 is disposed inside the steering column and transmits a steering angle signal to the controller 58. The signal is also transmitted to the driving support device ECU 60. The torque sensor 54 is disposed inside the steering column, for example, and transmits a signal corresponding to the steering torque to the controller 58 by detecting torsion of a torsion bar attached between the input shaft and the output shaft. The assist motor 56 is, for example, a DC motor disposed in the column housing portion, and assists the driver's steering operation by outputting torque necessary for steering the vehicle. The torque output from the assist motor 56 is deflected by the worm gear and the wheel gear, decelerated and transmitted to the column shaft, and finally changes the direction of the wheel.

  The controller 58 is necessary for steering the vehicle based on the steering torque signal from the torque sensor 54 and other vehicle state signals (vehicle speed, yaw rate, etc.) during normal times when lane departure warning control and lane keeping support control are not performed. A control signal is output to the drive circuit of the assist motor 56 so as to output the steering force. Further, when the lane departure warning control or the lane keeping support control is performed, the auxiliary steering force based on the instruction signal from the lane keeping control ECU 46 in addition to (or instead of) the normal steering force output. , The assist motor 56 is controlled.

  The driving support device ECU 60 is, for example, a computer unit having the same hardware configuration as the lane keeping control ECU 46. The driving support device ECU 60 analyzes the captured image of the in-vehicle camera 10 to determine the driver's looking aside, and performs predetermined driving support control based on the determination. This looking-aside state is determined based on the face direction in the left-right direction, the direction of the line of sight, and the face direction in the up-down direction. That is, (1) a state in which the driver's face direction or line-of-sight direction is estimated to be tilted by a predetermined angle or more with respect to the front side as a reference, and (2) the driver's face direction faces the front side. The driver is in a state of looking aside when any one of the states estimated to have tilted upwards or downwards by a predetermined angle (which may be different from the predetermined angle of (1)) continues for a predetermined time or longer. Judge that there is. In addition, about the method of detecting the looking-aside state of the left-right direction, since various methods are already known, description is abbreviate | omitted. Also, the state of facing the front does not mean exactly one point, but refers to the state of a predetermined range including the state where the driver is facing the front and the front is included in the effective field of view. Good.

  A side look in the vertical direction occurs, for example, when the driver is gazing at meters or when an object is sandwiched or removed from the sun visor. Then, it can be estimated by the following method that the driver's face is tilted by a predetermined angle or more in the vertical direction. First, as shown in FIG. 4, feature amounts A, B, and C in the vertical direction are detected from the driver's face in the captured image of the vehicle interior camera 10. For the detection, binarization processing, feature point extraction processing, pattern matching processing, or the like is used. The feature amount A is detected as a distance from the top of the eyebrow to the present (the unit may be the number of pixels or an actual distance on the image), and the feature amount B is detected from the present to the bottom of the nose. The feature amount C is detected as a distance from the nose to the cleft. Then, a reference amount corresponding to each feature amount is learned in advance. Specifically, for example, an average of each feature amount detected a plurality of times within a predetermined time (there may be processing such as removal of singular values) or the like can be used. This learning may be performed in an arbitrary period such as a predetermined time after the vehicle system is activated. Moreover, the reference amount at the time of the last driving | operation can also be used. In this case, it is preferable that confirmation that the drivers are the same is obtained.

  Then, using the fact that each feature amount changes small as the driver's face is tilted up and down, there is a state where each feature amount is equal to or less than a predetermined ratio with respect to the learned reference amount. When it continues for a predetermined time or more, it is determined that the driver is in an aside state. For example, assuming that the reference amount of the feature amount A is A1, the reference amount of the feature amount B is B1, and the reference amount of the feature amount C is C1, A ≦ A1 × L, B ≦ B1 × M, and C ≦ C1 When the state of × N continues for a predetermined time, it is determined that the driver is in an aside state. Here, L, M, and N are adaptive values determined by experiments and the like.

  As described above, in the vehicle driving support device 1 of the present embodiment, the sideways in the vertical direction as well as in the left-right direction are detected, so that the driver's sideways can be detected more appropriately.

  Hereinafter, an operation when the vehicle driving assistance device 1 detects a sideward state and performs predetermined driving assistance control will be described. FIG. 5 is a flowchart showing a flow of characteristic processing executed by the driving support device ECU 60.

  First, the driving support device ECU 60 receives a steering angle signal from the steering angle sensor 52, and determines whether or not the steering angle θ is within a predetermined angle θr from the reference angle to the left and right (S100). If it is not within the predetermined angle θr, no processing is performed and one routine of this flow is terminated. As described with reference to FIG. 2, this determination is made because the vehicle interior camera 10 images the driver's head 16 through the hole 14 </ b> A inside the steering wheel 14, and the steering steering angle θ is any of the reference angles. Is operated beyond the predetermined angle θr in this direction, because the driver's head 16 cannot be sufficiently imaged because of being obstructed by the spoke portion 18 of the steering wheel 14 or the like. Therefore, in order to prevent the driver from feeling annoyance due to an excessive alarm operation or the like, the alarm operation or the like is not performed in a state where reliability cannot be determined in the determination of the looking-aside state.

  When the steering angle θ is within a predetermined angle θr to the left and right from the reference angle, the detection data of the steering touch sensor 20 is referred to and it is determined whether or not the steering wheel 14 is held with both hands. (S110).

  If it is determined in S110 that the steering wheel 14 is being held with both hands, it is determined whether or not the vehicle is in a sideways state as described above (S120). That is, based on the state where the driver's face direction or the direction of the line of sight is inclined to a predetermined angle or more with respect to the state where the driver's face direction or line of sight is facing the front, or the state where the driver's face direction is facing the front When one of the states estimated to be tilted up or down by a predetermined angle or more continues for a predetermined time T1 or more, it is determined that the driver is in an aside state. If it is determined that the person is looking aside, an alarm operation is performed by the alarm device 30 (S130), and one routine of this flow is terminated. If it is determined not to look aside, no processing is performed and one routine of this flow is terminated.

  On the other hand, if it is determined in S110 that the steering wheel 14 is not held with both hands (that is, the state of the one-handed handle or the state of letting go), it is determined whether or not it is a side-by-side state ( S140). That is, based on the state where the driver's face direction or the direction of the line of sight is inclined to a predetermined angle or more with respect to the state where the driver's face direction or line of sight is facing the front, or the state where the driver's face direction is facing the front When one of the states estimated to be tilted up or down by a predetermined angle or more continues for a predetermined time T2 or more, it is determined that the driver is in an aside state. When it is determined that the vehicle is in the side-by-side state, an alarm operation is performed by the alarm device 30 and the lane keeping control system 40 is not operating (when the lane keeping control switch 44 is in an off state). This is operated (S150), and one routine of this flow is completed. Here, the predetermined time T2 is shorter than the predetermined time T1 used in S120. If it is determined not to look aside, no processing is performed and one routine of this flow is terminated.

  As described above, when it is determined that the driver is in the state of the one-handed handle or the hand-off operation, an alarm operation is performed at an earlier timing with respect to the looking-aside state. Further, the lane keeping control system 40 is activated to suppress the deviation of the own vehicle from the traveling lane due to the side look. That is, when it is considered that the alarm is highly urgent because the driver is in a one-handed handle state or in a state of letting go, the control for more actively suppressing the inconvenience due to looking aside is performed. As a result, predetermined driving support control for suppressing inconvenience due to looking aside can be appropriately performed.

  As mentioned above, although the best mode for carrying out the present invention has been described using one embodiment, the present invention is not limited to such one embodiment, and within the scope not departing from the gist of the present invention, Various modifications and substitutions can be made to the above-described embodiment.

  For example, as an example of the predetermined driving support control of S150 in the flowchart of FIG. 5, the alarm operation at the early timing and the operation of the lane keeping control system 40 are illustrated, but only one of these may be performed. . In addition to this, for example, control for instructing the brake device to output a braking force may be performed.

  Further, as shown in FIG. 6, when it is determined in S110 that the steering wheel 14 is not being held with both hands, it is determined whether the state is a hand-off operation state or a one-handed handle state (S135). In the case of the hand-off operation state, the alarm operation and the operation of the lane keeping control system 40 may be performed at an earlier timing than in the case of the one-hand handle state (S160, 170). In this case, each predetermined time is determined as T1 ≧ T2 ≧ T3. In this way, in a hand-off operation state in which the urgency of the alarm is considered to be higher than the state of a one-handed handle, control for more actively suppressing inconvenience due to looking aside is performed.

  Further, if the imaging mode of the vehicle interior camera 10 does not image the driver's head 16 through the hole 14A as in the embodiment, the determination in S100 is unnecessary. For example, this corresponds to the case where the vehicle interior camera 10 is disposed near the ceiling of the vehicle interior.

  In the determination of the look-aside state, the look-ahead state in the left-right direction and the up-down direction is detected. However, the look-ahead state only in the left-right direction or only the up-down direction may be detected.

  INDUSTRIAL APPLICABILITY The present invention can be used for an apparatus that performs control for suppressing at least inconvenience due to a driver's side aside. The appearance, weight, size, running performance, etc. of the vehicle to be mounted are not limited.

1 is a diagram illustrating an example of an overall configuration of a vehicle driving support apparatus 1 according to an embodiment of the present invention. It is a figure which shows a mode that the vehicle interior camera 10 images the driver | operator's head 16. FIG. It is a figure which shows an example of the whole structure of the lane keeping control system. It is a figure which shows a mode that the feature-value of an up-down direction is detected from a driver | operator's face. It is a flowchart which shows the flow of the characteristic process which ECU60 for driving assistance apparatuses performs. 12 is a flowchart showing another example of the flow of characteristic processing executed by the ECU 60 for driving support device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle driving assistance device 10 Car interior camera 12 Combination meter 14 Steering wheel 14A Hole 16 Driver's head 18 Spoke part 20 Steering touch sensor 30 Alarm device 40 Lane maintenance control system 42 Lane recognition camera 44 Lane maintenance control switch 46 Electronic control unit for lane keeping control 48 Buzzer 50 Steering device 52 Steering angle sensor 54 Torque sensor 56 Assist motor 58 Controller 60 Electronic control unit for driving support device

Claims (5)

Looking aside state detecting means for detecting the looking side state of the driver;
Holding state detecting means for detecting the holding state of the steering wheel by the driver;
Alarm means for alarming the driver, and
If the driver's side-by-side state is detected by the side-by-side state detection means and the holding state detection unit detects that the driver is holding the steering wheel with both hands, the side-by-side state is Controlling the alarm means to perform an alarm operation when continued for a first predetermined time;
When the driver's side-by-side state is detected by the side-by-side state detection unit and the holding state detection unit detects that the driver is not holding the steering wheel with both hands, the side-by-side state is The alarm means is controlled to perform an alarm operation when the second predetermined time that is shorter than the first predetermined time is continued.
Vehicle driving support device. Looking aside state detecting means for detecting the looking side state of the driver;
Holding state detecting means for detecting the holding state of the steering wheel by the driver;
Alarm means for alarming the driver, and
If the driver's side-by-side state is detected by the side-by-side state detection means and the holding state detection unit detects that the driver is holding the steering wheel with both hands, the side-by-side state is Controlling the alarm means to perform an alarm operation when continued for a first predetermined time;
When the driver's side-by-side state is detected by the side-by-side state detection means, and the holding state detection unit detects that the driver is holding the steering wheel with one hand, the side-by-side state is Controlling the alarm means to perform an alarm operation when a second predetermined time that is shorter than the first predetermined time is continued;
If the driver's side-by-side state is detected by the side-by-side state detection unit and the driver's holding state detection unit detects that the driver is not holding the steering wheel with any hand, the side-by-side state is detected. The alarm means is controlled to perform an alarm operation when a state continues for a third predetermined time that is shorter than the second predetermined time,
Vehicle driving support device. The vehicle driving support device according to claim 1 or 2 ,
Comprising imaging means for imaging the driver's head;
The looking-aside state detecting means is a means for detecting a feature amount in the vertical direction of the driver's face from a captured image of the imaging means, and detecting a driver's looking-aside state based on the change in the feature amount.
Vehicle driving support device. The vehicle driving support device according to claim 3 ,
The vehicle side assistance state detecting means is means for learning in advance a reference amount that serves as a reference for detecting a state of looking aside regarding the feature amount. The vehicle driving support device according to any one of claims 1 to 4 ,
A steering angle detector for detecting the steering angle;
The looking-aside state detecting means is a means for detecting a driver's looking-aside state based on an image captured by an imaging means that images the driver's head through a hole inside the steering wheel.
When the steering angle detected by the steering angle detector is outside a predetermined range, the predetermined driving support control is stopped.
Vehicle driving support device.

Images