JP2021089616A - Driving support device, and driving support system - Google Patents

Abstract

To provide a driving support device and a driving support system capable of estimating a gazing direction according to an occupant.SOLUTION: In a driving support device 10, an occupant direction detection unit 1 detects a direction of a driver 30's face, based on the face image of the driver 30 captured by an imaging unit 11, and a driving state detection unit 2 detects the driving state of a vehicle 50 based on a yaw rate of the vehicle 50 detected by a yaw rate sensor 12. A prediction time setting unit 3 calculates a cross-correlation function between the direction of the driver 30's face and the forwarding direction of the vehicle 50, and sets the prediction time of the driver 30 based on the correlation function. A gazing direction estimation unit 4 estimates the gazing direction of the driver 30 based on the driving state of the vehicle 50 and the prediction time of the driver 30.SELECTED DRAWING: Figure 1

Description

The present invention relates to a driving support device and a driving support system that support the driving of a vehicle.

In order to be able to drive the vehicle safely, occupants such as drivers always predict the point where the vehicle will reach and look at the point. Therefore, for example, as disclosed in Patent Documents 1 to 4, there are a driving support device and a driving support system that estimate the occupant's gaze point in the traveling direction of the vehicle and utilize it for driving support of the vehicle.

In Patent Document 1, the traveling locus of the vehicle is predicted based on the traveling state such as the vehicle speed and the yaw rate detected by the sensor, and the gaze point of the driver is estimated based on these. Then, the driver's gaze direction is detected based on the driver's face image captured by an imaging unit such as a camera, and the presence or absence of the driver's inattentive state is determined based on the gaze direction and the estimated gaze point. However, if there is an inattentive state, an alarm is output.

In Patent Document 2, the direction of the face and the line of sight intersects with the trajectory of the vehicle based on the direction of the driver's face and the line of sight detected from the image captured by the camera, the vehicle speed, and the track of the vehicle predicted from the navigation information. Estimate the gaze point to be performed, and calculate the driver look-ahead time until the gaze point is reached. In addition, when it is detected from the navigation information that the vehicle is entering or traveling on a curve and the driver look-ahead time is within a predetermined range, the driver drives based on the direction of the face and the line of sight, the vehicle speed, and the navigation information. Correct the front view range of the person. Then, when the direction of the face or the line of sight deviates from the front view range, the vehicle is controlled by outputting an alarm or the like.

In Patent Document 3, the driver's gaze point is estimated based on the running state such as the vehicle speed and the yaw rate detected by the sensor and the preset prediction time. In addition, a threshold value for inattentive determination (driver's front view range) is set based on the driving state and the gazing point. Then, based on the threshold value for inattentive judgment and the orientation of the driver's face detected from the image captured by the camera, the presence or absence of the driver's inattentive state is determined, and if there is inattentive state, an alarm is output. Vehicle control.

The device and system disclosed in Patent Document 4 detect road information such as the curve length, radius of curvature, or road surface friction coefficient of the road on which the vehicle travels, based on navigation information and images captured by the camera. In addition, the driver's prediction time is set based on the road information and the driving state such as the vehicle speed and the yaw rate detected by the sensor. Alternatively, the forecast time is input by the operation of the driver. Then, the driver's forward gaze distance (gaze point) is estimated based on the foreseeable time and the vehicle speed, and the acceleration of the vehicle during curve traveling is controlled based on the forward gaze distance and road information.

Japanese Unexamined Patent Publication No. 9-7100 Japanese Unexamined Patent Publication No. 2010-257293 Japanese Unexamined Patent Publication No. 2009-294735 JP-A-2009-78732

There are individual differences in the direction of the gaze point that the occupants such as the driver gaze at. Therefore, if the gaze direction is estimated without considering the characteristics of the occupant, the threshold value such as the visual field range for inattentive judgment cannot be set appropriately according to the occupant, and the presence or absence of the inattentive state of the occupant cannot be properly determined. It may not be possible, or it may not be possible to properly control the running of the vehicle. In particular, when the vehicle is traveling on a curved road, the gaze direction varies from side to side of the vehicle due to individual differences of the occupants, so that the judgment of the occupant's inattentive state and the vehicle's travel control may not be performed appropriately according to the occupant. ..

An object of the present invention is to provide a driving support device and a driving support system capable of estimating a gaze direction according to an occupant.

The driving support device according to the present invention is based on a traveling state detection unit that detects the traveling state of the vehicle, a gaze direction estimation unit that estimates the gaze direction of the occupant with respect to the traveling direction of the vehicle, and an image of the occupant captured. It is provided with a occupant orientation detection unit that detects the direction of the occupant, and a prediction time setting unit that sets the occupant's prediction time based on the detection result of the occupant orientation detection unit and the detection result of the traveling state detection unit, and estimates the gaze direction. The unit estimates the gaze direction of the occupant based on the detection result of the traveling state detection unit and the setting result of the prediction time setting unit.

Further, the driving support system according to the present invention includes the above-mentioned driving support device, a yaw rate sensor for detecting the yaw rate of the vehicle, and an imaging unit for imaging the occupants of the vehicle. The driving state of the vehicle is detected based on the output of the sensor.

According to the driving support device and the driving support system of the present invention, the prediction time setting unit is based on the direction of the occupant detected by the occupant orientation detection unit from the image of the occupant and the driving state of the vehicle detected by the traveling state detection unit. Set the forecast time according to the occupants of the vehicle. Then, the gaze direction estimation unit estimates the gaze direction of the occupant with respect to the traveling direction of the vehicle based on the foreseeable time of the occupant and the traveling state of the vehicle. That is, since not only the running state of the vehicle but also the characteristic of the occupant's unique prediction time is taken into consideration, the gaze direction according to the occupant can be estimated.

In the present invention, the traveling state detection unit detects the traveling direction of the vehicle based on the detection result of the yaw rate of the vehicle, and the occupant orientation detection unit detects the orientation of the occupant's face based on the occupant's face image. , The prediction time setting unit calculates a cross-correlation function between the traveling direction of the vehicle detected by the traveling state detection unit and the direction of the occupant's face detected by the occupant orientation detection unit, and is based on the cross-correlation function. The occupant's forecast time may be set.

Further, in the present invention, the prediction time setting unit may use the phase difference between the change in the traveling direction of the vehicle and the change in the direction of the occupant's face when the cross-correlation function is maximized as the occupant's prediction time.

Further, in the present invention, the prediction time setting unit may update the prediction time of the occupant when the traveling direction of the vehicle detected by the traveling state detection unit deviates from the predetermined straight-ahead range.

Further, in the present invention, in the above-mentioned driving support device, a visual field setting unit that sets the front visual field range of the occupant based on the occupant's gaze direction estimated by the gaze direction estimation unit, a setting result of the visual field setting unit, and an occupant. An inattentive determination unit for determining the presence or absence of an inattentive state of the occupant may be further provided based on the detection result of the orientation detection unit.

Further, in the present invention, the visual field setting unit may change the size of the front visual field range according to the foreseeing time detected by the foreseeing time setting unit.

Further, in the present invention, the inattentive determination unit sets the inattentive determination time based on the foresight time set by the foresight time setting unit, and the occupant orientation detected by the occupant orientation detection unit is set by the visual field setting unit. When the state of being out of the front view range continues for the inattentive determination time, it may be determined that the occupant has inattentive state.

Further, in the present invention, the inattentive determination unit may change the length of the inattentive determination time according to the foreseeing time set by the foresight time setting unit.

Further, the driving support system of the present invention may further include an alarm unit that outputs an alarm to the occupant when it is determined by the inattentive determination unit provided in the driving support device that the occupant is in an inattentive state. ..

According to the present invention, it is possible to provide a driving support device and a driving support system capable of estimating a gaze direction according to an occupant.

It is a block diagram of the driving support system of embodiment of this invention. It is a flowchart which showed the operation of the driving support system of FIG. It is a flowchart which showed the detail of the prediction time setting process of FIG. It is a figure which showed the setting example of the prediction time by the driving support device of FIG. It is a flowchart which showed the detail of the gaze direction estimation process of FIG. It is a figure which showed the estimation example of the gaze direction by the driving support device of FIG. It is a flowchart which showed the detail of the front view | field of view range setting process of FIG. It is a figure which showed the setting example of the front view range by the driving support device of FIG. It is a flowchart which showed the detail of the inattentiveness determination process of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each figure, the same parts or corresponding parts are designated by the same reference numerals.

FIG. 1 is a configuration diagram of the driving support system 100 of the first embodiment. The driving support system 100 is mounted on a vehicle 50 composed of a four-wheeled vehicle. The driving support system 100 includes a driving support device 10, an imaging unit 11, a yaw rate sensor 12, and an alarm unit 13.

The image pickup unit 11 is composed of, for example, an infrared camera, and captures a moving image of the face of the driver (occupant) 30 seated in the driver's seat of the vehicle 50. The yaw rate sensor 12 detects the yaw rate of the vehicle 50. The alarm unit 13 is composed of a speaker, a buzzer, lights, a display, a vibrator, or the like installed in the vehicle interior of the vehicle 50, and gives an auditory, visual, or tactile alarm to an occupant such as the driver 30. Output.

The driving support device 10 includes an occupant orientation detection unit 1, a traveling state detection unit 2, a prediction time setting unit 3, a gaze direction estimation unit 4, a visual field setting unit 5, and an inattentive determination unit 6. These blocks 1 to 6 are built in a microcomputer (not shown) that controls the overall operation of the driving support device 10, and the functions of the blocks 1 to 6 (details will be described later) are realized by software.

The occupant orientation detection unit 1 detects the orientation of the driver 30 as the orientation of the driver 30 based on the face image of the driver 30 captured by the imaging unit 11.

The traveling state detection unit 2 detects the yaw rate of the vehicle 50 or the traveling direction of the vehicle 50 as the traveling state of the vehicle 50 based on the detection result of the yaw rate sensor 12.

The prediction time setting unit 3 calculates a cross-correlation function between the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 and the direction of the driver 30 detected by the occupant orientation detection unit 1. Then, the prediction time setting unit 3 sets the prediction time of the driver 30 based on the cross-correlation function. The driver 30 predicts a point at which the vehicle 50 will reach after a certain time Δt has elapsed, and is driving while gazing at the point. In this case, the time Δt is the prediction time. In general, the foresight time of a less skilled driver tends to be shorter than the foresight time of a more skilled driver.

The gaze direction estimation unit 4 gaze direction of the driver 30 with respect to the traveling direction of the vehicle 50 based on the yaw rate of the vehicle 50 detected by the traveling state detection unit 2 and the prediction time set by the forecast time setting unit 3. To estimate.

The visual field setting unit 5 sets the front visual field range of the driver 30 based on the gaze direction of the driver 30 estimated by the gaze direction estimation unit 4.

The inattentive determination unit 6 determines whether or not the driver 30 is in an inattentive state based on the front visual field range set by the visual field setting unit 5 and the orientation of the driver 30 detected by the occupant orientation detection unit 1. Then, when the inattentive determination unit 6 determines that the driver 30 is in an inattentive state, the inattentive determination unit 6 outputs an alarm command to the alarm unit 13. The alarm unit 13 receives an alarm command from the inattentive determination unit 6 and outputs an alarm to an occupant such as the driver 30.

FIG. 2 is a flowchart showing the operation of the driving support system 100. This operation is started when the vehicle 50 is started (ignition is turned on), and is terminated when the vehicle 50 is stopped (ignition is turned off).

In FIG. 2, first, the imaging unit 11 images the face of the driver 30 (step S1), and the occupant-oriented detection unit 1 of the driving support device 10 is based on the face image of the driver 30 imaged by the imaging unit 11. Then, the orientation of the driver 30's face is detected at a predetermined cycle (step S2).

When the vehicle 50 travels, the yaw rate sensor 12 detects the yaw rate of the vehicle 50 at a predetermined cycle (step S3), and the traveling state detection unit 2 determines the traveling direction of the vehicle 50 based on the detection result of the yaw rate sensor 12. (Step S4).

Then, the prediction time setting unit 3 executes the prediction time setting process (step S5). FIG. 3 is a flowchart showing the details of the prediction time setting process.

In FIG. 3, if the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 is within a predetermined straight-ahead range set in advance (step S51: NO), the vehicle 50 is traveling on a straight road (step S51: NO). Step S52). In this case, the prediction time setting unit 3 maintains the prediction time of the current driver 30 stored in the internal memory of the microcomputer (not shown) (step S53). Therefore, the forecast time is not updated.

At the time of manufacturing the driving support device 10, a predetermined standard prediction time is stored in the internal memory of the microcomputer. Then, as will be described later, when the driving support device 10 is in operation (when the vehicle 50 is traveling), the prediction time set by the prediction time setting unit 3 is stored in the internal memory of the microcomputer, and the driver 30 is predicted. The time is updated.

When the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 deviates from the predetermined straight-ahead range (step S51: YES), the vehicle 50 is traveling on a curved road (step S54). In this case, the prediction time setting unit 3 calculates a cross-correlation function between the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 and the direction of the driver 30 detected by the occupant orientation detection unit 1 (step). S55).

FIG. 4 is a diagram showing an example of setting the prediction time by the prediction time setting unit 3. In FIG. 4A, the temporal change in the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 is shown by a solid line, and the time of the face orientation of the driver 30 detected by the occupant orientation detection unit 1 is shown. The change is shown by a broken line. The horizontal axis is the time t (seconds), and the vertical axis is the direction (tilt angle with respect to 0 ° in the reference direction).

The driver 30 predicts a point at which the vehicle 50 will reach, and while looking at the point, drives the vehicle 50 in response to changes in surrounding conditions such as roads. Therefore, for example, when the vehicle 50 travels on a curved road, the direction of the face of the driver 30 changes according to the bending direction and curvature of the curved road, as shown in FIG. 4A. The traveling direction of the vehicle 50 changes accordingly. The magnitude (amplitude) of the change differs between the direction of the face of the driver 30 and the traveling direction of the vehicle 50, but the mode of change is the same, and there is a phase difference (time lag) between the two.

The cross-correlation function between the direction of the driver 30's face and the traveling direction of the vehicle 50 shown in FIG. 4A calculated by the prediction time setting unit 3 in step S55 of FIG. 3 is shown by a solid line in FIG. 4B. It becomes a function as shown. In FIG. 4B, the horizontal axis is the phase difference Δt (seconds) and the vertical axis is the correlation coefficient.

In step S56 of FIG. 3, the prediction time setting unit 3 sets the phase difference Δt at which the cross-correlation function between the direction of the face of the driver 30 and the traveling direction of the vehicle 50 is maximized as the prediction time of the driver 30 and internally. The forecast time of the driver 30 stored in the memory is updated (step S57). In the example of FIG. 4B, since the phase difference Δt at which the cross-correlation function is maximized is “1 second”, the predictive time of the driver 30 is “1 second”. This prediction time is the current value, and it is possible to set this as the prediction time of the driver 30 as it is and update the prediction time, but if the prediction time is updated only by the current value, the accuracy of the prediction time is accurate. May cause an erroneous judgment at the time of inattentive judgment. Therefore, in step S57, for example, a process of calculating the average value of the current prediction time obtained from the phase difference Δt and the prediction time set in the internal memory and resetting this as the update value of the prediction time is performed. It is preferable to do so. In the case of FIG. 4B, the current prediction time is 1 second, but if the set prediction time is 2 seconds, 1.5 seconds is reset as a new prediction time and the prediction time is updated. To. In this way, by repeating the update of the prediction time while reflecting the past prediction time in the current prediction time, the prediction time peculiar to the driver 30 can be set accurately.

Next, the gaze direction estimation unit 4 executes the gaze direction estimation process (step S6 in FIG. 2). FIG. 5 is a flowchart showing the details of the gaze direction estimation process.

In FIG. 5, first, the gaze direction estimation unit 4 detects the current yaw rate of the vehicle 50 from the output of the traveling state detection unit 2 (step S61). Then, the gaze direction estimation unit 4 estimates the gaze direction of the driver 30 based on the yaw rate and the foresight time of the driver 30 set by the forecast time setting unit 3 (step S62).

FIG. 6 is a diagram showing an example of estimating the gaze direction by the gaze direction estimation unit 4. FIG. 6 shows a state in which the vehicle 50 and the curved road on which the vehicle 50 travels are viewed from above. The direction in which the vehicle 50 shown by the solid line in FIG. 6 is facing is the reference direction 0 °. The direction of the inclination angle φ with respect to the reference direction 0 ° is the gaze direction that the driver 30 gazes at, and the gaze point of the driver 30 is in this gaze direction φ. The gaze direction estimation unit 4 estimates the gaze direction φ of the driver 30 by the following equation based on the yaw rate ω of the vehicle 50 and the prediction time Δt of the driver 30.
φ = ωΔt / 2
Then, the gaze direction estimation unit 4 stores the estimated gaze direction φ of the driver 30 in the internal memory (step S63 in FIG. 5).

Next, the visual field setting unit 5 executes the front visual field range setting process (step S7 in FIG. 2). FIG. 7 is a flowchart showing the details of the front view range setting process. Further, FIG. 8 is a diagram showing an example of setting the front visual field range by the visual field setting unit 5. FIG. 8 shows a state in which the front view ranges A1 and A2 of the vehicle 50 and the driver 30 are viewed from above.

In FIG. 7, if the predictive time of the driver 30 set by the predictive time setting unit 3 is the same as or longer than the predetermined predetermined time set in advance (step S71: NO), the visual field setting unit 5 is shown in FIG. As shown in 8 (a), the front view range of the driver 30 is set to a predetermined normal range A1 (step S72 in FIG. 7). Specifically, the spread angle of the front viewing range with respect to the horizontal direction is set to the normal angle A1.

On the other hand, if the foreseeing time of the driver 30 is shorter than the predetermined time (step S71: YES in FIG. 7), the visual field setting unit 5 normally ranges the front visual field range of the driver 30 as shown in FIG. 8A. It is set in a predetermined range A2 narrower than A1 (step S73 in FIG. 7). Specifically, the spread angle of the front viewing range with respect to the horizontal direction is set to a predetermined angle A2 smaller than the normal angle A1.

Next, the visual field setting unit 5 adjusts the front visual field ranges A1 and A2 of the driver 30 in the left-right direction of the vehicle 50 based on the gaze direction φ of the driver 30 estimated by the gaze direction estimation unit 4 (FIG. Step S74 of 7.

At this time, as shown in FIGS. 8B to 8D, the front visual field ranges A1 and A2 are set so that the gaze direction φ comes to the center of the front visual field ranges A1 and A2. Specifically, as shown in FIG. 8B, when the gaze direction φ is 0 °, the vehicle 50 is traveling on a straight road, so the front viewing ranges A1 and A2 are shifted to the left and right of the vehicle 50. It is set directly in front of you. Further, as shown in FIG. 8C, when the gaze direction φ is + θ °, the vehicle 50 is traveling on a curved road turning right, so that the front viewing ranges A1 and A2 correspond to the gaze direction φ. , Shifted to the right side of the vehicle 50. Further, as shown in FIG. 8D, when the gaze direction φ is −θ °, the vehicle 50 is traveling on a curved road turning left, so that the front viewing ranges A1 and A2 correspond to the gaze direction φ. Then, it is shifted to the left side of the vehicle 50.

The visual field setting unit 5 stores data indicating the front visual field range of the driver 30 set in steps S72 to S74 of FIG. 7 as described above in the internal memory (step S75 of FIG. 7).

Next, the inattentive determination unit 6 executes the inattentive determination process (step S8 in FIG. 2). FIG. 9 is a flowchart showing the details of the inattentive determination process.

In FIG. 9, if the foresight time of the driver 30 set by the foresight time setting unit 3 is the same as or longer than the predetermined predetermined time set in advance (step S81: NO), the inattentive determination unit 6 will inspect. The determination time is set to a predetermined normal time (step S82).

If the foresight time of the driver 30 is shorter than the predetermined time (step S81: YES), the inattentive determination unit 6 sets the inattentive determination time to a predetermined time shorter than the normal time (step S83).

The predetermined time to be compared with the foresight time in step S81 of FIG. 9 may be the same as or different from the predetermined time to be compared with the foresight time in step S71 of FIG. ..

When the inattentive determination time is set as described above, the inattentive determination unit 6 detects the direction of the driver 30's face from the output of the occupant orientation detection unit 1 (step S84). Then, the direction of the face of the driver 30 is compared with the front view range set by the field view setting unit 5, and the state in which the direction of the face of the driver 30 deviates from the front view range continues for the inattentive determination time. If not (step S85: NO), the inattentive determination unit 6 determines that the driver 30 is not inattentive (step S86).

On the other hand, if the state in which the driver 30's face is out of the front view range continues for the inattentive determination time (step S85: YES), the inattentive determination unit 6 determines that the driver 30 has inattentive state (step S85: YES). Step S87). Then, the inattentive determination unit 6 outputs an alarm command, and in response to the alarm command, the alarm unit 13 outputs an alarm (step S88).

When the inattentive determination process is completed as described above, the process returns to step S1 of FIG. 2 and the subsequent processes are repeated.

According to the above embodiment, the direction of the driver 30's face detected by the occupant orientation detection unit 1 of the driving support device 10 from the face image of the driver 30 imaged by the imaging unit 11 and the yaw rate of the driving state detection unit 2. The prediction time setting unit 3 sets the prediction time according to the driver 30 based on the traveling direction of the vehicle 30 detected by the sensor 12. Then, the gaze direction estimation unit 4 estimates the gaze direction of the driver 30 based on the foreseeable time of the driver 30 and the traveling direction of the vehicle 30. That is, since not only the running state of the vehicle 30 but also the characteristic of the driver 30's unique prediction time is taken into consideration, the gaze direction according to the driver 30 can be estimated.

Further, in general, the direction of the face of the driver 30 looking ahead of the vehicle 50 changes and the traveling direction of the vehicle 50 changes according to changes in the surrounding conditions such as the road on which the vehicle 50 travels. In consideration of this, in the above embodiment, the prediction time setting unit 3 calculates a cross-correlation function between the traveling direction of the vehicle 50 and the face orientation of the driver 30, and drives based on the cross-correlation function. Since the foreseeing time of the person 30 is set, the foreseeing time according to the driver 30 can be set appropriately. Then, since the gaze direction estimation unit 4 estimates the gaze direction of the driver 30 based on the prediction time according to the driver 30 and the yaw rate of the vehicle 50, the gaze direction according to the driver 30 is estimated more appropriately. can do.

Further, when the vehicle 50 travels on a curved road, the traveling direction of the vehicle 50 deviates from the predetermined straight-ahead range. In consideration of this, in the above embodiment, every time the traveling direction of the vehicle 50 detected by the traveling state detection unit 2 deviates from the predetermined straight-ahead range, the foreseeing time setting unit 3 determines the traveling direction of the vehicle 50 and the occupant. The occupant's prediction time is updated based on the direction of the driver 30's face detected by the direction detection unit 1. Therefore, it is possible to more accurately estimate the traveling direction of the vehicle 50 and the gaze direction according to the driver 30.

Further, in the above embodiment, the visual field setting unit 5 appropriately sets the front visual field range of the driver 30 based on the gaze direction of the driver 30 estimated by the gaze direction estimation unit 4 and the yaw rate of the vehicle 50. can do. Then, based on the front view range of the driver 30 and the orientation of the driver 30's face detected by the occupant orientation detection unit 1, the inattentive determination unit 6 appropriately determines whether or not the driver 30 is in the inattentive state. can do.

Further, as described above, the lower the driving skill of the vehicle, the shorter the prediction time and the narrower the field of view in front of the vehicle. In consideration of this, in the above embodiment, when the prediction time of the driver 30 set by the prediction time setting unit 3 is shorter than the predetermined time, the front viewing range is set narrow and the prediction time of the driver 30 is set. If it is the same as or longer than the predetermined time, the front viewing range is set wide. Therefore, for the driver 30 who has a low driving skill of the vehicle 50 and a short prediction time, the front view range can be set narrow and the determination of the presence or absence of the inattentive state can be made strict. Further, for the driver 30 who has a high driving skill of the vehicle 50 and a long prediction time, the front view range can be set wide and the determination of the presence or absence of the inattentive state can be relaxed. That is, by setting the width of the front visual field range according to the foresight time affected by the driving skill of the driver 30, it is possible to more appropriately determine the presence or absence of the inattentive state.

Further, in the above embodiment, when the face orientation of the driver 30 detected by the occupant orientation detection unit 1 deviates from the front visual field range set by the visual field setting unit 5 and the inattentive determination time continues. The inattentive determination unit 6 determines that the driver 30 is in an inattentive state. Therefore, when the direction of the face of the driver 30 is momentarily deviated from the front view range, it is possible to avoid overdetermining that the driver 30 is in an inattentive state. Further, when the direction of the driver 30 is continuously deviated from the front view range to some extent, it can be reliably determined that there is an inattentive state.

Further, in the above embodiment, when the foresight time of the driver 30 set by the foresight time setting unit 3 is shorter than the predetermined time, the inattentive determination unit 6 sets the inattentive determination time shorter and the foresight time of the driver 30. When is equal to or longer than the predetermined time, the inattentive determination unit 6 sets the inattentive determination time longer. Therefore, for the driver 30 who has a low driving skill of the vehicle 50 and a short prediction time, the inattentiveness determination time can be set short and the determination of the presence or absence of the inattentive state can be made strict. Further, for the driver 30 who has a high driving skill of the vehicle 50 and a long prediction time, it is possible to set a long inattentive determination time to loosen the determination of the presence or absence of the inattentive state. That is, by setting the inattentive determination time according to the foresight time affected by the driving skill of the driver 30, it is possible to more appropriately determine the presence or absence of the inattentive state.

Further, in the above embodiment, when the inattentive determination unit 6 determines that the driver 30 is in the inattentive state, the alarm unit 13 outputs an alarm. Therefore, the driver 30 can stop the inattentive state and drive the vehicle 50 safely.

In addition to the above-described embodiments, the present invention can employ various embodiments as described below.

In the above embodiment, an example in which the direction of the driver 30's face is detected as the direction of the occupant based on the face image of the driver 30 captured by the imaging unit 11 is shown, but the present invention is limited to this. It's not something to do. In addition to this, for example, the line-of-sight direction of the driver 30 may be detected as the direction of the occupant based on the image of the inside of the vehicle captured by the image pickup unit 11. Then, the foreseeing time of the driver 30 may be set or the presence or absence of the inattentive state of the driver 30 may be determined based on the line-of-sight direction.

In the above-described embodiment, an example is shown in which the traveling state detection unit 2 detects the yaw rate of the vehicle 50 or detects the traveling direction of the vehicle 50 by the yaw rate sensor 12 as the traveling state of the vehicle 50. Is not limited to these. In addition to this, the traveling direction, turning degree, and the like of the vehicle may be detected as a traveling state based on, for example, a gyro sensor and navigation information.

In the above embodiment, an example in which the front view range and the inattentive determination time are set in two stages based on the foresight time of the driver 30 is shown, but the present invention is not limited to this. The front view range and the inattentive determination time may be set to three or more stages based on the foresight time of the driver 30.

In the above-described embodiment, the driver who has a lower driving skill level and a shorter forecasting time has a narrower front view range or a shorter inattentive judgment time to determine the inattentive state and give an alarm. Although a strict example is shown, the present invention is not limited to this. For example, from the viewpoint of preventing erroneous judgment of the inattentive state, conversely, the driver with lower driving skill and shorter prediction time may set a wider front view range or a longer inattentive judgment time. Therefore, the judgment of the inattentive state and the alarm may be loosened. That is, based on various viewpoints, the size of the front view range may be changed or the length of the inattentive determination time may be changed according to the driver's foresight time.

In the above embodiment, an example in which the present invention is applied to the driving support device 10 and the driving support system 100 that estimate the gaze direction of the driver 30 and determine the inattentive state of the driver 30 is shown. However, the present invention is also applicable to a driving support device and a driving support system that utilize the estimated driver's gaze direction for automatic driving control such as steering and speed of a vehicle, for example. The present invention is also applicable to a driving support device and a driving support system that estimate the gaze direction of an occupant other than the driver.

1 Passenger orientation detection unit 2 Driving state detection unit 3 Forecast time setting unit 4 Gaze direction estimation unit 5 Field of view setting unit 6 Inattentive judgment unit 10 Driving support device 11 Imaging unit 12 Yaw rate sensor 13 Alarm unit 30 Driver (occupant)
50 Vehicle 100 Driving support system A1, A2 Front view range Δt, phase difference, prediction time ω yaw rate φ gaze direction

Claims (10)

A running state detection unit that detects the running state of the vehicle,
In a driving support device including a gaze direction estimation unit that estimates the gaze direction of an occupant with respect to the traveling direction of the vehicle.
An occupant orientation detection unit that detects the orientation of the occupant based on the captured image of the occupant, and
A prediction time setting unit for setting the prediction time of the occupant based on the detection result of the occupant orientation detection unit and the detection result of the traveling state detection unit is further provided.
The gaze direction estimation unit is a driving support device that estimates the gaze direction of the occupant based on the detection result of the traveling state detection unit and the setting result of the prediction time setting unit. In the driving support device according to claim 1,
The traveling state detection unit detects the traveling direction of the vehicle based on the detection result of the yaw rate of the vehicle.
The occupant orientation detection unit detects the orientation of the occupant's face based on the occupant's face image.
The prediction time setting unit calculates a cross-correlation function between the traveling direction of the vehicle detected by the traveling state detection unit and the face orientation of the occupant detected by the occupant orientation detection unit, and the cross-correlation A driving support device characterized in that the prediction time of the occupant is set based on a function. In the driving support device according to claim 2,
The prediction time setting unit sets the phase difference between the change in the traveling direction of the vehicle and the change in the direction of the occupant's face when the cross-correlation function is maximized as the prediction time of the occupant. A characteristic driving support device. In the driving support device according to any one of claims 1 to 3.
The driving support device is characterized in that the forecast time setting unit updates the forecast time of the occupant when the traveling direction of the vehicle detected by the traveling state detection unit deviates from a predetermined straight-ahead range. In the driving support device according to any one of claims 1 to 4.
A visual field setting unit that sets the front view range of the occupant based on the gaze direction of the occupant estimated by the gaze direction estimation unit.
A driving support device further comprising an inattentive determination unit for determining the presence or absence of an inattentive state of the occupant based on a setting result of the visual field setting unit and a detection result of the occupant orientation detection unit. In the driving support device according to claim 5,
The visual field setting unit is a driving support device that changes the size of the front visual field range according to the forecast time detected by the forecast time setting unit. In the driving support device according to claim 5 or 6.
The inattentive determination unit sets the inattentive determination time based on the foresight time set by the foresight time setting unit, and the orientation of the occupant detected by the occupant orientation detection unit is set by the visual field setting unit. A driving support device, characterized in that, when the state of being out of the front view range continues for the inattentive determination time, it is determined that the occupant has inattentive state. In the driving support device according to claim 7.
The inattentive determination unit is a driving support device that changes the length of the inattentive determination time according to the foresight time detected by the inattentive time setting unit. The driving support device according to any one of claims 1 to 8.
A yaw rate sensor that detects the yaw rate of the vehicle and
Includes an imaging unit that captures images of the occupants of the vehicle.
The driving support system provided in the driving support device is characterized in that the driving state detecting unit detects the driving state of the vehicle based on the output of the yaw rate sensor. The driving support device according to any one of claims 5 to 8.
A yaw rate sensor that detects the yaw rate of the vehicle and
An imaging unit that images the occupants of the vehicle,
Including an alarm unit that outputs an alarm to the occupant
The traveling state detection unit provided in the driving support device detects the traveling state of the vehicle based on the output of the yaw rate sensor.
The driving support system is characterized in that the alarm unit outputs the alarm when it is determined by the inattentive determination unit provided in the driving support device that the occupant is in an inattentive state.

Images