JP2011128799A - Device and method for estimating driver state - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a state of a driver after concretely specifying an occurrence factor of a temporal change of an effective visual field angle of the driver. <P>SOLUTION: A stop state of an own vehicle is determined, and visual stimulation is displayed on a display part, when it is determined that the own vehicle is in the stop state. When the displayed visual stimulation is visually recognized by the driver, an effective visual field angle of the driver is derived and stored. Since the effective visual field angle of the driver is derived each time when the own vehicle is determined that it is in the stop state, the state of the driver is estimated according to a prescribed condition wherein a relation between a relative change amount of the effective visual field angle and the driver state is given. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a driver state estimating apparatus and a driver state estimating method for estimating a driver's state by determining whether or not the driver can recognize an object to be noted.

  It is generally known that about 90% of the amount of information that a driver of a vehicle acquires while driving the vehicle is acquired based on the visual ability of the driver of the vehicle. Therefore, if the driver's state is a state where the driver may not be able to recognize the object to be noted, grasp the driver's state due to visual ability and provide appropriate driving assistance is important.

  As a device for grasping the state of the driver, there is a driver state determination device disclosed in Patent Document 1. The driver state determination device disclosed in Patent Document 1 gives a visual stimulus to a driver at an arbitrary position in a predetermined peripheral visual field region, and the driver's line of sight when the arbitrary position is moved. The state of the driver is determined based on the direction, the line-of-sight stop time, and the line-of-sight trajectory. In addition, a specific example of detecting the driver's line-of-sight direction in the driver state determination device in Patent Document 1 is disclosed in Patent Document 2.

  By the way, the human visual field can be divided into a central visual field and a peripheral visual field. The central visual field is a range having a radius of about 1 ° centering on the line-of-sight direction. The central direction of the central visual field is called the central visual field direction. In the central field of view, the shape of the object, the shape of the character, and the color of the object or character can be distinguished finely. On the other hand, the peripheral visual field is a range of about 180 ° in the left and right directions and about 130 ° in the vertical direction deviating from the central visual field. In the peripheral field of view, the shape of a thing or character is not clearly recognized, but it is said that a visual stimulus that changes over time, such as a blinking light spot or a moving object, is detected.

  The range in which the driver can recognize the visual stimulus centering on the driver's central visual field direction is called the driver's effective visual field. The effective visual field changes depending on the driver's internal and external factors. The driver's internal factors are driver characteristics such as deterioration of visual ability, psychological state, and fatigue associated with aging. The external factors of the driver include the complexity of visual information that the driver recognizes from the central visual field or the driving environment such as the visual information amount in the peripheral visual field region.

  As a specific example where the effective field of view narrows due to external factors, the change in visual information during highway driving is large, that is, the change in scenery captured by the vision is large, increasing the driver's cognitive burden in the central field of view, The visual stimulus displayed in the peripheral visual field may not be noticed.

JP 2006-158740 A JP 2003-80969 A

  However, the driver state determination device disclosed in Patent Document 1 only determines whether the range of the effective visual field at a certain point is wider or narrower than the predetermined range. The effective visual field varies depending on the driver or the driving situation of the driver. For this reason, even when it is determined that the effective visual field is narrower than the predetermined range, the specific factor that the effective visual field is reduced is not specified, and it is difficult to perform appropriate driving support.

  For example, when it is determined that the effective visual field is narrower than a predetermined range, the effective visual field is actually narrow according to external factors such as the driving environment, or according to internal factors such as driver fatigue. The effective field of view may be narrow, but the cause of the narrow effective field of view is different. Therefore, depending on the driver state determination device disclosed in Patent Literature 1, there is a problem that the cause of the narrowing of the effective visual field is not specifically specified, and driving assistance becomes the same.

  The present invention solves the above-described conventional problems, and specifically identifies the cause of a change in the driver's effective visual field over time, and then estimates the driver's state. It is another object of the present invention to provide a driver state estimation method.

  A driver state estimation device according to the present invention is a driver state estimation device that estimates a state of a driver of a vehicle having a display area, has an imaging system, and uses the imaging system to image the driver. An image data acquisition unit that acquires data, a stop state determination unit that determines whether the speed of the vehicle is equal to or lower than a predetermined value based on the speed information of the vehicle, and a speed of the vehicle that is equal to or lower than the predetermined value If it is determined that the first visual stimulus is displayed on the display area, whether or not the first visual stimulus is visually recognized by the driver based on the image data of the driver is determined. The visual stimulus control unit for determining, and when it is determined that the first visual stimulus is visually recognized by the driver, the driver is based on the position where the first visual stimulus is displayed in the display area. Is the angle formed by the front direction and the line-of-sight direction. An effective viewing angle deriving unit for deriving an effective viewing angle, a storage unit for storing the first effective viewing angle, the first effective viewing angle, and a point before the first effective viewing angle is stored. And a driver state estimating unit that estimates the state of the driver based on a difference from the second effective viewing angle stored in the storage unit.

  The driver state estimation method of the present invention is a driver state estimation method for estimating the state of a driver of a vehicle having a display area, the driver state estimation method having an imaging system, and the driver using the imaging system. And determining whether the vehicle speed is equal to or lower than a predetermined value based on the vehicle speed information, and determining that the vehicle speed is equal to or lower than the predetermined value. The first visual stimulus is displayed in the display area, and it is determined whether or not the first visual stimulus is visually recognized by the driver based on the image data of the driver, and the first visual stimulus is determined. When it is determined that a stimulus is visually recognized by the driver, the first visual stimulus is an angle formed by the front direction of the driver and the line-of-sight direction based on the position displayed in the display area. Deriving one effective viewing angle and storing the first effective viewing angle The state of the driver based on the difference between the first effective viewing angle and the second effective viewing angle stored in the storage unit before the time when the first effective viewing angle is stored. Is estimated.

  According to the driver state estimation device and the driver state estimation method of the present invention, it is possible to estimate the driver's state after specifically specifying the cause of the temporal change in the driver's effective viewing angle. .

The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 1st Embodiment. The block diagram which shows the driver state estimation apparatus which concerns on 1st Embodiment. The flowchart explaining operation | movement of the driver state estimation apparatus which concerns on 1st Embodiment. Explanatory drawing regarding display of visual stimulus based on visual stimulus display distance or visual stimulus display angle The flowchart explaining the detail of operation | movement of the visual stimulus control part of the driver | operator state estimation apparatus which concerns on 1st Embodiment. The figure which showed an example of the comparison table for estimating a driver's condition The flowchart explaining the detail of operation | movement of the driver state estimation part of the driver state estimation apparatus which concerns on 1st Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 2nd Embodiment. The block diagram which shows the driver state estimation apparatus which concerns on 2nd Embodiment. The flowchart explaining operation | movement of the driver state estimation apparatus which concerns on 2nd Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 3rd Embodiment. The block diagram which shows the driver state estimation apparatus which concerns on 3rd Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 4th Embodiment. The block diagram which shows the driver state estimation apparatus which concerns on 4th Embodiment. The figure which shows an example of the relationship between an estimation threshold value and the spatial frequency of a foreground image The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 5th Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 6th Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 7th Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 8th Embodiment. The system block diagram which shows the structure of the driver state estimation system containing the driver state estimation apparatus which concerns on 9th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 shows a configuration of a driver state estimation system 1a including a driver state estimation device 100a according to the first embodiment. As shown in FIG. 1, the driver state estimation system 1a includes a driver state estimation device 100a, a speed information output unit 210, a display unit 230, and a driving support unit 240. The driver state estimation device 100a includes a stop state determination unit 101a, an image data acquisition unit 220, a visual stimulus control unit 102a, an effective visual field angle derivation unit 103a, a storage unit 104a, and a driver state estimation unit 105a. Hereinafter, the driver state estimation system 1a will be described as being mounted on a vehicle such as an automobile. The vehicle is referred to as “own vehicle”, and the driver of the vehicle is referred to as “driver”.

  The speed information output unit 210 outputs speed information related to the speed of the host vehicle to the driver state estimation device 100a. Specifically, the speed information output unit 210 acquires the turbine rotation speed of a torque converter included in the transmission, the rotation speed of the vehicle shaft of the transmission, and the like from the transmission, and the driver state estimation device 100a as speed information. Output to. Or the speed information output part 210 acquires the speed of the own vehicle from the speed sensor with which the vehicle was equipped, and outputs it to the driver state estimation apparatus 100a as speed information.

  The display unit 230 is, for example, the whole or part of the front window of the host vehicle. On the display unit 230, a visual stimulus (hereinafter referred to as a visual stimulus) is displayed by an action from the driver state estimating device 100a. The visual stimulus is information captured visually, such as a blinking light spot displayed on the display unit 230, a light spot that is simply displayed and disappears after a lapse of a certain time, and a symbol mark. The display unit 230 may be referred to as a display surface or a display area.

  The display unit 230 may be a head-up display (HUD) that displays visual stimuli on all or part of the front window of the host vehicle. The display unit 230 may be a head-mounted display (HMD) worn by the driver.

  The driver state estimation device 100a determines whether or not the host vehicle is stopped based on the acquired speed information. That is, the driver state estimation device 100a determines the traveling state of the host vehicle. If the driver state estimation device 100a determines that the vehicle is in a stopped state, the driver state estimation device 100a calculates the effective viewing angle of the driver. The driver state estimation device 100a estimates the driver's state change by comparing the calculated effective visual field angle with the effective visual field angle calculated at the past time point, and indicates the estimation result of the driver's state change. The result information is output to the driving support unit 240. Details of the internal configuration of the driver state estimation device 100a will be described later.

  The driving support unit 240 supports the driving operation of the driver based on the estimation result information. For example, when the estimation result information indicates “fatigue state”, the driving support unit 240 notifies the driver of a break. The notification content may be displayed as characters on the display unit 230 or the notification content may be output by voice.

  For example, when the estimation result information indicates “fatigue state” or “attentiveness state”, the driving support unit 240 performs control to increase the level of driving support for the driver. This control is a control for alerting or providing brake assistance.

  Specific examples of driving assistance related to alerting are shown. For example, in response to determining that the distance between the host vehicle and the passerby or the preceding vehicle is equal to or less than a predetermined reference value, the driving support unit 240 is notified so as to notify the driver of information indicating an alert. When the estimation result information indicates “fatigue state” or “attentiveness state”, the driving support unit 240 resets the predetermined reference value higher.

  A specific example of driving assistance related to brake assistance will be shown. For example, the driving support unit 240 is set in advance so as to automatically assist the brake when it is determined that the distance between the host vehicle and the passerby or the preceding vehicle is equal to or less than a predetermined reference value. In such a case, when the estimation result information indicates “fatigue state” or “attentiveness state”, the driving support unit 240 resets the predetermined reference value higher.

(Internal configuration of driver state estimation device 100a)
FIG. 2 shows a configuration diagram of the driver state estimation device 100a. The driver state estimation device 100a includes a stop state determination unit 101a, an image data acquisition unit 220, a visual stimulus control unit 102a, an effective visual field angle derivation unit 103a, a storage unit 104a, and a driver state estimation unit 105a. FIG. 3 shows a flowchart for explaining the operation of the driver state estimating apparatus 100a. Hereinafter, with reference to FIG.2 and FIG.3, the driver state estimation apparatus and method in 1st Embodiment are demonstrated.

  The stop state determination unit 101a acquires the speed information output from the speed information output unit 210. The stop state determination unit 101a determines whether the host vehicle is in a stop state based on the speed information (S101).

  The stop state refers to a case where the host vehicle is completely stopped, or a case where the vehicle is below a predetermined speed (the same applies to the second and subsequent embodiments). That is, the stop state determination unit 101a determines whether the speed of the host vehicle is equal to or lower than a predetermined speed. The predetermined speed can be arbitrarily set, and is, for example, 5 km / h.

  When it is determined that the host vehicle is in a stopped state, the stop state determining unit 101a outputs control information indicating that the host vehicle is in a stopped state to the visual stimulus control unit 102a (Yes in S101). When it is determined that the host vehicle is not in the stopped state, the stop state determining unit 101a waits until it is determined that the host vehicle is in the stopped state (No in S101).

  The image data acquisition unit 220 has an imaging system, captures an image of the driver (hereinafter, driver image) using the imaging system, and outputs the driver image data to the visual stimulus control unit 102a. The imaging system is, for example, a camera.

  The image data acquisition unit 220 captures a driver image every predetermined time, and the plurality of captured driver images are stored as driver image data in a memory included in the image data acquisition unit 220. Here, the unit of the predetermined time for imaging is expressed by fps (Frame Per Second). Note that, among the driver image data stored in the memory, the driver image data for which a predetermined time has elapsed from the time of imaging may be automatically deleted. It is desirable that the plurality of captured driver images be stored in the memory as driver image data to which the imaging time is added.

  Note that the image data acquisition unit 220 may be installed inside the host vehicle or outside the host vehicle. In addition, the image data acquisition unit 220 includes a memory, and a plurality of captured driver images are stored as driver image data in the memory included in the image data acquisition unit 220. However, the present invention is not limited thereto. That is, the memory is provided separately from the image data acquisition unit 220 and may be included in the driver state estimation device 100a, for example. In the following description, it is assumed that the image data acquisition unit 220 has a memory.

  The visual stimulus control unit 102a acquires driver image data from the image data acquisition unit 220 in response to acquiring control information indicating that the vehicle is in a stopped state from the stop state determination unit 101a. The visual stimulus control unit 102a derives the driver's line-of-sight direction and eye point based on the acquired driver image data. The visual stimulus control unit 102a specifies a reference determined based on the line-of-sight direction and the eyepoint in the display unit 230, and performs control for displaying the visual stimulus from the outer visual field direction to the inner visual field direction with respect to the reference ( S102).

  The driver image data acquired by the visual stimulus control unit 102a from the image data acquisition unit 220 is preferably the latest stored driver image data among the driver image data stored in the image data acquisition unit 220. .

  The driver image data acquired by the visual stimulus control unit 102a may be singular or plural. In a plurality of cases, the visual stimulus control unit 102a acquires the latest stored driver image data and the most recently stored driver image data. In the following description, it is assumed that the number is single.

  The visual stimulus control unit 102a detects the driver's pupil center point and corneal reflection center point from the driver image data, and based on the positional relationship between the pupil center point and the corneal reflection center point, the driver's line of sight Deriving the direction.

  The eye point refers to the center point between the left eye and the right eye of the driver. The visual stimulus control unit 102a detects the distance between the driver's pupil position and the camera from the driver image data, and derives an eye point based on the detected distance between the driver's pupil position and the camera.

  FIG. 4 shows a two-dimensional projection plane when the driver 400 faces the front direction of the host vehicle. In FIG. 4, the driver 400 is shown in a top view, and the front window 310 is all or part of the components of the display unit 230 shown in FIG.

As shown in FIG. 4, the visual stimulus is displayed at the intersection point between the front window 310 and the direction of the angle θ _k with the eye point 401 of the driver 400 as the center point and the line-of-sight direction 410 as a reference.

Here, θ _k is defined as the visual stimulus display angle. The parameter L _k is the horizontal distance between the line-of-sight direction 410 and the visual stimulus, and is defined as the visual stimulus display distance. The parameter k is an integer that satisfies 0 ≦ k ≦ n, and the parameter n is a positive integer.

  In addition, the line-of-sight direction 410 of the driver 400 is a linear direction that connects the eye point 401 and the gaze point of the driver 400. The line-of-sight direction is described as the front. The central visual field position 320 is an intersection position between the line-of-sight direction 410 and the front window 310. The visual field outward direction refers to a direction in which an area away from the central visual field position 320 is viewed, and the visual field inner direction refers to a direction in which an area near the central visual field position 320 is viewed.

  FIG. 5 is a flowchart illustrating a visual stimulus control method in the visual stimulus control unit 102a. Hereinafter, a control method in which the visual stimulus control unit 102a displays the visual stimulus on the display unit 230 will be described with reference to FIGS.

In FIG. 2, the visual stimulus control unit 102a sets an initial value θ ₀ of the visual stimulus display angle with the eye point 401 as the center point and the line-of-sight direction 410 as a reference (S102a), and displays the initial value θ ₀ as the visual stimulus display. The angle is set as the direction angle (S102b). The visual stimulus control unit 102a identifies the visual stimulus display position that is the intersection position of the set visual stimulus display direction and the front window 310 (S102c), and displays the visual stimulus at the visual stimulus display position (S102d).

Visual stimulation control unit 102a, by displaying the visual stimulus on the visual stimulus display position of the angle theta _0, determines whether or not the display end condition is satisfied (S102e). The case where the display end condition is satisfied refers to a case where the displayed visual stimulus is visually recognized by the driver or displayed at a preset visual stimulus display angle.

A method for determining whether or not the visual stimulus displayed at the visual stimulus display position at the angle θ ₀ has been visually recognized by the driver will be described below.

  The visual stimulus control unit 102a acquires driver image data from the image data acquisition unit 220 in response to the display of the visual stimulus at the visual stimulus display position. Here, the driver image data acquired from the image data acquisition unit 220 in response to the visual stimulus being displayed on the display unit 230 by the visual stimulus control unit 102 a is the driver image stored in the image data acquisition unit 220. Of the data, the driver image data stored most recently is desirable.

  Note that the driver image data acquired by the visual stimulus control unit 102a when the visual stimulus is displayed on the display unit 230 may be singular or plural. In a plurality of cases, the visual stimulus control unit 102a acquires the latest stored driver image data and the most recently stored driver image data. In the following description, it is assumed that the number is single.

  The visual stimulus control unit 102a derives the driver's line-of-sight direction based on the driver image data acquired from the image data acquisition unit 220, the angle of the driver's line-of-sight direction after the visual stimulus is displayed, and the visual stimulus It is determined whether the display angle is the same or substantially the same. Specifically, it is determined whether or not the difference between the angle of the line-of-sight direction and the visual stimulus display angle is equal to or less than a predetermined value set in advance. If it is determined that the driver has visually recognized and is equal to or greater than the predetermined value, it is determined that the driver has not visually recognized the visual stimulus.

When it is determined that the driver 400 has visually recognized the visual stimulus, the display end condition is satisfied, and the visual stimulus control unit 102a ends the display of the visual stimulus (S102e: Yes). On the other hand, when it is determined that the driver 400 does not visually recognize the visual stimulus, the display end condition is not satisfied (S102e: No), and the visual stimulus control unit 102a sets the visual stimulus display angle θ _k corresponding to the visual stimulus. The angle θ _{k + 1} is changed by a predetermined amount (S102f).

The visual stimulus control unit 102a resets the visual stimulus display direction angle as the angle of the visual stimulus display direction based on the visual stimulus display angle θ _{k + 1} after a predetermined time has elapsed (S102g). The visual stimulus control unit 102a specifies the visual stimulus display position 340 that is the intersection point between the reset direction of the visual stimulus display angle θ _{k + 1} and the front window 310 (S102c), and the specified visual stimulus display position 340 is determined. To control the display of visual stimuli.

Note that the display of the visual stimulus is based on the intersection of the direction of the initial value θ ₀ of the visual stimulus display angle and the intersection of the front window 310 with the direction of the preset visual stimulus display angle θ _n and the front window 310. Repeat until position 350. The visual stimulus display angle θ _n is an angle that is recognized as sufficiently recognizable as the visual field of the driver 400.

In this case, in the case where the display end condition shown in FIG. 5 is determined, if it is determined that the visual stimulus is displayed at the preset visual stimulus display angle θ _n , the display of the visual stimulus is ended. .

When it is determined that the visual stimulus is visually recognized by the driver, the visual stimulus control unit 102a outputs the driver's line-of-sight angle α _m (m: positive integer) to the effective viewing angle derivation unit 103a. α _m satisfies the condition of θ _n ≦ α _m ≦ θ ₀ . The subscript m indicates the time course, α _m is the driver's gaze direction angle acquired after α _m−1 , and α _m−1 is the driver acquired after α _m−2. Is the viewing direction angle. The parameter α ₀ is an effective viewing angle of the driver derived at the start of driving. Note that the driver's line-of-sight direction angle may be a visual stimulus display angle or an angle of the driver's line-of-sight direction derived from the driver image data.

When it is determined that the visual stimulus is displayed at the visual stimulus display angle θ _n of the display unit 230 and the driver does not recognize the visual stimulus, the driver does not visually recognize the visual stimulus. Is output to the effective viewing angle deriving unit 103a.

The effective visual field angle deriving unit 103a acquires the line-of-sight direction angle α _m output from the visual stimulus control unit 102a. The effective visual field deriving unit 103a derives the acquired gaze direction angle α _m as the driver's effective visual angle (S103).

The effective viewing angle deriving unit 103a generates effective viewing angle related information 50 based on the derived effective viewing angle α _m and outputs the information to the storage unit 104a. The storage unit 104a stores the effective viewing angle related information 50. On the other hand, the effective viewing angle deriving unit 103a outputs information related to the driver's effective viewing angle α _m to the driver state estimating unit 105a.

The effective viewing angle related information 50 will be described. The effective viewing angle related information 50 is derived by the effective viewing angle deriving unit 103a, the driver's effective viewing angle α _m , the derived date from which the effective viewing angle α _m is derived, the derived time, the derived location, the weather information, It includes driver information, operation start time, elapsed time from the operation start time, and the like.

  Specifically, the derivation date and the derivation time are specified by time information acquired from the timer unit (not shown) in the driver state estimation device 100a by the effective viewing angle derivation unit 103a. The derivation location and weather information are specified by information output from a navigation device (not shown) in the driver state estimation device 100a. For example, the driver information is stored in advance in the storage unit 104a. The driving start time is either when the engine of the host vehicle is started, when the engine of the host vehicle is started at the beginning of the day, or when input by a driver with a predetermined switch or the like. It is specified by time information output from a timer unit (not shown). The elapsed time from the operation start time is specified by the difference between the above-described derivation time and the operation start time.

The driver state estimation unit 105a acquires the driver's effective viewing angle α _m output from the effective viewing angle derivation unit 103a and the past effective viewing angle related information 50 stored in the storage unit 104a. The driver state estimating unit 105a outputs the effective visual field angle α _m output from the effective visual field angle deriving unit 103a and one or more effective visual field angles α _(m−p) included in the past effective visual field angle related information 50. And compare. The parameter p is an integer that satisfies 0 ≦ p ≦ m.

The driver state estimation unit 105a estimates the state of the driver based on at least one of a change amount, a change direction, and a change speed of the effective viewing angle α _{m with} respect to the effective viewing angle α _(m−p) . (S105). The driver state estimation unit 105a outputs the estimated driver state estimation result information to the driving support unit 240 (S106), and the operation of the driver state estimation device 100a ends.

A method by which the driver state estimating unit 105a estimates the driver's state will be described. FIG. 6 shows an example of contrast information for estimating the driver's state based on the change in the effective viewing angle α _m with respect to the effective viewing angle α _(mp) . The comparison information may be stored in advance by the driver state estimation unit 105a. Further, the storage unit 104a may store the information in advance, and the driver state estimation unit 105a may acquire the storage unit 104a.

FIG. 6 shows an example in which the parameter m is set to “4” and the parameter p is set to “3”. In FIG. 6, for example, when the value of the ratio of the effective viewing angle α ₄ , which is the latest information, to the effective viewing angle α ₀ at the start of operation is equal to or less than the threshold β ₁ , Formula 1 is established.

In FIG. 6, the effective viewing angle α ₄ , which is the latest information, is smaller than the effective viewing angle α ₁ at a predetermined time after the start of operation, and the effective viewing angle α derived after the predetermined time. _When smaller than ₃ , Equation 2 is established.

Further, in FIG. 6, the values of the ratio of the effective viewing angle α ₄ , which is the latest information, and all the effective viewing angles (α ₁ , α ₂ , α ₃ ) of the last three times are all from the threshold β ₂ . Is also very small, Equation 3 holds.

In FIG. 6, the threshold value β ₁ is a parameter for estimating whether or not the driver is in a fatigue state when the driver state estimation unit 105 a estimates the driver state. The threshold β ₂ is a parameter for estimating whether or not the driver is in a distracted state when the driver state is estimated by the driver state estimating unit 105a.

  If Equation 1 does not hold, the normal condition requirement is satisfied. In this case, the driver is estimated to be in a normal state, and the driver state estimation unit 105a outputs estimation result information indicating that the driver is in a normal state to the driving support unit 240.

  When Equation 1 is satisfied but Equation 2 and Equation 3 are not satisfied, the requirement for the state of caution is satisfied. In this case, it is estimated that the driver is in a state of caution, and the driver state estimation unit 105a outputs estimation result information indicating that the driver is in a state of caution to the driving support unit 240.

  If both Equation 1 and Equation 2 hold, the fatigue condition requirement is satisfied. In this case, it is estimated that the driver is in a fatigue state, and the driver state estimation unit 105a outputs estimation result information indicating that the driver is in a fatigue state to the driving support unit 240.

  If Equation 3 holds, the distraction requirement is satisfied. In this case, it is presumed that the driver is temporarily in a distracted state in which attention is paid to other visual objects, and the driver state estimating unit 105a estimates that the driver is in a distracted state. Information is output to the driving support unit 240. In addition, the estimation result information output from the driver state estimation unit 105a may be included in the effective viewing angle related information 50 and stored.

  A process in which the driver state estimating unit 105a estimates the driver state based on the comparison information shown in FIG. 6 will be described with reference to FIG. FIG. 7 is a flowchart illustrating details of the operation performed by the driver state estimation unit 105a.

The driver state estimation unit 105a acquires the effective viewing angle α _m output from the effective viewing angle deriving unit 103a and the past effective viewing angle related information 50 stored in the storage unit 104a. The driver state estimation unit 105a compares the effective viewing angle α _m output from the effective viewing angle derivation unit 103a with the effective viewing angle α _(m−p) included in the past effective viewing angle related information 50. .

  It is not efficient to estimate the driver state only with the absolute value of a single effective viewing angle derived at an arbitrary time. This is because the effective viewing angle varies depending on the driver or the driving situation when the effective viewing angle is derived. Therefore, in the first embodiment, based on an effective viewing angle derived at a certain time point and a relative change amount of at least two effective viewing angles of an effective viewing angle derived at the start of driving past that time. The driver state is estimated. It can be said that the driver state estimated based on the relative change amount of the at least two effective viewing angles is very accurate compared to the driver state estimated based on the single effective viewing angle. The same applies to each embodiment described later.

The driver state estimation unit 105a determines whether or not the value of the ratio of the effective viewing angle alpha _m is the latest information for the field of view angle alpha ₀ at the start of operation is the threshold value beta ₁ below (S105a). When the value of the ratio of the effective viewing angle α _m that is the latest information to the effective viewing angle α ₀ at the start of driving is larger than the threshold β ₁ (NO in S105a), the driver state estimation unit 105a indicates that the driver is normal. It is estimated that the vehicle is in a state (S105b), and the operation of the driver state estimation unit 105a ends.

When the value of the ratio of the effective viewing angle α _m that is the latest information with respect to the effective viewing angle α ₀ at the start of driving is equal to or less than the threshold β ₁ (YES in S105a), the driver state estimation unit 105a uses the latest information. Whether the value of the ratio between a certain effective viewing angle α _m and all the effective viewing angles (α _m−3 , α _m−2 , α _m−1 ) of the last three times is much smaller than the threshold β _2. Is determined (S105c).

The value of the ratio of the effective viewing angle α _m which is the latest information to all the effective viewing angles (α _m−3 , α _m−2 , α _m−1 ) of the last three times is much greater than the threshold β _2. (S105c YES), the driver state estimating unit 105a estimates that the driver is in a distracted state (S105d), and the operation of the driver state estimating unit 105a ends.

The values of the effective viewing angle α _m that is the latest information and all the effective viewing angles (α _m−3 , α _m−2 , α _m−1 ) of the last three times are values that are very smaller than the threshold β _2, respectively. If not (nO in S105c), the driver condition estimating unit 105a, a field of view angle alpha ₀ at the start of operation, an effective field angle alpha ₂ at an arbitrary time after the start of operation, effective is the latest information field The angle α _m is compared (S105e). This comparison, the driver state estimation unit 105a determines whether or not the date information is a valid field angle alpha _m gradually decreases from the effective viewing angle alpha ₀ at the start of operation (S105f).

When it is determined that the effective viewing angle α _m , which is the latest information, gradually decreases from the effective viewing angle α ₀ at the start of driving (YES in S105f), the driver state estimation unit 105a determines that the driver It is estimated that the vehicle is in a fatigued state (S105g), and the operation of the driver state estimating unit 105a ends.

When it is determined that the effective viewing angle α _m which is the latest information is not gradually decreasing from the effective viewing angle α ₀ at the start of driving (NO in S105f), the driver state estimation unit 105a. The driver state is estimated to be a state of caution (S105h), and the operation of the driver state estimation unit 105a ends.

  As described above, in the driver state estimation device 100a of the first embodiment, the state in which the host vehicle is stopped is determined, and the visual stimulus is displayed on the display unit 230 when it is determined that the vehicle is in a stopped state. The effective visual angle of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Since the driver's effective viewing angle is derived each time the host vehicle is determined to be in a stopped state, a predetermined condition (FIG. 6) that defines the relationship between the relative change amount of the effective viewing angle and the driver's state. The state of the driver is estimated.

  Therefore, the driver state estimation device 100a according to the first embodiment uses the estimation result information estimated by comparing the relative change amount of the driver's effective visual field angle derived over time as the driver's effective visual field. It can be identified as the cause that the angle changes with time. Furthermore, in the driver state estimation device 100a of the first embodiment, the effective viewing angle is derived when the host vehicle is in a stopped state. For this reason, the driver's load to see the central visual field direction is almost constant, and the cause of the effective visual angle changing with time is eliminated while eliminating the cause of the temporal change of the effective visual angle based on external factors. It can be specifically identified.

  In addition, the driver state based on the driver's effective viewing angle according to whether or not the driver's visual stimulus is visually recognized at a single point in time is not estimated. The driver state is estimated based on the relative change amount of the driver's effective visual field angle according to whether or not the driver's visual stimulation angle is visually recognized at at least two time points. For this reason, since the cause of the temporal relative change in the effective viewing angle is found from the estimation result information, the driver's estimation result information is highly accurate.

  Furthermore, the effective visual field angle of the driver is derived according to the presence or absence of the visual recognition of the driver when visual stimuli are sequentially displayed at predetermined time intervals from the visual field outer direction toward the visual field inner direction. For this reason, it is possible to estimate the driver's state in consideration of the process of changing the effective viewing angle, such as whether it is a distracted state, a fatigued state, or a cautionary state. Further, the estimated driver state can be utilized for more appropriate driving support.

  In the above description, the visual stimulus control unit 102a has been described as acquiring the latest stored driver image data among the driver image data stored in the image data acquisition unit 220. Absent. The driver state estimation device 100a includes a timer unit (not shown), manages the time when the control information indicating that the vehicle is stopped is acquired by the timer unit, and acquires the control information that indicates that the vehicle is stopped, or a predetermined time. The driver image data at a time before or after the time may be acquired from the image data acquisition unit 220.

  The image data acquisition unit 220 preferably includes a stereo camera or a compound eye camera that can acquire three-dimensional information. In the above description, the driver's line-of-sight direction and eye point are described as being calculated by the visual stimulus control unit 102a, but this is not a limitation. That is, the image data acquisition unit 220 may output a vector in the line-of-sight direction, or angles in the x, y, and z directions, and a position in the relative coordinates of the pupil position with respect to predetermined reference coordinates in the camera.

  In the above description, the driver image data acquired by the visual stimulus control unit 102a from the image data acquisition unit 220 is described as being singular. However, the present invention is not limited to this. That is, when there are a plurality of acquired driver image data, the visual stimulus control unit 102a derives the driver's line-of-sight direction and eye point based on the plurality of driver image data, or the driver performs a visual stimulus. It is possible to determine whether or not the user has visually confirmed. Specifically, the driver's line-of-sight direction and eye point derived from a plurality of driver image data may be averaged. Further, a change in the driver's line-of-sight direction may be detected based on a plurality of driver image data, and it may be determined whether the driver has recognized a visual stimulus based on the detection result.

(Second Embodiment)
A driver state estimation system 1b including a driver state estimation device 100b according to the second embodiment will be described with reference to FIG. FIG. 8 is a system configuration diagram showing a configuration of a driver state estimation system 1b including a driver state estimation device 100b according to the second embodiment. In FIG. 8, the same components as those in FIG. Hereinafter, in the second embodiment, components different from those in the first embodiment and operations of the components will be described, and description of the same contents will be omitted.

  As shown in FIG. 8, the driver state estimation system 1b includes a driver state estimation device 100b, a speed information output unit 210, a display unit 230, a driving support unit 240, a foreground imaging camera unit 250, and a communication unit 260. . The internal configuration of the driver state estimation device 100b will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100b.

  Since the display unit 230 and the driving support unit 240 have the same functions as the respective units in the first embodiment, description thereof is omitted.

  The foreground imaging camera unit 250 captures a foreground image every predetermined time, and a plurality of captured foreground image data is stored in a memory included in the foreground imaging camera unit 250. Here, the unit of the predetermined time for imaging is expressed in fps. Note that among the foreground image data stored in the memory, foreground image data for which a predetermined time has elapsed from the time of imaging may be automatically deleted. The foreground imaging camera unit 250 outputs the captured foreground image to the driver state estimation device 100b.

  The foreground imaging camera unit 250 may capture an image based on the control information output from the driver state estimation device 100b.

  In addition, although the foreground imaging camera unit 250 includes a memory and the plurality of captured foreground images are stored as foreground image data in the memory included in the foreground imaging camera unit 250, the present invention is not limited thereto. That is, the memory is provided separately from the foreground imaging camera unit 250 and may be included in the visual stimulus control unit 102b, for example. In the following description, it is assumed that the foreground imaging camera unit 250 has a memory.

  The communication unit 260 communicates wirelessly with a traffic light (not shown) based on an instruction from the driver state estimation device 100b, and acquires time information regarding the remaining time of the red light state in the traffic light. The communication unit 260 outputs time information regarding the remaining time of the red signal state to the driver state estimation device 100b.

(Internal configuration of driver state estimation device)
FIG. 9 shows an internal configuration of the driver state estimation device 100b of the second embodiment. As shown in FIG. 9, the driver state estimation device 100b includes a stop state determination unit 101b, an image data acquisition unit 220, a visual stimulus control unit 102b, an effective visual field angle derivation unit 103b, a storage unit 104b, and a driver state estimation unit 105b. And a traffic light state determination unit 106. FIG. 10 shows a flowchart for explaining the operation of the driver state estimating apparatus 100b shown in FIG. Hereinafter, the driver state estimation apparatus and method in the second embodiment will be described.

  The stop state determination unit 101b acquires the speed information output from the speed information output unit 210. The stop state determination unit 101b determines whether the host vehicle is in a stop state based on the speed information (S201). Here, the stop state refers to the case where the host vehicle is completely stopped or the vehicle is at a predetermined speed or less.

  When it is determined that the host vehicle is in a stopped state, the stop state determining unit 101b outputs control information indicating that the host vehicle is in a stopped state to the traffic light state determining unit 106 (Yes in S201). When it is determined that the host vehicle is not in the stopped state, the stop state determining unit 101b waits until it is determined that the host vehicle is in the stopped state (No in S201).

  The traffic light state determination unit 106 acquires foreground image data from the foreground imaging camera unit 250 in response to acquiring control information indicating that the vehicle is in a stopped state from the stop state determination unit 101b. Based on the acquired foreground image data, the traffic light state determination unit 106 determines whether or not the foreground image includes a traffic light. If the traffic light includes a traffic light, the traffic light state determination unit 106 determines whether or not the traffic light is in a red signal state ( S202).

  The foreground image data acquired by the traffic light state determination unit 106 from the foreground imaging camera unit 250 is preferably foreground image data stored most recently among the foreground image data stored in the foreground imaging camera unit 250.

  The foreground image data acquired by the traffic light state determination unit 106 may be singular or plural. In the case of a plurality of signals, the traffic light state determination unit 106 acquires the most recently stored foreground image data and the most recently stored foreground image data. In the following description, it is assumed that the number is single.

  When the traffic light state determination unit 106 determines that the traffic light of the captured foreground image is in a red signal state, the traffic light state determination unit 106 outputs control information indicating that the signal device wirelessly communicates with the traffic light to the communication unit 260. The traffic light state determination unit 106 acquires time information regarding the remaining time of the red signal state via the communication unit 260.

  The traffic light state determination unit 106 determines whether or not the remaining time is equal to or longer than a predetermined time set in advance based on the acquired time information regarding the remaining time of the red light state. When it is determined that the remaining time is equal to or longer than the predetermined time, the traffic light state determination unit 106 outputs control information indicating that the traffic light is in a red signal state to the visual stimulus control unit 102b (Yes in S202).

  The traffic light state determination unit 106 presets the remaining time of the red signal state when the red signal is not recognized in the foreground image data or when the remaining time of the red signal state is determined to be less than the predetermined time. The process waits until it is determined that the predetermined time is exceeded (No in S202).

  The visual stimulus control unit 102b acquires the operator image data from the image data acquisition unit 220 in response to the acquisition of the control information indicating the red signal state from the traffic light state determination unit 106. Since the operation of the visual stimulus control unit 102b has been described in the first embodiment, the description of the subsequent processing flow is omitted (S102).

  In addition, since the functions of the image data acquisition unit 220, the effective viewing angle derivation unit 103b, the storage unit 104b, and the driver state estimation unit 105b have been described in the first embodiment, description thereof will be omitted.

  As described above, in the driver state estimation device 100b according to the second embodiment, the visual stimulus is displayed on the display unit when it is determined that the state of the red signal of the traffic light continues for a predetermined time or more in addition to the host vehicle being stopped. 230. The effective visual angle of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Since the driver's effective viewing angle is derived each time the host vehicle is determined to be in a stopped state, the relationship between the relative change amount of the effective viewing angle and the driver state is determined according to a predetermined condition. The driver's condition is estimated.

  Therefore, the driver state estimation device 100b according to the second embodiment is more sensitive to the situation in which the host vehicle is in a traveling state during the process of deriving the effective viewing angle than the driver state estimation device 100a according to the first embodiment. In order to prevent this, the driver's state can be estimated in a safer state.

(Third embodiment)
A driver state estimation system 1c including a driver state estimation device 100c according to a third embodiment will be described with reference to FIG. FIG. 11 is a system configuration diagram showing a configuration of a driver state estimation system 1c including a driver state estimation device 100c according to the third embodiment. In FIG. 11, the same reference numerals are used for the same components as in FIG. Hereinafter, in the third embodiment, components different from those in the first embodiment and operations of the components will be described, and description of the same contents will be omitted.

  As illustrated in FIG. 11, the driver state estimation system 1c includes a driver state estimation device 100c, a speed information output unit 210, a display unit 230, a driving support unit 240, and an illuminance sensor 270. The internal configuration of the driver state estimation device 100c will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100c.

  Since the display unit 230 and the driving support unit 240 have the same functions as the respective units in the first embodiment, description thereof is omitted.

  The illuminance sensor 270 is a sensor installed in the host vehicle, and measures the brightness around the host vehicle every predetermined time. The illuminance sensor 270 outputs information related to the measured brightness around the host vehicle to the driver state estimation device 100c. The illuminance sensor 270 may measure the brightness around the host vehicle based on an instruction from the driver state estimation device 100c.

(Internal configuration of driver state estimation device)
FIG. 12 shows an internal configuration of the driver state estimation device 100c of the third embodiment. As shown in FIG. 12, the driver state estimation device 100c includes a stop state determination unit 101c, an image data acquisition unit 220, a visual stimulus control unit 102c, an effective visual field angle derivation unit 103c, a storage unit 104c, and a driver state estimation unit 105c. And a contrast adjusting unit 107. Hereinafter, the driver state estimation apparatus and method in the third embodiment will be described.

  The stop state determination unit 101c acquires the speed information output from the speed information output unit 210. The stop state determination unit 101c determines whether the host vehicle is in a stop state based on the speed information. Here, the stop state refers to the case where the host vehicle is completely stopped or the vehicle is at a predetermined speed or less.

  The contrast adjustment unit 107 acquires information about the brightness around the host vehicle output from the illuminance sensor 270. The contrast adjustment unit 107 makes the brightness contrast between the brightness around the host vehicle and the visual stimulus displayed by the visual stimulus control unit 102c constant according to the acquired information about the brightness around the host vehicle. To control.

  Specifically, the contrast adjustment unit 107 visually adjusts the ratio of the visual stimulus displayed by the visual stimulus control unit 102c and the brightness of the surroundings of the host vehicle output by the illuminance sensor 270 to be constant. Adjust the display brightness of the stimulus. For example, when the surroundings of the host vehicle are bright during the daytime or the like, adjustment is performed so that the luminance of the visual stimulus is increased. On the other hand, when the surroundings of the host vehicle are dark at night or the like, an adjustment is made so that the luminance of the visual stimulus is reduced.

  The contrast adjustment unit 107 outputs information regarding the brightness of the adjusted visual stimulus and control information indicating that the visual stimulus is displayed according to the adjusted brightness of the visual stimulus to the visual stimulus control unit 102c.

  The visual stimulus control unit 102c acquires control information indicating that the visual stimulus is displayed according to the luminance of the visual stimulus output by the contrast adjusting unit 107. In accordance with the acquired control information, the visual stimulus control unit 102b changes the luminance of the visual stimulus adjusted by the contrast adjustment unit 107 in order at predetermined time intervals from the driver's outer field of view to the inner field of view. The corresponding visual stimulus is displayed on the display unit 230.

  Note that the functions of the visual stimulus control unit 102c, the effective visual field angle deriving unit 103c, the storage unit 104c, and the driver state estimation unit 105c are the same as the functions in the first embodiment, and thus description thereof is omitted.

  As described above, in the driver state estimation device 100c of the third embodiment, the contrast of the visual stimulus is adjusted and displayed on the display unit 230 in consideration of the brightness around the host vehicle. The driver state estimation device 100c according to the embodiment can estimate the driver's state more accurately than the driver state estimation device 100a according to the first embodiment.

(Fourth embodiment)
A driver state estimation system 1d including a driver state estimation device 100d according to a fourth embodiment will be described with reference to FIG. FIG. 13 is a system configuration diagram illustrating a configuration of a driver state estimation system 1d including a driver state estimation device 100d according to the fourth embodiment. In FIG. 13, the same components as those in FIG. Hereinafter, in the fourth embodiment, components different from those in the first embodiment and operations of the components will be described, and description of the same contents will be omitted.

  As illustrated in FIG. 13, the driver state estimation system 1d includes a driver state estimation device 100d, a speed information output unit 210, a display unit 230, a driving support unit 240, and a foreground imaging camera 250. The internal configuration of the driver state estimation device 100d will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100d.

  Since the display unit 230 and the driving support unit 240 have the same functions as the respective units in the first embodiment, description thereof is omitted.

  The function of the foreground imaging camera unit 250 is the same as that described in the second embodiment, and a description thereof will be omitted.

(Internal configuration of driver state estimation device)
FIG. 14 shows the internal structure of the driver state estimation device 100d of the fourth embodiment. As shown in FIG. 14, the driver state estimation device 100d includes a stop state determination unit 101d, an image data acquisition unit 220, a visual stimulus control unit 102d, an effective visual field angle derivation unit 103d, a storage unit 104d, and a driver state estimation unit 105d. And a threshold adjustment unit 108. Hereinafter, the driver state estimation apparatus and method in the fourth embodiment will be described.

  The stop state determination unit 101d acquires the speed information output from the speed information output unit 210. The stop state determination unit 101d determines whether or not the host vehicle is in a stop state based on the speed information. Here, the stop state refers to the case where the host vehicle is completely stopped or the vehicle is at a predetermined speed or less.

  The functions of the image data acquisition unit 220, the visual stimulus control unit 102c, the effective viewing angle derivation unit 103c, and the storage unit 104c are the same as the functions in the first embodiment, and thus the description thereof is omitted.

The threshold adjustment unit 108 acquires the foreground image data output from the foreground imaging camera 250. The threshold adjustment unit 108 adjusts the threshold β ₁ for estimating the driver state according to the complexity of the foreground image indicated by the acquired foreground image data. Specifically, the threshold adjustment unit 108 extracts the spatial frequency of the foreground image based on the foreground image data output from the foreground imaging camera unit 250. The spatial frequency of the foreground image is a parameter indicating the complexity of the foreground image. Generally, when the spatial frequency of an image is low, the image is not complicated. Further, when the spatial frequency of the image is high, the image is considered to be complicated. The threshold adjustment unit 108 determines an estimated threshold corresponding to the extracted spatial frequency according to the relationship between the spatial frequency of the foreground image and the estimated threshold. This threshold value β ₁ is the same as that in the first embodiment.

Relationship between the spatial frequency and the threshold value beta ₁ of the foreground image will be described with reference to FIG. 15. FIG. 15 is a diagram illustrating an example of the relationship between the threshold and the spatial frequency of the foreground image. 15, when the low spatial frequencies of the foreground image, the threshold beta ₁ for estimating a driver's state is adjusted to a higher value. On the other hand, when the high spatial frequencies of the foreground image, the threshold beta ₁ is adjusted to a small value. The threshold adjustment unit 108 outputs information on the adjusted threshold to the driver state estimation unit 105d. In addition, the threshold adjustment processing by the threshold adjustment unit 108 may be performed before the driver state estimation processing by the driver state estimation unit 105d.

The driver state estimation unit 105d acquires information on the threshold value output from the threshold value adjustment unit 108. The driver state estimation unit 105d acquires the effective visual field angle α _m of the driver output from the effective visual field angle deriving unit 103d, and the past effective visual field angle related information 50 from the storage unit. Since the subsequent processing is the same as that of the first embodiment, description thereof is omitted.

  As described above, the driver state estimation device 100d according to the fourth embodiment is effective by adjusting the threshold for estimating the driver state according to the complexity of the foreground image of the subject vehicle to be captured. Since the driver state can be estimated in consideration of the influence of the difference in complexity of the foreground when the viewing angle is derived, the driver state can be estimated more accurately.

  Note that the relationship between the spatial frequency of the foreground image and the estimation threshold is not limited to the relationship illustrated in FIG. For example, the estimated threshold value may be adjusted by a predetermined amount when the extracted spatial frequency value is within a predetermined range.

(Fifth embodiment)
Next, a driver state estimation system 1e including a driver state estimation device 100e according to a fifth embodiment will be described with reference to FIG. FIG. 16 is a system configuration diagram illustrating a configuration of a driver state estimation system 1e including a driver state estimation device 100e according to the fifth embodiment. In FIG. 16, the same components as those in FIG. Hereinafter, in the fifth embodiment, components different from those in the first embodiment and operations of the components will be described, and description of the same contents will be omitted.

  As illustrated in FIG. 16, the driver state estimation system 1e includes a driver state estimation device 100e, a speed information output unit 210, a display unit 230, a driving support unit 240, and a navigation device 280. The internal configuration of the driver state estimating device 100e is the same as that of the driver state estimating device 100a shown in FIG.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100e.

  Since the display unit 230 and the driving support unit 240 have the same functions as the respective units in the first embodiment, description thereof is omitted.

  The navigation device 280 is installed in the host vehicle and recognizes the traveling road information on which the host vehicle is traveling based on the GPS (Global Positioning System) function during the operation of the vehicle. It outputs to the driver state estimation part 100e. Moreover, the navigation apparatus 280 may output the derivation | leading-out place of an effective visual field angle, and a weather information to the driver | operator state estimation apparatus 100e.

(Internal configuration of driver state estimation device)
As shown in FIG. 16, the driver state estimation device 100e includes a stop state determination unit 101e, an image data acquisition unit 220, a visual stimulus control unit 102e, an effective visual field angle derivation unit 103e, a storage unit 104e, and a driver state estimation unit 105e. Have Hereinafter, description of the component which performs the same operation | movement as each component of the driver state estimation apparatus 100a of 1st Embodiment is abbreviate | omitted.

  The stop state determination unit 101e acquires the speed information of the host vehicle output from the speed information output unit 210 and the traveling road information output from the navigation device 280, respectively. The stop state determination unit 101e determines whether or not the vehicle is traveling on a predetermined road at a predetermined speed based on the speed information and the traveling road information. Here, the predetermined road is a road where the host vehicle is expected to travel at a substantially constant speed like an expressway. The predetermined speed range is, for example, 80 [km] / h [hour] ± 5 [km] / h [hour].

  When the stop state determination unit 101e determines that the vehicle is traveling on a predetermined road at a predetermined substantially constant speed, the stop state determination unit 101e displays control information for displaying a visual stimulus as a visual stimulus control unit. To 102e. If the vehicle is not in a state of traveling on a predetermined road at a predetermined substantially constant speed, the stop state determination unit 101e waits until it is determined that the vehicle is traveling on a predetermined road at a predetermined speed. .

  The functions of the image data acquisition unit 220, the visual stimulus control unit 102c, the effective visual field angle derivation unit 103c, the storage unit 104c, and the driver state estimation unit 105c are the same as the functions in the first embodiment, respectively. Is omitted.

  Note that the storage unit 104e may store information such as “traveling speed” and “traveling road” as the effective viewing angle related information 50 in addition to the effective viewing angle related information 50 of the first embodiment.

The driver state estimation unit 105e estimates the driver state based on the derived relative change amount of the effective viewing angle, for example, using the comparison table shown in FIG. Further, “effective viewing angle α ₀ at the start of driving” in the first embodiment is read as “effective viewing angle α ₀ at the start of driving on the highway” in the fifth embodiment.

  As described above, in the driver state estimation device 100e of the fifth embodiment, a state in which the host vehicle is traveling on a predetermined road at a predetermined speed is determined, and visual stimulation is generated when it is determined as the state. It is displayed on the display unit 230. The effective visual field of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Each time it is determined that the host vehicle is traveling on a predetermined road at a predetermined speed, the driver's effective field of view is derived. Therefore, the relationship between the relative change in the effective field of view and the driver state The driver's state is estimated according to the predetermined condition (see FIG. 6).

  Therefore, the driver state estimation device 100e according to the fifth embodiment uses the driver state estimation result information estimated by comparing the relative change amount of the driver's effective visual field derived with time to the driver's state estimation result information. It can be identified as the cause of the change in the effective visual field over time. Furthermore, in the driver state estimation device 100e of the fifth embodiment, the effective visual field is derived when the host vehicle is traveling on a predetermined road at a predetermined speed. For this reason, the load of viewing the driver's central visual field direction is almost constant, and the effective visual field is not affected by changes in the effective visual field over time due to external factors even when the vehicle is not driving, such as when driving on an expressway. It is possible to specifically specify the cause of the change with time.

(Sixth embodiment)
A driver state estimation system 1f including a driver state estimation device 100f according to a sixth embodiment will be described with reference to FIG. FIG. 17 is a system configuration diagram illustrating a configuration of a driver state estimation system 1f including a driver state estimation device 100f according to the sixth embodiment. In FIG. 17, the same components as those in FIGS. 1 to 14 are denoted by the same reference numerals. Hereinafter, in the sixth embodiment, constituent elements different from those in the first to fifth embodiments and operations of the constituent elements will be described, and description thereof will be omitted unless they are the same.

  As shown in FIG. 17, the driver state estimation system 1f includes a driver state estimation device 100f, a speed information output unit 210, a display unit 230, a driving support unit 240, a foreground imaging camera unit 250, a communication unit 260, and illuminance. A sensor 270 is included. The internal configuration of the driver state estimation device 100f will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100f.

  The display unit 230, the driving support unit 240, the foreground imaging camera unit 250, the communication unit 260, and the illuminance sensor 270 have the same functions as the units in the first to fourth embodiments, and thus description thereof is omitted. .

  The driver state estimation device 100f includes a stop state determination unit 101f, an image data acquisition unit 220, a visual stimulus control unit 102f, an effective viewing angle derivation unit 103f, a storage unit 104f, a driver state estimation unit 105f, a traffic light state determination unit 106, A contrast adjustment unit 107 is included.

  Description of functions of the stop state determination unit 101f, the image data acquisition unit 220, the visual stimulus control unit 102f, the effective visual field angle derivation unit 103f, the storage unit 104f, the driver state estimation unit 105f, the traffic light state determination unit 106, and the contrast adjustment unit 107 Since it was demonstrated in Embodiment 1 thru | or Embodiment 4, description is abbreviate | omitted.

  As described above, in the driver state estimation device 100f of the sixth embodiment, the visual stimulus is displayed on the display unit when it is determined that the state of the red signal of the traffic light continues for a predetermined time or more in addition to the host vehicle being stopped. 230. The effective visual angle of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Since the driver's effective viewing angle is derived each time the host vehicle is determined to be in a stopped state, the relationship between the relative change amount of the effective viewing angle and the driver state is determined according to a predetermined condition. The driver's condition is estimated.

  Therefore, the driver state estimation device 100f according to the sixth embodiment is more sensitive to the situation in which the host vehicle is in a traveling state during the derivation process of the effective viewing angle than the driver state estimation device 100a according to the first embodiment. In order to prevent this, the driver's state can be estimated in a safer state.

  Furthermore, in order to adjust the contrast of the visual stimulus and display it on the display unit 230 in consideration of the brightness around the host vehicle, the driver state estimation device 100f of the sixth embodiment is the same as that of the first embodiment. Compared with the driver state estimation device 100a, the state of the driver can be estimated more accurately.

(Seventh embodiment)
A driver state estimation system 1g including a driver state estimation device 100g according to a seventh embodiment will be described with reference to FIG. FIG. 18 is a system configuration diagram illustrating a configuration of a driver state estimation system 1g including a driver state estimation device 100g according to the seventh embodiment. In FIG. 18, the same reference numerals are used for the same components as those in FIGS. 1 to 14. Hereinafter, in the seventh embodiment, constituent elements different from those in the first to fourth embodiments and operations of the constituent elements will be described, and descriptions thereof are omitted if they are not the same.

  As illustrated in FIG. 18, the driver state estimation system 1g includes a driver state estimation device 100g, a speed information output unit 210, a display unit 230, a driving support unit 240, a foreground imaging camera unit 250, and a communication unit 260. . The internal configuration of the driver state estimation device 100g will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100g.

  The display unit 230, the driving support unit 240, the foreground imaging camera unit 250, and the communication unit 260 have the same functions as the units in the first to sixth embodiments, and thus the description thereof is omitted.

  The driver state estimation device 100g includes a stop state determination unit 101g, an image data acquisition unit 220, a visual stimulus control unit 102g, an effective viewing angle derivation unit 103g, a storage unit 104g, a driver state estimation unit 105g, a traffic light state determination unit 106, A threshold adjustment unit 108 is included.

  Description of functions of the stop state determination unit 101g, the image data acquisition unit 220, the visual stimulus control unit 102g, the effective viewing angle derivation unit 103g, the storage unit 104g, the driver state estimation unit 105g, the traffic light state determination unit 106, and the threshold adjustment unit 108 Since it was demonstrated in Embodiment 1 thru | or Embodiment 4, description is abbreviate | omitted.

  As described above, in the driver state estimation device 100g according to the seventh embodiment, the visual stimulus is displayed on the display unit when it is determined that the state of the red signal of the traffic light continues for a predetermined time or more in addition to the host vehicle being stopped. 230. The effective visual angle of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Since the driver's effective viewing angle is derived each time the host vehicle is determined to be in a stopped state, the relationship between the relative change amount of the effective viewing angle and the driver state is determined according to a predetermined condition. The driver's condition is estimated.

  Therefore, the driver state estimation device 100g according to the seventh embodiment is more sensitive to the situation in which the host vehicle is in a traveling state during the process of deriving the effective viewing angle than the driver state estimation device 100a according to the first embodiment. In order to prevent this, the driver's state can be estimated in a safer state.

  Furthermore, by adjusting the threshold for estimating the driver's state according to the complexity of the foreground image of the subject vehicle being imaged, the effect of the difference in foreground complexity when deriving the effective viewing angle was taken into account Since the driver state can be estimated, the driver state can be estimated more accurately.

(Eighth embodiment)
A driver state estimation system 1h including a driver state estimation device 100h according to an eighth embodiment will be described with reference to FIG. FIG. 19 is a system configuration diagram showing a configuration of a driver state estimation system 1h including a driver state estimation device 100h according to the eighth embodiment. 19, the same reference numerals are used for the same components as those in FIGS. Hereinafter, in the eighth embodiment, components that are different from those in the first to fourth embodiments and operations of the components will be described, and descriptions of components that are not the same will be omitted.

  As illustrated in FIG. 19, the driver state system 1h includes a driver state estimation device 100h, a speed information output unit 210, a display unit 230, a driving support unit 240, a foreground imaging camera unit 250, and an illuminance sensor 270. The internal configuration of the driver state estimation device 100h will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100h.

  The display unit 230, the driving support unit 240, the foreground imaging camera unit 250, and the illuminance sensor 270 have the same functions as the units in the first to fourth embodiments, and thus the description thereof is omitted.

  The driver state estimation device 100h includes a stop state determination unit 101h, an image data acquisition unit 220, a visual stimulus control unit 102h, an effective viewing angle derivation unit 103h, a storage unit 104h, a driver state estimation unit 105h, a contrast adjustment unit 107, and A threshold adjustment unit 108 is included.

  The driver state estimation device 100h includes a stop state determination unit 101h, an image data acquisition unit 220, a visual stimulus control unit 102h, an effective viewing angle derivation unit 103h, a storage unit 104h, a driver state estimation unit 105h, a contrast adjustment unit 107, and Since the description of the function of the threshold adjustment unit 108 has been given in the first to fourth embodiments, the description thereof is omitted.

  Therefore, the driver state estimation device 100h according to the eighth embodiment adjusts the threshold for estimating the driver state according to the complexity of the foreground image of the subject vehicle to be captured, thereby adjusting the effective viewing angle. Since the driver state can be estimated in consideration of the influence of the difference in complexity of the foreground at the time of derivation, the driver state can be estimated more accurately.

  Furthermore, by adjusting the threshold for estimating the driver's state according to the complexity of the foreground image of the subject vehicle being imaged, the effect of the difference in foreground complexity when deriving the effective viewing angle was taken into account Since the driver state can be estimated, the driver state can be estimated more accurately.

(Ninth embodiment)
A driver state estimation system 1k including a driver state estimation device 100k according to a ninth embodiment will be described with reference to FIG. FIG. 20 is a system configuration diagram illustrating a configuration of a driver state estimation system 1k including a driver state estimation device 100k according to the ninth embodiment. 20, the same components as those in FIGS. 1 to 14 are denoted by the same reference numerals. Hereinafter, in the ninth embodiment, constituent elements different from those in the first to fourth embodiments and operations of the constituent elements will be described, and description thereof will be omitted if they are not the same.

  As shown in FIG. 20, the driver state estimation system 1k includes a driver state estimation device 100k, a speed information output unit 210, a display unit 230, a driving support unit 240, a foreground imaging camera unit 250, a communication unit 260, and an illuminance. A sensor 270 is included. The internal configuration of the driver state estimation device 100h will be described later.

  The speed information output unit 210 has the same function as the speed information output unit 210 in the first embodiment, and outputs speed information related to the speed of the host vehicle to the driver state estimation device 100k.

  The display unit 230, the driving support unit 240, the foreground imaging camera unit 250, the communication unit 260, and the illuminance sensor 270 have the same functions as the units in the first to fourth embodiments, and thus description thereof is omitted. .

  The driver state estimation device 100k includes a stop state determination unit 101k, an image data acquisition unit 220, a visual stimulus control unit 102k, an effective viewing angle derivation unit 103k, a storage unit 104k, a driver state estimation unit 105k, a contrast adjustment unit 107, and A threshold adjustment unit 108 is included.

  The driver state estimation device 100k includes a stop state determination unit 101k, an image data acquisition unit 220, a visual stimulus control unit 102k, an effective viewing angle derivation unit 103k, a storage unit 104k, a driver state estimation unit 105k, a contrast adjustment unit 107, and Since the description of the function of the threshold adjustment unit 108 has been given in the first to fourth embodiments, the description thereof is omitted.

  As described above, in the driver state estimation device 100k according to the ninth embodiment, the visual stimulus is displayed on the display unit when it is determined that the state of the red signal of the traffic light continues for a predetermined time or more in addition to the host vehicle being stopped. 230. The effective visual angle of the driver when the displayed visual stimulus is visually recognized by the driver is derived and stored. Since the driver's effective viewing angle is derived each time the host vehicle is determined to be in a stopped state, the relationship between the relative change amount of the effective viewing angle and the driver state is determined according to a predetermined condition. The driver's condition is estimated.

  Therefore, the driver state estimation device 100k according to the ninth embodiment is compared with the driver state estimation device 100a according to the first embodiment in a situation where the host vehicle is in a traveling state during the process of deriving the effective viewing angle. In order to prevent this, the driver's state can be estimated in a safer state.

  Therefore, the driver state estimation device 100k according to the ninth embodiment adjusts the threshold for estimating the driver state according to the complexity of the foreground image of the subject vehicle to be imaged, thereby adjusting the effective viewing angle. Since the driver state can be estimated in consideration of the influence of the difference in complexity of the foreground at the time of derivation, the driver state can be estimated more accurately.

  Furthermore, by adjusting the threshold for estimating the driver's state according to the complexity of the foreground image of the subject vehicle being imaged, the effect of the difference in foreground complexity when deriving the effective viewing angle was taken into account Since the driver state can be estimated, the driver state can be estimated more accurately.

  While various embodiments have been described above with reference to the accompanying drawings, it goes without saying that the input device of the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  In addition, the condition for the stop state determination units 101a, 101b, 101c, 101d, and 101e in the above-described embodiments to start displaying visual stimuli is a state in which the amount of visual information per unit time is a predetermined amount. It doesn't matter. For example, it is assumed that the threshold adjustment unit 108 determines that the spatial frequency in the foreground image in the traveling direction of the host vehicle captured by the foreground imaging camera unit 250 is within a predetermined range. Only in this case, the stop state determination unit 101d may output control information indicating that the visual stimulus control unit 102d displays the visual stimulus to the visual stimulus control unit 102d.

  The contrast adjustment unit 107 uses the foreground imaging camera 250 instead of the illuminance sensor 270, and outputs the foreground image in the traveling direction of the host vehicle output by the foreground imaging camera unit 250 and the luminance or color of the foreground image. Get information about. The contrast adjustment unit 107 may perform control to make the luminance contrast or color contrast between the foreground and the visual stimulus constant based on the acquired foreground image and information on the luminance or color of the foreground image. Absent.

  Specifically, the contrast adjustment unit 107 adjusts the luminance or color of the displayed visual stimulus based on information about the luminance or color of the foreground image output by the foreground imaging camera unit 250. The contrast adjustment unit 107 outputs information regarding the brightness or color of the adjusted visual stimulus to the visual stimulus control unit 102d. The visual stimulus control unit 102d displays the visual stimulus on the display unit 230 based on the information regarding the brightness or color of the adjusted visual stimulus.

Incidentally, the visual stimulation control unit 102a specifies the visual stimulus display positions 330 apart as the initial value L ₀ of the preset visual stimulus display distance from the center view position 320 of the driver 400, the specified visual stimulus display A visual stimulus may be displayed at position 330.

Note that when the driver 400 visually recognizes the visual stimulus displayed by the visual stimulus control unit 102a, the visual stimulus control unit 102a ends the display of the visual stimulus. However, the visual stimulus displayed by the visual stimulus control unit 102a may not be visually recognized by the driver. In this case, after a predetermined time has elapsed, the visual stimulus control unit 102a specifies the visual stimulus display position 340 that is separated by the visual stimulus display distance L ₁ (<visual stimulus display position L ₀ ). The visual stimulus control unit 102a may display a visual stimulus at the specified visual stimulus display position 340.

When the visual stimulus displayed by the visual stimulus control unit 102a is not yet visually recognized by the driver 400, the visual stimulus control unit 102a sequentially turns further toward the inside of the visual field than the visual stimulus display position 340 after a predetermined time has elapsed. Display visual stimuli. The display of the visual stimulus is repeated to appear on the central field visual stimulus display preset from the position 320 a distance L _n as distant visual stimulus display position 350 of the driver 400. The visual stimulus display distance L _n is at a position sufficiently close to the center visual field position 320 of the driver 400. The parameter L _k [m] indicating the visual stimulus display distance satisfies L _n ≦ L _k ≦ L ₀ .

In the first embodiment, the visual stimulus control unit 102a displays the visual stimulus displayed on the display unit 230 when the driver's line-of-sight angle α _m and the visual stimulus display angle θ _k are substantially the same. Is determined to be visually recognized by the driver. However, the determination of whether or not the visual stimulus is visually recognized by the driver is not limited to the case where the angle of the driver's line-of-sight direction and the visual stimulus display angle are substantially the same.

For example, the visual stimulation control unit 102a determines that the central visual field position 320 and the distance between the position of the driver's line of sight direction of the front window 310, or approximately the same or not and the visual stimulus display position L _k. When it is determined that the visual stimulus control unit 102a is substantially the same, the visual stimulus control unit 102a may determine that the driver has visually recognized the visual stimulus displayed on the display unit 230. The distance between the center visual field position 320 and the position of the front window 310 in the driver's line-of-sight direction is derived by the visual stimulus control unit 102a including the same configuration as the line-of-sight direction detection device disclosed in Patent Document 1. .

In order for Formula 3 shown in FIG. 6 to be established, the current effective visual field α _m derived by the effective visual field angle deriving unit 103a and all three effective visual fields (α _m−3 , α _m−2 , It was assumed that the ratio to α _m−1 ) was much smaller than the threshold value β ₂ . However, when the value of the ratio between the current effective visual field α _m and at least one of the last three effective visual fields (α _m-3 , α _m-2 , α _m-1 ) is much smaller than the threshold β ₂ In addition, Formula 3 shown in FIG. 6 may be established.

  The eye point 401 may be a position such as the center of the eyelips defined by JIS D0021 or the center of the eyelips of the left eye or the right eye. Further, the central visual field direction 410 may be a front center of an eye point defined by JIS D0021 or a standard eye point such as a left eye or a right eye.

  Further, in each of the embodiments described above, the line-of-sight direction 410 has been described as a linear direction that connects the eye point 401 of the driver 400 and the gaze point of the driver 400, but is not limited thereto. For example, the line-of-sight direction 410 is set in the direction of the position of the tail lamp of the preceding vehicle in the image based on the eye point 401 by image recognition of a peripheral image captured by a camera (not shown) that captures the periphery of the host vehicle. May be set. Alternatively, the line-of-sight direction 410 may be set to the direction of an object that is frequently watched by the driver while the host vehicle is stopped, with the eyepoint 401 as a reference, such as the position of a traffic light in the surrounding image. good.

  In each of the above-described embodiments, it has been described that the driving support unit 240 is not included in the driver state estimation device 100. However, the driving support unit 240 may be included in each driver state estimation device 100.

  In each embodiment, the stop state determination unit 101 does not need to output control information for displaying a visual stimulus to the visual stimulus control unit 102 every time the host vehicle is in a stop state. For example, the stop state determination unit 101 outputs control information indicating that a visual stimulus is displayed to the visual stimulus control unit 102 when the host vehicle is in a stopped state at the start of driving or every time a predetermined time elapses after starting driving. May be.

  Further, the display position of the visual stimulus displayed by the visual stimulus control unit 102 is not limited to the display position described in each of the above-described embodiments, and the display is performed in order from the outer side of the visual field toward the inner side of the visual field. Just do it. For example, in FIG. 2 or FIG. 3, in each of the above-described embodiments, the central visual field position 320 and the visual stimulus display positions 330 to 350 are arranged substantially linearly. However, the central visual field position 320 and the visual stimulus display positions 330 to 350 may be arranged randomly rather than linearly.

  Further, the visual stimulus may be displayed a plurality of times at the central visual field position 320 and the visual stimulus display positions 330 to 350 at substantially the same visual stimulus display angle. For example, the visual stimulus may be displayed at a position close to the driver after the visual stimulus is displayed at a position far from the driver at substantially the same visual stimulus display angle.

  In the second embodiment described above, it has been described that the process of adjusting the display brightness of the visual stimulus by the contrast adjusting unit 107 and the process of adjusting the threshold by the threshold adjusting unit 108 are performed independently. However, the adjustment process by the contrast adjustment unit 107 and the adjustment process by the estimated threshold value adjustment unit 109 may have a predetermined correlation.

  For example, when the display brightness of the visual stimulus is adjusted by the contrast adjusting unit 107 as the predetermined correlation, the adjustment of the estimated threshold by the threshold adjusting unit 108 may be unnecessary. Specifically, the contrast adjustment unit 107 outputs adjustment end information indicating that the display brightness of the visual stimulus is adjusted to the visual stimulus control unit 102. At this time, the visual stimulus control unit 102 acquires the adjustment end information output by the contrast adjustment unit 107, and outputs a predetermined estimated threshold value to the driver state estimation unit 105 without adjusting the estimated threshold value. May be.

  Furthermore, in each of the above-described embodiments, the size of the visual stimulus (light spot) displayed by the visual stimulus control unit 102 is constant, but the size may not be constant. For example, a visual stimulus having a larger diameter as a visual stimulus displayed at a position distant from the line-of-sight position 320 may be displayed using a cortical enlargement coefficient depending on a visual stimulus display angle with respect to the line-of-sight direction 410. .

In the first embodiment, the relative amount of change between the effective visual field α ₀ at the start of driving and the effective visual field α _m when the host vehicle is stopped is derived. However, when the relative change amount of α _m when the host vehicle is stopped is derived, it is not always necessary to compare with the effective visual field α ₀ at the start of driving. For example, instead of the effective visual field α ₀ at the start of operation, a comparison with a past effective visual field derived at the same point or a past effective visual field derived at the same time may be performed.

  The driver state estimation device and the driver state estimation method according to the present invention are useful as a device or method used for a part of a vehicle such as an automobile. It is also useful as an apparatus or method used in a part of a driving support apparatus that supports driving for a driver based on the estimated driver state.

1a-h, 1k Driver state estimation system 50 Effective viewing angle related information 100a-h, 100k Driver state estimation devices 101a-h, 101k Stop state determination units 102a-h, 102k Visual stimulus control units 103a-h, 103k Effective Viewing angle derivation units 104a to h, 104k, storage units 105a to h, 105k, driver state estimation unit 106, traffic light state determination unit 107, contrast adjustment unit 108, threshold adjustment unit 210, speed information output unit 220, image data acquisition unit 230, display unit 240, driving support Unit 250 foreground imaging camera unit 260 communication unit 270 illuminance sensor 280 navigation device 310 front window 320 center visual field position 330, 340, 350 visual stimulus display position 400 driver 401 eye point 410, 420, 430, 440 line-of-sight direction

Claims (21)

A driver state estimating device for estimating a state of a driver of a vehicle having a display area,
An image data acquisition unit that has an imaging system and acquires the image data of the driver using the imaging system;
A stop state determination unit that determines whether the speed of the vehicle is equal to or lower than a predetermined value based on the speed information of the vehicle;
When it is determined that the speed of the vehicle is equal to or lower than the predetermined value, the first visual stimulus is displayed on the display area, and the first visual stimulus is based on the image data of the driver. A visual stimulus control unit for determining whether or not the driver has visually recognized;
When it is determined that the first visual stimulus is visually recognized by the driver, the front direction and the line-of-sight direction of the driver are determined based on the position where the first visual stimulus is displayed in the display area. An effective viewing angle deriving unit for deriving a first effective viewing angle that is an angle formed;
A storage unit for storing the first effective viewing angle;
Based on the difference between the first effective viewing angle and the second effective viewing angle stored in the storage unit before the time when the first effective viewing angle is stored, the state of the driver is determined. A driver state estimation unit to estimate;
A driver state estimation device comprising:   The driver state estimation device according to claim 1, wherein the stop state determination unit determines whether or not the speed of the vehicle is zero.   When the visual stimulus control unit determines that the visual stimulus is not visually recognized by the driver, a second visual stimulus is displayed on the driver side with respect to the visual stimulus in the display area. The driver state estimation apparatus according to claim 1.   The driver state estimation device according to claim 1, wherein the stop state determination unit determines that the display of the visual stimulus starts when the vehicle is traveling at a constant speed. The storage unit stores an effective viewing angle at the start of operation of the vehicle,
When the driver state estimation unit establishes a predetermined relationship between the effective viewing angle derived by the effective viewing angle deriving unit and the effective viewing angle at the start of driving, the driver is in a fatigue state The driver state estimation device according to claim 1, wherein the driver state estimation device is estimated. The driver state estimation device according to claim 5,
The predetermined relationship is that a value of a ratio between the effective visual field angle derived by the effective visual field angle deriving unit and the effective visual field angle at the start of operation is equal to or less than a first threshold value, and the effective visual field angle is derived. The driver state estimation device, wherein the effective visual field angle derived by the unit is smaller than the effective visual field angle at the start of driving. The storage unit stores an effective viewing angle at the start of operation of the vehicle,
When the ratio of the effective visual field angle derived by the effective visual field angle deriving unit and the effective visual field angle derived immediately before the derivation is less than a second threshold, The driver state estimating apparatus according to claim 1, wherein the driver is estimated to be in a distracted state.   The driver state estimation apparatus according to claim 1, wherein the visual stimulus control unit sets a front front direction of the driver's eye point as a central visual field direction. The driver state estimation device according to claim 8,
The driver state estimation device, wherein the eye point is determined based on an eyeball position and a line-of-sight direction detected by a captured image acquired via an imaging device that captures an image including the driver. .   The visual stimulation control unit sets a direction of a predetermined object that is frequently watched by the driver as a central visual field direction from the driver's eye point as a starting point. The driver state estimation apparatus according to claim 1, wherein The driver state estimation device according to claim 10,
The driver state estimation device, wherein the predetermined object with a high frequency of being watched while the vehicle is stopped is a traffic light. The driver state estimation device according to claim 10,
The driver state estimation device, wherein the predetermined object with a high frequency of being watched while the vehicle is stopped is a preceding vehicle.   The driver state estimation apparatus according to claim 1, wherein the visual stimulus control unit displays the visual stimulus on a front window of the vehicle. The driver state estimation device according to claim 8,
The effective visual field angle deriving unit derives an effective visual angle as an angle determined by a visual stimulus display direction determined to be initially viewed by the visual stimulus control unit and a central visual field direction of the driver. A driver state estimating device.   The driver state estimation device according to claim 1, wherein the stop state determination unit determines display start of the visual stimulus based on a traffic light state transmitted from an immediately preceding traffic light. A contrast adjusting unit that adjusts display luminance or color of the visual stimulus displayed by the visual stimulus control unit according to the brightness of the surroundings of the vehicle,
The driver state estimation apparatus according to claim 1, wherein the visual stimulus control unit displays the visual stimulus adjusted by the contrast adjusting unit. The driver state estimating device according to claim 16, wherein
The said contrast adjustment part adjusts the display brightness | luminance or color of the said visual stimulus displayed by the said visual stimulus control part based on the captured image around the said vehicle, The driver state estimation apparatus characterized by the above-mentioned. The driver state estimation device according to claim 6 or 7,
An estimated threshold value adjusting unit for adjusting the first estimated threshold value or the second estimated threshold value according to the foreground of the vehicle displaying the visual stimulus displayed by the visual stimulus control unit. A driver state estimation device as a feature. When it is determined that the driver state estimation unit is in a fatigued state, a driving support unit that notifies the driver of a break, and
The driver state estimating device according to claim 1, further comprising: When it is determined that the driver state estimation unit is in a distracted state, a driving support unit that warns the driver;
The driver state estimating device according to claim 1, further comprising: A driver state estimation method for estimating the state of a driver of a vehicle having a display area,
Having an imaging system, obtaining image data of the driver using the imaging system,
Based on the speed information of the vehicle, it is determined whether the speed of the vehicle is a predetermined value or less,
When it is determined that the speed of the vehicle is equal to or lower than the predetermined value, the first visual stimulus is displayed on the display area, and the first visual stimulus is based on the image data of the driver. Determine if it was visible to the driver,
When it is determined that the first visual stimulus is visually recognized by the driver, the front direction and the line-of-sight direction of the driver are determined based on the position where the first visual stimulus is displayed in the display area. Deriving a first effective viewing angle, which is an angle formed;
Storing the first effective viewing angle;
Based on the difference between the first effective viewing angle and the second effective viewing angle stored in the storage unit before the time when the first effective viewing angle is stored, the state of the driver is determined. Driver state estimation method to be estimated.

Images