JP6481969B2 - Driver state estimation device - Google Patents

Description

  The present invention relates to a driver state estimation device, and more particularly, to a driver state estimation device that estimates the state of a driver driving a vehicle.

  It is desired to develop a vehicle that can reduce the probability of an accident caused by a traveling vehicle. According to statistics on the cause of accidents, vehicle accidents due to poor physical condition of the driver account for about 10% of the causes of accidents. Moreover, it is known that about 30% of these accidents caused by poor physical condition are poor physical condition related to brain diseases such as cerebral infarction. Many of the accidents resulting from these brain diseases are caused by sudden changes in the physical condition of the driver suffering from the brain disease, resulting in an inoperable state and an accident. Therefore, by detecting the occurrence of early symptoms related to the driver's brain disease at an early stage and taking appropriate measures such as notifying the driver, the vehicle is caused by deterioration of the driver's own disease or poor physical condition. The occurrence of an accident can be prevented in advance, and the probability of the occurrence of a vehicle accident can be reduced.

  Japanese Laid-Open Patent Publication No. 2017-16568 (Patent Document 1) describes a driver abnormality detection device. The driver abnormality detection device described in Patent Document 1 captures the head of a driver of a vehicle, and based on the captured image, the direction of the driver's line of sight, the height of the driver's eyes, and the driver's head The vehicle width direction position is obtained, and based on these, it is determined whether or not the driver has an abnormality.

JP 2017-16568

However, there is a problem that the direction of the driver's line of sight and the position of the head are easily affected by the external environment in which the vehicle is traveling, and the driver's abnormality is easily erroneously determined. Furthermore, according to the invention described in Patent Document 1, even when it is possible to detect an abnormality such as a driver's sleep or a side look, the driver has an abnormality such as a visual field defect based on a brain disease or the like. However, there is no significant change in the driver's head position and the like, and it is difficult to detect these abnormalities.
Accordingly, an object of the present invention is to provide a driver state estimation device that can detect an abnormality such as a driver's visual field defect.

  In order to solve the above-described problems, the present invention provides a driver state estimation device that estimates the state of a driver driving a vehicle, a steering sensor that detects a steering operation by the driver, and a driving that drives the vehicle. A driver camera that images the driver, an image analysis unit that detects the direction of the driver's line of sight based on an image captured by the driver camera, a steering operation detected by the steering sensor, and an image analysis unit A driver abnormality determination unit that determines a driver abnormality based on the direction of the line of sight, and the driver abnormality determination unit detects an expansion of the distribution with respect to the normal state of the driver's line of sight, And when the change with respect to the normal state of steering operation is detected, it is characterized by determining that a driver | operator is suspected of a visual field loss. .

  In the present invention configured as described above, the steering sensor detects the steering operation by the driver. On the other hand, the driver camera images a driver driving the vehicle, and the image analysis unit detects the direction of the driver's line of sight based on the captured image. The driver abnormality determination unit determines a driver abnormality based on the steering operation detected by the steering sensor and the line-of-sight direction detected by the image analysis unit. In particular, when an increase in the distribution of the driver's line of sight with respect to the normal state is detected and a change in the steering operation with respect to the normal state is detected, the driver abnormality determination unit suspects that the driver has a visual field defect. Is determined.

  As in the invention described in Patent Document 1, various devices for detecting an abnormality of a driver based on an image of a camera that images the driver have been proposed. However, when the driver has an abnormality such as a visual field defect due to a brain disease or the like, the position of the driver's head changes greatly, or the driver's line of sight deviates constantly from a predetermined range. Such an abnormality does not appear, and it is difficult to detect such an abnormality. Further, according to the research of the present inventors, it has been clarified that the steering operation changes when the driver has an abnormality such as a visual field loss. Since it is easily affected by the road surface or the like, there is a possibility of erroneous determination. Furthermore, according to the study of the present inventors, when the driver has an abnormality such as a visual field defect, the driver unconsciously distributes the direction of the line of sight in order to compensate for the information that is missing due to the visual field defect. It became clear that it expanded more than usual time without abnormality. According to the present invention configured as described above, the driver abnormality determination unit is configured when the driver's line-of-sight direction distribution is expanded with respect to the normal state and the steering operation is changed with respect to the normal state. Since it is determined that the driver is suspected of having visual field loss, it is possible to determine the driver's visual field loss while sufficiently suppressing erroneous determination.

  In the present invention, it is preferable that the camera further includes a vehicle environment camera that captures the surrounding environment of the vehicle, and the vehicle environment captured by the vehicle environment camera matches the direction of the line of sight detected by the image analysis unit. In this case, the driver abnormality determination unit is configured not to determine that the driver is suspected of having a visual field loss even if the distribution of the direction of the driver's line of sight is larger than the normal state.

  As described above, when the driver has an abnormality such as a visual field defect, the distribution of the direction of the line of sight has been clarified by the present inventors' research, but the distribution of the driver's line of sight is outside the vehicle. It may expand depending on the environment. According to the present invention configured as described above, when the outside environment captured by the outside environment camera matches the direction of the line of sight detected by the image analysis unit, the direction of the driver's line of sight is matched. Even if the distribution is enlarged, it is not determined that the driver is suspected of having a visual field defect, so that it is possible to suppress erroneous determination based on the gaze distribution.

  In the present invention, it is preferable that the vehicle further includes an outside environment camera that images the surrounding environment of the vehicle, and the driver abnormality determination unit includes a large number of objects to be visually recognized in the image captured by the outside environment camera. Is configured not to determine that the driver is suspected of having a visual field loss even when the distribution of the driver's line-of-sight direction is larger than the normal state.

  According to the present invention configured as described above, when a large number of objects to be viewed are included in the image captured by the outside-environment camera, the driver abnormality determination unit may cause the driver to be suspected of missing visual field. Since it is not determined that there is an object, it is possible to prevent an erroneous determination from occurring due to an increase in the distribution of the line-of-sight direction in an environment where there are many objects to be viewed.

  In the present invention, preferably, the image analysis unit is configured to extract a road sign from an image captured by an external environment camera, and the driver abnormality determination unit extracts the direction of the driver's line of sight by the image analysis unit. When the vehicle is directed to a road sign, it is determined that the direction of the driver's line of sight matches the outside environment captured by the outside environment camera.

  According to the present invention configured as described above, when the direction of the driver's line of sight is directed toward the road sign, the driver abnormality determination unit determines that the direction of the line of sight matches the environment outside the vehicle. In addition, since it is not determined that the driver is suspected of having a visual field defect, it is possible to suppress erroneous determination due to the driver turning his / her eyes on the road sign.

  In the present invention, preferably, the driver abnormality determination unit is configured when the distribution of the gaze direction of the driver is not biased even when the distribution of the gaze direction of the driver is expanded with respect to the normal state. It is configured not to determine that the driver is suspected of having a visual field defect.

  According to the inventor's research, when the driver has a visual field defect, the distribution in the direction of the line of sight expands, but the distribution of the line of sight expands evenly over the entire visual field. According to the present invention configured as described above, the driver abnormality determination unit does not determine that the driver is suspected of having a visual field loss when there is no bias in the distribution of the line-of-sight direction, and thus suppresses erroneous determination. can do.

  In the present invention, it is preferable that the driver abnormality determination unit is configured to change the setting of the driving support device mounted on the vehicle when it is determined that the driver is suspected of having a visual field loss. Yes.

  According to the present invention configured as described above, when it is determined that the driver is suspected of having a visual field loss, the setting of the driving support device mounted on the vehicle is changed, so the driver's state Accordingly, it is possible to perform appropriate driving support according to the situation, and it is possible to suppress the occurrence of an accident or the like until a driver with a visual field loss stops driving.

It is a block diagram showing the whole driver state estimating device by the embodiment of the present invention. It is a flowchart which shows the process performed in the control unit with which the driver | operator state estimation apparatus of embodiment of this invention is equipped. A photograph of a subject wearing visual field defect glasses for simulating a driver with visual field defect is shown. It is an example of the time-sequential data which shows the result of the driving | operation by a drive simulator about normal time and the time of visual field loss. It is a figure which shows the power spectrum density obtained by performing FFT analysis to the time series data shown in FIG. It is a figure which shows an example of gaze distribution when two test subjects drive on a drive simulator.

Next, a driver state estimation apparatus according to a preferred embodiment of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a block diagram showing an entire driver state estimation apparatus according to an embodiment of the present invention.

  As shown in FIG. 1, a driver state estimation apparatus 1 according to an embodiment of the present invention includes a control unit 2, a vehicle exterior camera 4, a steering sensor 6, a vehicle interior camera 8, an alarm device 10, and a communication device. 12 and. The control unit 2 is configured to be able to output a command signal to the connected driving support device 14.

  The control unit 2 is configured to receive each detection signal from the vehicle exterior camera 4, the steering sensor 6, and the vehicle interior camera 8 and output a control signal to the alarm device 10, the communication device 12, and the driving support device 14. Has been. Specifically, the control unit 2 includes an input interface circuit for inputting a detection signal from each device, an output interface circuit for outputting a control signal to each device, and a microprocessor for processing the input signal. , A memory, and a program (not shown) for operating them. Furthermore, the control unit 2 functions as an image analysis unit 16 and a driver abnormality determination unit 18 by the action of a built-in microprocessor or the like.

  The vehicle exterior camera 4 is a vehicle exterior camera provided so as to take an image of the surroundings of the vehicle on which the vehicle on which the driver state estimation device 1 is mounted is traveling. In the present embodiment, the vehicle exterior camera 4 is attached to the back side of a vehicle rearview mirror (not shown), and is provided so as to capture the traveling environment with a field of view substantially equivalent to the field of view of the driver of the vehicle. Video camera.

The steering sensor 6 is a sensor provided to detect a steering operation by the driver, and is configured to be able to detect the rotation angle of the steering shaft. In the present embodiment, a rotary encoder is used as the steering sensor 6.
The vehicle interior camera 8 is a driver camera disposed in the vehicle interior of the vehicle, and is mainly a video camera directed to photograph the head of the driver driving the vehicle from the front.

  The alarm device 10 is a device for visually and audibly transmitting the driver to the driver when a driver's physical condition is estimated by the driver abnormality determination unit 18 built in the control unit 2. . In the present embodiment, a display (not shown) and a speaker (not shown) provided in the driver's seat function as the alarm device 10. The alarm device 10 has a suspicion that the driver is in poor physical condition and may be incapable of driving. Therefore, the alarm 10 is displayed on the display so that the vehicle can be quickly stopped in a safe place, and the driver can hear the voice. It is comprised so that it may alert | report.

  When the driver abnormality determination unit 18 estimates that the driver's physical condition is poor, the communication device 12 is configured to report the fact to the nearby police and public institutions by radio. As a result, even when the driver falls into an inoperable state due to poor physical condition, it is possible to take measures to minimize damage.

  The driving support device 14 includes various sensors, steering actuators, brake actuators and the like (not shown) mounted on the vehicle. In the present embodiment, the driving support device 14 is configured to realize various functions such as a collision prevention function and a lane keep assist function by controlling the actuator based on detection signals of various sensors. Yes. Further, the driving support device 14 is configured to be able to change the setting of each function based on the command signal transmitted from the control unit 2.

Next, information processing in the control unit 2 will be described with reference to FIGS.
FIG. 2 is a flowchart showing processing executed in the control unit provided in the driver state estimation device of the present embodiment.

  The control unit 2 is configured to repeatedly execute the processing according to the flowchart shown in FIG. 2 at predetermined time intervals while the vehicle is traveling. First, in step S <b> 1 of FIG. 2, video image data captured by the vehicle interior camera 8 is input to the control unit 2. The image analysis unit 16 incorporated in the control unit 2 performs image analysis on the image input from the vehicle interior camera 8 and specifies the direction of the driver's line of sight at predetermined time intervals. Data on the direction of the line of sight of the driver obtained in the image analysis unit 16 is stored in the control unit 2.

  Next, in step S <b> 2, video image data captured by the vehicle exterior camera 4 is input to the control unit 2. The image analysis unit 16 built in the control unit 2 performs image analysis on the image input from the vehicle exterior camera 4 and specifies the vehicle exterior environment of the traveling vehicle at predetermined time intervals. Specifically, the image analysis unit 16 analyzes an image captured by the vehicle exterior camera 4, and the lane, the pedestrian, the preceding vehicle, the vehicle in the oncoming lane, the traffic signal, the road sign, and other signs on which the vehicle should travel. Etc. are extracted. Data on the environment outside the vehicle determined by the image analysis unit 16 is stored in the control unit 2.

Further, in step S 3, the steering angle [deg] detected by the steering sensor 6 is input to the control unit 2.
Next, in step S4, the control unit 2 performs FFT (Fast Fourier Transform) analysis on the time series waveform of the steer angle acquired in step S3, and calculates and stores the power spectral density (PSD) at each frequency. The control unit 2 stores in advance power spectrum density data obtained by FFT analysis of a normal steering operation waveform in a state where there is no abnormality in the driver.

  Furthermore, in step S5, the difference ΔPSD between the high frequency region of the power spectral density acquired in step S4 and the high frequency region of the power spectral density in the normal state of the driver is greater than or equal to a predetermined threshold value. It is determined whether or not there is. If the difference ΔPSD in the high frequency region of the power spectrum density is greater than or equal to a predetermined threshold value, the process proceeds to step S6. If it is less than the predetermined threshold value, “no abnormality” is determined in step S11, and the flowchart shown in FIG. The one-time process is terminated.

Here, an example of a change in the steering operation when the driver has a visual field defect will be described with reference to FIGS. 3 to 5.
FIG. 3 shows a photograph of a subject wearing visual field defect glasses for simulating a driver with visual field defect. FIG. 4 is an example of time-series data indicating the results of driving by the drive simulator for normal time and visual field loss. FIG. 5 is a diagram showing power spectral density obtained by performing FFT analysis on the time series data shown in FIG.

  Changes in the driver's steering operation in the presence of visual field loss were clarified by a drive simulator experiment with healthy subjects. Further, the driving behavior when the same driver caused visual field loss was measured by driving the drive simulator with the same subject wearing the visual field defect glasses shown in FIG. As shown in FIG. 3, the visual field defect glasses are glasses in which one half of the central axis of each spectacle lens (the right half as viewed from the subject in FIG. 3) is blacked out. One of these is blocked, and the state where the visual field is lost is simulated.

  FIG. 4 is a graph showing a result of driving the same lane by a drive simulator in a state where the same healthy subject wears visual field defect glasses and a state where he / she does not wear vision loss glasses. The upper part of FIG. 4 is a graph showing the steering operation in the drive simulator, and the lower part is a graph showing the lateral position (offset) of the vehicle traveling on the drive simulator. In each graph, the solid line indicates the driving result of a healthy subject, the broken line indicates the driving result of the subject who simulates the visual field defect, and the solid line and the broken line indicate the result of traveling in the same lane by the drive simulator.

  First, as is apparent from the upper graph of FIG. 4, the subject who simulates the visual field defect indicated by the broken line generally has more detailed vibration than the steering operation by the healthy subject indicated by the solid line. . For example, in part A in the graph, the operation amount of the subject who simulates the visual field defect is large, and in part B, large correction steering is seen.

  Furthermore, as shown in the lower graph of FIG. 4, the offset from the center of the lane is generally larger in the subject who simulated the visual field defect indicated by the broken line than the running position by the healthy subject indicated by the solid line. For example, in the portion C in the graph, the lateral position of the vehicle is greatly deviated from the center of the lane, and in the portion D, the traveling position is largely staggered. In addition, as seen in part E, the running position lacks stability.

  FIG. 5 is a graph showing the power spectral density obtained by performing FFT analysis on the time series waveform of the steering operation shown in the upper part of FIG. The left column of FIG. 5 shows the power spectral density in a healthy subject, and the right column shows the power spectral density in a subject simulating a visual field defect.

  As is clear from FIG. 5, in the subject who simulated the visual field defect shown in the right column, the spectrum in the high frequency region is larger than the power spectrum density in the healthy subject shown in the left column. In particular, the power spectrum in the frequency range of 0.25 Hz to 0.5 Hz surrounded by a thick frame in FIG. 5 is large in the subject who simulated the visual field defect.

  In the present embodiment, the power spectral density calculated in step S4 in FIG. 2 is integrated in the range of 0.25 Hz to 0.5 Hz, and the integrated value PSD1 is measured and stored in advance. The value PSDN is compared in step S5. If the difference ΔPSD between these integrated values PSD1 and PSDN is less than a predetermined threshold value, there is no suspicion of visual field loss, so the driver abnormality determination unit 18 built in the control unit 2 determines “no abnormality” (FIG. 2 step S5 → S11).

  In addition, according to the research of the present inventors, in a driver with a visual field defect, steering entropy is increased because the steering operation is larger and vibrational than when there is no visual field defect, and the steering operation is not smooth. It is also clear. However, when the vehicle speed is slow, such as when the vehicle is traveling on a congested road, or when normal driving action is difficult due to snow or road irregularities, the presence or absence of visual field loss is determined only from the steering operation. It is clear from the inventor's research that the determination is difficult. For this reason, in this embodiment, generation | occurrence | production of a misjudgment is suppressed by performing the process after step S6 of FIG.

  Next, in step S6 of FIG. 2, the image analysis unit 16 performs image analysis in step S1, and determination is made on the driver's line-of-sight direction data stored in the control unit 2. That is, according to the research of the present inventors, it is clear that when the driver is in a visual field deficient state, the direction of the line of sight is greatly moved to compensate for the portion of the visual field that is deficient compared with the normal state. In step S6, it is determined whether or not the distribution of the direction of the driver's line of sight in a predetermined period is larger than the distribution of the direction of the line of sight of the driver stored in advance. If the distribution of the line-of-sight direction is enlarged, the process proceeds to step S7. If not enlarged, “no abnormality” is determined in step S11, and the one-time process of the flowchart shown in FIG.

  Further, in step S 7, it is determined whether or not there is a bias in the distribution of the driver's line-of-sight direction that has been image-analyzed by the image analysis unit 16 in step S 1 and stored in the control unit 2. That is, according to the study of the present inventors, depending on the external environment of the vehicle being driven, the distribution of the direction of the line of sight may expand even for a healthy driver. There is no bias in the gaze direction distribution. On the other hand, when the distribution in the direction of the driver's line of sight is enlarged due to a visual field defect, it is clear that the distribution is biased. In step S7, if there is a bias in the line-of-sight direction distribution, the process proceeds to step S8. If there is no bias, it is determined that there is no abnormality in step S11, and one process of the flowchart shown in FIG. To do.

  Next, in step S8, based on the video image from the vehicle exterior camera 4 analyzed by the image analysis unit 16 in step S2, whether or not the expansion of the driver's line-of-sight direction distribution is due to the external environment. Is determined. If the expansion of the line-of-sight distribution is not due to the external environment, the process proceeds to step S9, and the driver is determined to be “suspected of visual field loss”. On the other hand, if it is due to the external environment, the process proceeds to step S11, where “no abnormality” is determined and the one-time process of the flowchart shown in FIG. That is, when there is a target that the driver should pay attention to around the traveling vehicle, and the driver's gaze is directed toward the target, the gaze distribution is expanded and the distribution is biased. The driver abnormality determination unit 18 determines “no abnormality”. That is, the driver abnormality determination unit 18 matches the direction of the driver's line of sight detected by image analysis of the video image captured by the vehicle interior camera 8 and the video image captured by the vehicle exterior camera 4. If it is, the driver does not determine that there is a suspicion of visual field loss even if the distribution in the direction of the driver's line of sight is larger than the normal state.

  Specifically, the image analysis unit 16 analyzes a video image obtained by the camera 4 outside the vehicle cabin, and detects targets such as pedestrians around the vehicle, preceding vehicles, vehicles on the opposite lane, traffic lights, road signs, and other signs. Extract and specify the position of the target. Next, the image analysis unit 16 identifies the direction of the driver's line of sight from the video image captured by the vehicle interior camera 8. The driver abnormality determination unit 18 collates the position of the target extracted from the image of the vehicle exterior camera 4 with the direction of the driver's line of sight identified from the image of the vehicle interior camera 8, and extracts the driver's line of sight. If the target is directed to a target (for example, a road sign), it is not determined that there is a suspicion of visual field loss.

  In this way, it is a natural driving action for the driver to look at various targets around the vehicle, and when the driver's line of sight is distributed in this way, the driver's line of sight Is consistent with the environment outside the vehicle imaged by the camera 4 outside the vehicle compartment, and it is determined that there is no abnormality. In addition, when a vehicle travels in an urban area or the like and there are a large number (for example, 10 or more) of recognition objects (targets) around the vehicle, the recognition object that the driver is looking at is specified. Difficult to do. Also in such a case, the driver abnormality determination unit 18 does not determine that “the visual field loss is suspected”, but proceeds to step S11 and ends the one-time process of the flowchart illustrated in FIG.

  If it is determined in step S9 that “the visual field defect is suspected”, the process further proceeds to step S10. In step S <b> 10, the driver abnormality determination unit 18 sends a signal to a display (not shown) and a speaker (not shown) in the passenger compartment, which is the alarm device 10, and issues an alarm to the driver. In the present embodiment, the driver abnormality determination unit 18 displays a message “There is a suspicion of visual field loss, please stop the vehicle immediately in a safe place” on the display (not shown). Is output from the speaker. Further, the driver abnormality determination unit 18 sends a command signal to the driving support device 14 to change the setting of the driving support device 14 according to the situation. Specifically, the lane keep assist control executed by the driving support device 14 is strengthened, and the control parameter of the lane keep assist control is changed so that the vehicle is more difficult to be removed from the lane.

  When the driving support device 14 has an automatic driving function, the driver abnormality determination unit 18 issues a command to the driving support device 14 so that the vehicle automatically stops at a safe place by automatic driving. The present invention can also be configured as described above. Further, the driver abnormality determination unit 18 sends a command signal to the communication device 12 to make an emergency call to the nearest police station or the like that the driver may be incapable of driving. The process ends. As described above, the driver state estimation device 1 includes an output device that outputs something to the alarm device 10, the communication device 12, the driving support device 14, and the like, and copes with a situation where the driver may be unable to drive. It is preferable.

Next, an example of the distribution of the driver's line of sight will be described with reference to FIG.
FIG. 6 is a diagram illustrating an example of a line-of-sight distribution when two subjects drive on a drive simulator.

  FIG. 6 is a diagram showing an example of a line-of-sight distribution when two subjects travel on a highway on a drive simulator. FIG. 6 (a) column shows that subject A does not wear visual field loss glasses. The case where it drove was shown and the (b) column has shown the case where the test subject drive | wears with the visual field defect | deletion glasses. Similarly, the column (c) in FIG. 6 shows a case where the subject B is driven without wearing visual field loss glasses, and the (d) column shows a case where the subject is driven while wearing visual field loss glasses. . FIG. 6 shows the gaze distribution while each subject is driving the drive simulator for 1 minute 30 seconds.

  As in the example shown in FIG. 6, in a situation where the subject in a normal state travels on the highway and there is no oncoming vehicle, the direction of the subject's line of sight is distributed in a relatively narrow range (column (a) in FIG. 6). And (c) column). On the other hand, in each of the subjects wearing the visual field defect glasses (columns (b) and (d) in FIG. 6), the direction of the line of sight is distributed over a wider range than the state of not wearing them. Moreover, in the state where the visual field defect glasses are not worn, the subject B has a line of sight distributed in a wider range than the subject A, and the distribution range varies from subject to subject. When is worn, the range in which the line of sight is distributed is larger than when not worn. For this reason, it is possible to detect the visual field loss of the driver by measuring and storing the distribution of the line of sight in a healthy state for each driver and comparing it with the distribution of the line of sight during driving.

  In this embodiment, the variance of plot points indicating the driver's line of sight distributed in the field of view is calculated in the vertical and horizontal directions, and the range of the line of sight is evaluated by the average value thereof. Yes. In other words, in the present embodiment, the variance of the distribution in the direction of the line of sight in the normal time of the driver is measured, and the value is stored in advance. Further, in step S5 of FIG. 2, the variance of the distribution in the direction of the line of sight during the driving of the driver is calculated, and when the obtained variance is larger than a prestored variance by a predetermined value or more. It is determined that the line-of-sight distribution is widened.

  Further, as shown in columns (b) and (d) of FIG. 6, it can be seen that in both the subjects A and B, the visual field distribution is biased to either the left or right direction when wearing visual field defect glasses. . That is, in a state where the visual field loss glasses are not worn, the subject's line of sight is distributed approximately evenly from side to side centering on the center of the lane to be traveled (columns (a) and (c) in FIG. 6). On the other hand, it can be seen that in the state of wearing the visual field defect glasses, the gaze distribution is biased to the right side or the left side with respect to the center of the lane. It has been clarified by the inventor's research that such a bias in the gaze distribution is a tendency peculiar to the driver who has caused the visual field defect. In the present embodiment, when the driver's line-of-sight distribution is biased to the right or left by a predetermined ratio or more with respect to the center of the lane, it is determined that the line-of-sight distribution is biased, and such a bias is seen. If not, it is determined that there is no suspicion of visual field loss (step S7 in FIG. 2).

  Furthermore, the results shown in FIG. 6 are obtained when the vehicle travels on a highway, but according to the experiment of the present inventors, when the vehicle travels in an environment where there are many objects to be viewed, such as an urban area. The driver's line-of-sight distribution varied greatly depending on the position of the object to be viewed. For this reason, in such a driving environment, it has been difficult to clearly identify the presence or absence of a driver's visual field loss based only on the gaze distribution. In particular, when there are many sighting objects such as pedestrians, traffic lights, road signs, etc. in the driver's field of view, even if the image of the camera 4 outside the vehicle is collated with the direction of the driver's line of sight, the driver It was difficult to identify which object is looking at. For this reason, when there are a large number of objects to be viewed in the environment outside the vehicle imaged by the vehicle exterior camera 4, it is determined that “there is no suspicion of visual field loss”, and “there is no suspicion of visual field loss”. (Step S8 in FIG. 2).

  Even if the vehicle is traveling on an expressway or the like and there are few objects to be viewed, if the driver's line of sight is directed to the preceding vehicle or oncoming vehicle, the distribution of the driver's line of sight is Enlarging and uneven distribution occurs. In such a case, the direction of the driver's line of sight and the environment outside the vehicle are collated, and if they match (if the line of sight is directed toward the visual target), it is determined that there is no suspicion of visual field loss. If they do not match, the present invention can be configured to determine that there is a suspicion of visual field loss.

  According to the driver state estimation device 1 of the embodiment of the present invention, the driver abnormality determination unit 18 expands the distribution of the direction of the driver's line of sight relative to the normal state (step S6 in FIG. 2), and the steering. When the operation changes with respect to the normal state (step S5 in FIG. 2), it is determined that the driver is suspected of having a visual field loss. Therefore, the visual field loss of the driver is determined while sufficiently suppressing erroneous determination. be able to.

  Further, according to the driver state estimation device 1 of the present embodiment, when the outside environment captured by the vehicle exterior camera 4 matches the direction of the line of sight detected by the image analysis unit 16, the driver Even if the distribution in the direction of the line of sight is expanded, it is not determined that the driver is suspected of having a visual field loss (step S8 in FIG. 2), so that the occurrence of erroneous determination based on the line-of-sight distribution can be suppressed.

  Furthermore, according to the driver state estimation device 1 of the present embodiment, when many visual objects are included in the image captured by the vehicle exterior camera 4, the driver abnormality determination unit 18 Since it is not determined that there is a suspicion of visual field loss (step S8 in FIG. 2), it is possible to prevent an erroneous determination from occurring due to an increase in the distribution in the direction of the line of sight in an environment where there are many visual objects.

  Further, according to the driver state estimation device 1 of the present embodiment, when the driver's line of sight is directed to the road sign, the driver abnormality determination unit 18 matches the line of sight and the outside environment. Since it is determined that the driver is suspected of having a visual field loss (step S8 in FIG. 2), erroneous determination due to the driver turning his / her gaze on the road sign can be suppressed.

  Furthermore, according to the driver state estimation apparatus 1 of the present embodiment, the driver abnormality determination unit 18 does not determine that the driver is suspected of having a visual field loss when the distribution of the gaze direction is not biased (see FIG. 2 step S7), it is possible to suppress erroneous determination.

  Further, according to the driver state estimating device 1 of the present embodiment, when it is determined that the driver is suspected of having a visual field loss, the setting of the driving support device 14 mounted on the vehicle is changed ( Since step S10) in FIG. 2 makes it possible to perform appropriate driving support according to the driver's condition, an accident or the like occurs until the driver with a visual field loss stops driving. Can be suppressed.

  As mentioned above, although preferable embodiment of this invention was described, a various change can be added to embodiment mentioned above.

1 Driver condition estimation device 2 Control unit 4 Outside camera (outside vehicle camera)
6 Steering sensor 8 Car interior camera (driver camera)
DESCRIPTION OF SYMBOLS 10 Alarm device 12 Communication device 14 Driving support device 16 Image analysis part 18 Driver abnormality determination part

Claims (6)

A driver state estimating device for estimating a state of a driver driving a vehicle,
A steering sensor for detecting a steering operation by the driver;
A driver camera that images the driver driving the vehicle;
An image analysis unit for detecting the direction of the driver's line of sight based on the image captured by the driver camera;
A driver abnormality determination unit that determines a driver's abnormality based on a steering operation detected by the steering sensor and a line-of-sight direction detected by the image analysis unit;
When the driver abnormality determination unit detects an increase in the distribution of the driver's line of sight relative to the normal state and detects a change in the steering operation relative to the normal state, the driver is suspected of having a visual field loss. A driver state estimation device configured to determine.   Further, the vehicle has an outside environment camera that images the surrounding environment of the vehicle, and when the outside environment imaged by the outside environment camera matches the direction of the line of sight detected by the image analysis unit, the driving is performed. The driver abnormality determination unit is configured not to determine that the driver is suspected of having a visual field loss even if the distribution of the driver's line-of-sight direction is larger than a normal state. Driver state estimation device.   Furthermore, the vehicle has an outside environment camera that captures the surrounding environment of the vehicle, and the driver abnormality determination unit is configured to display a driver when the image captured by the outside environment camera includes many visual objects. The driver state estimation device according to claim 1, wherein the driver state estimation device is configured not to determine that the driver is suspected of having a visual field loss even when the distribution of the line-of-sight direction is larger than the normal state.   The image analysis unit is configured to extract a road sign from an image captured by the external environment camera, and the driver abnormality determination unit is configured to extract the direction of the driver's line of sight from the image analysis unit. 3. The driver state estimating apparatus according to claim 2, wherein the driver state estimating device is configured to determine that the direction of the driver's line of sight matches the environment outside the vehicle imaged by the vehicle environment camera. .   Even if the driver's line-of-sight direction distribution is larger than the normal state, the driver abnormality determination unit may cause a visual field defect to the driver if the line-of-sight direction distribution is not biased. The driver state estimation apparatus according to any one of claims 1 to 4, wherein the driver state estimation apparatus is configured not to determine that there is a suspicion.   The driver abnormality determination unit is configured to change a setting of a driving support device mounted on a vehicle when it is determined that the driver is suspected of having a visual field defect. The driver state estimation apparatus according to any one of the above.