JP2006043429A - Apparatus for judging load of visual recognition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To judge a load of visual recognition to an indirect visibility except for the front of a driver with accuracy. <P>SOLUTION: The visual point coordinate of a driver is detected by a visual point detecting part 51, and the presence or absence of blinking of the driver is detected by a blinking detecting part 52 to watch the driver, and furthermore, it is judged by a calculating part 43 for a visual point distance based on the visual point coordinate that the visual point moves between the front direct visibility and the rear and side indirect visibility devices, and the presence or absence of blinking between the front direct visibility and indirect visibility is judged at the same time by a blinking presence judgement part 44 during the visual point movement based on the presence or absence of blinking. A blinking pattern judgement part 48 judges the load of visual recognition to the indirect visibility during the visual point move by combination patterns of the visual point movement direction between the front direct visibility and the rear indirect visibility device and the presence or absence of the blinking. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a visual load determination device that measures a visual load required to recognize a surrounding object by, for example, a driver driving a vehicle.

  2. Description of the Related Art Conventionally, as a technique for estimating a vehicle driver's cognitive state and calculating a driving action risk with high accuracy, for example, a driving action risk calculating device described in Patent Document 1 below is known. Yes.

  This driving behavior risk degree computing device determines the traveling direction of the vehicle based on the steering angle detected by the steering angle sensor, detects the driver's viewpoint behavior, The degree of driving behavior risk is calculated in consideration of how often the direction of travel of the vehicle is seen for a certain amount of time. According to this driving behavior risk calculating device, for example, when the traveling direction is viewed for a predetermined threshold time or more, or when the traveling direction is viewed for a predetermined threshold number of times or more, the driving behavior risk is lowered while the traveling direction is decreased. Is not seen for a predetermined threshold time or when the traveling direction is not seen for a predetermined threshold number of times or more, the driving action risk is increased.

  In order to avoid the annoyance (moderate feeling and anxiety) with respect to the driving operation by the information as the amount of information for driving support to the driver increases, including the technology described in Patent Document 1 described above. It is necessary to monitor the state of the driver such as the driver's attention state and control information to be presented based on the monitoring result.

On the other hand, in the driving behavior risk calculation device described in the above-mentioned Patent Document 1, as described in claim 2, the “traveling time (gaze time)” and “frequency of watching” the traveling direction of the vehicle. In order to calculate the driving risk using at least one visual behavior of “(gazing frequency)” and to further improve the accuracy of the risk calculation, the “visual attention state” is described. Is detected. In this visual attention state detection, the driver's attention state is estimated by classifying visual behavior into, for example, gaze, blink, saccade eye movement, following eye movement, and convergence movement. For example, if the visual behavior is gaze, the driver is paying attention to the driving behavior, but if the visual behavior is blinking or congested, it is detected that the driver is not paying attention to the driving. ing.
JP 2003-99899 A (Claim 2, Claim 3)

  However, in the driving behavior risk calculation device described in Patent Document 1, monitoring of the driving state by the driver's gaze time, gaze frequency, and blink rate is a technique for visual behavior in the forward direct field of view. It did not consider visual behavior other than in front of the vehicle. On the other hand, the driver who is driving the vehicle needs to grasp not only the front which is the traveling direction but also the surrounding situation including the rear side by visually recognizing and performing an appropriate driving action. Accordingly, there is a problem that the driver's condition cannot be sufficiently monitored unless the visual behavior detection accuracy is increased for the rear side. Even when applied to the indirect field of view, there is a problem that the visual action detection accuracy is insufficient.

  By the way, for example, with regard to the rear side indirect view using a side mirror mounted on the vehicle, the dangerous state for the driver is (1) Although the situation should be directed to the rear side, the rear side indirect view The situation where the line of sight is not directed and the information is not collected, (2) The information is directed toward the rear side indirect field of view with the attention to the rear side, but the situation of the rear side Is in a state that cannot be fully understood.

  These two states are also applicable to the forward direct field of view without using a mirror or the like, but in the forward direct field of view, the information can be obtained almost simply by directing the line of sight, whereas the rear side indirect Not necessarily in sight. The state (2) is a state in which the visual load is high and the information on the rear side cannot be determined. In the rear side indirect view where the visual load is higher than the front direct view, it is necessary to detect the state (2). Sex is manifested.

  Specifically, it is necessary to effectively use the driver's caution resources especially in situations where the driving lane must be merged in a limited time and road section, such as a merging scene. When the state is detected, it is difficult to visually recognize both the front side and the rear side, and there is a possibility of oversight of other vehicles. On the other hand, it is necessary not to give a sense of anxiety, and the necessity of providing driving support information increases depending on the driving ability.

  Therefore, although it is necessary to detect the state of (2) in order to determine the necessity of driving assistance, in the above-mentioned Patent Document 1, the line of sight is directed to the rear side indirect view by monitoring the line of sight behavior. It is possible to determine the situation of (1) above by determining whether or not, but it is not possible to determine the situation of (2) above.

  The reason for this is that when moving the line of sight to the back side indirect view, it is assumed that the line of sight is separated from the front, and there is a limit to the time that the line of sight can be separated from the front, and it does not exceed a certain gaze time. Even if the load increases, the gaze time in the rear side indirect view does not increase. Also, if the driving load exceeds a certain threshold, the driving action may be omitted, and this also applies to the visual act, so if the visual load of the rear side indirect view increases, some drivers may omit the visual action. Because there is.

  Also, in the past, when the determination of the visual load based on the number of blinks per hour (blink rate) is applied to the rear side indirect view, the blink rate is usually reduced due to the amount of movement of the line of sight or the change of view. There is a problem that the state (2) cannot be determined by the blink rate.

  Then, this invention is proposed in view of the above-mentioned situation, and it aims at providing the visual load judgment apparatus which can judge the visual load with respect to the indirect visual field including direct visual fields other than the front correctly. To do.

  The present invention relates to a visual load determination device for determining a visual load of a driver who visually recognizes a forward direct view and an indirect view including a direct view other than the forward, and detects the viewpoint coordinates of the driver by the viewpoint coordinate detection means. And monitoring the driver by detecting the presence or absence of the driver's blink action by the blink state detection means, and further, based on the viewpoint coordinates detected by the viewpoint coordinate detection means by the viewpoint movement determination means, Based on the presence / absence of the blink action detected by the blink state detection means by the blink action presence / absence judging means, while determining that the viewpoint movement is a viewpoint movement between the forward direct view and the indirect view. It is determined whether or not there is a blink action during the viewpoint movement period between the forward direct view and the indirect view determined by the viewpoint movement determination means.

  The visual load determination device includes a direction of the viewpoint movement between the forward direct view and the indirect view determined by the viewpoint movement determination unit, and the presence or absence of the blink action determined by the blink action presence determination unit. The above-described problem is solved by determining the combination pattern by the pattern determining unit and determining the visual load on the indirect field of view by the visual load determining unit based on the pattern determined by the pattern determining unit.

  According to the visual load determination device according to the present invention, the visual load on the indirect field of view is determined based on the combination pattern of the direction of the viewpoint movement between the front direct field of view and the indirect field of view and the presence or absence of the blink action. It is possible to accurately determine the visual load on the indirect field of view other than the front.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The visual load determination device to which the present invention is applied is a viewpoint shift between a viewpoint to a rear side indirect viewing device that allows a driver to visually recognize a rear side indirectly such as a door mirror and a viewpoint to a front direct field of view. Whether the demand for information provided by the rear side indirect vision device, i.e., information on the rear side, is high due to the change in the blink pattern, which is a combination pattern with the presence or absence of the blink action when By determining, the driver's visual load is measured.

[Principle for measuring visual load]
First, before describing the configuration and operation of a visual load determination device to which the present invention is applied, the principle for determining the visual load when a driver who is a measurement target recognizes an object will be described.

  The driver performs a blink action while driving and moves the viewpoint between the viewpoint to the rear side indirect vision device and the viewpoint to the front vision, but the viewpoint movement is performed with blinking. Probability is high, and the visual load determination device to which the present invention is applied determines the combination of the reciprocation of viewpoint movement and the presence or absence of blinks.

  Here, there are three types of blinking actions of the driver. In other words, the blink action is a blink when the driver's intention is clearly involved, voluntary blinks that are consciously closed or closed in accordance with a signal, and external reflex-induced stimulation is clear Reflective blink, which is a blink that is a blink that closes the eyelid reflexively when it is surprised by a sudden sound or light, etc. There are blinks that cannot identify external reflex-induced stimuli and that are deeply related to the driver's cognitive and psychological state.

  On the other hand, in the visual load judgment device to which the present invention is applied, among the three types of blink actions, attention is paid to the spontaneous blink that reflects the state of cognition and psychology. The features of this spontaneous blink include (1) a decrease in the blink rate when attention and attention are directed to visual information, (2) an increase in the blink rate when awareness is directed to thinking activities, (3) There is a blink at the end of recognition judgment, and (4) an increase in the occurrence rate of cluster blinks during fatigue.

  On the other hand, when the viewpoint moves to the rear side indirect vision device, the presence or absence of blinks depends on the combined conditions of spontaneous blinks (1) to (4). In the apparatus, the visual load is determined by using the feature of (1) “suppression of blinking action when attention / interest is focused on visual information”. This is due to the human nature that blinking usually does not occur when the demand for visual information is high, between the viewpoint to the rear side indirect vision device and the viewpoint to the front direct vision. The same state is used when moving the viewpoint.

  When the presence or absence of the blink action at the time of such a viewpoint movement is applied when the viewpoint is moved between the viewpoint to the rear side indirect vision device and the viewpoint to the front direct vision, the viewpoint movement as shown in FIG. It is possible to derive a blink pattern that is a combination pattern with or without blink action. This blink pattern consists of a forward direction that moves from the viewpoint to the front direct view to the viewpoint to the rear side indirect vision device and a return direction that moves from the viewpoint to the rear side indirect vision device to the front direct view. The first blink pattern to the fourth blink pattern can be classified according to the presence / absence of the blink action.

  The first blink pattern includes a forward direction in which the view moves from the viewpoint toward the front direct view to the view point toward the rear side indirect view device, and a return path that moves from the view point toward the rear side indirect view device to the view point toward the front direct view. This is a case where a blink action occurs in both directions.

  In addition, the second blink pattern has no blink action in the forward direction from the viewpoint to the front direct view to the rear side indirect view device, and the front direct view from the view to the rear side indirect view device. This is a case where a blink action occurs in the backward direction of moving to the viewpoint.

  In addition, the third blink pattern moves from the viewpoint to the front direct view to the viewpoint to the rear side indirect view device, and from the viewpoint to the rear side indirect view device to the point to the front direct view. This is a case where the blink action does not occur both in the return direction.

  Furthermore, in the fourth blink pattern, a blink action occurs in the forward direction from the viewpoint to the front direct view to the viewpoint to the rear side indirect view device, and the front blinks from the viewpoint to the rear side indirect view device. This is a case in which there is no blinking action in the return direction in which the view point moves directly to the view point.

  Usually, in a state where the visual load is low and the degree of request for obtaining visual information is not so high, the first blink pattern or the second blink pattern is shown from the combined state of the spontaneous blink elements. On the other hand, when the viewing load increases at the time of lane change or merging in a place where there is a lot of traffic flow, the degree of demand for obtaining visual information increases, and the blink action at the time of moving the viewpoint is suppressed. The ratio of the pattern or the fourth blink pattern increases.

  Such a blink pattern change is considered to be determined by a relative relationship between the road environment during driving and the driving ability of the driver. Accordingly, the visual load determination device can determine the visual load on the rear side of the driver by detecting a change in the blink pattern.

  Specifically, it is observed from the visual behavior during actual driving that there is a correlation between the change in the blink pattern and the visual load, and the visual load determination device determines that the third blink Using the increase in the frequency of occurrence of the pattern or the fourth blink pattern, it is determined that the visual load for performing the viewpoint movement to the rear side indirect vision device is high.

  As shown in the conceptual diagram of FIG. 2, the visual load determination device detects the movement of the viewpoint, and as shown in FIG. 2A, the rising edge of the viewpoint movement detection signal is followed from the viewpoint to the front direct view. Assuming that the forward direction of movement to the viewpoint to the side indirect vision device is the forward direction, and that the falling edge of the signal is the backward direction of movement from the viewpoint to the rear side indirect vision device to the viewpoint to the front direct vision, FIG. As shown in b), the presence or absence of a blink action at the time of rising of the viewpoint movement detection signal is detected.

  Then, as shown in FIG. 2 (c), the visual load determination device determines that the blink pattern when the blink is present, and the blink pattern when the blink is absent when the blink is absent in the figure. When the blink pattern is the third blink pattern or the fourth blink pattern, that is, when it corresponds to the blink pattern with no blink on the return path, as shown by x in FIG. Furthermore, it can be determined that the visual load on the rear side is high.

[Configuration of Visual Load Determination Device According to First Embodiment]
The present invention is applied to the visual load determination device 1 according to the first embodiment configured as shown in FIGS. As shown in FIG. 3, the visual load determination device 1 is configured to be connected to a line-of-sight / blink state measurement device 2, a vehicle speed sensor 3, an information provision device 4, and an information presentation device 5. Such a visual load determination device 1 inputs the viewpoint coordinate signal and the blink signal from the line-of-sight / blink state measurement device 2 and determines the visual load for the driver's rear side indirect view from the vehicle speed sensor 3. The vehicle speed signal is input, the visual load calculation unit 11 determines the visual load, and the information provision determination unit 12 determines the information presentation method provided by the information providing device 4 based on the determination result. As a result, the information presented to the information presentation device 5 reflects the determination result of the visual load.

  Specifically, as shown in FIG. 4, the visual load determination device 1 is an information control device between an information presentation device 5 that presents driving support information such as a navigation device or an IT device and the visual load determination device 1. 21, and further includes a face image camera 31 having the driver's face as an imaging range 31 a and a line-of-sight / blink state measuring device 2 connected to each other and mounted on the host vehicle 20. Thus, when the visual load determination device 1 is mounted on the host vehicle 20, the face image camera 31 is provided at a position where the driver's eyeball portion can be imaged, and the vehicle speed sensor 3 is connected to the vehicle drive portion. The visual load determination device 1, the information control device 21, and the information presentation device 5 are provided, for example, in an instrument panel of the vehicle, and the information presentation device 5 is visible from the driver. Provided at various positions.

  The line-of-sight / blink state measuring device 2 has a so-called eye tracker system, and measures the viewpoint position and the blink state of the driver. The line-of-sight / blink state measurement device 2 performs infrared corneal reflection and image processing to measure the viewpoint position and the blink state. As a result, the line-of-sight / blink state measurement device 2 creates viewpoint coordinate data and blink data according to the time axis and sends the viewpoint coordinate data and blink data to the visual load determination device 1. The line-of-sight / blink state measuring device 2 may perform measurement processing separately for the viewpoint coordinate data and the blink data, and when only the viewpoint movement signal is measured and the viewpoint movement signal becomes a predetermined value, It may be determined that the eye state is present, and blink data may be extracted.

  The eye-gaze / blink state measuring device 2 detects the state by a face image camera 31 comprising a contact type covering the head like a hat and two stereo cameras fixed on the instrument panel as an eye tracker system. The non-contact type to perform can be used. On the other hand, the gaze / blink state measuring apparatus 2 of the present example aims to determine whether or not the gaze is directed to a rear side indirect vision device having a certain area such as a door mirror or a room mirror. A slightly inferior but inexpensive non-contact type is used. Thereby, since it is a non-contact type with respect to a driver | operator, there exists an advantage that a measurement load is not given.

  The vehicle speed sensor 3 creates vehicle speed data indicating whether or not the host vehicle is traveling and sends the vehicle speed data to the visual load determination device 1.

  The visual load determination device 1 calculates the visual load during driving by the driver using the viewpoint coordinate data and blink data from the line-of-sight / blink state measurement device 2 and the vehicle speed data from the vehicle speed sensor 3. The processing contents of the visual load determination device 1 will be described later.

  The information providing device 4 calculates the current position of the vehicle on which the visual load determination device 1 is mounted, traffic condition information such as road traffic congestion, and travel guidance information for promoting travel on the recommended route. In the information providing apparatus 4, the information provision determining unit 12 controls the presentation timing and the presentation contents of the traffic condition information and the travel guide information presented to the driver.

  The information provision judgment unit 12 presents the traffic status information and the travel guide information sent from the information provision device 4 by the information presentation device 5 based on the visual load calculated by the visual load judgment device 1. Calculate the presentation timing. Specifically, when the driver's visual burden is high, the presentation content is easily recognized, and the amount of information that can be recognized within a limited time is small. In a situation where there is a large amount of information, the information presentation device 5 presents the presentation timing of the information with the lower priority order later than the presentation timing of the information with the higher priority order.

[Functional configuration of visual load determination device 1]
The visual load determination device 1 configured as described above stores program data for calculating the visual load in a ROM (Read Only Memory) or the like (not shown) and executes the program data by a CPU (Central Processing Unit). Thus, as shown in FIG. 5, the visual load calculation unit 11 and the information provision determination unit 12 have functions, and the visual load calculation unit 11 includes the respective units 41 to 49. And the visual load judgment apparatus 1 performs information processing by each part 41-49, and information provision judgment whether the pop-up information by a navigation system, driving support information, such as ITS driving | operation information is shown by the information presentation apparatus 5 is provided. The information is determined by the unit 12 and the information control device 21 is controlled.

  Here, the line-of-sight / blink state measurement device 2 includes, for example, an eye tracker system, and includes a viewpoint detection unit 51 that generates viewpoint coordinate data and a blink state detection unit 52 that generates blink data. ing. The viewpoint detection unit 51 and the blink state detection unit 52 perform predetermined image processing using the face image data captured by the face image camera 31, and the viewpoint detection unit 51 includes viewpoint coordinate data indicating the driver's viewpoint position. The blink state detection unit 52 generates blink data indicating the presence / absence of the driver's blink action, and outputs the blink data to the visual load determination device 1 every predetermined period, for example.

  The visual load determination device 1 includes a process execution determination unit 41 connected to the viewpoint detection unit 51, a travel state determination unit 42 connected to the vehicle speed sensor 3, a viewpoint movement amount calculation unit 43, a blink presence / absence determination unit 44, and a viewpoint stop. A coordinate calculation unit 45, a stop point position determination unit 46, a stop point position temporary storage unit 47, a viewpoint movement blink pattern determination unit 48, and a high load blink pattern frequency calculation unit 49 connected to the information provision determination unit 12 are provided. .

  The traveling state determination unit 42 determines whether the host vehicle 20 is traveling or stopped from the vehicle speed signal from the vehicle speed sensor 3, and outputs the determination result to the processing execution determination unit 41 as traveling state information.

  The process execution determination unit 41 determines whether or not to execute processing by each of the units 41 to 49 and the information provision determination unit 12 described later based on the driving state information input from the driving state determination unit 42. Then, the viewpoint coordinate data is read from the viewpoint detection unit 51 and output to the viewpoint movement amount calculation unit 43, and the blink data is read from the blink state detection unit 52 and output to the blink presence / absence determination unit 44.

  The viewpoint movement amount calculation unit 43 calculates the data indicating the movement amount between the viewpoints and sends it to the viewpoint stop coordinate calculation unit 45 when a plurality of viewpoint coordinate data that are temporally moved back and forth are sent from the process execution determination unit 41. . The data indicating the movement amount of the viewpoint indicates the movement distance per unit time of the plurality of viewpoint coordinate data.

  When the data indicating the movement amount of the viewpoint is sent from the viewpoint movement amount calculation unit 43, the viewpoint stationary coordinate calculation unit 45 determines the stationary state of the viewpoint. At this time, the viewpoint stop coordinate calculation unit 45 normally stops when the driver recognizes the visual recognition target, because each viewpoint moves slightly, so that the movement per unit time is equal to or less than a predetermined value. Judge. Thereby, the viewpoint stationary coordinate calculation unit 45 determines whether or not the viewpoint is in a stationary state for each viewpoint coordinate data. Then, the viewpoint stop coordinate calculation unit 45 sends the viewpoint coordinate data and the stop determination result to the stop point position determination unit 46.

  The stop point position determination unit 46 determines whether the stop point coordinate based on the stop determination result sent from the viewpoint stop coordinate calculation unit 45 is a coordinate range in which it is determined that the rear side indirect viewing device such as a door mirror is viewed, or the own vehicle It is determined whether the coordinate range is determined to be 20 when looking forward in the traveling direction. Then, the stop point position determination unit 46 discards the viewpoint coordinate data and the stop determination result when the rear side indirect view device or the coordinate range that is not determined to be looking forward in the traveling direction, When the coordinate range is determined to be looking at the indirect vision device or the front in the traveling direction, the viewpoint coordinate data and the stop determination result are stored in the stop point temporary storage unit 47.

  When the blinking presence / absence determining unit 44 receives a plurality of blink data that are temporally moved from the process execution determining unit 41 and the viewpoint movement amount calculating unit 43 transmits data indicating the movement amount of the viewpoint, the viewpoint is changed. It is determined whether a blink action has occurred when moving. Thereby, the blink presence / absence determination unit 44 extracts the viewpoint coordinate data detected when the blink action occurs. Then, the blink presence / absence determination unit 44 sends the viewpoint coordinate data when the blink action occurs and the viewpoint coordinate data based on the viewpoint movement not accompanied by the blink action as the blink presence / absence determination result to the blink pattern determination unit 48 when the line of sight moves. .

  The eye movement blink pattern determination unit 48 uses the viewpoint coordinate data and the stop determination result stored in the stop point temporary storage unit 47, the blink presence / absence determination result and the viewpoint coordinate data sent from the blink presence / absence determination unit 44. Then, it is determined whether any of the blink patterns shown in FIG. Then, the eye-gaze blink pattern determination unit 48 sends information indicating the blink pattern determined to be applicable from the viewpoint movement and the presence / absence of the blink to the high-load blink pattern frequency calculation unit 49 as a determination result.

  The high-load blink pattern frequency calculation unit 49 is identified as having a high visual load when visually recognizing the rear side indirect vision device among the blink patterns sent from the blink pattern determination unit 48 when moving the line of sight. The frequency of the blink pattern or the fourth blink pattern is calculated. When the high-load blink pattern frequency calculation unit 49 determines that the frequency of the third blink pattern or the fourth blink pattern is high, the high-load blink pattern frequency calculation unit 49 sends the fact to the information provision determination unit 12.

  Thereby, the visual load determination device 1 controls the information presentation timing of the information presentation device 5 and the information presentation content by controlling the information presentation of the information control device 21 by the information provision judgment unit 12.

[Visual load calculation processing by visual load determination device 1]
Next, a processing procedure for calculating the visual load by the visual load determination device 1 having the functional configuration as described above will be described with reference to the flowchart of FIG.

  First, in step S1, the visual load determination device 1 reads the vehicle speed signal from the vehicle speed sensor 3 by the travel state determination unit 42 and sends the determination result of determining the travel state to the process execution determination unit 41, and in step S2, the process execution is performed. The determination unit 41 determines whether or not the traveling state of the host vehicle 20 is stopped. Then, the process execution determination unit 41 ends the process when it is determined that the host vehicle 20 is stopped, and proceeds to step S3 when it is determined that the vehicle 20 is not stopped. The viewpoint coordinate data is read from the viewpoint detection unit 51 and output to the viewpoint movement amount calculation unit 43.

  Next, the viewpoint movement amount calculation unit 43 calculates the viewpoint movement speed by calculating the movement amounts of the plurality of viewpoint coordinate data per unit time in step S4, and in step S5, the viewpoint movement speed calculated in step S4. It is determined whether or not the viewpoint is moving by determining whether or not the speed is higher than a predetermined speed. As the predetermined speed with respect to the viewpoint moving speed, a speed at which the viewpoint moves more than a moving amount that is not recognized to be stopped per unit time is stored in advance by experiments or the like. If the visual load determination device 1 determines that the viewpoint movement amount calculation unit 43 is moving the viewpoint, the visual load determination device 1 outputs the viewpoint coordinate data to the blink presence / absence determination unit 44 and proceeds to step S7. If the viewpoint movement amount calculation unit 43 determines that the viewpoint is not moving, the viewpoint coordinate data is output to the viewpoint stop coordinate calculation unit 45, and the process proceeds to step S6.

  In step S <b> 6, the viewpoint stop coordinate calculation unit 45 calculates the average coordinates of the viewpoint coordinate data from the viewpoint movement amount calculation unit 43, calculates the stop point coordinates, and outputs them to the stop point position determination unit 46. Then, the process proceeds to step S9.

  On the other hand, in step S7, the blink presence / absence determination unit 44 receives blink data acquired in the same period as the viewpoint coordinate data used to determine that the viewpoint is moving in step S5 from the process execution determination unit 41. It is acquired and it is determined from the blink data whether or not a blink during viewpoint movement has occurred. In step S <b> 8, the blink presence / absence determination unit 44 increments a counter value that counts that a blink has occurred during movement of the viewpoint.

  In the next step S9, whether or not the stop point coordinates calculated in step S6 by the stop point position determination unit 46 are within the viewpoint range of a rear side indirect viewing device such as a mirror registered in advance. To determine whether the stop point coordinates are a stop point in the forward direct view or a stop point in the rear side indirect view. Then, when the stop point coordinates are a stop point in the rear side indirect field of view, the stop point position determination unit 46 uses a mirror gaze that is used to determine the moving direction of the viewpoint in a process described later as a determination result. The flag value is set to “1” and stored in the stopping point position temporary storage unit 47, and the process returns to step S1.

  On the other hand, if the stop point coordinates are not within the viewpoint range of the rear side indirect field of view device such as a mirror, in step S11, by referring to the value of the mirror gaze flag by the eye movement pattern blink determining unit 48. Determine whether the viewpoint has moved from the rear side indirect view device such as a mirror immediately before, and determine whether the stop point coordinates immediately before the current process were a rear side indirect view device such as a mirror. To do. As a result, the viewpoint movement started immediately before the stop coordinates other than the rear side indirect viewing device such as the mirror in the current process are changed from the rear side indirect viewing device such as the mirror to the forward direct view direction. It is determined whether or not the viewpoint is moved. If it is determined that the viewpoint movement is not started from a rear side indirect viewing device such as a mirror, the value of the mirror gaze flag is set to “0” in step S12, and the process returns to step S1. If it is determined that the viewpoint movement is started from the rear side indirect viewing device such as a mirror, the value of the mirror gaze flag is set to “0” in step S13, and the process proceeds to step S14.

  Next, in step S14, the blink pattern determination unit 48 at the time of line-of-sight movement refers to the count value set in step S8 to determine whether or not a blink has occurred during the viewpoint movement. If it is determined that the process has been completed, the process returns to step S1. As a result, the visual load determination device 1 causes the third blink pattern and the fourth blink that no blink occurs on the return path in which the viewpoint moves from the viewpoint toward the rear side indirect vision device to the viewpoint toward the front direct vision. Since it does not correspond to the pattern, the process returns to step S1.

  On the other hand, if it is determined that no blink has occurred during the movement of the viewpoint, the blink pattern determination unit 48 at the time of eye movement moves the blink pattern determination result to the high load blink pattern frequency calculation unit 49. Then, in step S15, the high-load blink pattern frequency calculation unit 49 generates a third blink pattern in which no blink occurs on the return path from the viewpoint to the rear side indirect view device to the view to the front direct view or Since it corresponds to the fourth blink pattern, the counter value of the number of times the blink pattern with a high visual load is detected is counted up, and in step S16, the third blink pattern or the fourth blink pattern with a high visual load is detected. Calculate the frequency of occurrence per unit time.

  Next, the high-load blink pattern frequency calculation unit 49 determines whether the occurrence frequency per unit time of the third blink pattern or the fourth blink pattern calculated in step S16 exceeds a predetermined value. This predetermined value is a value that considers that the frequency with which the third blink pattern or the fourth blink pattern is detected is higher as the driver feels higher in visual recognition load. The frequency at which the third blink pattern or the fourth blink pattern is repeated in a scene where there is another vehicle on the rear side at the time of merging, or the driver with low driving ability is the third blink pattern or the fourth blink pattern The frequency etc. which repeat is obtained by experiment.

  If the high-load blink pattern frequency calculation unit 49 determines that the occurrence frequency per unit time of the third blink pattern or the fourth blink pattern exceeds a predetermined value, the operation is performed in step S18. The value of the support information provision flag is set to “1”, the value of the low importance information stop flag is set to “1” in step S19, and the processing is performed after each flag is sent to the information provision determination unit 12. finish. On the other hand, if the high load blink pattern frequency calculation unit 49 determines that the frequency of occurrence of the third blink pattern or the fourth blink pattern per unit time does not exceed the predetermined value, the high load blink pattern frequency calculation unit 49 reduces the frequency in step S19. After the value of the importance level information stop flag is set to “1” and the flag is sent to the information provision determination unit 12, the process is terminated.

  Thereafter, the information provision determination unit 12 controls the information supplied from the information control device 21 to the information presentation device 5 according to the flag value from the high load blink pattern frequency calculation unit 49.

  Here, information to be presented to a driver with a high visual load includes information with low urgency and information for supporting driving operation. This low urgency information is a pop-up information such as a navigation system or a warning such as a fuel warning that does not cause a problem even if the information presentation time is delayed for a certain period, for example, several seconds. Such information with low urgency may hinder recognition of the rear side for a driver with a high visual load, so information provision is stopped or put on hold for several seconds. Therefore, when the value of the low importance level information stop flag is set to “1”, the information provision determination unit 12 displays an information control signal for stopping or holding the presentation of information with low urgency level. 12 is output. And the information provision judgment part 12 makes the output of the information with low urgency preset among the information which should be output to the information presentation apparatus 5 from the information provision apparatus 4 stop or hold | maintain.

  In addition, as described above, the information for assisting driving operation is likely to cause insufficient recognition of the rear side when the visual load is high as described above. Information for supplementing, for example, the presence or absence of other vehicles on the rear side existing within a predetermined distance. It is desirable to provide information supporting the driving operation as information other than visual information such as voice to a driver who has a high visual load. Therefore, when the value of the driving support information provision flag is set to “1”, the information provision determination unit 12 outputs an information control signal that presents information for assisting the driving operation to the information provision determination unit 12. . And the information provision judgment part 12 acquires the information which supports the driving operation set beforehand from the information provision apparatus 4, and makes the information presentation apparatus 5 present.

  Thus, even if the frequency of detecting the third blink pattern or the fourth blink pattern in step S17 is not high, by stopping or holding off the presentation of information with low urgency, It is avoided that the visual load is further hindered. On the other hand, in the case where the third blink pattern or the fourth blink pattern is detected at a high frequency in step S17, in addition to the stop or hold of the presentation of information with low urgency, the rear side situation Information that facilitates grasping is presented to reduce the visual load.

  In this example, the information provision determination unit 12 stops providing information with low urgency when a blink pattern with a high visual load of the third blink pattern or the fourth blink pattern is detected once. Alternatively, when the blink pattern having a high visual load of the third blink pattern or the fourth blink pattern exceeds twice, information for assisting the driving operation is presented.

  In the visual load determination device 1 that performs such processing, a merging scene is given as a typical driving scene when the visual load of the rear side indirect field of view obtained through a mirror or the like is high. In the scene, as shown in FIG. 7, the viewpoint coordinates and the presence or absence of blinks are detected.

  According to FIG. 7, the viewpoint coordinates move among the right door mirror, the front, the room mirror, the meter, the switches, and the like, and the visual load determination device 1 detects the presence or absence of blinks when the viewpoint movement occurs. To do. In the visual load determination device 1, the case indicated by ● in the figure is “viewpoint movement with blinking”, the case of ○ is “viewpoint movement without blinking”, and ◎ is “not blinking completely” By determining “half-open state”, the blink pattern can be determined.

  The visual load determination device 1 detects the line-of-sight position and the presence or absence of blinks as shown in FIG. 7A when the visual load in which no other vehicle is present behind the host vehicle 20 does not increase. . Accordingly, the visual load determination device 1 can detect that the frequency of blinking is low when the viewpoint moves forward from the right door mirror.

  On the other hand, the visual load determination device 1 has a line-of-sight position and a blink when there is another vehicle behind the host vehicle 20 and the visual load becomes high in order to check the traveling lane, distance, and the like. When the presence / absence is detected as shown in FIG. 7B, it can be detected that the frequency of blinking is high when the viewpoint is moved forward from the right door mirror.

[Effect of the first embodiment]
As described above in detail, according to the visual load determination device 1 according to the first embodiment to which the present invention is applied, the eye movement and the blink movement are detected, and the rear side indirect visual field device moves directly to the front direct field. Since the rear side visual load is determined using the change in the blink pattern when the direction of the viewpoint is moved, the state of the visual load can be determined without imposing a load on the driver. Therefore, according to the visual load determination device 1, the way of providing information can be changed according to the visual load, and it is possible to suppress giving a sense of humor and anxiety by presenting information when the visual load is high. be able to. Moreover, in this visual load judgment apparatus 1, the range of driver state monitoring, such as at the time of lane change or merging, can be greatly improved.

  Moreover, according to this visual recognition load judgment apparatus 1, when it is judged that it is the 3rd blink pattern or the 4th blink pattern among the 1st-4th blink patterns, it is visually recognized with respect to a rear side indirect view. Since it is determined that the load is high, the visual calculation load can be determined in real time even with a calculation function with a low processing capability by simplifying the calculation method.

  Further, according to the visual load determination device 1, the third blink pattern or the fourth blink pattern is detected, the visual load on the rear side is high, and the driving ability of the driver corresponding to the driving environment is sufficiently high. When it is determined that there is no information, the presentation of information with low urgency is stopped or put on hold, so that it is possible to avoid an increase in visual load and a decrease in driving ability by providing information with low urgency.

  Furthermore, according to the visual load determination device 1, when the third blink pattern or the fourth blink pattern is detected at a predetermined frequency, it is determined that the visual load on the rear side is very high. Since the information for assisting the driving operation is presented, the visual load can be reduced efficiently and the rear side situation can be quickly grasped. Moreover, in this case, it is possible to suppress a reduction in the viewing load by presenting information by voice instead of visual information.

[Second Embodiment]
Next, the visual load determination device 1 according to the second embodiment will be described. In addition, about the part similar to the above-mentioned 1st Embodiment, the detailed description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  In addition to the functions described in the first embodiment, the visual load determination device 1 according to the second embodiment measures the phase difference and eye movement amount of the blinking time of each of the driver's right eye and left eye. The visual load is determined to be high based on the phase difference and the line-of-sight movement amount. In the first embodiment described above, the direct field of view is the front of the vehicle to be visually recognized by the driver, and the indirect field of view is the viewpoint to the rear side indirect field of view device, but in the second embodiment, indirect The field of view is a field of view including a direct field of view other than the front.

[Configuration of Visual Load Determination Device 1]
As shown in FIG. 8, the visual load determination device 1 according to the second embodiment is connected to the blink presence / absence determination unit 44 in addition to the visual load determination device 1 according to the first embodiment. A road environment acquisition unit 71 that includes a phase difference calculation unit 61 and acquires road environment information including weather, road surface, road line type, road width, curvature, and the like, and a traffic congestion state (congestion degree). ) Including road condition information, a driving education database 73, a neurological disease determination unit 74 for determining the possibility of cranial nerve disease, and a vehicle control unit 75 including an automatic driving device. It is connected.

  In addition, the blink state detection unit 52 of the gaze / blink state measurement device 2 in the second embodiment performs predetermined image processing on an image including both eyes of the driver so that the eyelid is opened and the pupil is opened. Eyeblink data is generated that distinguishes between an open eye state in which the eyelid can be confirmed and a closed eye state in which the eyelid is closed and the pupil is hidden.

  The blink presence / absence determination unit 44 changes from the opened eye state to the closed eye state for each of the right and left eyes of the driver from the blink data supplied from the blink state detection unit 52 via the process execution determination unit 41. Further, the blink time until the eye changes from the closed state to the open state is acquired. Then, the blink presence / absence determination unit 44 supplies the blink time of each of the left and right eyes to the left and right eye blink phase difference calculation unit 61.

  When the right and left eye blink phase difference calculation unit 61 acquires the blink time of the right eye and the blink time of the left eye, it calculates the time difference between the blink times and calculates the phase difference of the left and right eyes. The phase difference between the left and right eyes is supplied to the information provision determination unit 12 and used for calculating the visual load by the information provision determination unit 12.

  Specifically, as shown in FIG. 9, when the driver's left eye changes in the order of the open eye state, the closed eye state, and the open eye state as shown in FIG. 9A, there is no phase difference between the left and right eyes. 9B, the right eye of the driver changes in the order of the open eye state, the closed eye state, and the open eye state in synchronization with the left eye of the driver. On the other hand, when the phase difference between the left and right eyes occurs, even when the left eye of the driver is determined to be closed, the right eye of the driver is determined to be open as shown in FIG. Even when the left eye of the driver is determined to be in the open state, the right eye of the driver is determined to be in the closed state.

  When the left and right eye blink phase difference calculation unit 61 determines that there is a phase difference of the blink time, the information provision determination unit 12 calculates the phase difference and the viewpoint movement amount calculated by the viewpoint movement amount calculation unit 43. Read data. Thereby, the information provision judgment part 12 judges whether a driver | operator's present visual burden is a high state, or the possibility of a disease is high. At this time, as illustrated in FIG. 10, the information provision determination unit 12 refers to a condition table that describes a correspondence relationship between a preset condition regarding the phase difference between the left and right eyes and a condition regarding the line-of-sight movement amount. In addition, the judgment reference value of the phase difference between the left and right eyes and the amount of eye movement that the visual load is high or the possibility of the disease is high may be derived from the results of the experiment on the subject. Alternatively, it may be determined comprehensively by learning and judging, and may be appropriately changed according to various situations.

  When it is determined by the information provision determination unit 12 that the visual load is high, the information provision determination unit 12 determines whether or not to provide driving support information such as driving information, as in the first embodiment. The control device 21 is controlled. In addition, when the information provision determining unit 12 determines that there is a high possibility of a disease, the information control device 21 and the display unit 5 alert the neurological disease determining unit 74 to that effect.

  As described above, in the second embodiment, when it is determined that the visual load is high, the road environment information acquired by the road environment acquisition unit 71 and the road condition acquired by the road condition acquisition unit 72 are provided as information provision determination units. 12 and the road environment and the road situation when it is determined that the visual load is high, the own vehicle information acquired by the vehicle control unit 75, and driver-specific information such as age, Learn the situation where the price increases. Then, the information provision determination unit 12 acquires from the driving education method database 73 a driving education method that matches the road environment, road conditions, own vehicle information, and driver-specific information that increases the visual load, and the information control device 21. To present.

  In this way, the education program stored in the driving education method database 73 is presented to the driver to educate the driving method when the visual load becomes high. In addition, as a timing which shows an educational program, it is desirable to carry out at the time of a stop or parking with sufficient driving operation, for example.

[Visual load calculation processing by visual load determination device 1]
Next, a processing procedure for calculating the visual load by the visual load determination device 1 having the functional configuration as described above will be described with reference to the flowchart of FIG. Note that the same step number is used for the same processing as in the first embodiment described above.

  First, in step S1, the visual load determination device 1 reads the vehicle speed signal from the vehicle speed sensor 3 by the travel state determination unit 42 and sends the determination result of determining the travel state to the process execution determination unit 41, and in step S2, the process execution is performed. The determination unit 41 determines whether or not the traveling state of the host vehicle 20 is stopped.

  Then, in step S2, the process execution determining unit 41 ends the process when it is determined that the host vehicle 20 is stopped, and proceeds to step S3 when it is determined that the vehicle is not stopped. The viewpoint coordinate data is read from the viewpoint detection unit 51 and output to the viewpoint movement amount calculation unit 43. Note that, when the driver is resting at rest, etc., when determining a neurological disease or the like using the left and right eye blink phase difference, the process proceeds to step S3 without performing the process of step S2.

  Next, the viewpoint movement amount calculation unit 43 calculates the viewpoint movement speed by calculating the movement amounts of the plurality of viewpoint coordinate data per unit time in step S4, and in step S21, the viewpoint movement speed calculated in step S4. It is determined whether or not a viewpoint movement corresponding to a saccade (a jumping movement, a rapid eye movement) is performed by determining whether or not is equal to or higher than a predetermined speed. As the predetermined speed for this viewpoint movement speed, a speed that is recognized as a saccade is set. Therefore, in step S4, the viewpoint movement amount calculation unit 43 proceeds to step S22 if the viewpoint movement speed corresponding to the saccade is equal to or higher, and returns to step S1 otherwise.

  In step S22, the viewpoint movement amount calculation unit 43 acquires the line-of-sight movement amount for the line-of-sight movement recognized as a saccade, and advances the process to step S23. In step S23, it is determined whether or not the movement of the line of sight is recognized as a saccade, and the amount of movement of the line of sight is a movement amount that easily induces a reflective blink. For example, it is recognized that a reflective blink is likely to occur when the line-of-sight movement amount corresponds to a viewing angle of 20 degrees or more from the current viewpoint that is directly in the forward visual field. Accordingly, in step S23, the viewpoint movement amount calculation unit 43 determines whether or not the line-of-sight movement amount calculated in step S22 is a line-of-sight movement amount of 20 degrees or more within the visual angle. The process proceeds to step S24, and if not, the process proceeds to step S28.

  Here, in order to determine the level of the visual load from the phase difference between the left and right eyes in the subsequent stage, it is desirable to use data at the time of the reflective blink, not the voluntary blink or the spontaneous blink. In addition, during a normal life, it is said that the amount of line-of-sight movement during saccade is within 15 degrees in viewing angle, which accounts for 85%. Therefore, the line-of-sight movement amount corresponding to 15 degrees or more at the viewing angle is defined as a relatively large line-of-sight movement amount, and when the vehicle is driven, it is a condition that the line-of-sight movement amount corresponds to 20 degrees or more at the viewing angle. Assume that high and low are judged. Note that the line-of-sight movement to the door mirror as the rear side indirect vision device is about 35 to 40 degrees in view angle.

  In step S24, the right and left eye blink phase difference calculation unit 61 reads the right eye blink time and the left eye blink time calculated by the blink presence / absence determination unit 44. The blinking time of the right eye and the blinking time of the left eye are the reflections when it is determined in step S21 that the line of sight is equivalent to a saccade, and in step S23, the line of sight movement is determined to be 20 degrees or more in the viewing angle. It was obtained by detecting sexual blinks.

  In the next step S25, the left and right eye blink phase difference calculation unit 61 calculates the time difference between the right eye blink time and the left eye blink time read in step S24, thereby calculating the left and right eye phase difference. Calculate.

  In the next step S26, the left and right eye blink phase difference calculation unit 61, as shown in FIG. 9, for example, when the open / closed state of the left eye changes as shown in FIG. When it changes as shown in B), it is determined that there is no phase difference between the left and right eyes, and when the right eye changes as shown in FIG. 9C, it is determined that there is a phase difference between the left and right eyes. When it is detected that there is a phase difference between the left and right eyes, the left and right eye blink phase difference calculation unit 61 compares the phase difference between the left and right eyes with a predetermined time, and the phase difference between the left and right eyes is equal to or longer than the predetermined time. It is determined whether or not.

  Here, the blinking time varies depending on various conditions such as individual differences and mental and physical conditions, but is generally about 0.1 seconds. Therefore, the predetermined period compared with the phase difference between the left and right eyes is set to 0.02 seconds or more, for example. Note that the blink time of the order of 0.02 seconds is a range that can be measured by a device such as an existing line-of-sight measuring instrument or an eye mark camera, and can be realized by using the device as the blink state detection unit 52. It is. The predetermined time for the phase difference between the left and right eyes may be obtained by calculating the average and frequency distribution of the blink time by the left and right eye blink phase difference calculation unit 61. If it is determined in step S26 that the phase difference between the left and right eyes is not 0.02 seconds or more, the condition table shown in FIG. 10 corresponds to “the visual load is within the capability range”. To return.

  On the other hand, if the phase difference between the left and right eyes is 0.02 seconds or more in step S26, it corresponds to “the visual load exceeds the capability range” in the condition table of FIG. Proceed.

  In the next step S27, the information provision determination unit 12 stops presenting low-importance pop-up information or the like and presents road conditions such as the degree of congestion for driving support. At this time, the road environment and the road situation when it is determined that the visual load is high are acquired from the road environment acquisition unit 71 and the road state acquisition unit 72, and the own vehicle information is acquired from the vehicle control unit 75, Add information specific to the driver such as age to learn the situation where the visual burden increases.

  In step S27, a driving education method that matches the road environment, road conditions, host vehicle information, and driver-specific information that increases the visual load is obtained from the driving education method database 73, and the information control device 21 Let them present. Thus, the education program stored in the driving education method database 73 is presented to the driver to educate the driving method when the visual load increases.

  As described above, when the line-of-sight movement is recognized as a saccade and the amount of line-of-sight movement is 20 degrees or more in the visual angle, the image visually recognized during the line-of-sight movement is recognized as noise and is not transmitted to the driver's visual center. A blinking action is performed. This blink action is classified as a reflective blink. On the other hand, in order to acquire the current visual information with a high visual load and no time for blinking, it is considered that an action for suppressing the blinking action occurs as a spontaneous blink. Therefore, in such a saccade, particularly in elderly people whose physical functions have deteriorated, a phase difference is likely to occur in the blinking eyes of the left and right eyes, and it can be determined that the current driving load and viewing load are high. . In terms of the phenomenon, the phase difference increases in the blinking time for both eyes, and the accuracy of grasping the distance to the surrounding object and grasping the speed is expected due to the difference in the visual start and end times of the left and right eyes. It can be said that it is a high situation as a visual load. On the other hand, in step S27, an education program is presented to the driver to educate the driving method when the visual load increases.

  Further, in step S23, in which it is determined that the line-of-sight movement amount is less than the line-of-sight movement amount corresponding to 20 degrees or more at the viewing angle, the right eye calculated by the blink presence / absence determination unit 44 is the same as in step S24. The blinking time of the left eye and the blinking time of the left eye are read, and in step S29, as in step S25, the time difference between the blinking time of the right eye and the blinking time of the left eye read in step S28 is calculated. The phase difference between the left and right eyes is calculated.

  In step S30, as in step S26, it is determined whether or not the phase difference between the left and right eyes is equal to or longer than a predetermined time. If it is determined in step S30 that the phase difference between the left and right eyes is not 0.02 seconds or more, the process returns to step S1 because it corresponds to “healthy person, general” in the condition table of FIG.

  On the other hand, if it is determined in step S30 that the phase difference between the left and right eyes is 0.02 seconds or more, the condition table of FIG. To proceed.

  In step S <b> 31, the information provision determination unit 12 provides information to a driver who may have a disease. Here, when the phase difference between the left and right eyes occurs although the amount of line-of-sight movement is not large, there may be a cranial nervous system disease such as multiple sclerosis. Therefore, in this step S31, the neurological disease determination unit 74 determines that the information provision determination unit 12 has detected the high possibility of the disease, and notifies the driver using the information presentation device 21 and the display unit 5. , Do a warning. Furthermore, the vehicle control unit 75 can control various devices of the vehicle (starting of an automatic driving device, etc.) and contribute to a stable driving operation.

[Effects of Second Embodiment]
As described above in detail, according to the visual load determination device 1 according to the second embodiment to which the present invention is applied, the blink presence / absence determination unit 44 performs a viewpoint movement between the forward direct view and the indirect view. The presence or absence of blinks in the right eye and left eye of the driver is determined, the blink time of each of the right eye and left eye when there is the blink, and the left and right eye blink phase difference calculation unit 61 Because the phase difference between the left and right eyes is calculated from the time difference between the blink time of the right eye and the blink time of the left eye, the phase difference between the left and right eyes is also used to determine the visual load. It is possible to accurately determine the visual load on the indirect field of view. That is, the determination of the visual load based on the blink pattern shown in FIG. 1 depends on the individual characteristics or the accuracy of determining the visual load is not sufficient when the absolute number of blink data is small. As shown in FIG. 10, the phase difference between the left and right eyes can also be used to determine the visual load, thereby improving the accuracy of the visual load determination.

  Specifically, in the first embodiment, when it corresponds to the third blink pattern or the fourth blink pattern, it is determined that the visual load is high. If detected, it can be determined that the visual load is high. Thereby, even if it is a case where the number of blink data decreases by an individual, visual recognition load can be judged correctly. Further, in the second embodiment, as the indirect field of view, the front direct field of view and the front side are used without using the viewpoint to the rear side indirect field device that allows the driver such as a door mirror to visually recognize the rear side. The visual load can be determined only by the movement of the line of sight between the direct visual fields other than.

  Further, according to the visual load determination device 1, a combination pattern of the viewpoint movement amount and the left-right eye phase difference as shown in FIG. 10 is determined, and the left-right eye phase difference is large and the line-of-sight movement amount is large. When the combination pattern is a predetermined combination pattern, it can be determined that the visual load is high.

  Furthermore, according to the visual load determination device 1, it is possible to acquire road conditions when it is determined that the visual load is high and present information that supports driving operations corresponding to the road conditions. It is possible to stop presenting information such as pop-up information and to present road conditions such as the degree of congestion for driving assistance.

  Furthermore, according to this visual load determination device 1, since it is possible to acquire the road environment when it is determined that the visual load is high and to present information that supports the driving operation corresponding to the road environment, it is less important. It is possible to stop presenting information such as pop-up information and to present road environment such as weather and road type for driving assistance.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

  That is, the visual load determination device 1 has been described with respect to the case of determining the visual load for grasping the situation of the rear side. Yes, it can be applied to a wide range of scenes that include indirect visual recognition.

It is a figure for demonstrating the blink pattern divided according to the relationship between the outward path | route of the viewpoint to a back side indirect vision device, and the viewpoint to a front direct view, a return path, and the presence or absence of a blink action. It is a processing conceptual diagram of the visual load judgment device concerning a 1st embodiment to which the present invention is applied, (a) is change of viewpoint movement, (b) is change of presence or absence of blink, (c) is at the time of viewpoint movement. (D) is a figure which shows a blink pattern. It is a block diagram which shows the functional structure of the visual load judgment apparatus which concerns on 1st Embodiment to which this invention is applied. It is a block diagram which shows the structure at the time of mounting the visual load judgment apparatus which concerns on 1st Embodiment to which this invention is applied to the own vehicle. It is a block diagram which shows the functional internal structure of the visual load judgment apparatus which concerns on 1st Embodiment to which this invention is applied. It is a flowchart which shows the process sequence of the visual load judgment process of the visual load judgment apparatus which concerns on 1st Embodiment to which this invention is applied. It is a figure for demonstrating the judgment of the viewpoint coordinate in a merge scene, the presence or absence of a blink, and a blink pattern, Comprising: (a) is the case where there is no other vehicle in the rear side, (b) is the other in the rear side This is the case when a vehicle is present. It is a block diagram which shows the functional internal structure of the visual load judgment apparatus which concerns on 2nd Embodiment to which this invention is applied. It is a figure for demonstrating the phase difference of a left eye, (A) is a change of the open state of a left eye, (B) is the open state of the right eye when there is no phase difference with respect to the open state of a left eye. Change (C) is a change in the open state of the right eye when there is a phase difference with respect to the open / closed state of the left eye. It is a figure which shows the condition table which judges visual recognition load from the relationship between the phase difference of right-and-left eyes, and a gaze movement amount. It is a flowchart which shows the process sequence of the visual load judgment process of the visual load judgment apparatus which concerns on 2nd Embodiment to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Visual load judgment apparatus 2 Gaze / blink state measurement apparatus 3 Vehicle speed sensor 4 Information provision apparatus 5 Information presentation apparatus 11 Visual load calculation part 12 Information provision judgment part 21 Information control apparatus 31 Face image camera 41 Process execution judgment part 42 Running state judgment Unit 43 Viewpoint movement amount calculation unit 44 Blink presence / absence determination unit 45 Viewpoint stop coordinate calculation unit 46 Stoppoint position determination unit 47 Stoppoint position temporary storage unit 48 Eyeblink movement blink pattern determination unit 49 High load blink pattern frequency calculation unit 51 viewpoint detection unit 52 blink state detection unit 61 left and right eye blink phase difference calculation unit 71 road environment acquisition unit 72 road condition acquisition unit 73 driving education method database 74 neurological disease determination unit 75 vehicle control unit

Claims (10)

In the visual load determination device that determines the visual load of the driver who visually recognizes the front direct view and the indirect view including the direct view other than the front,
Viewpoint coordinate detection means for detecting the driver's viewpoint coordinates;
Blink state detecting means for detecting presence or absence of the driver's blink action;
Viewpoint movement determining means for determining that the viewpoint movement is a viewpoint movement between the forward direct view and the indirect view based on the viewpoint coordinates detected by the viewpoint coordinate detection means;
Based on the presence or absence of the blink action detected by the blink state detection means, the presence or absence of the blink action in the viewpoint movement period between the front direct view and the indirect view determined by the viewpoint movement determination means. Means for determining the presence or absence of a blink action to determine;
Determining a combination pattern of the direction of the viewpoint movement between the forward direct view and the indirect view determined by the viewpoint movement determination unit and the presence or absence of the blink action determined by the blink action presence / absence determination unit; Pattern judgment means;
A visual load determination device comprising: a visual load determination unit that determines a visual load with respect to the indirect field of view based on the pattern determined by the pattern determination unit. The pattern determining means determines a pattern without blinking when moving the viewpoint from the indirect field of view to the front direct field of view,
The visual load determination means determines that the visual load on the indirect view is high when it is determined that the pattern has no blinking action when the viewpoint moves from the indirect view to the front direct view. The visual load determination device according to claim 1, wherein the visual load determination device is a visual load determination device.   The visual load determination means stops or suspends presentation of information with low urgency when the pattern determination means detects a pattern without blinking when the viewpoint moves from the indirect view to the front direct view. The visual load judgment device according to claim 2, wherein:   The visual load determination means presents information for assisting a driving operation when the pattern determination means detects a pattern with no blinking action at a predetermined frequency when moving the viewpoint from the indirect field of view to the front direct field of view. The visual load determination device according to claim 2, wherein the visual load determination device is a visual load determination device. The indirect field of view is a rear side situation recognized by the driver by a mirror,
The said visual load judgment means judges the visual burden with respect to a back side indirect visual field based on the pattern judged by the said pattern judgment means. The one of Claim 1 thru | or 4 characterized by the above-mentioned. Visual load determination device. The blinking presence / absence judging means judges whether or not the driver's right eye and left eye each blink when the viewpoint moves between the front direct view and the indirect view, and there is the blink Measure the blink time of each of the right and left eyes in
It further comprises left and right eye blink phase difference calculating means for calculating the phase difference between the left and right eyes from the time difference between the blink time of the right eye and the blink time of the left eye measured by the blink presence / absence determining means. The visual load judgment device according to any one of claims 1 to 5.   The visual recognition load according to claim 6, wherein the visual load determination unit determines a visual load on the indirect field of view based on a phase difference between the left and right eyes calculated by the left and right eye blink phase difference calculation unit. Load judgment device.   The visual load determination means includes the amount of viewpoint movement between the front direct view and the indirect view determined by the viewpoint movement determination means, and the position of the left and right eyes determined by the left and right eye blink phase difference calculation means. The visual load determination device according to claim 7, wherein a combination pattern with a phase difference is determined and it is determined that the visual load is high when the combination pattern is a predetermined combination pattern. Road condition acquisition means for detecting a road condition including the degree of congestion around the vehicle,
When the visual load determination means determines that the visual load is high, the visual load determination means obtains a road condition when the visual load is determined to be high, and presents information that supports driving operation corresponding to the road condition The visual load determination device according to claim 1, wherein the visual load determination device is a visual load determination device. Road environment acquisition means for detecting road environment including weather or road type around the vehicle,
When the visual load determination unit determines that the visual load is high, the visual load determination unit acquires the road environment when the visual load is determined to be high, and presents information that supports driving operation corresponding to the road environment. The visual load determination device according to claim 1, wherein the visual load determination device is a visual load determination device.

Images