JP2012201232A - Visual line direction detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visual line direction detecting device for enabling more accurate calibration without giving a burden to a driver by collecting information for the calibration in a sufficiently wider range for no congestion to occur.SOLUTION: When operating an ODO meter reset button or being alarmed that a fuel level falls, a driver looks at an ODO meter or a fuel display. At this point, congestion occurs. During travel at a fixed speed or higher and a fixed distance away from a frontward vehicle, when first operating a brake, the driver looks at the other vehicle. At this point, no congestion occurs. When the congestion occurs (S322, S342, S362) and no congestion occurs as well (S384), the visual line direction detecting device automatically acquires calibration information to perform calibration, and so it can detect an accurate visual line direction.

Description

The present invention relates to a gaze direction detection device that detects a gaze direction of a driver who drives a vehicle.

A conventional gaze direction detection device includes a camera that captures a driver's face, and uses filter information (gaze detection model) that associates an image of the driver's face with the driver's gaze direction. Then, a process of detecting the driver's line-of-sight direction from the driver's face image taken by the camera is performed.
Of these, the filter information varies depending on individual differences among drivers, and it is necessary to perform correction (calibration) corresponding thereto.
As an example of this, when a switch of equipment such as a navigation device or an air conditioner provided in the host vehicle is operated, the equipment is operated assuming that the driver's line of sight is facing the direction of the equipment. A method is known in which a driver's face is photographed and calibration is performed based on image information (calibration information) of the photographed image (Patent Document 1).

JP 2010-30361 A

However, these pieces of equipment are only located within the reach of the driver sitting in the driver's seat, that is, within the range where the congestion of the driver's eyes approaching the center occurs. It has been.
Therefore, even if it is attempted to collect calibration information based on these, it can only be performed within the limited range, and thus only calibration with accuracy within this range can be performed.
In order to increase this accuracy, it is conceivable to separately perform a process of collecting calibration information. However, it is also conceivable that the driver is burdened with an operation load for executing this process or a burden associated with this process.
Therefore, the present invention extends the range of collecting calibration information to a range where congestion does not occur, performs more accurate calibration, and does not impose a burden on the driver due to the expansion of this range. An object is to provide a detection device.

  In order to achieve the above object, the calibration means constituting the gaze direction detecting device according to claim 1 of the present invention includes a non-congestion acquisition means for collecting calibration information at the time of non-congestion, and calibration information at the time of congestion. And a means for acquiring during congestion.

The non-congestion acquisition unit is configured to perform calibration information on the first image captured by the imaging unit when a first event occurs in which the driver's line of sight faces the first direction when no congestion occurs in the driver's eyes. Get as.
When the second event occurs when the driver's eyes are congested and the driver's line of sight is in the second direction, the acquisition unit at the time of congestion is used as calibration information. get.
Then, the calibration unit performs the calibration of the filter information based on the calibration information obtained by the non-congestion acquisition unit and the congestion acquisition unit.

  In this way, it is possible to collect both calibration information when congestion occurs and when congestion does not occur, so that accurate calibration of filter information can be performed.

  In the present invention, since the calibration information is collected when the first event or the second event unrelated to the collection of the calibration information occurs, the calibration information is collected up to the range where congestion occurs. Even if the range is expanded, calibration information can be collected without burdening the driver.

Next, in addition to the configuration of the gaze direction detection device according to claim 1, a gaze direction detection device according to claim 2 of the present invention includes a brake detection means for detecting depression of the brake of the host vehicle, and a gaze direction. It is determined whether the vehicle is traveling in a safe traveling state in which the separation distance from another vehicle traveling in front of the host vehicle including the detection device is more than a distance that can prevent a collision of the host vehicle with the other vehicle if a brake operation is performed. Traveling state determination means.
Then, the non-congestion acquisition means, as a first event, when there is a first stepping on the brake when the host vehicle is traveling in a safe traveling state, the driver's line of sight is directed to other vehicles whose first direction is the first direction. The first image taken by the photographing means when the brake is depressed is acquired as calibration information on the assumption that there is no congestion.
Therefore, in the present invention, calibration information when congestion is not generated can be acquired only by first depressing the brake in the safe driving state.
Next, the gaze direction detecting device according to claim 3 of the present invention is the equipment of the own vehicle as a second event in addition to the configuration of the gaze direction detecting device according to any one of claims 1 and 2. Is provided with first correspondence display means for performing a first correspondence display.
Then, when the equipment is operated, the acquisition means at the time of the congestion is assumed that the driver's line of sight is directed to the first correspondence display in the second direction and the congestion occurs, and the first correspondence display is present. The second image taken by the photographing means is acquired as calibration information.
For this reason, in the present invention, driving that is suitable for the first correspondence display that is displayed when the equipment is operated as calibration information when the equipment is operated only by operating the equipment of the automobile. The second image obtained by photographing the person's face can be reliably acquired.
Next, in addition to the configuration of the gaze direction detecting device according to any one of claims 1 to 3, the gaze direction detecting device according to claim 4 of the present invention provides notification to the driver as a second event. If done, second correspondence display means for performing a second correspondence display is provided.
Then, when the notification is given, the acquisition means at the time of congestion is taken when the second correspondence display is present on the assumption that the driver's line of sight is facing the second correspondence display in the second direction and congestion has occurred. A second image taken by the means is acquired as calibration information.
Therefore, in the present invention, the driver's face that is suitable for the second correspondence display that is displayed when the notification is made as the calibration information when the congestion occurs only by notifying the driver. It is possible to reliably acquire an image obtained by shooting.
Next, a gaze direction detection device according to a fifth aspect of the present invention is an image of a driver photographed by a photographing means in addition to the configuration of the gaze direction detection device according to any one of the first to fourth aspects. The face orientation calculating means for calculating the face orientation according to the above and the face orientation calculated by the face orientation calculating means when the second event occurs are scheduled when the driver's line of sight is in the second direction. And face direction determining means for determining whether the face is within the range of the face direction.
Then, when it is determined that the face direction calculated by the face direction calculating unit is not within the range of the planned face direction, the acquisition prohibiting unit uses the second image as calibration information. It is prohibited to acquire.
For this reason, in the present invention, even when the second event occurs, it is prohibited to acquire the second image as calibration information when the driver's face orientation deviates more than expected. So accurate calibration can be performed.

It is a block diagram of the gaze direction detecting device of this embodiment. It is a flowchart of the gaze direction monitoring process performed in this embodiment. It is a flowchart of the calibration information acquisition process performed in this embodiment. It is a flowchart of the calibration process performed in this embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[1. overall structure]
As shown in FIG. 1, the gaze direction detection device 1 of the present embodiment includes a central control device 3 that performs gaze direction monitoring processing described later, a face direction detector 4, a gaze direction detector 5, and an interface unit. 6 is provided.

  The central control device 3 includes a CPU 30 that executes various processes according to a program related to the line-of-sight direction monitoring process, a ROM 31 that stores the programs, and a RAM 32 that can write and erase various information used in the processes executed by the CPU 30. It has.

  Among these, the RAM 32 is set with storage areas for storing image information of images at the time of congestion and images at the time of non-congestion captured by the camera 7 described later in the line-of-sight direction monitoring process. Are referred to as an image storage unit 32a during congestion and an image storage unit 32b during non-congestion, respectively. Note that the image information stored in the image storage unit 32a during congestion and the image storage unit 32b during non-congestion are collectively referred to as calibration information.

  Further, in the RAM 32, a filter information storage unit 32c that stores filter information for detecting the driver's line-of-sight direction based on an image obtained by photographing the driver's face with the camera 7 is set.

The central control device 3 is communicably connected to various devices on the CAN via the interface unit 6.
The devices that can communicate with the central control device 3 include an ODO meter reset button 20, an ODO meter 21, a head-up display (hereinafter “HUD”), in addition to a camera 7 and a speaker 8 that are mainly used in a gaze direction monitoring process described later. A lighting display button 30, a brightness adjustment button 31 for HUD, and a projector 32 for HUD.

  Further, as devices capable of communicating with the central control device 3, there are a gasoline remaining amount indicator 40, a gasoline remaining amount indicator 41, a speed detector 50, a brake depression amount detector 51, and a forward monitoring device 52. There are an auto cruise device 100, an auto cruise operation state indicator 101, a cooling water temperature warning light 102, and the like.

  Among these, the camera 7 is installed in the instrument panel, and is arranged at a position where the face of the driver sitting in the driver's seat facing forward can be photographed from the front. The camera 7 may be installed other than the instrument panel. For example, you may install in the upper part of a steering column.

The speaker 8 outputs a sound or an alarm sound that is instructed to be output in a visual line direction monitoring process or the like.
The ODO meter reset button 20 is a button that can return the display of the ODO meter 21 to 0 when pressed. The ODO meter reset button 20 and the ODO meter 21 are disposed adjacent to each other on the instrument panel, and the ODO meter reset button 20 is disposed at a position operable from the driver on the instrument panel. The ODO meter 21 is disposed from the driver on the instrument panel. It is arranged at a position where it can be visually recognized.

  The projector 32 for HUD is a device that displays an image that appears to be displayed on a virtual image surface outside the vehicle over the windshield as viewed from the driver on a windshield (not shown).

  The HUD lighting display button 30 and the HUD luminance adjustment button 31 are a button for turning on and off the HUD function including the projector 32 and a button for adjusting the luminance of the image projected by the projector 32.

The lighting display button 30 and the brightness adjustment button 31 are arranged on the instrument panel, and the projector 32 is arranged in the instrument panel.
The gasoline remaining amount notification device 40 is a device that detects the remaining amount of gasoline in a gasoline tank (not shown) and notifies when the remaining amount has decreased.

  The gasoline remaining amount indicator 41 is a device that displays the amount of gasoline in the gasoline tank detected by the gasoline remaining amount indicator 40, and is installed at a position visible from the driver on the instrument panel.

The speed detector 50 is a device that detects the speed of the host vehicle.
The brake depression amount detector 51 is a device that detects the depression amount of the brake pedal of the host vehicle.

The forward monitoring device 52 is a device that mainly measures the distance to the amount of other vehicles traveling in front of the host vehicle. The distance measurement by the front monitoring device 52 may be performed by a millimeter wave, or a Lidar or a camera may be used.
[2. Gaze direction monitoring process]
Next, the gaze direction monitoring process executed by the central controller 3 will be described.

In the following description, it is assumed that the step numbers are executed in order from the smallest to the largest, unless otherwise specified.
This line-of-sight direction monitoring process is started when a start button (not shown) is operated.

  As shown in FIG. 2, as soon as the line-of-sight direction monitoring process is started, the process of S10 is executed. In S10, control for capturing the driver's face is executed by the camera 7, and an image of the driver's face is captured. The process of acquiring the image information is executed.

  The process of photographing the driver's face by the camera 7 performed in S10 is always performed while the line-of-sight direction monitoring process is being performed, and the following process is performed for each frame image captured in S10. Is done.

In S12, a process for determining whether a predetermined update period has elapsed since the previous calibration process (S50) was performed.
This update period may be, for example, 30 minutes.

  If it is determined in the process of S12 that the update period has elapsed (S12: YES), the process of S30 is executed next. If it is determined that the update period has not elapsed (S12: NO), then the process of S14 is performed. Is executed.

In S30 and S50, calibration information acquisition processing and calibration processing to be described later are executed.
In S14, the line-of-sight direction of the driver when the driver's face is photographed in S10 is determined from the image information of the image obtained by photographing the driver's face in the process of S10 and the filter information stored in the filter information storage unit 32c. Processing to detect is executed.

In S16, a process of determining whether the driver's line-of-sight direction detected in S14 is in a predetermined range assumed when looking away is executed.
If it is determined that the driver is not looking away in the process of S16, the process of S10 and subsequent steps is executed again. If it is determined that the driver is looking away, the process of emitting an alarm sound from the speaker 8 is executed in S18. Then, the processing after S10 is executed again.
[3. Calibration information acquisition process]
Next, calibration information acquisition processing executed by the central controller 10 will be described with reference to FIG.

When the calibration information acquisition process is started, the processes of S32 to S38 are sequentially executed.
In S32, it is determined whether or not the ODO meter reset button 20 is pressed when the driver's face image is captured in S10. If it is determined that the ODO meter reset button 20 is pressed (S32: YES), the process of S320 is executed (1). If not pressed, the process of S34 is executed.

  In S320, the face orientation detector 4 acquires the image information of the image taken in S10 and the face detection information (driver's face image and face acquired separately) for detecting the driver's face orientation from the image information. And the driver's face orientation is detected from the face orientation direction planned when the ODO meter 21 is viewed. A process for determining whether the angle is within the range of ± 5 degrees is executed.

  This face orientation detection information associates an image of the driver's face acquired before executing the gaze direction monitoring process of FIG. 2 and photographed with the driver facing the designated direction and the designated direction. Information. This face direction detection information may be updated periodically.

  If the driver's face orientation is within ± 5 degrees (S320: YES), the image information of the image taken by the camera 17 in S10 is stored in the RAM 13 as an image whose line of sight is facing the ODO meter 21. After the storing process is executed (S322), the process proceeds to the calibration process S50 (5). If not stored (S320: NO), the process proceeds to S14, and the image acquired in S10 is determined to look away ( Used in S16).

  In S34, when the driver's face image is photographed in S10, it is determined whether the HUD lighting display button 30 or the brightness adjustment button 31 is operated. If it is determined that the driver is operated (S34: YES), the process of S340 is performed. Is executed (2), and if not pressed (S34: NO), the process of S36 is executed.

  In S340, when the face direction detector 15 detects the driver's face direction based on the face direction detection information from the image information of the image taken in S10, and the driver's face direction looks at the HUD 32 A process for determining whether the angle is within ± 5 degrees from the face orientation is executed.

  When the driver's face orientation is within ± 5 degrees (S340: YES), in S10, the image is directed in the direction in which the image projected by the HUD projector 32 is displayed. After the process of storing the image information of the image photographed by the camera 7 in the RAM 13 is executed (S342), the process proceeds to the calibration process S50 (5), and if it does not fit (S340: NO), the process of S14 is performed. Transition.

  In S36, when the driver's face image is photographed in S10, it is determined whether an alarm is issued from the gasoline remaining amount detection unit. If it is determined that the alarm is issued (S36: YES), the process of S360 is executed. (3) If not issued, the process of S38 is executed.

  In S360, the face direction detector 15 detects the driver's face direction from the image information of the image taken in S10 based on the face direction detection information, and the driver's face direction is determined based on the remaining gasoline indicator. A process of determining whether the face is within ± 5 degrees from the face orientation when viewing 41 is executed.

  If the driver's face orientation is within ± 5 degrees (S360: YES), the image information of the image taken by the camera 7 in S10 as an image whose line of sight is facing the gasoline remaining amount indicator 41 is displayed. Is stored in the RAM 13 (S362), the process proceeds to the calibration process (S50) (5), and if not (S340: NO), the process proceeds to S14.

  In S38, when the driver's face image is photographed in S10, it is determined whether the host vehicle travels and the speed exceeds 10 km / h. If it is determined that the speed exceeds (S38: YES), The process of S380 is executed (4), and if not exceeded, the process proceeds to S14.

  In S380, whether or not the distance from the other vehicle traveling in front of the host vehicle is a safe driving state that is more than a distance that can prevent a collision of the host vehicle with the other vehicle if the brake operation is performed, Is executed to determine whether the host vehicle is 10 m or more away from other vehicles traveling ahead.

  If it is determined that the host vehicle is separated from the other vehicle by 10 m or more (S380: YES), the process of S382 is executed. If it is determined that the vehicle is not separated by 10 m or more (S380: NO), the process of S14 is performed. Migrate to

  In S382, a process of determining whether the brake that the driver has depressed in the safe driving state is the first one is executed. If it is determined that the brake is first depressed (S382: YES), the process of S384 is executed. If it is determined that the brake is not initially depressed (S382: NO), the process of S14 is performed. Migrate to

In S384, the image information of the image taken by the camera 7 in S10 is stored in the RAM 13 as an image having the driver's line of sight on the vehicle ahead (S384), and the process proceeds to the calibration process (S50). To do.
[4. Calibration process]
Next, the calibration process (S50) executed by the central controller 10 will be described with reference to FIG.

  The calibration process is an algorithm that detects the gaze direction based on the captured image, and corrects the gaze direction based on the actual gaze direction and the captured image in that state. This is a process for enabling detection with high accuracy.

  When the calibration process (S50) is started, whether or not the information acquired in the calibration information acquisition process (S30) is non-congestion information, that is, the driver's line-of-sight direction is directed to another vehicle. A process for determining whether or not the information is information is executed (S52). If it is determined that the information at the time of non-congestion is acquired (S52: YES), the process at S54 is executed and it is determined that the information at the time of congestion is acquired. (S52: NO), the process of S56 is executed.

  In S54, based on the image stored in the RAM 13, that is, the driver's face image captured by the camera 7 when the driver's line of sight is facing another vehicle in S10, and the face direction detection information, The process for calculating the direction is executed, and the process of S56 is executed.

  In S56, when S54 is executed, the image information stored in the non-congestion image storage unit 32a is updated with the image information stored in the RAM 13, and when S320, S340, and S360 are executed, the image during congestion is displayed. The image information corresponding to each display stored in the storage unit 32 b is updated with the image information stored in the RAM 13.

  In S58, based on the updated calibration information and the face orientation calculated in S54, S320, S340, and S360, the display 21, 32, 41 and the line-of-sight direction to other vehicles traveling in front are detected. A line-of-sight detection model that is filter information for detecting the line-of-sight direction from an image of the driver's face based on the relationship between the detected line-of-sight direction and the positions of the displays 21, 32, 41 and other vehicles traveling ahead. Is generated (corrected), that is, calibration is executed.

Thereafter, the calibration process is terminated, and the process proceeds to S12 in FIG.
[5. effect]
As described above, in the present embodiment, calibration information is collected during non-congestion (S384) and calibration information is collected during congestion (S322, S342, S362).
In the calibration process (S50), the filter information is calibrated based on the collected calibration information (S58).

  In this way, it is possible to collect information for calibration both when congestion occurs and when it does not occur, so that accurate calibration of filter information can be performed.

  In the present embodiment, calibration information can be collected without imposing any burden on the driver even if the range for collecting calibration information is expanded to a range where congestion does not occur.

Next, in the present embodiment, when the host vehicle and the other vehicle are traveling with a separation distance (10 m) or more (S380: YES), the brake is first depressed (S382: YES). Because the driver's line of sight is located in another vehicle that is traveling ahead with high probability, it is determined that congestion has not occurred and the driver's line of sight is facing the other vehicle. ing.
Therefore, in the present embodiment, when the brake is stepped on for the first time, it is estimated that the driver's line of sight is in the direction (first direction) in which the driver's line of sight faces the other vehicle (first event). In this case, the image (first image) taken by the camera 7 at this time is acquired as calibration information.
Therefore, if configured as in the present embodiment, calibration information when no congestion has occurred can be acquired, so that accurate calibration can be performed.
Next, in the present embodiment, when an operation tool such as the ODO meter reset button 20 is operated (S32, S34: YES), the driver's line of sight faces the corresponding display (first corresponding display) (second display). An image (second image) photographed by the camera 7 is acquired as calibration information on the assumption that an event (second event) in which it is estimated that the driver's line of sight is present in (direction) (S322, S342).
Therefore, if configured as in the present embodiment, calibration information when congestion occurs can be acquired, so that accurate calibration can be performed.
Further, in the present embodiment, when the remaining amount of gasoline is notified (S36: YES), driving in a direction (second direction) in which the driver's line of sight faces a display corresponding to the notification (second correspondence display). The second image captured by the camera 7 at this time is acquired as calibration information on the assumption that an event (second event) that is estimated to have a person's line of sight has occurred (S362).
Therefore, if configured as in the present embodiment, calibration information when congestion occurs can be acquired, so that accurate calibration can be performed.
In the present embodiment, when the driver's face is facing in a direction in which the driver's line of sight is not assumed to be facing in the second direction (S320, S340, S360: NO), the driver's face is Since the captured image is prevented from being acquired as a calibration image, accurate calibration can be performed.
(Other embodiments)
The ODO meter 21 and the like have been exemplified as a display for the driver to turn his eyes when congestion occurs. In addition, the remaining amount of the washer liquid and the navigation device when the navigation device is operated are shown. A display screen, a display screen of an audio device when audio is operated, and the like can be considered.

Moreover, as a case where congestion has occurred, a case where the door mirror is viewed when the angle adjustment button of the door mirror is operated may be considered.
In S320, S340, and S360 of the above-described embodiment, the allowable range is ± 5 degrees. However, for example, this may be ± 1, and the range may be enlarged or reduced according to accuracy.

Moreover, although the reference | standard that the own vehicle is drive | working at 10 km / h was used as a reference | standard as a reference | standard which judges that the own vehicle is drive | working (S38), it is not restricted to this.
In addition, although the distance that can prevent a collision of the host vehicle with another vehicle is 10 m, the distance is not limited to this.
(Correspondence)
The imaging means recited in the claims corresponds to the camera 7 of the embodiment, the line-of-sight direction detection means corresponds to the processes of S10 and S14, and the non-congestion acquisition means corresponds to the processes of S38, S380 to S384. The congestion acquisition process corresponds to the processes of S32, S34, S36, S320, S322, S340, S342, S360, and S362.

The equipment described in the claims corresponds to the ODO meter reset button 20, the lighting display button 30 for HUD, and the brightness adjustment button 31 for HUD.
The brake detection means described in the claims corresponds to the brake depression amount detector 51 of the embodiment, the running state determination means corresponds to the processing of S380, and the first correspondence display means is the projection of the ODO meter 21, HUD. The second correspondence display means corresponds to the gasoline remaining amount indicator 41.

  The face direction calculation means described in the claims corresponds to the process of calculating the face direction in S320, S340, and S360 of the present embodiment, and the face direction determination means corresponds to S322, S342, and S362, and the acquisition prohibition means. Corresponds to the processing of S320, S340, and S360.

  Further, the present invention is not limited to the above-described embodiment as long as it matches the gist of the invention described in the claims.

DESCRIPTION OF SYMBOLS 1 ... Gaze direction detection apparatus, 3 ... Central control apparatus, 4 ... Face direction detector, 5 ... Gaze direction detector, 6 ... Interface part, 7 ... Camera, 8 ... Speaker, 10 ... Central control apparatus, 17 ... Camera, 20 ... ODO meter reset button, 21 ... ODO meter, 30 ... lighting display button, 31 ... brightness adjustment button, 32 ... projector, 32a ... non-congestion image storage unit, 32b ... congestion image storage unit, 32c ... filter information Storage part 40 ... gasoline remaining amount indicator 41 ... gasoline remaining amount indicator 50 ... speed detector 51 ... brake depression amount detector 52 ... forward monitoring device.

Claims (5)

Photographing means for photographing the driver's face;
Gaze direction detection that detects the driver's gaze direction from the driver's face image captured by the imaging means, using filter information that associates the driver's face image with the driver's gaze direction Means,
In a gaze direction detection device comprising calibration means for performing calibration of the filter information,
The calibration means includes
When a first event occurs in which the driver's line of sight faces the first direction when no congestion occurs in the driver's eyes, the first image captured by the imaging unit is acquired as calibration information. Acquisition means;
Acquisition at the time of congestion when the second event occurs when the driver's eyes are congested and the driver's line of sight is in the second direction, and the second image captured by the imaging unit is acquired as calibration information. Means,
With
A line-of-sight direction detection apparatus, wherein the filter information is calibrated based on the calibration information obtained by the non-congestion acquisition unit and the congestion acquisition unit. In the gaze direction detecting device according to claim 1,
Brake detecting means for detecting depression of the brake of the own vehicle;
Drive in a safe driving state where the distance from the other vehicle traveling in front of the host vehicle provided with the line-of-sight direction detection device is more than a distance that can prevent a collision of the host vehicle with the other vehicle if a brake operation is performed. Traveling state determining means for determining whether or not
With
The non-congestion acquisition means includes:
As the first event, when the host vehicle is traveling in the safe driving state and the first brake is depressed, the driver's line of sight is directed to the other vehicle in the first direction and congestion occurs. A line-of-sight direction detection apparatus, wherein the first image captured by the imaging unit is acquired as calibration information when it has been stepped on. In the gaze direction detecting device according to any one of claims 1 and 2,
When the equipment of the host vehicle is operated as the second event, the first correspondence display means for performing a first correspondence display,
The acquisition means at the time of congestion is
When the equipment is operated, it is assumed that the driver's line of sight is directed to the first correspondence display in the second direction and congestion occurs. The gaze direction detecting device, wherein the second image captured by the step (2) is acquired as calibration information. In the gaze direction detecting device according to any one of claims 1 to 3,
A second correspondence display means for performing a second correspondence display corresponding to the notification when the notification is made to the driver as the second event;
The acquisition means at the time of congestion is
When the notification is given, it is assumed that the driver's line of sight is facing the second correspondence display in the second direction and congestion is occurring. An eye-gaze direction detecting apparatus that acquires the second image as calibration information. In the gaze direction detecting device according to any one of claims 1 to 4,
A face orientation calculating means for calculating a face orientation from an image of the driver photographed by the photographing means;
A face for determining whether the face orientation calculated by the face orientation calculation means when the second event occurs is within the range of the face orientation planned when the driver's line of sight is facing the second direction Direction determination means;
With
The acquisition means at the time of congestion is
An acquisition prohibiting unit that prohibits acquisition of the second image as calibration information when it is determined that the face orientation calculated by the face orientation calculating unit is not within the range of the planned face orientation; A gaze direction detecting device characterized by the above.

Images