JP2019046240A - Sight line direction detecting apparatus and calibration method - Google Patents

Abstract

To provide a technique which reduces restriction in calibration required for increasing accuracy in detection of a sight line direction and speed of processing of the detection.SOLUTION: A sight line direction detecting apparatus has a determining unit, a generating unit, and a resuming unit. The determining unit determines whether or not calibration regarding detection of a sight line is interrupted. If the determining unit determines that the calibration is interrupted, the generating unit generates data obtained in the middle of calibration. The resuming unit resumes the calibration based on the data obtained in the middle of calibration.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for detecting a line-of-sight direction.

  In recent years, development of a driver monitor system for monitoring the state of a driver has been progressing for the purpose of supporting safe driving of a vehicle. One of the detection targets of the driver monitor system is the driver's line-of-sight direction.

  As a representative method for detecting the gaze direction, (1) a method of detecting the gaze direction based on the position of the moving point with respect to the reference point with the reference point as the eye and the moving point as the center of the iris (also referred to as “eye-iris method”). (2) A method of detecting the direction of the line of sight based on the position of the moving point relative to the reference point with the reference point as the corneal reflection point (Purkinje image) and the moving point as the center of the pupil (“pupil-corneal reflection method”) And so on).

  The "eye-iris method" has the merit of being inexpensive because it can be realized by using an image taken with a visible light camera, but it detects the direction of the eye because the position of the eye changes due to changes in facial expressions. There is a demerit that accuracy is low.

  On the other hand, the “pupil-corneal reflection method” has the advantage of high accuracy in detecting the corneal reflection point because it is easy to detect the corneal reflection point accurately, but an infrared light source and an infrared camera are required for photographing the corneal reflection point. Therefore, there is a demerit that it is expensive.

  In addition, since the object watched by the human is imaged in the fovea deviating from the center of the retina, the human gaze direction is slightly deviated from the direction of the center of the iris or pupil. There are individual differences in the slight deviation. Therefore, no matter what gaze direction detection method is used, calibration is required to increase the gaze direction detection accuracy and the detection processing speed.

JP 2010-30361 A

  In the calibration of a general gaze direction detecting device, the user needs to gaze at a plurality of fixed gaze points in order or the user needs to gaze at the moving gaze points with his eyes. For this reason, a certain amount of time is required from the start to the completion of the calibration of the gaze direction detecting device.

  When the gaze direction detection device detects the driver's gaze, it is desirable to calibrate the gaze direction detection device before starting the vehicle, but if the gaze direction detection device is calibrated before starting the vehicle, the gaze direction There is a problem that the start of the vehicle is delayed by the time required from the start to the completion of the calibration of the detection device.

  Note that the gaze direction detection calibration proposed in Patent Document 1 is a calibration that completes the process instantaneously, limited to a gaze state in one direction, such as gaze of an in-vehicle device provided in a certain place. is there. For this reason, the gaze direction detection calibration proposed in Patent Document 1 does not have a problem related to the time required from the start to completion of calibration, but the user gazes at a plurality of fixed gaze points in order. In addition, there is a problem in that it cannot be applied to calibration that requires a certain amount of time from start to completion, such as when the user gazes at a moving gaze point.

  In view of the above-described problems, an object of the present invention is to provide a technique for reducing restrictions relating to calibration necessary for increasing the detection accuracy and the detection processing speed in the line-of-sight direction.

  A gaze direction detection device according to the present invention includes a determination unit that determines whether or not calibration is interrupted with respect to gaze direction detection, and a generation unit that generates intermediate data of the calibration when the determination unit determines that there is an interruption And a resuming unit that resumes the calibration with the content based on the midway data of the calibration (first configuration).

  Further, in the gaze direction detecting device having the first configuration, the calibration intermediate data may include a configuration (second configuration) including personal identification information.

  Further, the gaze direction detecting device having the first or second configuration may have a configuration (third configuration) including a storage unit that stores data during the calibration.

  The gaze direction detecting device having the first or second configuration may further include: a transmission unit that transmits the calibration intermediate data to a server; and a reception unit that receives the calibration intermediate data from the server. The restarting unit may be configured to restart the calibration with a content based on the halfway data of the calibration received by the receiving unit (fourth configuration).

  Further, in the gaze direction detecting device having any one of the first to fourth configurations, a normality determination unit that determines which section is normal in the midway data of the calibration is provided, and the resuming unit includes: A configuration (fifth configuration) may be employed in which the calibration is restarted with the content based on the section determined to be normal by the normality determination unit.

  Further, in the gaze direction detection device having any one of the first to fifth configurations, the determination unit determines whether or not the calibration is interrupted based on a gaze direction detection result during the calibration. (6th structure) may be sufficient.

  Further, in the gaze direction detection device having any one of the first to sixth configurations, the determination unit determines whether or not the calibration is interrupted based on a situation of a vehicle on which the gaze direction detection device is mounted. It may be a configuration (seventh configuration).

  Further, the gaze direction detecting device having any one of the first to seventh configurations includes a proposing unit that proposes resumption of the calibration based on a situation of a vehicle on which the gaze direction detecting device is mounted (first) 8 configuration).

  The calibration method according to the present invention is a calibration method for a gaze direction detection device, wherein a determination step for determining whether or not calibration has been interrupted for gaze direction detection is performed, and when the determination unit determines that there is an interruption. It is a structure (9th structure) provided with the production | generation process which produces | generates the said data in the middle of the said calibration, and the restart process which restarts the said calibration by the content based on the data in the middle of the said calibration.

  According to the present invention, it is possible to suspend and resume calibration necessary for increasing the detection accuracy and detection processing speed of the line of sight, thereby reducing the restrictions on calibration necessary to increase the detection accuracy of the line of sight. can do.

The figure which shows the structure of the gaze direction detection apparatus which concerns on 1st Embodiment. Diagram showing fixed gaze points The flowchart which shows the calibration operation | movement of the gaze direction detection apparatus which concerns on 1st Embodiment. The flowchart which shows the operation example regarding the 2nd determination example of the presence or absence of interruption A flowchart showing an operation example of the normality determination unit Diagram showing fixed gaze points Diagram showing fixed gaze points The figure which shows the structure of the gaze direction detection apparatus which concerns on 2nd Embodiment. The flowchart which shows the calibration operation | movement of the gaze direction detection apparatus which concerns on 2nd Embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

<1. Configuration of Eye Gaze Direction Detection Device According to First Embodiment>
FIG. 1 is a diagram illustrating a configuration of a gaze direction detection device according to the first embodiment. A line-of-sight detection device 101 according to the first embodiment (hereinafter abbreviated as a line-of-sight detection device 101) is mounted on a vehicle.

  In the interior of the vehicle in which the line-of-sight direction detection device 101 is mounted, there are an infrared camera 1, an infrared light emitting device 2, a display 3, a meter 4, a speaker 5, a microphone 6, a start switch 7, an interruption switch 8, and a restart switch 9. Provided and connected to the line-of-sight detection device 101. Part or all of the infrared camera 1, the infrared light emitting device 2, the display 3, the meter 4, the speaker 5, the microphone 6, the start switch 7, the interruption switch 8, and the restart switch 9 are incorporated in the line-of-sight direction detection device 101. May be.

  The infrared camera 1 captures the driver's face. The infrared light emitting device 2 irradiates the driver's face with infrared light. The infrared camera 1 and the infrared light emitting device 2 are installed, for example, in a vehicle steering.

  The display 3 displays an image. As the display 3, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, a head-up display that projects an image on a windshield of a vehicle, or the like can be used. The meter 4 displays the speed of the vehicle, the engine speed of the vehicle, the fuel remaining amount of the vehicle, and the like.

  The speaker 5 converts the audio signal from the line-of-sight direction detection device 101 into sound. The microphone 6 converts the sound into a sound signal and sends it to the line-of-sight direction detection device 101. The microphone 6 mainly collects sound emitted by the driver.

  The start switch 7 is a mechanical switch for the driver to instruct the visual line direction detection device 101 to start calibration. The interruption switch 8 is a mechanical switch for the driver to instruct the visual line direction detection device 101 to interrupt the calibration. The restart switch 9 is a mechanical switch for the driver to instruct the line-of-sight direction detection device 101 to restart calibration. Instead of these mechanical switches, for example, a touch panel arranged on the screen of a liquid crystal display or an organic EL display may be used.

  The line-of-sight direction detection apparatus 101 includes a microcomputer as a control unit that controls the entire apparatus. Specifically, the line-of-sight detection device 101 includes a CPU 11 that realizes various control functions by performing arithmetic processing, a nonvolatile memory 12 that stores various data, and a RAM 13 that is a work area for arithmetic processing. Prepare. The nonvolatile memory 12 is composed of, for example, a hard disk or a flash memory. The function of controlling each part of the line-of-sight direction detection device 101 is realized by the CPU 11 executing arithmetic processing according to a program 12 a stored in advance in the nonvolatile memory 12.

  The line-of-sight direction detection apparatus 101 includes an image processing unit 14 that processes image data captured by the infrared camera 1. In the present embodiment, the image processing unit 14 is a hardware circuit. The image processing unit 14 performs predetermined image processing on a signal of image data input from the infrared camera 1 to generate digital image data in a predetermined format. The predetermined image processing includes, for example, A / D conversion, luminance correction, contrast correction, and the like. The image data processed by the image processing unit 14 is stored in the RAM 13 in the MPEG-4 format, for example, as a captured video obtained by the infrared camera 1.

  In addition, vehicle status information provided from various ECUs (Electronic Control Units), various sensors, and various in-vehicle devices is input to the CPU 11 via a controller area network (CAN) bus (not shown). The vehicle status information may include, for example, steering information, brake information, accelerator information, position information, door opening / closing information, and the like. Furthermore, map information provided from a navigation device or the like is input to the CPU 11 via a CAN (not shown).

  The CPU 11 includes a gaze direction calculation unit 11a, a start unit 11b, a determination unit 11c, a generation unit 11d, a resumption unit 11e, an HMI (Human Machine Interface) control unit 11f, a personal identification unit 11g, and a normal determination unit. 11h and a suggestion unit 11i.

  The line-of-sight direction calculation unit 11a obtains the moving point with respect to the corneal reflection point (Purkinje image) from the image data stored in the RAM 13 to determine the position of the center of the pupil, and complete calibration data 12c stored in the nonvolatile memory 12 or The driver's line-of-sight direction is detected using the standard calibration data 12d. The driver's line-of-sight direction can be represented by, for example, a virtual plane that faces the driver in front of the driver and the driver's line-of-sight direction vector passes through the virtual plane (two-dimensional coordinates). The method of using the driver's line-of-sight direction detected by the line-of-sight direction calculation unit 11a is not particularly limited. For example, it can be used as a material for determining whether or not the automatic operation can be switched to the manual operation.

  The start unit 11 b starts calibration in response to an operation of the start switch 7 by the driver or a voice input of a start instruction using the microphone 6. In the present embodiment, the calibration is completed when the driver gazes at the nine fixed gaze points indicated by the circled numbers in FIG. 2 in the numerical order. Note that the number of times of data sampling at each fixed gaze point may be one, but it is desirable to obtain an average value by a predetermined number of times. The position of the fixed gaze point may be notified to the driver using, for example, the display 3 or the meter 4. The order in which the fixed gaze points are watched may be notified to the driver by sequentially displaying the fixed gaze points to be watched or by voice guidance using the speaker 5.

  The determination unit 11c determines whether or not the calibration is interrupted. A specific determination method example will be described later.

  When the determination unit 11c determines that there is an interruption, the generation unit 11d generates “intermediate calibration data 12b” that is data related to the calibration between the start and the end of calibration, and the “intermediate calibration data” 12b "is stored in the nonvolatile memory 12. For example, when the calibration is interrupted at the stage where the driver's gaze ends for four fixed gaze points, “halfway calibration data 12b” is generated with sampling data relating to the four fixed gaze points.

  When the calibration is completed, the generation unit 11d generates “complete calibration data 12c” and stores the “complete calibration data 12c” in the nonvolatile memory 12. In the present embodiment, “completed calibration data 12c” is generated with sampling data relating to nine fixed gaze points.

  The resuming unit 11e resumes the calibration with the content based on the “halfway calibration data 12b” in response to the operation of the resuming switch 9 by the driver or the voice input of the resuming instruction using the microphone 6. For example, when “halfway calibration data 12b” is generated with sampling data relating to four fixed gaze points, calibration is resumed from the fifth fixed gaze point. After calibration is restarted from the fifth fixed gaze point, for example, when calibration is interrupted again at the stage where the driver's gaze is finished for two more fixed gaze points, the “halfway calibration data 12b” is updated. Then, “halfway calibration data 12b” is generated with the sampling data relating to the six fixed gaze points.

  The HMI control unit 11 f outputs to the display 3, to the meter 4, to the speaker 5, to input from the microphone 6, to input from the start switch 7, and from the interruption switch 8 and from the restart switch 9. Process the input.

  The personal identification unit 11g performs personal identification of the driver. As a method for personal identification, for example, face authentication or fingerprint authentication can be used. In this case, the facial features and fingerprint features become the personal identification information. In addition, for example, a driver-specific operation unit (for example, four mechanical switches and selection keys on four touch panels when four persons can be identified) is provided, and based on an operation on the driver-specific operation unit, The personal identification unit 11g may perform personal identification of the driver.

  The personal identification information is included in each of “halfway calibration data 12b” and “completed calibration data 12c”. As a result, the calibration can be interrupted, resumed, and completed for each driver. Then, once calibration for a specific driver is completed, re-calibration is not necessary for the specific driver.

  Unlike the present embodiment, the CPU 11 may be configured not to include the personal identification unit 11g. In this case, calibration for a single driver is assumed.

  The normality determination unit 11h determines which section (which sampling data) is normal in the “halfway calibration data 12b”. For example, the interval determined to be not normal by the normality determination unit 11h may be sampled again when the calibration data is resumed. Thereby, the “completed calibration data 12c” becomes more appropriate data, and the detection accuracy of the line-of-sight direction is improved. A specific determination method example will be described later.

  The proposing unit 11i proposes resumption of calibration based on the situation of the vehicle. For example, what is necessary is just to suggest by the display using the display 3 and the meter 4, and the sound using the speaker 5. FIG. For example, the proposing unit 11i proposes to resume calibration when it is determined that the risk is low even if calibration is resumed based on vehicle speed, vehicle shift, vehicle position, map information, and the like. do it. By the proposal of the proposing unit 11i, the possibility that the driver misses the opportunity of resuming calibration is reduced, and as a result, the time required for completing calibration can be shortened.

<2. Operation of Gaze Direction Detection Device According to First Embodiment>
FIG. 3A is a flowchart showing the calibration operation of the eye gaze direction detecting device 101.

  When the calibration is started, the determination unit 11c determines whether or not the calibration is interrupted (step S10). When it is determined that the calibration is interrupted except when the interrupt switch 8 is pressed, it is desirable to notify the user that the calibration is interrupted using the display 3, the meter 4, the speaker 5, and the like.

(First example of whether to interrupt)
For example, when the interruption switch 8 is pressed, the determination unit 11c determines that the calibration is interrupted.

(Second example of whether to interrupt)
In addition, for example, in the gaze of the fixed gaze point, the number of sampling data whose distance between the virtual plane coordinate of the fixed gaze point and the coordinate on the virtual plane indicating the gaze direction (gaze direction coordinate) is greater than the threshold T1 is equal to or greater than the predetermined value V1. In this case, the determination unit 11c determines that the calibration is interrupted. FIG. 3B is a flowchart illustrating an operation example of this determination. The process of the flowchart shown in FIG. 3B is executed for each fixed gaze point. The determination unit 11c sets each parameter C, EC, and SC to 0 (step S101). Next, the determination unit 11c acquires the coordinates (Xn, Yn) on the virtual plane of the fixed gaze point (step S102). Next, the determination unit 11c reads “standard calibration data 12d” from the nonvolatile memory 12 (step S103). Next, using the “standard calibration data 12d”, the determination unit 11c calculates the line-of-sight direction coordinates (Xc, Yc) for the Cth sampling (step S104). Then, the determination unit 11c determines whether or not the distance between the coordinate (Xn, Yn) on the virtual plane of the fixed gaze point and the line-of-sight direction coordinate (Xc, Yc) of the number of sampling times C is greater than the threshold T1 ( Step S105). If the distance is greater than the threshold value T1, the determination unit 11c increments the parameter EC and the parameter C by one (step S106), and determines whether the parameter EC indicating the number of errors is equal to or greater than the predetermined value V1. (Step S107). If the parameter EC is not equal to or greater than the predetermined value V1, the process returns to step S104. If the parameter EC is equal to or greater than the predetermined value V1, the determination unit 11c determines that the calibration has been interrupted (step S108), and then ends the flow operation of FIG. 3B. On the other hand, if the distance is not greater than the threshold value T1, the determination unit 11c increments the parameter SC and the parameter C by one (step S109), and whether the parameter SC indicating the number of successes is equal to or greater than the specified value V2. Is determined (step S110). If the parameter SC is not greater than the specified value V2, the process returns to step S104. If the parameter SC is equal to or greater than the specified value V2, the flow operation of FIG. 3B is terminated. According to the flow operation of FIG. 3B, for example, as shown in FIG. 4, when the first fixed gaze point is being watched, the virtual plane in which the value of the radius R1 is the threshold value T1 with the first fixed gaze point as the center. When the number of sampling data in which coordinates on the virtual plane representing the line-of-sight direction appear outside the upper circle C1 is equal to or greater than the predetermined value V1, the determination unit 11c determines that the calibration is interrupted.

  In this determination, it is assumed that the driver performs an action other than calibration (for example, start of driving of the vehicle) during the calibration, and when the driver performs an action other than calibration, the determination unit 11c Therefore, it can be determined that the calibration is interrupted.

  Further, when this determination is performed, since “complete calibration data 12c” has not yet been generated, the line-of-sight calculation unit 11a calculates “standard calibration data 12d” in the calculation of coordinates on the virtual plane representing the line-of-sight direction. Is used. The “standard calibration data 12d” is human average calibration data. Instead of “standard calibration data 12d”, “halfway calibration data 12b” may be used. The “halfway calibration data 12b” can be used when it is constituted by sampling data relating to two or more fixed gaze points. Since the human cornea shape is substantially bilaterally symmetric and vertically symmetric, a pseudo “complete calibration data 12c” is generated from the “halfway calibration data 12b” using the symmetry, and the pseudo The “completed calibration data 12c” may be used.

(Third example of whether to interrupt)
For example, when it is estimated that the driver is driving the vehicle, the determination unit 11c determines that the calibration is interrupted. Thereby, it is possible to prevent the calibration from continuing in a state where safety cannot be ensured. For example, it may be estimated whether the driver is driving the vehicle based on the steering information, brake information, and accelerator information. However, even when driving, when driving on a gentle downhill or the like, steering, braking operation, or accelerator operation may not be performed, so the driver drives the vehicle depending on the position of the vehicle. It may be estimated that the driver is driving the vehicle when the position of the vehicle is changing.

(Fourth example of whether to interrupt)
For example, when it is recognized that the vehicle door has changed from a closed state to an open state based on the door opening / closing information, the determination unit 11c determines that the calibration is interrupted. Thereby, it is possible to prevent the calibration from being continued in a state where there is a high possibility that the driver is not facing forward.

  If it is determined in step S10 that the calibration is not interrupted, the process proceeds to step S40 described later. On the other hand, if it is determined in step S10 that the calibration is interrupted, the generation unit 11d generates “halfway calibration data 12b” (step S20), and the “halfway calibration data 12b” is stored in the nonvolatile memory 12. Is stored (saved) in a nonvolatile manner (step S21). Since the nonvolatile memory 12 stores the “halfway calibration data 12b”, a communication function with the server as in the second embodiment to be described later is unnecessary.

  Here, the normality determination unit 11h determines that the sampling data in which the distance between the coordinates on the virtual plane of the fixed gaze point and the coordinates on the virtual plane representing the line-of-sight direction is larger than the allowable value A1 in the gaze of the fixed gaze point is not normal. FIG. 3C is a flowchart illustrating an operation example of this determination. The process of the flowchart shown in FIG. 3C is executed for each sampling data. The normality determination unit 11h acquires the coordinates (Xn, Yn) on the virtual plane of the fixed gaze point (step S201). Next, the normality determination unit 11h reads “standard calibration data 12d” from the nonvolatile memory 12 (step S202). Next, the normality determination unit 11h calculates the line-of-sight direction coordinates (X, Y) using the “standard calibration data 12d” (step S203). Then, the normality determination unit 11h determines whether or not the distance between the coordinate (Xn, Yn) on the virtual plane of the fixed gaze point and the line-of-sight direction coordinate (X, Y) is greater than the allowable value A1 (step S204). . If the distance is greater than the allowable value A1, the normality determination unit 11h determines that the sampling data that is the current determination target is not normal (step S205), and then ends the flow operation of FIG. 3C. On the other hand, if the distance is not greater than the allowable value A1, the normality determination unit 11h determines that the sampling data that is the current determination target is normal (step S206), and then ends the flow operation of FIG. 3C. According to the flow operation of FIG. 3C, for example, as shown in FIG. 5, when gazing at the first fixed gaze point, the normality determination unit 11h has the radius R1 as the center with the first fixed gaze point as the center. It is determined that the sampling data in which the coordinates on the virtual plane representing the line-of-sight direction appear outside the circle C1 on the virtual plane that is the allowable value A1 is not normal. The allowable value A1 is smaller than the threshold value T1.

  When the process of step S21 ends, the CPU 11 determines whether the calibration is resumed by the resuming unit 11e (step S30).

  If the calibration is not resumed by the resume unit 11e, the CPU 11 continues the determination in step S30. On the other hand, if calibration is resumed by the resuming unit 11e, the process proceeds to step S40.

  In step S40, the CPU 11 determines whether calibration is completed.

  If calibration has not been completed, the process returns to step S10. On the other hand, when the calibration is completed, the generation unit 11d generates “complete calibration data 12c” (step S50), and stores (saves) the “complete calibration data 12c” in the nonvolatile memory 12 in a nonvolatile manner. (Step S51). The flow operation of FIG. 3A ends when the process of step S51 ends.

  The above-described flow operation in FIG. 3A enables the calibration to be interrupted and resumed. Thereby, the problem that the start of the vehicle is delayed by the time required from the start to the completion of the calibration of the gaze direction detecting device can be solved. For example, the calibration operation can be performed by using a short stoppage time of the vehicle due to a red light. Also, not only can the calibration be interrupted and resumed in the period from one engine start to engine stop (one trip), but also the calibration work can be continued by repeatedly suspending and restarting the calibration through multiple trips. You can also. Therefore, it is possible to reduce restrictions on calibration necessary for improving the detection accuracy of the line-of-sight direction.

<3. Second Embodiment>
FIG. 6 is a diagram illustrating a configuration of a gaze direction detection device according to the second embodiment. The line-of-sight detection device 102 according to the second embodiment (hereinafter abbreviated as the line-of-sight detection device 102) has a configuration in which a communication unit 15 is added to the line-of-sight direction detection device 101. The communication unit 15 communicates with the server 200 provided in the server center.

  The “halfway calibration data”, “complete calibration data”, and “standard calibration data” stored in the nonvolatile memory 12 of the visual line direction detection device 101 in the first embodiment are stored in the server 200 in this embodiment. Saved. The “standard calibration data” may be stored not in the server 200 but in the nonvolatile memory 12 of the line-of-sight direction detection device 102.

  FIG. 7 is a flowchart showing the calibration operation of the gaze direction detecting device 102. The flowchart shown in FIG. 7 is obtained by replacing steps S21 and S51 in the flowchart shown in FIG. 3A with steps S22 and S52, respectively, and adding step S31 between steps S30 and S40.

  In the present embodiment, when the process of step S20 ends, the communication unit 15 transmits “halfway calibration data” to the server 200 (step S22). When the calibration is resumed, the communication unit 15 transmits the driver's personal identification information to the server 200, thereby obtaining “intermediate calibration data” corresponding to the driver's personal identification information transmitted to the server 200. The requested “intermediate calibration data” is received from the server 200 (step S31).

  Since the “halfway calibration data” is stored in the server 200, the vehicle on which the gaze direction detection device 102 that generated the “halfway calibration data” is mounted and the vehicle on which the gaze direction detection device 102 that resumes calibration is mounted. Even if the vehicle is different, the flow operation shown in FIG. 7 can be realized. Therefore, this embodiment is more convenient than the first embodiment.

  In the present embodiment, when the process of step S50 ends, the communication unit 15 transmits “completed calibration data” to the server 200 (step S52). When the line-of-sight direction calculation unit 11 a detects the line-of-sight direction of the driver, the communication unit 15 transmits the driver personal identification information to the server 200, thereby corresponding to the driver personal identification information transmitted to the server 200. “Complete calibration data” is requested, and the requested “complete calibration data” is received from the server 200. Thereby, once the calibration is completed, the “completed calibration data” can be shared by a plurality of vehicles on which the line-of-sight direction detection device 102 is mounted. Therefore, this embodiment is more convenient than the first embodiment.

<4. Modification>
The above-described embodiment is an example in all respects and should not be considered as limiting, and the technical scope of the present invention is indicated by the scope of the claims, not the description of the above-described embodiment. Therefore, it should be understood that all modifications that come within the meaning and range of equivalency of the claims are embraced.

  For example, in the above-described embodiment, the driver's gaze direction is detected using the “pupil-corneal reflection method”, but the driver's gaze direction is detected using another method such as the “eye-iris method”. May be.

  In the above-described embodiment, the calibration in which the driver gazes at the nine fixed gaze points in order is adopted, but the number of fixed gaze points may be a plurality other than nine. As the number of fixed gaze points increases, the detection accuracy in the gaze direction improves, but the time required for calibration increases. Therefore, the number of fixed gaze points may be determined in consideration of this trade-off relationship.

  Moreover, you may employ | adopt the calibration in which a driver gazes at a moving gaze point with the eyes instead of the calibration in which a driver gazes a plurality of fixed gaze points in order.

11c determination unit 11d generation unit 11e resumption unit 11h normality determination unit 11i proposal unit 12 nonvolatile memory 15 communication unit 101 gaze direction detection device 102 according to the first embodiment 102 gaze direction detection device according to the second embodiment 200 server

Claims (9)

A determination unit that determines whether or not calibration is interrupted with respect to the gaze direction detection;
A generation unit that generates data during the calibration when the determination unit determines that there is an interruption;
A resuming unit for resuming the calibration with the content based on the halfway data of the calibration;
A gaze direction detecting device.   The gaze direction detection device according to claim 1, wherein the calibration intermediate data includes personal identification information.   The line-of-sight direction detection apparatus according to claim 1, further comprising a storage unit that stores data during the calibration. A transmission unit for transmitting data during the calibration to the server;
A receiving unit that receives the midway data of the calibration from the server,
The gaze direction detection device according to claim 1, wherein the resuming unit resumes the calibration with a content based on halfway data of the calibration received by the receiving unit. A normality determination unit that determines which section is normal in the midway data of the calibration,
The line-of-sight direction detection apparatus according to claim 1, wherein the resuming unit resumes the calibration with a content based on a section determined to be normal by the normality determining unit.   The gaze direction detection device according to claim 1, wherein the determination unit determines whether or not the calibration is interrupted based on a gaze direction detection result during the calibration.   The gaze direction detection device according to any one of claims 1 to 6, wherein the determination unit determines whether or not the calibration is interrupted based on a situation of a vehicle on which the gaze direction detection device is mounted.   The gaze direction detection device according to any one of claims 1 to 7, further comprising a proposing unit that proposes resumption of the calibration based on a situation of a vehicle on which the gaze direction detection device is mounted. A method for calibrating a gaze direction detecting device,
A determination step for determining whether or not calibration is interrupted with respect to the gaze direction detection;
A generation step of generating halfway data of the calibration when the determination unit determines that there is an interruption;
A resuming step of resuming the calibration with the content based on mid-calibration data;
A calibration method comprising:

Images