JP2019152957A - Calibration device - Google Patents

Abstract

To calibrate detection result of a sight line.SOLUTION: A calibration device (100) includes sight line detection means (11, 12, 31) which performs a sight line detection process for detecting a sight line of a user, manual input receiving means (13, 32) capable of receiving manual input by a user, presentation means (22) which presents information indicating user's gaze position on the basis of the detection result of the sight line detection process, and correction means (15) which, when a correction instruction for the gaze position is input via the manual input receiving means after the information indicating user's gaze position is presented by the presentation means, reflects an error with respect to the gaze position corresponding to the correction instruction to the next sight line detection process. When the sight line detection process is performed for a plurality of times and the correction instruction is input for a plurality of times, the correction means reflects an average of errors of gaze positions respectively correspond to the plurality of correction instructions to the next sight line detection process.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technical field of a calibration apparatus that performs calibration related to an operation input by a line of sight.

  For example, when a cross key provided on the steering wheel accepts an operator's input, an in-vehicle device that exists on the operator's line of sight is selected as an operation target. An apparatus for controlling an in-vehicle device to be operated based on an operator input received by a cross key has been proposed (see Patent Document 1). Alternatively, the in-vehicle device group is identified based on the selected vertical direction when the in-vehicle device group is identified based on the gaze direction of the occupant and the vertical direction selection unit is operated in a state where the in-vehicle device group is identified. An apparatus for specifying any on-vehicle device in the group has been proposed (see Patent Document 2).

  As a technique for correcting the line-of-sight position, for example, a display on which a cursor on the display screen moves following the line-of-sight position and a return key is pressed at the line-of-sight movement position can perform input corresponding to the display screen. In the apparatus, the gaze position data is corrected with the correction data, the actual gaze position is calculated, and after the gaze position data is displayed, when the correction operation is performed, the gaze position data and the gaze position data calculated immediately before are displayed. A technique for calculating and adjusting the amount of misalignment has been proposed (see Patent Document 3). Alternatively, an apparatus that adjusts gaze detection for detecting the user's gaze position based on the position of the gaze target accompanying the user operation and the user's gaze position at the time of the user operation has been proposed (see Patent Document 4). . Alternatively, it is a method for correcting the detected value of the operator's line of sight while the operator is performing an operation using the surgical tool, and detecting the tip of the surgical tool as a reference point, A method for correcting the detected value of the line of sight corresponding to the error between the detected value and the reference point has been proposed (see Patent Document 5). Alternatively, a technique for acquiring a correction coefficient for correcting an error when the operator performs a line-of-sight input based on the movement of the line of sight detected during the operation of the operator on the object displayed on the display screen is proposed. (See Patent Document 6). Alternatively, a technique has been proposed in which a correction target is displayed on the display unit, and a deviation between the coordinate position where the correction target is displayed and the gaze position of the detected line of sight is calculated and the deviation is corrected ( (See Patent Document 7). Alternatively, a technique for detecting the operator's line-of-sight position in the display area of the display unit based on the line-of-sight direction and correcting the detected line-of-sight position based on the indicated position that is the position in the display area designated by the operator. It has been proposed (see Patent Document 8). Alternatively, every time the user selects an object, a temporary correction coefficient is obtained and based on the relationship between the selected object and other objects, the temporary correction coefficient calculated when the degree of gaze of the user is high is emphasized. A technique for calculating the correction coefficient is proposed (see Patent Document 9).

Japanese Patent Laying-Open No. 2015-151059 International Publication No. 2012/176535 Japanese Patent Laid-Open No. 08-030380 International Publication No. 2016/139850 JP 2017-176414 A JP2015-152939A Japanese Patent Application Laid-Open No. 11-076165 Japanese Patent Laying-Open No. 2015-046111 JP 2012-065781 A

  In the techniques described in Patent Documents 1 and 2, it is assumed that the detection accuracy of line-of-sight detection is relatively high. However, in order to satisfy the premise, the configuration relating to the line-of-sight detection becomes relatively complicated and the cost becomes relatively high. When the detection accuracy of the line-of-sight detection is relatively low, it is necessary to use a correction (calibration) technique as described in Patent Documents 3 to 9, for example. However, the techniques described in Patent Documents 3 to 9 have room for improvement.

  This invention is made | formed in view of the said situation, and makes it a subject to provide the calibration apparatus which can calibrate the detection result of a gaze.

  A calibration apparatus according to an aspect of the present invention is a calibration apparatus that performs calibration related to an operation input using a line of sight in a device operation apparatus that can accept an operation input using a line of sight, and performs a line-of-sight detection process that detects a user's line of sight. Line-of-sight detection means, manual input reception means capable of receiving input by the user's hand, presentation means for presenting information indicating the user's gaze position to the user based on a result of the line-of-sight detection process, After the information indicating the gaze position is presented by the presentation unit, when an instruction to correct the gaze position is input via the manual input reception unit, an error related to the gaze position corresponding to the correction instruction is And a correction unit that reflects the line-of-sight detection process, wherein the correction unit performs the line-of-sight detection process a plurality of times and the correction instruction is continuously input a plurality of times. , The average value of the error of the plurality of the gaze position corresponding respectively to the plurality of times of correction instructions, is that reflected in the next of the sight line detection processing.

It is a block diagram which shows the structure of the apparatus operating device which concerns on embodiment. It is a conceptual diagram which shows the operation concept by a driver. It is a flowchart which shows the calibration process which concerns on embodiment.

  An embodiment according to a calibration apparatus will be described with reference to FIGS. In the following embodiments, an apparatus operating device is taken as an example of a calibration device.

(Constitution)
The configuration of the device operating device according to the embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of a device operating device according to the embodiment.

  In FIG. 1, the device operating device 100 is mounted on a vehicle 1. The vehicle 1 includes a plurality of in-vehicle devices such as a navigation device 201, an audio device 202, an air conditioner 203, a power window 204, and a mirror 205, for example.

  The device operating device 100 is configured to be able to operate the plurality of in-vehicle devices. The device operating device 100 includes a calculation unit 10, a storage device 21, a display unit 22, a line-of-sight detection sensor 31, and a manual operation detection sensor 32. The calculation unit 10 includes a line-of-sight detection unit 11, a gaze point detection unit 12, a manual operation detection unit 13, and an in-vehicle device control unit 14 as processing blocks that are logically realized therein or as processing circuits that are physically realized. And a calibration unit 15.

  The line-of-sight detection sensor 31 includes, for example, an IR (InfraRed) camera for line-of-sight measurement and an LED (Light Emitting Diode) light source (for example, near infrared illumination). The camera is installed so as to be able to capture the face (in particular, eyes) of the user (here, the driver of the vehicle 1). The said LED light source is installed so that the light from this LED light source may be irradiated to a user's face.

  The line-of-sight detection unit 11 of the calculation unit 10 detects the user's line-of-sight direction from the image captured by the camera. Specifically, the line-of-sight detection unit 11 detects the user's line-of-sight direction by the corneal reflection method. In addition, about the cornea reflection method, since the existing various aspects are applicable, the detailed description is abbreviate | omitted.

  The gaze point detection unit 12 of the calculation unit 10 retains the line of sight when the line of sight remains for a predetermined time (for example, 0.2 seconds) based on the line of sight detected by the line of sight detection unit 11. Is detected as a point of interest.

  The manual operation detection sensor 32 includes a cross key (see FIG. 2) provided on the steering wheel of the vehicle 1. The cross key includes a button for instructing each of the upper, lower, left, and right directions, and a button for instructing “OK”.

  The manual operation detection unit 13 of the calculation unit 10 detects a manual operation by the user based on the output from the manual operation detection sensor 32.

  The in-vehicle device control unit 14 of the calculation unit 10 controls a plurality of in-vehicle devices such as the navigation device 201 based on the output from at least one of the gazing point detection unit 12 and the manual operation detection unit 13.

  The calibration unit 15 of the calculation unit 10 calibrates the detection result obtained by the line-of-sight detection unit 11. Details of the calibration unit 15 will be described later.

  The display unit 22 includes, for example, a HUD (Head-Up Display), a MID (Multi Information Display), or a liquid crystal display. The display unit 22 may be uniquely provided in the device operating device 100, or may be shared with, for example, the display unit of the navigation device 201.

(Device operation)
Next, the operation concept of the in-vehicle device using the device operation device 100 will be described with reference to FIG. FIG. 2 is a conceptual diagram showing the concept of operation by the driver.

  On the display unit 22, for example, operation candidates related to at least some of the in-vehicle devices among the plurality of in-vehicle devices such as the navigation device 201 are displayed. For example, in the case of the audio device 202, for example, icons and symbols indicating power ON / OFF, volume increase / decrease, channel selection, and the like are displayed as operation candidates. For example, in the case of the air conditioner 203, for example, icons and symbols indicating ON / OFF of the power supply, raising / lowering of the set temperature, and the like are displayed as operation candidates.

  The display unit 22 further displays information (for example, a symbol or the like) indicating the user's gaze point. For example, the cross key provided in the hand operation detection sensor 32 when the operation candidate desired by the user overlaps with the information indicating the gazing point (that is, when the desired operation candidate is selected by the user's line of sight). When the button indicating “OK” is pressed, an operation corresponding to the operation candidate is performed.

(Problems with eye-gaze input)
When operating a vehicle-mounted device using a user's line-of-sight detection result, if the line-of-sight detection accuracy is relatively low, for example, an operation different from the operation desired by the user may be performed. On the other hand, when trying to sufficiently increase the detection accuracy of the line of sight, the configuration of the line-of-sight detection device (for example, the line-of-sight detection sensor 31) becomes relatively complicated, or the cost of the device becomes relatively high.

  Therefore, in the device operating device 100, by introducing the calibration unit 15 for calibrating the detection result by the line-of-sight detection unit 11 or the like, the detection accuracy of the line-of-sight detection is suppressed and the detection accuracy of the line of sight is improved. I am trying.

(Calibration process)
Next, the calibration process performed by the device operating device 100 will be described with reference to the flowchart of FIG.

  In FIG. 3, the device operating apparatus 100 determines whether or not a calibration processing start trigger related to line-of-sight detection is ON (step S <b> 101). The device operating apparatus 100 determines that the start trigger is ON, for example, when a predetermined switch or button is pressed. If it is determined in step S101 that the start trigger is not ON (step S101: No), the processing illustrated in FIG. 3 is terminated. Thereafter, after a predetermined period (for example, several tens of milliseconds to several hundreds of milliseconds) elapses, the process of step S101 is performed.

  If it is determined in step S101 that the start trigger is ON (step S101: Yes), the line-of-sight detection unit 11 detects the user's line-of-sight direction. The gaze point detection unit 12 specifies a gaze point based on the gaze direction detected by the gaze detection unit 11 (step S102). Next, gaze point information is displayed on the display unit 22 based on the processing result of step S102. As a result, gaze point information is presented to the user (step S103).

  Next, the manual operation detection unit 13 detects a manual operation by the user based on the output from the manual operation detection sensor 32. In particular, the manual operation detection unit 13 specifies which button (that is, the manual operation target) of the cross key (see FIG. 2) is pressed (step S104).

  Next, the device operating apparatus 100 determines whether or not the information related to the gaze point presented in the process of step S103 matches the result of the process of step S104 (step S105). In the process of step S104, the device operating apparatus 100 determines “match” when the button indicating “OK” on the cross key is pressed. On the other hand, in the process of step S104, the device operating apparatus 100 determines “mismatch” when any of the buttons for instructing the up / down / left / right directions of the cross key is pressed.

  When it is determined that they match in the determination in step S105 (step S105: Yes), the device operating device 100 performs an operation input based on the user's line of sight from the next time (that is, after the time when it is determined to match in the determination in step S105). (Step S106). This is because if it is determined to be coincident in the determination in step S105, the gaze point information and the user's intention are coincident and there is no need for calibration.

  On the other hand, when it is determined that there is a mismatch in the determination in step S105 (step S105: No), the calibration unit 15 continues to determine “mismatch” in the determination in step S105 N times (N is a natural number of 2 or more). It is determined whether it has been done (step S107).

  In the determination of step S107, when it is determined that the determination of “mismatch” is not N consecutive times (that is, less than N times) (step S107: No), the calibration unit 15 determines an error (that is, a gaze point). Is stored in the storage device 21 (step S108).

  In the present embodiment, if any of the buttons for designating the up, down, left, and right directions of the cross key is pressed in the processing of step S104, it is assumed that an error occurs in the direction corresponding to the pressed button. . The error when the button for instructing the up / down / left / right directions of the cross key is pressed once is determined in advance. The calibration unit 15 obtains an error according to the number of times the button for instructing the up / down / left / right directions of the cross key is pressed in the process of step S104, and stores the error in the storage device 21.

  As can be seen from the flowchart of FIG. 3, even if it is continuously determined as “mismatch” in the determination of step S105, if the determination of “mismatch” is less than N times, the processing of steps S102 to S105 is performed again. Done. In the determination in step S105, when it is determined that “disagreement” is repeated n times (n is a natural number less than N) continuously, the calibration unit 15 obtains an average value of errors at each time and calculates the average value (ie, average error). ) Is stored in the storage device 21.

  After the process of step S108, the calibration unit 15 reflects the error obtained in the process of step S108 on detection by at least one of the line-of-sight detection unit 11 and the gaze point detection unit 12 (step S109). As a result, the detection by the line-of-sight detection unit 11 or the like is calibrated. Thereafter, the processing after step S102 is performed.

  In the determination of step S107, when it is determined that the determination of “mismatch” is N times consecutively (step S107: Yes), the calibration unit 15 resets the error stored in the storage device 21 (step S110). ). In this case, for example, the device operating apparatus 100 may notify the user of a message indicating that recalibration is recommended. Thereafter, the process of step S101 is performed after a predetermined period.

(Technical effect)
According to the device operating apparatus 100, the detection by the line-of-sight detection unit 11 and the like is calibrated by the calibration process described above. For this reason, in the said device operating device 100, it is not necessary to obtain | require comparatively high detection accuracy for the gaze detection sensor 31, the gaze detection part 11, and the gaze point detection part 12. FIG. In other words, in the device operating apparatus 100, the above-described calibration process is introduced on the assumption that the detection accuracy of the line-of-sight detection sensor 31, the line-of-sight detection unit 11, and the gaze point detection unit 12 is not so high. Therefore, according to the apparatus operating device 100, the line-of-sight detection accuracy can be improved by the calibration process described above. In addition, for example, the configuration of the device related to the line-of-sight detection such as the line-of-sight detection sensor 31 can be made relatively simple, and the cost of the device can be suppressed.

  In the device operating device 100, the gaze point based on the gaze direction detected by the gaze detection unit 11 based on the operation result of the cross key (see FIG. 2) as a part of the manual operation detection sensor 32 is the user's intention. It is determined whether or not they match. For this reason, according to the said device operating device 100, the shift | offset | difference of the detection result by the gaze detection part 11 grade | etc., Can be determined comparatively easily.

<Modification>
In the calibration process shown in FIG. 3, an average error is obtained in the process of step S108 (when it is determined as “mismatch” n times consecutively in the determination of step S105). However, each time it is determined as “mismatch” in the determination in step S105, the error of each time may be simply reflected in the detection by at least one of the line-of-sight detection unit 11 and the gaze point detection unit 12.

  Various aspects of the invention derived from the embodiments and modifications described above will be described below.

  A calibration apparatus according to an aspect of the invention is a calibration apparatus that performs calibration related to an operation input using the line of sight in a device operation apparatus that can accept an operation input using the line of sight, and performs a line of sight detection process that detects a user's line of sight. Detection means; manual input acceptance means capable of accepting input by the user's hand; presentation means for presenting information indicating the user's gaze position to the user based on a result of the line-of-sight detection process; and the presentation When an instruction to correct the gaze position is input via the manual input receiving unit after the information indicating the gaze position is presented by the means, an error relating to the gaze position corresponding to the correction instruction is Correction means for reflecting in the line-of-sight detection process, wherein the correction means is configured to perform the line-of-sight detection process a plurality of times, and the correction instruction is continuously input a plurality of times. The mean value of the error of the plurality of the gaze positions respectively corresponding to the plurality of times of correction instructions, is that reflected in the next of the sight line detection processing.

  In the above-described embodiment, the line-of-sight detection sensor 11, the line-of-sight detection unit 11, and the gazing point detection unit 12 correspond to an example of a line-of-sight detection unit, and the manual operation detection sensor 32 and the manual operation detection unit 13 are manual input reception units. The display unit 22 corresponds to an example of a presentation unit, and the calibration unit 15 corresponds to an example of a correction unit.

  In the calibration device, the line-of-sight detection process is calibrated in accordance with a gaze position correction instruction. According to the calibration apparatus, the detection result of the line of sight can be calibrated, and thus the calibration related to the operation input by the line of sight is performed. As a result, according to the calibration apparatus, it is possible to improve the detection accuracy related to the line-of-sight detection process.

  The present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit or concept of the invention that can be read from the claims and the entire specification. Moreover, it is included in the technical scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Calculation part, 11 ... Gaze detection part, 12 ... Gaze point detection part, 13 ... Manual operation detection part, 14 ... In-vehicle apparatus control part, 15 ... Calibration part, 21 ... Memory | storage device, 22 ... Display part , 31 ... gaze detection sensor, 32 ... manual operation detection sensor, 100 ... equipment operation device

Claims (1)

A calibration device for performing calibration related to the operation input by the line of sight in the device operation device capable of receiving the operation input by the line of sight,
Eye-gaze detection means for performing eye-gaze detection processing for detecting the user's eye gaze;
Manual input receiving means capable of receiving input by the user's hand;
Presenting means for presenting information indicating the gaze position of the user to the user based on the result of the line-of-sight detection process;
After the information indicating the gaze position is presented by the presenting means, when an instruction to correct the gaze position is input via the manual input receiving means, an error relating to the gaze position corresponding to the correction instruction, Correction means for reflecting to the next gaze detection process;
With
The correction means, when the line-of-sight detection process is performed a plurality of times and the correction instruction is continuously input a plurality of times, calculates an average value of errors of the plurality of gaze positions respectively corresponding to the plurality of correction instructions. This is reflected in the next line-of-sight detection process.

Images