JP6827132B2 - Operating device, control method, program and storage medium - Google Patents

Description

The present invention relates to a method of accurately accepting an operation intended by an operator.

Conventionally, there has been known a technique for supporting an operator's selection operation based on the operator's line of sight. For example, Patent Document 1 discloses a technique for changing a selection criterion so that an object image in the line of sight of a user can be easily selected in an information processing apparatus that selects an object image displayed on a display screen. There is.

Japanese Unexamined Patent Publication No. 2012-22632

In a system in which the display object and the operation target are separate bodies, the line of sight deviates from the display object when the operator operates the operation target. On the other hand, the technique described in Patent Document 1 is not applicable in such a case because it is based on the premise that the displayed object and the operation target are on the extension line of the operator's line of sight.

As an example of the problem to be solved by the present invention, the above is given as an example. An object of the present invention is to provide an operation device capable of suitably accepting an operation on an operation target intended by an operator.

The invention according to claim 1 corresponds to a display unit that is an operation device and outputs a plurality of operation targets , and a plurality of operation targets that are separate from the display unit and are output to the display unit. An operation unit including an operation area to be operated, a line-of-sight detection unit that detects the line of sight of an operator who operates the operation target, and a line of sight of the operator detected by the line-of-sight detection unit among the plurality of operation targets for a predetermined time. It is characterized by including a control unit that estimates the operation target directed as described above as an operation candidate and expands the area of the operation area of the operation unit corresponding to the estimated operation candidate from the initial setting.

The invention according to claim 12 includes an operation unit that is separate from a display unit that outputs a plurality of operation targets and includes an operation area corresponding to each of the plurality of operation targets output to the display unit. A control method executed by the operating device, wherein the line-of-sight detection step of detecting the line of sight of the operator who operates the operation target and the line of sight of the operator detected in the line-of-sight detection step among the plurality of operation targets are It is characterized by having a control step of estimating an operation target directed to a predetermined time or more as an operation candidate and expanding the area of the operation area of the operation unit corresponding to the estimated operation candidate from the initial setting.

The invention according to claim 13 is separate from the display unit that outputs a plurality of operation targets, and is electrically operated from the operation unit having an operation area corresponding to each of the plurality of operation targets output to the display unit. A program executed by a computer that receives a signal, the line-of-sight detection unit that detects the line of sight of the operator who operates the operation target, and the operator among the plurality of operation targets that the line-of-sight detection unit detects. An operation target whose line of sight is directed for a predetermined time or longer is estimated as an operation candidate, and the computer is made to function as a control unit that expands the area of the operation area of the operation unit corresponding to the estimated operation candidate from the initial setting. It is a feature.

It is a schematic diagram of an operation system. This is an example of a display screen displayed on the display of an in-vehicle device. The outline configuration of the in-vehicle device is shown. The functional blocks related to the operation detection processing of the operation system are shown. Each division setting of the operation area after the initial state and the operation prediction is shown. It is a flowchart of operation detection processing. It is a display screen example of the display which concerns on the modification.

In a preferred embodiment of the present invention, the operation device includes an operation unit having an operation area corresponding to each of a plurality of output operation objects, a line-of-sight detection unit that detects the line of sight of an operator who operates the operation object, and the like. Of the plurality of operation targets, the operation target detected by the line-of-sight detection unit and directed to the operator's line of sight for a predetermined time or longer is estimated as an operation candidate, and the operation of the operation unit corresponding to the estimated operation candidate is performed. It includes a control unit that changes the area of the area from the initial setting.

The operation device includes an operation unit, a line-of-sight detection unit, and a control unit. The operation unit includes an operation area corresponding to each of a plurality of output operation targets. The line-of-sight detection unit detects the line of sight of the operator who operates the operation target. Of the plurality of operation targets, the control unit estimates the operation target detected by the line-of-sight detection unit to which the operator's line of sight is directed for a predetermined time or longer as an operation candidate. Then, the control unit changes the area of the operation area of the operation unit corresponding to the estimated operation candidate from the initial setting. According to this aspect, the operating device can accurately estimate the operation candidates to be operated based on the line of sight of the operator, and appropriately determine the area of the operating area so that the operator can perform the intended operation.

In one aspect of the operation device, the control unit controls an operation target for which the line of sight is finally directed for a predetermined time or longer in a period from when the line of sight is directed to a display unit for displaying the operation target until the line of sight is separated. , Presumed to be the operation candidate. According to this aspect, the operation device can more accurately estimate the operation candidate to be operated based on the line of sight of the operator.

In another aspect of the operation device, when the operation candidate is estimated, the control unit emphasizes and outputs the operation candidate. According to this aspect, the operating device can preferably make the operator visually recognize the estimated operation candidate.

In another aspect of the operation device, the control unit makes another operation area different from the operation area of the operation unit corresponding to the operation candidate inoperable. According to this aspect, the operating device can suitably suppress the operator from erroneously detecting an unintended operation.

In another aspect of the operating device, the operating device determines whether the operator has pressed the operating area of the operating unit corresponding to the operating candidate estimated by the control unit within a predetermined time. It also has a part.

In another aspect of the operation device, when the operation determination unit does not press the operation area of the operation unit corresponding to the operation candidate estimated by the control unit within a predetermined time, the operation determination unit does not press. The area of the operation area of the operation unit corresponding to the operation candidate is returned to the initial setting. According to this aspect, the operating device can suitably restore the area of the operating area changed based on the line-of-sight detection unrelated to the operation.

In another aspect of the operation device, the operation device presses the operation area of the operation unit corresponding to the operation candidate estimated by the control unit and the display unit displaying the map and the operation target with a predetermined pressure or more. When it is detected that the operation has been performed, it is determined that the operator has selected the operation candidate, and when the contact at a pressure lower than the predetermined pressure is detected, it is determined that the operation for the map has been performed. , Are further prepared. In this aspect, the operating device can favorably identify the operation on the map and the operation on the operation target and accept each operation.

In another aspect of the operation device, the operation determination unit detects that the operator has pressed the operation area of the operation unit corresponding to the operation candidate estimated by the control unit with less than a predetermined pressure. At that time, the area of the operation area of the operation unit corresponding to the operation candidate is returned to the initial setting. According to this aspect, the area of the operation area can be suitably returned to the default state when the operation on the operation target is not intended.

In another aspect of the operating device, the operating device is mounted on a moving body. Preferably, the operating unit is provided on the steering of the moving body.

In another aspect of the operating device, the operating object is displayed at a position on the line of sight that does not overlap with the operating unit. Also in this aspect, the operating device can accurately estimate the operation candidates to be operated based on the line of sight of the operator, and can change the area of the operating area so that the operator can perform the intended operation.

In another preferred embodiment of the present invention, the control method is executed by an operation device including an operation unit having an operation area corresponding to each of a plurality of output operation objects, and is performed by an operator who operates the operation object. Of the line-of-sight detection step for detecting the line of sight and the plurality of operation targets, the operation target for which the line of sight of the operator detected in the line-of-sight detection step is directed for a predetermined time or longer is estimated as an operation candidate, and the estimated operation is performed. It has a control step of changing the area of the operation area of the operation unit corresponding to the candidate from the initial setting. By executing this control method, the operating device can accurately estimate the operation candidates to be operated based on the line of sight of the operator, and appropriately determine the area of the operating area so that the operator can perform the intended operation. it can.

In another preferred embodiment of the present invention, a program executed by a computer that receives an electric signal from an operation unit having an operation area corresponding to each of a plurality of output operation objects, and is an operation for operating the operation object. Among the plurality of operation targets, the line-of-sight detection unit that detects the line of sight of a person and the operation target that the line-of-sight of the operator detected by the line-of-sight detection unit is directed to for a predetermined time or longer are estimated and estimated as operation candidates. The computer is made to function as a control unit that changes the area of the operation area of the operation unit corresponding to the operation candidate from the initial setting. By executing this program, the computer can accurately estimate the operation candidates to be operated based on the line of sight of the operator, and appropriately determine the area of the operation area so that the operator can perform the intended operation. Preferably, the program is recorded on a recording medium.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[Overview of operating system]
FIG. 1 is a schematic view of an operation system 100 according to an embodiment. The operation system 100 includes an on-board unit 1 installed in the interior of the vehicle Ve, a line-of-sight detection camera 2, and a touch pad 3 provided on the steering wheel. The operation system 100 preferably accepts the operation of the touch pad 3 on the vehicle-mounted device 1.

The on-board unit 1 has a navigation function for guiding a route from a departure place to a destination, and displays a map or the like on a display 44. The vehicle-mounted device 1 is electrically connected to the touch pad 3 by wire or wirelessly, and receives a detection signal (also referred to as “detection signal Sd”) indicating a user's touch position. Further, the vehicle-mounted device 1 is electrically connected to the line-of-sight detection camera 2 by wire or wirelessly, and receives an image (also referred to as “camera image Im”) from the line-of-sight detection camera 2.

The vehicle-mounted device 1 displays a plurality of selection items (also referred to as “selection item Si”) that can be selected by operating the touch pad 3 on the display 44 at a predetermined timing, and the detection signal Sd received from the touch pad 3 Based on, it recognizes which of the displayed selection items Si is selected, and executes the operation corresponding to the selected selection item Si. At this time, the vehicle-mounted device 1 specifies the selection item Si in the driver's line-of-sight direction and improves the operability by increasing the operation area of the touch pad 3 corresponding to the specified selection item Si. The in-vehicle device 1 may be, for example, a stationary navigation device installed in the vehicle Ve, or a portable terminal such as a PND (Portable Navigation Device). The in-vehicle device 1 and the touch pad 3 described later are examples of the "operation device" in the present invention.

The line-of-sight detection camera 2 is installed at a position where the driver's face is within the shooting range, and transmits the generated camera image Im to the vehicle-mounted device 2. The line-of-sight detection camera 2 may be a general camera that captures visible light, or may be an infrared camera having an LED irradiation function. The camera image Im is used to detect the direction of the driver's line of sight.

The touch pad 3 (steering remote controller) is an operation interface provided on the steering wheel (steering wheel) of the vehicle Ve, and is electrically connected to the vehicle-mounted device 2. The touch pad 3 has a touch position detecting device 6 shown in FIG. 4, and detects a user's touch position. The touch pad 3 generates a detection signal Sd indicating the coordinates of the detected touch position (also referred to as “touch coordinates”) and supplies the detection signal Sd to the vehicle-mounted device 1. In this embodiment, the touch pad 3 accepts an operation of selecting the selection item Si displayed on the display 44. Hereinafter, the detection area of the touch pad 3 that accepts the operation of selecting the selection item Si is referred to as "operation area T", and each of the division areas when the operation area is divided for each associated selection item Si is "divided". Also called "operation area Td". As will be described later, the size of the division operation region Td corresponding to each selection item Si varies depending on the selection item Si to which the driver looks. The touch pad 3 is an example of the "operation unit" in the present invention, the operation area T is an example of the "operation area" in the present invention, and the selection item Si is an example of the "operation target".

FIG. 2 is an example of a display screen of the display 44 of the vehicle-mounted device 1. On the display screen of FIG. 2, the vehicle-mounted device 1 displays a map around the current position and pops up a window 8 showing four selection items Si (Si1 to Si4). In this example, the vehicle-mounted device 1 displays the selection items Si1 to Si4 corresponding to each of the four recommended songs automatically selected at a predetermined timing on the window 8. In the example of FIG. 2, the selection items Si1 to Si4 are displayed separated by a frame having the same shape. In this case, the vehicle-mounted device 1 recognizes the touch position based on the detection signal Sd supplied from the touch pad 3, and reproduces the music of the selection item Si corresponding to the division operation area Td to which the recognized touch position belongs. In this case, as will be described later, the vehicle-mounted device 1 determines the size of each divided operation region Td according to the line-of-sight direction of the driver recognized based on the camera image Im.

[Configuration of in-vehicle device]
FIG. 3 shows a schematic configuration of the vehicle-mounted device 1 when the vehicle-mounted device 1 is a general navigation device. As shown in FIG. 3, the in-vehicle device 1 includes an autonomous positioning device 10, a GPS receiver 18, a system controller 20, a disk drive 31, a data storage unit 36, a communication interface 37, a communication device 38, a display unit 40, and an audio output. It includes a unit 50 and an input device 60.

The self-supporting positioning device 10 includes an acceleration sensor 11, an angular velocity sensor 12, and a distance sensor 13. The acceleration sensor 11 is composed of, for example, a piezoelectric element, detects the acceleration of the vehicle, and outputs acceleration data. The angular velocity sensor 12 is composed of, for example, a vibration gyro, detects the angular velocity of the vehicle when the direction of the vehicle is changed, and outputs angular velocity data and relative orientation data. The distance sensor 13 measures a vehicle speed pulse composed of a pulse signal generated with the rotation of the wheels of the vehicle.

The GPS receiver 18 receives radio waves 19 that carry downlink data including positioning data from a plurality of GPS satellites. The positioning data is used to detect the absolute position of the vehicle from latitude and longitude information and the like.

The system controller 20 includes an interface 21, a CPU 22, a ROM 23, and a RAM 24, and controls the entire vehicle-mounted device 1.

The interface 21 performs an interface operation with the acceleration sensor 11, the angular velocity sensor 12, the distance sensor 13, and the GPS receiver 18. Then, from these, vehicle speed pulse, acceleration data, relative orientation data, angular velocity data, GPS positioning data, absolute orientation data, and the like are input to the system controller 20. The CPU 22 controls the entire system controller 20. The ROM 23 has a non-volatile memory (not shown) or the like in which a control program or the like for controlling the system controller 20 is stored. The RAM 24 readablely stores various data such as route data preset by the user via the input device 60, and provides a working area to the CPU 22.

The system controller 20, a disk drive 31 such as a CD-ROM drive or a DVD-ROM drive, a data storage unit 36, a communication interface 37, a display unit 40, an audio output unit 50, and an input device 60 are connected to each other via a bus line 30. It is connected to the.

Under the control of the system controller 20, the disc drive 31 reads and outputs content data such as music data and video data from a disc 33 such as a CD or DVD. The disk drive 31 may be either a CD-ROM drive or a DVD-ROM drive, or may be a CD-DVD compatible drive.

The data storage unit 36 is composed of, for example, an HDD or the like, and stores various data used for navigation processing such as map data. The communication device 38 is composed of, for example, an FM tuner, a beacon receiver, a mobile phone, a dedicated communication card, or the like, and is distributed from a VICS (registered trademark, Vehicle Information Communication System) center via a communication interface 37. Receive road traffic information such as traffic information and other information.

The display unit 40 displays various display data on a display device such as a display under the control of the system controller 20. Specifically, the system controller 20 reads map data from the data storage unit 36. The display unit 40 displays the map data read from the data storage unit 36 by the system controller 20 on a display screen such as a display. The display unit 40 includes a graphic controller 41 that controls the entire display unit 40 based on control data sent from the CPU 22 via the bus line 30, and a memory such as a VRAM (Video RAM), and can display image information immediately. It includes a buffer memory 42 that is temporarily stored, a display control unit 43 that controls the display of a display 44 such as a liquid crystal display or a CRT (Cathode Ray Tube) based on image data output from the graphic controller 41, and a display 44. .. The display 44 is composed of, for example, a liquid crystal display device having a diagonal of about 5 to 10 inches, and is mounted near the front panel in the vehicle.

The audio output unit 50 includes a D / A converter 51 that performs D / A conversion of audio digital data transmitted from the disk drive 31 or the RAM 24 or the like via the bus line 30 under the control of the system controller 20, and a D / A. It includes an amplifier (AMP) 52 that amplifies the audio analog signal output from the converter 51, and a speaker 53 that converts the amplified audio analog signal into audio and outputs it into the vehicle.

The input device 60 includes keys, switches, buttons, a remote controller, a voice input device, and the like for inputting various commands and data. The input device 60 is arranged around the front panel and the display 44 of the main body of the in-vehicle electronic system mounted in the vehicle. When the display 44 is of the touch panel type, the touch panel provided on the display screen of the display 44 also functions as the input device 60.

The CPU 22 executes a program stored in the ROM 23 or the like in advance to execute the "line-of-sight detection unit", the "control unit", the "operation determination unit" of the present invention, and the "computer" that executes the program of the present invention. Functions as.

[Operation detection processing]
Next, the operation detection process in this embodiment will be described.

(1) Functional block diagram 4 shows a functional block diagram related to an operation detection process of the operation system 100. As shown in the figure, the CPU 22 of the vehicle-mounted device 1 functionally includes a line-of-sight detection unit 5A, a touch pad area division determination unit 5B, and an operation determination unit 5C.

The line-of-sight detection unit 5A detects the driver's line of sight by analyzing the camera image Im received from the line-of-sight detection camera 2 by the vehicle-mounted device 1 based on a known line-of-sight recognition technique. In this case, the line-of-sight detection unit 5A extracts the display area of the driver's face and eyes from the camera image Im based on a known image recognition technique, and positions the iris, pupil, and / and corneal reflex from the extracted area. By detecting, the driver's line-of-sight direction is specified. The line-of-sight detection unit 5A supplies information regarding the line-of-sight direction of the specified driver (also referred to as “line-of-sight information IE”) to the touch pad area division determination unit 5B.

The touch pad area division determination unit 5B predicts the selection item Si selected by the driver and the division operation area Td to be pressed based on the line-of-sight information IE, and the division operation area Td corresponding to each selection item Si according to the prediction result. Determine the range. In the present embodiment, the touchpad area division determination unit 5B is used for a predetermined time (for example, also referred to as a “display gaze period”) in a period from when the driver turns his / her line of sight to the display 44 until he / she turns his / her line of sight from the display 44. When there is a selection item Si whose line of sight is continuously directed for 100 milliseconds or more, it is predicted that the driver selects the division operation region Td corresponding to the selection item Si. The above-mentioned predetermined time is set to a time longer than the time when the line of sight passes through the selection item Si when the line of sight is crossed, based on an experiment or the like. In this way, the touchpad area division determination unit 5B sets the above-mentioned predetermined time as a threshold value, and predicts the division operation area Td corresponding to the selection item Si through which the line of sight passes when the line of sight is simply crossed. Is preferably excluded from.

When the touch pad area division determination unit 5B determines that a plurality of selection items Si have been gazed at for a predetermined time or longer during the display gaze period, the touch pad area division determination unit 5B finally turns to the selection item Si for which the gaze is directed for a predetermined time or longer. It is predicted that the driver will select the corresponding split operation area Td. In general, when one selection item Si is selected from a plurality of selection items Si, there is a tendency to pay attention to the selection item Si to be selected next. Therefore, the touch pad area division determination unit 5B can accurately predict the division operation area Td predicted to be selected by the driver according to this aspect.

Then, the touch pad area division determination unit 5B sets the division of the operation area T so as to increase the division operation area Td corresponding to the selection item Si predicted to be selected by the driver. Then, the touch pad area division determination unit 5B supplies information regarding the division setting of the operation area T (also referred to as “division setting information Is”) to the operation determination unit 5C.

Here, a method of determining the range of the division operation area Td will be described with reference to FIG. FIG. 5A is a diagram showing a division setting of the operation area T on the touch pad 3 in the initial state when the four selection items Si1 to Si4 shown in FIG. 2 are displayed. Here, the touch coordinates indicating each position in the operation area T are represented by a rectangular two-dimensional coordinate space having the upper left corner as the origin (0, 0) and the lower right corner as (1000, 700). Further, the upper left split operation area Td1 corresponds to the upper left selection item Si1 “1.XXXXXX” in FIG. 2, and the upper right split operation area Td2 corresponds to the upper right selection item Si2 “2.YYYYYY” in FIG. The lower left split operation area Td3 corresponds to the lower left selection item Si3 “3.ZZZZZZ” in FIG. 2, and the lower right split operation area Td4 corresponds to the lower right selection item Si4 “4.VVVVVV” in FIG. Corresponds to.

As shown in FIG. 5A, the divided operation areas Td1 to Td4 in the initial state are all congruent and have the same area. Specifically, each of the division operation areas Td1 to Td4 is a rectangular area having a width of "500" and a height of "350".

5 (B) shows the case where the driver turns his / her line of sight to the selection item Si1 of FIG. 2 for 200 milliseconds, the selection item Si2 for 200 milliseconds, and the selection item Si4 for 20 milliseconds in order during the display gaze period. It is a figure which shows the allocation of the division operation area Td. In this case, the touchpad area division determination unit 5B is selected item Si2 in which the line of sight is finally directed for a predetermined time or longer among the selection items Si1 and Si2 for which the line of sight is directed for a predetermined time (100 milliseconds or more here). It is predicted that the division operation area Td2 corresponding to is selected. Therefore, in the example of FIG. 5B, the touchpad area division determination unit 5B expands the division operation area Td2 by "100" in both vertical and horizontal directions from the initial state. Further, as the division operation area Td2 is expanded, the touchpad area division determination unit 5B reduces the division operation areas Td1, Td3, and Td4, which are other division operation areas Td, by the amount overlapping with the expansion area of the division operation area Td2. It is decreasing. In FIG. 5B, the division operation area Td1 has a width of "100" smaller than the initial state, the division operation area Td4 has a vertical width of "100" smaller than the initial state, and the division operation area Td3 is in the upper right. The number of rectangular areas of "100" in the vertical and horizontal corners is reduced from the initial state.

Next, the operation determination unit 5C will be described again with reference to FIG. The operation determination unit 5C identifies the selected division operation area Td based on the division setting information Is generated by the touch pad area division determination unit 5B and the detection signal Sd generated by the touch position detection device 6 of the touch pad 3. To do. Then, the operation determination unit 5C determines that the selection item Si corresponding to the specified division operation area Td is selected, and controls the display unit 40 and the audio output unit 50 so as to execute the operation corresponding to the selection item Si. To do. Specifically, when the operation determination unit 5C receives the detection signal Sd generated by the touch position detection device 6 from the touch pad 3, the operation determination unit 5C divides the division operation area Td to which the touch coordinates indicated by the detection signal Sd belong. It is specified by referring to Is. Then, the operation determination unit 5C recognizes the selection item Si corresponding to the division operation region Td to which the touch coordinates indicated by the detection signal Sd belong as the execution target.

Preferably, the operation determination unit 5C operates when the detection signal Sd is not received even after a lapse of a predetermined time (for example, 5 seconds) or more after the touch pad area division determination unit 5B generates the division setting information Is. It is preferable to return the division setting of the area T to the initial state shown in FIG. 5 (A). As a result, the operation determination unit 5C returns the division setting of the operation area T to the initial setting even when the driver gazes at the display 44 which is not related to the operation, and erroneously detects the operation. Can be prevented.

(2) Processing Flow FIG. 6 is a flowchart showing a procedure of operation detection processing according to this embodiment. The vehicle-mounted device 1 executes the processing of the flowchart shown in FIG. 6 after the system is started and when the selection screen of the selection item Si as shown in FIG. 2 is displayed.

First, the touch pad area division determination unit 5B of the vehicle-mounted device 1 sets the division setting of the operation area T that defines the range of each division operation area Td to the initial setting shown in FIG. 5 (A) (step S101). Next, the line-of-sight detection unit 5A of the vehicle-mounted device 1 acquires the camera image Im from the line-of-sight detection camera 2 (step S102). Then, the line-of-sight detection unit 5A detects the driver's line of sight by analyzing the camera image Im by a known line-of-sight recognition technique (step S103). In this case, the line-of-sight detection unit 5A calculates, for example, the position of the driver's eyes and the line-of-sight direction in the vehicle Ve, and supplies these information to the touch pad area division determination unit 5B as the line-of-sight information IE.

Next, the touchpad area division determination unit 5B determines whether or not the line of sight detected by the line-of-sight detection unit 5A in step S103 is directed to the display 44 (step S104). In this case, the touch pad area division determination unit 5B provides, for example, line-of-sight information Ie, information on the installation position of the vehicle-mounted device 1 in the vehicle Ve stored in advance, and information on the arrangement of the display 44 in the vehicle-mounted device 1. Based on this, it is determined whether or not the display 44 exists in the line-of-sight direction. Then, when the touchpad area division determination unit 5B determines that the line of sight is directed to the display 44 (step S104; Yes), the line of sight is changed based on the information of the display position of each selection item Si on the display 44. The selection item Si on the directed display 44 is specified (step S105). Then, the process returns to step S102. In this way, the touchpad area division determination unit 5B repeatedly executes the process of specifying the selection item Si to which the line of sight is directed during the display gaze period, and the time series of the selection item Si of the line of sight destination during the display gaze period. Accumulate information.

On the other hand, when the touch pad area division determination unit 5B determines that the line of sight is not directed to the display 44 (step S104; No), it determines that the display gaze period has ended, and in step S105 during the display gaze period. Based on the accumulated information, the selected division operation area Td is predicted (step S106). Specifically, the touchpad area division determination unit 5B performs a division operation corresponding to the selection item Si whose line of sight is finally directed for a predetermined time or longer among the selection items Si whose line of sight is directed for a predetermined time or longer during the display gaze period. The region Td is predicted as the split operation region Td pressed by the driver. As a result, the touchpad area division determination unit 5B predicts the division operation area Td corresponding to the selection item Si that was last watched before pressing the division operation area Td as the division operation area Td pressed by the driver.

Next, the touch pad area division determination unit 5B determines whether or not the division operation area Td pressed by the driver can be predicted (step S107). That is, the touch pad region division determination unit 5B determines whether or not there is a selection item Si whose line of sight is directed for a predetermined time or longer during the display gaze period. Then, when the touch pad area division determination unit 5B can predict the division operation area Td to be pressed by the driver (step S107; Yes), the division setting of the operation area T in which the division operation area Td to be predicted to be pressed is expanded. (Step S108). Then, the touch pad area division determination unit 5B supplies the division setting information Is indicating the setting content in step S108 to the operation determination unit 5C. This makes it easier to select the divided operation area Td where the pressing is expected, and even when the driver operates the touchpad 3 in a state where the touchpad 3 cannot be gazed by the driving operation, erroneous operation detection is preferable. Can be suppressed.

The touchpad area division determination unit 5B may execute the process of step S108 when it is determined that the driver has turned his / her line of sight to the touchpad 3. In this way, the touchpad area division determination unit 5B sets the division of the operation area T by expanding the division operation area Td that is predicted to be pressed only in the situation where the driver can determine that the touchpad 3 will be operated from now on. You may go.

On the other hand, when the division operation area Td pressed by the driver cannot be predicted (step S107; No), the touchpad area division determination unit 5B determines that none of the selection items Si are being gazed during the latest display gaze period. , Return the process to step S102.

After setting the division of the operation area T in step S108, the operation determination unit 5C determines whether or not the touch pad 3 has detected a touch (step S109). Specifically, the operation determination unit 5C determines whether or not the vehicle-mounted device 1 has received the detection signal Sd from the touch pad 3. Then, when the touch pad 3 detects the touch (step S109; Yes), the operation determination unit 5C specifies the touch position (that is, the touch coordinates) based on the detection signal Sd (step S110). Then, the operation determination unit 5C identifies the division operation area Td to which the touch coordinates specified in step S110 belong by referring to the division setting information Is, and executes the selection item Si corresponding to the specified division operation area Td. It is determined as the selection item Si to be selected (step S111). Then, the operation determination unit 5C executes the operation according to the selection item Si determined in step S111 by controlling the display unit 40 and the audio output unit 50 (step S112).

On the other hand, when the operation determination unit 5C determines that the touch detection has not been performed (step S109; No), it determines whether or not 5 seconds or more have elapsed since the division setting of the operation area T in step S108 has elapsed (step). S113). Then, when the operation determination unit 5C determines that 5 seconds or more have elapsed since the division setting of the operation area T in step S108 (step S113; Yes), the operation area T division setting is returned to the initial setting (step S114). ). That is, in this case, the operation determination unit 5C determines that the touch pad 3 linked to the line of sight detected in the immediately preceding display gaze period has not been operated. In this case, the process is returned to step S102 again. On the other hand, if 5 seconds or more have not passed since the operation area T was divided and set in step S108 (step S113; No), the operation determination unit 5C has a touch pad 3 linked to the line of sight detected in the immediately preceding display gaze period. It is determined that the operation of (1) may be performed, and the touch detection by the touch pad 3 is continuously performed in step S109.

As described above, the vehicle-mounted device 1 according to the present embodiment detects the line of sight of the operator who operates the touch pad 3, and the line of sight of the operator is predetermined among the plurality of selection items Si displayed on the display 44. It is estimated that the selection item Si that has been directed for more than an hour is an operation candidate. Then, the vehicle-mounted device 1 changes the area of the divided operation area Td corresponding to the estimated operation candidate from the initial setting. As a result, the on-board unit 1 can suitably perform the operation for the selection item Si intended by the operator even in a situation where the operator is driving and cannot gaze at the touch pad 3.

[Modification example]
Hereinafter, each modification of the embodiment will be described. In addition, each of these modified examples may be applied in combination to the above-described embodiment.

(Modification example 1)
When the on-board unit 1 determines the selection item Si to be executed in step S111 of FIG. 6, the in-vehicle device 1 invalidates the touch operation to the division operation area Td other than the division operation area Td expanded in step S108 (that is, cannot be operated). May be set to. That is, in this case, even when the vehicle-mounted device 1 detects the touch position in the division operation area Td other than the division operation area Td where the pressing is predicted in step S110, it corresponds to the division operation area Td. The operation according to the selection item Si is not executed. As a result, the vehicle-mounted device 1 can more reliably suppress false detection of the operation on the touch pad 3.

(Modification 2)
The touch pad 3 is not limited to the application for selecting the selection item Si. In addition to this, the touch pad 3 may accept various operations such as enlargement, reduction, and scrolling of the map displayed on the display 44.

In this case, for example, the vehicle-mounted device 1 determines that the operation (input) for selecting the selection item Si has been performed only when the touch pad 3 detects that the pressure is pressed at a predetermined pressure or higher, and the above-mentioned predetermined pressure. When the touch pad 3 detects a contact of less than or equal to less than, it is determined that the operation is for the map displayed on the display 44. In this case, the detection signal Sd includes information on the contact pressure in addition to the coordinate information of the touch position, and the vehicle-mounted device 1 has a contact pressure with the touch pad 3 equal to or higher than a predetermined pressure based on the detection signal Sd. Judge whether or not there is. By doing so, the on-board unit 1 can suitably accept both the operation on the map and the operation related to the selection of the selection item Si by the touch pad 3.

Preferably, when the vehicle-mounted device 1 determines that the touch operation to the touch pad 3 has been performed at a pressure lower than the predetermined pressure, that is, when it is determined that the operation for the map has been performed, the division setting of the operation area T is set. It is good to return to the default setting. Thereby, when the driver does not have the intention to select the selection item Si, the division setting of the operation area T can be returned to the initial setting.

In addition, the in-vehicle device 1 does not distinguish between the operation on the map and the operation related to the selection of the selection item Si based on the pressing pressure on the touch pad 3, but based on the contact time or the number of contacts with the touch pad 3, these operations The operation may be identified. For example, the vehicle-mounted device 1 is an operation related to selection of the selection item Si when it detects a contact in which the contact time with the touch pad 3 is less than a predetermined time, or when it detects a plurality of contacts in a short time. Is determined. On the other hand, when the in-vehicle device 1 detects a contact in which the contact time with the touch pad 3 is longer than a predetermined time, or does not detect continuous contact in a short time, it determines that the operation is for the map. As described above, the on-board unit 1 can preferably distinguish between the operation on the map and the operation related to the selection of the selection item Si, depending on the contact time or the number of contacts with the touch pad 3.

(Modification 3)
In step S107 of FIG. 6, when the divided operation area Td to be pressed can be predicted, the vehicle-mounted device 1 may highlight the selection item Si on the display 44 corresponding to the divided operation area Td.

FIG. 7 shows a display example of the display 44 according to this modification. In the example of FIG. 7, as a result of the driver's line of sight being directed to the selection item Si2, the vehicle-mounted device 1 predicts that the corresponding division operation area Td2 shown in FIG. 5B is pressed, and the division operation area The operation area T is divided and set so that Td2 is enlarged. In this case, the on-board unit 1 emphasizes by enlarging the selection item Si2 corresponding to the division operation area Td2 predicted to be pressed and applying a edging effect. According to this aspect, the driver can visually confirm the prediction result by the vehicle-mounted device 1. For example, when the selection item Si different from the selection item Si to be selected is highlighted, the driver looks at the selection item Si of interest again after 5 seconds have elapsed based on step S113 of FIG. By pointing, you can prevent erroneous operation.

(Modification example 4)
In the embodiment, the selection item Si and the division operation area Td were four each. However, the number of selection items Si and the division operation region Td applicable to the present invention is not limited to four, and may be any number of two or more. Further, the arrangement of the selection item Si and the division region Td is not limited to the embodiment, and may be any arrangement as long as there are any number of 2 or more. Even in this case, each selection region Si has a corresponding relationship with any of the division operation regions Td.

1 On-board unit 2 Eye-gaze detection camera 3 Touchpad 5A Eye-gaze detection unit 5B Touchpad area division determination unit 5C Operation determination unit

Claims (14)

An operation unit that is separate from the display unit that outputs a plurality of operation targets and has an operation area corresponding to each of the plurality of operation targets output to the display unit .
A line-of-sight detection unit that detects the line of sight of the operator who operates the operation target,
Of the plurality of operation targets, the operation target detected by the line-of-sight detection unit and directed to the operator's line of sight for a predetermined time or longer is estimated as an operation candidate, and the operation of the operation unit corresponding to the estimated operation candidate is performed. A control unit that expands the area of the area from the initial setting,
An operating device characterized by comprising. The control unit presumes that the operation target whose line of sight is finally directed to the predetermined time or longer during the period from when the line of sight is directed to the display unit displaying the operation target until when the line of sight is separated from the display unit is the operation candidate. The operating device according to claim 1. The operation device according to claim 1 or 2, wherein when the control unit estimates the operation candidate, the operation candidate is emphasized and output. The operation device according to any one of claims 1 to 3, wherein the control unit makes another operation area different from the operation area of the operation unit corresponding to the operation candidate inoperable. Claims 1 to 4 further include an operation determination unit that detects whether or not the operator has pressed the operation area of the operation unit corresponding to the operation candidate estimated by the control unit within a predetermined time. The operating device according to any one of the above. When the operator does not press the operation area of the operation unit corresponding to the operation candidate estimated by the control unit within a predetermined time, the operation determination unit of the operation unit corresponding to the operation candidate. The operating device according to claim 5, wherein the area of the operating area is returned to the initial setting. A display unit that displays a map and the operation target,
When it is detected that the operation area of the operation unit corresponding to the operation candidate estimated by the control unit is pressed at a predetermined pressure or higher, it is determined that the operator has selected the operation candidate, and the predetermined pressure is determined. An operation determination unit that determines that an operation on the map has been performed when a contact with less than is detected is detected.
The operating device according to any one of claims 1 to 4, further comprising. When the operator detects that the operation area of the operation unit corresponding to the operation candidate estimated by the control unit is pressed with a pressure lower than a predetermined pressure, the operation determination unit corresponds to the operation candidate. The operating device according to claim 7, wherein the area of the operating area of the operating unit is returned to the initial setting. The operating device according to any one of claims 1 to 8, wherein the operating device is mounted on a moving body. The operating device according to claim 9, wherein the operating unit is provided on the steering wheel of the moving body. The operating device according to any one of claims 1 to 10, wherein the operating target is displayed at a position on the line of sight that does not overlap with the operating unit. It is a control method executed by an operation device including an operation unit that is separate from a display unit that outputs a plurality of operation targets and has an operation area corresponding to each of the plurality of operation targets output to the display unit.
A line-of-sight detection process that detects the line of sight of the operator who operates the operation target,
Among the plurality of operation targets, the operation target whose line of sight of the operator detected in the line-of-sight detection step is directed to a predetermined time or longer is estimated as an operation candidate, and the operation of the operation unit corresponding to the estimated operation candidate is performed. A control process that expands the area of the area from the initial setting,
A control method characterized by having. A program executed by a computer that receives electrical signals from an operation unit that is separate from the display unit that outputs a plurality of operation targets and has an operation area corresponding to each of the plurality of operation targets output to the display unit. hand,
A line-of-sight detection unit that detects the line of sight of the operator who operates the operation target,
Of the plurality of operation targets, the operation target detected by the line-of-sight detection unit and directed to the operator's line of sight for a predetermined time or longer is estimated as an operation candidate, and the operation of the operation unit corresponding to the estimated operation candidate is performed. Control unit that expands the area of the area from the initial setting
A program characterized in that the computer functions as a device. A recording medium on which the program according to claim 13 is recorded.