JPH09212082A - Visual line input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the occurrence of the frame out and out-of-focus of the eyeballs. SOLUTION: The eyeball position in a picked up image and the focusing degree of the eyeball are detected from the image of the eyeball subjected to image pickup by a visual line input means. A marker indicating the eyeball position and the focusing degree display markers 32 respectively corresponding to focusing time, non-focusing time and eyeball non-detection time are displayed on a display screen. The deviation rate from the focusing position of the eyeball may be displayed in multiple stages together with the direction of the deviation and the focusing degree display markers 32 may be displayed in the visual line position on the screen.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for ensuring the detection accuracy of a line of sight in an apparatus for inputting an instruction by the line of sight of an operator.

[0002]

2. Description of the Related Art As a typical conventional information input device,
A mechanical switch, an electric switch, an electronic switch, a switch using voice recognition, and the like are known.
Such an information input device requires a series of operations of visually searching for a switch to be operated and manually operating the switch. Therefore, for example, a complicated operation such as destination input of a vehicle-mounted navigation system is required. When performing (scale operation, scrolling, point setting), many loops are performed to visually check the map information, manually operate it, and visually check the result again. From the viewpoint of operability, Not always a good man-machine interface.

Further, since the switch must be installed within easy reach, there are restrictions on the design of the device.
For this reason, a remote control method using a remote controller (remote control) is generally used. However, considering that the device is mounted in a vehicle, for example, there is no effective plan for the installation location of the remote control, and vibration of the vehicle Or lane change,
There is a problem that the remote control may move carelessly due to running on a curve.

Besides the remote controller, an input device using voice recognition has been proposed, but since the words of the command must be remembered, if many operations are to be performed by voice, the command word is forgotten. Or mistakes may occur.
In addition, since the user is required to speak, there are problems that some people like and dislike it, and that verbal communication is obstructed during the switch operation.

As described above, a visual operation such as confirmation of the switch position and confirmation of the switch operation state is inevitably generated in the switch operation. Therefore, it is considered that if the information can be transmitted only by the line of sight, the intention of the switch operator can be transmitted to the control device without using finger operation or voice.

Further, if the line of sight is used, one gaze area and line of sight movement can be associated with one information input, so that problems such as forgetting a command and mistakes can be solved.
Therefore, a line-of-sight information input device has already been put to practical use, for example, for a single-lens reflex camera or a video camera.

[0007]

However, in order to use the line-of-sight direction as an interface tool, high measurement accuracy is generally required. In particular, in the case of measuring the line of sight without contact, generally, a camera captures a two-dimensional image of the face, the eyeball portion of the image is extracted, and the line-of-sight direction is specified. In this case, it is a major issue to achieve both a wide field of view that does not restrain the movement of the user and high measurement accuracy.

That is, in order to obtain high measurement accuracy, a camera for observing a narrow field of view must be used, and the user must be aware that the eyeball does not fall out of the field of view. As a result, there is a problem in that the usability of the man-machine interface is hindered. In the application example to the single-lens reflex camera or the video camera described above, since the user's eyeball is always in the finder, there is no concern that the eyeball will be out of frame during use, and a narrow photographing field of view and high measurement accuracy can both be achieved.

Further, for example, as described in Japanese Patent Application Laid-Open No. 1-274736, there is disclosed a technique in which a display means for displaying a gaze target is provided in the field of view of an observation system to notify an operator of a gaze position. There is. However, when trying to use the line-of-sight input device in free space by using a completely non-contact line-of-sight measuring means, the line-of-sight of the user should be kept in view of the camera in order to prevent the operator's eyeball from being framed out as described above. It is necessary to provide information such as where in the visual field and how the focus is.

It is also possible to prevent the frame out of the eyeball by causing the image input section for detecting the line of sight to follow the movement of the eyeball. However, in order to perform the follow-up control, the autofocus mechanism is used. And hardware such as a swing mechanism are required, which leads to an increase in system cost. Therefore, the present invention has been made by paying attention to such conventional problems, by displaying the eyeball position and the focus state on the captured image of the operator's eyeball on the screen,
An object of the present invention is to provide a line-of-sight input device that can easily hold the operator's eyeball position and quickly return the eyeball to a predetermined position.

[0011]

In order to achieve the above object, the invention as set forth in claim 1 detects a visual line direction of an operator and controls a peripheral device based on the detected visual line direction. In the input device, the operator's eyeball is imaged for detecting the operator's gaze direction, and the eyeball position in the image obtained from the imaged image of the eyeball and the degree of focus of the eyeball are displayed on the screen. I did it.

According to a second aspect of the present invention, an image input means for picking up an eyeball of an operator, an eyeball position detecting means for detecting an eyeball position in an image from an image picked up by the image inputting means, and the image inputting means. Focusing degree detecting means for detecting a focusing degree of the eyeball in the image from the captured image, display means for displaying information for giving an instruction by the operator's line of sight, and the detected line-of-sight direction of the operator. And a display control means for displaying the focus degree on the display means.

According to a third aspect of the present invention, the display means displays a focus degree pattern corresponding to the detected focus degree. According to a fourth aspect of the present invention, the focus degree pattern is set when the eyeball in the captured image is in focus,
Different patterns corresponding to the three cases of non-focusing and non-eyeball detection were used.

According to a fifth aspect of the present invention, the focusing degree pattern is a pattern with which a direction of deviation from a predetermined position where the eyeball in the captured image is focused with respect to the eyeball position of the operator can be determined. . In the invention according to claim 6, the size of the focusing degree pattern is changed in multiple steps according to the amount of deviation from a predetermined position where the operator's eyeball is focused.

According to a seventh aspect of the present invention, the focus degree pattern is displayed at a position where the line of sight of the display means is directed.

[0016]

According to the first aspect of the invention, the operator detects the eyeball position and the focus degree in the captured image from the captured image of the operator's eyeball and displays them on the display means. Can easily check the eyeball state, so that the eyeball position can be easily held at a predetermined position where the eyeball is in focus, and when the eyeball position is separated from the predetermined position, the eyeball can be quickly returned to the line of sight. The direction can be detected.

According to the second aspect of the present invention, the eyeball portion of the operator is imaged by the image inputting means, and the eyeball position in the image of the operator is detected from the picked-up image by the eyeball position detecting means. By detecting the focus degree of the eyeball in the captured image by the focus degree detection means and displaying the eyeball position and the focus degree in the image on the display means, the operator's eyeball is out of frame or defocused. At this time, by confirming the eyeball position and the in-focus state displayed on the display means, it is possible to quickly and easily return the position of the operator's eyeball to a predetermined position and detect the line-of-sight direction.

According to the third aspect of the present invention, the operator can more easily visually recognize the focus degree by displaying the pattern according to the detected focus degree on the display means. . According to the invention as set forth in claim 4, it is possible to easily determine whether or not the eyeball position of the operator is within a predetermined position, and to confirm that the eyeball is not detected when the eyeball is framed out. Therefore, the operator can more clearly understand the input state of the line of sight.

According to the fifth aspect of the present invention, it is easy to determine whether the operator's eyeball position is at a predetermined position where the eyeball in the captured image is focused, or is too close or too far from the predetermined position. Therefore, the eyeball position can be quickly returned to the predetermined position where the eyeball is focused. According to the sixth aspect of the invention, when the eyeball position is closer to the in-focus eyeball position, the in-focus degree pattern is displayed larger, while when it is far from the in-focus eyeball position, the in-focus degree pattern is decreased. By displaying, it is possible to intuitively determine the direction in which the eyeball position is to be moved, and it is possible to quickly perform a returning operation to the predetermined position of the eyeball position.

According to the invention of claim 7, since the pattern of the degree of focus is displayed in the direction of the line of sight of the operator, it is not necessary to move the line of sight in the display means in order to confirm the degree of focus. Since the state of the eyeball position can be recognized promptly, the eyeball position can be returned to the predetermined position more quickly.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIG.
This will be described with reference to FIG. The first embodiment is applied to a vehicle-mounted navigation device that inputs a destination according to the direction of the operator's line of sight. The destination setting of this navigation device is performed while the vehicle is stopped because it involves complicated operations.

FIG. 1 shows the configuration of the line-of-sight input device of this embodiment.
This will be described with reference to FIG. FIG. 1 is an overall configuration diagram of the line-of-sight input device according to the present embodiment, FIG. 2 shows a state when mounted on a vehicle, and FIG. 3 shows an example of a navigation display screen. The outline of the line-of-sight input method will be described below. An illumination A102 that emits invisible light such as a near-infrared LED is arranged in the vicinity of an operator (the driver of the vehicle in this case) 101, and an illumination B103 having the same specifications as the illumination A is relative to the illumination A102. Is arranged at a known position, and the operator 10 controls the illumination A102 and the illumination B103 by the illumination light emission control unit 104.
It emits light toward 1 under the specified conditions.

On the other hand, the operator's eyes are included by the image input unit 105 such as a CCD camera having a sensitivity to the infrared region, which is arranged so as to have the light irradiation direction of the illumination A102 and the photographing direction coaxial with each other. The face area is imaged. The image input unit 105 corresponds to the image input means. Then, the captured image signal is converted into digital data by the A / D converter 106 and stored in the image memory 107.

From the imaged image stored in the image memory 107, the retina reflection image (pupil position) is extracted by the pupil extraction unit 108, and the cornea reflection image of the illumination A102 is extracted from the vicinity of the pupil position by the cornea reflection image extraction unit 109. To do. Then, the focus determination unit 110 calculates a focus degree indicating the degree of focus of the corneal reflection image from the extraction result of the corneal reflection image, and determines whether or not it is in focus.

A gaze position calculation unit 111 calculates a gaze position from the positions of the extracted pupil and corneal reflection image, and a gaze determination unit 112 determines whether or not a particular position is gazed. The eyeball state calculation unit 113 calculates the eyeball position in the captured image and the degree of focus of the eyeball based on the result of pupil extraction and the result of focus determination, and controls the display on the navigation display screen 114. This function corresponds to the eyeball position detection means, the focus degree detection means, and the display control means, and the navigation display screen 11
4 corresponds to the display means.

On the other hand, the navigation map control unit 115 controls the display of the navigation display screen 114 according to the result of the gaze determination, and displays the navigation display screen 114 for navigation. Further, prior to the measurement of the eyeball position, in order to perform the calibration work, the calibration index control unit 116 draws the calibration index at a specific position on the screen, and the predetermined conversion coefficient necessary for the gaze position calculation is obtained from the calibration result. It is calculated by the calibration data calculation unit 117. Then, the calculated calibration data is stored in the calibration data storage unit 118.

Further, the destination input start switch 119 is turned on to start the destination input, and the calibration start switch 120 is turned on to start the calibration. The overall control unit 121 controls the series of processes described above. next,
The on-vehicle state of the line-of-sight input device according to the present embodiment will be described with reference to FIG. In FIG. 2, a navigation display screen 114 for displaying a navigation map screen is arranged near the steering wheel 201, and a destination input start switch 119 installed near the navigation display screen 114 is used to instruct the start of destination input. Similarly, a calibration start switch 120 provided near the navigation display screen 114 is used to instruct the start of calibration.

Below the navigation display screen 114, an image input section 105, an illumination A102 and an illumination B103 are arranged. Then, various icons 31, 32, 33, 34, 35a, 35b, 35c, 35d are displayed on the navigation display screen 114 as shown in FIG. Since the display states of these icons differ depending on the state of navigation control, details will be described later.

Next, the visual line inputting method according to the present embodiment will be described in detail with reference to FIGS. 1 and 2. In the measurement of the line-of-sight direction, it is necessary to perform calibration work to clarify the relationship between the line-of-sight direction and the position of the index displayed on the navigation display screen. Therefore, this proofreading work will be described below while explaining a specific eye-gaze input method.

The operator 101 starts the calibration by pressing the calibration start switch 120 arranged near the navigation display screen 114. During this calibration, the navigation display screen 114 is shown in FIG.
The indexes shown in are sequentially displayed one by one. The operator keeps watching the index while being displayed. Image input section 10
5 inputs the image of the face area including the operator's eyeball, and A
The data is converted into digital data by the / D converter 106 and stored in the image memory 107. At this time, the illumination A102 lights up,
An image A when the illumination B103 is off and an image B when the illumination B103 is on and the illumination A102 is off are input.

The pupil extraction unit 108 calculates the difference between the image A and the image B. Since the pupil of the image A is observed brightly by the reflected light from the retina and the pupil of the image B is observed dark, the difference between the images is that the pupil region is emphasized. Each pupil region is numbered by binarizing this difference image with a fixed threshold value and further performing labeling processing.

In this difference image, in addition to the retina reflection image, for example, a spectacle lens reflection image, a spectacle frame reflection image,
Various noises may be included, such as part of the face caused by changes in external lighting. Since these noises generally have an indefinite shape and an indefinite area, they can be distinguished from a retinal reflection image observed as a circle or an ellipse having an area expected in advance. Here, discrimination is performed based on the area and shape of the pupil region.

Area R of each region obtained as a result of labeling
_iIs a predetermined threshold value S ₁, S_Two(However,
S₁<S_Two) Compared to S₁<R_i<S_TwoTo satisfy
Extract only the area. Where S₁, S_TwoOf the camera
Area of the pupil (pixels in the image estimated from the imaging magnification
Just set it to (number).

The reflection image of the spectacle lens is also observed as a circular area. For example, if the aperture of the lens is narrowed and the area of the reflection image of the spectacle lens is set smaller than the area of the retina reflection image, Both can be distinguished. Next, for a region that remains unidentified, the ratio F of the region area to the area of the circumscribed rectangle is calculated. Since the retina reflection image is observed as a circle or an ellipse,
While the ratio F is equal to or greater than the predetermined value Fth, for example, the spectacle frame reflection has a long and narrow shape along the frame, and therefore even if it has the same area as the retina reflection image, F becomes small and can be identified. Is.

As a result of the above discrimination, the remaining area is determined as a retina reflection image. Further, when the retinal reflection image is not found due to blinking or frame-out of the eyeball position, a display indicating that the retinal reflection image is not found is displayed as shown in FIG. At this time, the above processing is repeated by inputting an image again. Next, the processing for the area determined to be the retina reflection image will be described.

The corneal reflection image extraction unit 109 obtains the barycentric position (Xg, Yg) of the extracted retina reflection image, and sets a small area including the retina reflection image centered on the barycentric position on the difference image. A point having the maximum brightness in this small area is defined as a corneal reflection image. In the focus determination unit 110, the brightness value I of the extracted corneal reflection image is calculated, and α · I (0 <α <
Binarization is performed using 1) as a threshold value. That is, 1 if the luminance value of the pixel of the difference image is larger than α · I, 0 if it is smaller.
Set. Then, the number K of pixels having a luminance value larger than α · I is calculated.

Here, if the difference image is an in-focus image, the luminance peak for the corneal reflection image in the small area becomes sharp and K has a small value. On the other hand, if the difference image is an out-of-focus image, the luminance peak of the corneal reflection image becomes gentle and K has a large value. Therefore, when K is larger than a preset value, it is determined that the input image is out of focus.

When the input image is not out of focus, an image focused on the eyeball is obtained, and when the pupil center of gravity is observed near the center of the image, the calibration data calculation unit 11
In Fig. 7, two parameters δx = Xg-from the center of gravity (Xg, Yg) of the pupil on the image and the coordinates (Px, Py) of the corneal reflection image.
Obtain Px, δy = Yg-Py. The eyeball state display control unit 113 creates a display pattern indicating the presence or absence of focus on the center of gravity (Xg, Yg) of the pupil and the corneal reflection image, and displays it on the navigation display screen 114. An example of this display pattern is shown in FIG. In FIG. 4, the position of the calibration index 41 is regarded as the position of the center of gravity of the retina reflection image, and a moving frame 42 is provided outside the position of the calibration index in the imaging field of view of the eyeball.
The position of 41 indicates the position of the eyeball in the visual field. When the corneal reflection image is in focus, the calibration index 41 is displayed as a filled pattern, and when the corneal reflection image is not in focus, it is displayed as a white pattern.

As described above, since the position of the eyeball in the photographing field of view and whether or not the eyeball is in focus are displayed, the operator himself can easily check the frameout of the eyeball and the out-of-focus. It is possible for the operator to return to the in-focus fixed position by moving the head position or the like depending on the situation. The above-described calculation processing of δx and δy is repeated until calibration data is obtained N times for the same index. The N pieces of δx and δy obtained as a result are averaged and stored in the calibration data storage unit 118 as average calibration data δxi and δyi for the index i.

Then, the average calibration data is calculated for all the prepared indexes (in the present embodiment, 5 points in the horizontal direction and 5 points in the vertical direction as shown in FIG. 5 in total), and calculated in the horizontal direction. Group of averaged calibration data δx and averaged calibration data δy calculated in the vertical direction
For each group of, the index position and the conversion formulas of δx and δy are determined by the least squares method. That is, in general, in a region where the line-of-sight angle is small, the coefficients a and b are determined on the assumption that the values of δx and δy and the index position are represented by the linear expression y = ax + b.

The calibration work is completed by the above processing. It should be noted that this calibration work may be carried out once, saved, and reused later if necessary, and does not need to be carried out every time the line-of-sight input processing is performed. Next, a procedure for inputting a destination to a vehicle-mounted navigation device using the line-of-sight input device will be described with reference to FIGS. 2, 3 and 6.

When it becomes necessary to input the destination of the navigation device, the destination input start switch 119 shown in FIG.
When is pressed, the navigation display screen 114 is displayed as shown in FIG. 3, and the destination input is accepted. Further, when the destination input start switch 119 is pressed, δx and δy are continuously calculated by the above-mentioned processing at any time thereafter, and converted into the position coordinates on the navigation screen 115 by the conversion formula obtained at the time of calibration. The gaze area display marker shown in FIG. 3 at the converted position
Superimpose 31 on the navigation display screen. This δ
In parallel with the calculation of x and δy, the pupil position in the captured image and the focusing degree to the corneal reflection image are calculated, and the eyeball position and the focusing degree in the captured image are displayed in the upper left part of the navigation display screen 114. The focus degree display pattern 32 is displayed.

With this focus degree display pattern 32, it is possible to determine whether the operator should move up, down, left or right by drawing the eyeball display pattern 37 in the moving frame 36 which is likened to the field of view. . That is, the head may be moved vertically and horizontally so that the eyeball display pattern 37 is located at the center of the moving frame 36. The eyeball display pattern
Similar to the calibration index of FIG. 4, 37 is displayed as a filled pattern when in focus and as a white pattern when out of focus.

Next, the operator 101 sets the wide area display marker 33 or the detailed display marker set below the navigation display screen 115.
When the user gazes at 34, the gaze area display marker 31 moves onto the gazed marker. Here, considering the case of gazing at the wide area display marker 33, for example, gazing at the wide area display marker 33,
When the eyes are closed for a while (about 300 msec), the wide area display marker 33
Is selected and confirmed. This is 100m for one gaze position calculation
When it is assumed that sec is required, the determination may be made based on the fact that the eyeball was not found three times in a row. Since a normal blink completes in about 100 msec, selection confirmation does not occur due to careless blinking.

When the wide area display marker 33 is selected and confirmed, the scale of the map displayed on the navigation display screen 115 is changed to one step wide area (scale ratio is increased), and the map changed to the wide area is displayed again. . The destination can be displayed on the screen by selecting the wide area display marker 33 multiple times (S61).
Next, by paying attention to one of the scroll markers 35a, 35b, 35c, 35d set on the four sides of the navigation display screen 115 and closing the eye for a while (about 300 msec), the scroll direction is selected and confirmed. , Scrolls the screen in a predetermined direction indicated by the scroll marker. This is repeated multiple times as necessary so that the vicinity of the destination is displayed in the center of the screen (S62).

Then, when the vicinity of the destination is displayed in the center of the screen, the detailed display marker 34 is selected and confirmed and the screen is enlarged and displayed. This step is repeated multiple times so that the true destination is displayed in detail on the screen (S63). In this way, when the true destination is displayed in detail on the screen, the operator 101
Pays attention to the destination that he / she wants to set, and when the gaze area display marker 31 points to the destination, closes his eyes for a while (about 300 msec) to determine the destination (S64).

When the destination is fixed, the navigation device calculates the optimum route from the current position to the destination by the navigation map control unit 114 and displays it on the map on the navigation display screen 115. Then, the navigation display screen 115 is set to a screen for displaying the current position, and the navigation operation toward the destination is started.

As described above, in the present embodiment, the navigation display screen displays whether or not the eyeball is in the photographing field of view and whether or not the eyeball is in focus. Can easily confirm the out-of-focus or out-of-focus of the eyeball, and can move the head position or the like depending on the situation to detect the eyeball and quickly return to a predetermined position where the eye is in focus.

Next, a second embodiment will be described in which the operator can easily move his or her own eyeball to a fixed position. In the present embodiment, when the position of the operator is too close according to the degree of focus of the eyeball, the focus degree display pattern on the navigation display screen is displayed large, and when the operator is too far, the focus degree is displayed. By displaying the display pattern in a small size, it is possible to easily understand the correspondence that the operator should take to move the eyeball to the predetermined position where the eyeball is focused.

The details of the method of displaying the focus degree on the navigation display screen will be described below. After extracting the retina reflection image and calculating the focus degree of the corneal reflection image in the same manner as in the first embodiment, when it is determined that the eyeball is out of focus, the deviation of the eyeball from the in-focus position is performed. To judge. That is, when it is determined that the eye is not in focus, the area S (pixels) of the retina reflection image is calculated, and when this S is larger than a preset threshold Sth, the operator approaches the camera. It is determined to be too far (rear focus), and when S is smaller than Sth, it is determined that the operator is too far from the camera (front focus).

As shown in FIG. 7, at the time of rear focus, the eyeball display pattern 72 of the focus degree display pattern 71 is enlarged,
If the eyeball display is made small during the front focus, the operator can easily judge whether he should move away or approach by looking at the focus degree display pattern 71 on the navigation display screen. That is, the operator may move the head back and forth so that the eyeball display pattern 72 has a normal size or the eyeball display pattern 72 has a filled display.

Here, when the eyeball of the operator is framed out, the eyeball cannot be detected, and it is no longer instructed where to move his or her eyeball. Therefore, for example, the eyeball position when looking at the navigation display screen of the navigation device is considered to be almost constant for each operator. Therefore, the image input unit is designed so that the eyeball can be captured in advance at the position where the navigation display screen is normally viewed. 105, Illumination A 102, and Illumination B
Change the orientation of 103. As a result, it is possible to avoid the situation where the instruction of the eyeball position cannot be obtained.

By making this adjustment each time the operator changes, it is possible to reduce the possibility that the eyeball position of the operator will be out of frame during navigation control. As described above, in the present embodiment, the displacement of the operator's eyeball position is visualized not only in the up-down and left-right directions but also in the front-back direction, so that the eyeball can be immediately moved to the fixed position. become able to. As a result, a man-machine interface with high operability can be constructed.

Next, a third embodiment in which the eyeball state of the operator is displayed in the gaze area will be described. According to the line-of-sight input device according to the present embodiment, as shown in FIG. 8, the eye-gaze area display marker 31 that performs line-of-sight input has the operator's eyeball position and the degree of focusing on the eyeball (in the first embodiment). (Corresponding to the second embodiment), or a state of deviation of the eyeball position from a predetermined in-focus position (corresponding to the second embodiment) is displayed at any time.

Therefore, the operator can simultaneously grasp the in-focus position and the focus state of the own eyeball position and the deviation of the eyeball position, and easily return the eyeball position to the predetermined position for focusing. Therefore, it is possible to improve the line-of-sight measurement accuracy in a narrow field of view and perform more stable line-of-sight input. As described above, according to the present embodiment, it is possible to obtain the effect that the eyeball state of the operator can be recognized more quickly and more naturally.

In the three embodiments described above, the position of the eyeball and the degree of focus are displayed as an artificial image created by a computer. However, this is an original image captured by a camera (illumination A102). The image when it is lit) may be displayed superimposed. Further, the above-described three embodiments have been described based on an example in which the line-of-sight input device of the present invention is applied to a navigation device, but the present invention is not limited to this, and a line-of-sight for controlling a specific peripheral device can be used. It can be widely used as an input device.

[Brief description of drawings]

FIG. 1 is a basic configuration diagram common to the first to third embodiments.

FIG. 2 is a diagram showing a state in which a visual line detection device common to the first to third embodiments is mounted on a vehicle.

FIG. 3 is a diagram showing an example of a navigation display screen.

FIG. 4 is a diagram showing an example of a display of a calibration index.

FIG. 5 is a diagram showing a display position of a calibration index.

FIG. 6 is a diagram showing a flowchart of a destination setting process and a navigation display screen at each stage.

FIG. 7 is a diagram showing a navigation display screen according to the second embodiment.

FIG. 8 is a diagram showing a navigation display screen according to the third embodiment.

[Explanation of symbols]

 101 Operator 102 Illumination A 103 Illumination B 110 Focus determination unit 114 Navigation display screen 31 Gaze area display marker 32,71,81 Focus degree display marker 37,72 Eyeball display marker

Claims (7)

[Claims] 1. A visual line input device for detecting a visual line direction of an operator and controlling peripheral devices based on the detected visual line direction, wherein an eyeball of the operator is imaged for detecting the visual line direction of the operator. A line-of-sight input device displaying an eyeball position in the image obtained from a captured image of the eyeball and a focus degree of the eyeball on a screen. 2. An image input means for picking up an eyeball of an operator, an eyeball position detecting means for detecting an eyeball position in the image from an image picked up by the image input means, and an image picked up by the image input means Focusing degree detection means for detecting the focusing degree of the eyeball in the inside, display means for displaying information for giving an instruction by the operator's line of sight, and the detected operator's line-of-sight direction and focus degree The line-of-sight input device according to claim 1, comprising display control means for displaying on the display means. 3. The line-of-sight input apparatus according to claim 2, wherein the display means displays a focus degree pattern corresponding to the detected focus degree. 4. The focusing degree pattern is a different pattern corresponding to three cases of focusing, non-focusing, and non-detecting an eyeball in the captured image. Eye gaze input device. 5. The focus degree pattern is a pattern in which a direction of deviation from a predetermined position where the eyeball in the captured image is focused with respect to the eyeball position of the operator can be determined.
The line-of-sight input device described in. 6. The line-of-sight input according to claim 5, wherein the focus degree pattern is configured such that the size of the pattern is changed in multiple stages according to the amount of deviation from a predetermined position where the operator's eye is focused. apparatus. 7. The focus degree pattern is displayed at a position to which the line of sight of the display means is directed.
The visual line input device according to claim 6.