JPH09167049A - Line of sight input device for console - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately move an input position mark to a necessary position by moving the input position mark like a cursor pointer in steps to the side of an area to which the line of sight an operator is directed. SOLUTION: A pupil extraction part 141 extracts a retia reflection image (pupil) from face image stored in an image memory 102. A gaze position calculation part 144 calculates a gaze position from the pupil position extracted by the pupil extraction part 141 and a cornea reflection position extracted by a cornea reflection image extraction part 142. A gaze area determination part 150 determines the area in which the line of sight direction is from the calculated gaze position. A cursor movement control part 172 performs control for moving the pointer cursor on a navigation image to the gaze position calculated by the gaze position calculation part 144 at a time in quick mode and to the area, where the line of sight of the operator is, determined by the gaze area determination part 150 in steps by a specific quantity in step mode.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a console line-of-sight input device, and more particularly to a console line-of-sight input device for various devices with a display on which an input position mark such as a pointer cursor is displayed.

[0002]

2. Description of the Related Art Conventionally, as an input device, a mechanical switch, an electric switch, an electronic switch, a switch utilizing voice recognition, or the like has been known.

[0003] In addition, research on information input by the line of sight has been conducted, and practical examples have been seen for single-lens reflex cameras and video cameras.

There have been many proposals for measuring the line of sight in a non-contact manner. Generally, a CCD camera captures a two-dimensional image of a face,
It is performed to extract the eyeball image and specify the line-of-sight direction.

If the gaze point of the operator can be determined from the measured line-of-sight direction, various devices can be controlled.

For example, Japanese Patent Application Laid-Open No. 4-101126 discloses a method capable of performing line-of-sight control such as focus control and exposure control of a still camera. In this invention, the gazing point is determined by obtaining the frequency of the line of sight of the operator, the dwell time, and the trajectory.

More specifically, the viewfinder of the camera is divided into a plurality of small areas, which small area the calculated gaze direction belongs to is calculated, and then the most probable probability is obtained by referring to the time-continuous measurement results. Has a high gaze area.
In addition, as another example, a method of determining a gaze area using fuzzy theory is described.

[0008]

The conventional mechanical switch, electric switch, and electronic switch input device described above have the following problems.

(A) Since a series of operations of manually searching for a switch to be operated and manually operating the switch are required, for example, complicated operations such as inputting a destination of an in-vehicle navigation system (scale conversion, scrolling). , Point setting), a map is visually checked, operated by hand, and the result is visually checked again and again, which means that it is not necessarily an excellent man-machine interface from the viewpoint of operability. hard.

(B) The switch must be installed within a reach of the operator, which causes restrictions on the design of the device.

(C) Although remote control by a remote controller has become common, there is no effective plan for the installation location of the remote controller in consideration of mounting on the vehicle, and the remote control is performed by vehicle vibration, lane change, or curve running. The controller moves carelessly.

Further, the conventional input device based on voice recognition has the following problems.

(A) The command word must be remembered, and if many operations are to be performed by voice, the command word will be forgotten or mistaken.

(B) Since the user is required to speak, some people like and dislike it.

(C) Communication of words is hindered during switch operation.

(D) Unreadable words cannot be pronounced.

As described above, the switch operation always involves visual work such as confirmation of the switch position and confirmation of the switch operation state.

Therefore, it is considered that if the information can be transmitted only by the line of sight, the intention of the switch operator can be transmitted to the control device without using finger operation or voice.

If the line of sight is used, a specific gaze area or a specific line of sight movement can be associated with a specific information input, and these can be confirmed on the input console. Solvable. Furthermore, if you use your line of sight, it will not hinder verbal communication,
The above problem can be solved.

However, the conventional line-of-sight input device has the following problems.

(A) Since a history of past measurements is used by using a kind of low-pass filter, it is difficult to deal with quick movements of the line of sight. The measurement result may be one step behind the actual eye movement.

(B) Since the gazing point is determined on a region-by-region basis, it is difficult to apply it to, for example, an application system in which a small switch panel is operated with the line of sight.

(C) In order to obtain the gazing point with high accuracy, it is inevitable to improve the accuracy of line-of-sight measurement. This is not a good idea as it will increase the cost of the system.

The present invention has been made by paying attention to the problems as described above. Even if the accuracy of the line-of-sight measurement is poor, it is possible to specify the position with high accuracy, and the position can be specified quickly and accurately. An object of the present invention is to provide an improved visual axis input device for consoles.

[0025]

In order to achieve the above-mentioned object, the invention of claim 1 displays the input position mark on the screen and sets the display position of the input position mark as the input position. In, an image input means for inputting an image of the operator's face area, a gaze position measuring means for measuring the gaze position of the operator from the image data by the image input means, and a gaze position measured by the gaze position measuring means A gaze area determining unit that determines an area to which the operator's line-of-sight direction belongs, and an input position mark movement control unit that moves the input position mark stepwise in the direction of the area determined by the gaze area determining unit There is something.

In the console line-of-sight input device according to the present invention, the input position mark such as the cursor pointer is step-moved, that is, inching (pitch moved) toward the region to which the operator's line-of-sight direction belongs, and the input position mark is required. It can be moved exactly to the position.

According to a second aspect of the present invention, in the console line-of-sight input device according to the first aspect, the step movement amount is changed to change the step movement amount of the input position mark in accordance with the gaze time of the same region of the operator. It has a means.

In the console line-of-sight input device according to the present invention, the amount of one step movement of the input position mark can be increased as the operator gazes at the same region longer.

The invention of claim 3 is the same as claim 1 or 2.
In the console line-of-sight input device described above, the input position mark movement control means, a step mode for moving the input position mark stepwise in the direction of the area determined by the gaze area determining means, and the gaze position measuring means. There is a quick mode in which the input position mark is moved to the measured gaze position all at once, and one of the modes is selected and set.

In the console visual line input device according to the present invention, the step mode in which the input position mark is stepwise moved to the side of the region to which the operator's line-of-sight direction belongs, and the conventional step position mode in which the input position mark is moved to the operator's gaze position at once. The quick mode equivalent to the one is used properly.

According to a fourth aspect of the present invention, in the visual line input device for a console according to the third aspect, there is an eyeball state detecting section for detecting the blink of the operator, and mode switching is detected by the eyeball state detecting section. The mode is switched according to the blink of the operator.

In the console visual line input device according to the present invention, the operator blinks to switch between the step mode and the quick mode.

According to a fifth aspect of the invention, in the console line-of-sight input apparatus according to the third aspect, there is gaze time measuring means for measuring that the gaze time at the same position has reached a predetermined value. The mode is switched by measuring that the gazing time at the same position has reached a predetermined value by the means.

In the console visual line input device according to the present invention, the operator switches between the step mode and the quick mode by gazing at the same position for a predetermined time.

According to a sixth aspect of the present invention, in the console line-of-sight input device according to any of the third to fifth aspects, the input for changing the shape and color of the input position mark in the step mode and the quick mode is performed. It has a position mark shape / color control means.

In the console line-of-sight input device according to the present invention, the shape and color of the input position mark are changed between the step mode and the quick mode. Whether the step position mode or the quick mode is selected depending on the shape and color of the input position mark. I understand.

[0037]

In the console visual line input device according to the present invention, since the input position mark is stepwise moved to the side of the region to which the visual line direction of the operator belongs, the rough visual line direction is measured even if the visual line measurement accuracy is poor. If possible, it is possible to move the input position mark cursor pointer to a predetermined position with high accuracy, which enables high-accuracy position specification and reduces the size of various markers on the console screen. The degree of freedom increases.

Further, if the moving direction of the line-of-sight and the moving direction of the input position mark are made to coincide with each other, the input position mark moves to the right one step when glancing at the right. You can

In the console line-of-sight input device according to the second aspect, the longer the operator looks at the same area, the larger the step movement amount of the input position mark can be made. Therefore, the input can be made according to the intention of the operator. The amount of one step movement of the position mark changes, so that the input position mark can be accurately moved to the required position and moved quickly with good operability.

In the console line-of-sight input apparatus according to the third aspect, the step mode in which the input position mark is stepwise moved to the side of the region to which the operator's line-of-sight direction belongs, and the conventional input position mark is moved all at once to the operator's gaze position. Since the quick mode equivalent to the one of the above is used separately, the input position mark is randomly accessed on the display screen to quickly move the input position mark to the vicinity of the target position in the quick mode, then switch to the step mode and accurately. The input position mark can be moved to the target position, and the time required for line-of-sight input can be shortened. In other words, high-speed and high-accuracy position input becomes possible.

In the console line-of-sight input device according to the fourth aspect, since the operator switches between the step mode and the quick mode by blinking, the mode switching can be performed in a non-contact manner without any key operation and without trouble. Be seen.

In the console line-of-sight input device according to the fifth aspect, since the operator switches between the step mode and the quick mode by gazing at the same position for a predetermined time, the mode switching also requires a key operation. Without contact and without hassle.

In the console line-of-sight input device according to the sixth aspect, since the shape and color of the input position mark are changed between the step mode and the quick mode, the shape of the input position mark,
Easier to use because you can see whether it is step mode or quick mode from the color.

[0044]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

1 and 2 show an embodiment in which the line-of-sight input device according to the present invention is applied to a destination input device in a vehicle-mounted navigation device.

FIG. 1 shows an on-vehicle state of the navigation device. The navigation device 10 is composed of a color liquid crystal display or the like, and has a panel-shaped screen display (console monitor) 20 for displaying a navigation map and the like, and a destination input start switch 30 installed near the screen display 20. And a calibration start switch 40.

An example of screen display by the screen display 20 is shown in FIG.
As shown in (a) and (b), in this navigation screen, a navigation map is displayed on the entire screen,
A pointer cursor A, which is an input position mark, and scroll markers B for up, down, left and right on this navigation map.
The detailed display marker C, the wide area display marker D, and the eyeball state figure E are displayed.

The destination input start switch 30 is a push button switch for instructing the start of destination input, and when the destination input start switch 30 is pressed, the navigation device 10
Enters destination input mode.

The calibration start switch 40 is a push button switch for instructing the start of calibration. When the calibration start switch 40 is pressed, the navigation device 10 enters the calibration mode.

Below the screen display 20, an image input section 100 including image pickup means such as a CCD camera for inputting an image of the operator's face area, and the image input section 100 are arranged so as to form a coaxial system. First illuminator 110 and first illuminator 1
A second illuminator 120 which is arranged apart from the image input unit 10 and forms a non-coaxial system with the image input unit 100 is provided. The first illuminator 110 and the second illuminator 120 are illuminators having the same optical specifications, and emit infrared LED light.

In FIG. 1, 50 is a stearing wheel and 60 is a front windshield glass.

FIG. 2 shows the overall structure of a vehicle-mounted navigation device to which the line-of-sight input device according to the present invention is applied.

This device includes an illumination light emission control section 130 for controlling the light emission operation of the first illuminator 110 and the second illuminator 120.
And an A / D converter 101 for converting an analog image signal input from the image input unit 100 into digital image data.
And the input digital image data (face image data)
In the image memory (frame memory) 102 for holding the gaze position, the gaze position measuring section 140 for measuring the gaze position of the operator from the face image data, the gaze area determining section 150, the eyeball state detecting section 160, and the screen display 20. A cursor display control unit 170 that controls the display of the pointer cursor A on the navigation screen, a navigation map control unit 180 that controls the map display on the screen display device 20 that controls the map display according to the result of the line-of-sight input, and an individual It has a calibration processing unit 190 that collects calibration data for reducing the influence of eyeball size and seating position deviation, and an overall control unit 200 that controls the entire apparatus.

The line-of-sight position measuring unit 140 is provided in the image memory 10
A pupil extraction unit 141 that extracts a retina reflection image (pupil) from the face image data stored in 2 and a corneal reflection image of the first illuminator 110 from a pupil vicinity position that is known from the pupil position extracted by the pupil extraction unit 141. Corneal reflection image extraction unit 1 for extraction
42, a focus determination unit 143 that determines and calculates the degree of focus on the corneal reflection image from the extraction result of the corneal reflection image by the corneal reflection image extraction unit 142, the pupil position and the corneal reflection image extracted by the pupil extraction unit 141. It includes a gaze position calculation unit 144 that calculates a gaze position based on the calibration data described below from the corneal reflection position extracted by the extraction unit 142.

The gaze area determining unit 150 determines the area to which the operator's gaze direction belongs based on the gaze position calculated by the gaze position calculating unit 144. The gaze area is defined in nine areas, that is, a central area, an immediately upper area, an upper right area, a right area, a lower right area, a right lower area, a lower left area, a left area and an upper left area.

The eyeball state detecting section 160 includes a pupil extracting section 14
The position of the eyeball in the camera field of view and the degree of focus on the eyeball are calculated from the pupil extraction result by 1 and the focus determination result by the focus determination unit 143 to detect a characteristic eyeball state such as a long blink. .

The cursor display control section 170 has a quick mode in which the cursor pointer is moved all at once to a gaze position in accordance with a characteristic eyeball state such as a long blink detected by the eyeball state detecting section 160, and a direction of the line of sight of the cursor pointer. The cursor movement mode control unit 171 that switches the cursor movement mode (gaze input mode) to the step mode in which the cursor is moved by a predetermined number of steps, and on the navigation image according to the gaze position calculated by the gaze position calculation unit 144 and the cursor movement mode. A cursor movement control unit 172 that controls the movement of the pointer cursor A, and a cursor display shape / color control unit 173 that controls the shape and color of the pointer cursor A displayed on the navigation image according to the cursor movement mode mode. ing.

FIG. 3A shows a screen display example in the quick mode, and FIG. 3B shows a screen display example in the step mode. In this example, the pointer cursor A has a rectangular shape in the quick mode and an elliptical shape in the step mode. After that, move the square pointer cursor to A
a, an elliptic pointer cursor may be distinguished from Ab.

In the quick mode, the cursor movement control unit 172 controls the cursor cursor A on the navigation image to move to the gaze position calculated by the gaze position calculation unit 144 at once, and in the step mode, the gaze area determination unit 150 determines. Then, control is performed to stepwise move to the region to which the operator's line-of-sight direction belongs by a predetermined amount.

As shown in FIG. 4, the calibration processing unit 190 uses the calibration target display control unit 191 for drawing the calibration target F at a specific position on the screen of the screen indicator 20, and the calibration result. It has a calibration data calculation unit 192 that calculates a conversion coefficient necessary for calculating the gaze position and a calibration data storage unit 193 that stores the calibration data.

Next, the operation of the visual line input device according to the present invention will be described.

Calibration work is required in advance to measure the line-of-sight direction, and the calibration work is performed as the calibration work prior to the destination input work. The procedure of the calibration work will be described below.

[Calibration work] (1) The operator starts the calibration work by pressing the calibration start switch 40 installed near the screen display 20. During calibration, as shown in FIG. 4A, the optotype control unit 191 sequentially presents the calibration optotypes F one by one on the navigation display screen. The operator continues to gaze at the calibration target F during that time.

(2) Face image data including the operator's eyeball is input from the image input unit 100, and the face image data is converted into digital data by the A / D converter 101 and stored in the image memory 102. It At this time, an image a when the first illuminator 110 is turned on and the second illuminator 120 is turned off and an image b when the first illuminator 110 is turned off and the second illuminator 120 is turned on are almost the same. Enter at the same time.

(3) Next, the pupil extraction unit 141 calculates the difference between the image a and the image b. In the image a, the pupil is brightly observed by the reflected light from the retina, and in the image b, the pupil is observed dark. Therefore, the difference between the two is that the pupil region is emphasized. The difference image c is binarized with a fixed threshold value and further subjected to labeling processing,
Number the areas.

In the difference image c, in addition to the retinal reflection image,
For example, various noises such as a spectacle lens reflection image, a spectacle frame reflection image, and a part of a face caused by a change in external illumination may be included.

These noises generally have an indefinite shape and an indefinite area, and can be discriminated from a retina reflection image observed as a circle or an ellipse having an area expected in advance. Here, identification based on the area and shape of the region is performed.

As a result of the labeling, the area Ri of each region obtained is set to a predetermined threshold value S1, S2 (S1 <
Compared with S2), only the region satisfying S1 <Ri <S2 is extracted. Here, S1 and S2 are predicted pupil diameters (diameter 2 to 8 mm) estimated from the photographing magnification of the CCD camera.
(The number of pixels in the pupil is determined as how many pixels are observed as the area).

Although the spectacle lens reflection image is also observed as a circular area, if the area of the spectacle lens reflection image is made smaller than the area of the retinal reflection image by, for example, narrowing the diaphragm of the lens, both areas will be different depending on the area. Can be identified.

Then, the ratio F of the area of the remaining area to the circumscribed rectangle is calculated. Since the retinal reflection image is observed in a circular or elliptical shape, the ratio F is equal to or greater than a certain value Fth. On the other hand, for example, spectacle frame reflection is an elongated region along the frame, and thus is temporarily equivalent to the retinal reflection image. Even if it has the area of F, F becomes small and can be identified.

As a result of the above discrimination, the remaining area is determined as a retina reflection image, and the barycentric position of this retina reflection image, that is, the pupil barycentric coordinate (Xg, Yg) is obtained.

If the corresponding area cannot be found,
The operator determines that his eyes are closed.

The flow of the series of processes (1) and (2) above is summarized in FIG.

(4) Next, the corneal reflection image extraction unit 142
, The coordinate of the center of gravity of the pupil in the difference image c (Xg, Y
A small area including the retinal reflection image is defined around g), and a point having the maximum brightness in this small area is set as a corneal reflection image, and corneal reflection image coordinates (Px, Py) are obtained.

(5) Next, in the focus determination unit 143, the brightness value I of the corneal reflection image is obtained, and each pixel in the small area for searching the corneal reflection image is α · I (0 <α <1). Binarize. If the pixel value of the difference image c is larger than α · I, set 1 to
If it is smaller, 0 is given, and the number K of pixels brighter than α · I obtained as a result of binarization is calculated.

If the difference image c is a focused image, the brightness peak of the corneal reflection image becomes sharp,
The number of pixels K has a small value. On the other hand, if the difference image c is an out-of-focus image, the luminance peak of the corneal reflection image becomes gentle and the number of pixels K becomes a large value.
If the number of pixels K is larger than a predetermined value,
It is determined that the input image is an out-of-focus image.

(6) If an image focused on the eyeball is obtained, the calibration data calculation unit 192 causes the coordinates of the center of gravity of the pupil (Xg, Yg) and the coordinates of the corneal reflection image (Px, Py) on the image.
From the two line-of-sight parameters δx = Xg−Px, δy =
Calculate Yg-Py.

(7) In the eyeball state detecting section 160, the pupil barycentric coordinates (Xg,
Yg) and the determination output of the focus determination unit 143 create a display pattern indicating whether or not the corneal reflection image is in focus, and display it on the screen display 2.

An example of the display pattern is shown in FIG. FIG.
In (a), the position of the calibration target F is regarded as the position of the center of gravity of the retina reflection image, a moving frame G is provided outside the position, and the position of the eyeball in the visual field is adjusted by the calibration target position in the moving frame G. Is represented. Further, when the corneal reflection image is in focus, the calibration target F is displayed in a filled pattern as shown in the figure, and when the corneal reflection image is not in focus, it is displayed in a blank pattern.

From this display pattern, the operator himself can confirm the frame-out of the eyeball and the out-of-focus condition, and it becomes easy to move the head position and hold it at a fixed position according to the situation.

(8) The steps (1) to (7) are repeated for the calibration target F at the same position until calibration data is obtained N times. As a result, the obtained N line-of-sight parameters δx, δ
y is averaged to obtain calibration data δxi and δyi as calibration results for the calibration target F, and the calibration data δxi and δyi are stored in the calibration data storage unit 193.

(9) All prepared calibration targets F
(In this embodiment, as shown in FIG.
Points (vertical direction, 5 points in the vertical direction), when the loop of (1) to (7) ends, for each group of calibration data δx calculated in the horizontal direction and each group of calibration data δy calculated in the vertical direction. , The conversion equation of the target position and the line-of-sight parameter δx or δy is determined by the least squares method.

Generally, in a region where the line-of-sight declination is small, the line-of-sight parameters δx, δy and the target position are linear expressions y.
= Ax + b, the coefficients a and b are determined.

In FIG. 4 (b), the numbers indicate the order in which the calibration target F is displayed on the screen.

(10) From the above calibration results, the predicted value in each direction, which is expected to be obtained when the operator slightly looks away from the screen of the screen display 20, is calculated from δxi and δyi. It is assumed that the coefficients a and b calculated above can be applied to a slightly deflected area, and the predicted value when slightly deflected to the right is δRight = δx5 − | δx1 −δx5 | / 4 The predicted value when the eyes are slightly deflected to the left is δLeft = δx1 + | δx1−δx5 | / 4 It is assumed that the predicted value δLower = δy10 + │δy10 -δy6 │ / 4 when the eyes are turned away (note that the symbol ││ represents an absolute value).

The calibration work is completed by the above processing.
This calibration work does not have to be performed every time and may be performed once. If the calibration data thus obtained is stored in the memory, it can be reused later.

If it is necessary to input a destination for navigation, a destination input operation is performed. The procedure for inputting a destination will be described below with reference to FIG.

[Destination input work] (1) When the destination input start switch 120 is pressed, the display screen of the screen display 20 becomes the navigation as shown in FIG. It is ready to accept input.

(2) When the destination input start switch 120 is pressed, thereafter, the same processing as (2) to (7) in the above [calibration work] is continuously performed, and the center of gravity of the pupil (Xg , Yg) and the corneal reflection image coordinates (Px, Py), two line-of-sight parameters δx = Xg−Px, δy = Y
g-Py and the degree of focusing on the corneal reflection image are calculated.

(3) The line-of-sight parameters δx and δy calculated in (2) are (9) in [Calibration work] each time.
3 is converted into the position coordinates on the vignetting screen of the screen display device 20 by the conversion formula already obtained, and a pointer cursor Aa (gaze area display) and eyeball state as shown in FIG. Figure E is superimposed.

(4) The destination to be set is the screen display 2
When the operator does not appear on the navigation screen 0, the operator looks at the wide area display marker D set at the lower right of the navigation screen, and the pointer cursor Aa moves to the position shown in FIG.
As shown in (a), it moves to the vicinity of the wide area display marker D. Since the accuracy of the line-of-sight measurement device is limited, the pointer cursor Aa is not always the wide area display marker D.
Do not always overlap.

(5) When the pointer cursor Aa is moved to the vicinity of the wide area display marker D and the eyes are closed for a while,
For example, when the eye is closed for 300 msec, the eyeball state detection unit 1
At 60, a closed eye (closed eyes) is detected because the retina reflection image does not exist, and based on this, the cursor movement mode of the cursor movement control unit 172 is instructed by the cursor movement mode control unit 171. Switch from step mode to step mode.

A blink of 300 msec is 3 when the calculation of the gaze position requires 100 msec.
The determination may be made based on the fact that no retinal reflection image was found consecutively. Further, since a normal blink is completed in about 100 msec, there is little risk that mode switching will occur due to careless blinking.

(6) When the cursor movement mode is switched, the cursor display shape / color control unit 173 operates according to the signal output from the cursor movement mode control unit 171, to notify the operator that the cursor movement mode has been switched. , The shape of the pointer cursor A is changed to an ellipse (see FIG. 3B). The same effect can be obtained even if the color of the pointer cursor A is changed.

(7) Subsequently, the pupil and corneal reflection images are extracted by executing the same processing as (2) to (7) in [Calibration work], and the line-of-sight parameters δx and δy are calculated.

Furthermore, in the gaze area determining section 150,
δRight, δLeft, δUpper, δLower are compared with δx, δy to determine which of the regions shown in FIG. 6 the operator is gazing at.

Which region in FIG. 6 is being watched is evaluated by the following judgment formula.

At the time of gazing at the central region CN δRight <δx <δLeft and δUpper <δy <δLo
wer At the time of watching the region UP just above δRight <δx <δLeft and δy ≦ δUpper At the time of watching the upper right region RU δx ≦ δRight and δy ≦ δUpper At the time of watching the right region RR δx ≦ δRight and δUpper <δy <δLower of the lower right region RU At gazing δx ≤ δRight and δLower ≤ δy At the time of gazing directly below the region DW δRight <δx <δLeft and at the time of gazing at δLower ≤ δy lower right region δLeft <δx and δLower ≤ δy At the time of gazing at the region LL δLeft ≤ δx <and <x δy <δLower At the time of gazing the upper left area LU δLeft ≦ δx and δy ≦ δUpper (8) By the operation of the cursor movement control unit 172, the display position of the pointer cursor Ab is a predetermined amount in the direction of the gazing area calculated in (7), For example, the pointer cursor Ab is moved one step by one dot, and the cursor display shape / color control unit 173 redraws the pointer cursor Ab on the navigation screen of the screen display 20. The movement of the pointer cursor Ab is performed based on the state transition diagram of FIG.

In FIG. 7, (x, y) is the previous pointer cursor position, (x ', y') is the new pointer cursor position, Δx is the unit movement step amount in the x direction, and Δy is CN, UP, RU, RR, RD, DW, L, which shows the unit movement step amount in the y direction and has an arrow
D, LL, and LU indicate the areas that are focused on.

The operator confirms the result of the movement of the pointer cursor Ab, and again repeats the action of gently shaking his eyes or head in the direction in which he wants to move the pointer cursor Ab.

Then, (7) and (8) are continuously executed, and the pointer cursor A gradually approaches the target position, in this case, the wide area display marker D.

(9) As shown in FIG. 8B, the pointer cursor A overlaps the wide area display marker D,
When the eyes are closed for about 300 msec, the closed eyes are detected as in (5), and the detection of the closed eyes determines that the wide area display marker D has been selected.

Then, the navigation map control unit 180
By this, the scale of the map is changed by one step and the map is displayed again. At the same time, the cursor movement mode is set to the step mode.

(10) When there is no destination on the map screen after being redisplayed, (4) to (9) are executed again. Generally, when the steps (4) to (9) are performed a plurality of times, the destination can be displayed on the screen.

(11) When the destination is displayed on the screen or the vicinity of the destination is displayed, the vicinity of the scroll marker B set on the four sides of the navigation screen of the screen display device 20 is watched for a while (( (About 300 msec) Close your eyes and change the cursor movement mode.
, The cursor pointer Ab is finely moved to be superimposed on the scroll marker B, and again (30
The eyes are closed and the screen scroll is selected and confirmed, and the navigation map control unit 180 scrolls the screen.

After scrolling for one step, the cursor movement mode is automatically switched to the quick mode,
Switches the display of the pointer cursor A.

(12) Perform (11) a plurality of times so that the vicinity of the destination is displayed in the center of the screen.

(13) When the vicinity of the destination is displayed in the center of the screen, the detailed display marker C is selected and confirmed and the screen is enlarged and displayed, as shown in FIG. 8 (d). The selection confirmation of the detailed display marker C is performed by the wide area display marker D described above.
The selection may be performed similarly to (13) is repeated multiple times so that the vicinity of the true destination is displayed in detail on the screen.

(14) When the true destination is displayed in detail on the screen, the operator looks at the destination to be set. As a result, the pointer cursor Aa moves at once to the vicinity of the destination, which is the gaze position, in the quick mode.

Thereafter, the cursor movement mode is switched to the step mode by closing the eyes for about 300 msec, and the pointer cursor Ab is moved to the step mode as shown in FIG.
Step to a predetermined position to match the destination.

When the pointer cursor Ab coincides with the destination, the position is fixed by closing the eyes for about 300 msec.

When the destination is determined as described above, the navigation map control unit 180 calculates the optimum route from the current position to the destination and sets it on the map.

The navigation screen of the screen display device 20 returns to the screen as shown in FIG. 3A, and the navigation toward the destination is started.

Next, another embodiment of the visual axis input device for console according to the present invention will be described with reference to FIGS.

In the embodiment shown in FIG. 9, a step movement amount changing section 210 is provided. The step movement amount changing unit 210 takes in the determination result of the gaze area from the gaze area determining unit 150 and determines that the same area is continuously gazed during the sampling of the face image a plurality of times, in other words, If the gazing time of the same area reaches a predetermined value, the step movement amount (Δx, Δy) of the pointer cursor Ab per one time is increased.

Thus, in the above-mentioned [destination input work], even if the mode is changed to the step mode at a position apart from the input target position, the pointer cursor Ab can be quickly moved to the target position.

As a matter of course, the step movement amount of the pointer cursor Ab per time may be gradually increased according to the gaze time of the same area.

In the embodiment shown in FIG. 10, a gaze time measuring section 220 is provided. The gaze time measurement unit 220 takes in the calculation result of the gaze position from the gaze position calculation unit 150, measures that the gaze time at the same position has reached a predetermined value, and the cursor movement mode control unit 17
Notification to 1.

Upon receiving the notification from the gaze time measuring unit 220, the cursor moving mode control unit 171 switches the cursor moving mode instead of blinking.

Thus, in this embodiment, the operator switches between the step mode and the quick mode by gazing at the same position for a predetermined time.

As described above, the destination input of the vehicle-mounted navigation device is taken as an example of the embodiment of the present invention. However, the console line-of-sight input device according to the present invention is not limited to this. The present invention can also be applied as input means in CAD in computers, word processors, file operations, and the like.

[Brief description of the drawings]

FIG. 1 is an explanatory view showing a vehicle-mounted state of a vehicle-mounted navigation device to which a console line-of-sight input device according to the present invention is applied.

FIG. 2 is a block diagram showing an overall configuration of a vehicle-mounted navigation device to which the console line-of-sight input device according to the present invention is applied.

3A and 3B are explanatory diagrams showing screen display examples in a quick mode and a step mode in an in-vehicle navigation device to which the console line-of-sight input device according to the present invention is applied.

4 (a) and 4 (b) are explanatory views showing a screen display example at the time of calibration work in the vehicle-mounted navigation device to which the console line-of-sight input device according to the present invention is applied.

FIG. 5 is an explanatory diagram showing a process of extracting a retina reflection image in the console visual line input device according to the present invention.

FIG. 6 is an explanatory diagram showing an example of region setting for cursor step movement in the console line-of-sight input device according to the present invention.

FIG. 7 is a state transition diagram in the moving direction of the pointer cursor in the console visual line input device according to the present invention.

FIGS. 8A to 8E are explanatory views showing transitions of screen display in a destination setting process in a vehicle-mounted navigation device to which the console line-of-sight input device according to the present invention is applied.

FIG. 9 is a block diagram showing another embodiment of the vehicle-mounted navigation device to which the console line-of-sight input device according to the present invention is applied.

FIG. 10 is a block diagram showing another embodiment of the vehicle-mounted navigation device to which the console line-of-sight input device according to the present invention is applied.

[Explanation of symbols]

 10 navigation device 20 screen display 30 destination input start switch 40 calibration start switch 100 image input unit 110 first illuminator 120 second illuminator 130 illumination emission control unit 140 gaze position measurement unit 150 gaze area determination unit 160 eyeball state detection Part 170 Cursor display control part 180 Navigation map control part 190 Calibration processing part 200 Overall control part 210 Step movement amount changing part 220 Gaze time measuring part

Claims (6)

[Claims] 1. A console line-of-sight input device in which an input position mark is displayed on a screen and the display position of the input position mark is the input position, and an image input unit for inputting an image of an operator's facial region; and the image input unit. Gaze position measuring means for measuring the gaze position of the operator from the image data by the gaze area, gaze area determining means for determining an area to which the gaze direction of the operator belongs from the gaze position measured by the gaze position measuring means, and the gaze area A line-of-sight input device for a console, comprising: input position mark movement control means for stepwise moving the input position mark in the direction of the area determined by the determination means. 2. The console line-of-sight input apparatus according to claim 1, further comprising step movement amount changing means for changing the step movement amount of the input position mark in accordance with the gaze time of the same region of the operator. A console line-of-sight input device characterized in that 3. The console line-of-sight input device according to claim 1, wherein the input position mark movement control means moves the input position mark stepwise in the direction of the area determined by the gaze area determination means. Line-of-sight input for console, characterized by having a step mode and a quick mode in which the input position mark is moved to the gaze position measured by the gaze position measuring means all at once, and one of the modes is selected and set. apparatus. 4. The console line-of-sight input device according to claim 3, further comprising an eyeball state detection unit that detects an operator's blink, and the mode is changed by the operator's blink detected by the eyeball state detection unit. A line-of-sight input device for a console, which can be switched. 5. The console line-of-sight input apparatus according to claim 3, further comprising a gaze time measuring unit that measures that the gaze time at the same position has reached a predetermined value, and the gaze time measuring unit measures the gaze time at the same position. A line-of-sight input device for a console, wherein the mode is switched by measuring that the time has reached a predetermined value. 6. The console line-of-sight input device according to claim 3, wherein an input position mark shape and color control for changing the shape and color of the input position mark in the step mode and the quick mode. A line-of-sight input device for a console having means.