JPH06194564A - Equipment controlled by line of sight - Google Patents

Abstract

PURPOSE:To accomplish a sure and effective intension input without forcing a user to perform a complicated operation and without the possibility of malfunction. CONSTITUTION:The equipment is provided with a line of sight detecting means for detecting the user's line of sight and outputting information on the line of sight, display means 13-20 provided with one or plural display marks which respectively correspond to different operations of the equipment, a single input information recognizing means 12 and a control means for generating a control signal for controlling the operation of the equipment when the information on line of sight showing that the user closely observes specified one or plural display marks of the display means 13-20 is outputted by the line of sight detecting means and also a signal from the single information recognizing means 12 shows a prescribed state, and then, the operation to be controlled is selected among different operations by the line of sight detecting means, and the definition (generation of a control signal) is performed by the single information recognizing means 12.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of a visual axis control device equipped with visual axis detecting means for detecting the visual axis of a user and outputting visual axis information.

[0002]

2. Description of the Related Art Conventionally, various proposals have been made for detecting the line of sight. For example, the applicant of the present application discloses that
No. 241511, JP-A-1-274736, JP-A-2-264632, JP-A-4-138432 and the like are disclosed. Further, a technique that enables various controls of the device based on the detected line-of-sight information is also disclosed in Japanese Patent Laid-Open No. 2-323.
No. 12, JP-A-3-87818, JP-A-3-1090.
No. 30, U.S. Pat. No. 4,595,990 and the like.

Hereinafter, these conventional examples will be outlined.

Although various proposals have been made for detecting the line of sight as described above, photoelectric non-contact measurement has the least burden on the subject and is suitable for use in general equipment. Therefore, most of this kind.

For example, the applicant of the present application irradiates the eyeball of a user with infrared rays with an iRED (infrared light emitting diode), forms an image of the anterior segment image of the user, receives the image, and outputs the iRED from the output signal of the sensor. We have developed and put into practical use a technique for detecting the direction of the line of sight by estimating the corneal reflection image and the position of the pupil circle.

The line-of-sight detection will be briefly described below to the extent necessary for understanding the present invention described below.

FIG. 10 is an optical layout diagram showing an example of the case where the visual axis detecting means is incorporated in the single-lens reflex camera.

In FIG. 10, 101 is a taking lens and 1 is a taking lens.
Reference numeral 02 is a quick return mirror for guiding a part of the light flux passing through the taking lens 101 to a finder system (observation system), 103 is a display element, 104 is a focusing plate, 105 is a condenser lens, 106 is a penta roof prism, 107
Is a sub-mirror for guiding the light flux that has passed through the quick return mirror to the focus detection system, 108 is a multi-point focus detection device, and 109 is a camera for controlling the drive of the display element 103, focus detection calculation, lens drive, etc. Control circuit.

In the above structure, for example, the display element 103 and the multi-point focus detection device 108 are controlled (provided for line-of-sight control) based on line-of-sight information obtained by the line-of-sight detecting means described later. .

That is, the display element 103 is, for example, a two-layer type guest-host type liquid crystal element that does not use a polarizing plate, and displays a range-finding area (focus detection position) in the finder field. , One of a plurality of distance measurement areas on the display element 103 is selectively displayed based on the line-of-sight information, and among various operation mode displays on the display element 103, the operation mode to be used is the same as the line-of-sight information. Is selected and displayed.

Further, the multi-point focus detection device 108 detects the focus state based on the information of the selected distance measurement area among the plurality of distance measurement areas on the photographing screen. The selection of, for example, one distance measuring area in accordance with the photographer's intention from the area is performed using the line-of-sight information.

Next, an example of the line-of-sight detecting means will be described with reference to FIGS.

In FIG. 10, 110 is an eyepiece lens,
A dichroic mirror 110a that transmits visible light and reflects infrared light is installed obliquely inside the same and also serves as an optical path splitter. 111 is a light receiving lens. 113a, 113b
(Not shown in FIG. 10), 113c is, for example, a light emitting diode which is a lighting means. Reference numeral 114 denotes an image sensor having a configuration in which photoelectric element arrays are two-dimensionally arranged, and is arranged so as to be conjugate with the vicinity of the pupil of the eyeball E at a predetermined position with respect to the light receiving lens 111 and the eyepiece lens 110. Reference numeral 115 denotes a line-of-sight calculation processing circuit, which has a line-of-sight correction calculation, line-of-sight correction data storage, line-of-sight calculation function, and control functions of the infrared light emitting diodes 113a, 113b, 113c.

The above-described eyepiece lens 110 to the line-of-sight calculation processing circuit 115 constitute the line-of-sight detecting means for the eyeball E.

FIG. 11 is a perspective view showing a main part of the line-of-sight detecting means shown in FIG.

Infrared light emitting diodes 113a, 1 for illumination
The reference numerals 13b and 113c are used in pairs to detect the distance between the camera and the eyeball E of the observer. Depending on the posture of the camera, that is, when the camera is in the lateral position, the infrared light emitting diode 113a, In the case where the 113b is a set and is in the vertical position, the infrared light emitting diodes 113b and 113c are used as a set, respectively.

Although the means for detecting the attitude of the camera is not shown in FIG. 11, an attitude detecting means using a mercury switch or the like is effective.

FIG. 12 is for explaining the principle of visual axis detection and the output intensity distribution from the image sensor 114. More specifically, FIG. 12A is an explanatory view of the principle of visual axis detection. b) is the image sensor 1 of FIG.
It is explanatory drawing of the output intensity distribution from FIG.

Each infrared light emitting diode 113a, 113
b and 113c are arranged substantially symmetrically in the Z direction, and divergently illuminate the line of sight of the photographer. The infrared light emitting diode 11
The operation when using a pair of 3a and 113b is as follows.

The infrared light projected from the infrared light emitting diode 113b illuminates the cornea 116 of the eyeball E. At this time, the corneal reflection image d by a part of the infrared light reflected on the surface of the cornea 116 is condensed by the light receiving lens 111 and is re-imaged at the position d ′ on the image sensor 114.

Similarly, the infrared light projected from the infrared light emitting diode 113a illuminates the cornea 116 of the eyeball E. At this time, the cornea reflection image e by a part of the infrared light reflected on the surface of the cornea 116 is condensed by the light receiving lens 111 and is re-imaged at the position e ′ on the image sensor 114.

The light fluxes from the ends a and b of the iris 117 form the images of the ends a and b at the positions a ′ and b ′ on the image sensor 114 via the light receiving lens 111. The rotation angle θ of the optical axis a of the eyeball E with respect to the optical axis a of the light receiving lens 111.
Is small, the Z coordinate of the ends a and b of the iris 117 is Z
Letting a and Zb be the coordinates Zc of the central position c of the pupil 118.
Is expressed as Zc≈ (Za + Zb) / 2.

Further, since the Z coordinate of the midpoint of the corneal reflection images d and e and the Z coordinate Zo of the center of curvature o of the cornea 116 match, the Z coordinate of the generation position of the corneal reflection images d and e is Zd, Z.
e, a standard distance to the center c of the curvature center o the pupil 24 of the cornea 116 and L _oc, when the considered coefficient individual differences with respect to the distance L _oc and A1, the rotation angle of the optical axis v of the eyeball E theta
Is substantially satisfy a relational expression _{(A1 * L oc) * sinθ} ≒ Zc- (Zd + Ze) / 2 ......... (1). Therefore, the line-of-sight calculation processing circuit 1
15, the image sensor 114 as shown in FIG.
The rotation angle θ of the optical axis a of the eyeball E can be obtained by detecting the positions of the respective feature points (corneal reflection images d and e and the ends a and b of the iris 117) projected on the upper part. it can.
In this case equation _{(1), β (A1 * L oc)} * sinθ ≒ (Za' + Zb') / 2 - rewritten as (Zd' + Ze') / 2 ......... (2). However, β is a magnification determined by the position of the eyeball E with respect to the light receiving lens 111, and is substantially obtained as a function of the interval | Zd′−Ze ′ | of corneal reflection images.

Furthermore, Za ', Zb', Zd ', Ze'
Also, since it is a measured amount that can be calculated and extracted from the output of the image sensor 114, the eyeball rotation angle θ can be obtained using the above equation (2). The calculation for converting the eyeball rotation angle θ to the focused position on the focus can be easily performed.

The coefficient A1 representing the individual difference is fixed together with the eyeball optical axis deviation correction amount B1 which is not described here, and the photographer fixes the visual target arranged at a predetermined position in the viewfinder of the camera. It is obtained by matching the position of the target with the position of the fixation point calculated based on the above equation (2).

In the above description, the eyeball of the photographer is Z-.
Although an example in which the photographer rotates in the X plane (for example, horizontal plane) is shown, the same can be detected when the eyeball of the photographer rotates in the XY plane (for example, vertical plane).

When the control of the device is executed from the visual line information detected by the visual line detecting means as described above, various measures are required.

In general, the visual line input cannot be expected to be as accurate as a manual switch. The human eye moves fairly well, and does not always have a clear intention to operate the device. In addition, human beings are unfamiliar with the action of intentionally keeping an eye on one point, and an input system based on such a forced action causes fatigue of the user and causes a malfunction. There is also a problem with the accuracy of the line-of-sight direction detected by the photoelectric method.

Due to such a technical background, satisfying both efficient input and reliable input at the same time has always been an important issue for the line-of-sight input technique and the device control technique based on it.

The applicant of the present application discloses, in Japanese Patent Laid-Open No. 3-109030, one methodology for determining a point of interest.

That is, the direction of the line of sight of the user is repeatedly detected, and the position where the line of sight has stayed the longest within a predetermined time is set as the gazing point. Alternatively, the position with the highest frequency in the repeated detection a predetermined number of times is set as the gazing point. Alternatively, from the viewpoint that the number of gaze points is not always one, the idea is to set all gaze points at all positions where a stop for a certain time or more is recognized within a predetermined time.

Further, in view of using the line-of-sight information at the moment when another operation member (for example, a shutter button) is operated instead of the statistical method as described above, Japanese Patent Application Laid-Open No. 2-32312 by the applicant of the present application discloses. Can be mentioned.

This method can be said to be very good when the operation of the operation member and the movement of the line of sight have a high degree of inevitability, but otherwise it is an unnatural input operation and is troublesome. In addition, a problem remains when the information entered through the line of sight is different from the intention.

As another way of thinking, there is a way to make sure that confirmation is made after inputting with the line of sight. For example, according to U.S. Pat. No. 4,595,990, a selected display device in which numbers 1 to 10 are arranged has a cancel, veri
It is disclosed that an fy display is added. When one of 1 to 10 is selected by the line of sight in the first step, the display area of the selected number is lit up, and in the second step, it is confirmed by the line of sight whether or not it is the user's intention. Or, it is decided by verify input.

However, such an input form is very complicated and cannot be said to be appropriate as an intention input means for general equipment.

[0036]

In these known line-of-sight input devices and line-of-sight control devices, the line-of-sight detection technology is incomplete, and reliable operation itself has been a major technical goal. Therefore, the viewpoint of improving the input / control procedure and making it truly easy to use is lacking. In order to make the gaze input control a truly versatile and valuable technology, it is essential to make sure the intention input action, as well as the ease of input operation and high efficiency.

(Object of the Invention) An object of the present invention is to provide a line-of-sight control device capable of performing reliable and highly efficient intention input without forcing a user to perform a complicated operation and without fear of a malfunction. That is.

[0038]

SUMMARY OF THE INVENTION The present invention has a line-of-sight detection means for detecting the line-of-sight of a user and outputting line-of-sight information, and one or a plurality of display symbols, each of which has a different operation of a device. Corresponding display means, a single input information confirmation means, and a line-of-sight indicating that the line-of-sight detection means indicates that the user is gazing at one or more specific display symbols of the display means. When the information is output and the signal from the single information confirmation means indicates a predetermined state, a control means for generating a control signal for controlling the operation of the equipment is provided, and the line-of-sight detection means is provided. An operation is selected to be controlled from among different operations, and its confirmation (control signal generation) is performed by a single information confirmation means.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on the illustrated embodiments.

FIG. 1 is a view showing the display in the viewfinder of a single-lens reflex camera equipped with the line-of-sight control device of the first embodiment of the present invention.

In FIG. 1, around the finder screen 11, various display parts for inputting the line of sight are arranged. When the user looks at any one of the display parts, the camera operation corresponding to the displayed contents can be performed. It is supposed to be selected.

The display unit 12 is for activating eye-gaze input, the display units 13 to 15 are of a type having a plurality of selection techniques in association with one type of camera operation, and the display units 16 to 20.
Is a type that decides ON or OFF for a certain camera operation.

For example, the display units 13a and 13b are related to the automatic exposure mode (AE mode), and 13a is a display unit for selection of the aperture priority AE mode and 13b is the shutter speed priority AE mode. Further, 13c is a display unit in the manual exposure control mode. These three displays 13
Among a, 13b, and 13c, only the one showing the current state has a higher luminance than the others, and the display color can be distinguished from each other.

In order to select the desired AE mode to be changed from the above, the following may be done. That is, the display unit 12
When the user gazes at the line of sight, the line-of-sight input is activated, and in order to indicate that, the display unit 12 changes in brightness, color, or blinks. That is, the display form is different from that in the normal state.

Next, when the user looks at a desired one of the display portions 13a, 13b, 13c, the display form is changed in a highlighted manner to show that the mode to be displayed is selected.
If the selection is different from the previous state, the display section held until then becomes the normal non-selected display state. In this state, the line of sight is displayed on these display portions 13a, 13
If it is separated from b and 13c, the selection is confirmed, and the display unit 12 for activating the line-of-sight input returns to the standby state. Alternatively, the selection may be confirmed by continuing to look for a predetermined time. Furthermore, it is also possible to set the confirmation condition to look at the display unit 12 again. On the other hand, if you want to change your selection, you can continue to look at your new desired display. In order to make such an operation smooth, it is desirable to add a stop of the line of sight for a predetermined time as a selection condition. This is because, in order to search for a desired one among a plurality of selection techniques, the line of sight often passes over an unnecessary display unit in many cases.

By adopting the method of activating the line-of-sight input by gazing at the display unit 12 as described above, the line-of-sight can be passed irrespective of the intention, or the display unit can be seen only for confirmation, and for input. It becomes possible to distinguish the case of looking at the display unit, and a correct line-of-sight input system without malfunction can be obtained.

Steps 101 to 111 in FIG. 2 are a flow chart showing an example of the operation for selecting the above-mentioned AE mode, which is performed by the control means (not shown) of the camera having the above-mentioned configuration.

Further, in FIG. 1, the display units 14a, 14
Reference numeral b is for switching the photometric distribution, 14a is a display unit for selecting average photometry, and 14b is a display unit for selecting spot photometry. Further, the display units 15a and 15b are related to AF operation mode switching, 15a is a one-shot mode for shooting a still subject, and 15b is a servo mode selection for shooting a moving object.

These selection operations are performed in the same manner as in FIG. 2 above.

The display units 16 to 20 relate to ON / OFF control of a certain operation as described above.

For example, assuming that the camera embodying the present invention is provided with an auto-zoom function for automatically setting an appropriate magnification, the display unit 16 displays the O-zoom function of the auto-zoom function.
It also serves as an N / OFF state display and a switch. As for the display form, the current state is represented by brightness, color, etc., as in the previous case.

In order to reverse the ON / OFF state of the auto-zoom function, first, the display unit 12 is gazed at to activate the line-of-sight input. At this time, since the display unit 12 displays that the standby state has entered the activated state, the display form is in a state different from the normal state. Next, when the user looks at the display unit 16, the display is switched to a display showing the changed state corresponding to the inversion of the ON and OFF states. That is, the operating state of the auto zoom function is fixed. More specifically, the auto zoom start signal or the stop signal is output by the control means (not shown).

Here, if the line of sight is turned to another, the change is confirmed. Alternatively, the sequence may be determined by continuing to look at the display for a predetermined time, or may be determined by looking at the display unit 12 again.

The display unit 17 relates to the activation of the self-timer, and the display unit 18 relates to the operation of retracting the lens for confirming the depth of field.

Since it is usually desirable to release the self-timer by one release operation, even if the self-timer is set by the line-of-sight input, it is automatically reset after the release. Of course, it is desirable that the line-of-sight input operation can be reset by the line-of-sight by an ON / OFF inversion method.

Since the lens retraction operation for confirming the depth of field is an operation that may be repeatedly used for one composition setting, it is troublesome to activate the line-of-sight input on the display unit 12 each time. . For such features,
For example, the following measures should be taken.

The visual axis input activation state by gazing at the display unit 12 is not reset to the standby state even after the occurrence of the lens retraction operation if the input display gazed immediately after that is the display unit 18. The narrowing-down operation is controlled by a predetermined timer, and returns to the released state after a predetermined time has elapsed or by looking at the display unit 18 again. When the user wants to perform the retracting operation again, all he has to do is look at the display unit 18 again. In this case, the line-of-sight input may be distinguished such that it is not accepted for other displayed actions. The eye-gaze input activation state should be reset eventually, but the condition may be that the release operation is performed, the activation display section 12 is looked at again, or a predetermined time elapses. Alternatively, a composite condition in which those conditions are combined may be used.

The display unit 19 displays the date imprinting ON and OF.
This is related to F control, and the display unit 20 displays the flash O
It relates to N and OFF.

Since the procedure of these input operations is not so different from the above, the details will be omitted, but the input operation is performed in relation to the display unit 12 for activating the line-of-sight input.

By configuring the input sequence as described above, while avoiding malfunctions due to carelessly gazing at the input display, or malfunctions when viewing the display state merely to confirm the current value, A device sequence with high input efficiency can be configured.

(Second Embodiment) FIG. 3 is a top view of a single-lens reflex camera equipped with a line-of-sight control device according to a second embodiment of the present invention.

In FIG. 3, reference numeral 21 denotes a shutter button, 2
Reference numerals 2 and 23 are push buttons different from the shutter button 21. Although not shown, a display unit for selecting each operation mode shown in FIG. 1 is arranged around the finder screen.

In this embodiment, the line-of-sight input operation is activated by pressing the operation button. As the operation button, the shutter button 21 or other operation buttons,
That is, the push buttons 22 and 23 may be used.

The camera operation after the eye-gaze input is activated is the same as that of the first embodiment. That is, when the user looks at the display section for selecting a specific operation, the control means (not shown) selects the operation mode, and at the same time, switches the selected display section to a display form that can be distinguished from others for confirmation. When the line of sight is separated from the display unit, the operation mode corresponding to the display content, that is, the input is confirmed. Alternatively, the input is confirmed by continuing to watch for a predetermined time. Similar to the above, various input confirmation sequences can be combined depending on the peculiarity of each display operation.

When the operation buttons are used, a slightly different sequence configuration may be adopted. In the case of push buttons, it is possible to give different meanings to the moment of pushing and the moment of releasing.

The most natural input procedure is that the line-of-sight input is activated from the moment the push button is pressed, and this state continues as long as the operation button is kept pressed. By looking at the display section showing the operation mode to be selected during this time, the desired motion is input through the line of sight, and the camera displays to confirm the line-of-sight input item. The user confirms the response of the camera and releases the push button to confirm the input information at that moment. If you do not want to change the line-of-sight input contents during the input operation, you can always change the display you gaze at while the push button is being pressed.

In the case of such a line-of-sight input procedure, a manual operation of pushing the push button is complicated, but since the period for receiving the line-of-sight input can be defined by manual operation, there are few chances of malfunction and the operation of the camera. By gazing at the display unit showing the state, the merit of the line-of-sight input that a desired operation can be selected, that is, the display confirmation operation and the simultaneity and the sameness of the input operation are satisfied. In addition, the operation members arranged at various places can be integrated into a form of single button and line-of-sight designation, and operability is greatly improved.

(Third Embodiment) FIG. 4 is a view showing the display in the viewfinder of a single-lens reflex camera equipped with the line-of-sight control apparatus of the third embodiment of the present invention. The same parts as those in FIG. 1 are designated by the same reference numerals.

In the second embodiment, the idea is to start the device operation mode by another input in order to select the device operation mode by the line-of-sight input. However, in the third embodiment, the opposite is true. The form is assumed. That is, a mode is such that selection is performed by line-of-sight input, the camera displays a confirmation display and responds, and then a single input means is used to input an intention to confirm input information.

The difference between FIG. 4 and FIG. 1 is only that the display unit 12 for activating the line-of-sight input in FIG. 1 is replaced with the display unit 31 for confirming the line-of-sight input.

The operation in the case of such a configuration is performed by first looking at any one of the display sections 13 to 20 for selecting the operation mode, and the viewed display operation is selected to confirm that. The display status changes to. Next, the user looks at the confirmation display section 31,
The entered contents are confirmed. If the fixed display portion 31 is not watched within a predetermined time, the contents selected by the sight line are automatically canceled and the display is returned to the original state.

Steps 201 to 208 in FIG. 5 are a flowchart showing only a part of the above-mentioned operation, that is, the selection of the display unit 13.

As the confirmation display section, it may be used in a negative sense like the NG display section 32 as shown in FIG. In this case, the gazed and selected item is canceled only when the NG display unit 32 is gazed within a predetermined time, and is otherwise automatically determined.

It is needless to say that the confirmation procedure in the positive or negative sense does not depend on the line of sight and a manual member such as the push button shown in FIG. 3 may be used.

In any case, by unifying all the operation systems into a procedure of selecting and confirming while viewing the content display by line-of-sight input and confirming by a single operation common to all selection items, input efficiency is improved. Improvements and input certainty are obtained.

(Fourth Embodiment) The scope of the present invention is not limited to devices such as cameras. It can be used for various devices having a function of detecting the direction of the gazing point of the user.

Therefore, the fourth embodiment of the present invention shows an example in which the line-of-sight control device is applied as the input means of the computer.

FIG. 7 shows only the display contents of the monitor screen screen, but this computer is equipped with a device (not shown) for picking up an image of the face of the user facing this screen and detecting the gaze direction of the eyeball. Has been done. Such a device can be easily realized by widening the light receiving area of the sensor and increasing the number of pixels in the above-described visual axis detection device, and therefore the details thereof will be omitted here.

In FIG. 7A, reference numeral 41 is a monitor screen screen, and four display parts 42 ₁ , 43 ₁ , 44 ₁ , 4 showing command type items are provided on the upper part of the screen 31.
5 ₁ are lined up.

"FILE" on the display section 42 ₁ is a general term for commands relating to the handling of data files. “UTILITY” on the display unit 43 ₁ is a general term for dictionaries, other reference data, subprograms started from the currently running program, and the like. In addition, the display unit 44
“CHARACTER” of ₁ relates to the type and size of characters to be displayed and printed, and “EDITOR” of the display unit 45 ₁ represents various editing functions.

In this embodiment, there is a foot switch (not shown), which controls the line-of-sight input function. That is, the above-mentioned command 4 items are displayed on the monitor screen screen 4 when the user turns on the foot switch.
1 appears on the screen and is ready to receive a line-of-sight input. Alternatively, the display itself may always be present, but the line-of-sight input is permitted by turning on the foot switch.

When "FILE" on the display unit 42 ₁ is selected by the first line-of-sight input, the selected item table 42 ₂ regarding the operation method (item) of this "FILE" is displayed below the display unit 42 ₁ . It appears as shown in 7 (b). This the selected item table 42 ₂ the angular action associated with opening and closing the storage of data files are Clause representation, confirm to the user that the command entry computer side is selected in the line of sight is "FILE" There is also a meaning to make it happen. The user
After confirming that this is the one that suits one's intention and is not an erroneous input, once the foot switch is released, the display 42
Line-of-sight input of "FILE" of ₁ is permitted.

Next, by selecting one of the items in the selected item table 42 ₂ by the second line-of-sight input, the content of the command executed for the file is specified. Here, when the foot switch is released again as described above, this is confirmed and executed.

Conventionally, such an operation has been performed by a manual member such as a mouse or a joystick, but since the selection can be done by the line of sight, these files can be managed while the hands are concentrated on the keyboard.

FIG. 7C shows "CHARA" on the display unit 44 ₁ .
It is a figure which shows the case where a command related to CTER, that is, a command related to characters is input through the line of sight.

When "CHARACTER" on the display section 44 ₁ is looked at, the selected display table 44 ₂ appears on the monitor screen screen 41, and if the "SIZE" display for character size designation is selected, the line of sight is selected. , The second selected item table 44 ₃ appears (see FIG. 7C). A specific character size is specified by looking at the desired character size.

Steps 301 to 311 in FIG. 8 are a flow chart showing only a part of the above-mentioned operation, that is, the selection of "FILE" on the display section 42 ₁ and "CHARACTER" on the display section 43 ₁ .

In this embodiment, it may not be operated on the main monitor screen 41, or may be operated on the sub monitor dedicated to command input. In the latter case, the configuration such as a head mounted display has an advantage that the visual axis detection system can be relatively simple.

In the above description, the line-of-sight input is activated and confirmed by the foot switch, but such a function can be achieved by using any manual member, and the operation described above is also possible. As in the example, it is possible to have a display area for activating the line-of-sight input in the screen, and look at the display area to activate the line-of-sight input sequence to perform a two-step line-of-sight operation.

(Fifth Embodiment) FIG. 9 shows the fifth embodiment of the present invention.

FIG. 9A shows an example in which the user controls the line-of-sight of the air conditioner 51 placed indoors, and FIG. 9B shows a more specific detailed front view of the air conditioner 51. It is a figure.

The air conditioner 51 has an optical input window 57 for the line-of-sight detection system.
The user's line of sight is detected from here. A device for tracking a person is added to the line-of-sight detection device as described above, and a face portion is imaged and received by a sensor, so that the line-of-sight can be obtained even in the case where the position of the user is uncertain. Can be detected. Further, on the front surface of the air conditioner 51, line-of-sight input display sections 52 to 56 are provided.

By looking at the display section 52, the user activates the air conditioner 51 into the line-of-sight input state. On this occasion,
The display unit 52 notifies the user that the line-of-sight input has been activated by an operation such as turning on a lamp.

Next, for example, in order to instruct the cooling operation, the display unit 56 ("cool") may be looked at. If the selection display of any of the display units 53 to 56 to be described later is not watched within the predetermined time, the activation of the line-of-sight input is canceled and the process returns to the standby state.

The display section 55 is for instructing the heating operation. In addition, the display units 53 and 54 are respectively for raising and lowering the set temperature, and are provided with symbols such as arrows related to the up and down of the temperature.

The designation operation by these lines of sight can be performed in the same manner as in the above-described first embodiment and the like.

Since the operation designation of such a device is not as complicated as the command designation of the computer, the confirmation of the line-of-sight input information may be carried out with a certain degree of certainty and a sequence configuration may be performed. After seeing the display unit 56 and displaying the confirmation display, by looking at the display unit 52 again,
The input may be fixed and the line of sight input standby state may be returned. In this case, as a confirmation display that the display unit 56 is viewed, it is possible to use a sign such as a change in the brightness of the display unit 56, a change in the color, or a blinking.

Conventionally, such a device must be controlled using a manual operation panel provided in the device itself or a dedicated wireless or wired remote control device. With the configuration as in the embodiment, the user can quickly and reliably control the operation of the device without looking through any manual operation device, just by looking at it with his / her eyes.

Furthermore, according to this embodiment, since control information is not input simply by looking at the display section, the possibility of causing a malfunction can be extremely reduced.

According to each of the above-mentioned embodiments, first, information is selected by the line-of-sight input, and then, only when the information is intended or not,
The input display is not prepared because the confirmation input is performed by the line-of-sight information (second line-of-sight input or stop of line-of-sight input) or by the operation member, in other words, by a single input confirmation operation. It is possible to configure a device with high input efficiency while avoiding a malfunction due to the user's gaze, or a malfunction when viewing the display state simply for confirming the current value.

More specifically, for example, in US Pat. No. 4,595,990 described in the conventional example, one is selected from a plurality of selected values by the line of sight in the first step, and the selected value is selected in the second step. Whether or not it really meets the user's intention is confirmed by cancel input or verify input by the line of sight. According to this, for example, two line-of-sight input operations must be performed until one value is confirmed. 4 is taken as an example, it is assumed that unintentional line-of-sight input is sequentially performed with the display unit 13a (Av) and the display unit 13b (Tv), and finally the display unit 13c (M) is selected and confirmed. Assuming that, in the above-described conventional device, the display unit 13a is gaze-inputted, and since this is not intended, the cancel input is performed, and then the display unit 13b is gaze-inputted. Does not FIG perform cancel input, the last display unit 13c and the visual axis input, since this is is intended such performing the verify inputs, it takes 6 steps in total.

On the other hand, according to this embodiment, the display unit 1
3a is entered into the line of sight, and if this is not the intention, the line of sight is moved to the display unit 13b, and if this is also unintended, the line of sight is moved to the display unit 13c. This is achieved by moving the input, and it is possible to quickly and easily perform the input with high efficiency without forcing the user to discriminate the line-of-sight input one by one.

[0103]

As described above, according to the present invention,
A line-of-sight detection unit that detects the line-of-sight of the user and outputs line-of-sight information, and a display unit that has one or more display symbols, each of which is meant to correspond to a different operation of the device. Input information confirmation means, and the line-of-sight detection means outputs the line-of-sight information indicating that the user is gazing at the specific one or more display symbols of the display means, and the single information confirmation When the signal from the means indicates a predetermined state, a control means for generating a control signal is provided to control the operation of the equipment, and the operation is selected by the line-of-sight detection means to be controlled from different operations. The confirmation (generation of the control signal) is performed by a single information confirmation means.

Therefore, it is possible to perform a reliable and highly efficient intention input without forcing the user to perform a complicated operation and without fear of a malfunction.

[Brief description of drawings]

FIG. 1 is a diagram showing a display in a viewfinder of a single-lens reflex camera equipped with a line-of-sight control device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a partial operation of a portion of the camera of FIG. 1 according to the present invention.

FIG. 3 is a top view of a single-lens reflex camera equipped with the line-of-sight control device of the second embodiment of the present invention.

FIG. 4 is a diagram showing a display in a viewfinder of a single-lens reflex camera equipped with a line-of-sight control device according to a third embodiment of the present invention.

5 is a flowchart showing a partial operation of a portion of the camera of FIG. 4 according to the present invention.

FIG. 6 is a diagram showing another example of display in the viewfinder of the camera of FIG.

FIG. 7 is a diagram showing a display on a screen of a monitor screen of a computer including a line-of-sight control device according to a fourth embodiment of the present invention.

FIG. 8 is a flowchart showing a partial operation of a portion of the computer of FIG. 7 according to the present invention.

FIG. 9 is a diagram according to a fifth embodiment of the present invention.

FIG. 10 is an optical layout diagram showing a conventional example in the case where a line-of-sight detecting means is incorporated in a single-lens reflex camera.

11 is a perspective view showing a main part of the line-of-sight detection means of FIG.

12 is a diagram for explaining the principle of line-of-sight detection in the line-of-sight detection means of FIG. 11 and the output intensity distribution from the image sensor of FIG.

[Explanation of symbols]

12-20 display unit 21 shutter button 22, 23 the push button 31 confirm the display unit 32 NG display part 42 _1, 43 _1, 44 _1, 45 ₁ display unit 42 _2, 44 _2, 44 ₃ the selected item table 51 Air Conditioning 52 to 56 display

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location G03B 13/02 7139-2K 9119-2K G03B 3/00 A

Claims (1)

[Claims] 1. A line-of-sight detecting means for detecting the line-of-sight of a user and outputting line-of-sight information, and a display having one or a plurality of display symbols, each of which has a meaning so as to correspond to a different operation of a device. Means, a single input information confirmation means, the gaze detection means outputs gaze information indicating that the user is gazing a specific one or a plurality of display symbols of the display means, and, A line-of-sight control device comprising: a control device that generates a control signal to control the operation of the device when a signal from a single information confirmation device indicates a predetermined state.