JPH1039995A - Line-of-sight/voice input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate the complication of a mouse operation, to eliminate the need of mounting a line-of-sight detector and to facilitate utilization by catching and deducing the moving distance and moving direction of a mouse cursor by the line of sight of an operator and instructing a click operation or the like by recognizing the voice of the operator. SOLUTION: An image processing circuit 2 inputs the video image of the operator by a camera 1 and converts the video signals to image data processable by a computer and a line-of-sight recognition part 3 recognizes the position of the eyes of the operator from the image data. A line-of-sight position calculation part 4 calculates which coordinate on a display the line of sight is paying attention to from the positional relation of pupils to the eyes of the operator. Also, a voice conversion circuit 6 converts the voice of the operator inputted by a microphone 5 to voice data and a voice recognition part 7 automatically voice-recognizes the voice data and analyzes an instruction given by the operator. A control part 8 controls the movement and operation of the mouse cursor and performs display by the coordinate position of the line of sight and the instruction obtained from the voice.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an input device of an information processing device such as a computer, and more particularly to a visual line / voice input device.

[0002]

2. Description of the Related Art Conventionally, a mouse, a joystick and the like have been used as a pointing device for pointing on a display screen among input means of an information processing apparatus such as a computer. For the purpose of removing and improving the operability of computer input, there has been proposed a method of realizing equivalent functions of a mouse or the like using a line of sight of an operator (operator).

For example, Japanese Patent Application Laid-Open No. 6-242883 discloses that an optical sensor is provided in goggles, infrared rays from a light source of the optical sensor pass through a first half mirror, and
The light reflected from the half mirror is applied to the eyeball, and the light reflected from the eyeball is reflected by the second and first half mirrors, and the CC
It is incident on the D camera, the incident light is converted into an electric signal by the CCD camera, the image is processed by the image processor, the outline is identified by the outline processor, and the direction of the line of sight is displayed on the display by the coordinate position detector. There has been proposed an input device that detects the coordinates. In other words, in this device, an optical sensor is disposed on goggles, infrared light is applied to the eyeball from the light source unit of the optical sensor, and the coordinate position of the line of sight of the operator's attention on the screen is detected by detecting reflected light from the eyeball. Is what you do.

[0004]

Conventionally, input processing such as movement of a mouse pointer or a cursor, selection of an icon, screen movement, and line drawing is performed by manually operating an input device (pointing device) such as a mouse or a joystick. I went by. For this reason, it is difficult to use these devices when the hand cannot be used or when an operator with a physical disability operates.

On the other hand, even when the operator's line of sight is used as the input means, the operator needs to wear devices such as an eye camera and goggles, which is very troublesome. In particular, when working for a long time on the screen, wearing goggles or the like causes fatigue and lowers working efficiency.

Accordingly, the present invention has been made in view of the above circumstances, and has as its object the movement of a mouse cursor and the like, the selection of icons and the window (screen) performed by a pointing device such as a mouse and a joystick. It is an object of the present invention to provide an input device capable of easily performing operations such as movement using an operator's line of sight without mounting equipment such as an eye camera and goggles on the operator. It is another object of the present invention to provide an input device that realizes a function equivalent to a mouse without using a hand by performing button operations of a mouse and a joystick by voice.

[0007]

In order to achieve the above object, the present invention comprises a camera for inputting a line of sight of the operator at a predetermined position separated from the operator, and based on an input signal from the camera. Image processing means for acquiring information on the position and movement of the line of sight of the operator on the display screen of the display device, and performing a pointing operation such as cursor movement on the display screen of the display device based on the acquired information. An input device of an information processing apparatus characterized in that the input apparatus performs the processing.

According to the present invention, there is provided a microphone for collecting voices emitted by the operator, and voice recognition means for recognizing voice information of the operator input from the microphone, and a mouse based on the recognized voice information. And a joystick button operation.

The outline of the present invention will be described below.

The present invention provides a display device (2 in FIG. 2).
Through the camera (1 in FIG. 2), the coordinate position of the line of sight on the display of the operator (5 in FIG. 2) is calculated by the calculator (3 in FIG. 2). The operation of moving is performed.

In the present invention, the voice of the operator input through the microphone (4 in FIG. 2) is automatically recognized by the voice recognition means of the computer, and the same operation as the conventional mouse button operation is performed.

According to the present invention, an operation using a mouse or the like can be performed by
By performing with the operator's line of sight and voice,
The complexity of mouse operation is eliminated, and the space for a conventional mouse is not required. Further, the operator does not need to wear the gaze detection device, is easy to use, and does not have to inconvenience the wearing of the gaze detection device.

As described above, according to the present invention, even when the user cannot freely operate with a mouse or the like or a finger with a joystick, a function equivalent to a mouse can be easily used. It is possible to do. In particular, the present invention makes it possible to use an information processing device and a game device for a disabled person or the like.

[0014]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a configuration of an embodiment of the present invention. Referring to FIG. 1, an embodiment of the present invention comprises:
An image output from the image signal conversion circuit 2 includes a camera 1 for shooting an operator, and an image processing circuit 2 for inputting a video shot by the camera 1 and converting the video signal into image data that can be processed by a computer. The data is input to the line-of-sight recognition unit 3, and the line-of-sight recognition unit 3 recognizes the position of the eyes of the operator from the input image data.

The line-of-sight position calculation unit 4 calculates which coordinate on the display the line of sight focuses on from the positional relationship of the pupil with respect to the eyes of the operator.

Also, in the embodiment of the present invention, a microphone 5 for picking up the voice of the operator, a voice conversion circuit 6 for converting the voice data inputted by the microphone 5 into voice data which can be handled by a computer, A voice recognition unit 7 that recognizes and analyzes instructions given by the operator.

The control unit 8 controls the movement and operation of the mouse cursor based on the coordinate position of the line of sight obtained from the line of sight position calculation unit 4 and the instruction obtained from the voice recognition unit 7, and displays it on the display 9.

The operation of the embodiment of the present invention will be described with reference to FIG.

Normally, mouse information is divided into information obtained by moving the mouse and information obtained by pressing (clicking) a button attached to the mouse.

The operation of moving the mouse cursor is performed by moving the mouse body in the direction in which the mouse cursor is to be moved. In the embodiment of the present invention, these mouse cursor movement functions are realized by the camera 1, the image conversion circuit 2, the gaze recognition unit 3, and the gaze position calculation unit 4.

The line-of-sight recognition unit 3 recognizes the position of the eyes of the operator and the position of the pupil in the eyes by using, for example, a pattern recognition technique. This information is input to the line-of-sight position calculation unit 4, where the position of the line of sight of the operator is calculated from these positional relationships.

On the other hand, in the embodiment of the present invention, the operation of the button of the conventional mouse is performed by the voice input of the operator. The button pressing function of the mouse is realized by the microphone 5, the voice conversion circuit 5, and the voice recognition device 7 in the embodiment of the present invention.

The voice recognition device 7 analyzes an operator's command corresponding to turning on and off a conventional mouse button using voice recognition technology. Thereby, the function of the mouse can be realized without using a hand.

[0025]

EXAMPLES Next, in order to explain the above-mentioned embodiment of the present invention in more detail, the following description will be made based on examples showing specific examples.

First, FIG. 2 is a diagram schematically showing a schematic configuration of an embodiment of the present invention. Referring to FIG. 2, an operator 10 performs an operation while viewing the display screen of display 9.
At that time, the camera 1 keeps monitoring the line of sight of the operator 10 and moves the mouse cursor to a position where the line of sight of the operator 10 stops on the display screen.

At this point, if the operator 10 speaks a voice, for example, "click", the microphone 5 detects this voice and the control unit of the computer performs a single click operation. For example, if you pronounce "double",
Perform a double-click action. Other voices are ignored. As a result, operations using a mouse cursor (cursor movement, icon selection, window generation, movement, and the like) can be performed on the display screen without using a mouse.

The camera according to the embodiment of the present invention may be provided at any place other than the display device, as long as the position of the line of sight of the operator can be identified.

Further, since a plurality of cameras are provided, it is possible to improve the measurement accuracy and reliability of the line of sight.

In the present invention, the term "mouse" includes a device having a function equivalent to a mouse, that is, an input device called a pointing device of a computer, such as a joystick, a trackball, a joypad, and a light pen. Shall be considered.

[0031]

As described above, according to the present invention,
Eliminates the need to manually operate the mouse and joystick,
Further, since the operator does not need to equip a device for gaze detection, there is an effect that even a physically handicapped person can easily perform a pointing operation on the display screen.

Further, according to the present invention, since a mouse and a joystick are not required, there is an advantage that a working space such as a mouse space is not required.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a diagram schematically showing the appearance of one embodiment of the present invention.

[Description of Signs] 1 camera 2 signal conversion circuit 3 gaze recognition unit 4 gaze position calculation unit 5 microphone 6 voice signal conversion unit 7 voice recognition unit 8 control unit 9 display 10 operator

Claims (4)

[Claims] 1. An input device, wherein a moving distance and a moving direction of a mouse cursor are determined by capturing an operator's line of sight with a camera, and an instruction such as a click operation is performed by recognizing an operator's voice. 2. A camera for inputting a line of sight of the operator at a predetermined position separated from the operator, and based on an input signal from the camera, a line of sight of the operator on a display screen of a display device. An input device for an information processing apparatus, comprising: image processing means for acquiring position and movement information; and performing a pointing operation such as moving a cursor on a display screen of the display device based on the acquired information. . 3. A microphone for collecting voices emitted by the operator, and voice recognition means for recognizing voice information of the operator input from the microphone, wherein a mouse or a joystick is used based on the recognized voice information. 3. The input device for an information processing device according to claim 2, wherein a button operation is performed. 4. An input device for an information processing apparatus according to claim 2, wherein said camera is provided on or near said display device facing said operator.