JPH07160410A - Computer input system - Google Patents

Abstract

PURPOSE:To specify a desired window screen for input without moving a pointing device such as a mouse and hand. CONSTITUTION:The visual point of a wearing person is detected by a glasses type sensor 44 and its visual point information is converted by a coordinate position conversion module 45 into coordinate position on a CRT 42. A windows displayed on the CRT 42 is controlled by a window system 46 based on the information on the coordinate position.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a personal computer, a workstation, a large-scale computer or the like used for consumer and general industries, and a computer for performing an input operation to various computer facilities using a window system. It is about the input system.

[0002]

2. Description of the Related Art A conventional technique will be described by taking a workstation as an example. FIG. 14 shows the equipment configuration of a conventionally used workstation, and FIG. 15 is a flowchart showing a typical keyboard input procedure for a conventionally used workstation with a window system. FIG. 16 is an example of a screen displayed on the CRT screen of the workstation. In FIG. 14, 11
Is the workstation itself, 12 is a CR that displays information
T and 13 are keyboards as input equipment, 14 is a mouse, and 15 is a window system mounted on the workstation body. In FIG. 16, 31, 32,
22 is a window screen displayed on the CRT screen.

Next, the operation of the conventional workstation will be described with reference to FIGS. When creating a program in a workstation using the window system 15 or performing various command input operations, the operator normally displays a plurality of window screens 31, 32, 33 (step 21). A window screen is used for reference, and a window screen is used for input. The operator thinks, for example, "Let's display XX in the window 31,""Let's create a program in the window 32" (step 22). At this time, it becomes necessary to sequentially perform an input operation using the keyboard 13 on a plurality of windows. For this reason, the operator first moves the mouse 14 to the input target window (step 23). The window system 15 detects the coordinates of the mouse 14,
The window at the position where the mouse 14 exists is activated (key input enabled state) (step 24). The operator confirms that the window to be input is activated and operates the keyboard 13 to start the input operation (step 25). After that, in order to perform the same input operation on another window, the operator thinks which window he / she wants to operate (step 22), and again moves the mouse 14 to that window (step 22). Step 23). After that, this series of operations is repeated every time the window that the operator wants to input changes.

[0004]

Since various computer input systems having a conventional window system have the above-described configuration, when an input is made from a keyboard to a certain window screen, an instruction device such as a mouse is first used. It was necessary to move to the window, and the operation was complicated.

The present invention has been made to solve the above problems, and provides a computer input system capable of designating a window screen to be input without moving a pointing device such as a mouse or a hand. It is an object.

[0006]

A first aspect of the present invention is directed to a display device (CRT 42) for displaying information, an input device (keyboard 43) for inputting information, and irradiating the pupil of an eye with reference light to scatter it. The eyeglass-type sensor 44 that detects the viewpoint of the wearer by detecting light, the coordinate position conversion module 45 that converts the viewpoint information from the sensor into the coordinate position on the display device, and the information of the coordinate position And a window system 46 for controlling the windows displayed on the display device.

A second aspect of the invention is to display a display device (CRT 42) for displaying information, an input device (keyboard 43) for inputting information, and irradiate the pupil of the eye with reference light and detect the scattered light. A spectacle-type sensor 44 having a function of detecting the viewpoint of the wearer and converting the viewpoint information into coordinate positions on the display device, and a window displayed on the display device based on the information on the coordinate positions. And a window system 46 for controlling.

According to a third aspect of the present invention, the display device (CRT 42) for displaying information, the input device (keyboard 43) for inputting information, and the pupil of the eye are irradiated with the reference light and the scattered light is detected. Eyeglass-type sensor 44 for detecting the viewpoint of the wearer, computer equipment (workstation main body 1) incorporating a coordinate position conversion module for converting viewpoint information from the sensor into coordinate positions on the display device, and coordinates thereof. And a window system 46 for controlling the window displayed on the display device based on the position information.

In a fourth aspect of the present invention, the display device (CRT 42) for displaying information, the input device (keyboard 43) for inputting information, and the pupil of the eye are irradiated with the reference light and the scattered light is detected. A spectacle-type sensor 44 for detecting the viewpoint of the wearer, a function for converting viewpoint information from the sensor into a coordinate position on the display device, and a window displayed on the display device based on the information on the coordinate position. And a window system 46 having a control function.

A fifth aspect of the invention is to display a display device (CRT 42) for displaying information, an input device (keyboard 43) for inputting information, and illuminate the pupil of the eye with reference light and detect the scattered light. An eyeglass-type sensor 44 for detecting the viewpoint of the wearer, and a coordinate position conversion module 45 for converting viewpoint information from the sensor into coordinate positions on the display device.
And a window system 46 having a function of controlling a window displayed on the display device based on the coordinate position information, and a foot-operated input device 107 for supporting the control of the window. It is what

A sixth aspect of the invention is a display device (CRT 42) for displaying information and a spectacle type sensor 44 for illuminating the pupil of the eye with reference light and detecting the scattered light to detect the viewpoint of the wearer. And a coordinate position conversion module 45 for converting viewpoint information from the sensor into a coordinate position on the display device, and a window system having a function of controlling a window displayed on the display device based on the information on the coordinate position. 46 and an input device such as a keyboard 43 incorporating a dedicated key 43a for supporting the control of the window.

According to a seventh aspect of the present invention, a display device (CRT 42) for displaying information, an input device (keyboard 43) for inputting information, and a pupil of an eye are irradiated with reference light and the scattered light is detected. An eyeglass-type sensor 44 for detecting the viewpoint of the wearer, and a coordinate position conversion module 45 for converting viewpoint information from the sensor into coordinate positions on the display device.
And a window system 46 having a function of controlling the window displayed on the display device based on the coordinate position information, and a voice input device 127 for supporting the control of the window. It is a thing.

[0013]

In the first aspect of the invention, the viewpoint of the wearer is detected by the eyeglass type sensor, and the viewpoint information is converted into the coordinate position on the display device by the coordinate position conversion module. The window displayed on the display device is controlled by the window system based on the coordinate position information.

In the second invention, the eyeglass-type sensor detects the viewpoint of the wearer, and the viewpoint information is converted into coordinate positions on the display device. The window displayed on the display device is controlled by the window system based on the coordinate position information.

In the third invention, the viewpoint of the wearer is detected by a spectacle-type sensor, and the viewpoint information is converted into coordinate positions on the display device by computer equipment. The window displayed on the display device is controlled by the window system based on the coordinate position information.

In the fourth invention, the viewpoint of the wearer is detected by a spectacle-type sensor, the viewpoint information is converted into coordinate positions on the display device by the window system, and the window displayed on the display device is It is controlled based on the information of the coordinate position.

In the fifth aspect of the invention, the viewpoint of the wearer is detected by the eyeglass type sensor, and the viewpoint information is converted into the coordinate position on the display device by the coordinate position conversion module. The window displayed on the display device is controlled by the window system based on the coordinate position information. Control of the window is supported by a foot-operated input device.

In the sixth invention, the viewpoint of the wearer is detected by a spectacle-type sensor, and the viewpoint information is converted into coordinate positions on the display device by the coordinate position conversion module. The window displayed on the display device is controlled by the window system based on the coordinate position information. Control of the window is supported by an input device such as a keyboard that incorporates dedicated keys.

In the seventh invention, the viewpoint of the wearer is detected by a spectacle-type sensor, and the viewpoint information is converted into coordinate positions on the display device by the coordinate position conversion module. The window displayed on the display device is controlled by the window system based on the coordinate position information. Control of the window is supported by the voice input device.

[0020]

【Example】

Example 1. Embodiments of the present invention will be described below with reference to the drawings. 1 is a block diagram of a computer input system according to a first embodiment of the present invention, in which 41 is a workstation body and 4
2 is a CRT as a display device for displaying information, 43 is a keyboard as an input device, 44 is a spectacle-type device that illuminates the pupil of the eye with reference light and detects the scattered light to detect the viewpoint of the wearer. A sensor, 45 is a coordinate position conversion module for converting the viewpoint information from the sensor 44 into a coordinate position on the CRT 42, and 46 is a C position based on the coordinate position information.
This is a window system that controls the windows displayed on the RT 42.

FIG. 2 is a principle diagram of the spectacle-type sensor 44. As shown in FIG. 2, the eyeglass-type sensor 44 calculates the orientation of the eye 201 in real time by illuminating the human eye (pupil) 201 with the reference light 202 and detecting the angle of scattering in real time. To do. The scattered light can be detected by receiving the light with the CCD 203 or the like. Since the light received by the CCD 203 becomes a digital signal, the process of determining the orientation of the eye 201 is easy.

FIG. 3 is a block diagram showing the relationship between the eyeglass-type sensor 44 and the coordinate position conversion module 45. As shown in FIG. 3, in the eyeglass-type sensor 44, calculation of the viewpoint and detection of the position of the screen are performed by a dedicated circuit chip provided in advance, and the result is always the I / O port 20.
Written to 4. The coordinate position conversion module 45 side
Coordinate conversion is performed based on the information written in the I / O port 205. Since there is a processing cycle even though it is always, the actual data update cycle is, for example, about 100 ms. As a function of the coordinate position conversion module 45, by calculating the deviation of each point based on the viewpoint information and the screen position information, it is possible to know where to look with respect to the screen, and then the actual number of display dots ( For example, 1280x
(1024 dots), the above deviation is converted into a dot position, and the conversion result is output to the I / O port 205 and transferred to the workstation.

FIG. 4 is a block diagram for explaining an X window as a configuration of the window system 46. In FIG. 4, the window manager 211 detects coordinate axes and activates windows. The window manager 211 stands up as a client (a kind of processing process) 208 at the same time when the X window software 206 is activated, and always sees the movement of a mouse (not shown) as a pointer. Also,
The current display status of the windows is also grasped at the same time, and the active or inactive process is automatically performed by seeing the coordinates of these windows and the mouse pointer.

FIG. 5 is a flow chart showing an input procedure for the workstation according to the first embodiment. FIG. 6 is a diagram showing an example of a screen displayed on the CRT screen of the workstation.

Next, the operation of the first embodiment will be described with reference to FIGS. When creating a program or performing various command input operations, the operator first displays a plurality of window screens (step 51). For each window on the CRT 42 displayed as shown in FIG. 6, the operator thinks, for example, "Let's display XX in the window 61", "Let's create a program in the window 62", etc. (step 52). The operator's viewpoint naturally moves to the window that the operator wants to input (step 53), and the spectacle-type sensor 4 worn by the operator 4
From 4, information regarding the viewpoint and where on the screen the user is looking are always sent to the coordinate position conversion module 45. The coordinate position conversion module 45 converts the information into the coordinate position on the CRT 42, and the window system 46 detects the coordinate position and activates the window screen existing at the coordinate (key input enabled state). (Step 54). The operator confirms that the window to be input is activated and operates the keyboard to start the input operation (step 55). After this, in order to perform the same input operation on another window,
The operator thinks which window he wants to operate (step 52), and the viewpoint naturally moves (step 53). After that, this series of operations is repeated every time the window that the operator wants to input changes.

In the actual key input operation, it is conceivable that the operator makes a blind touch input while looking at a book or the like outside the screen. In this case, the window does not become active with the above mechanism, but in order to respond to this, when the viewpoint moves out of the window or out of the screen, the function that keeps the last active window active is provided. It is possible to deal with this by adding it to the window system 46. Although not described in the first embodiment, not only the workstation but also various computer equipment such as a personal computer and a large-scale computer using a window system can be used as the computer equipment.

Embodiment 2 (corresponding to claim 2). Although the independent hardware is used as the coordinate position conversion module in the first embodiment, the function of the coordinate position conversion module is incorporated in the eyeglass-type sensor 44 as shown in FIG. It is possible to directly output the coordinate information from the sensor 44.

Embodiment 3 (corresponding to claim 3). Although independent hardware is used as the coordinate position conversion module in the first embodiment, as shown in FIG. 8 in the third embodiment, the coordinate position conversion module itself is incorporated into the workstation main body 41 in the form of a board,
The viewpoint information input from the communication port (not shown) in the workstation main body 41 is converted into coordinate information on the board, and the coordinate information is output to another input port (not shown) of the workstation main body 41. Such as
The coordinate information may be transferred to the window system 46.

Embodiment 4 (corresponding to claim 4). Although the independent hardware is used as the coordinate position conversion module in the first embodiment, the function of the coordinate position conversion module is realized as one function of the window system 46 as shown in FIG. It is also possible to adopt a configuration in which the viewpoint information from the glasses-type sensor 44 input to the station body 41 is automatically converted into coordinate information by the window system 46.

Embodiment 5 (corresponding to claim 5). In the first embodiment, the window system is provided with a corresponding function in consideration of the case where the operator inputs by blind touch. However, since the active state continuation / non-continuation is instructed, the fifth embodiment will be described. As shown in FIG. 10, the foot-operated input device 107 may be prepared to support blind touch. When it is desired to continue the active state of a specific window, the state is locked by continuously stepping on the foot input device 107.

Embodiment 6 (corresponding to claim 6). In the first embodiment, the window system itself is provided with a corresponding function in consideration of the case where the operator inputs by blind touch. However, since the active state continuation / non-continuation is instructed, the sixth embodiment will be described. As shown in FIG. 11, a dedicated key 43a is prepared on the keyboard 43, and it is possible to support blind touch. When it is desired to keep the active state of a specific window, the state is locked by pressing the dedicated key 43a.

Embodiment 7 (corresponding to claim 7). In the first embodiment described above, the window system is provided with the corresponding function in consideration of the case where the operator inputs by blind touch.
As a configuration using the voice input device 127 as shown in
It is possible to support blind touch. By using the voice input device 127, when the user wants to continue the active state of the particular window, he / she can be notified to the window system 46 by voice.

FIG. 13 is a block diagram showing the functional arrangement of the voice input device 127. In FIG. 13, reference numeral 301 is a signal processing unit that inputs a voice and performs signal processing such as frequency analysis, and 302 is a signal matching unit that matches a registered voice with the above-described signal-processed voice and searches for a matching signal string. , 303
Is a processing execution unit that, if there is a matching signal string, executes the processing registered for it.

[0034]

As described above, according to the first aspect of the invention, a spectacle-type sensor for detecting the viewpoint of the wearer, and a coordinate conversion module for converting viewpoint information from the sensor into coordinate positions on the display device. Since it is configured to include the window system that controls the window displayed on the display device based on the information of the coordinate position, I want to input just by moving the viewpoint without operating the pointing device such as the mouse or moving the hand. The effect is that the window screen can be specified.

According to the second aspect of the present invention, the spectacle type sensor having a function of detecting the viewpoint of the wearer and converting the viewpoint information into the coordinate position on the display device, and the information of the coordinate position based on the information. Since the window system for controlling the window displayed on the display device is included, it is possible to output the coordinate position information directly from the spectacle-type sensor, and thus the same effect as described above can be obtained.

According to the third invention, a spectacle-type sensor for detecting the viewpoint of the wearer, and a computer facility incorporating a coordinate conversion module for converting the viewpoint information from the sensor into coordinate positions on the display device, Since the window system for controlling the window displayed on the display device based on the information on the coordinate position is included, the same effect as described above can be obtained.

According to the fourth aspect of the invention, based on the eyeglass-type sensor for detecting the viewpoint of the wearer, the function of converting the viewpoint information from the sensor into the coordinate position on the display device, and the information of the coordinate position. Since the window system having the function of controlling the window displayed on the display device is included, the same effect as the above can be obtained.

According to the fifth invention, a spectacle-type sensor for detecting the viewpoint of the wearer, a coordinate position conversion module for converting the viewpoint information from the sensor into a coordinate position on the display device, and the coordinate position A window system having the function of controlling the window displayed on the display device based on the information,
Since it is configured to include a foot-operated input device for supporting the control of the window, the same effect as described above can be obtained, and a specific window can be kept active by the foot-operated input device. There is also an effect that it is possible to support blind touch.

According to the sixth aspect of the invention, the spectacle-type sensor, the coordinate position conversion module, and the window system, and the input device such as a keyboard in which a dedicated key is incorporated to support the control of the window are also provided. In addition to the effects similar to those described above, there is also an effect that a specific window can be kept in an active state by using the input device, and a blind touch can be supported.

According to the seventh aspect of the invention, since the spectacle-type sensor, the coordinate position conversion module, and the window system and the voice input device for supporting the control of the window are also provided, the same effect as the above can be obtained. There is also an effect that the voice input device can convey the intention to the window system by voice when the user wants to keep the active state of the specific window by the voice input device.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a computer input system according to a first embodiment of the present invention.

FIG. 2 is a principle diagram of the spectacle-type sensor in FIG.

FIG. 3 is a block diagram showing the relationship between the eyeglass-type sensor and the coordinate position conversion module in FIG.

4 is a block diagram for explaining an X window as a configuration of a window system 46 in FIG.

FIG. 5 is a flowchart showing an input procedure for a workstation according to the first embodiment.

FIG. 6 is a CR of the workstation in the first embodiment.
It is a figure which shows the example of a screen currently displayed on T screen.

FIG. 7 is a configuration diagram of a computer input system according to a second embodiment of the present invention.

FIG. 8 is a configuration diagram of a computer input system according to a third embodiment of the present invention.

FIG. 9 is a configuration diagram of a computer input system according to a fourth embodiment of the present invention.

FIG. 10 is a configuration diagram of a computer input system according to a fifth embodiment of the present invention.

FIG. 11 is a configuration diagram of a computer input system according to a sixth embodiment of the present invention.

FIG. 12 is a configuration diagram of a computer input system according to a seventh embodiment of the present invention.

FIG. 13 is a block diagram illustrating a functional configuration of a voice input device according to a seventh embodiment.

FIG. 14 is a configuration diagram of a conventional computer input system.

FIG. 15 is a flowchart showing an input procedure for a workstation according to a conventional example.

FIG. 16: CRT of workstation in conventional example
It is a figure which shows the example of a screen currently displayed on the screen.

[Explanation of symbols]

 41 Workstation Main Body 42 CRT (Display Device) 43 Keyboard (Input Device) 44 Glasses Sensor 45 Coordinate Position Conversion Module 46 Window System 61, 62, 63 Window 95 Workstation 107 Foot-operated Input Device 127 Voice Input Device

Claims (7)

[Claims] 1. A display device for displaying information, an input device for inputting information, and an eyeglass-type sensor for illuminating the pupil of the eye with reference light and detecting the scattered light to detect the viewpoint of the wearer. And a coordinate position conversion module for converting viewpoint information from the sensor into a coordinate position on the display device, and a window system for controlling a window displayed on the display device based on the information on the coordinate position. A computer input system characterized by. 2. A display device for displaying information, an input device for inputting information, and a reference light irradiating the pupil of the eye, and the scattered light is detected to detect the viewpoint of the wearer and the viewpoint. A spectacle-type sensor having a function of converting information to a coordinate position on the display device, and a window system for controlling a window displayed on the display device based on the information on the coordinate position. Computer input system. 3. A display device for displaying information, an input device for inputting information, and a spectacle-type sensor for illuminating the pupil of the eye with reference light and detecting the scattered light to detect the viewpoint of the wearer. And a computer facility incorporating a coordinate position conversion module for converting the viewpoint information from the sensor into a coordinate position on the display device, and a window for controlling the window displayed on the display device based on the information on the coordinate position. And a computer input system. 4. A display device for displaying information, an input device for inputting information, and a spectacle-type sensor for detecting the viewpoint of the wearer by irradiating the pupil of the eye with reference light and detecting the scattered light. And a window system having a function of converting viewpoint information from the sensor into a coordinate position on the display device and a function of controlling a window displayed on the display device based on the information of the coordinate position. Characteristic computer input system. 5. A display device for displaying information, an input device for inputting information, and an eyeglass-type sensor for detecting the viewpoint of the wearer by irradiating the pupil of the eye with reference light and detecting the scattered light. A coordinate position conversion module for converting viewpoint information from the sensor into a coordinate position on the display device; and a window system having a function of controlling a window displayed on the display device based on the coordinate position information. And a foot-operated input device for supporting the control of the window described above. 6. A display device for displaying information, an eyeglass-type sensor for detecting the viewpoint of the wearer by irradiating the pupil of the eye with reference light and detecting the scattered light, and viewpoint information from the sensor. To a coordinate position conversion module for converting the above to a coordinate position on the display device, a window system having a function of controlling a window displayed on the display device based on the information of the coordinate position, and supporting the control of the window. Therefore, a computer input system having an input device such as a keyboard incorporating a dedicated key. 7. A display device for displaying information, an input device for inputting information, and an eyeglass-type sensor for detecting the viewpoint of the wearer by irradiating the pupil of the eye with reference light and detecting the scattered light. A coordinate position conversion module for converting viewpoint information from the sensor into a coordinate position on the display device; and a window system having a function of controlling a window displayed on the display device based on the coordinate position information. And a voice input device for supporting the control of the window, a computer input system.