JP2000242394A - Virtual keyboard system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To add a keyboard image to a display device and to perform operation equivalent to input using a physical keyboard by providing a fingertip sensor which detects the relative position from a reference sensor and the movement of a fingertip and an input key detecting means which specifies the key at the position of the fingertip. SOLUTION: The reference sensor 1 which detects the relative position and movement of the fingertip is set where the movement is small at the time of the original keyboard input, e.g. on the left wrist of the operator. Further, sensors 2a, 2b, 2c... are fitted to fingertips with which the key input is done. Key-top array images 41, 42, 43... of a built-in virtual keyboard are sent to a personal computer display unit 9 and displayed in a window. Further, the key top array images 41, 42, 43... are compared with pieces 13a, 13b, 13c... of position information on fingertip position detection sensors 2a, 2b, 2c... and an input key detecting means 23 judges which key is pressed in which timing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to a "command",
The present invention relates to a virtual keyboard system for a computer application device such as an information processing device, a financial accounting device, a music device, a control device, and a game device for inputting “character” and “numeric value” from a keyboard.

[0002]

2. Description of the Related Art Conventionally, input of "commands", "characters" and "numerical values" of keyboard-type input devices such as information processing devices, financial accounting devices, music devices, control devices, game devices, etc. are naturally performed from a keyboard. Had been Further, a method of acquiring input key data by a visual sensor such as a television sensor has been proposed.

[0003]

However, the above-mentioned method has various problems. The first problem is that the detection of a key pressed by a finger can only be performed by an input screen. The reason is that the key code information of the key pressed by the movement of the finger is directly sent to the device main body and input, so that the input result cannot be understood unless the input screen is viewed.

[0004] A second problem is that any operator cannot use the most suitable keyboard for the purpose.
The reason is that the key top surface is hidden by hand and cannot be seen, so it is necessary to learn the key arrangement of the keyboard. Therefore, switching between keyboards having different key arrangements requires the operator to memorize all key arrangements on different keyboards, which is burdensome.

The present invention has been made under such a background, and even without a physical input device such as a keyboard,
It is an object of the present invention to provide a virtual keyboard system that displays a keyboard image on a display device and displays the position of an operator's finger with respect to the keyboard image, thereby performing an operation equivalent to input using a physical keyboard. .

[0006]

According to a first aspect of the present invention, there is provided an input system for inputting a key from a virtual keyboard to a computer application device, wherein the input system is mounted at a predetermined position and a reference position of the virtual keyboard is set. Reference sensor, and a fingertip sensor for detecting the relative position of the fingertip performing the key input from the reference sensor and the movement of the fingertip,
First display means for displaying a key top image of the virtual keyboard; second display means for displaying the position of the fingertip detected by the fingertip sensor on the displayed keytop image; Input key detection means for detecting that a key input has been performed by the detection signal, and specifying a key at the position of the fingertip displayed on the key top image; and a key of the input key detected by the input key detection means An input key code generating means for generating a code and transmitting the code to the computer application device is provided.

According to a second aspect of the present invention, there is provided the virtual keyboard system according to the first aspect, wherein the display of the key top image is directly displayed on an operation screen of the computer application device.

According to a third aspect of the present invention, there is provided the virtual keyboard system according to the first aspect, wherein the display of the key top image is displayed by being overlaid on an operation screen of the computer application device. I do.

According to a fourth aspect of the present invention, there is provided the virtual keyboard system according to any one of the first to third aspects, wherein the fingertip sensor is incorporated in a dedicated glove.

According to a fifth aspect of the present invention, there is provided the virtual keyboard system according to any one of the first to third aspects, wherein the fingertip sensor is attached to a fingertip for performing a key input.

According to a sixth aspect of the present invention, in the virtual keyboard system according to any one of the first to third aspects, the relative position of the fingertip is detected by measuring a myoelectric potential of a finger. provide.

According to the present invention, the key arrangement of the keyboard and the position of the fingertip are superimposed on the virtual space, respectively.
By substituting an actual keyboard operation, an input operation can be performed without an actual keyboard. If you move your finger to press any key in accordance with the keyboard image and the movement to the key pressed position on the virtual keyboard is detected, the color or shape of the marker or the corresponding key top image changes, or the sound changes. Pressing feedback is given to the operator in the event of occurrence, and input is performed at the same time. This detection accuracy can also be optimized by performing calibration and sensitivity adjustment. At this time, for example, it is possible to set such that the marker is not displayed when the marker of each finger is separated from the virtual keyboard surface by a certain amount or more.

The present invention comprises detection means for detecting the position and movement of a fingertip, display means for displaying a virtual keyboard and a finger position, and control software. The detecting means first sets a reference position of the virtual keyboard as a reference, for example, by attaching a reference sensor to a wrist of the left hand or the like.
The position of the reference sensor can be attached to other than the operator's body depending on the operating environment. Each finger is provided with a sensor capable of detecting the relative position of the tip of the finger with respect to the position reference of the virtual keyboard.

The display means displays the position of each fingertip with respect to the key arrangement image of the keyboard based on the virtual keyboard position reference and the relative position of each finger. The display is provided with a dedicated display device or screen, a keyboard input-only area is provided in a part of the information display screen, and the information is displayed by being overlaid on the information display screen. On the information display screen, the keyboard image is fixedly displayed, a marker indicating the position of each finger is displayed on the keyboard image, and the marker is shaken in accordance with the movement of the finger. The display shake is adjusted by adjusting the detection cycle of the fingertip position or adjusting the threshold value of the movement amount.

[0015]

Embodiments of the present invention will be described below in detail with reference to the drawings. In this embodiment, a case where a virtual keyboard is used as a data input keyboard of a personal computer will be described. FIG. 1 is a block diagram showing the basic configuration of this embodiment. In this figure, reference numeral 21 denotes a fingertip position and movement detecting means for detecting a relative position and a movement of a finger for performing a key input. The position data of the detected fingertip is displayed on the virtual keyboard display means 2.
2 to display the position of the fingertip on the virtual keyboard of the personal computer display 9. The fingertip position / movement detecting means 2
The detection data of No. 1 is also given to the input key detection means 23, which key is detected, and the detection result is notified to the input key code generation means 24, where the key code based on the detection result is generated. And input it to the personal computer 25.

FIG. 2 is a diagram showing a basic sensor arrangement and a state in which a virtual keyboard is displayed in a separate window on the same screen as the computer to be operated among display means of the display screen. In this figure, a reference sensor 1 serving as a reference for detecting the relative position and movement of a fingertip is set at a position where movement is small at the time of an original keyboard input, such as the operator's left wrist. In addition, the fingertip sensor 2
By attaching a, 2b, 2c,..., it is possible to detect the position of each finger relative to the reference sensor 1.

The display screen 8 of the personal computer is displayed as a window in the display 9 and displays keyboard input information 14. Similarly, the virtual keyboard screen 11 is also displayed on the display 9 by another window. In this virtual keyboard screen 11, 41, 42, 43,.
* Is a key top arrangement image, 13a, 13b,
13c,... Are the fingertip sensors 2a, 2b, 2c,.
This is a display of fingertip position information corresponding to.

FIG. 3 is a diagram showing a state in which a display screen of a virtual keyboard is prepared separately from a computer to be operated among display means of the display screen. FIG. 4 is a diagram showing a state in which a virtual keyboard is displayed as an overlay on the same window as the computer to be operated, of the display means of the display screen.

Next, the operation of this embodiment will be described with reference to FIGS. Virtual keyboard display means 2
2 is an image of the key top arrangement images 41, 42, 43,... Of the built-in virtual keyboard.
To display it in a window. Further, the key top arrangement images 41, 42, 43,... And the fingertip position detection sensors 2a, 2b, 2c,.
a, 13b, 13c,... are compared with each other to determine which key was pressed at which timing.
, And displays the information on the virtual keyboard screen 11 with the fingertip position information 13a, 13b, 13c,... Changed in display color, and transmits the information to the input key code generation means 24, where the key code data is The data is sent to the personal computer 25 and input.

The above-mentioned determination of which key is pressed at which timing can have a function of calibrating fine adjustment as needed. The process of determining the press can be defined in advance or can be adjusted by the operator by software.
In addition, by using the keyboard of the actual machine to imitate the key top layout position information and the timing information of pressing,
It is also possible to easily use a keyboard with a special arrangement.

The operation of one embodiment of the present invention has been described above in detail with reference to the drawings. However, the present invention is not limited to this embodiment, and a design change or the like may be made without departing from the gist of the present invention. The present invention is also included in the present invention. For example, using a hardware device with the same configuration, a pointing device that detects a pointing position by sliding a finger on a flat pad, or a type of input that inputs a command by giving meaning to a finger action (action) It can also be applied to equipment.

A pointing device, such as a touch pad, which detects the amount of movement on a plane can be operated virtually. As a result, by moving the fingers of both hands as if there is an actual keyboard, it is possible to perform an input on an arbitrary surface or in a space in the same manner as an actual keyboard. For example, if you need computer-assisted work support but want to keep both hands free, the information display will show your clothes,
This is possible by mounting a display device on a hood, a mask, glasses, a head-mounted display, or the like.
Information input can be performed if there is any pointing device if only a command selection operation. However, inputting characters and numbers takes time with a pointing device, and voice input depends on the usage environment. Therefore, it is more efficient to input the characters and numbers using a keyboard.

According to the present invention, even in a use environment where extra hardware such as a keyboard cannot be carried, if there is a space corresponding to the keyboard, it is possible to reproduce an operation environment for keyboard input. For example, if you want to type a character string for information retrieval or recording while performing equipment maintenance while referring to maintenance information data using a computer, use a normal keyboard on an appropriate surface or space. By performing the hitting operation, a character string serving as a search key can be input. In addition, by seeing the key top image of the keyboard, it is also possible to use different keyboard images depending on the target, improving the efficiency of input work,
Speeding up and facilitation are possible.

Also, since a three-dimensional gesture can be recognized, it can be extended to a three-dimensional pointing device used for rotating and supporting three-dimensional data. In addition, `` It is possible to overcome the conditions on the operating side, which are originally inconvenient to hit the keyboard such as gloves and bandages, '' and `` The virtual keyboard key arrangement and pitch can be set to arbitrary values By changing the threshold, the amount of movement of the finger and the threshold value for the judgment of the pressing can be reduced to reduce physical fatigue such as tenosynovitis, and a quiet operating environment due to the absence of a tapping sound. Become.

[0025]

As described above, the first effect of the present invention is that even if there is no keyboard, any operator can perform input as if there is a normal keyboard. The reason is that input can be performed without confirming the position and movement of the finger with respect to the key top image even without a keyboard. The second effect is that any operator can use the most suitable keyboard for the purpose. The reason is that the key arrangement and the pressing timing can be adjusted according to the characteristics of the fingers of the individual without being restricted by the physical shape.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an embodiment of the present invention.

FIG. 2 is a diagram showing a sensor layout which is a basic embodiment of the present invention, and a state in which a virtual keyboard screen is displayed in a separate window on the same screen as a computer to be operated, among display means of a display screen.

FIG. 3 is a diagram showing a state in which a display screen of a virtual keyboard is prepared separately from a computer to be operated among display means of the display screen.

FIG. 4 is a diagram showing a state in which a virtual keyboard is overlaid and displayed on the same screen as a computer to be operated, among display means on the display screen.

[Explanation of symbols]

 1 Reference sensors 2a, 2b, 2c,... Fingertip sensors 41, 42, 43,.・ ・ ・ Fingertip position information 14 Keyboard input information

Claims (6)

[Claims] 1. An input system for inputting a key from a virtual keyboard to a computer application device, comprising: a reference sensor attached to a predetermined position, the reference position of the virtual keyboard being set; A fingertip sensor for detecting a relative position from a sensor and a movement of the fingertip; first display means for displaying a keytop image of the virtual keyboard; and a fingertip detected by the fingertip sensor on the displayed keytop image. Second display means for displaying a position of the fingertip; an input key for detecting that a key input has been performed by a detection signal from the fingertip sensor and identifying a key at the position of the fingertip displayed on the keytop image Detecting means for generating a key code of an input key detected by the input key detecting means; A virtual keyboard system comprising input key code generation means for transmitting to an application device. 2. The virtual keyboard system according to claim 1, wherein the display of the key top image is directly displayed on an operation screen of the computer application device. 3. The virtual keyboard system according to claim 1, wherein the display of the key top image is displayed as being overlaid on an operation screen of the computer application device. 4. The virtual keyboard system according to claim 1, wherein the fingertip sensor is incorporated in a special glove. 5. The virtual keyboard system according to claim 1, wherein the fingertip sensor is attached to a fingertip for performing a key input. 6. The virtual keyboard system according to claim 1, wherein the relative position of the fingertip is detected by measuring a myoelectric potential of a finger.