JP2010282575A - Input device and program for input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input device which can display input content at an appropriate position even when being in use, in a wearable input device. <P>SOLUTION: The input device has a finger sensor which is worn by at least one of a finger and a hand of a user and which detects spatial operation of the finger to generate detection signals, and display sections 42L and 42R which are worn by the user and which display input information corresponding to each of the detection signals, and calculates spatial posture of the hand based on each detection signal, and displays the input information at display positions on the display sections 42L and 42R associated with the posture. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention belongs to the technical field of input devices and programs for input devices. More specifically, the present invention belongs to the technical field of an input device that outputs an input code corresponding to a detection signal obtained by detecting a spatial motion of a user's finger and a program for the input device.

  In general, in a so-called computer device or the like, a keyboard on which keys that are operated by a finger corresponding to each character are arranged is used as an input device when inputting characters such as alphabets. This keyboard requires a space for its arrangement, and it is desired to reduce the size of the keyboard from the viewpoint of space saving. On the other hand, considering that the keys on the keyboard are operated by fingers, there is a limit to downsizing the keyboard from the viewpoint of operability.

  Therefore, a virtual keyboard device that does not require a space for keyboard layout has been proposed as a kind of the input device. In this type of keyboard device, a sensor is attached to a finger, a spatial motion of the finger is detected by the sensor, and an input code corresponding to an operation signal indicating the detected motion is output. The prior art regarding such a virtual keyboard device is described, for example, in Patent Document 1 below. In Patent Document 1, in a so-called wearable input device, a character or the like corresponding to an output input code is displayed on a computer arranged at a position where the user can visually recognize it.

Japanese Patent Laid-Open No. 10-307663

  However, in the conventional configuration disclosed in Patent Document 1 described above, the input content as the input code cannot be confirmed without a display member fixed at a visually recognizable position. As a result, there is a problem in that the wearable input device cannot be moved and is not convenient as a wearable input device.

  Therefore, the present invention has been made in view of the above problems, and an example of the purpose thereof is an input device capable of displaying input contents at an appropriate position even during use in a wearable input device, and It is to provide a program for the input device.

  In order to solve the above-described problem, the invention according to claim 1 is mounted on a mounting portion provided on at least one of the user's finger or hand, and the spatial movement of the user's finger. Detection means such as a finger sensor that detects a detection signal and input information determination means such as an input code output unit that determines input information corresponding to the detection signal based on the detection signal from the detection means And a display unit such as a display unit that displays the input information that is worn by the user and determined by the input information determination unit, and a hand space based on the detection signal generated by the detection unit Posture information, which is information indicating a specific posture, is calculated, and an input code output for displaying the input information determined by the input information determination means at a display position on the display means associated with the posture information. And a display control means such as a department.

  In order to solve the above problem, the invention according to claim 2 is the input device according to claim 1, wherein the display control means is combined with the input information already displayed on the display means. The input information determined by the input information determining means is displayed at the display position.

  In order to solve the above-mentioned problem, the invention according to claim 3 is the input device according to claim 1 or 2, wherein the display means is attached to either the hand or the forearm, and the display control is performed. The means calculates one of the rotation angles as the posture information based on the detection signal generated by the detection means, and is preset for each rotation angle based on the calculated rotation angle. The input information determined by the input information determination means is displayed at the display position.

  In order to solve the above-described problem, the invention according to claim 4 is the input device according to any one of claims 1 to 3, wherein the display position is associated with the display position. The posture information is configured to be the display position at which the user can visually recognize the input information displayed at the display position.

  In order to solve the above-mentioned problem, the invention according to claim 5 is the input device according to any one of claims 1 to 4, wherein the display means is a display surface of the input information in the display means. It is mounted so that the directions of the normals raised in the direction are a plurality of directions in which the display means can be visually recognized by the user.

  In order to solve the above-described problem, the invention according to claim 6 causes a computer to function as the input device according to any one of claims 1 to 5 excluding the detection unit.

  According to the first aspect of the present invention, since the determined input information is displayed at the display position associated with the posture information indicating the spatial posture of the hand, the user's posture or the like changes However, the input information can be displayed at a display position on an appropriate display means.

  According to the invention described in claim 6, by reading and executing the input device program by the computer, the determined input information is associated with the posture information indicating the spatial posture of the hand. Since the computer functions so as to display at a display position, input information can be displayed at an appropriate display position on the display means even when the posture of the user changes.

  Therefore, according to the first or sixth aspect of the invention, in the input device that outputs input information corresponding to the detection signal obtained by detecting the spatial movement of the user's finger, the input device is in use. The input content can be displayed at an appropriate position.

  According to the invention described in claim 2, in addition to the effect of the invention described in claim 1, in addition to the input information already displayed on the display means, the input information determined by the input information determination means For example, even when a phrase composed of a plurality of characters is input, the entire phrase can be displayed at an appropriate position and its contents can be recognized.

  According to the invention described in claim 3, in addition to the effect of the invention described in claim 1 or 2, the display means is attached to either one of the hand or the forearm, and the rotation angle of either one of them is determined. Since it is calculated as posture information and input information is displayed at a display position preset for each rotation angle based on the calculated rotation angle, the display position is optimized according to the rotation of the hand or forearm. Thus, the input information can always be displayed at the optimum display position.

  According to the invention described in claim 4, in addition to the effect of the invention described in any one of claims 1 to 3, the display position where the input information is displayed corresponds to the posture information corresponding to the user. Since the input information displayed at the display position is the display position at which the input information is visible, the input information can be displayed at the display position on the display means that is always visible even when the posture of the user changes. .

  According to the invention described in claim 5, in addition to the effect of the invention described in any one of claims 1 to 4, the direction of the normal line set up on the display surface in the display means is changed from the user to the display means. Since the display means is mounted so as to be in a plurality of visually recognizable directions, the input information can be easily displayed according to the user's posture.

It is a figure which shows the structure of a virtual keyboard apparatus. It is a figure which shows the other structure of a virtual keyboard apparatus. It is a figure which shows a finger sensor. It is a graph which shows the output voltage of the finger sensor in a stationary state. It is a graph which shows the output voltage of the finger sensor at the time of key depression. It is a figure which shows the structure of the input code output part containing a pattern judgment part. It is a figure which shows the motion pattern when a finger | toe moves to the left direction memorize | stored in the motion pattern dictionary part, (A) shows the motion pattern of an X-axis direction, (B) is the motion pattern of a Y-axis direction. Indicates. It is a figure which shows the operation | movement pattern at the time of a finger | toe moving rightward memorize | stored in the operation | movement pattern dictionary part, (A) shows the operation pattern of an X-axis direction, (B) is the operation pattern of a Y-axis direction. Indicates. It is a figure which shows the operation | movement pattern at the time of a finger | toe moving straight ahead memorize | stored in the operation | movement pattern dictionary part, (A) shows the operation pattern of an X-axis direction, (B) is a Y-axis direction. An operation pattern is shown. It is a figure which shows the operation | movement pattern at the time of a finger | toe moving to the near front memorize | stored in the operation | movement pattern dictionary part, (A) shows the operation pattern of an X-axis direction, (B) is a Y-axis direction. An operation pattern is shown. It is a figure which shows the motion pattern at the time of a finger | toe moving to the left tip direction memorize | stored in the motion pattern dictionary part, (A) shows the motion pattern of the X-axis direction, (B) is the Y-axis direction. An operation pattern is shown. It is a figure which shows the operation | movement pattern at the time of a finger | toe moving to the right tip direction memorize | stored in the operation | movement pattern dictionary part, (A) shows the operation pattern of an X-axis direction, (B) is a Y-axis direction. An operation pattern is shown. It is a figure which shows the operation | movement pattern at the time of a finger | toe moving to the left front direction memorize | stored in the operation | movement pattern dictionary part, (A) shows an operation pattern of the X-axis direction, (B) shows the Y-axis direction. An operation pattern is shown. It is a figure which shows the motion pattern at the time of a finger | toe moving to the right front direction memorize | stored in the motion pattern dictionary part, (A) shows the motion pattern of the X-axis direction, (B) is the Y-axis direction. An operation pattern is shown. It is a flowchart figure which shows learning operation | movement of a virtual keyboard apparatus. It is a figure which shows the input system containing a virtual keyboard apparatus. It is a figure which illustrates arrangement | positioning etc. in the hand of the display part which concerns on embodiment, (a) is a figure which illustrates the external appearance of the said display part arrange | positioned at the right and left hand, (b) is each display part of right and left It is a figure which shows the example of a display. It is a flowchart which shows the display operation which concerns on embodiment. It is a figure which shows the structure for the attitude | position detection of the wrist which concerns on embodiment, (a) is an external view of the left index finger with which the finger sensor is mounted | worn, (b) is a detection axis | shaft etc. of attitude | position detection about a left hand (C) is a figure which illustrates the content of the rotation angle-display area corresponding | compatible table which concerns on embodiment. It is a figure which shows the determinant for the attitude | position detection of the wrist which concerns on embodiment, (a) is a basic formula of the said determinant, (b) And (c) is an application example of the said determinant. It is an external view of the left hand which illustrates the result of display operation concerning an embodiment, (a) is the 1st example and (b) is the 2nd example. It is a flowchart which shows the display operation which concerns on a 1st modification, (a) is a flowchart which shows the said whole display operation, (b) is a flowchart which shows the detail of a stationary state determination operation | movement. It is a flowchart which shows the display operation which concerns on a 2nd modification. It is a figure which illustrates arrangement | positioning etc. in the hand of the display part which concern on 3rd modification, (a) is an external view which shows a 1st example, (b) is an external view which shows a 2nd example, (c ) Is an external view showing a third example, and (d) is an external view showing a fourth example.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The embodiment described below is an embodiment when the present invention is applied to a virtual keyboard device.

(I) Outline of Virtual Keyboard Device First, an outline of a virtual keyboard device according to an embodiment will be described.

  FIG. 1 shows the configuration of the virtual keyboard device. In FIG. 1, fingers 14-1 to 14-5 and 14-6 to 14-10 of the left hand 10 </ b> L and the right hand 10 </ b> R as an example of a user's wearing part are respectively provided with finger sensors 16-as an example of detection means. 1 to 16-5 and 16-6 to 16-10 are mounted.

  The finger sensor 16 is a sensor that detects a spatial movement of the finger and outputs an operation signal that changes in accordance with the movement, and is configured by, for example, an acceleration sensor or a gyro sensor. Details of the finger sensor 16 will be described later.

  For example, the backs of the left hand 10L and the right hand 10R are equipped with data collection units 18 and 20, respectively. The data collection units 18 and 20 have finger sensors 16-1 to 16-5 and 16-6 to 16 respectively. The operation signal from −10 is supplied via the harnesses 22-1 to 22-5 and 22-6 to 22-10. The data collection units 18 and 20 have radio units 24 and 26, respectively, and operation signals from the radio units 24 and 26 are input code output units 28 as an example of input code determination means and display control means. Is received by the radio unit 30. At this time, the operation signals collected by the data collection units 18 and 20 may be received by the input code output unit 28 via a wire (harness).

  In addition to the wireless unit 30, the input code output unit 28 includes an operation pattern dictionary unit 32 and an operation pattern determination unit 34.

  The motion pattern dictionary unit 32 is a storage unit that stores dictionary pattern information indicating a pattern of finger motion signals. Details of the dictionary pattern information will be described later with reference to FIGS.

  On the other hand, the operation pattern determination unit 34 compares the operation signal corresponding to the pattern information stored in the operation pattern dictionary unit 32 with the operation signal supplied from the wireless unit 30 and corresponds to the detected operation signal. Output the input code.

  The output input code is transmitted from the wireless unit 36 provided in the input code output unit 28 to the wireless unit 40 of the character display 38 as an example of the display control means. The character display 38 has a display unit 42, and a character code 44, for example, is displayed on the display unit 42 as an input code input by the operation of a finger. In this case as well, the input code output unit 28 and the character display 38 may be connected by wire (harness) and communicated.

  FIG. 2 shows another configuration of the virtual keyboard device.

  In the configuration of FIG. 1, the data collection units 18 and 20 are attached to the left hand 10L and the right hand 10R, respectively, but in the configuration of FIG. 2, no data collection unit is provided. The operation signals from the finger sensors 16-1 to 16-5 and 16-6 to 16-10 are input to the input code output unit 28 via the harnesses 22-1 to 22-5 and 22-6 to 22-10. Is supplied to the operation pattern determination unit 34.

  In the embodiment described in detail below, in the configuration of the virtual keyboard device shown in FIG. 1 and FIG. 2, the character display 38 and the input code output unit 28 are the left hand character display 38 and the input code, respectively. Two sets of output unit 28, right hand character display 38 and input code output unit 28 are provided. The input code output unit 28 for the left hand outputs an input code detected as input by the operation of the fingers 14-1 to 14-5 of the left hand 10L, and the display unit 42 in the character display 38 for the left hand. Is a character, number or symbol or the input code itself (hereinafter referred to as the character, number or symbol or the input code itself) as an example of input information corresponding to the input code output from the left-hand input code output unit 28. Simply referred to as “characters”). On the other hand, the input code output unit 28 for the right hand outputs an input code detected as input by the operation of the right 14R fingers 14-6 to 14-10, and the display unit 42 in the character display 38 for the right hand. Displays a character or the like corresponding to the input code output from the input code output unit 28 for the right hand. Both the character corresponding to the input code output from the input code output unit 28 for the left hand and the character corresponding to the input code output from the input code output unit 28 for the right hand You may display on both or any one of the display part 42 in the display 38, and the display part 42 in the character display 38 for right hands.

  In FIG. 3, an acceleration sensor is shown as an example of the finger sensor 16.

  In FIG. 3, a finger sensor 16 is a three-axis acceleration sensor, and detects acceleration due to finger movement on each of the X axis, the Y axis, and the Z axis. The Z axis indicates the direction 54 of gravitational acceleration.

  FIG. 4 shows an example of the output voltage of the finger sensor 16 when the finger is stationary.

  In FIG. 4, it can be seen that the output voltage in the Z-axis direction differs from that in the X-axis direction and the Y-axis direction by the amount of gravitational acceleration (see reference numeral 46).

  FIG. 5 shows the output voltage of the finger sensor 16 when the key is depressed in the space.

  FIG. 5 shows the output voltage of each finger sensor 16 when each finger (mainly the little finger 14-5) of the left hand 10L is operated so as to push down the key of the letter “Q” in the virtual keyboard device according to the embodiment. Yes. The key arrangement of the virtual keyboard device according to the present invention employs a so-called JIS (Japanese Industrial Standard) arrangement. In addition, the upper, middle, and lower graphs in FIG. 5 show the X-axis direction, the Y-axis direction, and the Z-axis direction of each finger sensor 16 attached to the little finger 14-5, the ring finger 14-4, and the thumb 14-1, respectively. The output voltages in the X-axis direction, Y-axis direction, and Z-axis direction of the finger sensor 16 attached to the other middle finger 14-3 and index finger 14-2 are not shown.

  In FIG. 5, it can be seen that the operation of the little finger 14-5 is large during the time when the operation of depressing the key of the letter “Q” is performed (the time indicated by reference numeral 48 in FIG. 5). That is, as shown in the upper part of FIG. 5, changes in the output voltage in the X-axis direction, the Y-axis direction, and the Z-axis direction of the finger sensor 16-5 attached to the little finger 14-5 are attached to other fingers. It can be seen that this is larger than the change in output voltage of the finger sensor 16 in the X-axis direction, Y-axis direction, and Z-axis direction.

  FIG. 6 shows the configuration of the input code output unit 28 including the operation pattern determination unit 34.

  In FIG. 6, for example, regarding the left hand 10L, as described above, the finger sensor 16 is attached to each of the fingers 14-1 to 14-5, and one finger sensor 16 has three axes (X axis, Y axis, and (Z-axis) operation signal is output. Therefore, 5 × 3 = 15 operation signals 50 are supplied to the operation pattern determination unit 34.

  Here, in the virtual keyboard device according to the embodiment, learning is performed by mapping 15 motion signals 50, that is, 15 data, to each character 52 using a neural network, and the motion pattern dictionary unit 32 performs the above-described learning. Create or update dictionary pattern information.

  Here, the contents of the dictionary pattern information will be specifically described with reference to FIGS.

  First, FIG. 7 shows an operation pattern stored in the operation pattern dictionary unit 32 when the finger 14 moves to the left, and FIG. 7A shows an operation pattern in the X-axis direction. (B) shows an operation pattern in the Y-axis direction.

  Similarly, FIG. 8 shows the motion pattern when the finger 14 moves rightward, FIG. 9 shows the motion pattern when the finger 14 moves straight ahead, and FIG. 10 shows the motion pattern when the finger 14 moves. FIG. 11 shows the motion pattern when the finger 14 moves in the left-hand direction, and FIG. 12 shows the motion pattern when the finger 14 moves in the right-hand direction. FIG. 13 shows the operation pattern when the finger 14 moves in the left front direction, and FIG. 14 shows the operation pattern when the finger 14 moves in the right front direction.

  Then, as illustrated in FIGS. 7 to 14, the operation pattern determination unit 34 causes the output voltage in the X-axis direction based on the output voltage of the finger sensor 16 to exceed the threshold value 1 set in advance in the X-axis direction. (In the + direction) or the threshold value 2 that has been preset in the X-axis direction downward (in the-direction), and the output voltage in the Y-axis direction is preset in the Y-axis direction. The motion pattern of the finger 14 is detected based on the situation where the threshold value 1 is broken upward (in the + direction) or the threshold value 2 preset in the Y-axis direction is broken down (in the − direction). Then, based on the movement direction of the finger 14 determined by the detected movement pattern of the finger 14, a corresponding input code is output.

  Next, the operation as the learning related to the creation or update of the dictionary pattern information described above will be specifically described with reference to FIG.

  FIG. 15 is a flowchart showing the learning operation of the virtual keyboard device according to the embodiment. The learning operation is performed for each hand 10 in principle.

  In FIG. 15, the power is turned on in step S10, and the process proceeds to step S12. If the learning mode is determined by the input operator in step S12, the process proceeds to step S14, and the keyboard is connected to each input code output unit 28 (see FIG. 1 or 2).

  In step S16, an operation of pressing the keyboard is performed. In step S18, acceleration data of the finger sensor 16 at the time of key pressing is read. In step S20, data processing of a necessary portion of the acceleration data (for example, from a change point of the acceleration data). Data processing at ± 500 ms).

  In step S22, learning is performed with a neural network. In step S24, new dictionary pattern information is constructed. In step S26, if the input operator does not finish learning, the process returns to step S16. If it is determined that the learning has ended, the process proceeds to step S28. If it is determined in step S12 that the learning mode is not determined by the input operator, the process proceeds to step S28.

  In step S28, the acceleration data of the finger sensor is read, and in step S30, the acceleration data is compared with the data table value to determine whether or not the hand or finger has moved.

  If “No” in the step S30, the process returns to the step S28. If “Yes” in the step S30, the process proceeds to a step S32 to process data at a necessary portion of the acceleration data (for example, ± 500 ms from the change point of the finger sensor). Data processing (see reference numeral 48 in FIG. 5)).

  In step S34, the movement of the finger based on the acceleration data is compared with the dictionary pattern information held in the movement pattern dictionary unit 32 as a database. In step S36, the input code closest to the movement of the finger is output. The input code is transmitted to a character output device such as a personal computer, and the process proceeds to step S40. In addition to being transmitted to the character output device, the determined input code is also output for each hand on the character display 38 corresponding to each hand.

  If it is determined in step S40 that the input operator does not continue the input, the power is turned off in step S42. If the input operator continues the input in step S40, the process proceeds to step S44. If it is determined in step S44 that the input operator has entered an accurate character according to the judgment of the input operator by looking at the display screen 60 (see FIG. 16 described later) or the display unit 42 of each character display 38. On the other hand, if the input operator does not input an accurate character according to the judgment of the input operator by looking at the display screen 60 or the display unit 42 of each character display 38 in step S44, the process proceeds to step S28. Proceed to S12.

  In the flowchart of FIG. 15, the determination of the input operator in steps S12, S26, S40, and S44 will be described. An operation button is provided in the data collection unit 18 or 20, and the input operator (that is, the user) The operation corresponding to the determination is performed on the operation button, and the virtual keyboard device performs the determination according to the operation performed on the operation button.

  FIG. 16 shows a configuration of an input system including the virtual keyboard device according to the embodiment. In FIG. 16, the character display 38 illustrated in FIG. 1 or 2 is not shown.

  In FIG. 16, the operation signal 50 from the finger sensor 16 attached to each finger is supplied to the operation pattern determination unit 34 of the corresponding input code output unit 28. Then, the motion pattern determination unit 34 compares the motion signal 50 from the finger sensor 16 with the motion signal corresponding to the pattern information stored in the motion pattern dictionary unit 32, and inputs corresponding to the detected motion signal 50. The code is supplied to the corresponding character display 38 and personal computer 56.

  The personal computer 56 outputs the input code to the display device 58, and the characters 59 indicated by the input code, for example, “A”, “B”, “C”,... Column 59 are displayed on the display screen 60 of the display device 58. Is done.

(II) Configuration and Operation of Character Display According to the Embodiment Next, the configuration and operation of each input code output unit 28 and each character display 38 and related members according to the embodiment will be described with reference to FIGS. Will be described. FIG. 17 is a diagram illustrating the arrangement of the display unit 42 in the hand according to the embodiment, FIG. 18 is a flowchart illustrating the display operation according to the embodiment, and FIG. 19 is a diagram illustrating wrist posture detection according to the embodiment. It is a figure which shows the structure for. 20 is a diagram illustrating a determinant for wrist posture detection according to the embodiment, and FIG. 21 is an external view of the left hand 10L illustrating the result of the display operation according to the embodiment.

  As described above, the input code output unit 28 and the character display 38 according to the embodiment are provided for the left hand and the right hand, and the display unit 42L and the display unit 42R as examples of the respective display means are, for example, As illustrated in FIG. 17A, the left-hand display unit 42L covers the entire upper part of the left hand 10L, and the right-hand display unit 42R covers the entire upper part of the right hand 10R. It is mounted so as to wrap around the back.

  More specifically, the display unit 42L for the left hand is configured by, for example, an organic EL (Electro Luminescence) display device having flexibility, for example, from the vicinity of the base of the thumb 14-1 to the little finger 14- It is mounted so as to cover the entire left back near the base of 5. Similarly, the display unit 42R for the right hand is configured by a flexible display device, and is mounted so as to cover, for example, the entire right hand back from the base of the thumb 14-6 to the base of the little finger 14-10. Yes.

  In addition to these, in each of the display portions 42L and 42R, it is possible to perform display by dividing each band-like display area parallel to each extended finger 14 and extending from the base of each finger 14 in the wrist direction. It is said that. That is, in the display portion 42L of the left hand 10L, for example, a strip shape that is parallel to the thumb 14-1 and the index finger 14-2 in an aligned state and extends from the base of the thumb 14-1 and the index finger 14-2 in the wrist direction. It is possible to display characters and the like using only the display area (in other words, not using the other display area of the display unit 42L).

  Then, as illustrated in FIG. 17B, the input code detected as input by the operation of the fingers 14-1 to 14-5 of the left hand 10L as described above is displayed on the display unit 42L for the left hand as described above. A character or the like (character code 44L) corresponding to is displayed. On the other hand, on the display unit 42R for the right hand, characters and the like (character code 44R) corresponding to the input code detected as being input by the operation of the fingers 14-6 to 14-10 of the right hand 10R are displayed. The

  Next, the display operation according to the embodiment, which is mainly executed for each hand, mainly with respect to the input code output unit 28 and the character display 38 of each hand, will be specifically described with reference to FIGS.

  As shown in FIG. 18, as a display operation according to the embodiment, first, as step S <b> 1, each input code output unit 28 receives any operation signal from the data collection unit 18 or 20 corresponding to the corresponding hand. It is monitored whether or not 50 is output. When no operation signal 50 is output (step S1; NO), the monitoring is continued. On the other hand, when any of the operation signals 50 is output (step S1; YES), each input code output unit 28 next detects the posture of the hand at that point in step S2, and the user is the most. A display area on the display section 42L or 42R that is easy to visually recognize (that is, the display area for each of the display sections 42L and 42R that can be displayed separately) is selected. In the operation of step S2, each input code output unit 28 sets, for each corresponding hand 10, the extension direction when the left and right arms and hands are straightly extended forward of the body as the “y-axis” direction. The rotation angle of the left and right wrists around the axial direction is detected as the posture, and the display area on the display unit 42L or 42R that is most easily visible to the user is selected according to the detected posture. The operation in step S2 will be described in detail later with reference to FIGS.

  When a new display area is selected in each of the display units 42L and 42R by the operation of step S2, each input code output unit 28 next selects the newly selected display area until just before that in step S3. It is determined whether or not it is necessary to change the display area from the immediately preceding one by determining whether or not each display area is the same. If the display area does not need to be changed (step S3; NO), each input code output unit 28 proceeds to the operation of step S5 described later. On the other hand, if the display area needs to be changed (step S3). YES), as step S4, the display area is changed to the new display area selected in the operation of step S2.

  Next, in step S5, each input code output unit 28 optimizes the display start position and display end position of characters and the like corresponding to the input code to be displayed in the existing or newly selected display area. In the operation in step S5, the positions of the beginning and end of sentences and the line breaks of characters and the like that are actually displayed in the display area to be used are optimized so as to be more easily visible.

  In addition to the operation signal 50 newly input in step S1, the optimization operation targets operation signals 50 corresponding to all input codes displayed on the display units 42L and 42R so far. Done. That is, when an input operation for a new character or the like is executed, the display area is changed including the already input character or the like (in other words, as a series of sentences including the input character or the like) Optimization of the display start position and the like is executed. As a result, each input code output unit 28 generates display position information indicating the optimized display position and the like, and the generated display position information is displayed as characters and the like displayed at the optimized display position and the like. Are added to the corresponding input codes and output to the corresponding character display 38 respectively.

  After that, in step S6, each character display unit 38 adds a character or the like corresponding to the input code output from the corresponding input code output unit 28 to the corresponding input code output unit 28 and outputs it from the corresponding input code output unit 28. It is displayed at the display position on the display unit 42L or 42R indicated by the display position information that has been received. Then, each input code output unit 28, as a step S7, for example, confirms whether or not the operation signal 50 for a certain time or more has been output from the data collection unit 18 or 20, or the like. It is confirmed whether or not the input operation is finished and the display operation according to the embodiment is finished. When the display operation is not ended (step S7; NO), each input code output unit 28 returns to step S1 and repeats the series of operations described above. If the display operation is ended (step S7; YES), the display operation according to the embodiment is finished as it is.

  Next, the display area selection operation according to step S2 will be described in detail with reference to FIGS.

  First, as described above with reference to FIGS. 1, 2, and 3, each finger 14 is equipped with a finger sensor 16 whose Z axis is the direction opposite to the direction of gravity 54. Specifically, as shown in FIG. 19A by taking the index finger 14-2 of the left hand 10L as an example, the finger sensor 16 includes a middle joint of each finger 14, that is, a first joint and a second joint of each finger. It is mounted on the back side of the hand. At this time, as shown in FIG. 19 (a), each finger sensor 16 is mounted so that the Z-axis is opposite to the direction of gravity 54, and the X-axis and Y-axis are respectively shown in FIG. 19 (a). Direction. Note that the finger sensor 16 may be mounted at any position other than the middle section as long as the spatial movement of the finger 14 can be detected.

  Then, for each of the left hand 10L and the right hand 10R to which the finger sensor 16 is attached in such a state, as shown in FIG. 19B, the rotation angle about the y-axis about the wrist 15L is illustrated as an example of the left hand 10L. The input code output unit 28 calculates the acceleration data for each axis output from each finger sensor 16. In FIG. 19B, the X axis, the Y axis, and the Z axis for each finger sensor 16 are described only for the finger sensor 16-1 attached to the thumb 14-1, and the other finger sensors 16 are shown. The description is omitted.

  More specifically, first, using the acceleration data for each axis from each finger sensor 16, the rotation angle θ around each y axis for each finger 14 is expressed by the determinant shown in FIG. Calculate by solving.

  Here, as the rotation angle θ, the rotation angle of the wrist is 0 degree when each palm is directed vertically downward. For the left hand 10L, the rotation angle range from 0 degrees to -180 degrees until the palm turns counterclockwise by rotating the wrist counterclockwise is reversed, and the wrist is rotated again by rotating the wrist clockwise. The range of the rotation angle until it faces vertically upward is assumed to be 0 degree to +180 degree. On the other hand, for the right hand 10R, the range of the rotation angle from the above-mentioned “rotation angle of 0 degree” to the rotation of the wrist in the clockwise direction until the palm turns vertically upward is changed from 0 degree to −180 degrees. The range of the rotation angle from when the wrist is rotated in the clockwise direction until the palm faces vertically upward is set to 0 degrees to +180 degrees. Here, it should be noted that the sign (positive / negative) of the rotation angle is reversed between the left and right hands.

  In addition, the determinant shown in FIG. 20A is applied to the acceleration data output from the finger sensor 16 for all fingers 14 for each finger 14, and “x”, “ “y” and “z” are acceleration data in the x-axis direction, acceleration data in the y-axis direction, and acceleration data in the z-axis direction, respectively, from the finger sensor 16. Further, “1 g” in the determinant is the gravitational acceleration. Then, the input code output unit 28 displays the acceleration data “x” in the x-axis direction, the acceleration data “y” in the y-axis direction, and the acceleration data “z” in the z-axis direction from each finger sensor 16 in FIG. The rotation angle θ is calculated for each finger 14 by substituting into the determinant shown in FIG.

  Accordingly, the input code output unit 28 then averages the rotation angle θ calculated for each finger 14 separately for the left hand 10L and the right hand 10R. More specifically, the rotation angles θ calculated for the little fingers 14-1 to 14-5 are averaged to obtain the rotation angle for the left hand 10L, and are calculated for the little fingers 14-10 to 14-6. The rotation angle θ is averaged to obtain the rotation angle for the right hand 10R. Then, the average value for each hand is set as the rotation angle of the wrist of each hand.

  In calculating the rotation angle of the wrist, the rotation angle θ of the finger 14 calculated for the representative fingers 14 (for example, the middle finger 14-3 and the middle finger 14-8) corresponding to the left and right wrists is used. It can also be configured to have a wrist rotation angle.

  Further, in calculating the rotation angle θ for each finger 14, the rotation of the finger 14 is obtained with an average value of the rotation angle θ before and after a predetermined time provided for data processing exemplified by reference numeral 48 in FIG. 5. It is also possible to set the angle θ.

More specifically, for example, a timing 0.25 seconds before the start of the predetermined time is set as time t1, and a timing when 0.25 seconds elapses from the predetermined time is set as time t2. FIG. 20B shows acceleration data “x _t1 ” in the x-axis direction, acceleration data “y _t1 ” in the y-axis direction, and acceleration data “z _t1 ” in the z-axis direction from the finger sensor 16 at time t1. Substituting into the determinant and solving, further, the acceleration data “x _t2 ” in the x-axis direction, the acceleration data “y _t2 ” in the y-axis direction, and the acceleration data “z _t2 ” in the z-axis direction from the finger sensor 16 at time t2. By substituting and solving the determinants shown in FIG. 20C, the average value of the rotation angles θ calculated using the respective determinants is used as the rotation angle θ for the finger 14. In this case, the acceleration data “x _t1 ” in the x-axis direction, the acceleration data “y _t1 ” in the y-axis direction, and the acceleration data “z _t1 ” in the z-axis direction from the finger sensor 16 at time t1 are shown in FIG. Only the rotation angle θ obtained by substituting into the determinant shown in FIG. 5 may be used as the rotation angle θ for the finger 14, or the acceleration data “x _t2 ” in the x-axis direction from the finger sensor 16 at time t2. , Only the rotation angle θ obtained by substituting the acceleration data “y _t2 ” in the y-axis direction and the acceleration data “z _t2 ” in the z-axis direction into the determinant shown in FIG. The rotation angle θ may be set as follows.

  When the wrist rotation angle θ is calculated for each hand, based on the rotation angle-display area correspondence table stored in advance in a nonvolatile manner for each hand in a memory (not shown) or the like in the input code output unit 28 in advance. The display area corresponding to the rotation angle θ is selected in the display units 42L and 42R.

  Here, the rotation angle-display area correspondence table will be described with reference to FIG.

  First, as a premise, the display part 42L for the left hand 10L is assumed to be mounted as a whole so as to cover the entire upper part of the left hand 10L, and the display area facing the metacarpal bone of the thumb 14-5 is first. The display area facing the metacarpal bone of the index finger 14-2 is the second display area, the display area facing the metacarpal bone of the middle finger 14-3 is the third display area, and the ring finger 14 -4 is a fourth display region, and the display region facing the little finger 14-5 is a fifth display region. Further, the display part 42R for the right hand 10R is assumed to be mounted in a range covering the entire upper part of the right hand 10R, and the display area facing the metacarpal bone of the thumb 14-6 is the first display area. The display area facing the metacarpal bone of the index finger 14-7 is the second display area, the display area facing the metacarpal bone of the middle finger 14-8 is the third display area, and the ring finger 14-9 The display area facing the metacarpal bone is the fourth display area, and the display area facing the little finger 14-510 metacarpal is the fifth display area.

  Then, as the rotation angle-display area correspondence table for each hand, as in the rotation angle-display area correspondence table T illustrated in FIG. When the rotation angle θ is less than −90 degrees, the first display area and the second display area are associated with each other. When the rotation angle θ is −90 degrees or more and less than −0 degrees, the first display area to the third display area are associated with each other. When the angle θ is 0 degree or more and less than +45 degrees, the first display area to the fifth display area are associated with each other. When the rotation angle θ is +45 degrees or more and less than +90 degrees, the third display area to the fifth display area are associated with each other. When the rotation angle θ is not less than +90 degrees and less than +180 degrees, only the fourth display area and the fifth display area are associated with each other. The contents of the rotation angle-display area correspondence table T are common to the left hand 10L and the right hand 10R because the sign (positive / negative) of the rotation angle is reversed between the left and right hands as described above.

  Since the rotation angle-display area correspondence table T is stored in the input code output unit 28, if the rotation angle θ is calculated, it corresponds immediately (that is, a character or the like is displayed at the rotation angle θ). Display area can be selected.

  Then, according to the display operation described with reference to FIGS. 18 and 19, for example, when the left hand 10 </ b> L has the palm vertically downward (that is, when the rotation angle θ is 0 degree), for example, FIG. As shown in FIG. 21, characters and the like are displayed using all the display areas (first display area to fifth display area) of the display unit 10L, while the left hand 10L is, for example, counterclockwise from the state of FIG. When the image is rotated 90 degrees (ie, when the rotation angle θ is −90 degrees), characters and the like are displayed using only the first display area to the third display area as illustrated in FIG. 21B, for example. The Rukoto. By executing such a display operation, even when the wrist is rotated, characters and the like are displayed at a position optimal for visual recognition each time.

  In the case illustrated in FIG. 21, when the wrist 15L is rotated from the state of FIG. 21A to the state of FIG. 21B, the display unit 42L is already in the original state of FIG. The characters displayed on the screen (character code 44L) and the characters newly input when the state shown in FIG. 21B is entered (in other words, as a series of characters). The display is performed using only the first display area to the third display area exemplified in 21 (b).

  As described above, according to the operation of the virtual keyboard device according to the embodiment, the determined input code is displayed at the display position associated with the rotation angle θ indicating the spatial posture of the hand. Even when the posture of the user changes, characters and the like can be displayed at appropriate display positions on the display units 42L and 42R.

  Further, since the newly entered input code is displayed at the display position associated with the rotation angle θ together with the input code already displayed on the display units 42L and 42R, it is composed of, for example, a plurality of characters. Even when a phrase or the like is input, the entire phrase can be displayed at an appropriate position to recognize the content.

  Further, the display units 42L and 42R are mounted on the backs of the respective hands, the wrist rotation angle θ is calculated, and the display position set in advance for each rotation angle θ is calculated based on the calculated rotation angle θ. Since characters and the like are displayed, the display position is optimized according to the rotation of the hand (wrist), so that the characters and the like can always be displayed at the optimum display position.

  Furthermore, since the display position where the input code is displayed is a display position where the user can visually recognize at the corresponding rotation angle θ, even when the user's posture or the like changes, the display section 42L that can always be visually recognized. And characters can be displayed at display positions on 42R.

(III) Configuration and Operation of Character Display According to Modified Embodiment Next, the operation of the character display according to the modified embodiment of the present invention will be described with reference to FIGS.

(A) First Modification First, the display operation of the first modification according to the present invention will be described with reference to FIG. FIG. 22 is a flowchart showing the display operation according to the first modification. In FIG. 22, the same operation as the display operation according to the embodiment described with reference to FIG. 18 is denoted by the same step number, and detailed description thereof is omitted. Furthermore, since the configuration of the virtual keyboard device according to the first modification is basically the same as the configuration of the virtual keyboard device according to the embodiment, in the following description, the components of the virtual keyboard device according to the embodiment will be described. Explanation will be made using the same member numbers.

  In the above-described embodiment, as illustrated in FIG. 18, the display operation according to the embodiment is started when the input operation of a new input code is executed. On the other hand, in the first variation described below, in addition to the input operation of a new input code, the stationary state after inputting the input code (the stationary state where each finger 14 is not moving, that is, the user Is in the state of trying to confirm the input content), and the display operation is started.

  More specifically, in the display operation according to the first modification, as shown in FIG. 22A, first, as step S1, each input code output unit 28 is an input code output unit corresponding to the corresponding hand. It is monitored whether any input code is output from 28. Then, when there is no output of any input code (step S1; NO), each input code output unit 28 next confirms whether or not each finger 14 is in a state where it can be determined as a stationary state in step S10. The confirmation operation in step S10 will be described in detail later.

  When the check operation in step S10 is not in a state determined to be stationary (step S10; NO), the input code output unit 28 continues the monitoring (see step S1).

  On the other hand, when any input code is output in the confirmation operation in step S1 (step S1; YES), or in the confirmation operation in step S10, it is possible to determine a stationary state (step S10; YES). Thereafter, the input code output unit 28 and the character display unit 38 perform the operations in steps S2 to S7 similar to those in the embodiment, and terminate the display operation according to the first modification.

  Next, the stationary state confirmation operation according to step S10 will be specifically described with reference to FIG.

  In the confirmation operation according to step S10, the input code output unit 28 first acquires the current value of each acceleration data from each finger sensor 16 as step S100. Next, as step S101, the input code output unit 28 calculates the rate of change between the current acceleration data and the immediately preceding acceleration data. Specifically, for calculating the fluctuation rate, first, the maximum value and the minimum value in the acceleration data of this time are acquired during the learning operation shown in FIG. 15, and an intermediate value between these values (that is, (maximum value + minimum value)). ÷ 2) is calculated. Then, the value obtained by subtracting the current acceleration data from the previous acceleration data divided by the intermediate value (that is, (immediate acceleration data−current acceleration data) / intermediate value) is set as the variation rate in step S101. .

  Next, in step S102, the input code output unit 28 is preset and stored in the input code output unit 28 as a variation rate calculated in the operation of step S101 and a criterion for determining whether or not it is stationary. Compare thresholds.

  When the fluctuation rate is a value equal to or less than the threshold value (step S102; threshold value or less), the input code output unit 28 next determines that the current finger 14 can be determined to be stationary as step S103 (step S10). ; See YES), the process proceeds to step S2 shown in FIG. On the other hand, when the variation rate is a value larger than the threshold value (step S102; greater than the threshold value), the input code output unit 28 next assumes that the current finger 14 cannot be determined to be stationary in step S104 (step S104). S10; NO), the process proceeds to step S1 shown in FIG.

(B) Second Modification Next, the display operation of the second modification according to the present invention will be described with reference to FIG. FIG. 23 is a flowchart showing the display operation according to the second modification. Also, in FIG. 23, the same step number is assigned to the same display operation as the display operation according to the embodiment described with reference to FIG. 18 and the display operation according to the first modification described with reference to FIG. Description of is omitted. Furthermore, since the configuration of the virtual keyboard device according to the second modified embodiment is also basically the same as the configuration of the virtual keyboard device according to the embodiment, in the following description, the constituent members of the virtual keyboard device according to the embodiment It demonstrates using the same member number.

  In the above-described embodiment, the display operation according to the embodiment is started as a result of the input operation of a new input code as shown in FIG. 18, but in the second modification described below, Only the presence / absence of the stationary state after the input code input described in the first modification is detected and the display operation is started.

  More specifically, in the display operation according to the second modified embodiment, as shown in FIG. 23, first, in step S10, each input code output unit 28 performs the same stationary state confirmation operation as in the first modified embodiment.

  And when it is not in the state determined to be a still state in the confirmation operation of the step S10 (step S10; NO), the input code output unit 28 continues the confirmation operation.

  On the other hand, when it is determined that the stationary state can be determined in the confirmation operation in step S10 (step S10; YES), the input code output unit 28 and the character display 38 thereafter perform steps S2 to S7 similar to the embodiment. The operation is executed, and the display operation according to the second modification is completed.

  As described above, according to the operation of the virtual keyboard device according to the first modified embodiment or the operation of the virtual keyboard device according to the second modified embodiment, in addition to the effect of the operation of the virtual keyboard device according to the embodiment, each finger Since the display area for displaying characters and the like is changed in consideration of the 14 stationary states, the characters and the like can be displayed at the display positions on the display units 42L and 42R more appropriately.

(C) Third Modification Finally, a modification of the display unit 42L or 42R itself will be described with reference to FIG.

  In the above-described embodiments and modifications, only one of the display units 42L and 42R is mounted as the flexible display units 42L and 42R on the back portions of the left hand 10L and the right hand 10R, respectively. Explained the case. However, in addition to this, for example, as illustrated only for the left hand 10L in FIG. 24A, two or more flexible display portions 42L (42R) are attached to the back of each hand for each hand. You may comprise as follows. In this case, since the number of variations for changing the display area with respect to the visibility seen from the user increases, the display position of the input code can be determined / changed with higher visibility.

  Further, as illustrated in FIGS. 24B to 24D, when the display portions 42L and 42R made of a display device such as a liquid crystal display, which is not flexible, are mounted on the backs of the respective hands. It is also possible to apply the present invention.

  That is, for example, when one non-flexible display unit 42L (42R) is mounted as illustrated in FIG. 24B, the display unit 42L (42R) is displayed in a plurality of displays. By dividing into regions, it is possible to obtain the same operational effects as those of the above-described embodiment or each modified embodiment as long as the rotation angle θ is centered on 0 degree. For example, when two or more display portions 42L (42R) having no flexibility are mounted as illustrated in FIG. 24C or FIG. 24D, the inside of each display portion 42L (42R) is provided. By dividing the display area into a plurality of display areas, it is possible to obtain the same operational effects as those of the above-described embodiment or each modified embodiment over a wider range of the rotation angle θ.

  In addition, about the display parts 42L and 42R which concern on embodiment and each modification, it comprises so that it may mount | wear to the part close | similar to the hand in a forearm part not only the back of a hand as illustrated in FIG.17, FIG.21 or FIG.24. You can also

  Further, for each of the display units 42L and 42R, the direction of the normal line standing on the display surface in each of the display units 42L and 42R is changed from the user to the display unit 42L in each part such as a hand or a forearm to be worn. And 42R are preferably mounted so as to be in a plurality of visible directions. As a result, it is possible to display characters or the like that are easily visible in accordance with the posture of the user.

  Further, in the above-described embodiment and each modification, the case where the finger sensor 16 or the like is directly attached to the corresponding finger 14 or the back of the hand has been described, but in addition to this, gloves are applied to the left hand 10L and the right hand 10R, respectively. The corresponding finger sensor 16 or the like can be mounted on, for example, the outer side (back side of the hand) of the part corresponding to the finger 14 or the back of the hand in the glove.

  In this case, it is not necessary to wear the finger sensor 16 corresponding to each finger 14 every time the virtual keyboard device according to the embodiment or each modified form is to be operated, and the same configuration is obtained by simply wearing gloves. This is extremely convenient for the user.

  Furthermore, the programs corresponding to the flowcharts shown in FIGS. 15, 18, 22 and 23 are recorded in a recording medium such as a flexible disk or a hard disk, or obtained via a network such as the Internet, By reading and executing these with a general-purpose microcomputer, the microcomputer can be used as the input code output unit 28 and the character display 38 according to the embodiment or each modified embodiment.

  As described above, the present invention can be used in the field of input devices, and particularly when applied to the field of input devices using a virtual keyboard device, a particularly remarkable effect can be obtained.

10L Left hand 10R Right hand 14, 14-1, 14-2, 14-3, 14-4, 14-5, 14-6, 14-7, 14-8, 14-9, 14-10 Finger 15L Wrist 16, 16-1, 16-2, 16-3, 16-4, 16-5, 16-6, 16-7, 16-8, 16-9, 16-10 Finger sensor 18, 20 Data collection unit 22-1 , 22-2, 22-3, 22-4, 22-5, 22-6, 22-7, 22-8, 22-9, 22-10 harness 24, 26 radio unit 28 input code output unit 30, 40 Radio unit 32 Operation pattern dictionary unit 34 Operation pattern determination unit 42, 42L, 42R Display unit 44, 44L, 44R Character code 50 Operation signal 52 Character 54 Gravity direction 58 Display device 59 Column 60 Display screen

Claims (6)

A detecting unit that is mounted on a mounting unit provided on at least one of the user's finger or hand and detects a spatial motion of the user's finger to generate a detection signal;
Input information determination means for determining input information corresponding to the detection signal based on the detection signal from the detection means;
Display means for displaying the input information attached to the user and determined by the input information determination means;
Based on the detection signal generated by the detection unit, posture information that is information indicating a spatial posture of a hand is calculated, and the display position on the display unit associated with the posture information is Display control means for displaying the input information determined by the input information determination means;
An input device comprising: The input device according to claim 1,
The display control means displays the input information determined by the input information determination means together with the input information already displayed on the display means at the display position. The input device according to claim 1 or 2,
The display means is attached to either the hand or the forearm,
The display control means calculates any one of the rotation angles as the posture information based on the detection signal generated by the detection means, and previously calculates each rotation angle based on the calculated rotation angle. An input device that displays the input information determined by the input information determination means at the set display position. The input device according to any one of claims 1 to 3,
The input device, wherein the display position is the display position at which the user can visually recognize the input information displayed at the display position in the posture information associated with the display position. The input device according to any one of claims 1 to 4,
The display means is mounted so that directions of normals on the display surface of the input information in the display means are a plurality of directions in which the user can visually recognize the display means. Input device.   A program for an input device, which causes a computer to function as the input device according to any one of claims 1 to 5 excluding the detection means.

Images