JP2021033888A - Keyboard control program, keyboard and keyboard control system - Google Patents

Abstract

To provide a keyboard control program, a keyboard and a keyboard control system that control display of the keyboard according to a skill level of a user.SOLUTION: A keyboard 1 comprises printing display controlling means 102 that performs display controlling of printing of a key group 12 of the keyboard 1 on the basis of visual line state information calculated from visual line information 111 or visual line information 111 which is obtained by detecting a visual line VL of a user 3 operating the keyboard 1 when an application program 210 of PC 2 displays content instructing operation of the keyboard 1 on a display unit 22.SELECTED DRAWING: Figure 1

Description

The present invention relates to keyboard control programs, keyboards and keyboard control systems.

As a conventional technique, a keyboard control program that assists input by indicating a key to be operated next to a keyboard as a typing practice has been proposed (see, for example, Patent Document 1).

The keyboard control program disclosed in Patent Document 1 displays a character string for typing practice on the display unit of a computer, and the user next operates the keyboard provided with an LED on the key top of each key. Control the LED of the key to be turned on to encourage the user's operation.

Japanese Unexamined Patent Publication No. 2003-9230

However, the keyboard control program of Patent Document 1 described above controls the lighting of the LED of the key that the user should operate next by lighting the key, and guides the user's line of sight to the keyboard. , It was not always suitable for practicing the typing method (hereinafter referred to as "touch typing") in which typing is performed without shifting the line of sight to the keyboard. In other words, it is useful for users who do not remember the keyboard layout to remember the layout, but it hinders touch typing practice for users who remember the keyboard layout, and it depends on the user's proficiency level. It could not be said that it controlled the display of the keyboard.

An object of the present invention is to provide a keyboard control program, a keyboard, and a keyboard control system that control a keyboard display according to a user's proficiency level.

One aspect of the present invention provides the following keyboard control program, keyboard and keyboard control system in order to achieve the above object.

[1] Keyboard
When the application displays a display of the content instructing the operation of the keyboard on the display unit, the line-of-sight information obtained by detecting the line-of-sight of the user who operates the keyboard or the line-of-sight state information calculated from the line-of-sight information is used. A keyboard control program that functions as a print display control means for controlling the display of printing on the keyboard based on the above.
[2] When the user operates the keyboard according to the display content of the application, the print display control means directs the user's line of sight toward the keyboard from the line-of-sight information or the line-of-sight state information. The keyboard control program according to the above [1], which controls the display so that the print on the keyboard is not displayed when it is determined that the keyboard is printed.
[3] When the user operates the keyboard according to the display content of the application, the print display control means directs the user's line of sight toward the keyboard from the line-of-sight information or the line-of-sight state information. The keyboard control program according to the above [1], which controls the display so as to emphasize the printing of the keys of the keyboard to be operated next when it is determined that the keyboard is printed.
[4] Measurement information obtained by measuring the operation signal received from the keyboard with respect to the display when the line-of-sight information or the line-of-sight state information and a display of contents instructing the operation of the keyboard by the application are displayed on the display unit. Further function as an evaluation means to calculate the evaluation value based on
The keyboard control program according to any one of [1] to [3], which switches the operation of the print display control means based on the evaluation value calculated by the evaluation means.
[5] When the application displays a display of the content instructing the operation of the keyboard on the display unit, the line-of-sight information obtained by detecting the line-of-sight of the user who operates the keyboard or the line-of-sight calculated from the line-of-sight information. A keyboard having a print display control means for displaying and controlling the print of the keyboard based on state information.
[6] An information processing device that executes an application and controls the display of the execution contents of the application on the display unit.
It has a keyboard that outputs operation signals according to the operation to the information processing device and controls the display contents of key printing.
Whether or not the line of sight of the user who operates the keyboard is directed toward the keyboard when the application of the information processing device displays a display of the content instructing the operation of the keyboard on the display unit. A keyboard control system that displays and controls the printing of the keyboard based on the content of the line of sight of the user and the measurement information measured when the keyboard is operated according to the display content of the application.

According to the invention according to claim 1, 5 or 6, the display of the keyboard can be controlled according to the proficiency level of the user.
According to the invention of claim 2, when the user operates the keyboard according to the display content of the application, if the user's line of sight is toward the keyboard, the display is controlled so as not to display the print on the keyboard. be able to.
According to the invention of claim 3, when the user operates the keyboard according to the display content of the application, if the user's line of sight is toward the keyboard, the key of the keyboard to be operated next is printed. The display can be controlled so as to be displayed.
According to the invention of claim 4, the evaluation value can be calculated based on the content of the line of sight of the user and the measurement information, and the operation of the print display can be switched based on the calculated evaluation value.

FIG. 1 is a schematic view showing an example of the configuration of the keyboard control system according to the embodiment. FIG. 2 is a block diagram showing a configuration example of the keyboard according to the embodiment. FIG. 3 is a block diagram showing a configuration example of a PC according to the embodiment. FIG. 4 is a configuration diagram showing information communicated by the keyboard and the PC. FIG. 5 is a flowchart showing an example of the mode selection operation of the keyboard. 6 (a)-(c) are plan views of a keyboard for explaining an example of support mode operation. 7 (a)-(c) are plan views of a keyboard for explaining a modified example of the support mode operation. FIG. 8 is a flowchart showing an example of keyboard support mode operation. 9 (a)-(c) are plan views of a keyboard for explaining an example of the penalty mode operation. 10 (a)-(c) are plan views of a keyboard for explaining a modified example of the support mode operation. FIG. 11 is a flowchart showing an example of keyboard support mode operation.

[Embodiment]
(Keyboard control system configuration)
FIG. 1 is a schematic view showing an example of the configuration of the keyboard control system according to the embodiment.

This keyboard control system is configured by connecting a keyboard 1 and a PC (Personal Computer) 2 so as to be able to communicate with each other by a USB (Universal Social Bus) cable, wireless communication, or the like.

The PC 2 includes electronic components such as a CPU and a flash memory having a function for processing information in the main body, and also includes a display unit 22 such as an LCD (Liquid Crystal Display) that displays characters, images, and the like.

The keyboard 1 outputs an operation signal to the PC 2 according to the operation content of the user 3, and obtains line-of-sight information obtained from the line-of-sight VL (VLd and VLk) and / or information based on the line-of-sight information (details will be described later). The display on the keys is changed according to the situation, and the main body is equipped with electronic parts such as a CPU (Central Processing Unit) and a flash memory having a function for processing information, and is composed of a plurality of keys. The projector 13 as a print display means for displaying characters and images on each of the key group 12 operated by the hands 31l and 31r of the 3rd hand and the keys of the key group 12, and the line-of-sight detection device for detecting the line-of-sight VL of the user 3. The line-of-sight detection camera 14 and the arm motion detection camera 15 as an arm motion detection device for detecting the motions of the hands 31l and 31r of the user 3 are provided. A part or all of the projector 13, the line-of-sight detection camera 14, and the arm motion detection camera 15 included in the keyboard 1 may be provided in the PC 2 or may be used in combination with the PC 2, or may be configured as an independent device. You may. Further, the projector 13 may be provided in the PC 2 or may be configured to be used in combination with the PC 2, or may be configured as an independent device.

As an example, the keyboard 1 evaluates the skill level of the user 3 regarding keyboard operation, calculates an evaluation value, and uses the evaluation value and the line-of-sight information obtained from the line-of-sight VL of the user 3 and / or information based on the line-of-sight information. The display / non-display of characters and images on the key group 12 is changed accordingly.

(Keyboard configuration)
FIG. 2 is a block diagram showing a configuration example of the keyboard 1 according to the embodiment.

The keyboard 1 includes a control unit 10 composed of a CPU and the like to control each unit and execute various programs, a storage unit 11 composed of a storage medium such as a flash memory and storing information, and a key group 12. It includes a projector 13, a line-of-sight detection camera 14, an arm motion detection camera 15, and a communication unit 16 that communicates with the PC 2.

By executing the keyboard control program 110 described later, the control unit 10 includes the typing detection means 100, the operation signal output means 101, the print display control means 102, the line-of-sight detection means 103, the arm movement detection means 104, the application communication means 105, and the like. It functions as a typing evaluation means 106 and the like.

The typing detecting means 100 detects an operation on the key group 12.

The operation signal output means 101 outputs an operation signal to the PC 2 via the communication unit 16 based on the operation content detected by the typing detection means 100.

The print display control means 102 displays characters or images on the key group 12 based on at least one of the line-of-sight information 111, the line-of-sight state information calculated from the line-of-sight information 111, the arm movement information 112, and the application measurement information 113. To control.

The line-of-sight detection means 103 records the line-of-sight VL of the user 3 detected by the line-of-sight detection camera 14 together with the time series as line-of-sight information 111.

The arm movement detecting means 104 records the movements of the hands 31l and 31r of the arm movement detecting camera 15 user 3 together with time series as arm movement information 112.

The application communication means 105 communicates with the typing application (210, FIG. 3) operating on the PC 2 via the communication unit 16, acquires the application measurement information (211 and FIG. 3) measured by the typing application 210, and measures the application. It is stored in the storage unit 11 as information 113.

The typing evaluation means 106 has a proficiency level regarding the operation of the keyboard 1 of the user 3 based on at least one of the line-of-sight information 111, the line-of-sight state information calculated from the line-of-sight information 111, the arm movement information 112, and the application measurement information 113. Is evaluated and stored in the storage unit 11 as evaluation information 114.

The storage unit 11 includes a keyboard control program 110 that operates the control unit 10 as each of the above-mentioned means 100-106, line-of-sight information 111, line-of-sight state information calculated from the line-of-sight information 111, arm movement information 112, and application measurement information 113. The evaluation information 114 and the like are stored.

(PC configuration)
FIG. 3 is a block diagram showing a configuration example of the PC 2 according to the embodiment.

The PC 2 is composed of a CPU and the like, a control unit 20 that controls each unit and executes various programs, a storage unit 21 that is composed of a storage medium such as a flash memory and stores information, a display unit 22, and a keyboard. A communication unit 23 that communicates with 1 is provided.

By executing the typing application 210 described later, the control unit 20 functions as a problem display means 200, an operation signal input means 201, an operation completion time measuring means 202, a score measuring means 203, a keyboard communication means 204, and the like.

The question display means 200 displays a character string that can be input by the keyboard 1 on the display unit 22 as a question for typing practice. Images and the like may be included in the displayed contents as necessary.

The operation signal input means 201 receives an operation signal input from the keyboard 1 via the communication unit 23.

The operation completion time measuring means 202 measures the operation completion time from when the problem display means 200 displays the problem on the display unit 22 until all the operation signals are input to the problem, and the application measurement information 211. Record in.

The score measuring means 203 measures the score from the correctness and the correctness ratio of the operation signal for the problem, and records it in the application measurement information 211.

The keyboard communication means 204 communicates with the keyboard 1 and transmits the application measurement information 211 in response to the request of the keyboard 1.

The storage unit 21 stores the typing application 210, the application measurement information 211, and the like that cause the control unit 20 to operate as the above-mentioned means 200-204.

FIG. 4 is a configuration diagram showing information communicated by the keyboard 1 and the PC 2.

The keyboard 1 functions as the application communication means 105 by executing the keyboard control program 110, and the application communication means 105 communicates with the PC 2 via the communication unit 16 and mainly outputs an operation signal. The PC 2 functions as a keyboard communication means 204 by executing the typing application 210, and the keyboard communication means 204 communicates with the keyboard 1 via the communication unit 23 and mainly outputs application measurement information.

A part or all of the print display control means 102, the line-of-sight detection means 103, the arm movement detection means 104, and the typing evaluation means 106 of the keyboard 1 may be made to function on the PC2, or the problem display means 200 of the PC2. , A part or all of the operation completion time measuring means 202 and the score measuring means 203 may be made to function by the keyboard 1. Further, all means may be integrated in either the keyboard 1 or the PC 2. That is, the keyboard control program 110 and the typing application 210 cooperate with each other, and in the modified example, one may include a part or all of the other.

(Keyboard operation)
Next, the operation of this embodiment will be described separately for (1) basic operation, (2) mode selection operation, (3) support mode operation, and (4) penalty mode operation.

(1) Basic Operation First, the print display control means 102 of the keyboard 1 controls the projector 13 and controls the display of predetermined characters and images on the key tops of the key group 12 as an initial state.

User 3 operates the keyboard 1 to practice typing. The typing detection means 100 of the keyboard 1 detects an operation on the key group 12, and the operation signal output means 101 outputs an operation signal to the PC 2 according to the operation content of the user 3. The PC 2 executes the typing application 210 based on the operation signal input from the keyboard 1.

By executing the typing application 210, the PC 2 operates as a question display means 200, and displays a character string on the display unit 22 as a question for typing practice.

The user 3 confirms the character string for typing practice displayed on the display unit 22, and operates the keyboard 1 to input the displayed character string. At this time, in general, if the user 3 has a low level of proficiency in typing (mainly touch typing), the line-of-sight VLk tends toward the keyboard 1, and if the level of proficiency is high, the line-of-sight VLd points toward the display unit 22. Will increase. The keyboard control program 110 and the typing application 210 operate as described below in order to decrease the frequency of the line-of-sight VLk and increase the frequency of the line-of-sight VLd according to the proficiency level.

(2) Mode selection operation FIG. 5 is a flowchart showing an example of the mode selection operation of the keyboard 1.

First, the print display control means 102 refers to the evaluation information 114, and when the evaluation value does not exist (S1; Yes), the print display control means 102 is set to the “support mode” as the operation mode (S2). Here, the case where the evaluation value does not exist is the case where the user 3 operates the keyboard 1 and the PC 2 for the first time. In addition, the "support mode" reduces the frequency, staying time, moving speed, moving acceleration, moving direction, etc. of the line-of-sight VLk from the line-of-sight VLk to the line-of-sight VLk of the user 3 with low proficiency, and the frequency of the line-of-sight VLd and stay This is an operation mode for increasing the moving speed, moving acceleration, moving direction, etc. from the time, the line of sight VLk to the line of sight VLd. Further, the operation mode may be used to encourage the hands 31l and 31r to return to the home position after the key operation.

Next, the problem display means 200 of the PC 2 displays the problem as described in "(1) Basic operation", and the user 3 operates the keyboard 1. The keyboard 1 and the PC 2 operate as described below to evaluate the proficiency level of the user 3 (S3).

The operation completion time measuring means 202 of the PC 2 measures the operation completion time from when the problem display means 200 displays the problem on the display unit 22 until all the operation signals are input to the problem, and measures the application. Record in information 211. It is evaluated that the shorter the operation completion time, the higher the proficiency level.

Further, the score measuring means 203 of the PC 2 measures the score from the correctness and the correctness ratio of the operation signal for the problem, and records it in the application measurement information 211. The higher the percentage of correct answers, the higher the proficiency level.

The keyboard communication means 204 of the PC 2 transmits the application measurement information 211 recorded by the operation completion time measuring means 202 and the score measuring means 203 to the keyboard 1 in response to the request of the application communication means 105 of the keyboard 1. The transmission timing of the application measurement information 211 may be for each predetermined time, for each unit of the question to be asked by the question display means 200, or for each predetermined number of questions. The application communication means 105 acquires the application measurement information 211 and stores it in the storage unit 11 as the application measurement information 113.

Further, the line-of-sight detecting means 103 of the keyboard 1 detects the line-of-sight VL of the user 3 and records it as the line-of-sight information 111 together with the time series. The typing evaluation means 106 refers to a position having a line of sight from the line-of-sight information 111 and the position information of the keyboard (assuming that the relative position with the line-of-sight detection camera 14 is registered in advance) or only from the line-of-sight information 111. It is evaluated from the staying time, the moving speed of the line of sight, the acceleration of the line of sight, any or any combination of the moving directions of the line of sight, or the rate of change thereof (hereinafter referred to as "line-of-sight state information"). It may be evaluated that the longer the VL is the line-of-sight VLd to the display unit 22, the higher the proficiency level.

The arm motion detecting means 104 of the keyboard 1 detects the motions of the hands 31l and 31r of the user 3 and records them as arm motion information 112 together with the time series. The typing evaluation means 106 uses the arm movement information 112 and the keyboard position information (assuming that the relative position with the arm movement detection camera 15 is registered in advance), or only the line-of-sight information 111, the hand 31l and the hand 31l. It is evaluated from the staying time, movement speed, acceleration, any or any combination of movement directions, the rate of change of these, etc. (hereinafter referred to as "arm movement state information") with respect to a certain position of 31r, but for example, simply by hand. It may be evaluated that 31l and 31r are in the correct home position, and the higher the ratio of operating with the correct finger, the higher the proficiency level.

Next, the typing evaluation means 106 is proficient in operating the keyboard 1 of the user 3 based on at least one information of the line-of-sight information 111 and the arm movement information 112, or the line-of-sight state information and the arm movement state information and the application measurement information 113. The degree is evaluated (S3) and stored in the storage unit 11 as evaluation information 114.

The print display control means 102 refers to the evaluation information 114 (S4), and when the evaluation value is equal to or higher than a predetermined threshold value (S4; Yes), the print display control means 102 is set to "penalty mode" as the operation mode. (S5). The "penalty mode" further reduces the frequency, staying time, moving speed, moving acceleration, moving direction, etc. of the line-of-sight VLk from the line-of-sight VLd to the line-of-sight VLk of the highly proficient user 3, and the frequency and staying time of the line-of-sight VLd. , This is an operation mode for further increasing the moving speed, moving acceleration, moving direction, etc. from the line of sight VLk to the line of sight VLd. Further, the operation mode may be used to reduce that the hands 31l and 31r do not return to the home position after the key operation.

When the evaluation value is smaller than the predetermined threshold value (S4; No), the operation mode of the print display control means 102 is set to "support mode" (S5).

Next, the operation of the print display control means 102 in each mode will be described.

(3) Support Mode Operation FIG. 8 is a flowchart showing an example of the support mode operation of the keyboard 1. 6 (a)-(c) are plan views of the keyboard 1 for explaining an example of the support mode operation.

First, in the support mode, as shown in FIG. 6A, the print display control means 102 of the keyboard 1 controls the projector 13 so as not to display the characters and images of the key tops of the key group 12 in the initial state. Display control. This is a display control for preventing the user 3 from looking at the keyboard 1 as much as possible.

Next, the question display means 200 of the PC 2 displays a character string on the display unit 22 as a question for typing practice. At the same time, the application communication means 105 of the keyboard 1 receives the text information of the problem sentence from the PC 2 (S10).

The print display control means 102 of the keyboard 1 controls one character at a time from the first character of n = (S11), and as shown in FIG. 6B, the line of sight detecting means 103 is the line of sight VLk1 of the user 3 and the keyboard 1 is the line of sight VLk1. (S12; Yes), the key of the nth character to be input, for example, "S" is displayed and controlled as the character 130s on the key 120s (S13).

Further, when the line-of-sight VLd of the user 3 is the display unit 22 and is not on the keyboard 1 (S12; No), "S" is not displayed as the character 130s on the nth character key to be input, for example, the key 120s. Display control (S14). However, if the input is not completed for a certain period of time (S15; No), "S" is displayed and controlled as the character 130s on the key 120s (S13). In the display control of displaying "S" as the character 130s on the key 120s without displaying the character or image of the key top of the key group 12 as the initial state, the display of the key 120s is emphasized with respect to other keys. Therefore, for example, the characters and images of the key tops of the key group 12 are normally displayed in the initial state, and the characters 130s are displayed brighter on the keys 120s or highlighted in different colors. You may do.

If the above operation is not completed for inputting all of the question sentences (S16; No), the next character n = n + 1th character is also performed (S17), and the operation is performed until all of the problem sentences are input (S16; Yes). ).

Further, the support mode may operate as described in the following modification.

7 (a)-(c) are plan views of the keyboard 1 for explaining a modified example of the support mode operation.

First, as shown in FIG. 7A, the print display control means 102 stores in advance the keys that the user 3 frequently inputs while pointing the line-of-sight VLk2 toward the keyboard 1 as the weak keys 120q.

Next, the print display control means 102 of the keyboard 1 controls the projector 13 and controls the display so as not to display the characters and images of the key tops of the key group 12 in the initial state.

Next, the question display means 200 of the PC 2 displays a character string on the display unit 22 as a question for typing practice. At the same time, the application communication means 105 of the keyboard 1 receives the text information of the problem sentence from the PC 2.

The print display control means 102 of the keyboard 1 controls character by character in order from the first character of n =, and as shown in FIG. 7B, when the question key which is the nth character is the weak key 120q, FIG. As shown in 7 (c), "Q" is displayed and controlled as the character 130q on the key 120q, which is the nth character to be input. As an initial state, the characters and images of the key tops of the key group 12 may be normally displayed, and the characters 130q may be displayed brighter on the keys 120q or highlighted by displaying them in different colors.

(4) Penalty mode operation
FIG. 11 is a flowchart showing an example of the support mode operation of the keyboard 1. 9 (a)-(c) are plan views of the keyboard 1 for explaining an example of the penalty mode operation.

First, in the penalty mode, as shown in FIG. 9A, the print display control means 102 of the keyboard 1 controls the projector 13 and displays all the characters and images of the key tops of the key group 12 in the initial state. Display control. This is a display control on the premise that the user 3 does not look at the keyboard 1.

Next, the question display means 200 of the PC 2 displays a character string on the display unit 22 as a question for typing practice. At the same time, the application communication means 105 of the keyboard 1 receives the text information of the problem sentence from the PC 2 (S20).

The print display control means 102 of the keyboard 1 controls one character at a time from the first character of n = (S21), and as shown in FIG. 9B, the line of sight detecting means 103 is the line of sight VLk3 of the user 3 and the keyboard 1 is the line of sight VLk3. (S22; Yes), the display of the nth character key to be input, for example, the display of "D" which is the character 130d of the key 120d is turned off (S23). This is done as a penalty for not being able to touch-type. The character whose display has been turned off may be displayed as the next character, or may be displayed after a certain period of time has elapsed. The display may be controlled so that the character 130d of the key 120d is displayed darker or displayed in a different color so that it is difficult to see.

Further, when the line-of-sight VLd of the user 3 is the display unit 22 and is not on the keyboard 1 (S22; No), the nth key to be input, for example, the display of "D" which is the character 130d of the key 120d is displayed. The display is continuously controlled (S24). However, if the input is not completed for a certain period of time (S25; No), the display of "D", which is the character 130d of the key 120d, is turned off and controlled (S23).

If the above operation is not completed for inputting all of the question sentences (S26; No), the next character n = n + 1th character is also performed (S27), and the operation is performed until all of the problem sentences are input (S26; Yes). ).

Further, the penalty mode may operate as described in the following modification.

10 (a)-(c) are plan views of the keyboard 1 for explaining a modified example of the support mode operation.

First, as shown in FIG. 10A, the print display control means 102 of the keyboard 1 controls the projector 13 and controls the display so as to display all the characters and images of the key tops of the key group 12 in the initial state.

Next, the question display means 200 of the PC 2 displays a character string on the display unit 22 as a question for typing practice. At the same time, the application communication means 105 of the keyboard 1 receives the text information of the problem sentence from the PC 2.

The print display control means 102 of the keyboard 1 controls character by character in order from the first character of n =, and as shown in FIG. 10B, the line-of-sight detection means 103 has the line-of-sight VLk3 of the user 3 on the keyboard 1. When the key is detected and the key is operated with the finger 310l1, as shown in FIG. 10C, the input nth key, for example, the key 120d and the keys 120e, 120s, 120f and 120c around the key 120d The display of the character of is turned off. This is done as a penalty for not being able to touch-type, and also turns off the display of the surrounding characters that seem to have confirmed printing. The range for turning off the display may be wider. The character whose display has been turned off may be displayed as the next character, or may be displayed after a certain period of time has elapsed. The display may be controlled so that the keys 120d and the keys 120e, 120s, 120f and 120c around the key 120d are displayed darker or displayed in different colors to make them difficult to see.

(Effect of embodiment)
According to the above-described embodiment, the user 3 is proficient in operating the keyboard 1 based on at least one information of the line-of-sight information 111 and the arm movement information 112, or the line-of-sight state information and the arm movement state information and the application measurement information 113. The degree is evaluated, and the operation mode of the print display control means 102 of the keyboard 1 can be selected from the "support mode" or the "penalty mode" based on the evaluation result. In the "support mode", the print is displayed only in the necessary situations. In the "penalty mode", printing is turned off depending on the situation and a penalty is imposed when touch typing cannot be performed. Therefore, the user 3 depends on the proficiency level of the user 3. The display of the keyboard 1 can be controlled so that the touch typing of the keyboard 1 is improved.

Further, since the typing application 210 and the keyboard control program 110 are configured to be able to communicate with each other, the specifications of the typing application 210 can be freely changed by matching the specifications of the keyboard control program 110 of the keyboard 1. As long as the basic specifications of the keyboard control program 110 are common, the specifications may be freely changed.

[Other embodiments]
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, the mode is not limited to two, "support mode" and "penalty mode", and three or more modes may be prepared and switched according to the evaluation value. Further, instead of the evaluation value, an arbitrary feature amount may be calculated from the line of sight of the user 3 and the operation content, and the mode may be switched based on the feature amount, or the print content of the key may be switched without using the mode. May be changed.

In addition to turning on / off the printing of the key, the printed content may be changed based on the line-of-sight VL of the user 3. The printed content may be any content that can be assigned to keys such as characters and symbols in a normal QWETY array, functions when a function key is pressed, characters and symbols when a shift key is pressed, and shortcut key contents.

Further, although the projector 13 has been described as an example of the print display device, any device that can change the print display of the keyboard based on the control content of the print display control means 102 is sufficient, and has, for example, a display function. You may use a printed key top.

Further, although the line-of-sight detection camera 14 has been described as an example of the line-of-sight detection device, any device that can detect the line-of-sight and transmit the line-of-sight position information to the line-of-sight detection means 103 may be used. May be adopted.

Further, although the arm operation detection camera 15 has been described as an example of the arm motion detection device, any device capable of detecting the motions of the hands 31l and 31r and transmitting the arm motion motion information 112 to the arm motion detection means 104. For example, a motion capture glove, a finger cot with a sensor, or the like may be adopted.

In the above embodiment, the functions of the means 100-106 and 200-204 of the control unit 10 are realized by a program, but all or a part of each means may be realized by hardware such as ASIC. Further, the program used in the above embodiment can be stored in a recording medium such as a CD-ROM and provided. Further, the replacement, deletion, addition, etc. of the above steps described in the above embodiment can be performed without changing the gist of the present invention.

1: Keyboard 2: PC
3: User 10: Control unit 11: Storage unit 12: Key group 13: Projector 14: Line-of-sight detection camera 15: Arm motion detection camera 16: Communication unit 20: Control unit 21: Storage unit 22: Display unit 23: Communication unit 31l, 31r: Hand 100: Typing detection means 101: Operation signal output means 102: Print display control means 103: Line-of-sight detection means 104: Arm motion detection means 105: Application communication means 106: Typing evaluation means 110: Keyboard control program 111: Line-of-sight information 112: Arm movement information 113: Application measurement information 114: Evaluation information 200: Problem display means 201: Operation signal input means 202: Operation completion time measuring means 203: Score measuring means 204: Keyboard communication means 210: Typing application 211: Application measurement information

Claims (6)

Keyboard,
When the application displays a display of the content instructing the operation of the keyboard on the display unit, the line-of-sight information obtained by detecting the line-of-sight of the user who operates the keyboard or the line-of-sight state information calculated from the line-of-sight information is used. A keyboard control program that functions as a print display control means for controlling the display of printing on the keyboard based on the above. When the user operates the keyboard according to the display content of the application, the print display control means determines that the user's line of sight is directed toward the keyboard from the line-of-sight information or the line-of-sight state information. The keyboard control program according to claim 1, wherein when it is determined, the display is controlled so that the print on the keyboard is not displayed. When the user operates the keyboard according to the display content of the application, the print display control means determines that the user's line of sight is directed toward the keyboard from the line-of-sight information or the line-of-sight state information. The keyboard control program according to claim 1, wherein when it is determined, the display is controlled so that the printing of the key of the keyboard to be operated next is emphasized and displayed. Based on the line-of-sight information or the line-of-sight state information, and measurement information obtained by measuring an operation signal received from the keyboard with respect to the display when the display of the content instructing the operation of the keyboard is displayed on the display unit. To further function as an evaluation means for calculating evaluation values
The keyboard control program according to any one of claims 1-3, which switches the operation of the print display control means based on the evaluation value calculated by the evaluation means. When the application displays a display of the content instructing the operation of the keyboard on the display unit, the line-of-sight information obtained by detecting the line-of-sight of the user who operates the keyboard or the line-of-sight state information calculated from the line-of-sight information is used. A keyboard having a print display control means for displaying and controlling the print of the keyboard based on the above. An information processing device that executes an application and controls the display of the execution contents of the application on the display unit.
It has a keyboard that outputs an operation signal according to the operation to the information processing device and controls the display contents of key printing.
Whether or not the line of sight of the user who operates the keyboard is directed toward the keyboard when the application of the information processing device displays a display of the content instructing the operation of the keyboard on the display unit. A keyboard control system that displays and controls the printing of the keyboard based on the content of the line of sight of the user and the measurement information measured when the keyboard is operated according to the display content of the application.

Images