JP2015176277A - key input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a key input device capable of reducing the number of operations for selecting a target key from among a plurality of keys.SOLUTION: A key input device for inputting any of a plurality of keys grouped into a plurality of hierarchies includes: a sensor data acquisition part for acquiring selecting operation information corresponding to a plurality of mutually different selecting operations corresponding to the number of hierarchies; a control part for determining which of the plurality of keys has been selected on the basis of the selecting operation information acquired by the sensor data acquisition part; and an execution processing part for executing key input processing on the basis of the determination result of the control part.

Description

  The present invention relates to a key input device that inputs one of a plurality of keys based on an operation of an operator.

  Conventionally, a key input device that can be operated only with operation data such as the position and posture of a hand is difficult to install input hardware such as a keyboard and a mouse, and is an effective technology as an input device for devices such as digital home appliances. In addition, when a key must be repeatedly selected from a large number of keys, such as character input, it is also important to devise a key display method so that the target key can be found quickly.

  For example, Patent Document 1 and Patent Document 2 disclose conventional techniques for performing an input operation from hand movement data. In Patent Document 1, an operator is compared with a virtual QWERTY keyboard in which a so-called QWERTY keyboard in which alphabets are arranged in the order of Q, W, E, R, T, Y from the left of the uppermost alphabet is projected on a desk. An apparatus for performing an input operation and determining an input key from the position and movement of a fingertip at that time is disclosed. Patent Document 2 discloses an apparatus for selecting an item by twisting a wrist for a pie menu in which menu items are arranged in a circle.

  As a technique for simplifying key access by grouping a large number of keys and icons into a hierarchical structure and partially expanding in accordance with the operation of the operator, for example, Patent Document 3 and Patent Document 4 are known. Are disclosed. Patent Document 3 discloses a device in which information related to an icon is dynamically expanded and displayed when an icon is dragged and dropped onto an empty area of a display unit on a computer connected to a mouse. Yes. Patent Document 4 discloses that a device having a touch screen detects a tap or flick action, and displays a key related to the key at the position where the action has occurred in a superimposed manner. An apparatus is disclosed in which the number of candidates to be displayed on one screen is reduced and the number of times of screen switching is reduced to allow quick input.

Japanese Patent No. 48468871 Japanese Patent No. 4489825 JP 2003-233447 A JP 2011-233051 A

  The technique disclosed in Patent Document 1 is to input a keyboard having the same QWERTY layout as that of a conventional hardware keyboard on a desk, and is an input method that is familiar to an operator. There is a problem that is limited. Also, both hands are required for comfortable input, and high-speed input with one hand is difficult.

  In addition, the technique disclosed in Patent Document 2 performs two types of operations, item selection and execution, each time the menu is expanded to a lower level in order to perform a selection operation from a large number of items in a hierarchical pie menu. Must be repeated and another action must be performed to return to the menu one level above. Also, an operation for quickly moving up and down the hierarchy with only one type of operation is not disclosed, and it is not suitable for continuous key input.

  In addition, since the techniques disclosed in Patent Document 1 and Patent Document 2 both select items according to the position of the operator's hand and the twist angle of the wrist, the operator maintains a predetermined position and posture. You have to do another action to execute the selected item. This causes an increase in erroneous input and lowers the reliability of operation in an operating environment without a physical guide.

  On the other hand, Patent Document 3 and Patent Document 4, which are techniques for efficiently accessing items by hierarchizing items, are based on the assumption that a hardware mouse, a touch panel, or the like is mounted, and even in a non-contact environment where there is no event such as click or touch. It is not disclosed that it can be used.

  The present invention has been made, for example, to solve the above-described problems, and can reduce the number of operations for selecting a target key from a plurality of keys even in a non-contact environment. An object of the present invention is to provide a possible key input device.

  The key input device according to the present invention is a key input device for inputting any one of a plurality of keys grouped in a plurality of hierarchies. Selection operation information corresponding to a plurality of different selection operations according to the number of hierarchies is provided. A sensor data acquisition unit to be acquired, a control unit that determines which of a plurality of keys has been selected based on selection operation information acquired by the sensor data acquisition unit, and a key input based on a determination result of the control unit And an execution processing unit that executes processing.

  According to the key input device of the present invention, for a plurality of keys grouped in a plurality of layers, selection operation information corresponding to a plurality of different selection operations according to the number of layers is acquired, and the acquired selection operation information By determining which of the plurality of keys has been selected based on the above, it is possible to reduce the number of operations for selecting a target key from among the plurality of keys.

2 is a configuration diagram illustrating an example of a configuration of a key input device according to Embodiment 1. FIG. 3 is a functional block diagram illustrating an example of a functional configuration of a key input device according to Embodiment 1. FIG. 3 is a flowchart illustrating an example of an operation flow of the key input device according to the first embodiment. 6 is an explanatory diagram illustrating an example of a virtual object displayed by the key input device according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of a change in a virtual object displayed by the key input device according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of a change in a first hierarchy of a virtual object displayed by the key input device according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of a change in a second hierarchy of a virtual object displayed by the key input device according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of a change in a virtual object displayed by the key input device according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of a change in a virtual object displayed by the key input device according to Embodiment 1. FIG. 10 is an explanatory diagram illustrating an example of a virtual object displayed by a key input device according to Embodiment 2. FIG. 10 is an explanatory diagram illustrating an example of a virtual object displayed by a key input device according to Embodiment 3. FIG.

  Embodiments of the present invention will be described below. The following embodiment is an example of the present invention, and the present invention is not limited to the following embodiment.

Embodiment 1 FIG.
FIG. 1 is a configuration diagram showing an example of a configuration of a key input device according to Embodiment 1 showing an embodiment of the present invention.
In FIG. 1, a sensor 10 acquires position data indicating the position of a part of the human body and shape data indicating the shape of a part of the human body. For example, position data and shape data of a part of a human body such as a hand or a foot are acquired by infrared rays or image data.
The memory 11 stores a program or data for realizing each function of the key input device described later. For example, the memory 11 is a ROM (Read Only Memory), a HDD (HARD Disk Drive), an SSD (Solid State Drive), or the like. Composed.

A CPU (Central Processing Unit) 12 reads data acquired by the sensor 10 and a program or data stored in the memory 11 to realize each function of a key input device to be described later, and issues a display command to the display device 13. Output.
The display device 13 presents an image based on a display command from the CPU 12. For example, it is composed of a head-mounted display, a liquid crystal display, a tablet, etc., and provides an image that is visible to the operator.

FIG. 2 is a functional block diagram showing an example of a functional configuration of the key input device according to the first embodiment showing an embodiment of the present invention. In FIG. 2, the key input device includes a sensor data acquisition unit 20, a control unit 21, a display unit 22, and an execution processing unit 23.
The sensor data acquisition unit 20 acquires motion information of a part of the human body based on the data from the sensor 10. In the present embodiment, a posture acquisition unit 200, a translational displacement amount acquisition unit 201, and a rotational displacement amount acquisition unit 202 are provided.

  The posture acquisition unit 200 acquires shape data from the sensor 10 and acquires posture information corresponding to the posture of a part of the human body. In the present embodiment, posture information corresponding to the posture of a human hand such as “ken” “goo” or “par” is detected.

  The translation displacement amount acquisition unit 201 acquires position data from the sensor 10 and detects a displacement amount per unit time due to partial translation of a part of the human body in the three-dimensional directions of the horizontal, vertical, and depth directions. In the present embodiment, the movement operation of the palm is detected.

  The rotational displacement amount acquisition unit 202 acquires position data and shape data of the sensor 10 and detects a rotation amount and a rotation direction for each unit time of rotation of a part of the human body. In the present embodiment, wrist twisting motion is detected.

  In the present embodiment, a plurality of keys are grouped into a plurality of hierarchies, and when one of a plurality of groups in each hierarchy is selected and a target key is selected in the last hierarchy, a plurality of keys are selected. A specific key can be selected from among the keys. That is, a plurality of groups in each hierarchy or a plurality of keys in the last hierarchy are set as selection items corresponding to the respective hierarchy, and when any of the plurality of selection items is selected in each hierarchy, From among them, for example, a unique key is specified.

  Here, the selection operation for selecting one of the plurality of selection items is set for each layer, and the selection operations are set to be different from each other. In the present embodiment, a plurality of keys are grouped into three layers, and the selection operation for selecting a selection item in the first layer is performed so that the palm is parallel to the surface of the body (chest and abdomen). When the arm is open, the arm is extended or contracted forward, and the palm is moved away from or close to the body (chest or abdomen) (movement operation in the depth direction). The selection operation for selecting a selection item in the second hierarchy is to open the palm so that it is parallel to the surface of the body (chest and abdomen) and move the arm upward, downward, leftward, rightward, etc. It is set to an operation (moving operation in the plane direction) that moves the palm in a direction parallel to the surface of the body (chest or abdomen). The selection operation for selecting a selection item in the third hierarchy is set to an operation (rotation operation) of twisting the wrist to the left and right with the palm open so as to be parallel to the surface of the body (chest and abdomen). .

  In the present embodiment, the translation displacement amount acquisition unit 201 selects the selection operation information corresponding to the movement operation in the depth direction, which is the selection operation for selecting the selection item in the first hierarchy, and the selection item in the second hierarchy. Selection operation information corresponding to the movement operation in the planar direction, which is a selection operation for selecting, and the selection corresponding to the rotation operation, which is the selection operation for selecting the selection item in the third hierarchy, by the rotational displacement amount acquisition unit 202 Get operation information.

  The control unit 21 determines which of the plurality of keys has been selected based on the selection operation information acquired by the sensor data acquisition unit 20. In the present embodiment, each layer corresponds to each layer based on at least one of the movement amount and the movement direction of the movement operation in the depth direction, the movement operation in the plane direction, and the plurality of selection operations of the rotation operation. It is detected which of the selected items has been selected, and it is determined which of the plurality of keys has been selected based on the detection result. Further, it is detected that the selection operation has been switched to the opposite direction based on the operation direction of the selection operation, and it is determined whether or not the key input process can be executed based on the detection result. Further, the control unit 21 controls the display unit 22 based on the selection operation information acquired by the sensor data acquisition unit 20.

  In the present embodiment, the control unit 21 includes a first layer display control unit 210, a second layer display control unit 211, a third layer display control unit 212, a key input determination unit 213, and a layer structure data storage unit 214. . The key input determination unit 213 includes a display object displacement amount detection unit 215, a key selection detection unit 216, and a selection determination determination unit 217.

  The hierarchical structure data storage unit 214 stores hierarchical structure data of a plurality of keys grouped into a plurality of hierarchies. In the present embodiment, since a plurality of keys are grouped into three layers, selection items in each layer (a plurality of groups or a plurality of keys in the last layer) for each of the three layers. Is stored.

  The first hierarchical display control unit 210 includes posture information acquired by the posture acquisition unit 200, selection operation information corresponding to the movement operation in the depth direction acquired by the parallel displacement amount acquisition unit 201, and a hierarchical structure data storage unit Based on the hierarchical structure data stored in 214, control information for controlling the display of the first hierarchy is output. In the present embodiment, control is performed so that the selected selection item is clearly shown based on the movement amount of the movement operation in the depth direction. Further, display or non-display is controlled based on the posture information.

  The second hierarchy display control unit 211 includes control information for controlling the display of the first hierarchy by the first hierarchy display control unit 210, the attitude information acquired by the attitude acquisition unit 200, and the translation displacement amount acquisition unit 201. Based on the acquired selection operation information corresponding to the movement operation in the planar direction and the hierarchical structure data stored in the hierarchical structure data storage unit 214, control information for controlling the display of the second hierarchy is output. In the present embodiment, control is performed so that the selected selection item is clearly specified based on the movement amount of the movement operation in the planar direction. Further, display or non-display is controlled based on the posture information.

  The third hierarchy display control unit 212 is obtained by the control information for controlling the display of the second hierarchy by the second hierarchy display control unit 211, the attitude information acquired by the attitude acquisition unit 200, and the rotational displacement amount acquisition unit 202. The display of the third hierarchy is controlled based on the selected action information corresponding to the rotated action and the hierarchical structure data stored in the hierarchical data storage unit 214. In the present embodiment, control information for controlling the selected selection item to be specified based on the amount of rotation operation is output. Further, based on the operation amount of the rotation operation, control is performed so as to clearly indicate that the key input process is executed when the selection operation is switched in the opposite direction. Further, display or non-display is controlled based on the posture information.

  The key input determination unit 213 detects the selected key and the state in which the key input process is executed based on the control information for controlling the display of the third hierarchy by the third hierarchy display control unit 212, The execution processing unit 23 is controlled.

The display object displacement amount detection unit 215 detects the displacement amount of display in each layer based on the control information for controlling the display of the third layer by the third layer display control unit 212.
The key selection detection unit 216 determines which one of a plurality of keys is selected by determining which selection item is selected in each layer based on the display displacement detected by the display object displacement detection unit 215. Detect.
The selection determination determining unit 217 detects that the key input process is being executed based on the display displacement detected by the display object displacement detection unit 215, and the selection item is selected from the detection result. It is determined that it has been determined, and the execution processing unit 23 is controlled.

  Based on the control of the control unit 21, the display unit 22 displays a virtual object including a selection item corresponding to each layer for each layer. For example, a virtual object for key input operation is displayed on the display device 13 such as a head-mounted display, and includes a first layer display unit 220, a second layer display unit 221, and a third layer display unit 222.

  The first hierarchy display unit 220 displays a virtual object corresponding to the first hierarchy based on the control information output from the first hierarchy display control unit 210. The second hierarchy display unit 221 displays a virtual object corresponding to the second hierarchy based on the control information output from the second hierarchy display control unit 211. The third hierarchy display unit 222 displays a virtual object corresponding to the third hierarchy based on the control information output from the third hierarchy display control unit 212.

  The execution processing unit 23 executes key input processing based on the control of the key input determination unit 213.

  As described above, the programs and data for realizing the functions of the components shown in FIG. 2 are stored in the memory 11, and the CPU 12 appropriately reads out the programs or data stored in the memory 11 and uses the data acquired by the sensor 10. By executing the program, the function of each component is realized.

  Note that the configuration and functional configuration of the key input device shown in FIGS. 1 and 2 are the same in the following embodiments, and thus the description thereof is omitted in the following embodiments.

Next, the operation will be described.
FIG. 3 is a flowchart showing an example of the operation flow of the key input device according to the first embodiment showing one embodiment of the present invention.
FIG. 4 is an explanatory diagram showing an example of a virtual object displayed by the key input device according to the first embodiment showing an embodiment of the present invention.

  In FIG. 4, the virtual object has three types of configurations: an icon 40, an icon 41, and an input controller 42. The icon 40 indicates a first-level key group (selection item) based on a character type such as “Hiragana” or “English”. The icon 41 indicates a second-level key group (selection item) such as “A row” and “Ka row” in “Hiragana”. The input controller 42 indicates keys (selection items) of the third hierarchy (lowermost layer) such as “A”, “I”, “U” of “A row” of “Hiragana”, and selection selection of the key. Key input operations are visible.

  In addition, the state in which the key group of the first layer is selected and changed according to the movement amount of the movement operation in the depth direction acquired by the sensor data acquisition unit 20 is displayed, and according to the movement amount of the movement operation in the plane direction. The state where the key group of the second layer is selected and changed is displayed, the state where the key of the third layer is selected and changed according to the operation amount of the rotation operation is displayed, and the state where the key input process is executed is displayed. Is displayed.

  In FIG. 3, first, the sensor 10 detects a part of the operator's body and outputs shape data corresponding to the posture of the part of the body. The posture acquisition unit 200 detects posture information from the shape data. The process is started when the posture information matches a predetermined posture for instructing the start of input (step S1).

  The posture for instructing the start of input can be arbitrarily set such as a posture of a hand or a finger that is a part of the human body, a gesture, or the like. For example, the posture may be set such as opening or closing the hand, “Janken”, or a specific gesture such as waving.

  When the posture of a part of the body matches the posture for instructing the start of input (Yes in step S1), the display unit 22 displays the virtual object and sets the selection item (character group) for initial selection to the selected state ( Step S2).

  FIG. 5 is an explanatory diagram showing an example of a change in the virtual object displayed by the key input device according to the first embodiment showing an embodiment of the present invention.

  When the display unit 22 displays nothing (the left side in FIG. 5), when the operator takes a posture like “Janken” “par”, which is a posture for instructing the start of input, the sensor data acquisition unit 20 The posture acquisition unit 200 detects posture information from the shape data of the posture. Based on the posture information, for example, the first layer display control unit 210, the second layer display control unit 211, and the third layer display control unit 212 correspond to the postures in which the operator's hand 30 instructs input start. Judge whether to do.

  Then, if it is determined that the posture is instructed to start input, the first layer display control unit 210 displays control information for controlling the display of the key group icon 40 of the first layer which is the display of the first layer. It outputs to the 1st hierarchy display part 220, and the 1st hierarchy display part 220 displays this. The second tier display control unit 211 outputs control information for controlling the display of the second tier key group icon 41, which is the second tier display, to the second tier display unit 221, and the second tier display unit 221. Displays this. The third layer display control unit 212 outputs control information for controlling the input controller 42 of the third layer key, which is a display of the third layer, to the third layer display unit 222, and the third layer display unit 222 outputs this control information. Is displayed (right side of FIG. 5).

  At the same time, the first hierarchy display control unit 210 sets one icon 40 (not shown) in the list as an initial selection icon, and the first hierarchy display unit 220 displays this icon in a selected state according to a procedure described later. As this initial selection icon, an icon at a fixed position may be designated every time, or an icon selected in the last input from the history of input operations may be used.

  The selection state display of the icon 40 means that the second hierarchy group icon 41 and the input controller 42 are superimposed on the icon 40. In the example of FIG. 4, the icon of the key group “Hiragana” is in the selected state. In the superimposed display, the virtual object on the back is drawn with a lighter color or lower opacity.

  Thereafter, the operation from step S4 to step S11 is repeated until key input is performed until the operator takes a posture like “Janken” “goo” which is a posture for instructing the end of input (Yes in step S3). (No in step S3). Details of the operation in step S3 will be described later.

  After detecting the input start posture, the sensor 10 detects a part of the operator's body and outputs position data and shape data corresponding to the posture of the part of the body. Then, the posture acquisition unit 200 detects posture information from the shape data, and the parallel displacement amount acquisition unit 201 performs a partial movement of the human body in the three-dimensional directions of the horizontal, vertical, and depth directions from the position data. The amount of displacement per unit time is detected, and the rotational displacement amount acquisition unit 202 detects the amount of rotation and the direction of rotation of each part of the human body from the position data and shape data.

  The rotational displacement amount acquisition unit 202 determines whether or not the rotational displacement amount (rotational motion amount) of a part of the human body is greater than or equal to a predetermined threshold value (step S4), and the rotational displacement amount becomes the predetermined threshold value. If not (No in step S4), the process proceeds to step S5.

  Then, the translation displacement amount acquisition unit 201 determines whether or not the displacement amount (movement amount of the movement operation in the depth direction) of a part of the human body in the depth direction is greater than or equal to a predetermined threshold (step S5). When the amount of displacement in the depth direction is equal to or greater than a predetermined threshold (Yes in step S5), the selection of the first layer key group is changed (step S6).

FIG. 6 is an explanatory diagram showing an example of a change in the first hierarchy of the virtual object displayed by the key input device according to the first embodiment showing an embodiment of the present invention.
When the translation displacement amount acquisition unit 201 detects that the operator's hand 30 has been displaced in the depth direction by a predetermined threshold or more, the first layer display control unit 210 uses the first layer from the detected displacement amount. The selection of the key group icon 40 is changed. The first hierarchy display unit 220 receives this selection change control information from the first hierarchy display control unit 210 and sets the newly selected icon to the selected state display and the other icons to the non-selected state display. In the example of FIG. 6, the selection is changed from the “Hiragana” group to the “English” group.

  In step S5, when the moving displacement amount in the depth direction does not satisfy the predetermined threshold (No in step S5), the process proceeds to step S7. Then, the translation displacement amount acquisition unit 201 determines whether or not the displacement amount (movement amount of the movement operation in the plane direction) of a part of the human body in the plane direction is greater than or equal to a predetermined threshold (step S7). When the movement displacement amount in the plane direction is equal to or greater than the predetermined threshold (Yes in Step S7), the selection of the key group of the second hierarchy is changed (Step S8).

FIG. 7 is an explanatory diagram showing an example of a change in the second hierarchy of the virtual object displayed by the key input device according to the first embodiment showing an embodiment of the present invention.
When the translation displacement amount acquisition unit 201 detects that the operator's hand 30 is displaced by a predetermined threshold or more in a direction other than the depth, the second hierarchy display control unit 211 detects the displacement direction (operation direction). Based on the above, the selection of the key group in the second hierarchy is changed, and the newly selected icon 41 is displayed in the selected state, and the icon that has been in the selected state is displayed in the non-selected state.

  In step S7, when the amount of displacement in the plane direction is less than the predetermined threshold (No in step S7), the process proceeds to step S3.

  The selection change of the key group icon 41 in the second hierarchy is performed by changing the icon in the same direction as the detected displacement direction from the currently selected icon in the second hierarchy group icon list display. This is done by using a selection icon.

  The selection state display of the key group icon 41 in the second hierarchy is to superimpose the input controller 42 in which keys belonging to the small group are arranged at the position of the selected small group icon. At this time, if the input controller is already displayed for another small group, it is hidden. In the example of FIG. 7, the selection is changed from the group “A” to the group “K”.

  In step S4, when the rotational displacement amount is equal to or greater than a predetermined threshold (Yes in step S4), the selection of the key in the third hierarchy is changed (step S9).

  8 and 9 are explanatory diagrams illustrating an example of a change in a virtual object displayed by the key input device according to the first embodiment showing an embodiment of the present invention. FIG. 8 shows an input operation using the input controller 42 corresponding to the second hierarchy group “Ka row” of the first hierarchy group “Hiragana”. FIG. 9 shows an input operation using the input controller 42 corresponding to the second hierarchy group “A row” of the first hierarchy group “Hiragana”. The input controller 42 has a dial shape with a key 90 arranged in a circle and has an input decision cursor 91.

  When the key “A” is input from the state shown in FIG. 9A using the input controller 42 shown in FIG. 9, when the operator twists the wrist counterclockwise over a predetermined threshold value, the rotational displacement amount is acquired. The unit 202 detects that the operator's hand 30 has been displaced in the counterclockwise rotation direction by a predetermined threshold value or more. The third hierarchy display control unit 212 changes the selection of the key of the third hierarchy from the detected displacement, displays the newly selected key 90 as the selected state, and displays the key 90 that has been in the selected state until then. Is not selected. The operator twists his / her wrist while checking the input controller 42 and turns the input controller 42 counterclockwise until the input determination cursor 91 and the key “A” overlap. When the input determination cursor and the key “A” overlap, the key “A” is selected (FIG. 9B). In addition, if the input is not decided, the selection can be canceled by rotating again in the forward direction (counterclockwise).

  The rotational displacement amount acquisition unit 202 determines whether or not the rotational displacement amount (rotation motion amount) of a part of the human body has been switched in the opposite direction (step S10). (No in step S10), the process returns to step S9, and when it is switched to the opposite direction (Yes in step S10), it is determined that the key input is determined (step S11).

  When the execution of the key “A” is determined from the state of FIG. 9B, when the operator twists the wrist clockwise more than a predetermined threshold, the rotational displacement amount acquisition unit 202 causes the operator's hand 30 to be predetermined. It is detected that the rotation direction has been switched in the opposite direction beyond the threshold. The third hierarchy display control unit 212 determines that the key input process of the third hierarchy can be executed based on the detection of the switching in the opposite direction of the rotation, and clearly indicates that the key input process is executed. To be controlled. Based on this control, the third layer display unit 222 performs display, and the state shown in FIG.

The display object displacement amount detection unit 215 detects the displacement amount of display in each layer based on the control information for controlling the display of the third layer by the third layer display control unit 212. In the present embodiment, the control information for controlling the display of the third hierarchy by the third hierarchy display control unit 212 is the control information by the first hierarchy display control unit 210 and the second hierarchy display control unit 211. It is configured so that the display displacement amount in each layer can be detected based on the control information of the third layer display control unit 212.
For example, the display object displacement amount detection unit 215 calculates the displacement amount from the reference position on the display screen of the virtual object using the display state of the virtual object based on the control information, and stores the displacement history for a predetermined period. .

  The key selection detection unit 216 determines which one of a plurality of keys is selected by determining which selection item is selected in each layer based on the display displacement detected by the display object displacement detection unit 215. Detect. Here, it is detected that the key “A” is selected.

In addition, the selection determination determination unit 217 detects that the key input process is being executed based on the display displacement detected by the display object displacement detection unit 215, and determines the selection item from the detection result. It is determined that the selection has been determined, and the execution processing unit 23 is controlled.
The execution processing unit 23 executes key input processing based on the control of the key input determination unit 213. Then, the key “A” is input.

  When inputting the key “O” from this state, similarly, the operator twists the wrist clockwise more than a predetermined threshold value. Then, the display of the input controller 42 starts rotating clockwise in conjunction with the wrist rotation operation. When the operator rotates the wrist until the key “o” overlaps the cursor position and then reversely rotates counterclockwise, it is detected that the key has been switched in the opposite direction, and the key “o” is input. If an incorrect key is entered, the previous entry can be canceled by entering the “BS” key.

  When the virtual object is displayed and the posture information detected by the posture acquisition unit 200 matches a predetermined posture for instructing the end of input, the processing is ended and the virtual object is not displayed. . (Yes in step S3).

  In the state in which the virtual object is displayed, when the operator takes a posture such as “Janken” which is a posture for instructing the end of input, the posture acquisition unit 200 of the sensor data acquisition unit 20 performs the posture. Attitude information is detected from the shape data. Based on the posture information, for example, the first layer display control unit 210, the second layer display control unit 211, and the third layer display control unit 212 correspond to the postures in which the operator's hand 30 instructs the end of input. Judge whether to do.

  When detecting the posture instructing the end of input by the operator's hand 301, the first layer display control unit 210, the second layer display control unit 211, and the third layer display control unit 212 control all the virtual objects as non-display. The information is reset, and the first layer display unit 220, the second layer display unit 221, and the third layer display unit 222 reflect the information on the display device 13 in response thereto.

  The posture for instructing the end of input can be arbitrarily set within a range that does not overlap with the input start operation, such as a posture of a hand or a finger that is a part of the human body, a gesture, or the like. For example, the posture may be set such as opening or closing the hand, “Janken”, or a specific gesture such as waving.

As described above, according to the present embodiment, selection operation information corresponding to a plurality of different selection operations according to the number of layers is acquired and acquired for a plurality of keys grouped in a plurality of layers. By selecting which one of a plurality of keys has been selected based on the selection operation information, selecting any of the selection items of the corresponding layer according to the selection operation information of the different selection operations set for each layer Therefore, the operation for determining the selection item selected for each hierarchy can be omitted, and the number of operations for selecting a target key from a plurality of keys can be reduced. Therefore, since the target key can be easily accessed from among a large number of keys, it is possible to continuously input at a high speed.
That is, since a different selection operation is assigned to each key group hierarchy, key access can be made hierarchically without requiring a determination operation. For example, key group switching (changing the selection item (character type) in the first layer, changing the selection item in the second layer) can be operated only by parallel movement of the three-dimensional hand, so that it can be easily operated.

  Further, in the present embodiment, based on at least one of the operation amount and the operation direction of the selection operation, it is detected which of the selection items corresponding to the layer is selected for each layer, and based on the detection result It is determined which of a plurality of keys is selected. For example, regarding the movement operation of the palm in the depth direction, the selection of the key group of the first hierarchy is changed based on the displacement amount of the palm. In addition, regarding the movement of the palm in the plane direction, the selection of the key group of the second hierarchy is changed based on the displacement direction of the palm. As a result, the key group or the selected state of the key is linked to the movement amount or movement direction of the operator, so that the input operation is facilitated.

Further, in this embodiment, when it is detected that the selection operation has been switched in the opposite direction, it is determined that the key input process can be executed, and the key input process is executed. Thus, the operator can instruct input execution of the selected key by switching the selection operation in the opposite direction, so that it is possible to instruct input execution reliably without requiring a precise operation.
For example, in the operation of the input controller, the operator can easily obtain an actual feeling of the operation by designating the key at a clear operation point that is the reverse turning position. Furthermore, the rotation operation with the wrist as an axis is an operation that can be performed at hand, which is less likely to shake compared to the parallel movement operation. By using this for the key selection execution operation, the input reliability and high speed are increased.

  In the present embodiment, the selection operation includes a movement operation of a part of the human body and a rotation operation of a part of the human body. For example, the movement operation in the depth direction of the palm, the movement operation in the plane direction of the palm, and the rotation operation of the palm are used as the selection operation. Thereby, it is possible to realize key input with only a part of the human body. In addition, motion blur can be reduced. Therefore, for example, it is possible to operate with one hand, and it is possible to reliably instruct execution of input without requiring a precise operation.

  Moreover, in this Embodiment, the display part 22 which displays the virtual object containing the selection item corresponding to the hierarchy for every hierarchy is provided, and the control part 21 is based on the selection operation | movement information corresponding to an operator's selection operation | movement. The display unit 22 is controlled. Thereby, the display operation of the virtual object and the selection state of the key group or key are linked with the selection operation of the operator, so that the input operation is facilitated.

  In the present embodiment, display and non-display of the virtual object are controlled based on posture information corresponding to a portion of the posture of the human body. Thereby, for example, the operation can be performed with one hand, and the operation becomes easy.

  Further, in the present embodiment, the virtual object can visually recognize each selection item of at least two layers. For example, the input controller of the third layer is superimposed on the list of icons of the key group (character type) of the second layer and displayed in a multi-dimensional manner. As a result, it is easy to visually recognize the hierarchical structure of the keys, and it is easy to understand where to move the hand to switch to a different key group, so that a smooth operation is possible.

  In the present embodiment, a plurality of keys have a hierarchical structure from the first hierarchy to the third hierarchy, and a target key group is selectively expanded and displayed in accordance with an operation. As a result, the number of selection items displayed at a time can be reduced, and the operator can easily find the key.

  Further, in the present embodiment, in the superimposed display, the virtual object on the back is drawn with a lighter color or lower opacity. This makes it easier for the operator to visually understand which operation is currently being performed.

  By arranging the keys on the input controller so that they are equally distributed to the left and right of the input determination cursor, even if there is a limit in the range of rotation such as a human wrist, it is possible to access with a small amount of rotation.

  In the present embodiment, a case has been described in which a plurality of keys are grouped into three hierarchies. However, the number of hierarchies is not limited to three hierarchies as long as keys are grouped into a hierarchical structure. Absent.

  Further, in the present embodiment, the control unit 21 detects the displacement amount of the virtual object based on the control information for controlling the display unit 22, and based on the displacement amount of the display, which selection item is in each hierarchy. Although it has been described that it is detected which of a plurality of keys has been selected by determining whether it has been selected, and it is determined that the key input process of the selected key can be performed, the sensor data acquisition unit 20 Based on the acquired selection operation information, it is detected directly from the selection operation information which of the plurality of keys has been selected, and it is determined that the key input process of the selected key can be executed. Also good.

Embodiment 2. FIG.
In the above-described embodiment, the case where the icons of the key groups in the second hierarchy are arranged radially is described, but the arrangement method may not be radial. An embodiment in the case of arranging them sequentially in the vertical and horizontal directions will be described.

FIG. 10 is an explanatory diagram showing an example of a virtual object displayed by the key input device according to the second embodiment showing an embodiment of the present invention.
As shown in FIG. 10, if the icons are arranged sequentially in the vertical and horizontal directions, more icons can be arranged compared to the radial arrangement.

Embodiment 3 FIG.
In the above-described embodiment, the case where the shape of the input controller of the key of the third hierarchy is circular has been described. However, the shape of the input controller may not be circular. An embodiment in the case of a strip shape will be described.

FIG. 11 is an explanatory diagram showing an example of a virtual object (input controller) displayed by the key input device according to the third embodiment showing an embodiment of the present invention.
In FIG. 11, it is assumed that the controller has a band shape, and the keys 90 are arranged at equal intervals with the input determination cursor 91 and the reference position marker 901 at the center of the band. The direction of the band (key arrangement) at this time is not limited to the horizontal direction as shown in FIG. 11, and may be, for example, a shape extending in the vertical direction or a shape extending obliquely.

A procedure for key input operation using a band-like input controller will be described.
When the operator tilts his / her hand clockwise or counterclockwise by a wrist twisting operation, the rotational displacement amount acquisition unit 202 causes the operator's hand 30 to rotate clockwise or counterclockwise as in the first embodiment. It is detected that the displacement is greater than a predetermined threshold in the direction. The third hierarchy display control unit 212 changes the selection of the key of the third hierarchy from the detected displacement, displays the newly selected key 90 as the selected state, and displays the key 90 that has been in the selected state until then. Is not selected. In the present embodiment, the key on the input controller starts to move right or left at a constant speed. The key continues to move in the same direction while the operator tilts his hand in the same direction or until the leftmost key overlaps the position of the input determination cursor 91, and the key overlapping the input determination cursor at each time point Selected state.

  When the operator turns the wrist while the arbitrary key is in the selected state and tilts the hand in the direction opposite to the conventional twisting operation, the key on the input controller also turns back and moves. At this time, the key continues to move while the operator tilts his hand in the same direction or until the rightmost key on the controller overlaps the input decision cursor. Then, when the amount of displacement in the reverse direction after this switching becomes equal to or greater than a predetermined threshold, the rotational displacement amount acquisition unit 202 detects that the operator's hand 30 has been switched in the opposite direction beyond the predetermined threshold. . The third hierarchy display control unit 212 determines that the key input process of the third hierarchy can be executed based on the detection that the switching is performed in the opposite direction, and clearly indicates that the key input process is executed. To be controlled. Based on this control, the third hierarchy display unit 222 performs display. Thereafter, the key input process is executed as in the first embodiment.

  In the band-shaped input controller, a key that is moved out of the band area by movement may not be displayed. Alternatively, the keys may be arranged in a loop, and for example, a key that is off the right end may appear and be displayed on the left end.

  Further, in the key arrangement, it is not necessary to display all the keys included in the input controller side by side. Each time, only the keys that can ride over the area of the band of the input controller are displayed side by side, the key at the end in the direction of travel is removed from the band when the key is moving, and the key that has been hidden from the other end It may be repeated to display one of them.

  As described above, according to the present embodiment, by using a strip-shaped input controller, the number of keys included in one controller can be increased compared to a circular input controller. Allows you to enter more keys.

Embodiment 4 FIG.
In the above-described embodiment, the case where a plurality of keys are grouped into three hierarchies has been described. However, an embodiment in which a plurality of keys are grouped into two hierarchies will be described.

  In the present embodiment, the virtual object includes a key group icon of the first layer corresponding to the selection item of the first layer and a key input controller of the second layer corresponding to the selection item of the second layer.

  Similar to the first embodiment, when the operator performs an operation to instruct the start of input, the key group icon of the first hierarchy is displayed, and an arbitrary icon is selected as the initial selection icon. On the selected icon, an input controller with keys arranged in a circle or band as in the above-described embodiment is superimposed and displayed. After that, key selection and execution processing are performed according to the same procedure as in the above-described embodiment according to the rotation caused by wrist twist and the turning operation.

  As described above, according to the present embodiment, the number of keys is smaller than in the case of the first embodiment, and the number of keys is sufficient to be grouped into two layers. Key access is easier. Further, it is possible to cope with a case where there is no sufficient operation (motion) space in the three-dimensional direction and a key input operation is desired only by the two-dimensional operation.

Embodiment 5 FIG.
In the first embodiment, the first-level key group icon is selected by the amount of displacement in the depth direction of the hand, and the second-level key group icon is selected by the amount of displacement in a two-dimensional direction other than the depth. However, the moving direction for performing each operation does not necessarily have to be this combination. For example, the first layer group icon may be selected by the amount of displacement in the vertical direction, and the second layer group icon may be selected by the amount of displacement in the other two-dimensional directions. In this way, it is possible to cope with a case where it is desired to perform group expansion operations on the horizontal and depth planes, such as when the display device is in the downward direction or when the space area that can be operated in the vertical direction is limited.

Embodiment 6 FIG.
In the first embodiment, the icons of the key groups in the first hierarchy are arranged two-dimensionally in the vertical and horizontal directions, but a display method in which the icons are arranged in the depth direction may be used. In this case, the second layer key group icon and the input controller are displayed in a form that pops up on the target first layer key group icon, for example. By doing so, it is possible to display in visual synchronization with the actual operation, and it is possible to more intuitively change the selection of the key group in the first layer.

  As mentioned above, although embodiment of this invention was described, you may implement in combination of 2 or more among these embodiment. Alternatively, one of these embodiments may be partially implemented. Alternatively, two or more of these embodiments may be partially combined. In addition, this invention is not limited to these embodiment, A various change is possible as needed. For example, an operation with a human hand may be an operation with a foot. By doing so, it becomes an apparatus that can be operated even by an operator who is handicapped.

DESCRIPTION OF SYMBOLS 10 Sensor, 11 Memory, 12 CPU, 13 Display apparatus, 20 Sensor data acquisition part, 21 Control part, 22 Display part, 23 Execution processing part, 200 Posture acquisition part, 201 Translation displacement amount acquisition part, 202 Rotation displacement amount acquisition , 210 First layer display control unit, 211 Second layer display control unit, 212 Third layer display control unit, 213 Key input determination unit, 214 Hierarchical structure data storage unit, 215 Display object displacement amount detection unit, 216 Key selection Detection unit, 217 selection determination determination unit, 220 first layer display unit, 221 second layer display unit,
222 Third layer display section.

Claims (7)

In a key input device for inputting any of a plurality of keys grouped in a plurality of layers,
A sensor data acquisition unit for acquiring selection operation information corresponding to a plurality of different selection operations according to the number of layers;
A control unit for determining which of the plurality of keys is selected based on the selection operation information acquired by the sensor data acquisition unit;
A key input device comprising: an execution processing unit that executes a key input process based on a determination result of the control unit. The sensor data acquisition unit acquires at least one of an operation amount and an operation direction of the plurality of selection operations,
The control unit detects which of the selection items corresponding to the hierarchy is selected for each hierarchy based on at least one of the operation amount and the operation direction of the selection operation, and based on the detection result, The key input device according to claim 1, wherein which of the plurality of keys is selected is determined. The control unit detects that the selection operation has been switched to the opposite direction based on the operation direction of the selection operation, and determines whether or not the key input process can be performed based on the detection result.
The said execution process part performs the key input process of the said key determined by the said control part, when it determines with execution of a key input process being possible by the said control part. Key input device.   The key input device according to claim 1, wherein the selection operation includes a movement operation of a part of a human body and a rotation operation of a part of the human body. A display unit for displaying a virtual object including a selection item corresponding to the hierarchy for each hierarchy;
The key input device according to claim 1, wherein the control unit includes a display control unit that controls the display unit based on the selection operation information. The sensor data acquisition unit acquires posture information corresponding to a posture of a part of the human body,
The key input device according to claim 5, wherein the display control unit controls display and non-display of the virtual object based on posture information of a part of the human body acquired from the sensor data. .   The key input device according to claim 5, wherein the virtual object is capable of visually recognizing the selection items of each of at least two of the hierarchies.

Images