JP5790257B2 - Character input device, character input program, and character input method - Google Patents

Description

  The present invention relates to a character input device, a character input program, and a character input method for receiving character input by a user.

  In the technology of space gestures in which information devices are operated by performing gestures, hand gestures, etc. from a remote location, the information devices recognize and input characters drawn in space by the user. is there. For example, a technique is known in which a user draws a trajectory drawn in space using a tool that reflects light and performs character recognition from the trajectory. In this technique, the user designates confirmation / cancellation of the recognition result character displayed on the screen by drawing a predetermined figure such as ○ or ×. Such a technique enables a user to input characters into an intuitive information device.

  A technique for remotely controlling an electronic device by displaying a menu on a screen triggered by recognition of a user action such as making the hand a predetermined shape is known. In addition, a technique for recognizing a meaningful input other than position information by detecting a locus of a pointer with respect to a specific area is known.

JP 2008-234277 A JP-A-9-128141 JP-A-10-93120

  In the technology of character input by spatial operation, there are various problems regarding how to correct misrecognized characters and predictive input of character strings.

  For example, the technique for confirming / cancelling a recognized character displayed on the screen by drawing a predetermined figure such as the above-mentioned ○ or × must designate confirmation / cancellation for each character. Annoying and inconvenient. In addition, the technique for displaying a menu on the screen with some special action has problems such as having to remember a special action and having to perform a special action every time the menu is opened, which is bothersome and inconvenient for the user. is there.

  Even if the menu is displayed on the screen, the operation using the menu becomes inconvenient when the position of the character drawn on the screen by the user's space operation is far from the menu. Also, if the space operation has to be performed so that the character is drawn near the displayed menu, the user's free space operation character drawing is obstructed, which is bothersome and inconvenient for the user.

  In one aspect, an object of the present invention is to provide a technique for improving the convenience of a character input operation including a menu operation when inputting characters by moving an object in space.

  In one aspect, the character input device includes a position information acquisition unit that acquires position information of a specific object in space, a position information holding unit that stores position information, and a pointer display on the screen of the display device based on the position information. A pointer position determining unit that determines a position, a pointer display unit that displays a pointer at the determined display position of the pointer on the screen of the display device, and a movement of a specific object based on the position information accumulated in the position information holding unit To the circular menu based on the positional relationship between the circular menu and the pointer, the character display unit for specifying the character drawn by the menu, the menu display unit for displaying the circular menu surrounding the movement locus of the pointer corresponding to the specified character A menu selection unit for determining the selection operation.

  In one aspect, the convenience of character input operations including menu operations is improved when character input is performed by movement of an object in space.

It is a figure which shows the structural example of the character input device by this Embodiment. It is a figure which shows the hardware structural example of the computer which implement | achieves the character input device by this Embodiment. It is a figure explaining the example of the user's character input operation by this Embodiment. It is a figure which shows the example of the cyclic | annular menu by this Embodiment. It is a figure explaining the example of operation to the circular menu by this Embodiment. It is a figure which shows the example of the cyclic | annular menu by which the prediction character string by this Embodiment was displayed. 5 is a flowchart of character specification / circular menu display processing by the character input device according to the first embodiment; It is a cyclic | annular menu operation process flowchart by the character input device of this Embodiment. It is an input character string prediction process flowchart by the character input device of this Embodiment. It is a prediction character string allocation display processing flowchart by the character input device of this Embodiment. It is a figure explaining the display of the cyclic | annular menu by this Embodiment 2. FIG. 12 is a flowchart of character specification / circular menu display processing by the character input device according to the second embodiment.

  Hereinafter, the present embodiment will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration example of a character input device according to the present embodiment.

  A character input device 10 shown in FIG. 1 is a device that recognizes a trajectory of a specific object in a space and performs character input processing. The specific object is, for example, a part of a user's body that inputs characters in space, or a device that the user wears on the body. The character input device 10 recognizes and inputs a character drawn intuitively, for example, when a user moves a hand etc. in space.

  The character input device 10 includes a position information acquisition unit 11, a pointer position determination unit 12, a pointer display unit 13, a position information holding unit 14, a character specifying unit 15, a menu display unit 16, a character string holding unit 17, and a character string prediction unit 18. , A predicted character string assigning unit 19, a menu selecting unit 110, and a menu resetting unit 111.

  The position information acquisition unit 11 acquires position information of a specific object in space from the tracking device 20. The tracking device 20 is a device that identifies a specific object existing in space and tracks its movement.

  The pointer position determination unit 12 determines the display position of the pointer on the screen of the display device 30 based on the position information acquired by the position information acquisition unit 11. The pointer display unit 13 displays a pointer at the display position on the screen of the display device 30 determined by the pointer position determination unit 12.

  The position information acquired by the position information acquisition unit 11 is stored in the position information holding unit 14. The position information holding unit 14 is a storage unit that accumulates and stores the position information acquired by the position information acquisition unit 11. The continuous position information accumulated in the position information acquisition unit 11 is information on the trajectory of a specific object.

  The character specifying unit 15 specifies a character drawn by the movement of a specific object in space based on the position information stored in the position information holding unit 14. At this time, the character identification unit 15 identifies a character drawn in space by performing character recognition on the trajectory of a specific object. The character specifying unit 15 includes a character shape candidate extracting unit 151, a character recognizing unit 152, and a recognized character specifying unit 153.

  The character shape candidate extraction unit 151 extracts, from the position information stored in the position information holding unit 14, information on the locus of the portion estimated that the character is drawn by the movement of the specific object. The character recognition unit 152 performs character recognition on the trajectory information extracted by the character shape candidate extraction unit 151, and obtains a recognition certainty indicating the certainty of the recognized character. The recognized character specifying unit 153 is configured to draw a recognized character indicating the maximum recognized certainty by the movement of a specific object when there is a recognized character having a certainty of recognition or more among the recognized characters by the character recognizing unit 152. Identified as a character.

  The menu display unit 16 displays on the display device 30 a circular menu that assists the user to input characters so as to surround the movement trajectory of the pointer corresponding to the character specified by the character specifying unit 15. The pointer movement trajectory corresponding to the specified character is a pointer movement trajectory corresponding to the trajectory extracted by the character shape candidate extraction unit 151 when the specified character is recognized. The menu display unit 16 includes a menu position determination unit 161. The menu position determination unit 161 determines the display position and size of the circular menu.

  The character string holding unit 17 is a storage unit that stores a character string obtained by additionally recording the characters specified by the character specifying unit 15 in the order of drawing. The character string predicting unit 18 predicts a character string that the user intends to input from the character strings stored in the character string holding unit 17. The predicted character string assigning unit 19 assigns the character string predicted by the character string predicting unit 18 to the circular menu displayed on the screen of the display device 30.

  The menu selection unit 110 determines a user's selection operation for the circular menu based on the positional relationship between the circular menu on the screen of the display device 30 and the pointer. When the selection operation for the circular menu is determined, processing corresponding to the item selected from the circular menu is performed. The menu reset unit 111 deletes the circular menu according to the positional relationship between the circular menu on the screen of the display device 30 and the pointer.

  The character input device 10 of the present embodiment improves the convenience of character input operations including menu operations when inputting characters by moving an object in space.

  FIG. 2 is a diagram illustrating a hardware configuration example of a computer that implements the character input device according to the present embodiment.

  A computer 1 that implements the character input device 10 of the present embodiment shown in FIG. 1 includes, for example, a CPU (Central Processing Unit) 2, a memory 3 as a main memory 3, a storage device 4, a communication device 5, a medium reading / writing A device 6, an input device 7, an output device 8 and the like are provided. The storage device 4 is, for example, an HDD (Hard Disk Drive). The medium reading / writing device 6 is, for example, a CD-R (Compact Disc Recordable) drive or a DVD-R (Digital Versatile Disc Recordable) drive. The input device 7 is, for example, a keyboard / mouse. In the present embodiment, a camera that implements the tracking device 20 is also an input device. The output device 8 is, for example, a display device 30 such as a display.

  The character input device 10 and each functional unit included in the character input device 10 illustrated in FIG. 1 can be realized by hardware such as the CPU 2 and the memory 3 included in the computer 1 and a software program. A program that can be executed by the computer 1 is stored in the storage device 4, read into the memory 3 at the time of execution, and executed by the CPU 2.

  The computer 1 can also read a program directly from a portable recording medium and execute processing according to the program. The computer 1 can also sequentially execute processing according to the received program every time the program is transferred from the server computer. Further, this program can be recorded on a recording medium readable by the computer 1.

  Below, the process by the character input device 10 of this Embodiment is demonstrated, showing a more specific example.

[Embodiment 1]
FIG. 3 is a diagram for explaining an example of a user character input operation according to the present embodiment.

  As shown in FIG. 3, in the example of the first embodiment, the user who is slightly away from the display device 30 moves the hand 40 while facing the screen 31 of the display device 30, thereby performing a character input operation. I do. In the first embodiment, the user's hand 40 is a specific object whose position is to be acquired by the tracking device 20. The camera 21 images the user who is in front of the screen 31 of the display device 30. Although not shown in FIG. 3, the camera 21 and the display device 30 are connected to the computer 1 that implements the character input device 10.

  The camera 21 is required to have a performance capable of inputting video in real time using a visual sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. For example, a commercially available general web camera may be used.

  The display device 30 is required to be capable of sufficiently displaying various display contents by the character input device 10 of the present embodiment. For example, a resolution and display speed comparable to those of commercially available liquid crystal monitors are sufficient. The display device 30 may be any system such as an organic EL (Electroluminescence) system, a liquid crystal system, or a projection system.

  In the first embodiment, the tracking device 20 is realized by the camera 21 and software that analyzes the camera video and tracks the presence and position of a specific object.

  A technology that searches for a part similar to a specific figure or color distribution from the camera image, and the current frame is compared with the previous frame in the camera image to track how each part in the camera image has moved. There is technology to do. By combining these technologies, it is possible to track a specific object in the camera image. For example, an open source software library called OpenCV (registered trademark) includes these functions as standard.

  There is also a device called a distance image sensor that can sense the distance to each part of the image. If a distance image sensor is used, it is possible to capture a three-dimensional image, and more accurate tracking is possible.

  In the first embodiment, the tracking device 20 tracks the position of the user's hand 40. It is generally difficult to extract the hand 40 at high speed and with high accuracy. For example, by setting restrictions such that the palm of the hand 40 is always directed toward the camera, the hand 40 is always held, and the skin color portion that protrudes most from the camera 21 is regarded as the hand 40. In addition, it is possible to extract the hand 40 with high accuracy.

  In the character input device 10, the position information acquisition unit 11 acquires position information of the user's hand 40 in space from the tracking device 20. The position information obtained by the position information acquisition unit 11 may be three-dimensional coordinates including depth information, but is not particularly essential in the present embodiment. In the present embodiment, it is assumed that the position information acquisition unit 11 can obtain two-dimensional coordinates.

  The pointer position determination unit 12 determines the display position of the pointer 50 on the screen 31 of the display device 30 from the position information acquired by the position information acquisition unit 11. For example, when the obtained position information is the coordinates of the position where the hand 40 is captured on the camera image, the pointer position determination unit 12 sets the center position of the camera image to the center position of the screen 31 and sets the end of the camera image. The acquired coordinates are converted into coordinates on the screen 31 so that the position becomes the end position of the screen 31. The converted coordinates on the screen 31 become the coordinates of the determined display position of the pointer 50. Further, based on the calibration set in advance by the user, a range in which the tracked hand 40 can be moved without difficulty may be assigned to the screen 31, and the acquired position information may be converted into the display position of the pointer 50.

  The pointer display unit 13 displays the pointer 50 at the display position on the screen 31 of the display device 30 determined by the pointer position determination unit 12. As shown in FIG. 3, the pointer 50 is displayed at the display position of the screen 31 corresponding to the position of the user's hand 40 in the space, and follows the movement of the user's hand 40 in the space. The pointer 50 also moves. In the example shown in FIG. 3, the movement locus of the pointer 50 is displayed on the screen 31, but the design of whether or not to display the movement locus of the pointer 50 on the screen 31 is arbitrary.

  The position information holding unit 14 accumulates the position information obtained by the position information acquisition unit 11. Information obtained by arranging the position information acquired by the position information acquisition unit 11 in time series is information on the movement locus of the user's hand 40. That is, it can be said that the position information accumulated in the position information holding unit 14 is information on the movement locus of the user's hand 40.

  The position information stored in the position information holding unit 14 is, for example, until a character satisfying a certain degree of recognition certainty is obtained by character recognition or until the storage area allocated to the position information holding unit 14 is full. Retained. When the storage area allocated to the position information holding unit 14 becomes full, the old position information is deleted, and an area where new position information can be stored is made free. If the position information is acquired 20 times per second and the maximum time required for the user to write one character in the space is 3 seconds, the position information holding unit 14 has 20 × 3 = There should be an area that can store 60 or more pieces of position information.

  In the character specifying unit 15, the character shape candidate extracting unit 151 extracts a portion estimated to be a character from the information on the movement locus of the user's hand 40 accumulated in the position information holding unit 14. Here, the locus of the portion estimated to be a character extracted by the character shape candidate extraction unit 151 is referred to as a character shape candidate. The extraction of the character shape candidates can be performed based on, for example, the positional relationship between the coordinate points constituting the locus of the user's hand 40, the direction change of the locus, the size of the locus, the stop time, and the like.

  For example, characters are usually written from top to bottom and from left to right, so the part whose trajectory is moving from bottom to top and from right to left is just before starting to write the next character. There is a high possibility that it is a brush stroke. Therefore, it is considered that the portion immediately before such a stroke portion is likely to be the end of writing of a character, and the portion immediately after that is likely to be the start of writing of a character. In addition, since the trajectory stops frequently at the beginning and end of the character writing, the stopping portion can be extracted as a character delimiter candidate.

  In addition, among the candidates for the start point and the end point of the character, if the path between the points is too small or too large, the path It can be estimated that is not a character. The threshold for the size of the trajectory is, for example, determined by a number of subjects who input aerial handwritten characters in a natural size for each subject. A method of determining the confidence interval is conceivable.

  In the character identification unit 15, the character recognition unit 152 performs character recognition on the character shape candidates extracted by the character shape candidate extraction unit 151 to obtain the certainty as a character. In this embodiment, the certainty as a character obtained by character recognition is referred to as recognition confidence. For example, the character recognition unit 152 collates the extracted character candidate pattern with the template pattern of each character registered in advance, and obtains the similarity between the patterns. Here, the similarity obtained by collation is the recognition certainty.

  In the character identification unit 15, the recognized character identification unit 153 determines the recognized character for the character shape candidate having the maximum similarity when the character recognition by the character recognition unit 152 achieves a predetermined similarity or higher. , It is specified as a character drawn in the space by the user's hand 40.

  When the character is specified, the character specifying unit 15 resets continuous position information on which the character shape candidates are extracted, that is, the movement locus of the user's hand 40. The reset here refers to updating the movement trajectory of the user's hand 40 from which the character shape candidate is extracted to the movement trajectory of the user's hand 40 after the character shape candidate in which the character is specified. As a resetting method, for example, the position information from the position information holding unit 14 to the last point of the character shape candidate for which the character is specified is deleted. For example, a method may be considered in which the position information is next to the character shape candidate in which the character is specified.

  The character specifying unit 15 adds a character specified by character recognition to the character string held in the character string holding unit 17. Here, the character string held in the character string holding unit 17 is referred to as an undetermined character string. The unconfirmed character string indicates an input character string before Kana-Kanji conversion, predictive input, or character string transmission to the character input target application software.

  The character string predicting unit 18 predicts the character string that the user intends to input from the undetermined character string stored in the character string holding unit 17. For example, the character string predicting unit 18 extracts a character string that starts with the sound of an undetermined character string from a character string dictionary not shown in the configuration shown in FIG. The input character string prediction function by the character string prediction unit 18 of the present embodiment is the same function as the character string prediction input function and the like mounted on a mobile phone or the like. Here, the input character string predicted by the character string prediction unit 18 is referred to as a predicted character string.

  When the character specifying unit 15 specifies a character, the menu display unit 16 displays a circular menu so as to surround the movement trajectory of the pointer 50 corresponding to the specified character. Here, the movement locus of the pointer 50 corresponding to the specified character is referred to as a character locus.

  FIG. 4 is a diagram showing an example of a circular menu according to the present embodiment.

  When a character is specified by the character specifying unit 15, a circular menu 60 is displayed on the screen 31 of the display device 30 so as to surround the character locus 51 as shown in FIG. 4. In the example shown in FIG. 4, “NA” is displayed as the character specified by the character specifying unit 15 at the upper part of the area inside the area of the circular menu 60.

  The display position and size of the annular menu 60 are obtained by the menu position determination unit 161. When a character is specified by the character specifying unit 15, the menu position determining unit 161 sets the circular menu 60 based on the movement locus of the pointer 50 corresponding to the specified character, that is, the position and size of the character-like locus 51. Determine the display position and size.

  As a method for obtaining the position of the annular menu 60, for example, the coordinate of the center of gravity of the character locus 51 is used as the center coordinate of the annular menu 60, or the center coordinate of the circumscribed rectangle of the character locus 51 is used as the center coordinate of the annular menu 60. Possible methods. As a method of obtaining the radius as the size of the annular menu 60, for example, a method of obtaining the maximum value of the distance between the center coordinates of the annular menu 60 and the coordinates of the points constituting the character locus 51 by multiplying a constant value, For example, a method of multiplying the diagonal length of 51 circumscribed rectangles by a constant value may be considered. The radius of the circular menu 60 here is the radius of the inner circle in the circular menu 60 shown in FIG.

  Thus, various effective calculation methods for the display position and size of the circular menu 60 are conceivable. The menu position determination unit 161 determines the display position and size of the annular menu 60 so that the annular menu 60 is displayed so as to surround the character-shaped locus 51 with a certain amount of allowance.

  The menu display unit 16 displays the annular menu 60 on the screen 31 of the display device 30 based on the position and size obtained by the menu position determination unit 161. In FIG. 4, an example of a circular circular menu 60 is shown as the circular menu 60. However, if the circular menu 60 surrounds the character locus 51, the circular menu 60 has another shape such as a polygon. Also good.

  In the example shown in FIG. 4, an unconfirmed character string or a predicted character string is displayed at a position that does not interfere with the display of the circular menu 60. In the example shown in FIG. 4, the character string display unit 61 is a display column for an undetermined character string held in the character string holding unit 17. The predicted character string display unit 62 is a display column for a predicted character string obtained by the character string predicting unit 18.

  The items displayed on the circular menu 60 include items such as candidates for recognized characters other than the specified characters, various editing functions, and prediction character string selection.

  In the circular menu 60 shown in FIG. 4, items displayed as “TA”, “RU”, and “MA” are items in which candidates for recognized characters other than the character shape candidate in which the character “NA” is specified are presented. is there. When these items are selected from the circular menu 60, the specified character is replaced with the character of the selected item.

  In the circular menu 60 shown in FIG. 4, items displayed as “line feed”, “clear”, and “edit” are items of various editing functions. For example, “new line” is an editing function used when an unconfirmed character string is confirmed. “Clear” is an editing function used when, for example, erasing an unconfirmed character string in backspace. “Edit” is an editing function used in editing such as copying characters.

  In the circular menu 60 shown in FIG. 4, “predictive selection” is an item for predictive character string selection. When the item of “prediction selection” is selected from the circular menu 60, a transition to the circular menu 60 for selecting a prediction character string is performed.

  The user moves the pointer 50 by moving the hand 40 in the space, and performs an operation of selecting an item in the circular menu 60. Note that the user can also input the next character without selecting an item in the circular menu 60. For example, when the user operates the pointer 50 so as to overlap the character locus 51 within the area inside the circular menu 60, the character input device 10 deletes the previous character locus 51 and inputs the next character. It is reflected in the processing.

  FIG. 5 is a diagram for explaining an example of an operation on a circular menu according to the present embodiment.

  The menu selection unit 110 performs menu selection processing based on the positional relationship between the circular menu 60 and the pointer 50. The menu selection unit 110 performs a process according to the item selected from the circular menu 60 when it is determined that the user has performed an operation of selecting the circular menu 60.

  For example, a method of performing menu selection processing when the pointer 50 enters one of the items of the circular menu 60 or when the pointer 50 remains in the same item for a certain time or more is considered.

  FIG. 5A shows an example of a selection operation for the circular menu 60 by the user. In the example shown in FIG. 5A, the user moves the hand 40 in the space to move the pointer 50 on the screen 31 from the area inside the area of the circular menu 60 to the area of the target item of the circular menu 60. An operation of moving to the area inside the area of the circular menu 60 is performed. The menu selection unit 110 determines that the selection operation for the annular menu 60 has been performed when the pointer 50 moves from the inner area to the annular menu 60 area within a predetermined time and further moves to the inner area.

  By determining the selection operation to the circular menu 60 by the operation of the pointer 50 shown in FIG. 5A, the waiting time is unnecessary for the menu selection operation, and the pointer 50 is inadvertently moved outside the circular menu 60. It is possible to prevent erroneous operation of menu selection due to.

  FIG. 5B shows an example of a menu reset operation by the user. For example, a user who wants to delete the displayed circular menu 60 moves the hand 40 in the space to move the pointer 50 on the screen 31 from the inner region to the outer region of the circular menu 60 and to stop it. . The menu reset unit 111 deletes the circular menu 60 when the pointer 50 is in the area of the circular menu 60 or the area outside the area of the circular menu 60 for a predetermined time or more.

  Since the circular menu 60 may be an obstacle when the character writing position on the screen 31 is to be moved greatly, the operation for easily deleting the circular menu 60 as shown in FIG. It is valid.

  FIG. 6 is a diagram showing an example of a circular menu on which a predicted character string according to the present embodiment is displayed.

  For example, in the circular menu shown in FIG. 4, it is assumed that the item “predictive selection” is selected by the selection operation shown in FIG. At this time, as shown in FIG. 6, the predicted character string assigning unit 19 assigns each predicted character string obtained by the character string predicting unit 18 to an item on the circular menu 60. Thus, by assigning the predicted character string to the item of the circular menu 60, the user can easily select the predicted character string by the selection operation on the circular menu 60.

  When the item “Return” is selected in the circular menu 60 shown in FIG. 6, the circular menu 60 on the screen 31 returns to the circular menu 60 shown in FIG. 4.

  FIG. 7 is a flowchart of character specification / circular menu display processing by the character input device according to the first embodiment.

  In the character input device 10, the position information acquisition unit 11 acquires the position information of the user's hand 40 in the space from the tracking device 20 (step S10).

  The pointer position determination unit 12 determines the display position of the pointer 50 on the screen 31 of the display device 30 from the acquired position information (step S11). The pointer display unit 13 displays the pointer 50 at the determined display position on the screen 31 of the display device 30 (step S12).

  The character input device 10 additionally records the acquired position information in the position information holding unit 14 (step S13). The acquired position information is added to the information of the movement trajectory of the hand 40 of the user from which the character shape candidates are extracted.

  In the character specifying unit 15, the character shape candidate extracting unit 151 extracts character shape candidates from continuous position information stored in the position information holding unit 14, that is, information on the movement locus of the user's hand 40 (step S 14). The character recognition unit 152 performs character recognition on the extracted character shape candidates (step S15). The recognition confidence of the recognized character is obtained by character recognition for the character shape candidate. The recognized character identification unit 153 determines whether the recognition certainty is equal to or higher than a predetermined value (step S16). If the recognition certainty is not equal to or higher than the predetermined value (NO in step S16), the character input device 10 returns to the process in step S10 and acquires the position information of the next user's hand 40.

  If the recognition certainty is greater than or equal to a predetermined value (YES in step S16), the character recognized by the character recognition for the character shape candidate is specified as the character drawn by the movement of the user's hand 40 (step S17). The character specifying unit 15 resets the continuous position information for extracting the character shape candidates (step S18).

  The menu position determination unit 161 determines the display position and size of the annular menu 60 displayed so as to surround the character locus 51 from the position and size of the character locus 51, that is, the movement locus of the pointer 50 corresponding to the specified character. (Step S19). The menu display unit 16 displays the circular menu 60 having the determined size at the determined display position (step S20). The character input device 10 returns to the process of step S10, and acquires the position information of the next user's hand 40.

  As described above, by automatically displaying the annular menu 60 surrounding the character locus 51 drawn on the screen 31 by the character input operation by the user, the transition from the character input operation to the menu operation is facilitated. Further, since an area can be secured inside the area of the circular menu 60, a space for drawing the next character is guaranteed.

  FIG. 8 is a flowchart of a circular menu operation process performed by the character input device according to this embodiment.

  The circular menu operation process shown in FIG. 8 is an example of a process performed when the circular menu 60 is displayed on the screen 31 of the display device 30 in step S20 of FIG.

  When the circular menu 60 is displayed on the screen 31 of the display device 30 (step S20 in FIG. 7), the character input device 10 acquires the movement locus of the pointer 50 (step S30). The movement locus of the pointer 50 may be obtained by accumulating and recording the information on the display position of the pointer 50 obtained by the pointer position determination unit 12, or from the latest continuous position information held in the position information holding unit 14. You may make it ask. Here, it is only necessary to obtain the movement trajectory of the pointer 50 for the time necessary for determining the selection operation to the circular menu 60 and determining the deletion of the circular menu 60.

  The menu selection unit 110 determines whether or not the pointer 50 has moved from the area inside the circular menu 60 area to the area inside the circular menu 60 and further moved to the area inside the area of the circular menu 60 within a predetermined time. (Step S31).

  If the pointer 50 has moved from the inner area to the area of the circular menu 60 and further to the inner area (YES in step S31), the menu selection unit 110 determines that there has been an operation for selecting the circular menu 60 ( Step S32). The menu selection unit 110 executes a process corresponding to the item selected for the circular menu 60 (step S33). The character input device 10 proceeds to the process of step S36.

  If the pointer 50 has not moved from the inner area to the area of the circular menu 60 and further to the inner area (NO in step S31), the menu resetting unit 111 sets the pointer 50 to the area of the circular menu 60 or the outer area. It is determined whether or not there is more time (step S34). If the pointer 50 is not in the circular menu 60 area or the outer area for a predetermined time (NO in step S34), the character input device 10 proceeds to the process in step S37.

  If the pointer 50 is in the area of the circular menu 60 or the outer area for a predetermined time or longer (YES in step S34), the menu reset unit 111 deletes the circular menu 60 from the screen 31 (step S35). The character input device 10 proceeds to the process of step S36.

  The character input device 10 resets the movement locus of the pointer 50 (step S36). The reset here updates the trajectory to be determined for subsequent character input operations and operations on the circular menu 60 to a trajectory after the movement trajectory of the pointer 50 determined as YES in step S31 or step S34. That means. For example, the character input device 10 sets the start position of the trajectory that is the target of the character shape candidate to a position corresponding to the next point of the movement trajectory of the pointer 50 determined as YES in step S31 or step S34. I do.

  The character input device 10 determines whether or not the annular menu 60 is displayed on the screen 31 of the display device 30 (step S37). If the annular menu 60 is displayed (YES in step S37), the character input device 10 returns to the process in step S30. If the annular menu 60 is not displayed (NO in step S37), the character input device 10 ends the annular menu operation process.

  FIG. 9 is a flowchart of input character string prediction processing by the character input device of this embodiment.

  The input character string prediction process shown in FIG. 9 is an example of a process performed when a character drawn by the user's hand 40 in the space is specified in step S17 of FIG.

  When the character is specified (step S17 in FIG. 7), the character specifying unit 15 adds the specified character to the undetermined character string stored in the character string holding unit 17 (step S40). The character input device 10 displays the unconfirmed character string stored in the character string holding unit 17 on the character string display unit 61 of the screen 31 of the display device 30 (step S41).

  The character string predicting unit 18 predicts the character string that the user intends to input from the undetermined character string stored in the character string holding unit 17 (step S42). The character string predicting unit 18 displays each obtained predicted character string on the predicted character string display unit 62 of the screen 31 of the display device 30 (step S43).

  FIG. 10 is a predicted character string assignment display processing flowchart by the character input device of this embodiment.

  The circular menu operation processing shown in FIG. 10 is an example of processing performed when it is determined in the circular menu operation processing shown in FIG. 8 that the item “predictive selection” is selected from the circular menu 60.

  When the item “predictive selection” is selected from the circular menu 60, the predicted character string assigning unit 19 assigns each predicted character string obtained by the character string predicting unit 18 to each item of the circular menu 60. (Step S50). The menu display unit 16 changes the display of each item of the circular menu 60 according to the assignment of each predicted character string (step S51).

[Embodiment 2]
In the example of the first embodiment described above, the display position and size of the displayed circular menu 60 are determined according to the character locus 51 drawn at an arbitrary position on the screen 31 by moving the hand 40 by the user. ing. On the other hand, in the example of the second embodiment described below, the circular menu 60 is displayed at a certain size on the screen 31 in a certain size, and the user can arbitrarily specify the area inside the area of the circular menu 60. The display position of the pointer 50 moved on the screen 31 by moving the hand 40 at the position is converted. In the first embodiment and the second embodiment, the approach is different, but the user moves the hand 40 in the space, and the circular menu 60 is displayed around the characters drawn on the screen 31. The results are consistent at points.

  In the second embodiment, the operation of each functional unit in the character input device 10 shown in FIG. 1 is substantially the same as in the first embodiment. In the character input device 10, the operation differs greatly between the first embodiment and the second embodiment in the operation of the pointer position determination unit 12 and the operation of the menu display unit 16.

  In a state where the circular menu 60 is not displayed, the pointer position determination unit 12 determines the display position of the pointer 50 on the screen 31 of the display device 30 by the same processing as in the first embodiment. That is, the display position of the pointer 50 is obtained by converting the acquired position information using the default conversion parameter regardless of the display position of the subsequent circular menu 60. The position conversion using the default conversion parameters is, for example, a position conversion that associates the camera image with the screen 31 described in the first embodiment or a position conversion based on calibration set in advance by the user. is there.

  When a character is specified by the character specifying unit 15 in a state where the circular menu 60 is not displayed on the screen 31, the menu display unit 16 sets the circular menu 60 at a predetermined display position on the screen 31 of the display device 30. Display in size.

  In a state where the circular menu 60 is displayed on the screen 31 after the character is specified, the pointer position determination unit 12 is based on the display position and size of the circular menu 60, the position and size of the character locus 51, and the acquired position information. Thus, the display position of the pointer on the screen 31 of the display device 30 is determined. Specifically, after the circular menu 60 is displayed, the pointer position determination unit 12 converts the conversion parameter so that the character locus 51 fits in an area inside the displayed circular menu 60 with an appropriate margin. And the acquired position information is converted to determine the display position of the pointer 50.

For example, the center point of the circumscribed rectangle of the character locus 51 is (X _c , Y _c ), and the distance between the center point and the point having the maximum distance from the center point among the points constituting the character locus 51 is R _c. And Further, the center point of the circular menu 60 is (X _m , Y _m ), and the radius of the area inside the area of the circular menu 60 is R _m . At this time, an expression for obtaining the display position (X, Y) of the pointer 50 from the acquired position information (x, y) is, for example, as follows.

X = (x−X _c ) × R _m × α / R _c + X _m
Y = (y−Y _c ) × R _m × α / R _c + Y _m
Here, α is a constant and takes a value smaller than 1. As the value of α is smaller, the display position of the pointer 50 is assigned to an area smaller than the user's hand 40 swing. When the value of α is small, it is difficult to select the circular menu 60, but it is less likely that the user erroneously performs a menu selection operation when the user moves the hand 40 in the air to input characters. Conversely, when the value of α is large, it becomes easier to select the circular menu 60, but when the user moves the hand 40 in the air and inputs a character, the user selects the circular menu 60 by mistake. There is a high possibility that it will end.

  FIG. 11 is a diagram for explaining the display of the circular menu according to the second embodiment.

  FIG. 11A shows a state in which the user moves the pointer 50 on the screen 31 by the operation of the hand 40 when the circular menu 60 is not displayed. FIG. 11B shows a state after the character is specified by the character specifying unit 15 and the circular menu 60 is displayed.

  For example, it is assumed that the user is inputting characters by operating the pointer 50 on the right side of the screen 31 as shown in FIG. 11A by moving the hand 40 in the space. This state is maintained until the annular menu 60 is displayed.

  When the character drawn by the user is specified, the circular menu 60 is displayed in a predetermined size at a predetermined position, and the pointer 50 and the character locus 51 are displayed in the area inside the displayed circular menu 60 area. At this time, the user does not greatly change the position where the hand 40 is moved, and moves the hand 40 at the same space position as the initial position. The character input device 10 automatically converts the display position of the pointer 50 and the character locus 51 to the area inside the area of the annular menu 60 by using the conversion parameter obtained from the annular menu 60 and the original character locus 51. Yes.

  Thereafter, until the circular menu 60 is erased from the screen 31, the character input device 10 determines the display position of the pointer 50 using the conversion parameter obtained when the circular menu 60 is displayed, and places the pointer 50 on the display screen. indicate.

  FIG. 12 is a flowchart of character specification / circular menu display processing by the character input device according to the second embodiment.

  In the character input device 10, the position information acquisition unit 11 acquires the position information of the user's hand 40 in the space from the tracking device 20 (step S60).

  The pointer position determination unit 12 determines the display position of the pointer 50 on the screen 31 of the display device 30 by using the conversion parameter for the acquired position information (step S61). Here, the pointer position determination unit 12 uses the default conversion parameter before the character is specified and the circular menu 60 is displayed, and after the circular menu 60 is displayed, the pointer position determination unit 12 determines the conversion parameter obtained in step S69. Use. The pointer display unit 13 displays the pointer 50 at the determined display position on the screen 31 of the display device 30 (step S62).

  The character input device 10 additionally records the acquired position information in the position information holding unit 14 (step S63). The acquired position information is added to the information of the movement trajectory of the hand 40 of the user from which the character shape candidates are extracted.

  In the character specifying unit 15, the character shape candidate extracting unit 151 extracts character shape candidates from the continuous position information stored in the position information holding unit 14, that is, information on the movement locus of the user's hand 40 (step S 64). The character recognition unit 152 performs character recognition on the extracted character shape candidates (step S65). The recognition confidence of the recognized character is obtained by character recognition for the character shape candidate. The recognized character specifying unit 153 determines whether the recognition certainty is greater than or equal to a predetermined value (step S66). If the recognition certainty is not greater than or equal to the predetermined value (NO in step S66), the character input device 10 returns to the process in step S60 and acquires the position information of the next user's hand 40.

  If the recognition certainty is greater than or equal to a predetermined value (YES in step S66), the character recognized by the character recognition with respect to the character shape candidate is specified as the character drawn by the movement of the user's hand 40 (step S67). The menu display unit 16 displays a circular menu 60 having a predetermined size at a predetermined display position on the screen 31 of the display device 30 (step S68).

  The pointer position determination unit 12 calculates a conversion parameter based on the position and size of the circular menu 60 and the position and size of the character locus 51 (step S69). The pointer position determination unit 12 determines the display position after the conversion of the pointer 50 displayed on the screen 31 using the calculated conversion parameter (step S70). The pointer display unit 13 deletes the pointer 50 displayed on the screen 31, and displays the pointer 50 at the display position determined in step S70 (step S71). When the movement locus of the pointer 50 is displayed on the screen 31, the movement locus is also converted and displayed again.

  The character specifying unit 15 resets the continuous position information of the target from which the character shape candidates are extracted (step S72). The character input device 10 returns to the process of step S60, and acquires the position information of the next user's hand 40.

  Also in the second embodiment, the advantages obtained for the user are almost the same as those of the first embodiment. The second embodiment further has an advantage that the user can easily see because the annular menu 60 comes to a fixed position.

  Although the present embodiment has been described above, the present invention can naturally be modified in various ways within the scope of the gist thereof.

  For example, in the present embodiment, the position of a specific object in space is acquired using a camera image, but the present invention is not limited to this. For example, a position of a specific object in space may be acquired using a device including a terminal device worn by a user who can acquire the position using a gyro sensor, an acceleration sensor, an ultrasonic wave, or the like.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
A position information acquisition unit for acquiring position information of a specific object in space;
A position information holding unit for storing the position information;
A pointer position determination unit for determining a display position of the pointer on the screen of the display device based on the position information;
A pointer display unit for displaying a pointer at the display position of the pointer on the screen of the determined display device;
A character specifying unit for specifying a character drawn by movement of the specific object based on the position information stored in the position information holding unit;
A menu display unit for displaying a circular menu surrounding the movement locus of the pointer corresponding to the specified character;
A character input device comprising: a menu selection unit that determines a selection operation for the circular menu based on a positional relationship between the circular menu and the pointer.

(Appendix 2)
The said menu display part determines the display position and size of the said cyclic | annular menu based on the position and size of the movement locus | trajectory of the pointer corresponding to the specified character. Character input device.

(Appendix 3)
The menu display unit displays the circular menu in a predetermined size at a predetermined display position,
The pointer position determination unit, based on the display position and size of the circular menu, the position and size of the movement path of the pointer corresponding to the specified character, and the position information, the pointer position on the screen of the display device The character input device according to appendix 1, wherein a display position is determined.

(Appendix 4)
When the pointer moves from a region inside the circular menu region to the circular menu region and further moves to a region inside the circular menu region within a predetermined time, The character input device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein it is determined that a selection operation has been performed on a circular menu.

(Appendix 5)
A character string holding unit for storing a character string obtained by sequentially adding the specified characters;
A character string prediction unit that predicts an input character string from character strings stored in the character string holding unit;
The character input device according to any one of supplementary notes 1 to 4, further comprising: a predicted character string assigning unit that assigns the predicted input character string to the circular menu.

(Appendix 6)
Supplementary notes 1 to 5, further comprising: a menu reset unit that deletes the circular menu when the pointer is in the circular menu area or an area outside the circular menu area for a predetermined time or more. The character input device described in any of the above.

(Appendix 7)
Computer
Obtain location information of a specific object in space,
Based on the position information, the display position of the pointer on the screen of the display device is determined,
A pointer is displayed at the display position of the pointer on the determined display device screen;
Based on the position information stored and stored in the storage unit, the character drawn by the movement of the specific object is specified,
Displaying a circular menu surrounding the movement trajectory of the pointer corresponding to the identified character;
A character input program for executing a process of determining a selection operation for the circular menu based on a positional relationship between the circular menu and the pointer.

(Appendix 8)
Computer
Obtain location information of a specific object in space,
Based on the position information, the display position of the pointer on the screen of the display device is determined,
A pointer is displayed at the display position of the pointer on the determined display device screen;
Based on the position information stored and stored in the storage unit, the character drawn by the movement of the specific object is specified,
Displaying a circular menu surrounding the movement trajectory of the pointer corresponding to the identified character;
The character input method characterized by performing the process of judging selection operation to the said circular menu based on the positional relationship of the said circular menu and the said pointer.

DESCRIPTION OF SYMBOLS 10 Character input device 11 Position information acquisition part 12 Pointer position determination part 13 Pointer display part 14 Position information holding part 15 Character specific part 151 Character form candidate extraction part 152 Character recognition part 153 Recognition character specific part 16 Menu display part 161 Menu position determination part 17 Character string holding unit 18 Character string predicting unit 19 Predicted character string assigning unit 110 Menu selection unit 111 Menu reset unit 20 Tracking device 30 Display device

Claims (7)

A position information acquisition unit for acquiring position information of a specific object in space;
A position information holding unit for storing the position information;
A pointer position determination unit for determining a display position of the pointer on the screen of the display device based on the position information;
A pointer display unit for displaying a pointer at the display position of the pointer on the screen of the determined display device;
A character specifying unit for specifying a character drawn by movement of the specific object based on the position information stored in the position information holding unit;
A menu display unit for displaying a circular menu surrounding the movement locus of the pointer corresponding to the specified character;
A character input device comprising: a menu selection unit that determines a selection operation for the circular menu based on a positional relationship between the circular menu and the pointer. The said menu display part determines the display position and size of the said cyclic | annular menu based on the position and size of the movement locus | trajectory of the pointer corresponding to the specified character. Character input device. The menu display unit displays the circular menu in a predetermined size at a predetermined display position,
The pointer position determination unit, based on the display position and size of the circular menu, the position and size of the movement path of the pointer corresponding to the specified character, and the position information, the pointer position on the screen of the display device The character input device according to claim 1, wherein a display position is determined. When the pointer moves from a region inside the circular menu region to the circular menu region and further moves to a region inside the circular menu region within a predetermined time, The character input device according to any one of claims 1 to 3, wherein it is determined that an operation for selecting a circular menu has been performed. A character string holding unit for storing a character string obtained by sequentially adding the specified characters;
A character string prediction unit that predicts an input character string from character strings stored in the character string holding unit;
The character input device according to any one of claims 1 to 4, further comprising: a predicted character string assigning unit that assigns the predicted input character string to the circular menu. Computer
Obtain location information of a specific object in space,
Based on the position information, the display position of the pointer on the screen of the display device is determined,
A pointer is displayed at the display position of the pointer on the determined display device screen;
Based on the position information stored and stored in the storage unit, the character drawn by the movement of the specific object is specified,
Displaying a circular menu surrounding the movement trajectory of the pointer corresponding to the identified character;
A character input program for executing a process of determining a selection operation for the circular menu based on a positional relationship between the circular menu and the pointer. Computer
Obtain location information of a specific object in space,
Based on the position information, the display position of the pointer on the screen of the display device is determined,
A pointer is displayed at the display position of the pointer on the determined display device screen;
Based on the position information stored and stored in the storage unit, the character drawn by the movement of the specific object is specified,
Displaying a circular menu surrounding the movement trajectory of the pointer corresponding to the identified character;
The character input method characterized by performing the process of judging selection operation to the said circular menu based on the positional relationship of the said circular menu and the said pointer.

Images