JP2013196391A - Information processing apparatus, menu selection program and menu selection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve menu selection with a simple operation.SOLUTION: The information processing apparatus includes: a position information input section for inputting position information on a predetermined object; a circular motion calculation section for analyzing the change of the position information on the object obtained by the position information input section, and determining whether or not the object is performing the circular motion; a circuiting condition determination section for, when it is determined that the object is performing the circular motion by the circular motion calculation section, determining whether or not the object satisfies predetermined circuiting conditions; a menu selection mode control section for, on the basis of the determination result obtained by the circuiting condition determination section, executing a menu selection mode in which menu items are displayed at a display section, and a predetermined item is selected; and a menu selection section for, with respect to the menu item displayed at the display section in the menu selection mode, selecting the predetermined item on the basis of the central position of the circular motion obtained by the circular motion calculation section and the position information on the object obtained by the position information input section.

Description

  The present invention relates to an information processing apparatus, a menu selection program, and a menu selection method for realizing menu selection by a simple operation.

  Conventionally, in applications and the like, it has been an important issue to allow a user to perform comfortable menu opening and closing and selection operations. Therefore, conventionally, a technique is known in which a hand movement or the like is captured using a tracking device or the like that reads the movement of an object in space, and the position information of the hand is used for a user interface. For example, conventionally, a selection item displayed on the screen is captured by capturing a two-dimensional movement of the hand, and the selection item is determined using some determination method.

  Here, as a specific determination method, for example, there is a method of determining the selection content by twisting the wrist with respect to the pie menu arranged in a circle. Further, conventionally, there is a method for detecting repetition of a cyclic gesture such as rotating or reciprocating a hand, moving a selection cursor according to the number of detection times and speed, and determining a selection item by a non-circular gesture. is there.

  Conventionally, first, the position of the interactive device held in the user's hand is detected, and the device position is associated with the center of the spherical menu by a specific action by a gesture or the like. Next, conventionally, there is a method of selecting a menu by an operation of penetrating a menu item from the inside, returning to the inside again, and moving to another item to select another menu.

  Conventionally, there is a method in which a cursor corresponding to the position of the hand appears at the center of a circular menu when the hand is in a predetermined shape, and is selected and confirmed when the cursor is moved outside the options. Conventionally, there is a method of displaying a circular menu that surrounds the user within the reach of the user's hand together with the user video. Conventionally, there is a method of estimating a center position of a circular motion of a hand, obtaining a rotation angle, and selecting a menu item based on the rotation angle (see, for example, Patent Documents 1 to 6).

International Publication No. 2008/029467 Pamphlet JP 2010-0779848 A Special table 2007-506164 JP-A-9-128141 Special table 2010-534895 gazette JP 2011-145842 A

  However, for example, in the methods shown in Patent Document 1 and Patent Document 4 described above, it is necessary to make an unnatural hand movement such as twisting the wrist or making it a predetermined shape. You must have Further, in the methods as shown in Patent Document 2 and Patent Document 6, since the hand-circulating movement is assigned to the cursor movement, the pointing is relatively troublesome and tired. In the methods shown in Patent Document 3 and Patent Document 4, it is necessary to move the hand so as to penetrate the item from the position in the real space associated with the center of the circular or spherical menu. It is difficult to select when there are many. Furthermore, the technique as shown in Patent Document 5 requires an operation of aligning a hand with an object while looking at the screen, which increases body movement and makes it difficult to make an intuitive selection. That is, in the conventional technology as described above, the menu selection by a simple operation by the user cannot be realized.

  In view of such a problem, the disclosed technique aims to realize menu selection by a simple operation.

  An information processing apparatus according to one aspect of the disclosure analyzes a position information input unit that inputs position information of a predetermined object, a change in the position information of the object obtained by the position information input unit, and the object performs a circular motion. A circular motion calculating unit that determines whether the object is performing a circular motion by the circular motion calculating unit, and a circular condition that determines whether the object satisfies a predetermined circular condition A menu selection mode control unit for executing a menu selection mode for displaying a menu item on the display unit and selecting a predetermined item based on the determination result obtained by the determination unit, the circulation condition determination unit, and the menu selection For the menu item displayed on the display unit in the mode, the center position of the circular motion obtained by the circular motion calculation unit and the position of the object obtained by the position information input unit. And a menu selecting section for selecting the predetermined item based on the information.

  According to the disclosed technique, menu selection by a simple operation can be realized.

It is a figure which shows an example of a function structure of the information processing apparatus in this embodiment. It is a figure which shows an example of the hardware constitutions of the information processing apparatus in this embodiment. It is a figure which shows the specific example of operation which opens a menu. It is a figure which shows the specific example of a menu display. It is a figure which shows an example of menu information. It is a figure which shows the specific example of menu selection operation. It is a figure which shows the specific example of menu item confirmation operation. It is a flowchart (the 1) which shows an example of the menu selection process in this embodiment. It is a flowchart (the 2) which shows an example of the menu selection process in this embodiment. It is a flowchart (the 3) which shows an example of the menu selection process in this embodiment.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

<Information processing device: functional configuration example>
FIG. 1 is a diagram illustrating an example of a functional configuration of the information processing apparatus according to the present embodiment. An information processing apparatus 10 shown in FIG. 1 includes a tracking input unit (position information input unit) 11, a tracking position storage unit 12, a circular motion calculation unit 13, a circulation condition determination unit 14, and a menu selection mode control unit 15. , A circular movement storage unit 16, a menu selection determination unit 17, a menu selection control unit 18, a menu selection unit 19, and a menu information holding unit 20.

  The tracking input unit 11 inputs position information on a three-dimensional space of a predetermined object such as a user's hand obtained by the tracking device 21. Here, the tracking device 21 acquires an image using an image receiving device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). For example, the tracking device 21 performs image analysis from the acquired video, extracts a portion estimated to be a predetermined object, and acquires a position in the space including the depth from the position and size of the extracted portion.

  The predetermined object is not limited to the user's hand, for example, a part of the user's body other than the hand, an object worn on the body by the user, an object held by the user's hand, etc. Also good.

  Further, as the tracking device 21, for example, two image receiving devices installed a little apart from each other are used to match images obtained from the respective image receiving devices, and depth estimation is performed from a slight deviation of the images to determine the position of the user's hand. Information may be acquired. In addition, the tracking device 21 irradiates a user's hand with, for example, an infrared light emitting diode (LED) or an infrared laser, captures the reflected light with a sensor, and acquires position information by a method called Time Of Flight (TOF). May be. The tracking device 21 may acquire position information including the depth of the user's hand, for example, by analyzing the distortion of the projection pattern.

  The tracking device 21 can perform the presence / absence or tracking of a predetermined object such as a user's hand by using the above-described method. Specifically, as a technique for extracting the part of the user's hand to be tracked, for example, there is a technique of attaching an object of a specific color to the hand and extracting based on the specific color. Further, as a hand tracking method, for example, a predetermined shape pattern such as a specific shape of a hand is set in advance, and a hand portion is extracted by matching with the shape pattern of the hand, and then an optical flow is performed. There is a method of tracking the movement of the hand by a tracking method such as the above. The optical flow is a method of tracking the movement of an object by expressing the movement of the object as a vector in a visual expression (for example, a digital image that is continuous in time). As another hand extraction method, for example, there is a method of determining a hand portion from the skin color and three-dimensional shape of a hand when depth information is obtained first.

  Note that the hand extraction method and tracking method described above are not limited thereto. In this embodiment, the user's hand is used as the tracking portion. However, as long as there is no problem in operation, an object held in the hand, an object attached to the body, or other parts of the body may be tracked. .

  Further, the tracking input unit 11 acquires position information from the tracking device 21 at a short time interval of, for example, about 1/30 seconds. In addition, about a time interval, it is not limited to this, For example, position information should just be acquired sequentially by the space | interval which can catch a user's hand movement.

  The tracking position storage unit 12 stores the position information of the user's hand obtained by the tracking input unit 11. This memory may be left for the latest fixed time, and the old one may be discarded. That is, the fixed time is, for example, 10 seconds or 1 minute, and may be within a range necessary and sufficient for analyzing the circular motion of the user's hand. In the case where only the latest fixed time is continuously stored, a ring buffer or the like can be used as the tracking position storage unit 12, for example, but is not limited thereto.

  The circular motion calculation unit 13 analyzes the change in the position information of the user's hand obtained from the tracking position storage unit 12 and determines whether the movement of the user's hand is a circular motion. Specifically, the circular motion calculation unit 13 determines whether or not it is a circular motion depending on how much the change in the position information of the user's hand rotates (circular motion) with respect to a certain center.

  Note that when the circular motion calculation unit 13 recognizes that the change in the position information of the user's hand has moved by a predetermined angle from a reference position, for example, in a predetermined circular orbit, the circular motion calculation unit 13 It can be recognized that a circular orbit is drawn even if the hand is not turned around once.

  In addition, when it is determined that the user's hand is making a circular motion, the circular motion calculation unit 13 acquires various parameters related to the circular motion of the user's hand. Specifically, the circular motion calculation unit 13 acquires, for example, the center position of the circular motion as a parameter. The circular motion calculation unit 13 acquires at least one of, for example, a circular motion plane, the number of laps, a radius, and the like as parameters. Furthermore, the circular motion calculation unit 13 acquires, as parameters, the rotational direction of the circular motion, the inclination of the circular motion plane with respect to the tracking device 21, and the like.

  As for the object motion analysis method in the circular motion calculation unit 13, for example, first, the position information stored in the tracking position storage unit 12 is traced backward from the latest one, and a certain linearity is obtained by the least square method or the like. Summarize the parts where is recognized and replace with a straight line representing the part. Thereafter, the circular motion calculation unit 13 obtains the center point of the circumscribed rectangle at the intersection when a certain number of straight lines are aligned. Furthermore, the circular motion calculation unit 13 sequentially obtains the angles formed by the vectors extended to the center point from the adjacent points in time series order in the reverse direction of the time series using an outer product or the like, and the angles are reversed. Without checking, it is confirmed whether or not the accumulated angle satisfies a predetermined angle such as one round.

  Next, when the circular motion calculation unit 13 satisfies a predetermined angle and the average distance between the center point and each intersection in the range is R, the distance between the center point and the intersection is “a × R”. To “b × R”. Here, a and b described above are constants such that “0 <a <1” and “1 <b”. Further, when each distance value is within the range, the circular motion calculating unit 13 can further obtain a plane from each intersection by the least square method or the like.

  Further, if the deviation between the intersection and the plane is within the range, the circular motion calculation unit 13 considers that the predetermined object is performing a circular motion for a predetermined angle on the plane. Similarly, the circular motion calculation unit 13 can obtain the number of rounds of circular motion by retroactively analyzing past position information.

  Furthermore, the circular motion calculation unit 13 can calculate the radius of the circle by calculating the distance between the center point and each intersection within a range that is considered to be circular, for example, and taking the average thereof. Further, the circular motion calculation unit 13 can calculate the rotation direction of the circular motion and the inclination of the circular motion plane with respect to the tracking device 21 from the time series change of the position coordinates of the user's hand.

  Note that the center point and the plane may be different for each lap, but in such a case, for example, use the latest lap in the time series or use an average method. The error can be eliminated. Furthermore, in the present embodiment, there may be a case where the center and the plane are greatly different between the laps. In such a case, the irregular hand is selected by selecting the maximum number of laps satisfying a preset condition. Can be excluded.

  The circulation condition determination unit 14 performs determination based on a predetermined rotation condition as a determination condition for shifting to the menu selection mode. For example, the circulation condition determination unit 14 determines from the information obtained from the circular motion calculation unit 13 that there is a rotation of more than a certain number of rotations within a certain time. Here, when the state before entering the menu selection mode is a so-called free cursor state, the circulation condition determination unit 14 is less likely to occur in a natural operation at the time of the free cursor when the rotation condition is determined and the fixed number of rotations. Should be something. Here, the free cursor state means, for example, a state in which the cursor can freely move within the screen of the display unit 22 and a target on the screen can be selected, but is not limited thereto. Therefore, it is preferable that the circulation condition determination unit 14 sets a condition that can hardly occur unless it is intentionally circulated, for example, it is rotated twice or more within 2 seconds. Note that a plurality of conditions can be set as the above-described circulation conditions. For example, the circulation condition determination unit 14 may add a condition other than the circulation condition, for example, a condition that the radius of the circular motion is equal to or larger than a predetermined size or the direction of the circular motion is a predetermined direction. .

  The menu selection mode control unit 15 switches to the menu selection mode when a predetermined circulation condition is satisfied by the circulation condition determination unit 14. Here, switching to the menu selection mode means, for example, setting a value or a flag indicating that the mode has been switched, but is not limited thereto. In addition, the menu selection mode control unit 15 acquires each parameter described above from the circular motion calculation unit 13. Specifically, the menu selection mode control unit 15 acquires the center position of the circular motion from the circular motion calculation unit 13 as a parameter. In addition, the circular motion calculation unit 13 acquires information such as a circular motion plane, the number of laps, a radius, a rotation direction of the circular motion, and a tilt of the circular motion plane as parameters.

  The menu selection mode control unit 15 outputs information indicating that the mode has been switched and the parameters obtained from the circular motion calculation unit 13 to the circular motion storage unit 16.

  The circular motion storage unit 16 acquires and holds the center position of the circular motion, the circular motion plane, the radius of the circle, and the like when the menu selection mode control unit 15 switches to the menu selection mode. However, information such as the radius of the circle, the rotation direction of the circular motion, and the inclination of the circular motion plane may not be retained when not used for the determination by the menu selection unit 19.

  The menu selection determination unit 17 determines the item selected when the hand position obtained from the tracking input unit 11 is more than a certain distance from the circular motion plane held in the circular motion storage unit 16 in the menu selection mode. Alternatively, an operation such as cancellation is determined and the determined operation is executed. Note that “confirmation” includes, for example, a process of executing a predetermined execution event preset in the target menu item, but is not limited thereto. Canceling includes, for example, processing such as canceling the menu selection mode (clearing the flag or the like), returning to the state before selection is confirmed, or deleting the display of the menu item from the screen, but is not limited thereto. It is not a thing.

  Moreover, it is preferable that the above-mentioned fixed distance is sufficiently larger than the hand movement blur on the plane for menu selection. For example, even if a person moves his / her hand so as to move on the plane, in reality, when the deviation from the plane occurs, for example, about 5 cm at the maximum, the constant distance value in the menu selection determination unit 17 is about 5 cm. A sufficiently larger value, for example, about 10 cm is preferable. Further, since the menu selection determination unit 17 has two directions perpendicular to the circular motion plane, the movement of the hand in either direction can be determined as the selection item and the other can be canceled.

  When the menu selection control unit 18 is in the menu selection mode, the movement direction by the change in the position of the user's hand obtained from the tracking position storage unit 12 is predetermined with respect to the circular motion plane held in the circular motion storage unit 16. In the case of the direction, the change of the selection item by the menu selection unit 19 described later is invalidated. Specifically, for example, the menu selection control unit 18 determines whether the moving direction of the user's hand is parallel or perpendicular to the circular motion plane, and when the determined moving direction is vertical, the menu selection control unit 18 will be described later. The change of the menu item in the menu selection unit 19 to be performed is invalidated. Thereby, in this embodiment, the determination of menu item determination and cancellation by the menu selection determination unit 17 and the determination for changing the menu item can be processed separately.

  Here, in the present embodiment, if the menu selection determination unit 17 simply determines with the threshold of the distance from the plane, another item may be selected at the moment of the determinating operation depending on the hand shake. Therefore, in the menu selection control unit 18 described above, the distance is within the threshold of the distance according to the direction (movement direction) of the hand with respect to the circular motion plane held in the circular motion storage unit 16 so as not to occur. However, the selected item is not changed. The direction in which the hand advances with respect to the circular motion plane can be divided into two, for example, by being quantized by being relatively parallel or perpendicular.

  The menu selection unit 19 determines menu selection items based on the relationship between the hand position obtained from the tracking input unit 11 and the center position of the circle held by the circular motion storage unit in the menu selection mode. Specifically, for example, when menu items are arranged in a circle around the center position of the circle with reference to the center position of the circle, first, the menu selection unit 19 performs quantization based on the number of menu items. Thereafter, the menu selection unit 19 can select the corresponding menu item by associating the direction (angle) formed by the current hand position with the quantized value described above with reference to the center position of the circle. it can.

  The menu selection unit 19 can also quantize a certain range of the distance between the center position of the circle and the position of the hand based on the number of menu items and perform selection according to the value. Furthermore, the menu selection unit 19 can select the menu by both the angle and the distance by combining the two described above. In this case, for example, the menu selection unit 19 considers that the number of menu items is divided into n, quantizes a certain range of distance to n, and quantizes the angle by the number of menu items / n. In this way, in this embodiment, one of the menu items is assigned to a certain range of distances and angles. Further, by using the above-described method, this embodiment has an effect that the angle is not subdivided too much even when the number of menu items is large.

  The menu selection unit 19 can also produce an effect of changing the type of selection for a certain menu item selected by an angle using a distance value. Examples of selection types include dark and light colors for color selection, copy, delete, move, etc. for file selection, e-mail, phone call, schedule for name etc. This indicates an operation instruction after the selection such as asking is confirmed.

  In this case, the distance value is determined by comparing the radius of the circle held in the circular motion storage unit 16 with a reference that is arbitrarily set by the user by dividing it into almost equal, sufficiently small, or sufficiently large. Can do. For example, the menu selection unit 19 regards the value as “sufficiently small” when the distance value between the hand and the center point is, for example, about 5 cm or more smaller than the radius of the held circle or 1/2 or less of the radius of the circle. It can be regarded as “sufficiently large” when it is 5 cm or more larger than the radius of the circle formed, or when it is 1.5 times or more of the radius.

  Further, the menu selection unit 19 acquires the rotation direction of the circular motion and / or the inclination of the circular motion plane from the circular motion storage unit 16 before the menu display is performed as a process at the time of changing the menu. Change the type of menu to display based on the information. For example, the menu selection unit 19 may display different menus clockwise and counterclockwise even when the hand is rotated on the same circular motion plane. In addition, when the menu selection unit 19 moves the hand in a plane perpendicular to the tracking device 21, for example, the menu selection unit 19 displays an operation menu for the application (software) being displayed displayed on the display unit 22 at that time. You may change to a menu item. The menu selection unit 19 may change the menu item to a start menu of a frequently used application when the tracking device 21 is circularly moved in the front-rear depth direction, for example.

  The menu information holding unit 20 holds one or a plurality of menu information such as menu items to be displayed on the display unit 22, display positions of the respective items, execution events for the respective items, and the like. The menu information holding unit 20 outputs a target menu item to the display unit 22 when the menu selection unit 19 instructs to display or change the menu. Further, when the selection item is confirmed by the menu selection determination unit 17, the menu information holding unit 20 extracts an execution event corresponding to the content of the item, and executes the extracted execution event.

  The display unit 22 displays menus and selection items on the display device in the menu selection mode. The display format here is not particularly limited. For example, the menu items are arranged in an annular shape, vertically arranged, horizontally arranged, tiled, etc. There may be. In addition, it is preferable that the display unit 22 clearly indicates which item is selected when the user's hand is moved to which position, for example. For example, the display unit 22 can use a method of arranging menu items on a trajectory on the screen imitating the rotational movement trajectory of the hand, but is not limited thereto. The display unit 22 is not limited to the display of menus and selection items, and also displays the contents of an application being executed, for example. The display unit 22 is, for example, a liquid crystal display or a liquid crystal monitor.

  Here, in the example of FIG. 1, the tracking device 21 and the display unit 22 are provided outside the information processing apparatus 10, but are not limited thereto, and are provided integrally with the information processing apparatus 10, for example. It may be.

<Information processing device: hardware configuration example>
FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing apparatus according to the present embodiment. FIG. 2 shows a hardware configuration example of a computer that realizes the menu selection process in the present embodiment.

  A computer that implements the information processing apparatus 10 illustrated in FIG. 2 includes a central processing unit (CPU) 31, a memory 32, a storage device 33, a communication device 34, a drive device 35, an input device 36, and an output device 37. Have In FIG. 2, a recording medium 39 is also shown.

  The CPU 31 controls the entire information processing apparatus 10 in order to execute the menu selection process in the present embodiment. The CPU 31 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an operating system (OS) and an execution program stored in the memory 32. Thus, each process in the present embodiment is executed. Various information necessary during the execution of the program can be acquired from the storage device 33, and an execution result or the like can also be stored.

  Specifically, the CPU 31 reads the movement of the user's hand (position information, tracking information) and the like from the input device 36 and displays a menu on the screen of the output device 37 using the menu selection method in the present embodiment. Menu selection.

  Further, the CPU 31 reads various information and programs necessary for the present embodiment from the recording medium 39 via the drive device 35, and writes information such as the results and progress of the processing in the present embodiment to the recording medium 39. Control.

  The memory 32 is, for example, a main storage device. The memory 32 stores an execution program, input data, and the like according to the present embodiment, and executes menu selection processing and the like using the information. As the memory 32, for example, Read Only Memory (ROM), Random Access Memory (RAM), or the like can be used.

  The storage device 33 is an auxiliary storage device that stores an execution program, a control program provided in a computer, and the like, and performs input / output as necessary. As the storage device 33, for example, storage means such as a hard disk can be used. The storage device 33 can store various conditions, input data, output data, and the like set in advance in the present embodiment.

  For example, the communication device 34 uses a communication network represented by the Internet, a Local Area Network (LAN), or the like to connect to an external device and perform data transmission and reception. Note that the communication device 34 is connected to an external device to acquire, for example, a menu selection program and other input data in the present embodiment, and transmit information such as execution results and progress of the menu selection process to the external device. Can do. The communication device 34 may be connected to the tracking input unit 11 as an external device. Thereby, in this embodiment, information from the tracking input unit 11 can be acquired via the communication device 34.

  The drive device 35 can set a recording medium 39 on which an execution program such as menu selection processing in the present embodiment is recorded. Further, the drive device 35 reads out an execution program included in the recording medium 39 under the control of the CPU 31 and outputs it to the storage device 33 for installation. The drive device 35 can write various data to the recording medium 39 under the control of the CPU 31.

  The input device 36 inputs instruction information from the user. For example, a keyboard / mouse can be used as the input device 36. The input device 36 includes, for example, an image receiving device such as a camera that implements the tracking device 21.

  The output device 37 displays the menu selection process in this embodiment on the screen. As the output device 37, for example, a display device such as a display or a monitor can be used. However, the output device 37 is not limited to this, and a speaker or the like for outputting sound or the like may be provided.

  The bus 38 connects the CPU 31, the memory 32, the storage device 33, the communication device 34, the drive device 35, the input device 36, and the output device 37 so that data can be input and output.

  The recording medium 39 is a medium that can be connected to the drive device 35 and can read and write data. As the recording medium 39, for example, a portable recording medium such as a Compact Disc Recordable (CD-R) or a Digital Versatile Disc Recordable (DVD-R) can be used. Note that the recording medium 39 is not limited to the type described above, and may be, for example, a Universal Serial Bus (USB) memory. The recording medium 39 may be a semiconductor memory such as a flash memory, a hard disk drive, or the like.

  That is, each functional unit included in the information processing apparatus 10 can be realized by hardware and software programs such as the CPU 31 and the memory 32 included in the computer as illustrated in FIG. The computer-executable program is stored in the storage device 33, read into the memory 32 at the time of execution, and executed by the CPU 31.

  The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time a program is transferred from, for example, a server computer, the computer can sequentially receive the program at the communication device 34 and execute processing according to the received program.

  In the present embodiment, the image processing in the present embodiment can be executed with the hardware configuration described above. Also, by installing the program, the menu selection process can be easily realized with a general-purpose personal computer or the like.

<Specific example of menu selection operation>
Next, an example when this embodiment is seen from the user side is demonstrated using figures. FIG. 3 is a diagram illustrating a specific example of an operation for opening a menu. FIG. 3 shows a state in which the user tries to shift to the menu selection mode from the free cursor state by rotating the hand circularly.

  FIG. 3A shows the display screen 40 of the display unit 22 described above. In the free cursor state, the cursor 41 is displayed on the display screen 40 at a free position. In such a state, for example, as shown in FIG. 3B, when the user's hand 42 is rotated in a predetermined direction with respect to the circular motion plane 45, the user's obtained by the tracking input unit 11 can be obtained. Parameters such as a circular motion center position 44 and a circular motion plane 45 are held using the movement trajectory 43 of the hand 42. These pieces of information are calculated by the circular motion calculation unit 13. In the menu selection mode, parameters such as the circular motion center position 44 and the circular motion plane 45 are stored and held by the circular motion storage unit 16.

  Note that the circular motion plane 45 may not be parallel to the display screen 40. That is, when the user's hand 42 draws a circular orbit, the circular orbit may be drawn, for example, horizontally with the ground. The tracking device 21 can recognize the circular motion of the user's hand 42 even in the above-described operation, and can acquire the circular motion plane 45.

  FIG. 4 is a diagram showing a specific example of menu display. In the present embodiment, as shown in FIG. 4A when the determination condition by the circulation condition determination unit 14 set in advance is satisfied in correspondence with the operation of the user's hand 42 as shown in FIG. The menu item 50 is displayed on the display screen 40 of the display unit 22. The menu item 50 to be displayed is displayed at a predetermined display position based on the menu information held in advance in the menu information holding unit 20.

  For example, in the example shown in FIG. 4A, a plurality of menu items 50 are arranged and displayed in the same annular shape. Note that the display content is not limited to this, and may be arranged in a vertical row or horizontal row or vertically and horizontally at a predetermined interval, for example.

  Further, the number and types of items are not limited to the example of FIG. 4A. For example, the menu information held in the menu information holding unit 20 in advance according to the change condition by the menu selection unit 19 or the like. The contents and arrangement of menu items can be changed based on the above.

  Furthermore, the shape of the item displayed for each item is not limited to a rectangle as shown in FIG. 4A, and may be, for example, a circle, an ellipse, a star, or the like. You may change for every item. For example, when the size is changed for each item, the higher the priority or importance of the item, the different the size or shape of the item may be displayed.

  In the present embodiment, when the menu selection mode is entered as shown in FIG. 4A, the direction from the state immediately after the menu is displayed toward the position of the current hand 42 from the center of the held circular motion. The menu item corresponding to is selected.

For example, when the user positions the hand 42 in a predetermined direction with respect to the center position 44 of the circular motion, the menu item 50 corresponding to the position (“item 5” in the example of FIG. 4A). Is selected. At this time, the position of the user's hand can also select the corresponding menu item 50 based on an angle from a preset predetermined position (θ _{1 in} the example of FIG. 4B).

  Note that the selected menu item 50 can be highlighted. Highlighting means at least one of, for example, changing the color of other items to display, displaying larger than other items, or flashing items, but is not limited to this. is not.

  In the present embodiment, for example, the item information and item contents displayed on the display screen 40 as shown in FIG. 4A are held by the menu information holding unit 20.

  Here, FIG. 5 is a diagram illustrating an example of the menu information. In the example of FIG. 5, an example of the menu table is shown as the menu information. The menu table shown in FIG. 5 includes display items, position information, execution commands, and the like as items, but is not limited thereto.

  In the example of FIG. 5, display position information (x, y) for each display screen 40 is held for each display item (for example, items 1 to 8). In the present embodiment, an item corresponding to the position is selected based on the display position information and the current position information of the user's hand 42. The position information of each item is quantized so that any item can be easily selected even if there is a slight positional deviation.

  In the menu table shown in FIG. 5, predetermined events (for example, commands A to H) are associated with each item, and when an item is confirmed, an execution event corresponding to the item is executed.

  The execution command includes, for example, “open another screen”, “execute application”, “end application”, “save data”, and the like. As other execution commands, for example, if the menu item is a color selection, a dark color or a light color is selected, and if the menu item is a file selection, copy, delete, move, etc. are executed. Can do. Further, as another execution command, if the menu item is a person name or the like, an operation such as e-mailing, calling, or asking for a schedule can be performed, but is not limited thereto. In the present embodiment, the menu is selected by performing a predetermined operation by positioning the item to be selected by the user moving the hand 42.

  In the menu item shown in FIG. 5, each table may be set for each of a plurality of menus. Thereby, in this embodiment, based on the menu change instruction | indication etc. which are obtained by the menu selection part 19, the type of menu items etc. can be changed and it can display on the display part 22. FIG.

  FIG. 6 is a diagram illustrating a specific example of the menu selection operation. In the example of FIG. 6, the user performs menu selection in the menu selection mode. For example, the user selects an item to be selected from the menu items 50 displayed on the display screen 40 of the display unit 22, and straightly moves the hand 42 from the center position 44 of the circular motion to the position on the space corresponding to the item to be selected. Move.

  Here, the center position 44 of the circular motion is the central position of the circular motion of the hand 42 when opening the menu, and the user himself performs the operation of opening the menu. Therefore, the user can sensuously grasp the relative positional relationship of items to be selected from the center position 44 of the circular motion. Specifically, as shown in FIG. 6B, the user moves the hand 42 from the center position 44 of the circular motion in the direction of the item to be selected (in the example of FIG. 6B, the direction of arrow A). The corresponding item (“item 8” in the example of FIG. 6) is highlighted in comparison with the position information in the menu table.

In the present embodiment, not only the position information but also the angle information from the predetermined position (θ _{2 in} the example of FIG. 6B) is moved to collate with the position information of the menu table described above, The corresponding item may be highlighted. Thereby, the user can select an intuitive item.

  FIG. 7 is a diagram showing a specific example of the menu item confirmation operation. In the example of FIG. 7, an operation of moving the user's hand 42 with respect to the held circular motion plane 45 in a predetermined direction (in the direction of arrow B in the example of FIG. 6B) by a certain distance (for example, distance h) or more. , Assigned to the confirmation operation of the menu item. In addition, for the cancellation, an operation of moving the user's hand 42 in a direction opposite to the arrow B direction by a certain distance (for example, a distance h) or more is assigned to the circular motion plane 45. Thereby, the confirmation and cancellation of the menu item in this embodiment can be performed easily.

  Note that the direction of movement may be reversed from the above-described content, and a confirmation or cancellation operation may be assigned by moving the user's hand 42 by a different distance in the same direction. For example, if it is moved by the distance h, the menu item is confirmed, and if it is moved by the distance 3h in the same direction, it is canceled.

  Further, in the present embodiment, the menu item may be set in advance so as to be confirmed or canceled by another predetermined operation. As another predetermined operation, for example, the menu item may be confirmed and canceled by changing the shape of the hand (for example, opening or gripping the hand) or returning the palm. Further, as another predetermined operation, for example, using the speed information of the hand movement, etc., it is possible to perform control such as confirming the menu item when it is faster than the predetermined speed and canceling when it is slower than the predetermined speed. it can.

  In addition, as another predetermined operation, a process such as determining a menu item when the hand 42 is stopped (stopped) for a predetermined time can be performed. Specifically, for example, the menu selection determination unit 17 determines whether the tracking input point is stopped for a certain time, the input point is within a certain distance from the circular motion plane 45, and the input point is based on the radius from the circle center. A menu item can be confirmed when it is within a predetermined distance.

  In addition, the determination of the stop for a fixed time can be determined, for example, based on whether or not the movement range of the user's hand 42 within a predetermined time is equal to or less than a predetermined value. Further, the menu selection determination unit 17 can confirm the menu item selected when it is stopped and within the range, and can cancel the selection when it is out of the range.

  Further, as another predetermined operation, for example, a “confirmation button” or “cancel button” is displayed on the display screen 40, and the confirmation or cancellation operation is executed by moving the hand 42 to any position thereof. Can do. Note that the determination of the menu item determination and cancellation operations described above is performed by, for example, the menu selection determination unit 17.

<Specific Example of Canceling Operation in Menu Selection Determination Unit 17>
Here, regarding the specific example of the canceling operation in the menu selection determining unit 17 described above, the canceling operation can be performed when moving in a predetermined direction by a predetermined distance or more with reference to the circular motion plane as described above. As another example, for example, the cancel operation may be performed when the hand moves to a position away from the place where the circular movement is performed by a certain distance or more. In addition, as an above-mentioned example, although a user has lowered | hanged a hand etc., for example, it is not limited to this.

  In these determinations, for example, a cancel operation can be designated by setting the orientation of the circular motion plane and the content of hand movement for canceling. For example, the menu selection determination unit 17 can cancel the downward direction when a circle is drawn horizontally with the ground. Further, the menu selection determining unit 17 erroneously determines the cancel operation when the vertical circle is a direction in which the hand is naturally retracted (for example, a pulling direction for a front facing circle and a dominant hand direction for a side facing circle). It will never be done. Moreover, about the cancellation operation | movement in the menu selection determination part 17, a cancellation operation | movement can be performed by performing at least 1 among the several methods mentioned above, However, It is not limited to this, For example, A plurality of operations may be performed in a predetermined order.

  In addition, the menu selection determination unit 17 cancels when the distance between the circular motion position and the input point is more than a predetermined value determined based on the circular motion radius. The predetermined value can be obtained, for example, by linear conversion with respect to the circular motion radius. The distance between the circular motion position and the input point is the distance between the center of the circular motion and the input point. The distance between the circular motion position and the input point can also be the distance between the intersection of the perpendicular line drawn from the input point to the circular motion plane and the plane and the center of the circular motion. In this case, if the hand 42 moves away from the circle by a certain amount or more, it is canceled.

  Furthermore, the menu selection determination unit 17 can previously cancel an area (for example, the lower area of the display screen 40) and cancel it when the user's hand 42 moves there.

  As described above, in the present embodiment, the menu item can be selected at an absolute position from the direction between the center position arbitrarily determined by the user and the hand position. In the present embodiment, the selected menu item can be confirmed or canceled by moving the hand more than a certain amount in the vertical direction from the plane of hand movement for selection. In the present embodiment, the selection process can be easily performed with one hand.

  That is, in this embodiment, for example, menu selection can be performed intuitively based on the positional relationship between the center point on the space arbitrarily determined when the menu is opened and the position of the hand. In the present embodiment, for example, selection items can be selected and confirmed quickly by moving the hand in a direction perpendicular to the plane in the space defined when the menu is opened. In addition, in this embodiment, the menu can be quickly and intuitively selected without having to circulate (rotate or reciprocate) the hand when selecting a menu, align the cursor while viewing the screen, or make the hand a predetermined shape. Is possible.

<Example of menu selection processing procedure in this embodiment>
Next, an example of a menu selection processing procedure in the present embodiment will be described using a flowchart. Note that the menu selection process described below is a process that is executed in cooperation by installing a menu selection program (software) on hardware resources as shown in FIG. 8 to 10 are flowcharts (No. 1 to No. 3) showing an example of the menu selection process in the present embodiment.

In the examples shown in FIGS. 8 to 10, the menu selection process acquires the coordinate value X as position information by tracking input (S01) and stores the coordinate value (S02). The coordinate values at this time are stored as a coordinate string X _i (1 ≦ i ≦ n), which is a continuous coordinate string at predetermined time intervals.

Next, the menu selection process determines whether the menu selection mode (S03), (in S03, NO) if not the menu selection mode, calculates the center position C from the coordinate sequence _{X i} (S04). Thereafter, the menu selection process calculates an average distance R from the coordinate position of the center position (S05).

Next, in the menu selection process, it is determined whether or not the average distance R obtained in the process of S05 is smaller than a minimum distance R _min that is a preset constant value (R <R _min ?) (S06). In the menu selection process, when the average distance R is not smaller than the minimum distance R _min (NO in S06), “a × R”, “b × R” (0 <a <) using the constants a and b. 1, 1 <b) is compared with the distance r _i between each coordinate point X _i and the center point C (S07).

Here, the menu selection _process, r i <aR or _r i> bR become _{r i} to determine whether there (S08). In the menu selection process, when r _i satisfying the condition of S08 does not exist (NO in S08), ∠X _i CX _i −1 is calculated in order to obtain the angle a _i (S09). In the process of S09, for example, clockwise is positive and reverse is negative, but the present invention is not limited to this.

In the menu selection process, it is determined whether or not a _i and a _i −1 satisfying the angle “a _i × a _i −1” <0 exist (S10). Here, in the menu selection process, when a _i and a _i −1 exist, the number of times L is set to 0 (S11), and the process returns to S01.

In the menu selection process, when the average distance R is smaller than R _min in the process of S06 (YES in S06), the process of S11 described above is performed, and the process returns to S01. Also, the menu selection process, in the process of S08, (in S08, YES) satisfies _{r If i} is present in S08 case, a process of S11 described above, the process returns to S01.

In the menu selection process, when a _i and a _i −1 satisfying the conditions of S10 do not exist in the process of S10 (NO in S10), the sum W of all a _i is calculated (S12). Next, the menu selection process determines whether or not the absolute value | W | of the sum of angles W is 360 ° or more (| W | ≧ 360 °?) (S13), and | W | is 360 ° or more. (YES in S13), the number of laps L is increased by 1 (S14). In the menu selection process, the last one point is left and the stored coordinate value sequence is cleared (S15).

Next, in the menu selection process, it is determined whether or not the number of laps L is equal to or greater than a predetermined minimum value L _min (L ≧ L _min ?) (S16). In the menu selection process, when the number of laps L is equal to or greater than a preset minimum number of laps L _min (YES in S16), a circular motion plane is calculated (S17). In the process of S17, a unit vector in a direction formed by the outer product of the vector CX _i and the vector CX _i −1 is obtained, and an average N (normal line of the circular motion plane) of these vectors is obtained. In the menu selection process, the radius R, the center point C, and the normal N of the circular motion plane obtained by the above process are stored (S18). Next, the menu selection process shifts to the menu selection mode (S19), and returns to the process of S01.

Here, if the menu selection process is the menu selection mode in the above-described process of S03 (YES in S03), a transition process to the menu selection mode is performed. Specifically, the menu selection process calculates the distance v between the input point X and the circular motion plane (for example, the center point C and the normal N) (S20), and then the distance v is set to a predetermined value. It is determined whether or not it is greater than the maximum distance V _max (v> V _max ?) (S21).

Menu selection process (at S21, NO) if the distance v is not greater than the maximum distance _{V max} set in advance, obtains the angle t of the vector _{X n-1 X n} and N (S22). In the menu selection process, it is determined whether or not the angle t formed is in a relationship of “Tsh ≦ t ≦ 180−Tsh” as compared with a preset threshold value Tsh (S23). When the menu selection process satisfies the condition of S23 (YES in S23), a direction value D (0 ° ≦ D <360 °) from the center point C to the input point X is calculated (S24), and the direction value is further calculated. The number of menu items is quantized (S25). The quantized value at this time is d. Next, in the menu selection process, a menu item corresponding to the quantized value d is displayed (S26).

  In the menu selection process, when the condition of S23 is not satisfied (NO in S23), or after the process of S26 is completed, the menu and selection items are highlighted (S27), and the process returns to S01.

Also, the menu selection process, in the process of S21, (in S21, YES) maximum distance _{V max} is larger than if the distance v is preset, calculates the inner product of the vector _{C X} and the normal N (S28). Next, in the menu selection process, it is determined whether or not the multiplication result of the sum W of angles and the inner product u obtained in S28 is 0 or more (W × u ≧ 0?) (S29).

  Here, in the menu selection process, if the multiplication result of the sum of angles W and the inner product u is 0 or more (YES in S29), a selection confirmation process is performed (S30). Examples of the selection confirmation process include, but are not limited to, executing an execution event set in advance in a menu item corresponding to the position coordinates of the user's hand.

  In the menu selection process, if the multiplication result of the sum of angles W and the inner product u is not 0 or more in the process of S29 (NO in S29), a cancel process is performed (S31). Cancellation processing includes processing such as canceling the menu selection mode (clearing the flag, etc.), returning to the state before the selection is confirmed, and deleting the menu item display from the screen, but is limited to this. is not.

  As described above, according to the present embodiment, menu selection can be realized by a simple operation of the user. In the present embodiment, the confirmation and cancellation of the selected menu item and the menu change can also be realized by a simple operation. Thereby, the convenience of menu operation can be improved. In the present embodiment, a general computer system including a tracking device and a display device capable of obtaining depth information is installed as software operating on the computer system. The software portion may be realized by hardware that performs an equivalent function.

  Note that this embodiment can be applied to general non-contact information input technology by tracking the movement of an object in a space, for example.

  Each embodiment has been described in detail above, but is not limited to a specific embodiment, and various modifications and changes can be made within the scope described in the claims.

In addition, the following additional remarks are disclosed regarding the above Example.
(Appendix 1)
A position information input unit for inputting position information of a predetermined object;
Analyzing a change in the position information of the object obtained by the position information input unit, and determining whether the object is moving in a circular motion;
A circular condition determining unit that determines whether the object satisfies a predetermined circular condition when the circular movement calculating unit determines that the object is performing a circular movement;
A menu selection mode control unit for executing a menu selection mode for displaying a menu item on the display unit and selecting a predetermined item based on the determination result obtained by the circulation condition determination unit;
For the menu item displayed on the display unit in the menu selection mode, the center position of the circular motion obtained by the circular motion calculation unit and the position information of the object obtained by the position information input unit. An information processing apparatus comprising: a menu selection unit that selects the predetermined item based on the menu selection unit.
(Appendix 2)
The circular motion calculation unit calculates a circular motion plane by the circular motion,
After the predetermined item is selected by the menu selection unit, the selected predetermined item is confirmed or canceled based on the distance between the circular motion plane and the position information of the object obtained by the position information input unit. The information processing apparatus according to attachment 1, further comprising: a menu selection determination unit that performs
(Appendix 3)
In the menu selection mode, when the moving direction due to the change in the position information of the object obtained by the position information input unit is a predetermined direction with respect to the circular motion plane, the menu item is changed by the menu selection unit. The information processing apparatus according to attachment 2, further comprising a menu selection control unit that invalidates the menu item.
(Appendix 4)
The menu selection unit
The menu type to be displayed is changed based on at least one of the rotation direction of the object by the circular motion obtained by the circular motion calculation unit, the inclination of the circular motion plane, and the radius of the circle by the circular motion. The information processing apparatus according to appendix 2 or 3, characterized by the above.
(Appendix 5)
The menu selection determination unit
When the position information of the object obtained by the position information input unit is within a predetermined distance from the circular motion plane for a predetermined time and within a predetermined range from the display position of the predetermined item, The information processing apparatus according to any one of appendices 2 to 4, wherein the item is determined.
(Appendix 6)
Input positional information of a predetermined object, analyze changes in the positional information of the input object, determine whether the object is moving in a circular motion, and determine that the object is moving in a circular motion Determining whether or not the object satisfies a predetermined circulation condition, and displaying a menu item on the display unit based on the determination result and executing a menu selection mode for selecting the predetermined item, the menu selection mode A menu selection program for causing a computer to execute a process of selecting the predetermined item based on the center position of the circular motion and the position information of the object for the menu item sometimes displayed on the display unit.
(Appendix 7)
After calculating the circular motion plane by the circular motion and selecting the predetermined item, the selected predetermined item is confirmed or canceled based on the distance between the circular motion plane and the position information of the object. The menu selection program according to appendix 6, characterized in that:
(Appendix 8)
The supplementary note 7 is characterized in that, in the menu selection mode, the change of the menu item is invalidated when a moving direction due to a change in the position information of the object is a predetermined direction with respect to the circular motion plane. Menu selection program.
(Appendix 9)
(7) The menu type to be displayed is changed based on at least one of the rotation direction of the object by the circular motion, the inclination of the circular motion plane, and the radius of the circle by the circular motion. The menu selection program according to 8.
(Appendix 10)
The predetermined item is determined when the position information of the object is within a predetermined distance from the circular motion plane for a predetermined time and within a predetermined range from the display position of the predetermined item. The menu selection program according to any one of appendices 7 to 9.
(Appendix 11)
Input positional information of a predetermined object, analyze changes in the positional information of the input object, determine whether the object is moving in a circular motion, and determine that the object is moving in a circular motion Determining whether or not the object satisfies a predetermined circulation condition, and displaying a menu item on the display unit based on the determination result and executing a menu selection mode for selecting the predetermined item, the menu selection mode A menu selection method, comprising: processing for selecting the predetermined item based on the center position of the circular motion and the position information of the object for the menu item sometimes displayed on the display unit.
(Appendix 12)
After calculating the circular motion plane by the circular motion and selecting the predetermined item, the selected predetermined item is confirmed or canceled based on the distance between the circular motion plane and the position information of the object. The method for selecting a menu according to appendix 11, wherein:
(Appendix 13)
Item 13. The change of the menu item is invalidated when the moving direction due to the change of the position information of the object is a predetermined direction with respect to the circular motion plane in the menu selection mode. Menu selection method.
(Appendix 14)
Item 12. The menu item to be displayed is changed based on at least one of the rotation direction of the object due to the circular motion, the inclination of the circular motion plane, and the radius of the circle due to the circular motion. 14. The menu selection method according to 13.
(Appendix 15)
The predetermined item is determined when the position information of the object is within a predetermined distance from the circular motion plane for a predetermined time and within a predetermined range from the display position of the predetermined item. The menu selection method according to any one of appendices 12 to 14.

10 Information Processing Device 11 Tracking Input Unit (Position Information Input Unit)
DESCRIPTION OF SYMBOLS 12 Tracking position memory | storage part 13 Circular motion calculation part 14 Circulation condition determination part 15 Menu selection mode 16 Circular motion memory | storage part 17 Menu selection determination part 18 Menu selection control part 19 Menu selection part 20 Menu information holding part 21 Tracking device 22 Display part 31 CPU
32 Memory 33 Storage device 34 Communication device 35 Drive device 36 Input device 37 Output device 38 Bus 39 Recording medium 40 Display screen 41 Cursor 42 Hand 43 Moving track 44 Center 45 Circular motion plane 50 Menu item

Claims (7)

A position information input unit for inputting position information of a predetermined object;
Analyzing a change in the position information of the object obtained by the position information input unit, and determining whether the object is moving in a circular motion;
A circular condition determining unit that determines whether the object satisfies a predetermined circular condition when the circular movement calculating unit determines that the object is performing a circular movement;
A menu selection mode control unit for executing a menu selection mode for displaying a menu item on the display unit and selecting a predetermined item based on the determination result obtained by the circulation condition determination unit;
For the menu item displayed on the display unit in the menu selection mode, the center position of the circular motion obtained by the circular motion calculation unit and the position information of the object obtained by the position information input unit. An information processing apparatus comprising: a menu selection unit that selects the predetermined item based on the menu selection unit. The circular motion calculation unit calculates a circular motion plane by the circular motion,
After the predetermined item is selected by the menu selection unit, the selected predetermined item is confirmed or canceled based on the distance between the circular motion plane and the position information of the object obtained by the position information input unit. The information processing apparatus according to claim 1, further comprising a menu selection determination unit that performs the operation.   In the menu selection mode, when the moving direction due to the change in the position information of the object obtained by the position information input unit is a predetermined direction with respect to the circular motion plane, the menu item is changed by the menu selection unit. The information processing apparatus according to claim 2, further comprising a menu selection control unit that invalidates the menu. The menu selection unit
The menu type to be displayed is changed based on at least one of the rotation direction of the object by the circular motion obtained by the circular motion calculation unit, the inclination of the circular motion plane, and the radius of the circle by the circular motion. The information processing apparatus according to claim 2, wherein the information processing apparatus is an information processing apparatus. The menu selection determination unit
When the position information of the object obtained by the position information input unit is within a predetermined distance from the circular motion plane for a predetermined time and within a predetermined range from the display position of the predetermined item, The information processing apparatus according to claim 2, wherein an item is confirmed.   Input positional information of a predetermined object, analyze changes in the positional information of the input object, determine whether the object is moving in a circular motion, and determine that the object is moving in a circular motion Determining whether or not the object satisfies a predetermined circulation condition, and displaying a menu item on the display unit based on the determination result and executing a menu selection mode for selecting the predetermined item, the menu selection mode A menu selection program for causing a computer to execute a process of selecting the predetermined item based on the center position of the circular motion and the position information of the object for the menu item sometimes displayed on the display unit.   Input positional information of a predetermined object, analyze changes in the positional information of the input object, determine whether the object is moving in a circular motion, and determine that the object is moving in a circular motion Determining whether or not the object satisfies a predetermined circulation condition, and displaying a menu item on the display unit based on the determination result and executing a menu selection mode for selecting the predetermined item, the menu selection mode A menu selection method, comprising: processing for selecting the predetermined item based on the center position of the circular motion and the position information of the object for the menu item sometimes displayed on the display unit.

Images