JP2005070996A - Instruction input device, instruction input system, instruction input method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an instruction input device for executing instruction input by a natural operation with superior operability without constraints, and to provide an instruction input system, an instruction input method and a program . <P>SOLUTION: An operator moves an LED to a pointing direction. A 3D measuring device 20 detects rays of light emitted from the LED moving according to the operation of the predetermined site of the operator, and a personal computer body 32 detects the acceleration and speed of the LED based on the measured position information, and calculates a speed vector S from the speed at that time. Then, a cursor is displayed at the pointing position of the screen of a display 42. Also, the movement of the LED is determined in a coordinate system configured by the speed vector S and a plane vertical to the speed vector S; and when the movement of the LED is predetermined movement, the execution of processing corresponding to the movement is instructed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an instruction input device, an instruction input system, an instruction input method, and a program, and in particular, display of instruction information such as a cursor based on position information of light emitting means such as an LED that moves according to an operation of a predetermined part of an operator. The present invention relates to an instruction input device, an instruction input system, an instruction input method, and a program for controlling a position.

  As a device for inputting an instruction to a personal computer or the like, a mouse (two-dimensional mouse) that inputs an instruction to a personal computer using an xy coordinate value as a relative coordinate value has been widely used. In recent years, a technique for instructing and inputting a three-dimensional coordinate value using the two-dimensional mouse has been proposed. In this technique, for example, in addition to a conventional mouse ball for indicating a two-dimensional (xy) coordinate value, a mouse ball for indicating a third coordinate value (z coordinate value) is separately provided. A three-dimensional coordinate value is designated and input using the rotation of the mouse ball. This requires complex operations. Further, the operation of the two-dimensional mouse is constrained on a two-dimensional plane, and the operability is inferior as an instruction input device in a three-dimensional space.

  Therefore, the LED as a light source is mounted on the pen or attached to the operator's finger, etc., and a light receiving part for receiving light from the LED is attached to the upper part of the screen of the personal computer, and the position of the LED is detected to make it three-dimensional. An apparatus (three-dimensional mouse) capable of instructing and inputting coordinate values has been proposed (see, for example, Patent Document 1). In this apparatus, the angle of the light source with respect to the light receiving portion is obtained, and the three-dimensional position of the light source is obtained by the principle of triangulation. In addition, when inputting a signal such as a click, which is the basic operation of the mouse, using this device, a machine such as a switch that can be pressed by the operator with a finger is provided near the LED, and the switch is pressed. It is necessary to input to the basic software of a personal computer via a cable by performing a manual operation.

  In addition, as a device that can instruct and input a three-dimensional coordinate value, by rotating two two-dimensional balls, not only the movement amount in the x and y axis directions but also the angle and the rotation angle can be input, and the z axis A pointing device (see, for example, Patent Document 2) that can also input the direction, a first photosensor array in which light conversion elements are arranged in the first direction, and a second direction different from the first direction A non-contact method that generates a signal corresponding to a change in the output of a second optical sensor array in which light conversion elements are arranged in a direction, thereby determining the amount of movement (such as a cursor) and the direction of movement. There is known a position detection apparatus (see, for example, Patent Document 3).

  Furthermore, an object having a predetermined shape is recognized by a CCD camera and interpreted as a command for a personal computer or the like (for example, see Patent Document 4), or rotated with a finger in an electronic pen. A coordinate input electronic pen (see, for example, Patent Document 5) that determines the degree of change in graphical parameters (line thickness, color, shading, gray scale) according to the degree of rotation. An instruction input system (see, for example, Patent Document 6) that recognizes the movement of an RFID attached to a user's wrist with an antenna and interprets the movement pattern of the RFID as an input command to another device is known. Yes.

  Gyromouse of Gyration Inc. in the United States is also widely known as a pointing device. This Gyromouse has a built-in auto gyro and has a mechanism that can change the direction of the laser by aerial operation.

  Furthermore, in recent years, a system has been proposed in which, for example, a feature point indicating the direction of an arm or a finger is detected and an instruction is input by photographing a person's movement or gesture with a CCD camera and analyzing the image. In this system, two CCD cameras are provided, and for example, the direction indicated by the finger is detected from the movement of a specific part of the user.

  In addition, the present applicant uses a technique for detecting the position of the light emitting means by receiving light from the light emitting means, and uses feedback (change in focus metaphor, hue, etc.) to appeal to at least one of the five senses. To allow the user to recognize the passage state of the light emitting means with respect to each boundary of a plurality of divided areas in the three-dimensional space extending in front of the light, and to pass the boundary of each region of the light emitting means without using a mechanical mechanism In response to this, a three-dimensional instruction input device that can execute a two-dimensional mouse function such as click, double click, and drag has been proposed (see Japanese Patent Application No. 2003-96366). Furthermore, the position information measured by moving the light emitting means attached to the operator in the front-rear direction along the direction (z-axis direction) from the 3D measuring device toward the operator and receiving the light from the light emitting means. From the above, the acceleration and speed associated with the movement of the light emitting means are detected, and it is determined whether or not the movement of the light emitting means corresponds to a predetermined movement while taking into account the time associated with the movement. In such a case, there has been proposed an instruction input device that can instruct execution of processing (functions such as click, double click, and drag) associated with a predetermined motion (see Japanese Patent Application No. 2003-138645).

An apparatus that detects acceleration or speed using an acceleration sensor and inputs characters and figures is also known (see, for example, Patent Documents 7 and 8).
JP-A-10-9812 JP-A-5-150900 JP-A-11-24832 JP-A-11-110125 JP 2000-47805 A JP 2001-306235 A JP-A-8-16301 JP 2000-268130 A

  However, since the conventional three-dimensional mouse described above is operated by the operator pressing a switch provided in the vicinity of the LED with a finger, mechanical operation and components for realizing the same are required as in the two-dimensional mouse. To do. Further, since a signal such as a click, which is a basic operation of the mouse, is input via a wire, troublesome operations are caused when the operator operates. With respect to this point, wireless input is possible by using infrared rays to which Bluetooth or a code is added, but there arises a problem that the size and weight are increased and the cost is increased. Furthermore, since the position of the LED is obtained with absolute coordinates in this technique, there arises a problem that two light emitters must be mounted on the body part of the operator in order to input the indicated direction.

  In addition, other instruction input devices such as the above-described conventional pointing device and instruction input system also require mechanical operation in the same manner as the above-described three-dimensional mouse, and thus have poor operability. Gyromouse of Gyration Inc. of the United States also requires a mechanical operation with a finger, like a normal two-dimensional mouse, for operations such as clicking. In addition, the weight and size are relatively large and expensive.

  That is, the conventional instruction input device requires a relatively fine operation such as pressing a button, a switch, or the like, so that it is not convenient for a person who is relatively difficult to perform a mechanical operation.

  On the other hand, in a system in which a person's movements and gestures are photographed with a CCD camera and the direction of an arm or finger is detected, there are cases in which feature points cannot always be detected because there are individual differences in human parts. In addition, the CCD camera has a limited viewing angle, and there is a problem that an additional CCD camera must be added to expand it. In addition, since image processing is performed to detect feature points, there are problems that the number of objects to be analyzed is increased and the response is slow compared to other systems.

  In addition, the conventional three-dimensional instruction input device and instruction input device proposed by the present applicant must move the finger or hand on which the light emitting means is mounted while confirming a predetermined space (area) or plane. It is difficult to input instructions with natural movement.

  Furthermore, in an apparatus for inputting instructions using an acceleration sensor, there is a problem that the size and weight of the writing instrument increases and the usability is poor because the acceleration sensor is built in (attached to) a writing instrument such as a pen. In addition, since the accuracy of the acceleration and speed that can be detected becomes coarser than when the position of the light emitting means is measured and the acceleration and speed are detected, there is a problem that it is impossible to detect the detailed operation of the user. .

  The present invention has been proposed in order to solve the above-described problems, and is an instruction input device, an instruction input system, an instruction input method, and a program that are excellent in operability and can input instructions with a natural operation without difficulty. The purpose is to provide.

  In order to achieve the above object, the instruction input device of the present invention is position information indicating the position of the light emitting means measured by detecting the light emitted from the light emitting means that moves according to the operation of the predetermined part of the operator. Based on the position information input by the input means, a detection means for detecting the acceleration and speed of the light emitting means, and a position where the acceleration detected by the detection means exceeds a predetermined value. A velocity vector calculating unit that calculates a velocity vector indicating a moving direction of the light emitting unit as a base point based on the detected velocity, a line segment including the velocity vector calculated by the velocity vector calculating unit, and the light emission A table for calculating the intersection point with the virtual plane provided corresponding to the display means for displaying the instruction information for indicating the position indicated by the means as a display intersection point And a control means for controlling the display means so that the instruction information is displayed at a position corresponding to the position of the display intersection calculated by the display intersection calculation means of the display means. It is configured to include.

  The instruction input method of the present invention includes an input step of inputting position information indicating the position of the light emitting means measured by detecting light emitted from the light emitting means that moves according to the operation of the predetermined part of the operator; A detection step of detecting acceleration and speed of the light emitting means based on the positional information input in the input step, and a movement of the light emitting means based on a position where the acceleration detected by the detection step exceeds a predetermined value A velocity vector calculation step for calculating a velocity vector indicating the direction of the movement based on the detected velocity, a line segment including the velocity vector calculated by the velocity vector calculation step, and a position indicated by the light emitting means. A display intersection calculation step of calculating an intersection of a virtual plane provided corresponding to a display means for displaying instruction information for display as a display intersection; Includes a control step for controlling the display means such that the indication information at a position corresponding to the position of the display intersection calculated by the display intersection calculation step means is displayed, the.

  The program of the present invention includes an input step of inputting position information indicating the position of the light emitting means measured by detecting light emitted from the light emitting means moving according to the operation of the predetermined part of the operator to the computer, Based on the positional information input in the input step, a detection step of detecting the acceleration and speed of the light emitting means, and the light emitting means based on the position where the acceleration detected by the detection step exceeds a predetermined value A speed vector calculating step for calculating a speed vector indicating a moving direction based on the detected speed, a line segment including the speed vector calculated by the speed vector calculating step, and a position indicated by the light emitting means. Display intersection calculation for calculating an intersection with a virtual plane provided corresponding to a display means for displaying instruction information for display as a display intersection Degree and to execute a control process of controlling the display means so that said instruction information at a position corresponding to the position of the display intersections calculated by the display intersection calculating step of said display means is displayed.

  In the present invention, the light emitting means moves according to the operation of the predetermined part of the operator. This light emitting means may be attachable to an operator's finger or hand. The light emitting means is not particularly limited as long as it emits light, and may be, for example, an LED. The position of the light emitting means is measured by detecting the light emitted from the light emitting means.

  A virtual plane is provided corresponding to the display means for displaying the instruction information for indicating the position indicated by the light emitting means. The display means may be a display provided in a personal computer or the like, or may be a large screen that displays using a projector or the like. The instruction information displayed on the display means is not particularly limited, but can be a cursor used in, for example, a personal computer.

  When position information indicating the position of the light emitting means is input, the acceleration and speed of the light emitting means are detected based on the input position information. When the detected acceleration exceeds a predetermined value, a velocity vector indicating the moving direction of the light emitting means with the position where the detected acceleration exceeds the predetermined value as a base point is calculated. Next, the intersection between the line segment including the velocity vector and the virtual plane is calculated as a display intersection. The display means is controlled so that the instruction information is displayed at a position corresponding to the calculated position of the intersection for display.

  In this way, since the instruction information is displayed at the position corresponding to the intersection (display intersection) between the speed vector indicating the direction in which the light emitting unit has moved and the virtual plane, no mechanical operation is required, and the operability is excellent. It is possible to input instructions with natural operation without difficulty. For example, when it is desired to display the instruction information at a desired position, the instruction information can be easily displayed only by moving the light emitting means in the direction of the desired position.

  The instruction input device of the present invention has a stationary position where the light emitting means stops after the acceleration detected by the detecting means exceeds a predetermined value, and a movement position where the light emitting means moves from the stationary position as an end point. A movement vector calculation means for calculating a movement vector, a plane calculation means for calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position, and the plane calculation means. An intersection calculation means for calculating an intersection of the calculated plane and a line segment including the speed vector calculated by the speed vector calculation means; and the display intersection calculation means includes the speed vector. After calculating the intersection of the line segment and the virtual plane as a display intersection, and after the acceleration detected by the detection means exceeds the predetermined value Can be calculated as a line segment and, display intersection the intersection of said imaginary plane, including said movement position and the intersection calculated by the intersection point calculation means.

  In the instruction input method of the present invention, the stationary position where the light emitting means stops after the acceleration detected by the detecting step exceeds a predetermined value is set as a starting point, and the moving position where the light emitting means moves from the stationary position is set as an end point. A movement vector calculation step of calculating a movement vector that is a vector to be moved, a plane calculation step of calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position, and the plane calculation step An intersection calculation step of calculating an intersection of the calculated plane and a line segment including the velocity vector calculated by the velocity vector calculation means, wherein the display intersection calculation step includes the velocity vector After calculating the intersection of the line segment and the virtual plane as a display intersection, and after the acceleration detected by the detection step exceeds the predetermined value Can be calculated as a line segment and, display intersection the intersection of said imaginary plane, including said movement position and the intersection calculated by the intersection calculation step.

  The program of the present invention is a vector in which a starting position is a stationary position where the light emitting means is stationary after the acceleration detected by the detecting step exceeds a predetermined value, and a moving position where the light emitting means is moved from the stationary position is an end point. A movement vector calculation step for calculating the movement vector, a plane calculation step for calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position, and the plane calculation step. And an intersection calculation step for calculating an intersection of the line segment including the velocity vector calculated by the velocity vector calculation means, and the display intersection calculation step includes a line segment including the velocity vector. And the intersection between the virtual plane and the virtual plane is calculated as an intersection for display, and after the acceleration detected by the detection step exceeds the predetermined value, Serial intersection intersection calculated by the calculating step and the segment containing said movement position, wherein it is possible to calculate the virtual plane, an intersection of the display intersection.

  After the detected acceleration exceeds a predetermined value, the stationary position where the light emitting means stops after the detected acceleration exceeds the predetermined value is a starting point, and the moving position where the light emitting means moves from this stationary position is the end point. A movement vector which is a vector is calculated. Note that the stillness does not have to be completely stationary. For example, when the variation in the position of the light emitting means falls within a predetermined range, or when the acceleration and speed of the light emitting means are less than or equal to a predetermined value, the light emitting means is assumed to be stationary. can do.

  The movement direction of the light emitting means is indicated by the calculated movement vector. Subsequently, a plane perpendicular to the calculated movement vector and including the movement position is calculated. Further, the intersection of this plane and the line segment including the velocity vector is calculated. An intersection between the line segment including the intersection and the moving position and the virtual plane is calculated as a display intersection. The display means is controlled so that the instruction information is displayed at a position corresponding to the calculated position of the intersection for display.

  In this way, after the detected acceleration exceeds a predetermined value, a new display intersection is obtained from the stationary position and the moving position of the light emitting means, and the instruction information is displayed at a position corresponding to the display intersection. Therefore, no mechanical operation is required, the operability is excellent, and instructions can be input with a natural operation without difficulty. For example, after displaying the instruction information at a position corresponding to the display intersection between the velocity vector and the virtual plane, if the display position of the instruction information is to be moved to a desired position, the light emitting means is temporarily stopped. The instruction information can be easily moved to the desired position simply by moving from to the desired position.

  The moving position may be a moving position at every predetermined time, or may be a position when moving from a stationary position and resting again. Moreover, the intersection of the plane perpendicular to the movement vector and including the movement position and the line segment including the velocity vector may be fixed for a predetermined time. Thereby, it is not necessary to calculate the intersection for each movement position, and the amount of calculation can be reduced.

  The instruction input device of the present invention further includes processing means for performing predetermined processing using a coordinate system including a speed vector calculated by the speed vector calculation means and a plane perpendicular to the speed vector. You can also.

  The instruction input method of the present invention further includes a processing step of performing a predetermined process using a coordinate system constituted by the velocity vector calculated by the velocity vector calculating step and a plane perpendicular to the velocity vector. You can also.

  The program of the present invention may further include a processing step of performing a predetermined process using a coordinate system constituted by the velocity vector calculated by the velocity vector calculating step and a plane perpendicular to the velocity vector. .

  That is, even if the light emitting means is moved in an arbitrary direction even if the direction is not perpendicular to the screen of the display means, a predetermined coordinate system using a coordinate system composed of a velocity vector and a plane perpendicular to the velocity vector is used. Processing can be performed. Therefore, it is possible to input instructions smoothly with natural movement for the operator.

  Here, the predetermined process determines whether or not the movement of the light emitting means corresponds to a predetermined movement based on the acceleration and speed of the light emitting means in the coordinate system detected by the detection means. In addition to the determination, when it is determined that the movement of the light emitting means corresponds to a predetermined movement, a process for instructing execution of a process associated with the predetermined movement may be performed.

  In this way, by moving the light emitting means in an arbitrary direction, the light emitting means can be moved using a coordinate system composed of a velocity vector and a plane perpendicular to the velocity vector. In addition to forming a predetermined movement, it is possible to determine whether the movement of the light emitting means corresponds to the predetermined movement and smoothly execute the process associated with the predetermined movement. Can be directed.

  Here, the predetermined movement is not particularly limited, and may be one or plural. Moreover, the movement comprised by the some fine movement may be sufficient.

  For example, if the movement of a click, double click, or drag and drop on a personal computer is determined in advance, the movement of the light emitting means in a coordinate system composed of a velocity vector and a plane perpendicular to the velocity vector is determined in advance. It is determined whether or not it corresponds to the movement, and when it is determined that it corresponds, it is possible to instruct execution of click, double click, or drag and drop processing.

  Further, as the predetermined processing, for example, a velocity vector constituting the coordinate system is set as a z-axis, the z-axis itself is moved up and down, left and right, and a virtual object view (three-dimensionally expressed on the display means ( It is also possible to change the view).

  The control means may display a plurality of the instruction information corresponding to each of the latest display intersection calculated by the display intersection calculation means and a plurality of display intersection positions calculated in the past. The display means can be controlled.

  Thereby, a plurality of positions of the instruction information can be displayed, and the instruction position can be easily confirmed without losing sight of the display position of the instruction information.

  The control means can also control the display means so that the display state of the plurality of instruction information changes as time passes.

  As a result, it is possible to check the time that has elapsed since the instruction information was displayed, and the latest display position is not lost.

  The display means displays the instruction information and displays a predetermined object, and the control means displays the instruction information when the display position of the instruction information matches the display position of the predetermined object, and When the display position coincides with the display position of the predetermined object and the execution of the process associated with the predetermined movement is instructed by the processing means, the display is performed. The display means can be controlled to change the display state of the means.

  By changing the display state in this way, the positional relationship between the instruction information and the object can be confirmed. For example, such a change in display state can be applied to a display change in a shooting game or the like.

  The instruction input system of the present invention includes a light emitting means that moves according to an operation of a predetermined part of an operator, a measuring device that measures the position of the light emitting means by detecting light emitted from the light emitting means, And an instruction input device.

  Since this instruction input system also operates in the same manner as the above-described instruction input device, instruction input method, and program of the present invention, it does not require mechanical operation, has excellent operability, and inputs instructions with natural operation without difficulty. be able to.

  Further, the light emitting means can be provided in a mobile phone. In this way, by providing the light emitting means on the mobile phone that can be carried wirelessly, it is possible to easily input an instruction, and the power supply of the mobile phone can be shared with the power supply of the light emitting means.

  The storage medium for storing the above-described program of the present invention is not particularly limited, and may be a hard disk or a ROM. Further, it may be a CD-ROM, a DVD disk, a magneto-optical disk or an IC card. Furthermore, the program may be downloaded from a server or the like connected to the network.

  The present invention displays the instruction information at a position corresponding to the intersection of the velocity vector indicating the moving direction of the light emitting means and the virtual plane, and when the light emitting means further moves after the light emitting means is stationary, the new information on the virtual plane is displayed. Since a point of intersection is obtained and the instruction information is displayed at a position corresponding to the point of intersection, no mechanical operation is required, the operability is excellent, and the instruction can be input with a natural operation without difficulty. In addition, since a predetermined process is performed using a coordinate system composed of a velocity vector and a plane perpendicular to the velocity vector, no mechanical operation is required, the operability is excellent, and instructions are given with natural motion without difficulty. Can be entered.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing a configuration of an instruction input system including a personal computer 30 having a function as an instruction input device according to the first embodiment of the present invention. As illustrated, the instruction input system includes an LED 10, a 3D measuring device 20, and a personal computer 30.

  The LED 10 is attached to an operator's finger or hand.

  In the 3D measurement device 20, a light receiving unit (not illustrated) receives light emitted from the LED 10 to detect light from the LED 10 and measures a three-dimensional position of the LED 10. Position information (x, y, z coordinate values) indicating the measured three-dimensional position is output to the personal computer 30. The 3D measuring device 20 is not particularly limited as long as it is a device that can measure a three-dimensional position by detecting light emitted from a light emitter such as the LED 10, but is described in JP-A-10-9812. It can be configured using a position detection technique or the like.

  In this instruction input system, the LED 10 in the space in front of the screen of the display of the personal computer 30 is provided by installing the light receiving unit of the 3D measuring device 20 at a position that substantially matches the position of the screen of the display of the personal computer 30 (for example, the upper part of the display). The three-dimensional position of is measured. In addition, a virtual plane corresponding to the display screen of the personal computer 30 (which does not actually exist) is set at a position that substantially matches the display screen of the personal computer 30. The operator wearing the LED 10 can display a cursor at an arbitrary position on the display screen corresponding to the virtual plane by moving the hand wearing the LED 10 relative to the virtual plane.

  The coordinate system used in the 3D measurement apparatus 20 of this instruction input system is a coordinate system in which the origin is at the upper left corner of the virtual plane, the virtual plane is the xy plane, and the axis perpendicular to the virtual plane is the z axis. In this coordinate system, the direction from the virtual plane toward the operator is determined to be the + direction of the z axis, the right direction to the virtual plane is the + direction of the x axis, and the downward direction is the + direction of the y axis. Yes. Hereinafter, this coordinate system is referred to as a reference coordinate system.

  The virtual plane may be provided on any plane, but it is preferable to provide the virtual plane on a plane that substantially matches the screen of the display because the input position can be easily perceived when the operator inputs instructions. .

  FIG. 2 is a diagram showing a specific configuration of the personal computer 30. As shown in the figure, the personal computer main body 32 of the personal computer 30 includes a CPU 34, a ROM 36, a RAM 38, and an input / output interface (I / O) 40. The I / O 40 is connected to a 3D measurement device 20, a display 42, a speaker 44, and a hard disk drive (HDD) 46.

  The position information of the LED 10 output from the 3D measurement device 20 is input via the I / O 40.

  The HDD 46 detects the acceleration and speed of the LED 10, displays a cursor for indicating the position indicated by the LED 10 on the display 42 based on the acceleration and speed, and detects the movement of the LED 10 to perform predetermined processing. An instruction input processing routine program for instructing execution (hereinafter referred to as an instruction input processing program) and various setting information used when executing the instruction input processing program are stored. The CPU 34 loads the instruction input processing program and various setting information into the RAM 38 and executes the program.

  In the instruction input processing routine, the direction or movement indicated by the LED 10 is determined based on the detected acceleration and speed. By judging using the acceleration, the LED 10 is moved rapidly and not, that is, the operator is consciously moving the hand, and there is no intention to input the instruction, and the hand can be moved freely. Can be distinguished from the current state. Further, by determining using the speed, it can be recognized that the LED 10 has been moved by a certain distance, and the actual movement of the LED 10 can be determined. By using the speed as a criterion, it is possible to recognize a smaller movement than when the movement distance itself is calculated from the position information and used.

The various setting information includes an acceleration threshold A ₀ associated with the movement of the LED 10, a speed threshold V ₀ , a pointing mode (described later) time K 1, a command mode (described later) time K 2, and a time for determining the movement of the LED 10 Thresholds T1 ₀ and T2 ₀ are included.

  The storage medium for storing the instruction input processing program and various setting information is not limited to the HDD 46 and may be the ROM 36. Although not shown, a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card connected to the I / O 40 may be used. Furthermore, the program may be downloaded from a server or the like connected to the network.

  FIG. 3 is a diagram functionally showing the configuration of the personal computer main body 32.

  As shown in the figure, the personal computer main body 32 includes a command generation unit 50, a mouse driver 52, an OS 54, and an image driver 56.

  The command generation unit 50 receives position information indicating the three-dimensional position measured by the 3D measurement device 20 and detects the acceleration and speed of the LED 10. Further, based on the detected acceleration and speed, a command for pointing is generated and output to the mouse driver 52 so that the cursor is displayed on the display 42. Furthermore, based on the detected acceleration and speed, the movement of the LED 10 in the local coordinate system (described later) is a button operation in a general mouse equipped with buttons (hereinafter referred to as a two-dimensional mouse), that is, , Click, double-click, drag-and-drop, and a command corresponding to the determination result is generated and output to the mouse driver 52.

  A command is input from the command generation unit 50 to the mouse driver 52. Commands input to the mouse driver 52 are output to the OS 54.

  The OS 54 interprets a command from the mouse driver 52 and executes processing corresponding to the command.

  The image driver 56 performs processing for displaying an image corresponding to the position and movement of the LED 10 on the display 42 under the control of the OS 54.

  Although not shown, the personal computer main body 32 is also provided with a sound driver. The sound driver performs processing for outputting sound corresponding to the movement of the LED 10 from the speaker 44 under the control of the OS 54.

  Hereinafter, the flow of instruction input processing according to the present embodiment will be described in detail.

  The operator moves the hand wearing the LED 10 within the measurement area of the 3D measurement device 20. The position of the LED 10 changes according to the movement of the hand. The 3D measuring device 20 measures the three-dimensional position of the LED 10 at predetermined time intervals, and sequentially outputs position information indicating the three-dimensional position to the personal computer 30 (the personal computer body 32). The position information is used in an instruction input processing routine.

  4 and 5 are flowcharts showing the main routine of the instruction input process.

  In addition, although illustration is abbreviate | omitted in this flowchart, the positional information on LED10 is input into the personal computer 30 (PC main body 32) from the 3D measuring device 20 for every predetermined time. Then, the CPU 34 detects the acceleration and speed of the LED 10 based on the position information input for each input. For example, the speed can be detected by dividing a difference between coordinate values within a predetermined time by the predetermined time, and the acceleration can be detected by dividing the speed by the predetermined time.

At step 100, it is determined whether the acceleration exceeds the threshold value A _0. Here, when it is determined that acceleration ≦ A ₀ , the standby state is maintained until the next position information is input from the 3D measurement device 20. If it is determined that acceleration> A ₀ , the operator has moved his / her hand with will, and at this point, a pointing mode for displaying a cursor at a desired position is started.

  In step 102, the timer T is reset and started to measure the pointing mode time K1.

In step 104, it is determined whether the rate exceeds the threshold value V _0. The speed determined here is the speed associated with the movement of the LED 10 that has generated the acceleration exceeding the threshold A _{0 in} step 100. If it is determined in step 104 that the speed ≦ V ₀ , the pointing mode is terminated and the process returns to step 100. If it is determined that speed> V ₀ , the process proceeds to step 106.

In addition, the operation | movement by which the affirmation determination is carried out by step 100 and step 104 is an operation | movement which an operator moves the hand which mounted | wore LED10 rapidly in a pointing direction. Hereinafter, the operation of rapidly moving the hand in an arbitrary direction is referred to as push. While pointing mode is started by the push, the pointing mode is newly pushed until since the acceleration exceeding the threshold A ₀ is detected at step 100 until the time K1 has elapsed, or time K1 has elapsed It is continued until the operation is performed. Note that the direction of the operation (push) for which an affirmative determination is made in steps 100 and 104 does not have to be a direction perpendicular to the screen of the display 42 or the virtual plane, and may be an arbitrary direction.

  In step 106, a first pointing process routine incorporated in advance in the instruction input process program is executed as a subroutine of the instruction input process routine.

  The flow of the first pointing process will be described below with reference to the flowchart showing the first pointing process routine of FIG. 6 and the explanatory diagram of FIG.

In step 200, the position where acceleration is detected that exceeds the threshold A ₀ in step 100 (acceleration detecting position) as a base point, calculates the velocity vector S indicating the direction of movement of the LED 10. Since a vector in the (x, y, z) direction is obtained from the detected speed, the speed vector S can be obtained immediately.

  In step 202, a line segment M1 including the velocity vector S is calculated. The velocity vector S can be easily obtained geometrically, but the line segment M1 is a generalized parametric representation of the velocity vector S.

  In step 204, an intersection C1 between the line segment M1 and the virtual plane V is calculated. As described above, the virtual plane V is a plane corresponding to the screen of the display 42 set on a plane that substantially matches the screen of the display 42. The intersection C1 can be obtained geometrically as a point on the virtual plane V, and is uniquely calculated from an expression indicating the line segment M1 and an expression indicating the virtual plane V.

  In step 206, pointing processing is performed. Specifically, a command for pointing is generated so that the cursor is displayed at a position corresponding to the position of the point (intersection C1) on the virtual plane V calculated in step 204 on the screen of the display 42, and the mouse. Output to the driver 52. The OS 54 to which the command is input via the mouse driver 52 interprets the command and controls the display 42 so that the cursor is displayed at a position corresponding to the intersection C1 on the screen of the display 42.

  Thus, first, a point (intersection point C1) on the virtual plane V corresponding to the direction instructed by the first push is obtained, and the cursor is displayed on the screen of the display 42.

  In step 208, it is determined whether the LED 10 is stationary. Here, when a predetermined number or more of x, y, z coordinate values located within a predetermined range from the moving average of the x, y, z coordinate values of the LED 10 are input, it is determined that the LED 10 is stationary. The determination of stillness is not particularly limited to this, and it is determined that the camera is stationary when the distance between the x, y, and z coordinate values before the predetermined time and the latest x, y, and z coordinate values are within the predetermined distance. Alternatively, it may be determined that the camera is stationary when the acceleration and the speed are equal to or lower than predetermined values.

  If it is determined in step 208 that the camera is stationary, the stationary position P is detected in step 210. The x-, y-, and z-coordinate values when determined to be stationary are the stationary position P. The stationary position P detected here is used in a second pointing process described later.

After the completion of the first pointing process at step 106 in FIG. At step 108, it is determined whether the acceleration exceeds the threshold value A _0. Here, if it is determined that acceleration ≦ A _0, it is determined that the push operation has not started, and the process proceeds to step 110. In step 110, it is determined whether or not the timer T has reached the pointing mode time K1.

  If it is determined in step 110 that the timer T <K1, it is determined that the pointing mode time K1 has not elapsed, and the routine proceeds to step 112.

  In step 112, a second pointing process routine incorporated in advance in the instruction input process program as a subroutine of the instruction input process routine is executed.

  The flow of the second pointing process will be described below with reference to the flowchart showing the second pointing process routine of FIG. 7 and the explanatory diagram of FIG.

  In step 300, the movement position Q to which the LED 10 has moved is detected. Here, the moving position Q may not be stationary.

  In step 302, a movement vector PQ indicating the movement direction of the LED 10 is calculated from the stationary position P and the movement position Q.

  In step 304, a plane N perpendicular to the movement vector PQ and including the movement position Q is calculated. Specifically, two vectors (vectors orthogonal to the movement vector PQ) whose inner product with the movement vector PQ is 0 are obtained, and the movement position is perpendicular to the movement vector PQ from the two vectors and the movement position Q. A plane N including Q is calculated.

  In step 306, the intersection point O between the plane N calculated in step 304 and the line segment M1 calculated in step 206 is calculated.

  In step 308, a line segment M2 including the movement position Q and the intersection point O is calculated.

  In step 310, an intersection C2 between the line segment M2 and the virtual plane V is calculated. The intersection C2 is uniquely calculated as a point on the virtual plane V from an expression indicating the line segment M2 and an expression indicating the virtual plane V.

  In step 312, the pointing process is performed in the same manner as in step 206 of the first pointing process. As a result, the cursor displayed on the display 42 moves from the position corresponding to the intersection C1 to the position corresponding to the intersection C2.

  The second pointing process described above is repeated while negative determination continues in step 108 and step 110 in FIG. 4 (that is, until a new push operation is performed or time K1 elapses), so that new intersections are sequentially added. C2 is calculated and the display position of the cursor moves. Since the stationary position P and the line segment M1 are not changed, the intersection point C2 can be sequentially updated only by detecting the moving position Q. Here, since the position information is input from the 3D measurement device 20 every predetermined time, the second pointing process is executed every predetermined time. That is, the movement position Q is detected at every predetermined time, and the intersection C2 is updated.

  On the other hand, if it is determined in step 110 that the timer T ≧ K1, it is determined that the time K1 has been reached before a new push is performed, the pointing mode is terminated, and the process returns to step 100.

If it is determined in step 108 that acceleration> A _0, it is determined that a new push operation has started, and the process proceeds to step 114. In step 114, it is determined whether or not the timer T is less than the time K1. If it is determined that the timer T ≧ K1, the pointing mode is terminated and the process returns to step 100. If it is determined that the timer T <K1, it is determined that the push operation has been started before the time K1 has elapsed, and the routine proceeds to step 116.

At step 116, it is determined whether the rate exceeds the threshold value V _0. The speed determined here is the speed associated with the movement of the LED 10 that has generated the acceleration exceeding the threshold A _{0 in} step 108. If it is determined in step 116 that the speed ≦ V ₀ , the pointing mode is terminated and the process returns to step 100. If it is determined that speed> V _0, it is determined that a new push operation has been performed, the pointing mode is terminated, and the routine proceeds to step 118.

  In step 118, a timer T is started in order to time the command mode time K2. Here, a command mode for determining the movement of the LED 10 and instructing execution of a process corresponding to the determination result is started.

  In step 120 of FIG. 5, a coordinate system conversion formula is calculated.

  In this step, the speed vector S and the stationary position P by the push when the affirmative determination is made in steps 108 and 116 are obtained in the same manner as in steps 200 to 204 described above, and the positional information in the reference coordinate system input from the 3D measuring device 20 Is converted into position information in a coordinate system in which the velocity vector S is the z-axis and the stationary position P is the origin (hereinafter, this coordinate system is referred to as a local coordinate system).

  Specifically, first, the coordinate system A1 is obtained by performing a conversion in which the origin of the reference coordinate system is translated to match the stationary position P. The coordinate system A1 is rotated around the y-axis of the coordinate system A1 so that the zy plane of the coordinate system A1 and the line segment M1 including the velocity vector S and the stationary position P are overlapped to obtain the coordinate system A2. Further, the coordinate system A2 is rotated around the x-axis of the coordinate system A2 so that the line segment M1 overlaps the z-coordinate axis of A2, thereby obtaining the coordinate system A3. These three conversion expressions are combined into one matrix (for example, 3 × 3). This matrix is called a coordinate system conversion formula.

  Hereinafter, a coordinate system in which the z axis coincides with the velocity vector S and the origin is the stationary position P (that is, the coordinate system A3 obtained by conversion) is referred to as a local coordinate system, and x, yz in the local coordinate system. In order to distinguish the axis from the x, y, and z axes of the reference coordinate system, they are called local x, y, and z axes. A plane formed by the local x axis and the local y axis is referred to as a local plane L. FIG. 9 shows an example of the local coordinate system. This local coordinate system is used during the command mode. That is, the x, y, and z coordinate values input from the 3D measurement device 20 at predetermined time intervals are converted into the x, y, and z coordinate values of the local coordinate system using the coordinate system conversion formulas.

  In step 122, a command processing routine (described later) is activated to execute command processing. This command processing routine is a processing routine for determining the movement of the LED 10 on the local z-axis and instructing the execution of the processing associated with the movement. An instruction input processing program is previously provided as a subroutine of the instruction input processing routine. Built in.

  In step 124, it is determined whether the timer T has reached the command mode time K2. If it is determined that the timer T <K2, the command processing routine is repeated. If it is determined that the timer T ≧ K2, it is determined that the command mode time K2 has elapsed, and the process returns to step 100. As a result, the command mode ends.

  Alternatively, during execution of the command processing routine, the main routine always monitors the timer T to determine whether or not the time K2 has elapsed, and when it is determined that the time K2 has elapsed, the command processing routine is forcibly terminated by an interrupt. May be.

  Here, the processing of the command processing routine will be described.

  First, the movement of the LED 10 determined by the command processing routine will be described with reference to FIG. In the figure, the acceleration state is indicated by a solid arrow, and the speed state is indicated by a dotted arrow.

  The click movement is a movement within a preset time, and after rapidly moving in the direction approaching the local plane L (minus direction of the local z axis), it rapidly moves in the opposite direction (plus direction of the z axis). (That is, reciprocates once on the local z axis), and corresponds to the movement A in the figure.

  The double-click movement is a movement within a preset time, and is set as a movement that repeats the click movement twice continuously (that is, reciprocates twice on the local z-axis). Corresponds to the movement of

  The drag-and-drop movement is composed of a drag movement and a drop movement, and corresponds to the movement C in the figure. The drag is set as a movement that moves rapidly in the minus direction or on the local plane L after a predetermined time has elapsed after rapidly moving in the minus direction of the local z axis. Note that there is no time limit for the movement that slowly moves in the minus direction or on the local plane L. The drop is set as a movement that moves rapidly in the positive direction of the local z-axis.

  FIG. 11 is a diagram showing the movement on the local z-axis of the double-click movement (B) of FIG. 10 as time elapses. In this figure, the horizontal axis represents the time axis, and the vertical axis represents the z-axis. A thick solid line indicated by points B0 to B4 represents the locus of movement of the LED 10. A thin solid arrow indicates the acceleration state, and a dotted arrow indicates the speed state. Note that the movement of the LED 10 on the time axis indicated by the thick solid line, and the acceleration and speed accompanying the movement are actually very small, but are exaggerated in this figure.

  Hereinafter, the command processing routine will be described using the flowcharts of FIGS. 12 to 14 while comparing with FIG.

  Here, in order to determine the movement of the LED 10 on the local z axis, the acceleration and speed for determining the movement of the LED 10 are detected using the z coordinate value of the local coordinate system.

Also, in order to detect acceleration and speed with the direction approaching the local plane L as the negative direction of the local z axis and the direction away from the local plane L as the positive direction of the local z axis, the LED 10 is moved in a direction approaching the local plane L. If so, the sign of the detected acceleration and speed will be-, and if it is moved away from the local plane L, the sign of the detected acceleration and speed will be +. Since the threshold value A ₀ and the threshold value V ₀ are set as absolute values, when comparing the detected acceleration or speed with the threshold value, if necessary, the acceleration and the speed are set to positive values. It processes by attaching | subjecting the code | symbol.

In step 400 of the flowchart of FIG. 12, the detected acceleration - code values marked with is to determine whether it exceeds the threshold value A ₀ of the acceleration. As described above, when the LED 10 is moved in a direction approaching the local plane L, the signs of speed and acceleration become −. Therefore, in this case, the speed and acceleration are set to positive values by attaching a sign of-.

If it is determined in step 400 that −acceleration ≦ A ₀ , this subroutine is terminated and the process returns to the main routine. In FIG. 11, the state before point B0 corresponds to the standby state. If it is determined in step 124 of the main routine (FIG. 5) that the timer T ≦ K2, the command processing is repeated. If it is determined in step 400 that -acceleration> A ₀ (corresponding to point B0 in FIG. 11), the operator has moved his / her hand with will, so the routine proceeds to step 402 and timer T1 And timer T2 is started.

Here, the timer T1 is a timer used for measuring the time from the start of movement in a certain direction to the start of movement in a direction opposite to the certain direction. A threshold T1 ₀ is set in advance for the timer T1.

The timer T2 is a timer used for determining the double click movement. For timer T2, it is previously threshold T2 ₀ is set.

In step 404, the speed - code values marked with is to determine whether it exceeds the threshold value V _0. Here, if it is determined that −speed ≦ V ₀ , the process proceeds to step 416 to reset the timers T1 and T2 and end the process. If it is determined that −speed> V ₀ (corresponding to point B1 in FIG. 11), the process proceeds to step 406 to determine whether or not the acceleration exceeds the threshold A ₀ . In addition, in order to clarify the moving direction of LED10 here, + sign is attached to acceleration for convenience.

If it is determined in step 406 that + acceleration ≦ A ₀ , the process proceeds to step 408, and it is determined whether or not the timer T1 exceeds the threshold value T1 ₀ . Here, when it is determined that the timer T1 ≦ T1 _0, the process returns to step 406. If it is determined that + acceleration> A ₀ , the process proceeds to step 410, and it is determined whether or not the timer T1 is equal to or less than the threshold T1 ₀ .

If it is determined in step 410 that timer T1> T1 _0, it is determined that the movement of the LED 10 does not correspond to any of the preset movements, the process proceeds to step 416, and timers T1 and T2 are reset. And exit. If it is determined in step 410 that the timer T1 ≦ T1 ₀ , the movement of the LED 10 is a movement within a preset time, so that the process proceeds to step 412 and the speed is set to the threshold value V ₀ . Judge whether or not it has been exceeded.

If it is determined in step 412 that + speed ≦ V _0, it is determined that the movement of the LED 10 does not correspond to any of the preset movements, the process proceeds to step 416, and timers T1 and T2 are reset. And exit. If it is determined that + speed> V _0, it is determined that the movement of the LED 10 corresponds to the above-described click movement set in advance, the process proceeds to step 414, and click processing is performed (B2 in FIG. 11). Equivalent to a point).

  The click process will be specifically described with reference to the functional configuration diagram of FIG. 3. The command generation unit 50 outputs a command for instructing execution of the process associated with the click movement to the mouse driver 52. The command generation unit 50 outputs the command in the same format as a command by a button operation on the two-dimensional mouse. As a result, the OS 54 to which the command is input via the mouse driver 52 can interpret the command and execute appropriate processing in the same manner as the two-dimensional mouse.

  After executing the click process in step 414, the process proceeds to step 420 in FIG. 13, and the timer T1 is reset and started. The processing after this step is processing for determining whether or not the movement of the LED 10 is a double-clicking movement.

In step 422, the acceleration - code values marked with is to determine whether it exceeds the threshold value A _0. -If it is determined that acceleration ≤ A ₀ , the process proceeds to step 424 to determine whether or not the timer T1 has exceeded the threshold value T1 ₀ .

If it is determined in step 424 that the timer T1> T1 ₀ , the process proceeds to step 444, where the timers T1 and T2 are reset and terminated. If it is determined in step 424 that the timer T1 ≦ T1 ₀ , the process returns to step 422.

If it is determined in step 422 that −acceleration> A ₀ , the process proceeds to step 426, and it is determined whether or not the timer T1 is equal to or less than the threshold T1 ₀ .

In step 426, if it is determined that the timer T1> T1 _0, the process proceeds to step 444, and ends and resets the timer T1 and T2. If it is determined in step 426 that the timer T1 ≦ T1 ₀ , the process proceeds to step 428, and it is determined whether or not the speed has exceeded the threshold value V ₀ .

If it is determined in step 428 that −speed ≦ V ₀ , the process proceeds to step 444 to reset the timers T1 and T2 and end the process. If it is determined in step 428 that −speed> V ₀ , the process proceeds to step 430, where the timer T1 is reset and started (corresponding to point B3 in FIG. 11).

At step 432, then the detected acceleration is determined whether it exceeds the threshold value A _0. If it is determined that + acceleration ≦ A ₀ , the process proceeds to step 434 to determine whether or not the timer T1 exceeds the threshold value T1 ₀ .

In step 434, if it is determined that the timer T1> T1 _0, the process proceeds to step 444, and ends and resets the timer T1 and T2. If it is determined that timer T1 ≦ T1 ₀ , the process returns to step 432.

If it is determined in step 432 that + acceleration> A ₀ , the process proceeds to step 436, and it is determined whether or not the timer T1 is equal to or less than the threshold T1 ₀ .

In step 436, if it is determined that the timer T1> T1 _0, the process proceeds to step 444, and ends and resets the timer T1 and T2. If it is determined that the timer T1 ≦ T1 ₀ , the process proceeds to step 438, and it is determined whether or not the timer T2 is equal to or less than the threshold T2 ₀ .

The timer T2 measures the time from when the click movement starts. Threshold T2 ₀ is previously set as the limit of the time required for the movement of a double-click. Therefore, by comparing the timer T2 and the threshold T2 _0, it can be determined whether the movement of the LED10 corresponds to double-clicking movement.

In step 438, if it is determined that the timer T2> T2 _0, the motion of LED10 is determined not to correspond to the movement of the double-click, the process proceeds to reset the timer T1 and T2 is terminated to step 444 . If it is determined that the timer T2 ≦ T2 ₀ , the process proceeds to step 440, and it is determined whether or not the + speed exceeds the threshold value V ₀ .

If it is determined in step 440 that + speed ≦ V ₀ , the process proceeds to step 444, the timers T1 and T2 are reset, and the process ends. If it is determined that + speed> V _0, it is determined that the movement of the LED 10 corresponds to the above-described double-click movement set in advance, and the process proceeds to step 442 to perform double-click processing (in FIG. 8). , Corresponding to B4 point).

  Similarly to the click process, in the double click process, the command generation unit 50 outputs a command for instructing the execution of the process associated with the double click movement to the mouse driver 52. The OS 54 to which the command is input via the mouse driver 52 can interpret the command similarly to the two-dimensional mouse and execute an appropriate process corresponding to the double click movement.

  After step 442, the process shifts to step 444, the timers T1 and T2 are reset, and the process ends.

  Note that the double-click movement is not particularly limited to that shown in FIG. 11 as long as it satisfies the speed, acceleration, and time conditions described above. For example, the start point on the local z-axis of the second click (ie, point B2) need not be the same position as the start point on the local z-axis of the first click (ie, point B0).

On the other hand, in step 408 of FIG. 12, when it is determined that the timer T1> threshold T1 _0, it is determined that the movement of the LED10 corresponds to the motion of the drag that is set in advance, and proceeds to step 450 in FIG. 14 , Perform the drag process.

  Specifically, the command generation unit 50 outputs, to the mouse driver 52, a command for instructing execution of the process associated with the drag movement, similarly to the click process. The OS 54 to which the command is input via the mouse driver 52 interprets the command and executes an appropriate process corresponding to the drag movement.

  The drag process is usually a process performed after the click process. For example, an object such as an icon selected by the click process can be dragged (moved).

After step 450, in step 452, it is determined whether the acceleration exceeds the threshold value A _0. If it is determined that + acceleration ≦ A ₀ , the process returns to step 450 and the drag process is continued. If it is determined that + acceleration> A ₀ , the process proceeds to step 454 to determine whether or not the speed exceeds the threshold value V ₀ .

If it is determined in step 454 that + speed ≦ V ₀ , the process returns to step 450 and the drag process is continued. When it is determined that + speed> V ₀ , the movement of the LED 10 corresponds to the drop movement set in advance, and the process proceeds to step 456 to perform the drop process.

  In the drop process as well as the drag, the command generation unit 50 outputs a command for instructing execution of the process associated with the movement of the drag to the mouse driver 52. The OS 54 to which the command is input via the mouse driver 52 interprets the command and executes an appropriate process corresponding to the drag movement.

  After step 456, timers T1 and T2 are reset at step 458 and the process ends.

  The instruction input processing routine described above will be described with a specific example. For example, if the operator wants to open a file (its icon) displayed on the display 42 by double-clicking it, first the pointing mode is started by the first push, and the file that the operator wants to open. The LED 10 is moved so that the position where the icon is displayed matches the display position of the cursor (pointing). When pointing to the file is completed, the second push is performed to start the command mode, and a double click operation is performed. As a result, the file can be opened. If it is not necessary to correct the cursor position in the first push, the second push may be performed immediately.

Note that the above-described drop movement is not the movement of rapidly moving the LED 10 in the positive direction on the local z axis, but the movement of pushing the LED 10 against the local plane again (that is, rapidly in the negative direction on the local z axis). Moving motion). In this case, step 452 is replaced with a process for determining whether or not −acceleration> A ₀ , and step 454 is replaced with a process for determining whether or not −speed> V ₀ . As a result, the drop process is performed when the data is pushed again to the local plane. According to this method, the movement of the drop is close to the movement of the person “places”, and the operator can input instructions with a more natural movement.

  Further, when determining whether or not the movement of the LED 10 corresponds to a predetermined movement (click, double click, drop & drop), the determination is made by giving an allowable amount to the movement of the LED 10 or the predetermined movement. You may do it. For example, when the operator wears and moves the LED 10 on his / her hand, there may be a case where an unintended minute blur occurs due to the fluctuation of the hand. Accordingly, the determination means can input instructions more smoothly by determining the movement of the LED 10 or a predetermined movement with an allowable amount.

  As described above, the position information of the LED 10 is input, the acceleration and speed associated with the movement of the LED 10 are detected based on the position information, and the speed vector S indicating the moving direction of the LED 10 is calculated based on the detected acceleration and speed. Then, the line segment M1 including the velocity vector S is calculated, the intersection C1 between the line segment M1 and the virtual plane V provided corresponding to the display screen is calculated, and the position corresponding to the intersection C1 of the screen of the display 42 Since the cursor is displayed on the screen, it is possible to perform a pointing operation with a natural motion without performing a mechanical operation.

  Further, a movement vector PQ is calculated from the stationary position P where the LED 10 is stationary after the acceleration exceeds a predetermined value and the moving position Q moved from the stationary position, and a plane N perpendicular to the direction indicated by the movement vector PQ is calculated. And the intersection point O of the line segment M1 is calculated, the intersection point C2 of the line segment M2 including the intersection point O and the movement position Q and the virtual plane V is calculated, and the display position of the cursor is moved. The cursor position can be moved and fine-tuned with natural operations without difficulty.

  Furthermore, the direction (speed vector S) indicated by the operator can be obtained by the movement of the first LED 10 in the pointing operation. Therefore, if the LED 10 is moved in a desired direction at the initial stage of the pointing operation, even if the subsequent operation becomes unstable, the pointing can be performed at a desired position. It is easy to use even for those who have difficulty in performing the operation stably.

  Further, since the pointing process is performed according to the moving direction of the LED 10 without using the absolute coordinate value, the remote operation is possible.

  In addition, by detecting light from one light emitting means (LED 10) without using an acceleration sensor, the three-dimensional position can be measured to determine the direction and position indicated by the operator, which makes the system easy and It can be constructed at low cost.

According to the experiment, the threshold for acceleration is about 3x10 ^-7 m / sec ² and the threshold for velocity is preferably about 6x10 ^-6 m / sec. I know that. Since this value is several orders of magnitude smaller than the acceleration detected by a commercially available acceleration sensor, it is not suitable to use an acceleration sensor as a device for detecting the acceleration. Therefore, as in the present invention, the movement of the LED can be detected with high accuracy by detecting the acceleration and speed using a measuring device that detects the light emitted by the light emitting means and measures the three-dimensional position without using the acceleration sensor. The system can be constructed at a low cost with a simple structure and simple calculation.

In the present embodiment, the example in which the movement position Q is detected every predetermined time has been described.
The moving position Q may be a stationary position that is stationary after the stationary position P is detected.

  Alternatively, the intersection point O may be fixed for a predetermined time, and the intersection point C2 may be sequentially updated from the line segment M2 including the fixed intersection point O and the movement position Q. Thereby, the amount of calculation can be reduced compared with the case where the intersection point O is always recalculated. In this case, since an error may occur with a time lag, it is also preferable to recalculate the intersection point O at regular intervals. Further, the intersection point O may be recalculated when the angle formed by the line segment M1 and the line segment M2 exceeds a predetermined value. Further, without recalculating the intersection point O, the display of the cursor may be temporarily deleted and the push operation may be performed again.

  Furthermore, when the angle formed by the previous line segment M2 and the current line segment M2 exceeds a predetermined value, it may be determined that the pointing operation has become slow and the cursor display may be temporarily turned off.

  If the line segment M1 and the line segment M2 (plane N) are parallel, the intersection point O cannot be obtained. Assuming such a case, for example, before the step of calculating the intersection point O, a step for determining whether or not the line segment M1 and the plane N are parallel is provided in advance, and when it is determined that they are parallel The cursor may not be moved, or may be controlled so that the cursor disappears from the screen of the display 42, and the operator may perform an operation for pointing again.

  Further, as described above, whether the movement of the LED 10 in the coordinate system using the plane perpendicular to the velocity vector S as the xy plane and the velocity vector S as the z axis is a predetermined movement is determined by the acceleration and velocity. Since the process is instructed to execute the process associated with the determined movement, it is determined in advance by moving the LED in an arbitrary direction even if the direction is not perpendicular to the display screen. Therefore, it is possible to input instructions smoothly with natural movement for the operator.

  In the above-described embodiment, the example in which the second push is the same operation as the first push has been described. For example, the second push is the same as the click operation described above. Further, it may be an operation that makes one reciprocation in an arbitrary direction. Thereby, it is possible to input an instruction with a more natural movement for the operator.

  In the above-described embodiment, an example in which an instruction is input only by the movement direction or movement of the LED 10 has been described. For example, an instruction input switch or button is provided separately from the LED 10, and the movement direction of the LED 10 It is also possible to selectively use an instruction input by movement and an instruction input by a mechanical operation such as pressing a button as necessary.

  Alternatively, a battery may be connected to the LED 10 and the battery may be attached to the same hand as the LED 10. As a result, a cord for supplying power to the LED 10 is not required and can be operated wirelessly.

  In the above-described embodiment, the example in which the LED 10 is mounted on the operator's hand or finger has been described. For example, the LED 10 may be provided in a pen-type tool 12 as shown in FIG. The pen-type tool 12 may be provided with a power supply means such as a battery. Further, as shown in FIG. 16, the LED 10 may be provided in the mobile phone 16. In this case, the power source of the mobile phone 16 can be shared with the power source of the LED 10.

[Second Embodiment]
In the first embodiment, an example has been described in which when the LED 10 moves, a cursor is displayed at a position corresponding to only the calculated latest intersection C2. However, in this embodiment, not only the latest intersection C2 but also the latest intersection C2 is displayed. An example will be described in which a cursor is displayed at a corresponding position for intersections calculated in the past, and the display state of the plurality of cursors is changed as time passes.

  Note that the system configuration and the configuration of the personal computer 30 in the present embodiment are the same as those in the first embodiment, and a description thereof will be omitted.

  FIG. 17 is a diagram showing a display state of the cursor displayed on the display 42 in the present embodiment. As shown in the figure, the display 42 displays a plurality of cursors from the latest cursor to several steps before (here, a total of five cursors up to four steps before). The time elapsed since the cursors 60a, 60b, 60c, 60d, and 60e are displayed in this order is short. The display color of the cursor is darker in order of the shorter time that has elapsed since it was displayed.

  FIG. 18 is a flowchart showing the flow of the display control routine according to the present embodiment. This display control routine is included in the aforementioned instruction input processing program.

  In step 500, it is determined whether a predetermined time has elapsed. When it is determined that the predetermined time has elapsed, in step 502, the cursor display state is changed.

  Specifically, display control is performed so that the display colors of the cursors 60a, 60b, 60c, 60d, and 60e are reduced to the same extent.

  Although illustration is omitted, when the moving position Q is newly detected and a new intersection C2 with the virtual plane V is detected, the cursor 60e having the longest display time is deleted, and the display Control is performed so that the latest cursor is displayed at a position corresponding to the new intersection C2 on the screen of 42. In addition, the display states of the cursors 60a, 60b, 60c, and 60d are changed. That is, display control is performed so that the display color of the newly displayed cursor is the darkest and the display color of the cursor 60d is the lightest.

As described above, since the cursor is displayed at the corresponding position not only for the latest intersection C2 but also for the intersection calculated in the past, the display state of the cursors is changed as time elapses. The instruction input position can be easily confirmed without losing sight.
[Third Embodiment]
An example in which the instruction input system described in the first and second embodiments is applied to an entertainment system such as a game and an auxiliary system for operating a copying machine will be described. In the present embodiment, the personal computer 30 having the function as the instruction input device of the present invention is not shown.

  FIG. 19 is a diagram showing a case where the instruction input system is applied to a mole-tapping game. The screen shown is the screen of the display 42 of the personal computer 30. Animal faces appear randomly on the screen of the display 42. The user uses a pen-shaped tool 12 provided with the LED 10 (hereinafter referred to as the LED pen 12) and points or clicks on the appearing face according to the rule of tapping. When the face that appears is pointed or clicked, points are added accordingly.

  FIG. 20 is a diagram showing a case where the instruction input system is applied to a dart game. In this case, a pattern (such as a plurality of concentric circles) normally used for darts is displayed on the screen. When the user pretends to throw the LED pen 12, the CPU 34 of the personal computer 30 calculates the traveling direction of the LED 10 and calculates the position indicated by the LED 10 (that is, calculates the intersection C1 of the first embodiment). It is determined whether or not the pattern displayed on the screen is hit, and the score is added according to the hit location.

  FIG. 21 is a diagram showing a case where the instruction input system is applied to an invader game. Falling objects such as bombs are displayed on the screen. Falling objects fall from the top to the bottom of the screen. The user points or clicks the fallen object with the LED pen 12. When the falling object can be pointed or clicked, the falling object explodes and the score is added. Points may be added according to the type of fallen object. When the falling object reaches the ground surface (the bottom of the screen), the score may be subtracted according to the type of the falling object.

  FIG. 22A is a diagram showing a case where the instruction input system is applied to a hammer game. The screen is displayed in a three-dimensional representation using VRML (Virtual Reality Modeling Language) or the like. A hammer 62 is displayed at the center of the screen, and is fixed so that the bottom of the handle of the hammer 62 serves as a base point on the screen. The user can move the hammer 62 around the base point by dragging the LED pen 12. In the three-dimensional space represented on the screen, multiple objects such as comets move freely and at different speeds. The user moves the hammer 62 so as to strike an object by the drag movement of the LED pen 12. When the hammer 62 hits the object, points are added. The base point can be moved in a three-dimensional space by the drag and drop movement of the LED pen 12. In addition, as shown in FIG. 22B, a game in which an insect net 64 is displayed instead of the hammer 62 and an object is captured by the insect net 64 may be used. It is preferable that the hammer 62 and the insect net 64 exaggerate the three-dimensional depth expression.

  FIG. 23 shows a large scale of the game mechanism shown in FIGS. 22 (A) and 22 (B). As shown in the drawing, a three-dimensional space is expressed using a large screen 42 a instead of the display 42, and the LEDs 10 are provided at the uppermost and lowermost positions of the handle of the real insect net 68. On the large screen 42a, a plurality of virtual insects (dragonfly etc.) and a virtual insect net 72 corresponding to the real insect net 68 are displayed. Further, a three-dimensional recognition region in which the light receiving units 70 of the 3D measuring device 20 are arranged at four positions is provided in front of the large screen 42a. The 3D measuring device 20 can recognize the position and movement of the insect net 68 on the personal computer 30 side by measuring the three-dimensional positions of the two LEDs 10 and outputting the two pieces of positional information to the personal computer 30.

  The user moves the insect net 68 provided with two LEDs 10 in the three-dimensional recognition area so that the movement is recognized on the personal computer 30 side, and thereby the virtual insect net displayed on the large screen 42a. 72 can be moved. The user moves the insect net 68 to move the virtual insect net 72, and captures the virtual insect moving in the virtual space in a free direction and at a preset speed. Different points are added depending on the type of captured virtual insects.

  Here, it is assumed that the lowermost part (base point) of the pattern of the virtual insect net 72 is moved by the drag movement of the lowermost part of the pattern of the insect net 68. Further, by recognizing the movement of the top of the handle of the insect net 68, the position relative to the base point in the three-dimensional space can be calculated in the same manner as in the first embodiment. Since the length of the handle is fixed, it is only necessary to add a constant to the base point of the handle.

  Similarly, since the size of the net part of the insect net 68 is determined, the position of the net part can be easily obtained. Accordingly, the movement or trajectory of the net can be calculated in real time, and it can be determined whether or not the virtual insect has entered the net of the virtual insect net 72.

  This game can be used in game centers, homes with large projectors, and large screens. Although it is possible to provide a similar game using a motion capture device, it is expensive. On the other hand, if this instruction input system is used, it can be realized at a low cost, and only two light sources need be provided, so that power consumption (battery) can be reduced. In the case of motion capture, the power is usually supplied in a wired manner with a cord, and the user's operation is limited to some extent. However, since this instruction input system does not consume a large amount of power, it is supplied with a battery or the like. You only need to do this, and there are fewer restrictions on the operation.

  In addition, the HMD (head-mounted display) and data glove that are normally required for realizing VR (virtual reality) are not required in this instruction input system, so that the user's pleasant feeling of use is improved and the cost is reduced. This is also advantageous in terms of the aspect.

  FIG. 24 is a diagram showing a case where the instruction input system is applied to a falling object capturing game. The falling object is projected onto the large screen 42a by the projector 74. Falling objects are dropped randomly. The light receiving unit 70 of the 3D measurement device 20 is provided on the large screen 42a, and the position of the light source such as the LED 10 mounted on the user's hand or finger is measured.

  The user performs an operation of hitting the falling object displayed on the large screen 42a. The 3D measurement device 20 measures the three-dimensional position of the LED 10, detects the acceleration, speed, direction, and the like associated with the movement of the LED 10 based on position information indicating the three-dimensional position, and detects the moving direction and movement of the LED 10. Further, it is determined whether or not the fallen object has been struck from the position of the fallen object and a preset allowable amount. If it is determined that the object has been struck, an explosion image is displayed, and the display of the struck falling object is erased. These processes are performed by the personal computer 30.

  The score differs depending on the size and type of the fallen object, and when the fallen object falls to the ground (the bottom of the large screen 42a), the score may be subtracted depending on the type of fallen object. The deducted score is not necessarily the negative number of the score to be added.

  Further, as the game progresses, the remaining time can be displayed on the large screen 42a or music can be played to increase the excitement of the game. Unlike the hammer game shown in FIG. 23, this game does not require VRML software and can be realized at a lower cost.

  FIG. 25 is a diagram showing a case where the instruction input system is used to assist the copying machine operation. Among copying machines currently in widespread use, an operation screen (touch panel display) 82 is provided at the upper rear end of the copying machine 80 as shown. As described above, when the operation screen 82 is located at a relatively distant position, it becomes difficult for a child, a person who is relatively short, or a person with a physical disability to touch the operation screen 82 itself. It ’s hard.

  Accordingly, the light receiving unit of the 3D measuring apparatus 20 is provided on the side surface of the copying machine 80 or the side surface of the operation screen 82, and the LED 10 is attached to the user's finger or the tool such as the LED pen 12 is used. It is possible to easily input an instruction to the operation screen 82 provided at a relatively distant position.

  Thus, if the instruction input system is applied to a copying machine or the like, the operation of the touch panel display can be performed from a relatively remote place without touching the screen.

  Further, the present invention can be applied not only to copying machines but also to operation of touch panel displays such as ATMs of financial institutions. It can be applied to ON / OFF operation of a switch.

It is a figure which shows the structure of the instruction | indication input system containing the personal computer provided with the function as an instruction | indication input apparatus which concerns on the 1st Embodiment of this invention. It is the figure which showed the specific structure of the personal computer. It is the figure which showed the structure of the personal computer main body functionally. It is the flowchart which showed the main routine of instruction input processing. It is the flowchart which showed the main routine of instruction input processing. It is the flowchart which showed the 1st pointing processing routine. It is the flowchart which showed the 2nd pointing processing routine. It is explanatory drawing which demonstrated the process in a 1st pointing process and a 2nd pointing process geometrically. It is the figure which showed an example of the local coordinate system. It is the figure which showed the specific example of instruction | indication operation | movement in a local coordinate system. It is the figure which showed the motion on the local z-axis of the double click motion (B) of FIG. 10 according to progress of time. It is the flowchart which showed the command processing routine. It is the flowchart which showed the command processing routine. It is the flowchart which showed the command processing routine. It is the figure which showed the pen-type tool provided with LED10. It is the figure which showed the mobile telephone provided with LED10. It is a figure which shows the display state of the cursor displayed on the display in 2nd Embodiment. It is the flowchart which showed the display control routine. It is the figure which showed the case where the instruction | indication input system is applied to the mole-tapping game. It is the figure which showed the case where the instruction | indication input system is applied to a darts game. It is the figure which showed the case where the instruction | indication input system is applied to an invader game. FIG. 22A is a diagram showing a case where the instruction input system is applied to a hammer game, and FIG. 22B is a diagram showing an insect net displayed instead of the hammer of FIG. 22A. is there. It is the figure which showed the game which made large the mechanism of the game shown to FIG. 22 (A) and FIG. 22 (B). It is the figure which showed the case where the instruction | indication input system is applied to the falling object capture game. FIG. 3 is a diagram showing a case where an instruction input system is used to assist copying machine operation.

Explanation of symbols

10 LED
16 Mobile phone 20 3D measuring device 30 Personal computer 32 Personal computer body 34 CPU
36 ROM
38 RAM
40 I / O
42 Display 46 HDD
50 Command generator

Claims (17)

Input means for inputting position information indicating the position of the light emitting means measured by detecting light emitted from the light emitting means that moves according to the operation of the predetermined part of the operator;
Detecting means for detecting acceleration and speed of the light emitting means based on the positional information input by the input means;
A speed vector calculating means for calculating a speed vector indicating a moving direction of the light emitting means based on a position where the acceleration detected by the detecting means exceeds a predetermined value based on the detected speed;
An intersection between a line segment including the velocity vector calculated by the velocity vector calculating unit and a virtual plane provided corresponding to a display unit that displays instruction information for indicating a position instructed by the light emitting unit. A display intersection calculating means for calculating as a display intersection;
Control means for controlling the display means so that the instruction information is displayed at a position corresponding to the position of the display intersection calculated by the display intersection calculation means of the display means;
Instruction input device including Calculates a movement vector, which is a vector whose starting point is the rest position where the light emitting means is stationary after the acceleration detected by the detecting means exceeds a predetermined value, and whose moving position is the end point where the light emitting means has moved from the stationary position. Movement vector calculation means for
Plane calculation means for calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position;
An intersection calculation unit for calculating an intersection of the plane calculated by the plane calculation unit and a line segment including the velocity vector calculated by the velocity vector calculation unit;
The display intersection calculation means calculates an intersection between the line segment including the velocity vector and the virtual plane as a display intersection, and after the acceleration detected by the detection means exceeds the predetermined value, The instruction input device according to claim 1, wherein an intersection of the line segment including the intersection calculated by the intersection calculation means and the movement position and the virtual plane is calculated as a display intersection.   The processing unit according to claim 1, further comprising a processing unit that performs a predetermined process using a coordinate system including a speed vector calculated by the speed vector calculation unit and a plane perpendicular to the speed vector. Instruction input device.   The predetermined process determines whether or not the movement of the light emitting means corresponds to a predetermined movement based on the acceleration and speed of the light emitting means in the coordinate system detected by the detection means. The instruction input device according to claim 3, wherein when the movement of the light emitting means is determined to correspond to a predetermined movement, the instruction input device is a process that instructs execution of a process associated with the predetermined movement.   The control means is configured to display a plurality of the instruction information corresponding to each of the latest display intersection calculated by the display intersection calculation means and a plurality of display intersections calculated in the past. The instruction input device according to any one of claims 1 to 5, which controls display means.   The instruction input device according to claim 5, wherein the control unit controls the display unit so that a display state of the plurality of instruction information changes as time passes. The display means displays the instruction information and a predetermined object,
The control means matches the display position of the instruction information with the display position of the predetermined object, the display position of the instruction information and the display position of the predetermined object, and the processing means. 5. The display means is controlled so as to change the display state of the display means in any one of the cases where execution of processing associated with the predetermined movement is instructed by The instruction input device according to claim 6. Light emitting means that moves according to the operation of a predetermined part of the operator;
A measuring device that measures the position of the light emitting means by detecting light emitted from the light emitting means;
The instruction input device according to any one of claims 1 to 7,
Including instruction input system.   9. The instruction input system according to claim 8, wherein the light emitting means is provided in a mobile phone. An input step of inputting position information indicating the position of the light emitting means measured by detecting light emitted from the light emitting means that moves according to the operation of a predetermined part of the operator;
A detection step of detecting the acceleration and speed of the light emitting means based on the positional information input in the input step;
A velocity vector calculation step of calculating a velocity vector indicating the direction of movement of the light emitting means based on a position where the acceleration detected by the detection step exceeds a predetermined value, based on the detected velocity;
An intersection of a line segment including the velocity vector calculated by the velocity vector calculating step and a virtual plane provided corresponding to a display unit that displays instruction information for indicating a position instructed by the light emitting unit. A display intersection calculation step for calculating as a display intersection;
A control step of controlling the display means so that the instruction information is displayed at a position corresponding to the position of the display intersection calculated by the display intersection calculation step of the display means;
Instruction input method including Calculates a movement vector, which is a vector whose starting point is the rest position where the light emitting means is stationary after the acceleration detected in the detection step exceeds a predetermined value, and whose moving position is the end point where the light emitting means has moved from the stationary position. A moving vector calculation step to perform,
A plane calculation step of calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position;
An intersection calculation step of calculating an intersection of the plane calculated by the plane calculation step and a line segment including the velocity vector calculated by the velocity vector calculation means;
The display intersection calculation step calculates an intersection between the line segment including the velocity vector and the virtual plane as a display intersection, and after the acceleration detected by the detection step exceeds the predetermined value, The instruction input method according to claim 10, wherein an intersection of the line segment including the intersection calculated by the intersection calculation step and the movement position and the virtual plane is calculated as a display intersection.   12. The processing step according to claim 10 or 11, further comprising a processing step of performing a predetermined processing using a coordinate system configured by the velocity vector calculated by the velocity vector calculation step and a plane perpendicular to the velocity vector. Instruction input method.   The predetermined process determines whether or not the movement of the light emitting means corresponds to a predetermined movement based on the acceleration and speed of the light emitting means in the coordinate system detected by the detection step. 13. The instruction input method according to claim 12, which is a process for instructing execution of a process associated with the predetermined movement when it is determined that the movement of the light emitting means corresponds to the predetermined movement. On the computer,
An input step of inputting position information indicating the position of the light emitting means measured by detecting light emitted from the light emitting means that moves according to the operation of a predetermined part of the operator;
A detection step of detecting the acceleration and speed of the light emitting means based on the positional information input in the input step;
A velocity vector calculation step of calculating a velocity vector indicating the direction of movement of the light emitting means based on a position where the acceleration detected by the detection step exceeds a predetermined value, based on the detected velocity;
An intersection of a line segment including the velocity vector calculated by the velocity vector calculating step and a virtual plane provided corresponding to a display unit that displays instruction information for indicating a position instructed by the light emitting unit. A display intersection calculation step for calculating as a display intersection;
A control step of controlling the display means so that the instruction information is displayed at a position corresponding to the position of the display intersection calculated by the display intersection calculation step of the display means;
A program for running Calculates a movement vector, which is a vector whose starting point is the rest position where the light emitting means is stationary after the acceleration detected in the detection step exceeds a predetermined value, and whose moving position is the end point where the light emitting means has moved from the stationary position. A moving vector calculation step to perform,
A plane calculation step of calculating a plane perpendicular to the movement vector calculated by the movement vector calculation means and including the movement position;
An intersection calculation step of calculating an intersection of the plane calculated by the plane calculation step and a line segment including the velocity vector calculated by the velocity vector calculation means;
The display intersection calculation step calculates an intersection between the line segment including the velocity vector and the virtual plane as a display intersection, and after the acceleration detected by the detection step exceeds the predetermined value, The program according to claim 14, wherein an intersection of the line segment including the intersection calculated by the intersection calculation step and the moving position and the virtual plane is calculated as a display intersection.   16. The processing step according to claim 14 or 15, further comprising a processing step of performing a predetermined processing using a coordinate system configured by the velocity vector calculated by the velocity vector calculation step and a plane perpendicular to the velocity vector. program.   The predetermined process determines whether or not the movement of the light emitting means corresponds to a predetermined movement based on the acceleration and speed of the light emitting means in the coordinate system detected by the detection step. The program according to claim 16, which is a process for instructing execution of a process associated with the predetermined movement when it is determined that the movement of the light emitting means corresponds to the predetermined movement.

Images