JP2004303000A - Three-dimensional instruction input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To execute a function of a two-dimensional mouse such as click, double click or drag by use of three-dimensional information without using a mechanical mechanism. <P>SOLUTION: A space extending in the front of a screen is divided into three areas. When an LED 10 worn by a user goes and returns through the boundary between a semi-pressing layer and a full-pressing layer once in a certain fixed time while making the LED recognize the passage of the boundary of each area, the click operation of the two-dimensional mouse is executed, and when it goes and returns twice in the fixed time, the double click operation of the two-dimensional mouse is executed. When it is not returned to the semi-pressing layer for a fixed time after moved from the semi-pressing layer to the full-pressing layer, an object such as an icon is dragged, and when the LED 10 is returned to the semi-pressing layer, the operation of dropping the dragged object is executed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a three-dimensional instruction input device, and more particularly to a three-dimensional instruction input device for inputting an instruction based on three-dimensional information of a light emitting unit.
[0002]
[Prior art]
Conventionally, a two-dimensional mouse has been widely used as a device for inputting instructions to a personal computer or the like. The two-dimensional mouse inputs and inputs xy coordinate values to a personal computer as relative coordinate values. In recent years, a technique for inputting a third coordinate value using this two-dimensional mouse has been proposed.
[0003]
For example, a three-dimensional mouse can be easily achieved by attaching an LED as a light emitter to a finger and attaching a light receiving unit for receiving light from the LED to an upper portion of a screen of a personal computer (for example, see Patent Document 1). . The sensor used here determines the angle of the light source with respect to the sensor, and determines the three-dimensional position of the light source based on the principle of triangulation. The LED receives power as a light source from a personal computer, and signals such as a click, which is a basic operation of a mouse, are sent to the basic software of the personal computer through a wire. That is, a switch that can be pressed by a finger is provided near the light emitting body.
[0004]
In addition, by rotating two balls in two dimensions, not only a movement amount in the x and y axis directions can be input, but also an angle and a rotation angle can be input, and a pointing device that enables input in the z axis direction ( For example, see Patent Document 2), a first optical sensor array in which light converting elements are arranged in a first direction, and a second optical sensor array in which light converting elements are arranged in a second direction different from the first direction. Non-contact type position detection device that generates a signal corresponding to the output change of the optical sensor array when the optical sensor array moves, and thereby determines the amount of movement and the direction of movement (for example, Patent Document 3) Cf.) is known.
[0005]
Further, a remote control device (for example, see Patent Document 4) for recognizing an index with a CCD camera and interpreting the index as a command for a personal computer or the like, and a rotary switch rotated with a finger in an electronic pen are provided. A coordinate input electronic pen (see, for example, Patent Document 5) that determines the degree of change in graphical parameters (line segment thickness, color, shading, gray scale), and movement of an RFID attached to a user's wrist. After being recognized by the antenna, the movement pattern of the RFID is interpreted as an input command to another device. Similarly, when there is one antenna, an instruction input system that interprets a time-series pattern in which an RFID is recognized or not recognized as a command is known (for example, see Patent Document 6).
[0006]
Gyration Inc., USA Gyromouse has a built-in auto gyro and is a pointing device that can change the direction of the laser by air operation, but it is relatively heavy, the size is relatively large for the gyro, and it is somewhat expensive. is there.
[0007]
Furthermore, in recent years, a prototype of a device that combines a keyboard and a notepad with a keyboard of a personal computer that applies the concept of half-pressing in the shutter operation of a camera, taking advantage of the characteristic that characters are not input unless you press it strongly like a camera shutter. Was done.
[0008]
[Patent Document 1]
JP-A-10-9812
[Patent Document 2]
JP-A-5-150900
[Patent Document 3]
JP-A-11-24832
[Patent Document 4]
JP-A-11-110125
[Patent Document 5]
JP 2000-47805 A
[Patent Document 6]
JP 2001-306235 A
[0009]
[Problems to be solved by the invention]
However, the above-described conventional three-dimensional mouse has the same operability as a two-dimensional mouse, as is apparent from the fact that a signal such as a click is transmitted to the basic software of a personal computer via a cable. Other instruction input devices such as the above-described conventional pointing device and instruction input system have the same operability as a two-dimensional mouse.
[0010]
Further, US Gyration Inc. Gymouse Presenter of the company also relies on mechanical operation with a finger for operations such as clicking, like a normal two-dimensional mouse. Further, even in a device in which a keyboard and a notepad are overlapped, the operation of a cursor is restricted by two-dimensional information and does not use three-dimensional information.
[0011]
That is, there is a problem that the operability of the conventional instruction input device is restricted two-dimensionally.
[0012]
The present invention has been made in order to solve the above-described problem, and it is an object of the present invention to execute two-dimensional mouse functions such as click, double-click, and drag using three-dimensional information without using a mechanical mechanism. It is an object of the present invention to provide a three-dimensional instruction input device capable of performing the following.
[0013]
[Means for Solving the Problems]
In order to achieve the above object, a three-dimensional instruction input device according to the present invention includes: a light emitting unit that can be worn by an instruction input person; a light receiving unit that receives light emitted from the light emitting unit; A measuring means for measuring a three-dimensional position of the means, and a plurality of spaces divided from the screen in the direction of the instruction input person on the front of a screen for displaying predetermined information, the measurement being performed by the measuring means. Based on the three-dimensional position, the state in which the light-emitting means passes through the boundaries of the plurality of spaces is determined, and the feedback change appealing to at least one of the five senses of the instruction input person is made in accordance with the determined state. Execution means for executing a function as a three-dimensional mouse.
[0014]
In addition, the three-dimensional instruction input device of the present invention includes a light-emitting unit that can be attached to a hand or a finger of an instruction input person, receives light emitted from the light-emitting unit, and detects the three-dimensional position of the light-emitting unit. Measurement means for measuring from the obtained light, and a plane substantially coinciding with a screen for displaying predetermined information as an xy plane, and a space in front of the screen in a z-axis direction perpendicular to the xy plane. The area is divided into three areas of a first area, a second area, and a third area in order from the closest to the screen, and the three areas are obtained from the z coordinate values of the three-dimensional position of the light emitting means obtained by the measuring means. A way of passing the light emitting means with respect to each boundary dividing the area is determined, and selection and click are performed so that feedback different depending on the way of passing the light and appealing to at least one of the five senses of the instruction input person is applied. , Double chestnut Click, and execution means for executing the function of the two-dimensional mouse comprising a drag-and-drop can also be configured to include.
[0015]
The light emitting means can be mounted on the instruction input person, and is preferably mounted on a finger or a hand. The light emitting means is not particularly limited as long as it emits light, and may be, for example, an LED. The measuring means receives the light from the light emitting means and measures the three-dimensional position of the light emitting means from the received light.
[0016]
On the other hand, predetermined information is displayed on the screen visually recognized by the instruction input person. The execution means sets a plane substantially coincident with the screen as an xy plane, and sets a space in front of the screen in the first area and the second area in order from the screen in the z-axis direction perpendicular to the xy plane. The image is divided into three regions, a region and a third region. Further, the execution means determines a way of the light emitting means passing through each boundary of the three regions from the z-coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means. Provide feedback to appeal to at least one of the five senses of the input person.
[0017]
That is, by measuring the three-dimensional position of the light emitting unit that changes according to the finger or hand movement of the instruction input person by the measuring unit, the execution unit can determine the way of the light emitting unit passing through each boundary. Furthermore, since the execution means can make the instruction input person recognize the three-dimensional position of the light emitting means by feedback, the instruction input person confirms the position of the light emitting means, that is, the space in front of the screen is divided into three areas. The user can move his / her finger or hand on which the light emitting unit is mounted while checking the passing state of each boundary to be divided.
[0018]
The execution means executes the functions of the two-dimensional mouse including selection, click, double-click, drag and drop according to the way the light-emitting means passes through each boundary while giving feedback as described above.
[0019]
Therefore, according to the present invention, the function of the two-dimensional mouse can be executed by using the three-dimensional information without using a mechanical mechanism.
[0020]
The execution means of the present invention displays a small window on the xy plane of the light emitting means on the screen as the feedback, and in the small window, among images displayed on the screen, The image of a predetermined area including the position on the xy plane of the light emitting means is enlarged and displayed, and the magnification of the image displayed in the small window is changed according to the z coordinate value of the light emitting means. Thereby, it is possible to appeal to the vision of the instruction input person.
[0021]
The execution means causes a small window to be displayed at a position on the xy plane of the light emitting means on the screen as feedback. The shape of the small window is not particularly limited, and may be rectangular or circular. The execution means enlarges and displays, within the small window, an image of a predetermined area including the position of the light emitting means on the xy plane among the images displayed on the screen. Further, the magnification of the image displayed in the small window is changed according to the z-coordinate value of the light emitting means to appeal to the instruction input person's vision.
[0022]
As described above, by changing the enlargement ratio of the image displayed in the small window according to the z-coordinate value of the light emitting unit, the instruction input user can recognize the position of the light emitting unit in the z-axis direction.
[0023]
When the light emitting unit approaches a boundary between the first region and the second region from the second region side, the execution unit reduces the enlargement ratio as approaching the boundary. When passing through a boundary, the enlargement ratio is set to 1 so that the instruction input person can recognize the boundary.
[0024]
Thus, the instruction input person can recognize the state in which the light emitting unit is approaching the boundary by the change in the enlargement ratio. When the enlargement ratio becomes 1, it can be recognized that the light emitting means is passing through the boundary. Therefore, the instruction input person can clearly recognize the position of the light emitting unit in the z-axis direction, and can appropriately execute the function of the two-dimensional mouse by appropriately moving the finger or hand wearing the light emitting unit.
[0025]
Further, the execution means of the present invention can change at least one of hue, saturation, and lightness of an image displayed in the small window according to the z-coordinate value of the light emitting means.
[0026]
Thus, the instruction input person can recognize the position of the light emitting unit in the z-axis direction.
[0027]
Further, when the light emitting means moves from the third area toward the second area, the execution means of the present invention continuously changes the brightness in the small window according to the movement of the light emitting means. When the brightness reaches a predetermined value due to the movement of the light emitting means, the value is kept constant. When the light emitting means moves and exceeds a predetermined z coordinate value, the second The brightness is reduced again as approaching the boundary between the region and the first region, and in a predetermined region around the boundary including the boundary, the brightness in the small window is made the same as the brightness of the corresponding partial image of the screen. After the image displayed on the screen is viewed as it is, and the light-emitting means moves to the first area after passing through the boundary between the second area and the first area, the light-emitting means is in the small window. Before hue The color is changed to a different hue from that of the second area, and the light emitting means sharply increases the brightness as the distance from the boundary between the second area and the first area approaches the screen, so that the hue in the small window becomes clear. In such a way as to appeal to the vision of the instruction input person.
[0028]
As a result, the attributes (hue, brightness, saturation, etc.) of the small window change according to the z-coordinate value of the light emitting means, but the state in which the image on the screen is visible in the small window is maintained.
[0029]
Specifically, when the light emitting means moves from the third area toward the second area, the brightness of the small window is continuously and rapidly increased in accordance with the movement of the light emitting means. Thereby, the instruction input person can recognize that the light emitting means is heading to the second area.
[0030]
Further, the execution means keeps the value of the lightness constant when the lightness reaches a predetermined value due to the movement of the light emitting means. Thus, the instruction input person can recognize that the user is moving in the second area.
[0031]
Further, when the light emitting means moves and exceeds the predetermined z-coordinate value, the executing means decreases the brightness again as the light approaches the boundary between the second area and the first area. Thus, the instruction input person can recognize that the light emitting unit is approaching the first area.
[0032]
Further, the execution means sets the brightness in the small window to be the same as the brightness of the corresponding partial image of the screen when the light emitting means is in a predetermined area around the boundary including the second area and the first area. The image displayed on the screen as it is. Thereby, the instruction input person can recognize that the light emitting means is located near the boundary.
[0033]
Further, the execution means changes the hue in the small window to a hue different from that of the second area after the light emitting means moves to the first area after passing through the boundary between the second area and the first area. . Thereby, the instruction input person can recognize that the light emitting means has moved from the second area to the first area.
[0034]
Further, the executing means sharply increases the brightness as the light emitting means moves away from the boundary between the second area and the first area and approaches the screen, so that the hue in the small window becomes clear. Thus, the instruction input person can recognize that the light emitting means is moving in the first area.
[0035]
The executing means of the present invention can change the unevenness state of the image displayed in the small window according to the z-coordinate value of the light emitting means. Further, the execution means of the present invention can change the size and / or shape of the small window according to the z-coordinate value of the light emitting means.
[0036]
If the size of the small window is at least locally minimum when exceeding the boundary area, and the area of the same attribute is not changed when the light emitting means such as the LED is moving, the distance can be further increased. The feeling becomes easier to recognize.
[0037]
Thus, the instruction input person can recognize the position of the light emitting unit in the z-axis direction.
[0038]
In the present invention, the color of the outline of the small window can be arbitrarily set. The execution means calculates an exclusive OR of the color of the arbitrarily set outline of the small window and the color of the border area of the small window in the image displayed on the screen, and The color of the outline can also be determined.
[0039]
Thereby, even when the color of the outline of the set small window is the same as the color of the border area of the small window in the image displayed on the screen, the outline is displayed in a color different from that color. Therefore, it is possible to prevent the instruction input person from losing sight of the small window.
[0040]
In addition, not only the feedback by the small window but also the execution means of the present invention, as the feedback, appeals to the hearing of the instruction input person using sound when the light emitting means passes through each of the boundaries. You can also.
[0041]
Thus, similarly to the feedback by the small window, the instruction input person can recognize the position of the light emitting unit in the z-axis direction.
[0042]
The execution means of the present invention executes a selection function in a two-dimensional mouse when the light emitting means passes a boundary between the first area and the second area from the second area side. Can be. Also, the execution means has a function of selecting with a two-dimensional mouse when the light emitting means has once passed in one direction within a predetermined time once across a boundary between the first area and the second area. It may be executed. The executing means executes a selection function in a two-dimensional mouse when the light emitting means has once passed in one direction within another predetermined time within another boundary defined in the second area. You may do so.
[0043]
The execution means of the present invention, when the light-emitting means passes once in one direction within a predetermined time once the boundary between the first area and the second area, when passing in the opposite direction, A click function on a two-dimensional mouse can be performed.
[0044]
Further, the execution means of the present invention, when the light-emitting means has passed the boundary between the first area and the second area twice in the same direction within a predetermined time, a double-click with a two-dimensional mouse Function can be performed.
[0045]
Further, in the present invention, the first area is divided into an area close to the screen and an area close to the second area, and the execution means sets the boundary between the two areas by the light emitting means. When passing in one direction, the function of dragging in the two-dimensional mouse can be executed, and when passing in the opposite direction, the function of dropping in the two-dimensional mouse can be executed.
[0046]
As described above, the function of the two-dimensional mouse, such as selection, click, double-click, and drag and drop, is executed according to the way the light emitting means passes through each boundary, so that a mechanical mechanism is not used. .
[0047]
Further, the execution means of the present invention is configured such that, when the light-emitting means passes through a predetermined area near the first area in the second area, the instruction input by the instruction input person is x. The position on the -y plane may be maintained at the position on the xy plane of the light emitting means at the time of entering the predetermined area.
[0048]
Thus, when the vehicle passes through a predetermined area of the second area that is close to the first area, the position on the xy plane where the instruction is input by the instruction input person is fixed. 2D mouse functions such as selecting an object such as an icon or clicking on an object.
[0049]
In the present invention, the predetermined time may be changeable. Thereby, the usability of the instruction input person can be improved.
[0050]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0051]
FIG. 1 is a block diagram showing a configuration and a data flow of a three-dimensional instruction input device according to an embodiment of the present invention. As shown in the figure, the three-dimensional instruction input device includes a light-emitting body (LED) 10 mounted on a user's finger or hand, and 3D measurement for measuring a three-dimensional position of the LED 10 by receiving light from the LED 10. A click, double-click, drag-and-drop command (referred to as a mouse command) is generated from the locus of the z-coordinate value among the coordinate values indicating the three-dimensional position measured by the device 12 and the 3D measuring device 12. And a 3D mouse command generation device 14 for outputting to the personal computer 22 along with the xy coordinate values. Further, the personal computer 22 includes a mouse driver 16 that inputs a mouse command together with xy coordinate values from the 3D mouse command generation device 14, an image driver 20 that performs image processing, and a command input to the mouse driver 16. An OS 18 that performs a process related to an instruction input, for example, by activating the image driver 20 as needed, is stored.
[0052]
Here, the xy coordinate values are coordinate values when a plane substantially coincident with the screen of the personal computer 22 (see FIG. 2) is an xy plane, and the z coordinate values are perpendicular to the xy plane. These are coordinate values in the z-axis direction.
[0053]
Note that the hardware of the above-described three-dimensional instruction input device is configured using the position detection technology described in Japanese Patent Application Laid-Open No. 10-9812. Of course, any device capable of three-dimensional position measurement can be used.
[0054]
FIG. 2 is a diagram of the personal computer 22 as viewed from the side. The screen of the personal computer 22 is indicated by M. In the three-dimensional space extending in front of the screen M of the personal computer 22, the layer closest to the screen M is referred to as a fully-pressed lower layer, and the next closest layer is referred to as a fully-pressed upper layer. The boundary between the two is called boundary B. Further, the two layers on the near side are called a half-pressed layer (second layer), and are called a half-pressed lower layer and a half-pressed upper layer, respectively, from the side closer to the screen M. The boundary between the two is called boundary A. The boundary between the half-pressed layer and the fully-pressed layer is called a boundary K. The layer closest to the user, that is, the layer farthest from the screen M, is called a free layer (third layer). The boundary between the free layer and the half-pressed layer is called a boundary F.
[0055]
The free layer is a layer in which no action is taken simply by moving the cursor in response to the movement of the LED 10. This corresponds to a state where the mouse is moved without clicking the mouse in the two-dimensional mouse.
[0056]
If the LED 10 is on the layer before the half-pressed layer (upper half-pressed layer), the attribute of the extended cursor described later changes, and the user is selecting an object. When the LED 10 moves to the screen side of the half-pressed layer (lower half-pressed layer), the xy coordinate value of the cursor is fixed to the value when passing through the boundary A.
[0057]
Mouse commands are generated by the way the LED 10 passes between the half-pressed layer and the fully-pressed layer.
[0058]
Here, the extended cursor will be described. In the present embodiment, unlike the conventional cursor, the extended cursor surrounds a small area including a point indicated by the conventional cursor on the desktop of the personal computer 22 (hereinafter, referred to as a cursor point) with an arc or the like. It is displayed again as a small window. As shown in FIGS. 3A and 3B, the inside of a circle centered on the cursor point 30 is displayed in a small window (extended cursor) 32. The conventional cursor 34 is displayed inside the extended cursor 32, particularly at the center thereof.
[0059]
FIG. 3A shows a state in which the extended cursor 32 points to the center of the two rows of icons on the right side, and it can be seen that the desktop icons are somewhat enlarged.
[0060]
FIG. 3B shows the state of the extended cursor 32 on the screen when the LED 10 approaches the boundary K. As shown in the figure, the radius of the extended cursor 32 is minimized, and the image in the circle is the same as the background, that is, the magnification is 1.
[0061]
Thus, as the LED 10 approaches the boundary K from the outside in the half-pressed layer, the extended cursor 32 changes from the state where the image is enlarged so as to focus on the camera to the actual size, or the radius decreases. This constitutes a focus metaphor reminiscent of the aperture being stopped down.
[0062]
The OS displays the extended cursor 32 using the image driver 20. The conventional cursor 34 is displayed by the OS over the extended cursor 32 using the mouse driver 16.
[0063]
Next, the flow of processing of the three-dimensional instruction input device when a mouse command is generated will be described.
[0064]
As shown in FIG. 1, when the LED 10 is in front of the 3D measuring device 12, the position is measured by the 3D measuring device 12. Here, what is necessary for generating the mouse command is the z-coordinate value of the LED 10. The z-coordinate value is measured by the distance from the 3D measuring device 12. If the position of the screen M of the personal computer 22 matches the position of the 3D measuring device 12, this distance is the distance from the screen M. As shown by the arrow in FIG. 1, the direction extending forward is the positive direction of the z coordinate value.
[0065]
As the user moves the hand, the position of the LED 10 attached to the hand changes accordingly, and the three-dimensional coordinate value including the z-coordinate value is sent to the 3D mouse command generation device 14 through the 3D measurement device 12.
[0066]
The 3D mouse / command generating device 14 examines, from the time series of the z-coordinate value, how each of the boundary areas in front of the screen M has been passed within a certain period of time. This time can be arbitrarily set by the user. The 3D mouse command generation device 14 interprets the three important commands, click, double-click, and drag, depending on the method. The interpreted command signal is sent to the mouse driver 16 of the personal computer 22. At this point, the mouse driver 16 can receive the same data type, whether a two-dimensional mouse or a three-dimensional mouse. Therefore, the processing is performed on the personal computer 22 side thereafter. In this embodiment, since the extended cursor 32 is represented by a small window, the OS activates the image driver 20 or the like that processes the image as needed, and is shown in FIGS. 3A and 3B. The extension cursor 32 is displayed on the screen M of the personal computer 22 as described above. The mouse driver 16 is also activated at the same time, and the OS displays the conventional cursor 34 on the image of the extended cursor 32 on the screen M by the mouse driver 16.
[0067]
The image inside the extended cursor 32 is not different from the original image when the LED 10 is on the free layer, and is enlarged (the size of the extended cursor itself and the image displayed inside) when the LED 10 enters the half-pressed layer. Further, as the LED 10 moves from the half-pressed layer to the fully-pressed layer, that is, toward the boundary K, the magnification of the enlarged image becomes 1. In particular, the magnification is set to 1 even if the z-coordinate value of the LED 10 slightly moves within a predetermined distance just before the boundary K. This is a reproduction of the half-pressed state of the shutter of the camera, and also expresses that the hardness of the shutter is increased immediately before the shutter is lowered, and that there is no change.
[0068]
Hereinafter, the correspondence between the movement of the LED 10 and the generated mouse command will be described in detail with reference to FIG.
[0069]
FIG. 4 is a diagram of the personal computer 22 as viewed from above. In addition, the bold solid line and the dotted line shown in the figure indicate the locus corresponding to the movement of the LED 10. The bold solid line indicates that the xy coordinate value of the cursor point 30 is fixed (locked), and the dotted line portion indicates that the xy coordinate value is not fixed (unlocked). Is shown.
[0070]
When the LED 10 enters the lower half-pressed state from the upper half-pressed state through the boundary A, the xy coordinate value of the cursor point 30 is locked with the value at the time of passing through the boundary A. At this time, if the object is designated, if the LED passes through K later, the object is selected regardless of the xy coordinate value of the passing point at the time of passing. That is, after the LED 10 moves and passes through the boundary K and does not reach the boundary B but passes through the boundary K again in the reverse direction and returns to the upper half layer, a command of “click” is issued by the mouse of the personal computer 22. It is sent to the driver 16 (trajectory 40).
[0071]
Also, after the LED 10 enters the half-pressed lower layer from the upper half-pressed layer via the boundary A, does not return to the upper part of the half-pressed layer, crosses the boundary K once again, but does not reach the boundary B and reverses the boundary K again. When returning to the direction, a command of “double click” is sent to the personal computer 22. (Track 42).
[0072]
When the LED 10 passes the boundary A and the boundary K from the half-pressed upper layer and passes the next boundary B, the LED 10 enters a draggable area (fully pressed lower layer). At this point, the lock of the xy coordinate value of the cursor point 30 is released, and the object pointed to by the cursor point 30 is dragged according to the movement of the xy coordinate of the LED 10 in the lower layer of the full push. That is, a “drag” command is sent to the personal computer 22. When the LED 10 again passes through the boundary B in the reverse direction from the fully-pressed lower layer, and further passes through the boundary K and returns to the half-pressed layer, the xy coordinate values are locked at that point, and the object is dropped. That is, a "drop" command is sent to the personal computer 22 (locus 44).
[0073]
The correspondence between the lock state of the xy coordinate values and each boundary will be described in detail. Basically, the xy coordinate values are locked in any case when the LED 10 crosses the boundary A from the upper half-pressed layer to the lower half-pressed layer. Also, the lock is released when the vehicle has passed through the boundary A in the opposite direction in the case of a click, and at the time of returning to the boundary K for the second time in the case of the double click. Further, when the LED 10 enters the lower layer of the full push beyond the boundary B, the xy coordinates are unlocked. This is because the user has entered the draggable area. After the object is moved by dragging, when the LED 10 returns to the full-press upper layer again, it is locked again. This is a convenience to remove the object from the cursor. When the LED 10 returns to the half-pressed layer beyond the boundary K, the lock is released again. At this point, the object is not selected because the LED 10 has not returned to the upper half-pressed layer.
[0074]
In FIG. 4, the locus of the LED 10 at the time of clicking and double-clicking appears to move in the x or y coordinate values, but these are exaggeratedly expressed, and are actually at almost the same position. In the solid line, the value of the mouse driver 16 of the personal computer 22 is constant.
[0075]
As described above, the space extending in front of the screen is divided into three regions, and the extended cursor (focus metaphor) that appeals to the user's vision indicates that the LED 10 worn by the user has passed the boundary of each region. While recognizing, if you reciprocate the boundary between the half-pushed layer and the fully-pressed layer only once within a certain time, click the two-dimensional mouse, and if you reciprocate twice within a certain time, When the double-click operation of the two-dimensional mouse is performed, and the layer is not returned to the half-pressed layer for a certain period of time even after moving from the half-pressed layer to the fully-pressed layer, an object such as an icon indicated by the cursor is dragged, and the LED 10 When the mouse returns to the half-pressed layer, the dragged object is dropped. Mechanisms can be performed by LED10 movement in the air without using the (three-dimensional coordinate value).
[0076]
In the above-described embodiment, an example has been described in which the user recognizes that the LED 10 has passed through the boundary in the three-dimensional space using the extended cursor 32. Can be generated to assist the operation. Further, the user can freely change the maximum magnification ratio, color information such as hue, and the size (radius and the like) of the extended cursor 32 in each layer according to his / her preference and usability.
[0077]
FIG. 5 is a diagram illustrating an example of a change in brightness and hue of the extended cursor 32 according to the z-coordinate position of the LED 10. As shown, the hue changes from green to red at the boundary K. That is, when the user moves the LED 10 around the boundary K, the color changes, and the user is more aware of where the boundary is. The brightness is changed in the half-pushed layer (the left side of the boundary K in FIG. 5) and the fully-pushed layer (the right side of the boundary K in FIG. 5). In particular, in the half-pushed layer, the enlargement ratio of the image captured by the extended cursor toward the boundary K approaches 1 and the color approaches the hue of the image captured by the extended cursor, so that the focus is clearly focused. It is also possible to configure a metaphor.
[0078]
Also, the boundaries and layers in the above-described embodiment are examples, and the boundaries can be deleted and the two layers can be combined. In this case, it becomes possible by setting the attribute dependent on the layer to the same for the adjacent layer. The user can also freely set the z-coordinate value that defines the boundary. Further, by interpreting the meaning of each layer separately, for example, in an extreme case, it is possible to reverse the half-pushed layer and the fully-pushed layer. In this case as well, the mouse command may be activated by determining which boundary and how the LED 10 has passed.
[0079]
Note that at least one of the hue, saturation, and brightness of the image displayed on the extended cursor may be changed according to the z-coordinate value of the LED 10. In addition, the unevenness state of the image displayed on the extended cursor can be changed according to the z-coordinate value of the LED 10. Further, the size and shape of the extended cursor may be changed according to the z-coordinate value of the LED 10.
[0080]
The user may arbitrarily set the color of the outline of the extended cursor. Furthermore, the color of the outline of the extended cursor may be determined by taking the exclusive OR of the color of the outline of the extended cursor arbitrarily set and the color of the boundary area of the extended cursor in the image displayed on the screen. Good. With this, when the color of the contour set in advance for the extended cursor and the color occupying the boundary area of the contour of the extended cursor in the image displayed on the screen are the same, the color of the contour of the extended cursor is made different. , The extended cursor is clearly visible.
[0081]
In addition, the maximum allowable time until each operation is completed can be set by the user, but the click interval time required for identifying the click operation by the air operation may be longer than the set value of the two-dimensional mouse. preferable. Specifically, about 1.9 seconds of the double click, which exceeds 0.5 seconds that can be set in the basic software OS, was experimentally appropriate at present.
[0082]
【The invention's effect】
The three-dimensional instruction input device according to the present invention provides each of a plurality of divided regions in a three-dimensional space spread over the front of the screen by feedback (change in focus metaphor, hue, etc.) appealing to at least one of the five senses. A function of a two-dimensional mouse, such as clicking, double-clicking, and dragging, in which a user perceives a passing state of the light emitting means with respect to the boundary and according to a way of passing the light emitting means with respect to the boundary of each area without using a mechanical mechanism. Can be executed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration and a data flow of a three-dimensional instruction input device according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating space division when a personal computer is viewed from the side.
FIG. 3A is a diagram showing a state in which an extended cursor points to the center of two columns of icons on the right side as viewed from the front, and FIG. FIG. 7 is a diagram illustrating a state of an extended cursor on a screen when approaching.
FIG. 4 is a diagram illustrating space division and movement of LEDs in a space when the personal computer is viewed from above.
FIG. 5 is a diagram illustrating an example of changes in hue and brightness of an image in an extended cursor displayed on a screen.
[Explanation of symbols]
10 LED
12 3D measurement device
14 3D mouse command generator
16 Mouse Driver
18 OS
20 Image Driver
22 PC
32 Extended cursor

Claims (19)

A light emitting means that can be worn by an instruction input person;
Measuring means for receiving the light emitted from the light emitting means and measuring a three-dimensional position of the light emitting means based on the received position;
On the front of a screen displaying predetermined information, assuming a plurality of spaces divided from the screen in the direction of the instructor, based on a three-dimensional position measured by the measuring means, Executing means for executing a function as a two-dimensional mouse by determining a state of passing through the boundary of the plurality of spaces, and performing a feedback change appealing to at least one of the five senses of the instruction input person according to the determined state;
And a three-dimensional instruction input device. Light-emitting means that can be worn on the hand or finger of the instruction input person;
Measuring means for receiving light emitted from the light emitting means and measuring a three-dimensional position of the light emitting means from the received light;
A plane substantially coincident with the screen displaying the predetermined information is defined as an xy plane, and a space in front of the screen is defined as a first area in the z-axis direction perpendicular to the xy plane in the order from the screen closer to the screen. , The second area, and the third area are divided into three areas, and the light emission for each boundary that divides the three areas from the z-coordinate value of the three-dimensional position of the light emitting means obtained by the measuring means. Judging the way of passing through the means, and selecting, clicking, double-clicking, dragging and dropping, depending on the way of passing the means, and giving feedback giving at least one of the five senses of the instruction inputting person. Execution means for executing a function of a two-dimensional mouse including:
And a three-dimensional instruction input device. The execution means causes the small light window to be displayed at a position on the xy plane of the light emitting means on the screen as the feedback, and within the small window, among the images displayed on the screen, the light emitting means By enlarging and displaying an image of a predetermined area including the position on the xy plane, and changing the magnification of the image displayed in the small window according to the z-coordinate value of the light emitting means, 3. The three-dimensional instruction input device according to claim 2, which appeals to an instruction input person's vision. When the light emitting unit approaches a boundary between the first region and the second region from the second region side, the execution unit reduces the enlargement ratio as approaching the boundary. 4. The three-dimensional instruction input device according to claim 3, wherein the enlargement factor is set to 1 when passing through the boundary so that the instruction input person can recognize the boundary. 5. The execution unit according to claim 3, wherein at least one of hue, saturation, and brightness of an image displayed in the small window is changed according to az coordinate value of the light emitting unit. Three-dimensional instruction input device. The execution means, when the light emitting means moves from the third area toward the second area, continuously and rapidly increases the brightness in the small window according to the movement of the light emitting means; When the brightness reaches a predetermined value due to the movement of the light emitting means, the value is maintained constant, and when the light emitting means moves and exceeds a predetermined z coordinate value, the second area and the first The brightness is reduced again as approaching the border with the area, and in a predetermined area around the border including the border, the brightness in the small window is displayed on the screen in the same manner as the brightness of the corresponding partial image of the screen. After the light emitting means passes through the boundary between the second area and the first area and moves to the first area, the hue in the small window is changed to the second area. What is area 2 Hues, and the light emitting means rapidly increases the brightness as the distance from the boundary between the second area and the first area approaches the screen, so that the hues in the small window become clear. 6. The three-dimensional instruction input device according to claim 5, which appeals to an instruction input person's vision. 7. The three-dimensional instruction input device according to claim 3, wherein the execution unit changes an uneven state of an image displayed in the small window according to a z-coordinate value of the light emitting unit. 8. . The three-dimensional instruction input device according to any one of claims 3 to 7, wherein the execution unit changes a size and / or a shape of the small window according to a z-coordinate value of the light emitting unit. 9. The three-dimensional instruction input device according to claim 3, wherein an outline color of the small window can be arbitrarily set. The execution means calculates an exclusive OR of the color of the arbitrarily set outline of the small window and the color of the boundary area of the small window in the image displayed on the screen, and obtains the outline of the small window. The three-dimensional instruction input device according to claim 9, wherein the three-dimensional instruction input device determines a color of the three-dimensional image. The three-dimensional apparatus according to any one of claims 2 to 10, wherein the execution means appeals to the hearing of the instruction input person using sound as the feedback when the light emitting means passes through each of the boundaries. Instruction input device. 3. The execution unit executes a selection function in a two-dimensional mouse when the light emitting unit passes a boundary between the first region and the second region from the second region side. 4. The three-dimensional instruction input device according to claim 11. The executing means executes a selection function in a two-dimensional mouse when the light emitting means has once passed in one direction within another predetermined time within another boundary defined in the second area. The three-dimensional instruction input device according to any one of claims 2 to 11. The executing means captures the second area by dividing the second area into an area close to the first area and an area close to the third area, and the light emitting means sets a boundary between the two areas for a predetermined time. The three-dimensional instruction input device according to any one of claims 2 to 11, wherein a function of selecting with a two-dimensional mouse is executed when the user has passed through the mouse once in one direction. The execution means may include a two-dimensional mouse when the light emitting means passes through the boundary between the first area and the second area once in one direction within a predetermined time and then in the opposite direction. The three-dimensional instruction input device according to any one of claims 2 to 11, which performs a click function in. The execution means executes a double-click function of a two-dimensional mouse when the light emitting means has passed the boundary between the first area and the second area twice in the same direction within a predetermined time. The three-dimensional instruction input device according to any one of claims 2 to 11. The first area is divided into an area near the screen and an area near the second area,
The execution means executes a drag function of the two-dimensional mouse when the light emitting means passes through the boundary between the two regions in one direction, and executes the drag function of the two-dimensional mouse when the light emitting means passes the boundary in the opposite direction. The three-dimensional instruction input device according to any one of claims 12 to 14, which executes a drop function. The execution means, when the light-emitting means passes through a predetermined area near the first area of the second area, on the xy plane instructed by the instructor. The three-dimensional instruction input device according to any one of claims 2 to 17, wherein the position is maintained at the position on the xy plane of the light emitting unit at the time of entering the predetermined area. The three-dimensional instruction input device according to any one of claims 12 to 18, wherein the predetermined time is changeable.

Images