JPH0728591A - Space manipulation mouse system and space operation pattern input method - Google Patents

Abstract

PURPOSE:To recognize the space operation pattern of an operator by providing a space manipulation mouse with a motion signal generation means which detects the motion of the space manipulation mouse from the acceleration or the angular velocity in the two-dimensional direction or three-dimensional direction. CONSTITUTION:The operator moves a space manipulation mouse 1 on the virtual plane without using the reference surface as desks and grid reflection boards. The motion of the mouse 1 can be detected by being resolved into the horizontal and virtical directions based on the work of a horizontal motion detection element 3 and a virtical motion detection element 3 which are constructed by using piezoelectric elements. The detected two-directional motions, for example, the acceleration and angular velocity are converted into the space manipulation mouse or the signal representing the movement distance by motion detection parts 16a and 16b at the output. A transmission part 18 performs the required processing on the two motion signals and drives infrared ray emitting elements 5 and 6 to send it to the equipments to be controlled.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an input device used in a computer, a multimedia device or the like, and particularly to a three-dimensional input such as a space operation mouse for providing a man-machine interface environment with good operability. Regarding the device. Further, the present invention relates to a spatial motion pattern input system that uses a three-dimensional input device to realize an expanded input function according to a motion pattern of an operator.

[0002]

2. Description of the Related Art In recent years, a pointing device called a mouse has been widely used as an input device on a man-machine interface used in a computer and various systems using the computer.

In the system using this mouse, first,
The operator moves the mouse on a desk or the like to move a cursor that is interlocked with the mouse displayed on the display screen to display a desired object (image,
Move it up). Then, while the cursor is positioned on the object to be selected on the display screen, a confirmation switch called a click button of the mouse is pressed (or released) to make an input to the system. In this way, the mouse provides good operability not found in the keyboard.

However, the conventional mouse requires a large amount of operability because it is necessary to move the mouse by bringing it into contact with an operation surface such as a dedicated operation plate, and in addition, since there is a connection cable between the computer and the mouse. Was hindering me. That is, the operator cannot perform a pointing operation by freely setting the distance between the mouse and the control target device such as a computer in an arbitrary space or an arbitrary virtual plane.

Further, since the movement of the mouse is restricted by the above-mentioned operation surface, the mouse can detect only the movement on the plane, and cannot detect the spatial movement. Therefore, it is difficult to perform the pointing operation that reflects the spatial movement of the mouse.

In consideration of such a situation, recently, by capturing the three-dimensional movement of the operator, it is possible to perform a pointer operation on an arbitrary plane, and further a spatial pointer operation, so that a computer or a multimedia can be operated. A three-dimensional input device (Japanese Patent Laid-Open No. 3-192423), such as a spatial operation mouse, has emerged which enables a pointer operation easily even when the device or its display device is remote.

However, in the system using the above-mentioned three-dimensional input device, "When the operator moves the three-dimensional input device in space, a cursor that is interlocked with the three-dimensional input device is displayed on the display screen. A pointing operation such as "confirmation or selection is performed by moving the object to a desired object, and then pressing (or releasing) the click button" is mainly provided.

On the other hand, there has been proposed a system which can execute a function associated with each pattern by moving a conventional mouse on an operation surface to draw a predetermined pattern (Japanese Patent Laid-Open No. 4-7726). Japanese Patent Laid-Open No. 4-1
80119).

However, in these systems, since the mouse needs to be moved on the operation surface, there is a restriction that only simple patterns can be used. Also, it is unexpectedly difficult to draw a predetermined pattern neatly on the operation surface,
The pattern matching method has a problem in that the pattern drawn by the operator cannot be recognized.

[0010]

As described above, in the conventional system using the mouse, it is possible to detect the three-dimensional movement, and only the improvement such that the dedicated operation plate is unnecessary.

As a mouse to which a very simple pattern input function is added in addition to the function as a pointing device, a very limited pattern input function is given to a (two-dimensional) mouse requiring a dedicated operation plate or the like. It was still there.

The present invention has been made in consideration of the above circumstances, and provides a three-dimensional input device such as a spatial operation mouse that enables a pointer operation on an arbitrary virtual plane and further a spatial pointer operation. In the input system to be used, it is possible to recognize the spatial motion pattern of the operator's space and execute the control of the contents according to the spatial motion pattern to the computer or multimedia device, that is, a sensible man-machine interface environment. An object is to provide a spatial motion pattern input system that can be provided.

[0013]

In the space operation mouse system according to the present invention, at least one of the speed or acceleration in the predetermined three axial directions in the operation space or the angular speed or the angular acceleration around the three axes. And a motion signal generating means for converting the detected one or more quantities as they are or as related quantities to output as a motion signal, and the motion signal given by this motion signal generating means. A spatial operation mouse having a transmitting means for transmitting the control signal, a receiving means for receiving the control signal transmitted by the transmitting means of the spatial operating mouse, and a control obtained by the receiving means Based on the signal, the cursor or the display object on the display screen is changed to display the position indicated by the operator of the spatial operation mouse. And characterized by being provided with a control target device and a display means for changing a display screen in accordance with the motion signal.

Further, in the spatial motion pattern input method using the spatial manipulation mouse according to the present invention, the linear, planar or spatial motion in the three-dimensional space given by the operator is converted into the predetermined three motions in the manipulation space. A detection step of detecting at least two amounts of velocity or acceleration in one axial direction or angular velocity or angular acceleration around three axes, and an action in which the detected at least two detected amounts are patterned. A conversion step of converting to pattern data, the operation pattern data,
A comparison step of comparing with basic data related to a plurality of basic operation patterns registered in advance, an identification step of identifying the operation pattern based on the comparison result in the comparison step, and an identification result in the identification step And a control step of performing a predetermined control corresponding to the operation pattern.

On the other hand, according to the present invention, in the spatial operation pattern input method for causing the controlled device to execute the control of the content according to the spatial operation pattern of the spatial operation type input device,
From a motion detection step of detecting a motion amount in at least two axis directions out of motion amounts in predetermined three axis directions which are not parallel to each other in the space of the space operation type input device, and from at least two detection amounts detected in the motion detection step. A conversion step of converting the spatial motion amount into a motion vector sequence, an identification step of performing identification by comparing the motion vector sequence corresponding to a pre-registered basic motion pattern with the motion vector sequence, and the identification step. And an execution step of executing control based on the recognition result in step 1) on the device to be controlled.

Also preferably, the identifying step comprises
Generate at least one of a unit vector function based on a motion vector sequence obtained from the motion amount of the spatial operation type input device and a cumulative vector function based on a motion vector sequence obtained from the motion amount of the spatial operation type input device The vector operation generating step for performing at least one of the generated unit vector function and the cumulative vector function is compared with that obtained from the motion vector sequence corresponding to the basic motion pattern motion, and the spatial operation type input device is based on the comparison result. And an identifying step of identifying the spatial motion pattern of.

Further, preferably, the conversion step is
For the conversion table in which the number of unit vectors and the angle are registered for the spatial motion amount, a value obtained by temporally sampling the spatial motion amount including at least two detection amounts detected in the motion detection step is sequentially specified. It is characterized in that the result of converting the spatial motion amount into a motion vector sequence is obtained by giving it as a value.

Further, preferably, in the operation detecting step, a rotational operation amount for detecting a rotational operation amount about an axis in a direction orthogonal to each of two axial directions for detecting an operation amount of the spatial operation type input device. The method further comprises a detection step, and the conversion step is based on the detection result of the rotational movement amount detection step and is based on the movement amount in the two axial directions of the related spatial operation type input device detected in the movement detection step. It is characterized by further comprising a correction step of extracting a motion amount excluding a rotation motion amount about the axis.

[0019] Preferably, the execution step is
The shape of the spatial motion pattern recognized in the identifying step is displayed on a display screen.

[0020]

In the present invention, the spatial operation mouse is provided with the motion signal generating means for detecting the movement of the spatial operation mouse from the acceleration or the angular velocity in the two-dimensional direction or the three-dimensional direction. For this reason, it is possible to detect the operator's intended planar movement or spatial movement in an arbitrary operating space, and therefore, the large-screen image can be viewed by the spatial operation mouse as a computer pointer or from a distance. It can be used as a remote control device for systems and multimedia equipment to easily perform pointer operations and control operations.

Therefore, in the space operation mouse system using the space operation mouse of the present invention, the position pointed by the operator is displayed or the direction of the display screen is displayed by the change of the cursor or the display object on the display screen of the controlled device. By changing the, it is possible to realize a human interface environment with improved operability that has never existed before.

Further, in the spatial motion pattern input method according to the present invention, the basic motion pattern of the operator is registered in advance, and the motion pattern input by operating the spatial operation mouse is identified to correspond to each motion pattern. To execute the control.
Therefore, it is possible to control the control target device by using a natural human motion.

On the other hand, in the spatial motion pattern input method according to the present invention, the basic motion pattern of the operator is registered in advance, and the operator operates the spatial operation mouse (spatial operation type input device) to input it. The motion of the spatial motion pattern is detected and converted into a motion vector sequence, and the motion vector sequence corresponding to the pre-registered basic motion pattern is compared with the motion vector sequence for identification, and based on the recognition result. Executes control for controlled devices.

As described above, the movement of the spatial operation mouse is relatively obtained as a time-series small reference vector set by using the motion vector without measuring the spatial coordinates of the spatial operation mouse with respect to the reference position (origin). Can input an operation pattern. Further, in the present invention, instead of the conventional pattern matching method, the motion is decomposed into a motion vector sequence and compared with the basic motion pattern, so that the motion pattern obtained from an unstable mouse operation in space is recognized with high accuracy. It becomes possible to do.

Therefore, according to the present invention, it becomes possible to control the device to be controlled by using a natural human motion.

[0026]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic view of a spatial operation mouse according to the first embodiment of the present invention, and FIG. 2 is a schematic block diagram of the spatial operation mouse. Here, in this specification, the horizontal direction means the left and right direction as seen from the spatial operation mouse as indicated by the arrow 28 in FIG. 1, and the vertical direction means the up and down direction as seen from the mouse as indicated by the arrow 29. To do.

The spatial operation mouse of the present invention has a first movement detecting section 16a for detecting a horizontal movement, a second movement detecting section 16b for detecting a vertical movement, and a click operation by the operator. Switch unit 17 for switching
Output signal from the motion detector 16a, second motion detector 16
The transmitter 18 is provided for transmitting the output signal from the switch b and the output signal from the switch 17 to the control target device.

The first motion detector 16a includes a horizontal motion detecting element 2 for detecting a horizontal motion. The second motion detector 16b includes a vertical motion detector 3 that detects a vertical motion. The switch unit 17 includes the click button 4. The transmitter 18 includes infrared light emitting elements 5 and 6.

The operator holds the space operation mouse body 1 and
In any space, the space operation mouse body 1 is moved vertically and horizontally to operate. That is, the operator performs the moving operation of the spatial operation mouse body 1 on a virtual plane without using an existing reference surface such as a desk or a grating reflector.
The movement of the spatial operation mouse body 1 is changed to two directions of horizontal (left and right) and vertical (up and down) by the action of the horizontal direction motion detection element 2 and the vertical direction motion detection element 3 which are configured by using a piezoelectric element or the like. Disassembled and detected. The detected two-direction movements, such as acceleration and angular velocity, are detected by the movement detecting unit 1.
The signals 6a and 16b are converted into a predetermined motion signal, for example, a signal representing the speed or movement distance of the spatial operation mouse, and output. Alternatively, the acceleration and the angular velocity are output as they are. The transmission unit 18 performs necessary processing, such as format conversion, encoding, multiplexing, and modulation, on these two motion signals. After that, the transmitter 18
The infrared light emitting elements 5 and 6 are driven to transmit this signal to the control target device.

The operator operates the position of the cursor on the display screen by using such a spatial operation mouse, and presses the click button 4 of the switch section 17 before and / or after the operation (or Let go). The transmission unit 18 notifies the control target device that the click button 4 has been pressed (or released). The device to be controlled, which has received the motion signal and the signal indicating the click operation, executes a predetermined control operation according to the signal.

It is preferable that the light emitting element 5 emits infrared rays to the right space and the light emitting element 6 emits infrared rays to the left space, that is, if different directivities are assigned to both elements, the infrared radiation range becomes wide. . Particularly in the case of the space operation mouse operated in an arbitrary space, even if the operator shakes the space operation mouse to the left or right, infrared rays can be reliably transmitted to the control target device. Of course, the number of the light emitting elements may be one or three or more depending on the application.

In the embodiment shown in FIG. 1, the spatial operation mouse is provided with a motion detecting element for detecting the movement in the two axial directions of the vertical and horizontal directions. Instead, two motion detection elements that detect movement in the direction orthogonal to the vertical and horizontal directions (that is, the front-back direction) and the vertical direction, or two motion detection elements that detect movement in the front-back direction and the horizontal direction. May be provided in the space operation mouse. Also, the directions of the two axes to detect are
Unless it is parallel, it does not need to go straight. This also applies to the examples described later.

FIG. 3 is a conceptual diagram of the use of the spatial operation type computer system configured by applying the spatial operation mouse of this embodiment to the laptop computer 21.

According to the present invention, no special working environment or a restricted working environment for operating the mouse is required. That is, the operator may hold the space operation mouse body 1 and move it in any space (that is, in the air in front of the operator's eyes) up, down, left or right. This motion is caused by the horizontal motion detecting element 2 and the vertical motion detecting element 3 as described above.
Detected by. Then, a control signal corresponding to the detected movement is radiated from the infrared light emitting elements 5 and 6 to the laptop computer 21 and is received by the infrared light receiving element 23. Further, the control signal indicating the click operation is similarly transmitted from the spatial operation mouse to the laptop computer 21.

The laptop computer 21 obtains the amount of movement of the spatial operation mouse in accordance with the received control signal or by a predetermined arithmetic processing based on the control signal,
The control for moving the cursor on the display screen 22 is performed. If necessary, the display object at the cursor position on the display screen is subjected to a predetermined process, or a new input screen corresponding to the cursor position is displayed. Furthermore, in response to the click operation, it is possible to execute control such as performing a predetermined process associated with the display object (or the character string) at the cursor position in advance.

As described above, the operator uses the spatial operation mouse to move the cursor or the like displayed on the display screen 22 and executes the click operation to sequentially perform the system control.

A specific example of the input operation using the spatial operation mouse of the present invention will be briefly described below using some examples of display screens.

4 (a) and 4 (b) show an example in which the operation of selecting the F icon from the nine icons A to I is executed. First, as shown in FIG. 4A, it is assumed that there is a + -marked cursor in the portion D on the screen. When the operator moves the spatial operation mouse to the right, the cursor also moves to the right accordingly. Then, as shown in FIG. 4B, when the cursor moves to F, the operator presses the click button 4 of the spatial operation mouse to select the F icon for the laptop computer 21. Tell the thing. When the F icon is selected, the laptop computer 21 executes the process corresponding to the F icon.

5 (a) and 5 (b) are images showing the inside of the room. Here, an example in which an operation of selecting one object from this video is executed will be described. The portion indicated by the arrow icon on the display screen is changed into a mesh shape. For example, before the operation, if the arrow icon is at the door as shown in FIG. 5A, only the door is changed into a mesh shape. Next, when the operator moves the spatial operation mouse to the right and moves the icon to the word processor on the desk, only the word processor is changed into a mesh shape as shown in FIG. 5 (b). . From this mesh display, it can be recognized at a glance that the word processor is instructed. If the operator wants to select this word processor, he / she can press the click button 4 while the mesh is changing.

6 (a) and 6 (b) are images showing the corridor of the building. Here, an example will be described in which an operation for converting an image into a direction desired by an operator and displaying the image is executed.
For example, before the operation, as shown in FIG. 6 (a), an image viewed straight ahead in parallel with the passage is displayed. When the operator moves the space operation mouse held in his / her hand to the right, the displayed image gradually changes to the image looking to the right, in conjunction with it. Then, finally, as shown in FIG. 6B, the image changes to a front view of the right door.

Next, a configuration example in which the spatial operation mouse of the present invention is more concretely described.

FIG. 7A is a more specific block diagram of the spatial operation mouse of this embodiment. In addition, FIG.
FIG. 3 is a schematic block diagram showing an example of a controlled device.

This spatial operation mouse has a first motion sensor 30a for detecting vertical acceleration, an amplifier 31a for amplifying this output, and a speed detecting section 32a for integrating this output to obtain a speed. , A second motion sensor 30b for detecting horizontal acceleration, an amplifier 31b for amplifying this output, a speed detection unit 32b for integrating this output to obtain a speed, and the speed detection units 32a, 3a.
Infrared remote control transmission circuit 3 for transmitting the output of 2b
3 and an infrared light emitting element 34 driven by the transmitting circuit 33.

Here, piezoelectric elements are used as the first motion sensor 30a and the second motion sensor 30b. In addition,
A switch unit including a click button as shown in FIG. 2 is omitted in the drawing.

On the other hand, the device to be controlled, for example, a computer, includes an infrared light receiving element 35 for receiving the optical signal emitted from the infrared light emitting element 34 and an infrared remote control receiving circuit for converting the received signal into a predetermined form and outputting it. 36, a processing unit 37 that executes a predetermined process including screen control in response to the output, and a display screen 38.

In these configurations, the operator holds the main body of the space operation mouse and operates it by moving it in a two-dimensional direction such as up, down, left and right in a virtual plane of an arbitrary space. The movement of the spatial operation mouse is performed by the first movement sensor 30.
By the action of the a and the second motion sensor 30b, the acceleration is decomposed into, for example, two directions of horizontal and vertical directions and detected, and a voltage signal proportional to each acceleration is output.

The voltage signal corresponding to the detected acceleration in the two directions is a relatively weak signal. Therefore, the two voltage signals are amplified by the amplifiers 31a and 31b, respectively. At this time, if necessary, noise removal processing is performed.

Next, the two amplified signals are given to the speed detecting sections 32a and 32b, respectively. Here, 2
The voltage signals corresponding to the accelerations in the directions are converted into voltage signals corresponding to the moving speeds in the two directions by integration. FIG. 8A is a diagram showing a relationship between the voltage signal corresponding to the detected mouse acceleration and the voltage signal converted into the velocity. The broken line l1 indicates the acceleration sensor 30a or 3
It represents the output voltage from 0b. The solid line 12 indicates the speed detection unit 3
The output voltage converted into speed by the integrating circuit included in 2a or 32b is shown.

From the speed detectors 32a and 32b, the output voltage 12 converted into speed may be output as it is. Alternatively, as shown in FIG. 8B, the pulse signal may be converted into a pulse signal having a pulse density corresponding to the velocity and then output.

Further, the motion calculating section for detecting the velocity from the acceleration can be realized by using a digital integrating circuit or calculation by a microprocessor in addition to the integrating circuit.

Next, the infrared remote control transmission circuit 33, which has received each output of the speed detection sections 32a and 32b and the signal indicating the click operation from the switch section, responds to these signals.
Arbitrary processing such as format conversion, encoding, signal multiplexing, and modulation is performed. Then, the infrared light emitting element 34 is driven to transmit this signal to the control target device.

The signal corresponding to the moving speed of the spatial operation mouse and the signal indicating the click operation transmitted from the spatial operation mouse of FIG. 7A are the infrared light receiving element 35 of the control target device of FIG. 7B. Is received by. Then, the infrared remote controller receiving circuit 36
Thus, the process of returning to the predetermined control signal form is performed.

Upon receipt of this control signal, the processing unit 37 obtains the amount of movement of the spatial operation mouse by a predetermined arithmetic processing, and represents the amount of movement of the spatial operation mouse, the moving speed, their direction, and the click operation. You can get all the signals. The processing unit 37 controls the cursor on the display screen 38 in response to these signals. Depending on the case, a predetermined process, for example, a process of adding a color or a pattern to the display object at the cursor position after the movement is performed. Alternatively, control is performed to display a new input screen corresponding to the cursor position, such as an input screen in a layer below the current input screen. Furthermore, in response to the click operation, the process corresponding to the display object (or the character string) at the cursor position, for example, the display object itself can be processed. Alternatively, it is possible to execute control such as executing a command (including control other than screen control) indicated by the character string or the like.

Here, the speed detectors 32a, 32b
Is not provided on the side of the space operation mouse of FIG.
You may provide in the control target apparatus side of (b). Also, FIG.
It may be configured such that the space operation mouse side of (a) calculates up to the movement amount of the space operation mouse, and the calculated movement amount is given to the control target device side of FIG. 7 (b).

Next, an example in which a piezoelectric vibrating gyro is used as the motion detecting element will be described.

FIG. 9A shows an example of the structure of the piezoelectric vibrating gyro. In this piezoelectric vibrating gyro, a regular triangular prism constant elastic metal 91 is used as a vibrator. Longitudinal wave mode piezoelectric ceramics 92 to 94 are arranged on each surface of the constant elastic metal 91. As shown in FIG. 9B, an exciting power supply 96 is used to excite the vibrator 91 in the exciting piezoelectric ceramic 92. In this case, when stationary, the detection piezoelectric ceramic 9
An equal voltage is generated at 3,94. On the other hand, when the piezoelectric vibrating gyro rotates about the rotary shaft 95, a Coriolis force proportional to the angular velocity is generated in the direction of 90 degrees as viewed from the vibrating direction. As a result, a difference occurs between the voltage generated in the detection piezoelectric ceramic 93 and the voltage generated in the detection piezoelectric ceramic 94. Therefore, the angular velocity can be obtained by calculating the difference between the output voltages of the detection piezoelectric ceramics 93 and 94 using the subtraction device 97.
If the characteristics of the two detection piezoelectric ceramics 93 and 94 are the same, only the voltage signal proportional to the angular velocity appears at the output terminal 98.

If this piezoelectric vibration gyro is further applied to the spatial operation mouse of the present invention, not only the parallel movement but also the rotation around the axis can be detected. This application provides a wide range of applications in terms of the operability of the spatial manipulation mouse.

Even if the vibrator 91 does not rotate, if the body of the spatial operation mouse moves in a direction of 90 degrees with respect to the vibration direction of the exciting piezoelectric ceramic, the detecting piezoelectric ceramic 9
At 3, 94, a voltage difference proportional to the moving speed is generated. This can be used to detect the speed of the mouse body.

FIG. 10 shows an example of the structure of a spatial operation mouse provided with a camera shake correction circuit for stabilizing the pointing operation. The configuration of this space operation mouse is basically
The configuration is the same as that of the spatial operation mouse shown in FIG. However, the band limiter 39 is provided after the amplifiers 31a and 31b.
There is a difference in that a and 39b are provided respectively.

That is, since the operator manipulates the space operation mouse in his / her hand, the space operation mouse body inevitably vibrates slightly due to the hand shake of the operator. As a result, the pointing operation may not be performed accurately. Therefore, the band limiters 39a and 39b are used to remove the vibration component due to camera shake.

Normally, the frequency of camera shake is 0.5 Hz to 1
It is considered to be in the frequency range of about 5 Hz. Therefore,
It is effective to set the frequency removal range of the band limiters 39a and 39b to the above range. This makes it possible to detect the movement of the mouse only with respect to the motion intended by the operator, so that it is possible to avoid erroneous input that the operator does not intend. Therefore, the reliability of the space operation mouse can be improved.

As the band limiters 39a and 39b, a band limit filter, an integrating circuit, or an arithmetic circuit may be used, or they can be realized by software processing.

Depending on the application, the removal frequencies of the band limiters 39a and 39b may be set in different ranges.

On the other hand, 0.5 Hz, which is the frequency of camera shake,
There are cases where it is desired to operate the spatial operation mouse with a slow motion corresponding to a frequency range of about 15 Hz. In this case, the spatial operation mouse may be provided with two systems, a path in which the band limiters 39a and 39b are provided and a path in which they are not provided.
And even if the operation corresponds to the range of the removal frequency,
The signal may be output by switching the route depending on whether or not it is recognized as the operation intended by the operator in consideration of the moving distance obtained from the route without the band limiters 39a and 39b.

As described above, the space operation mouse which enables the pointer operation in an arbitrary space and can easily perform the pointer operation and the control operation even if the pointer operation and the control operation are performed away from the computer or the multimedia device or its display device is provided. can do.

FIG. 11 is a schematic configuration diagram of a spatial operation mouse according to the second embodiment of the present invention. Similar to the first embodiment, the horizontal motion detecting element 112 and the vertical motion detecting element 113 detect the two-dimensional motion of the spatial operation mouse body 111, and the resulting motion signal causes the controlled device to operate. Move the cursor on the display screen. The operation is performed by the cursor button 114 and the click button 117.

FIG. 12 is a conceptual diagram of the use of the spatial operation type video system when the spatial operation mouse of the second embodiment of the present invention is applied to a multimedia television. The operator can perform the operation while looking at the input screen displayed on the television screen, instead of using many function buttons as in the conventional button-controlled infrared remote control device.

When the operator moves the space operation mouse main body 111 vertically and horizontally, the movement is detected by the space operation mouse. Then, a control signal corresponding to the movement is generated in the spatial operation mouse and is emitted from the infrared light emitting elements 115 and 116. The emitted infrared rays are received by the infrared light receiving element 123 of the multimedia television main body 121.

When the operator pushes the cursor button 114 of the space operation mouse with the first finger (for example, thumb), the cursor is displayed on the display screen 122. The mouse body 111 is moved so that the cursor is moved onto the object to be clicked. Then, the operator presses the click button 117 with the second finger (for example, the index finger or the middle finger).

An example of the operation of this spatial operation type video system will be described with reference to the screen examples of FIGS. 13 (a) to 13 (d). First, it is assumed that the contents of channel A are displayed on the screen. The state of such a screen is shown in FIG.
Shown in. When the operator wants to display the channel D on the screen, for example, the operator first clicks the cursor button 114. Then, the input screen 124 is displayed as shown in FIG. At this time, on the input screen 124,
The letters A to F representing the channels are displayed. The current channel, A, is surrounded by a square cursor.
The operator moves the spatial operation mouse to move the cursor to D as shown in FIG. 13 (c). Then, the click button 117 is pressed, and then the cursor button 114 is pressed. Then, the channels are switched as shown in FIG. 13D, and the input screen 124 is erased.

Such an operation can be used not only for channel selection but also for various operations such as volume adjustment and color tone adjustment.

As described above, by using the spatial operation mouse of the present invention, unlike the conventional case of using the button control type infrared remote control device having many function buttons, the operator can perform an input operation while watching the television screen. It can be carried out. Therefore, the operator is freed from the burden of memorizing the functions of many buttons and the complicated button operation.
That is, the spatial operation mouse of the present invention can provide an operating environment that is very easy for the operator to use.

Here, with respect to the spatial operation mouse of the first embodiment having one click button and the spatial operation mouse of the second embodiment having two buttons of the cursor button and the click button, various types of click operation Will be described with reference to FIG. It should be noted that in FIG. 14, the operation of “pressing” does not indicate “keep pressing” but indicates “pressing and releasing”.

In the conventional mechanical mouse and optical mouse, the on / off switching of the cursor control is determined by whether or not the mouse body is brought into contact with the operation plate.
Therefore, when the mouse body is located at the edge of the operation plate and the user wants to move the cursor further on the display screen, the operator temporarily lifts the mouse and places the mouse in the movable area of the operation plate. Reinstall and move it again on the operation plate. However, in the spatial operation mouse of the present invention, the on / off switching of the cursor control can be easily instructed by pressing or releasing the button.
Alternatively, it is instructed by operating a button for on / off instruction.

First, in the spatial operation mouse of the second embodiment, a cursor button (A) for enabling cursor control and a click button (B) for confirming operation or selecting operation are separately provided. .

In operation type 1, the cursor button (A)
When you press, the cursor and the required input screen are displayed, and you can control the cursor. The operator moves the displayed cursor and then presses the click button (B). By this operation, confirmation input or selection input is made. Then, when the cursor button (A) is pressed again, the cursor and the input screen are erased and the cursor control becomes impossible.

In operation type 2, the cursor button (A)
While pressing and holding, the cursor and the necessary input screen are displayed and the cursor can be controlled. After the cursor is moved, the click button (B) is pressed for confirmation or selection. Then, when the cursor button (A) is released again, the cursor and the input screen are erased and the cursor control becomes impossible.

Next, the spatial operation mouse of the first embodiment is
One click button shares a cursor button for enabling cursor control and a click button for performing a confirmation operation or a selection operation.

In operation type 3, when the click button is pressed once, the cursor and the necessary input screen are displayed, and the cursor can be controlled. After moving the cursor, the confirmation or selection is performed by pressing the click button a preset number of times (for example, twice). Then, when the click button is pressed once again, the cursor and the input screen are erased and the cursor control becomes impossible.

In operation type 4, while holding down the click button, the cursor and the necessary input screen are displayed,
Cursor control is possible. After moving the cursor,
By releasing the click button, confirmation or selection is made, the cursor and input screen are erased, and cursor control becomes impossible.

The operation types 3 and 4 can also be used in the spatial operation mouse of the second embodiment.

It is preferable to select the optimum operation type for each of these operation types in consideration of the characteristics of the application.

The other operation types include, for example, the following methods. First, the control target device side enables cursor control and, if necessary, informs the operator of that fact. Then, on the side of the spatial operation mouse, the operation of the click button for performing the confirmation operation or the selection operation is performed.

The control target device controls the timing for enabling / disabling the cursor control and the control for the confirmation operation or the selection operation timing, and the space operation mouse side may or may not be provided with a click button. It is also possible to configure so that only the cursor is moved without any operation.

Next, a third embodiment of the present invention will be described.

The space operation mouse of the present invention can be applied not only to control icons and images on the screen using a cursor, but also allows the operator to hold the space operation mouse body and draw a predetermined operation pattern in the space. The function that can convey the intention is added. Therefore, the term "spatial operation mouse" is used not only to mean a mouse having a function as a pointing device, but also to have an expanded meaning including an input function according to a motion pattern.

Basically, a natural motion based on a normal human sense is performed in a three-dimensional space. Therefore, if the operator can recognize the pattern of the three-dimensional spatial motion, it is possible to provide an environment in which the operator can intuitively control the computer and the video equipment. That is, a person who uses sensory / reflexive actions such as physiological actions and habitual actions that are unconsciously performed in daily life.
It can be said that the machine interface is a human interface most suitable for sensory control.

The following are possible physiological and habitual movements used for sensory control. First,
Examples of physiological actions include actions associated with emotional emotions such as emotions and emotions, and actions defined by the structure of the human body. Next 5
One motion is an example of a physiological motion. (I) When you are surprised, your muscles atrophy and your body is shrugging for a moment. (Ii) When you are nervous, your limbs tremble. (Iii) Stand still when you are careful. (Iv) When attacking your opponent, stick your hand out or drop it from above. (V) When a right-handed person flips over the paper, flip it from the lower left to the upper right.

The following four actions are examples of habitual actions. (I) If yes, shake your head vertically. (Ii) When the volume is up, the volume is turned clockwise to operate the volume, and when the volume is down, the volume is turned counterclockwise. (Iii) When parting, raise your hand and shake to the left or right. (Iv) When attracting people, turn your palm up and shake it. As described above, in everyday life, there are many examples in which everybody habitually performs the same motion.

The use of physiological movements or habitual movements as an input pattern is excellent for the operator in that the movements can be instinctively sensed.

However, on a two-dimensional plane, it is difficult to express and recognize the motion. That is, the operation confirmation technique in the three-dimensional space is indispensable. Therefore, even if a motion pattern is input from the conventional two-dimensional mouse operated on the operation plate, it is not possible to recognize a human's sensory motion in the three-dimensional space. On the other hand, in the present invention, since the spatial operation mouse body 1 can be operated in the three-dimensional space, it is possible to realize a sensory control type human interface which cannot be achieved by the conventional two-dimensional mouse.

FIG. 15 is an example of a block diagram of a spatial operation mouse having an input function according to a spatial motion pattern.
This spatial operation mouse includes a first motion sensor 30a, an amplifier 31a, the band limiter 39a and the A / D converter 40 described above.
a, speed detector 32c, second motion sensor 30b, amplifier 31b, band limiter 39b, A / D converter 40 described above
b, a speed detection unit 32d, a motion recognition unit 41, a motion pattern memory 42, an infrared remote control transmission circuit 43, and an infrared light emitting element 34.

The basic structure is almost the same as that of the space operation mouse of FIG. 10, but is provided with a motion recognition section 41 and a motion pattern memory 42 for recognizing a motion pattern drawn in space by the operator. Is different.

First, the function of the pointing device is almost the same as that of FIG. 10, that is, the first motion sensor 30a, the amplifier 31a, and the band limiter 39 described above.
a, A / D converter 40a, speed detector 32c, second motion sensor 30b, amplifier 31b, band limiter 39b, A /
It is realized using the D converter 40b and the speed detection unit 32d.

The A / D converted signals are applied to the speed detectors 32c and 32d, respectively. Speed detector 32
In c and 32d, only the point that digital processing is performed is different, and the operation is the same as that in FIG. Therefore,
The details of the configuration and operation of this part have already been described, and will not be described. The speed detecting unit 32c,
A signal before A / D conversion may be applied to 32d.

Next, the process of recognizing an operation pattern using this spatial operation mouse and the control of the target device by this process will be described.

16 (a), 16 (b) and 16
(C) is an example of the input operation by such an operation pattern. Various operation patterns such as an operation of rotating the space operation mouse as shown in FIG. 16A, an up and down operation as shown in FIG. 16B, or an left and right operation as shown in FIG. 16C. Correspondingly registered control can be performed on a control target device such as a computer.
For example, when the device to be controlled has a built-in speaker, it is possible to perform control such that the volume increases when the spatial operation mouse is turned to the right and decreases when the spatial operation mouse is turned to the left. Alternatively, if the controlled device requests input for confirmation, shake the spatial operation mouse up and down to display "Ye
It is possible to perform various controls, such as "s" being transmitted and "No" being transmitted by shaking to the left or right.

First, the operator grasps the space operation mouse of this embodiment and moves his hand to draw a predetermined basic motion pattern in space. The movement of the space operation mouse is the first
The motion sensors 30a and 30b once decompose the motion into two directions and detect the motion. The signals representing the movements in the respective directions are amplified by the amplifiers 31a and 31b, and the excess components are removed by the band limiters 39a and 39b.
The signals are converted into digital signals by the / D converters 40a and 40b and given to the motion recognition unit 41, respectively.

On the other hand, the operation pattern memory 42 stores basic data corresponding to various predetermined basic operation patterns.

The motion recognition section 41 first converts a signal representing motion in two directions into motion pattern data having the same format as the basic data in order to pattern the motion of the operator in the three-dimensional space. Then, the operation pattern is identified by comparing the operation pattern data with the basic data. Then, the operation code indicating the corresponding basic operation pattern is acquired.

This operation code is transmitted to the controlled device by the infrared remote control transmission circuit 43 and the infrared light emitting element 34. The device to be controlled which has received this executes the control corresponding to the given operation code.

Here, by comparing the motion pattern data obtained by the motion of the operator with the basic data stored in the motion pattern memory 42, even if an attempt is made to identify the motion of the operator, it is difficult to identify the motion. As a result, it may be impossible to identify the motion. Therefore, for example, the similarity between the motion pattern data and the basic data is calculated, and the motion pattern having the basic data with the highest similarity to the measured motion pattern data is determined to be the motion performed by the operator. However, an operation code corresponding to the determined operation pattern may be obtained. Alternatively, the operation pattern may be identified using a well-known method such as neuro or fuzzy.

Further, when a plurality of basic data having a similar degree of similarity to the measured operation pattern data are detected, a plurality of sets of action codes and similarities are transmitted to the control target device side, and the control target device side is transmitted. The device may perform appropriate processing based on a given data set. Alternatively, the control target device side is informed that the operation pattern cannot be specified, and the control target device side is instructed to the operator to execute the input according to the operation pattern again by using the screen display or the synthesized sound. It may be configured.

There are various variations of motion recognition, and complex motions such as "8" and "x" can be recognized. Further, various kinds of data such as the moving speed, the acceleration, the size of the pattern, and the like can be used and combined to enable various types of control. For example,
If the volume increases when the space control mouse is turned to the right, draw a large circle (or at a faster speed) and make a smaller circle (or at a slower speed) when rotated to the right.
It is also possible to set so that the degree of increase in the volume is larger than when the knob is turned to the right.

Further, the basic operation pattern is assigned with the processing content corresponding to the meaning of the above-described normal human habitual operation, or the basic operation pattern is given an impression from the basic operation pattern. It is effective to assign processing contents that do not cause discomfort. By doing so, the operator can memorize the functions given to various basic operation patterns without difficulty, and
Very easy to use when used. As described above, by applying the space operation mouse of the present invention, an excellent man-machine interface environment can be provided.

Here, with respect to the two directions in which the motion should be detected, the motion is handled in the vertical and horizontal directions in this embodiment. Instead, it is possible to detect movements in two directions perpendicular to these directions (that is, the front-back direction) and the vertical direction, or in two directions, the front-back direction and the horizontal direction. Furthermore, it is possible to use a more complicated operation in a three-dimensional space as a basic operation pattern used for inputting an operation pattern by adding a motion detection unit and increasing the number of axes capable of detecting the motion. It is also effective to detect the rotation around the axis due to the action of twisting the wrist and use this as the action pattern or a part of the action pattern.

The two functions of the spatial operation mouse of this embodiment, that is, the so-called pointer function and this operation pattern input function, may be selectively used by the spatial operation mouse or by the controlled device. You may instruct. It is also possible to use the pointer function and the operation pattern input function in combination.

Further, the motion recognition unit 41 and the motion pattern memory 42 may be provided on the device to be controlled so that the spatial operation mouse outputs the data as it is, which is obtained by detecting the motion. good.

In this embodiment, an example has been described in which the operator holds the spatial operation mouse in his hand and operates it. Instead of holding the space operation mouse in the hand and operating it, the space operation mouse may be attached to another part of the operator's body, for example, the foot or the head, and the motion pattern of the attached part may be detected. Further, it may be attached to or incorporated in a device or tool operated by the operator to detect the operation pattern of these devices or the like.

As described above, according to the present invention, it is possible to provide a space operation mouse which enables not only pointer operation in an arbitrary space but also operation of an operator. Further, it is possible to provide a spatial operation mouse that enables easy pointer operation and control operation even if the mouse is away from the computer, the multimedia device, or the display device thereof.

Next, a fourth embodiment of the present invention will be described.

In the present embodiment, similarly to the third embodiment, a system for inputting a spatial motion pattern and executing a desired function using a three-dimensional input device such as a spatial operation mouse. Is. In this embodiment, the operation pattern recognition processing in the third embodiment is further detailed.
That is, as will be described later, the motion pattern in the operator's space is converted into a motion vector sequence which is a set of minute reference vectors, and the motion vector sequence is compared with the operator's basic motion pattern registered in advance. It is characterized in that it recognizes.

Using the spatial motion pattern input system of this embodiment, roughly, the operator can execute the function associated with this basic motion pattern by drawing it in space. . For example, the function shown in FIG. 17 can be realized. FIG. 17 shows an example of the input operation by the spatial motion pattern according to the present invention. As shown in the figure, the operator
By moving the triangles from to b to c in a triangular shape, a triangular item can be selected from the options displayed on the display screen 203 of the display device 202.

Next, the concept of the motion vector used in the present invention will be described. FIG. 18A shows the magnitude (V) and the direction (θ) of the movement of the spatial operation mouse 1 moved by the operator. As shown in FIG. 18B, a certain time (t)
The operation performed by the operator is represented by a set of a multiple of the size (n) of the reference vector and an angle with respect to the reference direction (for example, the horizontal direction). The motion expressed in this way is defined as a motion vector (Vt, θt). FIG.
As shown in (a), this motion vector is obtained by directly sampling the detection amount detected by the spatial operation mouse 1 for a certain period of time, and the velocity (v
x, vy) or acceleration (αx, αy) is a vector obtained by referring to a table with a specified value. The motion vector has its magnitude as the magnitude (n) of the reference vector.
It is expressed in multiples of. Therefore, by using this motion vector, the movement of the spatial operation mouse can be relatively obtained as a time-series set of minute reference vectors without measuring the spatial coordinates of the spatial operation mouse with respect to the reference position (origin). Becomes

Next, the difference between the so-called motion vector generally used for expressing motion and the motion vector used in the present invention will be described. As shown in FIG. 19, in the expression using the motion vector, a certain motion is captured only at the start point and the end point, and the magnitude (V) and the direction (θ) of the motion between the start point and the end point are used. On the other hand, in the expression by the motion vector in the present invention, the start point to the end point of a certain motion is regarded as a set of minute vectors, and the magnitude (Vt) and direction (θt) of each minute vector at a certain time (t) are used. . By this, even in unstable movement in space,
By following the temporal changes in the magnitude or direction of each motion vector, it is possible to relatively capture a certain motion.

FIG. 20 is a block diagram showing an example of the main configuration of the spatial motion pattern input system using the spatial operation mouse of the present invention. The spatial motion pattern input system according to the present exemplary embodiment includes a motion detection unit 204, a conversion unit 205 that performs conversion into a motion vector, a vector table 206 that shows a correspondence between a detection amount and a motion vector, and an identification unit 207.
And the execution unit 208.

The components shown in FIG. 20 can be appropriately arranged in the spatial operation mouse and the controlled devices. However, at least the motion detection unit 204 is mounted on the spatial operation mouse, and the execution unit 208 is mounted on the control target device. For example, the motion detection unit 204 is mounted on the spatial operation mouse, and the conversion unit 205, the vector table 206, the identification unit 207, and the execution unit 208 are mounted on the control target device. Alternatively,
Motion detection unit 204, conversion unit 205, and vector table 2
06 is mounted on the spatial operation mouse, and the identification unit 207 and the execution unit 208 are mounted on the control target device. Besides, various mounting methods are possible.

When the motion detecting unit 204 is mounted on the spatial operation mouse and the other is mounted on the controlled device, the motion detecting unit 20
The output signal of No. 4 is transmitted using the infrared remote control transmission circuit 33 as shown in FIG. 7A, and is received using the infrared remote control reception circuit 36 on the side of the controlled device as shown in FIG. 7A. Even when other mounting methods are adopted, signals can be passed from the spatial operation mouse side to the control target device side by the same method.

That is, the infrared remote controller transmission circuit 33 or the transmission unit 18 performs format conversion, encoding, or signal conversion to a signal at the time when all the processing necessary for inputting an operation pattern, which will be described later, is completed or a signal at the time of the processing. After performing processing such as multiplexing and modulation, the infrared light emitting elements 12, 1
3 is transmitted and this signal is transmitted to the controlled device.
If the control target device receives the signal, if the signal is in the middle of the operation pattern input process, the device to be controlled performs the subsequent steps to execute a predetermined control operation corresponding to the operation pattern input method.

In this embodiment, as the space operation mouse, the mouse already described in each embodiment of FIGS. 1 and 2 and FIG. 15 can be used. For example, when the spatial operation mouse described in the third embodiment is used, the motion detection unit 20
4 is a first motion sensor 30a, an amplifier 31a, a band limiter 39a, an A / D converter 40a, and a second motion sensor 30.
b, amplifier 31b, band limiter 39b, A / D converter 4
It can be configured using 0b. Also, the conversion unit 2
05, the vector table 206, and the identification unit 207 correspond to the motion recognition unit 41. The motion pattern memory 42 used by the motion recognition unit 41 stores basic motion patterns to be described later. The execution unit 208 corresponds to the execution function of the processing unit 37 in FIG. 7B, for example.

When the spatial operation mouse described above is used, the switch section 17 (not shown in FIG. 15) is provided with a click button for performing a confirmation operation or a selection operation. In some cases, a cursor button for enabling cursor control, an operation pattern input button for enabling an operation pattern input described later, and the like may be further provided for modification.

Here, Japanese Patent Laid-Open No. 3-192423 discloses a three-dimensional computer input device for detecting a motion in space and inputting it into a computer. However, it only discloses the concept of motion detection in space and can only be used for some pointing operations. On the other hand, an object of the present invention is to detect a motion of an operator in an arbitrary space by a spatial operation mouse or the like, input the motion as a meaningful motion, and perform motion recognition in a three-dimensional space. It is realized by using the concept of motion vector.

FIG. 21 shows a conversion unit 205 for converting the spatial motion amount obtained by the motion detection unit 204 into a motion pattern.
3 is a diagram for explaining the function of FIG.

The conversion unit 205 combines the detection amounts (Xt and Yt) in the two directions of horizontal (left and right) and vertical (up and down) detected by the motion detection unit 204 at a certain time (t) and combines them into the vector table 206. Is converted into a motion vector (Vt, θt) at high speed. Note that Vt represents the magnitude of the vector (a multiple of the reference vector) at a certain time (t), and θt represents the direction of the vector.
In the following description, a series of operations in space is treated as a velocity vector or acceleration vector sequence. In addition, the entire set of time series of motion vectors when the spatial operation mouse is moved is called a motion vector sequence (V {}, θ {}). The motion vector sequence (V {}, θ {}) is ((V1,
θ1), (V2, θ2), ..., (Vt, θt), ...
) Is a time series set of motion vectors.

Next, the spatial motion is calculated by the vector table 20.
An example of conversion into a motion vector will be described with reference to FIG.
The spatial motion of the operator is detected by the motion detection unit 204 by being decomposed into motions Xt and Yt in two directions as shown in FIG. As shown in FIG. 22B, the detection amount in the horizontal direction (for example, Xt = 2) and the detection amount in the vertical direction (for example, Yt =) obtained in the motion detection step at a certain time (t).
3) is based on a motion vector ((Vt, θ
t) = (V23, θ23)). FIG. 22 (c)
Represents the motion vector (V23, θ23). A coefficient based on the size (n) of the reference vector is registered in the vector table for obtaining the size of the motion vector.

FIG. 23 shows a discrimination unit 20 for discriminating by comparing the motion vector sequence (V {}, θ {}) converted by the conversion unit 205 with a basic motion pattern registered in advance.
7 is an example of an internal configuration of No. 7. As shown, the unit vector processing unit 224, the vector addition processing unit 225, and the determination unit 227 are used. However, it is also possible to adopt a configuration having only one of the unit vector processing unit 224 and the vector addition processing unit 225.

In the identifying unit 207, first, the unit vector processing unit 224 is performed using the motion vector sequence (V {}, θ {}).
And / or the vector addition processing unit 225. The result is compared with a basic motion pattern registered in advance by the motion vector sequence matching processing unit 226, and the determination unit 227 determines whether or not there is a basic motion pattern corresponding to the motion vector sequence.

FIG. 24 is a flow chart of processing by the unit vector processing section 224 of the identification section 207. Here, the motion vector (Vt, θt) at a certain time (t) is treated as a unit vector in the vector direction (θ) regardless of the magnitude of the vector. A group of unit vectors centered on the origin is called a unit vector function Sθ.

The unit vector processing section 224 first initializes the unit vector function Sθ (step S22).
8). In order to process the data of the motion vector sequence in time series, t is set to 1 (step S229). Time t
The direction θt of the vector at is called from the memory (step S230).

It is determined whether or not Sθ in that direction exists in the motion vector sequence up to that point (step S231). if,
If it does not exist, Sθt is set to 1 (step S232).

Then, it is determined whether the value of t is larger than the total number of samples of the motion vector sequence (step S233).
If smaller, add 1 to the value of t (step S23
4) Return to the step of calling the vector direction θt at time t from the memory.

Then, the same processing is repeated thereafter, and if the value of t becomes larger than the total number of samples of the motion vector sequence in step S233, this processing is ended.

By this processing, the motion vector caused by the motion of the operator is represented as a set of unit vectors (unit vector function) centered on the origin and having a size of 1.

Next, a concrete example of the unit vector processing will be described.

25 (a) to 25 (c) are explanatory views of the unit vector processing. For example, in a triangular motion vector string (the number of samples: 6) drawn in space as shown in FIG. 25A, the size of each motion vector is different,
The direction is composed of almost three directions. In this unit vector process from the motion vector sequence, attention is paid only to the vector direction (θ) of each motion vector as shown in FIG. 25B, and the process of obtaining the unit vector in the θ direction from the origin is repeated to obtain the motion vector sequence. An outline of the vector direction is obtained (steps S301 to S306). As a result, FIG. 25 (c)
As described above, this triangular motion vector sequence is represented as a unit vector in three directions.

As a result, triangles composed of almost the same vector direction can be identified as the same triangle regardless of the size of the shape or slight deformation. Alternatively,
When a circle is drawn in space, it can be identified by obtaining many unit vectors. Other than this, it is possible to identify various patterns.

In this unit vector processing, the direction of the unit vector may be roughly divided into several directions including the horizontal direction and the vertical direction. Furthermore, by using the information on the order in which the unit vectors appear,
For example, more motion patterns can be handled, such as identifying the rotation direction (clockwise or counterclockwise) in which a circle is drawn.

On the other hand, FIG. 26 is a flow chart of processing by the vector addition processing section 225 of the identification section 207. Here, the magnitude of the motion vector is added centered on the origin for each direction of the vector. The vector function obtained as a result of this addition is called a cumulative vector function Cθ with respect to the unit vector function in the unit vector processing.

The vector addition processing section 225 first initializes the cumulative vector function Cθ (step S23).
5). In order to process the data of the motion vector sequence in time series, t is set to 1 (step S236). Time t
The motion vector (vt, θt) at is called from the memory (step S237).

In the cumulative vector function Cθt-1 so far,
A vector of size (vt) is added in the direction (θt) (step S238).

Then, it is determined whether the value of t is larger than the total number of samples in the motion vector sequence (step S239).
If it is smaller, 1 is added to the value of t (step S24
0), and returns to the step (step S237) of calling the motion vector (Vt, θt) at time t from the memory.

Then, the same processing is repeated thereafter, and if the value of t becomes larger than the total number of samples of the motion vector sequence in step S239, this processing ends.

By this processing, the motion vector caused by the motion of the operator is represented as a set of vectors centered on the origin (cumulative vector function). By comparing this processing result with the basic operation pattern registered in advance, it is possible to identify whether the spatial operation pattern is the same as or different from the basic operation pattern in terms of shape, size and the like.

Next, a specific example of the addition vector process will be described.

27 (a) to 27 (c), FIG.
28A and 28B are diagrams for explaining a vector addition process for adding the magnitudes of motion vectors for each direction. In the unit vector processing described above, for example, as shown in FIG.
The shapes of the triangles drawn in space as shown in FIG. 7A and FIG.
A result like 5 (c) is obtained. The vector addition process is effective when it is necessary to distinguish the two. For example, a triangular motion vector string (sample number: 6) drawn in space as shown in FIG.
As shown in (b), paying attention to the vector size (V) and direction (θ) of each motion vector, repeating the process of adding the vector size for each vector direction from the origin, and constructing the entire motion vector sequence. Is obtained (steps S311 to S3
16). As a result, as shown in FIG. 27C, this triangular motion vector sequence is expressed as vectors having substantially the same size in the three directions.

Similarly, from the triangular motion vector sequence (number of samples: 5) drawn in space as shown in FIG. 28A, the result shown in FIG. 28B is obtained by the vector addition processing. To be

When comparing FIG. 27 (c) and FIG. 28 (b), the input spatial motion patterns are different from each other because the ratios of the sizes of the motion vectors forming the motion vectors are different. Can be identified.

Since the magnitude of the motion vector is expressed as a multiple of the reference vector, the vector addition process is facilitated by adding the coefficients recorded in the conversion table in advance. Also, this vector addition process may be roughly performed by dividing the direction of the motion vector into several directions including the horizontal direction and the vertical direction.

Next, the motion vector string matching processing unit 2
26, a matching process between the processing result of the unit vector processing unit 224 and a basic motion pattern registered in advance,
And / or a matching process is performed between the processing result of the vector addition processing unit 225 and a basic motion pattern registered in advance, and the determination unit 227 determines whether or not there is a basic motion pattern corresponding to the motion vector sequence.

FIG. 29 is a diagram showing an example of the flow of processing in the determination unit 227. For example, when the matching process in the motion vector sequence matching processing unit 226 can obtain the similarity (P) between the basic motion patterns of some identification candidates and the respective basic motion patterns, the similarity (P) is equal to the predetermined similarity. If it is larger than the reference value (Ref), the determining unit 227 determines that there is a basic motion pattern corresponding to the motion vector sequence, and if it is smaller than the similarity reference value (Ref), the basic motion pattern corresponding to the motion vector sequence. Is determined not to exist.

Then, based on this determination result, the identification unit 20
Recognition based on the degree of similarity in 7 is performed. Based on the result, the execution unit 208 executes control of the control target device with the basic data registered corresponding to the operation pattern.

Here, FIGS. 30A to 30E show an example of a method of identifying a motion vector sequence by a unit vector function. For example, when a unit vector function Sθ as shown in FIG. 30B is obtained from a triangular motion vector sequence (sample number: 6) drawn in space as shown in FIG. Expressed as a unit vector of direction. This processing result and pre-registered FIGS.
Identification is performed by comparing basic operation patterns such as 0 (e). Since this motion is represented as a unit vector in three directions, the basic motion patterns of the identification candidates are as shown in FIGS. 30 (d) and 30 (e).

Further, paying attention to the relationship of the vector directions (θ1, θ2, θ3) of the unit vector function, | θ1-θ2 | | θ2-θ3 | | θ3-θ1 | and the like are calculated.

This calculation is also performed for the basic operation patterns (FIG. 30 (d) and FIG. 30 (e)) of the identification candidates, and a comparison shows that the similarity in FIG. 30 (d) is found from the configuration in the vector direction. growing. In this way, the motion vector sequence can be identified by the unit vector function. Further, by calculating | θ1-θD1 | | θ2-θD2 | | θ3-θD3 |, etc., if the degree of similarity is increased or decreased, the identification is performed more accurately.

Thus, even without measuring the spatial coordinates of the spatial operation mouse with respect to the reference position (origin), the movement of the spatial operation mouse is relatively obtained as a time-series small reference vector set using the motion vector. Can input an operation pattern.

FIG. 31 is a flowchart showing an example of the process for inputting the motion pattern according to the present embodiment, which uses the above-described unit vector process and the determination method.

Next, the difference between the motion vector sequence matching of the present invention and the conventional pattern matching will be described.

32 (a) to 32 (g) are diagrams for explaining the difference between the motion vector sequence matching in the present invention and the conventional pattern matching. For example, when the operator wants to execute the control corresponding to the motion pattern “circle”, the place where the spatial operation mouse 1 should be correctly moved as shown in FIG. It is assumed that the motion of the locus shown in FIG. 32 (b) has been performed. At this time, in the conventional pattern matching processing, the locus of the entire motion obtained as shown in FIG. 32C or a set of characteristic points obtained from this locus,
Since matching is performed with the shape of the basic pattern registered in advance in FIG. 32 (d), the degree of similarity becomes small in the case of such an unstable operation input. Therefore, in the conventional pattern matching process,
It is considered that there are many cases in which the spatial motion pattern drawn by the operator cannot be recognized. On the other hand, in the motion vector sequence matching process in the present invention, based on the movement of the spatial operation mouse 1 by the operator, as shown in FIG.
A motion vector sequence is generated, and the result of the unit vector processing as shown in FIG.
Since the matching with a vector pattern such as 2 (g) is performed, the degree of similarity can be increased even for an input by an unstable operation. That is, by using the motion vector sequence matching, it is possible to correctly recognize an operation in an unstable space.

Here, FIG. 33 shows a configuration example of a system to which this embodiment is applied, and FIG. 34 shows a flow chart of an input operation example thereof.

The system shown in FIG. 33 has a space mouse body 1,
Motion detection unit 204, conversion unit 205, vector table 2
06, an identification unit 207, a basic operation pattern 209, an execution unit 208, an object designation unit 210, an object storage unit 211, a display control unit 212, and a display unit 213. In addition, here, it is assumed that the spatial mouse body 1 has at least the motion detecting element as described in the above-described embodiment. Also, the execution unit 208
In addition to the functions already mentioned,
The control corresponding to the code given from 10 is executed. The execution unit 208 executes the designated control for the display control unit 212 and a desired control target (not shown).

In this system, a control circuit (not shown) is used as a so-called pointing device for pointing an object displayed on the screen (herein referred to as an object pointing mode), and an input mode based on the above-mentioned operation pattern. (Here, it is called an operation pattern input mode) can be switched and used.

As described above, the motion pattern input mode is realized by the spatial mouse body 1, the motion detection unit 204, the conversion unit 205, the vector table 206, the identification unit 207, the basic motion pattern 209, and the execution unit 208.

The object designating mode is realized by the space mouse body 1, the object designating section 210, the object storing section 211, the display control section 212, the display section 213, and the executing section 208. Object specification part 2
10 is a function of causing the display control unit 212 to appropriately display an object on the screen of the display unit 213, and gives the result of detecting the movement of the spatial mouse body 1 to the display control unit 212 to move the cursor on the screen of the display unit 213. And the execution unit 208 or the display control unit 212 (when the function corresponding to the selected object is related to screen control) for the selected object or the code corresponding thereto in response to the instruction of the spatial mouse body 1. It has a function to output to. The object storage unit 211 stores information on various objects and codes indicating the functions associated with them.

In the above structure, first, the input mode is set by the user or the system.

When the motion pattern input mode is set (step S402), the user can input by holding the space operation mouse body and drawing a motion pattern in the space (step S403). Step S
This motion is detected in 404, a motion vector sequence is generated in step S405, and motion identification determination is performed using unit vector processing or the like in step 406. Then, in step S407, the corresponding control is executed.

To end the input mode, step S4
If the input mode is to be changed at 08, the process proceeds from step S409 to step S410.

On the other hand, when the object instruction input mode is set (step S410), the user grasps the body of the spatial operation mouse and moves the cursor displayed on the screen to the desired object to input an instruction. Is possible (S411). The control corresponding to the object instructed in step S412 is executed.

When ending the input mode, step S4
If the input mode is to be changed at step S13, the process proceeds from step S414 to step S402.

Next, the process of correcting a movement contrary to the operator's intention will be described.

FIGS. 35 (a) to 35 (d) are diagrams for explaining a correction unit for correcting the detection amount detected by the motion detection unit 204. As shown in FIG. If the operator does not hold the horizontal or vertical direction of the space operation mouse with the horizontal or vertical direction of the display screen in the hand, the cursor moves in a direction different from the movement of the hand in space. . For example, if the space operation mouse is held upside down with respect to the normal direction, the movement of the hand and the movement of the cursor are different by 180 degrees. In such a case, since the operator can visually check the movement of the cursor on the display screen, the horizontal or vertical direction of the space operation mouse may not immediately match the horizontal or vertical direction of the display screen. Easy to understand. Therefore, if the space operation mouse is immediately picked up again, comfortable operation can be performed thereafter.

However, while the spatial control mouse is being moved, unintentional rotation (due to the function of the human hand or arm) such as the twist of the wrist is added to change the hand movement and the cursor movement direction. Can be different. In other words, if the arm moves in a desired direction but the space operation mouse body 1 moves while rotating, the operator unexpectedly does not notice the cause, and as a result,
You may get the impression that the input device is not very user-friendly. For example, the operator moves the cursor 235 displayed on the display screen 203 of FIG.
, Y0) to the coordinates (X1, Y1) horizontally to the right. In this case, if the operator moves the spatial operation mouse with a twist of the wrist,
The space operation mouse is shown in FIGS. 35 (b), 35 (c) and 35.
It rotates in the order shown in (d). As a result, cursor 2
35 moves in different directions from the coordinates (XA, YA) to the coordinates (XB, YB) to the coordinates (XC, YC) in FIG.

Therefore, as shown in FIG. 36, the rotation amount detecting element 236 is installed in a direction (that is, the front-back direction) orthogonal to both the horizontal direction motion detecting element 2 and the vertical direction motion detecting element 3, and the rotation detection amount is determined. Accordingly, the distribution of the component of the motion amount such as the velocity or acceleration in the horizontal direction or the vertical direction in the detection axis direction is corrected.

FIG. 37 is a schematic block diagram of the spatial operation mouse in which the rotation amount detection unit 237 and the correction unit 238 are further provided in the spatial operation mouse of FIG. The movement of the spatial operation mouse by the operator is detected by the movement detection units 16a and 16b.
It is converted into a predetermined motion signal, for example, a signal representing the speed or movement distance of the spatial operation mouse. Based on the amount obtained by the rotation amount detection unit 237 at that time, horizontal (left and right) and vertical (up and down) The component of the movement amount in the two directions is corrected by the correction unit 238.

As a result, the performance of the spatial operation mouse as a pointing device can be improved, and
The operation pattern can be input accurately.

38 (a) and 38 (b) show the correction unit 2
It is a figure for demonstrating the process in 38. When the horizontal direction or the vertical direction of the spatial operation mouse is deviated from the horizontal direction or the vertical direction of the display screen due to the rotation due to the twist of the wrist or the like, the detection amount detected by the motion detecting units 16a and 16b is set to The direction (x ') and the vertical direction (y'). On the other hand, the detection amount when there is no deviation is set to the horizontal direction (x) and the vertical direction (y). Similarly, the direction of the motion vector when there is a deviation is (θ ′), and the direction of the motion vector when there is no deviation is (θ). In addition, the rotation amount detection unit 2
The amount obtained by 37 is (θm).

Then, as shown in FIG. 38 (a), when properly moving in the (θ) direction, if the position shifts by (θm) due to the rotation of the wrist, the direction of the motion vector becomes as shown in FIG. 38 (b). Is obtained as (θ '). That is, θ ′ = θ + θmx ′ = V · cos θ ′ y ′ = V · sin θ ′. Therefore, using the magnitude (V) of the motion vector, the direction (θ ′) of the motion vector that is erroneously obtained, and the amount (θm) obtained by the rotation amount detection unit 237, θ = θ′− The component distribution of the motion amount in the detection axis direction can be corrected by setting θmx = V · cos (θ′−θm) y = V · sin (θ′−θm).

Next, expanding the function of the spatial motion pattern input system of this embodiment from a two-dimensional motion pattern to a three-dimensional motion pattern will be described.

Up to this point, the movement detection in the two-axis directions has been described, but the present invention is not limited to this, and the front-back movement detection element 239 corresponding to the front-back direction 240 as seen from the spatial operation mouse as shown in FIG. Can be provided so that the spatial motion pattern can be input by three-dimensional motion.

FIG. 40 shows an example of a method of inputting a motion pattern by three-dimensional movement. The operator changes the spatial operation mouse 1 from a to b, c, d, e, f into a triangular pyramid shape. By moving, a triangular pyramid can be drawn on the display screen 203 of the display device 202.

The motion vector in such a three-dimensional space is represented by the magnitude (V) of the vector and its direction (θ, φ) as shown in FIG. 41 (a). Also, FIG.
1 (b) shows an example of a vector table that is referred to when converting a three-dimensional motion in a three-dimensional space into a motion vector.

By using this vector table and performing the same processing as the two-dimensional spatial motion pattern input already described, it is possible to input the three-dimensional spatial motion pattern.

Next, the spatial motion pattern input mode is turned on.
The / OFF operation will be described.

[0184] The main operation for the control target device by the normal spatial operation mouse is an operation of moving the cursor on the display screen in accordance with the movement of the hand. This operation realizes a so-called pointer function. Therefore, in order to enter the operation pattern input mode from the pointer function mode or the like with the spatial operation mouse, the following operation for changing the mode is required. For example, (1) Move the cursor to a certain position on the display screen (area where various tools can be selected, or area where a menu is selected).
It is conceivable to perform a predetermined simple movement such as (2) pressing the operation pattern input start button or (3) shaking up and down several times.

Also, when returning from this mode to the pointer function mode or the like, some operation or a predetermined rule is required. For example, (1) stipulate that it will be forced to come out after a certain period of time, (2) determine a stationary state (a state in which no amount of motion is detected), (3) release the operation pattern input button. (Or press again) is possible.

Further, since the operability after executing the control operation changes depending on the handling of the cursor when in the operation pattern input mode, the following can be considered depending on the application. (1) Delete the cursor from the display screen. However, the coordinates on the display screen when the operation pattern input mode is entered are retained. As a result, when the cursor is exited from this mode, the cursor reappears at the position where the operation pattern input mode was entered, so that the operation can be continued from that position after executing the tool selection and the like. (2) As usual, the cursor moves on the display screen according to the movement of the spatial operation mouse. As a result, the operator can give visual feedback, so that an accurate operation can be input. (3) The cursor is changed to a shape that makes it easy to understand that the operation pattern input mode is entered. In addition, you may combine some of these.

Further, normally, an arrow-shaped (or cross-shaped etc.) cursor 235 as shown in FIG. 42A is recognized as a result of being recognized by inputting an operation pattern.
When the deformation is performed as shown in (b), the operator can easily understand whether the operation pattern is correctly input. At the same time, omitting the screen display for confirmation (for example, "What is the current action?") And the operator's consent operation ("Yes" or "No") for each operation pattern input, You can move smoothly to the next operation. FIG. 42 (c)
Shows a modification of the cursor shape depending on the recognition result.

Further, since the operation in the space has an unstable surface, even when the cursor is to be moved linearly, the direction of the cursor to be moved is first recognized by the operation pattern and the shape of the cursor is indicated. It just needs to be transformed into a shape so that the cursor can be controlled only in that direction after that.

As described above in detail, according to this embodiment,
In a system using a three-dimensional input device such as a spatial operation mouse that enables a pointer operation on an arbitrary plane and further a spatial pointer operation, a computer or a multi-computer is recognized by recognizing an operation pattern in an operator's space. It is possible to realize a sensible man-machine interface environment that can control media devices.

FIG. 43 is a schematic configuration diagram of the spatial operation mouse according to the fifth embodiment. This spatial operation mouse has the same basic configuration and operation as the spatial operation mouse according to the above-described embodiment. However, the difference is that a motion detection element 7 for detecting the motion in the front-back direction is further provided.

That is, the present invention is not limited to the motion detection in the two-axis directions, and the motion detection units 2, 3 and 7 corresponding to the three-axis directions are provided as in the present embodiment, and the three-dimensional space is provided. It can be extended to detect movement in.

Since a normal display screen is a two-dimensional display,
In order to specify the cursor position on the screen, it is sufficient to provide the movement detecting units 2 and 3 for two axes as in the first embodiment, but the movement in the three axes of this embodiment can be detected. For the system having various pseudo three-dimensional displays, which are becoming popularly used recently,
It is very effective when used as a pointing device.

It is also possible to give a special role to the movement in the third axial direction. For example, the ratio of the movement amount of the cursor on the screen to the movement amount of the spatial operation mouse in the first or second axial direction may be determined by the third (virtual) position in the axial direction. it can.

As the detection of the movement of the third axis, the rotation around the axis may be detected by using the piezoelectric gyro shown in FIG. 9, instead of detecting the parallel movement by providing the movement detecting section 7.

On the other hand, in the spatial operation mouse according to the third embodiment described above, it is also possible to use an operation in a three-dimensional space as a basic operation pattern used for pattern input.

Needless to say, it may be further expanded to provide a detection unit having four or more axes.

In this case, the two axes may be given a role as a pointing device, and the other axes may be given a role of inputting an operation pattern.

As described above, according to the present invention, it is possible to perform a pointer operation on an arbitrary virtual plane, and further, a spatial pointer operation, recognize an operator's operation, and display a computer or a multimedia device or its display. It is possible to provide a spatial operation mouse that enables easy pointer operation and control operation even if the mouse is away from the device.

FIG. 44 is a schematic block diagram of a spatial operation mouse according to the sixth embodiment. This spatial operation mouse has the same basic configuration and operation as the above-described embodiment. The horizontal motion detecting element 2 and the vertical motion detecting element 3 detect the two-dimensional movement of the spatial operation mouse body 1, and the click button 4 accepts a click operation.

Here, in this embodiment, the spatial operation mouse and the control target device are connected using the connection cable 8. The connection is performed by inserting the connector 9 at the end of the connection cable 8 of the space operation mouse into the device to be controlled. The output signal from the space operation mouse is transmitted through the connection cable 8 and input to the control target device.

When it is not necessary for the system to make the space operation mouse wireless, this wired space operation mouse can be used without the need to consider the directivity of the transmission direction as in the infrared method. Transmission can be performed very reliably.

Although the present invention has been described with reference to various embodiments, the shape of the spatial operation mouse does not necessarily have to be the shape of a conventional so-called mouse, but may be changed depending on the purpose and use. It is possible to use it in various shapes.

Further, it is not always necessary to hold it in the hand for the operation, and the operator may directly or indirectly mount it on another device or tool for use, or may be built in.

Alternatively, if this spatial operation mouse is provided with other input means such as voice input, it is possible to provide a more varied operation environment.

The number of click buttons provided is arbitrary, and the shape thereof can be variously modified.

In each of the above-mentioned embodiments, it is preferable that the circuit inside the spatial operation mouse is integrated into one chip as much as possible.

Further, the present invention is not limited to the above-mentioned respective embodiments, and various modifications can be carried out without departing from the scope of the invention.

[0208]

As described above in detail, the present invention is provided with the motion detecting means for detecting the motion of the spatial operation mouse from the acceleration or the angular velocity in the two-dimensional direction or the three-dimensional direction.

Therefore, the operation intended by the operator can be detected by the spatial operation mouse in an arbitrary operation space, and the control signal can be easily transmitted to the controlled device by the transmitting means. Furthermore, by displaying the pointing position of the operator or changing the direction of the display screen by changing the cursor or the display object on the display screen of the controlled device, it is possible to realize an unprecedented human interface environment with easy operation. .

Alternatively, it is possible to realize the input means for controlling by using a natural human motion by providing the motion pattern storage means in which the motion of the operator is registered in advance and identifying the operation motion pattern.

Further, by providing a band limiting filter for removing the frequency of the hand shake of the operator, the pointing operation due to the hand shake can be made accurate.

On the other hand, in the spatial motion pattern input method according to the present invention, the motion of the spatial motion pattern input by the operator operating the spatial operation mouse is detected to detect the motion vector sequence (relative time series minute Reference vector set), and compares the motion vector sequence corresponding to the pre-registered basic motion pattern with the motion vector sequence for identification, and executes control based on this recognition result for the control target device. To do.

Therefore, the operation pattern can be input with high accuracy without measuring the spatial coordinates of the spatial operation mouse with respect to the reference position (origin).

As described above, according to the present invention, it is possible to control the device to be controlled by using a natural human motion.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a spatial operation mouse according to a first embodiment of the present invention.

FIG. 2 is a schematic block diagram of a spatial operation mouse according to the first embodiment of the present invention.

FIG. 3 is a conceptual diagram of use of the spatial operation type computer system using the mouse of FIG.

FIG. 4 is an explanatory diagram of a display screen and pointing operation of the computer shown in FIG.

5 is another explanatory diagram of the display screen and pointing operation of the computer of FIG.

6 is still another explanatory diagram of the display screen and pointing operation of the computer of FIG.

FIG. 7 is a block diagram of a spatial operation mouse and a control target device according to the first embodiment of the present invention.

FIG. 8 is a voltage waveform diagram corresponding to acceleration and velocity, and a diagram showing a relation between velocity and pulse density.

FIG. 9 is a configuration example of an angular velocity detection unit using a piezoelectric vibration gyro.

FIG. 10 is a schematic block diagram of a spatial operation mouse provided with a camera shake correction unit.

FIG. 11 is a schematic configuration diagram of a spatial operation mouse according to a second embodiment of the present invention.

12 is a conceptual diagram of use of the spatial operation type video system using the mouse of FIG.

13 is an explanatory diagram of a display screen and pointing operation of the system of FIG.

FIG. 14 is a diagram for explaining various click operations in the first and second embodiments.

FIG. 15 is a schematic block diagram of a spatial operation mouse according to a third embodiment of the present invention.

FIG. 16 is a diagram for explaining a detection operation pattern,

FIG. 17 is a diagram showing an example of an operation pattern input method according to the fourth embodiment of the present invention.

FIG. 18 is a diagram for explaining the concept of a motion vector.

FIG. 19 is a diagram for explaining a difference between a motion vector and a motion vector.

FIG. 20 is a block diagram showing a main configuration of a spatial motion pattern input system according to a fourth embodiment of the present invention.

FIG. 21 is a diagram for explaining the function of a conversion unit that converts a spatial motion amount into a motion pattern.

FIG. 22 is a diagram showing an example of converting a spatial motion into a motion vector by a vector table.

FIG. 23 is a block diagram showing a schematic configuration of an identification unit.

FIG. 24 is a flowchart of processing by a unit vector processing unit.

FIG. 25 is a diagram for explaining unit vector processing.

FIG. 26 is a flowchart of processing by the vector addition processing unit.

FIG. 27 is a diagram for explaining vector addition processing.

FIG. 28 is a diagram for explaining vector addition processing.

FIG. 29 is a diagram showing an example of the flow of processing in a determination unit.

FIG. 30 is a diagram for explaining an example of a method of identifying a motion vector sequence using a unit vector function.

FIG. 31 is a diagram showing a flow of motion pattern recognition processing by a unit vector function.

FIG. 32 is a diagram for explaining the difference between the motion vector sequence matching of the present invention and the conventional pattern matching.

FIG. 33 is a diagram showing a configuration example of a system to which the operation pattern input method according to the present embodiment is applied.

34 is a flow chart showing an input operation example of the system of FIG.

FIG. 35 is a diagram for explaining a process of correcting the detection amount detected by the motion detector.

FIG. 36 is a schematic view showing an example of the installation position of the rotation amount detection element.

FIG. 37 is a schematic block diagram showing a spatial operation mouse provided with a correction unit.

FIG. 38 is a diagram for explaining the processing in the correction unit.

FIG. 39 is a diagram showing an example of an installation position of a front-back direction detection element for expansion to input of a three-dimensional operation pattern.

FIG. 40 is a diagram showing an example of a three-dimensional operation pattern input method.

FIG. 41 is a diagram showing an example of a method of expressing a motion vector in a three-dimensional space, and an example of a vector table referred to when converting a motion in a three-dimensional space into a motion vector.

FIG. 42 is a diagram showing a modified example of a cursor according to the result recognized by inputting a motion pattern.

FIG. 43 is a configuration diagram of a spatial operation mouse according to a fifth embodiment of the present invention.

FIG. 44 is a configuration diagram of a spatial operation mouse according to a sixth embodiment of the present invention.

[Explanation of symbols]

1 ... Spatial control mouse body, 2 ... Horizontal motion detection element,
3 ... Vertical motion detection element, 4 ... Click button, 5,
6 ... Infrared light emitting element, 16a ... 1st motion detection part, 16b
... second motion detector, 17 ... switch, 18 ... transmitter,
21 ... Control target device, 22 ... Display screen, 23 ... Infrared light receiving element, 30a ... First motion sensor, 31a31b ... Amplifier, 39a ... Band limiter, 40a ... A / D converter, 32
c ... speed detection unit, 30b ... second motion sensor, 31b ... amplifier, 39b ... band limiter, 40b ... A / D converter, 3
2d ... Speed detection unit, 41 ... Motion recognition unit, 42 ... Motion pattern memory, 43 ... Infrared remote control transmission circuit, 34 ... Infrared light emitting element, 202 ... Display device, 203 ... Display screen,
235 ... Cursor, 236 ... Rotation amount detecting element, 239 ...
Front-back motion detector

Front Page Continuation (72) Inventor Fumimoto Fumiyoshi 1 Komukai Toshiba-cho, Kawasaki-shi, Kawasaki, Kanagawa Prefecture Corporate Research & Development Center

Claims (7)

[Claims] 1. Detecting at least one of speeds or accelerations in three predetermined axial directions in an operation space or angular velocities or angular accelerations around the three axes, and the detected one or more quantities. Has a motion signal generating means for outputting as it is or a related amount and outputting as a motion signal, and a transmitting means for transmitting the control signal including the motion signal given by the motion signal generating means. A space operation mouse, a receiving means for receiving the control signal transmitted by the transmitting means of the space operation mouse, and a cursor or a display object on the display screen based on the control signal obtained by the receiving means. Display means for changing and displaying the position pointed to by the operator of the spatial operation mouse, or changing the display screen according to the movement signal. A spatial operation mouse system comprising a controlled device. 2. A linear, planar, or spatial motion in a three-dimensional space given by an operator is defined as velocity or acceleration in the predetermined three axial directions or angular velocity or angle around the three axes in the operation space. A detection step of detecting at least two amounts of acceleration, a conversion step of converting the detected at least two detection amounts into motion pattern data in which the motion is patterned, the motion pattern data, and pre-registered A comparing step of comparing the basic data relating to the plurality of basic motion patterns, an identifying step of identifying the operation pattern on the basis of the comparison result in the comparing step, and a comparing step based on the identification result in the identifying step. And a control step for performing a predetermined control corresponding to the operation pattern. Spatial operation pattern input method using mouse. 3. A spatial motion pattern input method for causing a control target device to execute control of contents according to a spatial motion pattern of a spatial manipulation type input device, wherein a predetermined number of 3 which are not parallel to each other in a space of the spatial manipulation type input device. A motion detecting step of detecting at least two motion amounts in the axial direction among the motion amounts in the axial direction; and at least two detected in the motion detecting step.
A conversion step of converting the spatial motion amount consisting of two detection amounts into a motion vector sequence, and an identification step of comparing the motion vector sequence corresponding to a basic motion pattern registered in advance with the motion vector sequence for identification. And an executing step of executing control based on the recognition result in the identifying step with respect to a device to be controlled, the spatial operation pattern input method. 4. A unit vector function based on a motion vector sequence obtained from a motion amount of the spatial operation type input device and a motion vector sequence obtained from a motion amount of the spatial operation type input device in the identifying step. The vector function generating step for generating at least one of the cumulative vector functions set in step 1 and at least one of the generated unit vector function and cumulative vector function are compared with those obtained from the motion vector sequence corresponding to the basic motion pattern motion. 4. An identification step of identifying a spatial motion pattern of the spatial operation type input device based on the comparison result.
The spatial movement pattern input method described in. 5. The conversion step comprises a conversion table in which the number of unit vectors and angles are registered with respect to the spatial motion amount, and the spatial motion amount including at least two detection amounts detected in the motion detection step. The spatial motion pattern input method according to claim 3, wherein a result obtained by converting the spatial motion amount into a motion vector sequence is obtained by sequentially providing a value obtained by temporally sampling as a designated value. 6. The operation detecting step includes a rotational operation amount detecting step of detecting an amount of rotational operation about an axis in a direction orthogonal to each of two axis directions for detecting an amount of operation of the spatial operation type input device. Further, the converting step includes, based on the detection result of the rotational movement amount detecting step, a movement amount in the two axial directions of the related spatial operation type input device detected in the movement detecting step, The spatial motion pattern input method according to claim 3, further comprising a correction step of extracting a motion amount from which a rotation motion amount is removed. 7. The spatial motion pattern input method according to claim 3, wherein the execution step displays the shape of the spatial motion pattern recognized in the identification step on a display screen.