JPH11345073A - Portable information terminal operation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to release operator's hands from the input operation of information or a command by providing the operation system with an image forming device for picking up an image of an object and a mouse operation device for generating a mouse operation signal based on the picked-up image. SOLUTION: A portable information terminal is provided with an operation system 15 for realizing mouse operation by using a monitor camera 14. In the system 15, an image picked up by the camera 14 is supplied to an image recognizing circuit 16 as an image signal. The circuit 16 observes the picked-up image in time series to recognize the movement of the image and supplies a recognition signal to an image analysis circuit 17. The circuit 17 accesses a mouse operation signal previously stored in a memory 18 and supplies the accessed signal to the CPU 20. The CPU 20 transfers the mouse operation signal to a driver circuit 22 in accordance with a program stored in a program memory 21. A mouse pointer is displayed on a display screen 12 based on a picture signal outputted from the circuit 22.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a portable information terminal, so-called PDA (Personal Digital Assault).
The present invention relates to an operation system used when operating (i.

[0002]

2. Description of the Related Art Conventionally, an operation system for operating a portable information terminal using a keyboard or a pen is widely known.

For example, when inputting information or a command to a portable information terminal with a keyboard or pen, the portable information terminal is supported with one hand, and the keyboard or pen is operated with one hand. Therefore, both hands are blocked, and the freedom of action of the operator is hindered.

[0004] In addition, a cursor operation of a mouse or the like operated by hand is difficult to adjust to an object, and the hand must be moved each time the operation is performed.

[0005]

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a portable information terminal operation system capable of releasing the operator's hand from inputting information and instructions. Aim.

[0006]

Means for Solving the Problems In order to solve the above problems, the present invention has taken the following means.

According to a first aspect of the present invention, there is provided a portable information terminal operating system comprising: an image forming apparatus for capturing an image of an object; and a mouse operating device for generating a mouse operation signal based on the captured image. take.

According to this configuration, the operator does not need to operate the portable information terminal with his / her hand, and can release his / her hand from inputting information and commands. As a result, the operator can easily perform operations such as operating the communication device, operating the camera, and taking notes using the hand opposite to the hand supporting the portable information terminal.

According to a second aspect of the present invention, in the portable information terminal operating system according to the first aspect, the mouse operating device performs click, double-click, A configuration is employed in which the mouse operation signal for causing any of the drag operations to be performed is generated.

[0010] Thus, the portable information terminal can be operated by the movement of the operator, the hand does not need to be moved, and the operation is simplified.

[0011] For example, as in the third aspect of the present invention, the mouse operating device generates the mouse operation signal for executing the drag based on the movement of the operator's line of sight captured by the image forming apparatus. May be. Thereby, the operator can easily and quickly input information and commands simply by moving his / her eyes.

In this case, as in the invention according to claim 4,
The mouse operation device may generate the mouse operation signal for performing the click and the double click based on the blink of the operator captured by the image forming device, as in the invention according to claim 5, A mouse operation device may generate the mouse operation signal for performing the click and the double click based on the operator's nod and swing motion captured by the image forming apparatus.

According to a sixth aspect of the present invention, in the portable information terminal operating system according to the second aspect, the mouse operating device comprises:
The mouse operation signal for performing the click and the double click is generated based on the movement of the mouth of the operator captured by the image forming apparatus.

According to this configuration, the operator can easily and quickly input information and commands simply by moving the mouth.

[0015]

An embodiment of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 shows an overview of a portable information terminal to which the operation system according to the present invention is applied. This portable information terminal 1
The main body 11 includes a display screen 12 for visually displaying various information, a monitoring camera 14 as an image forming apparatus for optically recognizing an object illuminated by an infrared light emitting diode (LED) 13. Is provided. The display screen 12 may be composed of, for example, a liquid crystal display.
The surveillance camera 14 includes, for example, surveillance angles,
For example, it has 70 °.

FIG. 2 schematically shows a circuit configuration of the portable information terminal 10. As shown in FIG. The portable information terminal 10 includes an operation system 15 that implements a mouse operation using the above-described monitoring camera 14.
Is provided. In this operation system 15, the surveillance camera 14
The image captured by the image recognition circuit 16 is used as an image signal.
Supplied to

The image recognition circuit 16 observes the captured image in time series and recognizes the movement of the image. For example, the image analysis circuit 16 selects an operator's eyeball from the image, and specifies the operator's line of sight from the movement. In addition, the image recognition circuit 16 selects the operator's head from the image, specifies the nod or the swing of the operator from the motion, or selects the operator's mouth from the image, and selects “Yes” or “No” from the motion. , Etc. can be specified. A recognition signal corresponding to each of these operations is supplied to the image analysis circuit 17.

The image analysis circuit 17 calls a mouse operation signal stored in the memory 18 in advance based on the recognition signal supplied from the image recognition circuit 16. The mouse operation signal is supplied to a central processing unit (CPU) 20. C
The PU 20 passes a mouse operation signal to the driver circuit 22 according to a program stored in the program memory 21. A mouse pointer is displayed on the display screen 12 based on the image signal output from the driver circuit 22.

Operation keys 23 may be connected to the CPU 20, as shown in FIG. These operation keys 23
Is desirably blinked by the action of the light emitting diode 24.

Next, a method of operating the portable information terminal 10 will be described. First, the power supply of the portable information terminal 10 is turned on. Then, as shown in FIG.
0 is displayed at an arbitrary position on the display screen 12. CPU2
0 turns on the infrared light emitting diode 13 at the same time. The object irradiated by the infrared light emitting diode 13, that is, the operator 31 is captured by the monitoring camera 14.

Now, consider a case where the operator 31 moves the line of sight pointed to the mouse pointer 30 to another area without blinking. The movement of the eyeball caused by the movement of the line of sight is observed by the monitoring camera 14. This movement is recognized by the image recognition circuit 16, and as a result, corresponding to the movement of the line of sight,
A mouse operation signal for executing the mouse pointer movement is output from the image analysis circuit 17. The CPU 20 activates the driver circuit 22 according to the operation signal, and displays the display screen 1.
The mouse pointer 30 on the line 2 is moved according to the amount of movement of the line of sight.

Similarly, when one blink of the operator is recognized, a mouse operation signal for performing a click is generated, and
If the operator blinks twice without a break, a mouse operation signal for performing a double click is generated. If the movement of the line of sight is recognized without a break after blinking, a mouse operation signal for performing the drag is generated.

In addition, the image analysis circuit 17 may generate a mouse operation signal for performing a click based on the operator's nod, and a mouse operation signal for performing a double click based on the operator's swing. May be generated. In addition, a mouse operation signal for performing a click may be generated when the utterance of “Yes” is recognized from the movement of the mouth of the operator. Similarly, when the utterance of “No” is recognized, a double click is performed. A mouse movement signal may be generated. However, "yes" and "no" do not need to be actually generated, but need only be recognized by the movement of the mouth.

For example, consider the case where the operator's line of sight is directed away from the display screen 12 and toward the operation key 23. At this time, the image analysis circuit 17 outputs, instead of the mouse operation signal, an operation key recognition signal for specifying one operation key on which the user is looking. Based on the operation key recognition signal, the CPU 20 turns on the light emitting diode 24 corresponding to the specified operation key. The operator confirms the operation key that he / she wants to operate by lighting the operation key.

Subsequently, for example, when the operator nods, the image analysis circuit 17 outputs a key operation input instruction signal. As a result, the operator can operate the operation key 23 selected by the line of sight without touching the operation key.

[0027]

As described above, according to the portable information terminal operating system of the present invention, it is possible to release hands from input operations of information and commands, for example, keyboard input and pen input.

[Brief description of the drawings]

FIG. 1 is a perspective view showing a portable information terminal to which an operation system according to an embodiment of the present invention has been applied;

FIG. 2 is a block diagram showing a circuit configuration of the portable information terminal according to the embodiment.

FIG. 3 is a diagram showing an operation of the portable information terminal according to the embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Mobile information terminal 12 Display screen 13 Infrared LED 14 Surveillance camera 15 Operation system 16 Image recognition circuit 17 Image analysis circuit 31 Operator

Claims (6)

[Claims] An image forming apparatus for capturing an image of an object,
And a mouse operating device that generates a mouse operation signal based on the captured image. 2. The mouse operation device according to claim 1, wherein the mouse operation signal for performing one of a click operation, a double-click operation, and a drag operation is generated based on the movement of the operator captured by the image forming apparatus. The portable information terminal operation system according to claim 1, wherein 3. The mouse operation device according to claim 2, wherein the mouse operation device generates the mouse operation signal for performing the drag based on a movement of the operator's line of sight captured by the image forming device. Mobile information terminal operation system. 4. The mouse operation device according to claim 2, wherein the mouse operation device generates the mouse operation signal for performing the click and the double click based on a blink of the operator captured by the image forming device. Claim 3
The portable information terminal operation system described in the above. 5. The mouse operation device according to claim 1, wherein the mouse operation signal for performing the click and the double click is generated based on the operator's nod and swing motion captured by the image forming device. The portable information terminal operation system according to claim 2. 6. The mouse operation device according to claim 1, wherein the mouse operation signal for performing the click and the double click is generated based on a movement of a mouth of the operator captured by the image forming device. 2. The mobile information terminal operation system according to 2.