JP2003186620A - Information processor equipped with display unit with pointing function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processor equipped with a display unit having a pointing function which allows pointing by direct touch with finger and realizes an operation equivalent to the right and to the left buttons of a mouse by easily understandable and natural operation. <P>SOLUTION: The information processor equipped with the display unit having the pointing function which allows input of the coordinates of a touch position on a screen, is provided with a display unit 1 having the pointing function which has a transparent touch panel, by which the coordinates of the touch position can be inputted, and a control part 2, which controls the above coordinates as input information. The control part 2 stores in a storage part 12 in the control part 2, the coordinates of the touch position inputted by the touch panel 4, when an operation to touch to the touch panel 4 is performed, displays operation buttons in the screen of the display unit 3, and performs control corresponding to the button operations at the coordinates stored in the storage part 12, according to a touch operations to the display button area, including the operation buttons. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing device such as a personal computer or an electronic blackboard system having a display device with a pointing function capable of pointing a position on a screen with a finger or the like and inputting coordinates of the position. Regarding
In particular, the present invention relates to an information processing device including a display device with a pointing function that can realize a function equivalent to a button operation of a mouse.

[0002]

2. Description of the Related Art A personal computer has conventionally been provided with a pointing device called a mouse,
When the user moves the mouse to move the cursor displayed on the screen, the control unit of the personal computer acquires the coordinate information input from the mouse along with the movement. Then, when the user positions the cursor at a desired position and momentarily presses the button provided by the mouse in that state (hereinafter, this operation is referred to as a click operation), the control unit recognizes the click operation. Then, for example, what is instructed is recognized from the coordinate information input from the mouse at that time, and control according to the instruction is executed. Although such a function was effective in improving operability, in order to further improve operability, it is possible to indicate a position on the screen by touching a desired position on the screen with a finger or the like. Personal computers equipped with a display device with a pointing function have also been provided. In a personal computer using such a display device with a pointing function, as shown in FIG.
For example, when the cursor is at the position shown in (a),
Simply touching the position indicated by the star mark in (b) moves the cursor to the touch position shown in (c). That is, the action corresponding to the left mouse button being clicked at the touch position is performed. Similarly, even with electronic blackboard systems that replace whiteboards, in recent years,
A position pointed by a finger or the like on a large screen can be input to the computer system, and information written by the finger or the like can be input to the computer system in real time.

However, since such a display device with a pointing function does not have an operation button for performing a click operation like a mouse, there is a problem for that. For example, Windows (registered trademark), MAC,
General-purpose OS (operating system) such as Linux
In the APL (application) that operates under, the left button ON (click operation) performs menu display and selection, and the selected function is executed, while the right button ON also provides the function to operate. In a display device with a pointing function that does not have buttons, it is generally impossible to perform an operation associated with each of the two left and right buttons of the mouse. In addition, the general G provided by these OSs
As a UI (graphic user interface) function, when the cursor is positioned at a preset position without clicking, a balloon is displayed (for example, a frame is displayed in the vicinity and an explanatory note is displayed in the frame). However, such a function cannot be realized when a finger or the like is touched to give an instruction. That is, the mouse has two functions of simply positioning and performing a click operation at the positioned position, but in the display device with a pointing function, the touch operation is a positioning and clicking operation.

Therefore, for example, Japanese Patent Laid-Open No. 2000-819
In the prior art disclosed in Japanese Patent Laid-Open No. 51-51, a pen with buttons is used for touching, and each function of the mouse is enabled by switching the function at the time of touching with the button. Further, in the conventional technique disclosed in Japanese Patent Laid-Open No. 9-282100, an attempt is made to solve the above-mentioned problem of the display device with a pointing function by changing the touching method. For example, by pressing the whole body, pressing the right side of the fingertip, pressing the left side of the fingertip, and pressing the left side of the fingertip. Correspond to the operation. In addition, JP-A-11-110
In the conventional technique disclosed in Japanese Patent No. 132, the displayed cursor is divided into a plurality of areas corresponding to a plurality of buttons,
By changing the touch position on the displayed button,
It corresponds to the operation of the left and right buttons of the mouse. The following is known as a coordinate input / detection device of the display device with a pointing function according to the present invention. The first example is one in which wires are provided in a grid pattern in the XY directions on the coordinate input surface. By changing the current flowing through the wire using a dedicated pen that generates a magnetic field, Detect coordinates. In the second example, a resistance film such as conductive rubber is used for the coordinate input surface. When a pen or finger is brought into contact with the coordinate input surface to indicate a desired position, the resistance of that position on the coordinate input surface is changed. Since the value changes due to pressure, the coordinates of the position are detected by detecting the resistance value. This method is called a touch panel method, and there are various ways to realize it. Further, in the third example, the light beam is emitted from the two light emitting portions via the rotary polygon mirror to scan the coordinate input surface, and the reflected light from the reflecting member provided at the tip of the pen is changed to two. The position where the pen is inserted is detected by detecting with the light receiving part (Japanese Patent Laid-Open No. 57-21).
1637). Light is received by two light receiving parts and 3
Using the principle of angular surveying, the coordinates of the designated position are calculated. The coordinate input surface of this coordinate input / detection device is not a physical surface like the coordinate input surface of the coordinate input / detection devices of the first and second examples, but is a light beam emitted from the light emitting portion. The surface to be formed.

In addition to the above-mentioned Japanese Patent Laid-Open No. 57-212637, the prior art disclosed in Japanese Patent Laid-Open No. 9-91094 is an optical coordinate input / detection device shown as a third example. In the conventional technique, light emitting / receiving portions are provided in the lower left and right corners of the coordinate input surface, and the respective light emitting / receiving portions are provided.
The light beam is emitted from the light receiving part while changing the emission direction (emission angle), the coordinate input surface is scanned, and the light beam is folded back by the corner cube array (light retroreflecting means) provided on the upper and left and right sides of the panel. Then, the light beams returned by the respective light emitting / receiving units are detected (reflected in the returning direction). This makes it possible to detect the scanning angles of the left and right light emitting / receiving portions where the light beam is blocked by a finger or the like, and the coordinates of the designated position can be calculated using the principle of triangulation. In addition, JP-A-11-11011
The conventional technique disclosed in Japanese Patent No. 16 is also an optical coordinate input / detection device shown as a third example. In this conventional technique, light emitting / receiving portions are provided, for example, in upper and left corners of the coordinate input surface. The light beam is emitted from each light emission / reception part while changing the emission direction (emission angle), the coordinate input surface is scanned, and the light beam is folded back by the light retroreflecting means provided on the lower side and left and right sides of the panel. Then, the light beams returned by the respective light emitting / receiving units are detected (reflected in the returning direction). As a result, the coordinates of the instructed position are calculated by detecting the light blocking range of the scanning angles of the left and right light emitting / receiving parts where the light beam is blocked by the finger or the like. In addition, Japanese Patent Laid-Open No.
The prior art disclosed in Japanese Patent Application Publication No. 000-284895 also uses the optical coordinate input / detection device shown as the third example to distort the display plate when detecting the coordinates of the position similarly designated by the infrared beam. Even if the coordinate input pen is tilted, stable position detection can be performed.

[0006]

However, the conventional technique disclosed in Japanese Patent Laid-Open No. 2000-81951 requires a pen and cannot be operated with a fingertip.
The related art disclosed in Japanese Patent No. 282100 has a problem that the operation is delicate and a problem that the coordinate detection method is limited. Further, in the conventional technique disclosed in Japanese Patent Laid-Open No. 11-110132, there is a problem that the cursor display of a size that allows two or more button touches to be recognized is not practical because the cursor becomes too large. The object of the present invention is to solve such problems of the prior art,
Specifically, an information processing apparatus including a display device with a pointing function, which can be pointed by directly touching with a finger, and can realize an operation corresponding to the left and right buttons of a mouse by an easy-to-understand natural operation To provide.

[0007]

In order to solve the above-mentioned problems, the invention according to claim 1 is provided with a display device with a pointing function capable of inputting coordinates of a touched position on a display screen. In the information processing device, a display device with a pointing function having pointing means for inputting coordinates of a touched position on a display screen,
A control unit that performs control using the coordinates as input information and a coordinate storage unit that stores the coordinates are provided. When a touch operation is performed, the coordinates of the touch position input by the pointing unit are stored in the coordinate storage unit. At the same time, control is performed to display operation buttons on the display screen, and to perform control corresponding to button operation at the coordinates stored in the coordinate storage means in response to a touch operation on a display button area including the displayed operation button. It is characterized in that the means is configured. Further, in the invention according to claim 2, in the invention according to claim 1, there is provided clocking means for measuring the touch time,
If the measured touch time is determined to be within a predetermined time, the input coordinates are acquired, and if it is determined that the touch time is not within the predetermined time, an operation button is displayed on the screen. Configured. Further, the invention according to claim 3 is characterized in that, in the invention according to claim 2, the predetermined time can be set by an operating means. Further, in the invention according to claim 4, in the invention according to claim 1, claim 2, or claim 3, when a touch operation on the display button area is detected, the coordinates stored in the coordinate storage means are cursors. Keep it in place,
Furthermore, if the movement of the touch position is detected in the touch state after touching the display button area, the button display position is moved along with the movement of the touch position,
The control means is configured to move the cursor position at the same relative position with respect to the touch position. Further, in the invention according to claim 5, in the invention according to claim 4, when the touch position reaches the edge of the screen, the button display position is changed, and the next touch operation is performed on the operation button whose display position is changed. Up to this point, the feature is that processing is performed so that the button touch state is continued. Further, in the invention according to claim 6, in the invention according to claim 4, if movement of the touch position is detected while an area other than the display button area in the displayed mouse is touched, the mouse display position It is characterized in that the control means is configured so as to move only.

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a configuration block diagram showing a main part of an information processing apparatus including a display device with a pointing function according to each embodiment of the present invention. As shown in the figure, an information processing apparatus (hereinafter, simply referred to as an information processing apparatus) equipped with a display device with a pointing function of this embodiment includes a display device 1 with a pointing function and a control unit 2, and a display device 1 with a pointing function. Is realized by a touch panel system composed of a display device 3 and a transparent touch panel 4 overlaid on its screen. The control unit 2 also includes a CPU 11 that operates according to a program, a storage unit 12 that stores the program and various data, an input control unit 13 that acquires coordinate information and the like from the touch panel 4, and an image RAM that corresponds to the screen of the display device 3. A display control unit 14 and the like that has display control for writing image data in the image RAM is provided. In this embodiment, the control unit described in the claims is realized by the CPU 11, the input control unit 13, the display control unit 14, etc., the coordinate storage unit is realized by the storage unit 12, and the time measuring unit is shown by the CPU 11 and the figure. Not realized by a timer etc. With such a configuration, in the information processing apparatus according to the first exemplary embodiment of the present invention, at the cursor position shown in FIG.
When the position indicated by the star mark is touched, the CPU 11 of the control unit 2 obtains the coordinate information of the touch position from the touch panel 4 and moves the cursor position to FIG. That is, the display control unit 14 that has acquired the coordinate value of the touch position
2 clears the image data in the cursor image area shown in FIG. 2A in the image RAM, and the image data of the cursor image at the position of the acquired coordinate value corresponding to the position indicated by the star mark in FIG. 2B. To write. In addition, the display control unit 14
Can display an image of a mouse with an operation button near the cursor position as shown in FIG. In the case of using as in the case of FIGS. 2A and 2B described above, since the user simply touches the desired coordinate position directly, the time for touching the same position is extremely short. Therefore, in this embodiment, when the touch time is less than or equal to the predetermined time, the operation corresponding to the click of the left mouse button is performed as described above, and it is detected that there is a touch that exceeds the predetermined time at the same position. In this case, the mouse image is displayed near the cursor as shown in FIG. 2 (c). Since the length of touching time depends on the user, the predetermined time can be set by the operating means. Although not shown, a keyboard or a numeric keypad is provided or displayed, and a value is input using the keyboard or the numeric keypad.

Then, in the mouse display state as shown in FIG. 3A, the user displays the mouse left button as shown in FIG. 3B (strictly speaking, the left button area including the left button. ), The CPU 11 recognizes it and
An operation corresponding to the operation when the left mouse button is clicked at the cursor position shown in FIG. 3B is executed. Further, as shown in FIG. 3C, when the user touches the displayed right mouse button, the CPU 11 executes
The operation corresponding to the operation when the right mouse button is clicked at the cursor position shown in (c) is performed. Note that FIG.
When the touch on the mouse button displayed as shown in (b) and (c) is detected, the cursor position is not moved. Further, in this embodiment, a drag operation (pull movement operation) is possible. The drag operation refers to an operation of moving a mouse button (strictly speaking, an operation button area, the same applies hereinafter) in the displayed mouse. For example, in the Windows system, if the cursor is dragged while the left mouse button is pressed, a rectangular area having a diagonal line from the coordinate at which the drag is started to the coordinate at which the cursor is moved can be generated. Similar functions can be realized by simple drag operations.

FIG. 4A shows the displayed mouse and cursor before dragging. In this state, as shown in FIG. 4B, it is assumed that the user touches the left button of the displayed mouse. At this time, the cursor display position is not changed in this embodiment. Then, when the user performs a drag operation, the mouse display position is moved as indicated by the solid arrow in FIG. 4B according to the coordinates of the touch position detected accordingly, and FIG. As shown in, the cursor position is moved while maintaining the relative positional relationship between the mouse button coordinate position and the cursor coordinate position. In this way, the same drag operation as in the case of using the mouse can be performed. Thus, according to this embodiment, the same right button operation and left button operation as with a mouse can be performed. In the first embodiment described above, depending on the positional relationship between the cursor position and the mouse display position, when the displayed mouse is moved, the mouse button reaches the screen edge before the cursor, so the cursor movement range is limited. I will be lost. Therefore, in the second embodiment of the present invention, when the touch position reaches the edge of the screen, the button display position (mouse display position) is changed, and until the next touch operation on the operation button once released, Process so that the button touch state is still maintained. As shown in FIG. 5, even in this embodiment, in FIGS. 5 (a) and 5 (b),
It operates as shown in FIGS. 4 (a) and 4 (b). But,
In this embodiment, when the mouse display position reaches the screen edge before the cursor position as shown in FIG.
As shown in (d), the mouse display position is changed. When the displayed mouse position changes, the touch position moves away from the mouse button display position, so in the case of a normal drag operation, dragging is complete. However, in this embodiment, it is assumed that the mouse button remains touched until the mouse button is touched until the next touch is made. By doing so, FIG.
As shown in (e), even if the mouse button is touched again at a new mouse display position, as shown in FIGS.
It is possible to carry out the drag operation continued as in (f).

Further, in the third embodiment of the present invention shown in FIG. 6, only the mouse display position is moved by dragging the area other than the displayed mouse button in the first and second embodiments. Let That is, if the control unit 2 detects the movement of the touch position in a state where the region other than the display button region in the displayed mouse is touched, it instructs the display control unit 14 to move only the mouse display position. The image RAM is written in the same manner as described above. By doing so, it is possible to avoid that the original screen display cannot be performed due to the mouse display as shown in the first and second embodiments. Further, in the fourth embodiment shown in FIG. 7, after the mouse display, the display can be canceled. When the user touches an area other than the mouse while the mouse is displayed as shown in FIG. 7A, the control unit 2 recognizes it and recognizes the display control unit 14
To erase the mouse image written in the image RAM.

[0012]

As described above, according to the present invention,
In the invention according to claim 1, when a touch operation is performed on the display screen, the coordinates of the touch position are input, the coordinates are stored, and the operation button is displayed and displayed on the display screen. In response to the touch operation on the display button area including the operation button, the control corresponding to the button operation at the stored coordinates is performed, so that not only the finger can be directly touched to perform the pointing operation, but also an easy-to-understand natural With this simple operation, it is possible to realize the operation equivalent to the left and right buttons of the mouse. In addition, claim 2
In the invention described in claim 1, in the invention described in claim 1, the touch time is measured, and if it is determined that the measured touch time is within a predetermined time, the input coordinates are acquired,
If it is determined that the touch time is not within the predetermined time, the operation button is displayed on the screen, so that the operation button can be displayed by a simple operation, and therefore the effect according to claim 1 can be easily realized. be able to. In the invention according to claim 3, in the invention according to claim 2, since the predetermined time can be set by the operating means,
The predetermined time can be set to an appropriate value according to the operation speed of the user. According to the invention of claim 4, in the invention of claim 1, claim 2, or claim 3,
When a touch operation on the display button area is detected, the cursor is displayed at the stored coordinate position as it is, and if movement of the touch position is detected while the display button area is being touched, the touch position is detected. Since the button display position moves along with the movement and the cursor position moves so as to be the same relative position to the touch position, an easy-to-understand drag operation can be realized. In the invention according to claim 5, in the invention according to claim 4, when the touch position reaches the edge of the screen, the button display position is changed, and until the next touch operation to the operation button whose display position is changed. , It is treated as if the button touch state is kept, so when the touch position reaches the screen edge, the cursor moves to the screen edge by the subsequent touch position movement even if the cursor does not reach the screen edge. The cursor can be positioned anywhere on the screen. Further, in the invention according to claim 6, in the invention according to claim 4, if movement of the touch position is detected while being touched in an area other than the display button area in the displayed mouse, the mouse display position is detected. Since only the mouse moves, the original screen display can be performed for the displayed mouse.

[Brief description of drawings]

FIG. 1 is a configuration block diagram showing a main part of an information processing apparatus including a display device with a pointing function according to each embodiment of the present invention.

FIG. 2 is an explanatory diagram of an information processing apparatus including a display device with a pointing function according to the first embodiment of the present invention.

FIG. 3 is another explanatory diagram of the information processing apparatus including the display device with a pointing function according to the first embodiment of the present invention.

FIG. 4 is another explanatory diagram of the information processing apparatus including the display device with a pointing function according to the first embodiment of the present invention.

FIG. 5 is an explanatory diagram of an information processing apparatus including a display device with a pointing function according to a second embodiment of the present invention.

FIG. 6 is an explanatory diagram of an information processing device including a display device with a pointing function according to a third embodiment of the present invention.

FIG. 7 is an explanatory diagram of an information processing apparatus including a display device with a pointing function according to a fourth embodiment of the present invention.

FIG. 8 is an explanatory diagram of an information processing apparatus including a display device with a pointing function showing an example of a conventional technique.

[Explanation of symbols]

1 Display device with pointing function 2 control unit 3 display devices 4 touch panel 11 CPU 12 Memory 13 Input control section 14 Display control unit

Claims (6)

[Claims] 1. An information processing apparatus comprising a display device with a pointing function capable of inputting coordinates of a touched position on a display screen, comprising pointing means for inputting coordinates of the touched position on the display screen. A display device with a pointing function, a control means for controlling the coordinates as input information, and a coordinate storage means for storing the coordinates, and a coordinate of the touch position input by the pointing means when a touch operation is performed. Is stored in the coordinate storage means, the operation buttons are displayed on the display screen, and the button operation at the coordinates stored in the coordinate storage means is performed in response to the touch operation on the display button area including the displayed operation button. With pointing function, characterized in that the control means is configured to perform corresponding control An information processing apparatus equipped with the shown apparatus. 2. An information processing apparatus including the display device with a pointing function according to claim 1, further comprising a time measuring unit for measuring a touch time, and if it is determined that the measured touch time is within a predetermined time. The display device with a pointing function is characterized in that the control means is configured to obtain the input coordinates and display the operation button on the screen if the touch time is determined not to be within the predetermined time. Information processing equipment. 3. An information processing apparatus comprising the display device with a pointing function according to claim 2, wherein the predetermined time is configured to be set by an operating means. Information processing comprising a display device with a pointing function. apparatus. 4. An information processing apparatus comprising the display device with a pointing function according to claim 1, claim 2, or claim 3, wherein when a touch operation on the display button area is detected, the cursor is stored in coordinates. If the movement of the touch position is detected in the touched state after the display button area is touched, the button display position is moved along with the movement of the touch position. At the same time, an information processing apparatus including a display device with a pointing function, characterized in that the control means is configured to move the cursor position at the same relative position with respect to the touch position. 5. The information processing apparatus including the display device with a pointing function according to claim 4, wherein when the touch position reaches the edge of the screen, the button display position is changed and the operation button whose display position is changed is displayed. An information processing apparatus having a display device with a pointing function, characterized in that processing is performed so that the button touch state is maintained until the next touch operation. 6. The information processing apparatus comprising the display device with a pointing function according to claim 4, wherein movement of the touch position is detected when a region other than the display button region in the displayed mouse is touched. For example, an information processing apparatus having a display device with a pointing function, characterized in that the control means is configured to move only a mouse display position.