JP2011081447A - Information processing method and information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a function of displaying a mouse pointer to be operated by a touch panel without being hidden behind the finger. <P>SOLUTION: The appearance position of a mouse pointer with respect to the finger is automatically changed according to the moving direction and moving speed of the finger. The mouse pointer appears in a desired direction only by quickly moving the finger in the desired direction, thereby giving a natural operation feeling. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information input method and information processing apparatus for inputting coordinates.

  From a device equipped with an interface using a mouse as an input means such as that found in a terminal device of a recent electronic device, it is possible to browse a WEB browser or the like as many browsing applications through the Internet. In this WEB browser, there are cases where hyperlinks are pasted on various pages composed of characters and images. Moving the mouse pointer over a hyperlink changes the shape of the mouse pointer, making it easy for the operator to place the mouse pointer on the hyperlink and click the mouse to jump to that link destination Can be recognized. By clicking on the part on the hyperlink, a new WEB page linked to can be opened and further related information can be browsed.

  In recent years, the resolution of a liquid crystal display screen has increased, and it has become possible to display more characters and images on a WEB browser. As the resolution increases and the display size decreases, characters and images displayed on the screen are becoming finer. No matter how fine a character or image, the mouse can point to any part of the screen.

  On the other hand, in a device such as a portable device that is operated with a hand, it is necessary to perform a screen operation while being carried. Therefore, rather than an interface using a mouse, an interface having a transparent touch panel on the display of the apparatus and directly touching a part on the screen is widely used. An interface using a touch panel is effective as a means for quickly indicating an operation target placed on a target screen because it can intuitively operate a place desired to be pointed instantaneously.

  Therefore, it has become necessary to indicate such small characters and images in an apparatus using a touch panel and having means for displaying an instruction display such as a mouse pointer on the screen. FIG. 1 shows a positional relationship between coordinates at which the touch panel is pressed with a finger and coordinates at which the mouse pointer is displayed on the liquid crystal display unit. As shown in FIG. 1, the touch panel 1 is bonded to the liquid crystal display unit 2, and the operator places a finger on the touch panel 1. Based on the detected coordinates 4 on the touch panel, the apparatus displays the mouse pointer 5 as an image on the liquid crystal display unit 2 so that the detected coordinates 4 on the touch panel and the coordinates 6 pointed to by the mouse pointer are at the same position. FIG. 2 is a perspective view when the operator views the coordinates at which the touch panel 1 is pressed with the finger of FIG. 1 and the coordinates at which the mouse pointer is displayed. From the viewpoint of the operator, the operation is performed with the consciousness that the mouse pointer is directly under the finger as shown in FIG.

  However, since small characters and hyperlinks that are displayed on a high-resolution display screen in recent years are smaller than the fingertips, the mouse pointer 5 is under the finger of the user even when pointing with the finger. It is hidden behind and cannot be seen. Therefore, it is impossible to identify whether or not the hyperlink is in a selected state by changing the shape of the mouse pointer.

  In order to solve this problem, the following method has been conventionally used.

  The first method is to interrupt the operation of the touch panel or not to have the touch panel in the device. For example, a mouse pointer is displayed on the screen using a mouse to indicate a target hyperlink. .

  In a similar method, there is a method in which a mouse pointer is displayed remotely without using a mouse and the same operation is performed. For example, a commander capable of moving a cursor such as a remote controller of a television or a cross key of a video game is applicable.

  The second method is not to directly operate the transparent touch panel on the display screen with a finger, but to prepare a pen-like stylus that is thinner than the finger and point to a target hyperlink.

  The above first and second methods try to solve the problem by preparing a tool for another input means such as a mouse or a pen. There was also the hassle of bringing back a hand for a tool for another input means, and problems such as being unable to use if the tool was lost occurred.

The third method includes means for displaying a mouse pointer around the finger so that the mouse pointer is not hidden under the finger (see, for example, Patent Document 1). Further, the display position and orientation of the mouse pointer are configured to change depending on the area of the screen where the mouse pointer is located.
As long as the mouse pointer is directly under the finger, the mouse pointer is hidden behind the finger. In order to solve this problem, Patent Document 1 shows a state in which the mouse pointer is displayed on the left side of the finger when the finger is placed at the center of the screen as shown in FIG. In this state, the operator can see the mouse pointer displayed on the left side of the finger without being hidden by the finger. However, if the finger is moved to the right and the finger is moved to the right edge of the screen as shown in FIG. 4 or is placed on the right edge of the screen, the region shown in the hatched area 51 in FIG. It cannot be indicated by a pointer. FIG. 4 shows a state where the mouse pointer is displayed on the left side of the finger when the operator places his finger on the right edge of the screen. FIG. 5 shows a range in which the mouse pointer cannot point when a finger is placed on the right end of the screen shown in FIG.

  In order to solve these problems, the solution means of Patent Document 1 is that the finger touches the area near the right edge of the screen as shown in FIG. 6 in order to point the hatched area 51 at the right edge of the screen as shown in FIG. In this case, the mouse pointer 61 is displayed at the position of the hatched area 51 in FIG. 6 on the right side of the finger. 6 is an explanatory diagram in which the mouse pointer 61 is displayed on the right side of the finger by moving the finger to the left side, and the range 51 that cannot be indicated by the mouse pointer in FIG. 5 is indicated. With this method, the area at the right end of the screen can be pointed with the mouse pointer.

  Another method is to display the mouse pointer above the finger so that it is not hidden by the finger (see, for example, Patent Document 2). However, the mouse pointer protrudes from the screen near the top of the screen. Therefore, the mouse pointer is displayed directly below the finger, not above the finger, near the upper end of the screen. As a result, an area where the mouse pointer is hidden with a finger remains on the screen.

  Again, with respect to Patent Document 1, FIG. 7 shows a state in which the mouse pointer is pointing to the left side of the finger with the finger placed at the same position as in FIG. When the finger is placed at the same position as the finger touch position in FIG. 6 as shown in FIG. 7, it may be desired to display the mouse pointer on the left side of the finger (mouse pointer 71). That is, there arises a problem that it is desired to point two mouse pointer positions at the same finger position.

JP-A-11-24841 Japanese Patent Laid-Open No. 9-237157

  However, in the conventional technique described above, in the first method and the second method, when used in a portable device or the like, the stylus is lost or the convenience is reduced due to having to carry many extra functions. Can be considered.

  In the third method, the mouse pointer protrudes from the screen near the upper end of the screen. Therefore, near the upper end of the screen, the mouse pointer is displayed directly below the finger, not above the finger. Therefore, an area where the mouse pointer is hidden with a finger remains on the screen.

  Many applications have screen configurations in which many touch-selectable buttons and hyperlinks are scattered on the entire screen. Such an application may want to display the mouse pointer to the left of the finger when the finger is placed at the same position as when the mouse pointer is to be displayed at the right position of the finger at the finger touch position. That is, there arises an inconvenience of pointing to two different mouse pointer positions at the same finger position.

  In Patent Document 1, a mouse cursor appears outside the screen from the finger near the edge of the screen. Depending on where the operator presses the touch panel, the mouse cursor is automatically displayed in the left / right, up / down, and diagonal directions of the finger, but the mouse pointer is not always displayed where the operator intended. It is clear from the above description.

  Further, in Patent Document 2, as soon as the finger is released, the mouse pointer is returned directly below the finger. When the finger is touched again, the mouse pointer is displayed above the finger. If the position of the mouse pointer changes when the finger is released, there is a problem that the operator cannot know the position pointed immediately before releasing the finger. Further, in the method of Patent Document 2, when the tap or double tap is performed, the position pointed to by the mouse pointer moves back and forth between the two directly below and above the finger, so it cannot be determined which position is selected.

  In order to solve the above-described problems, the present invention provides a mouse pointer display means that can easily point to an arbitrary position on a screen without using a stylus or a mouse pointer by using a touch panel.

  The invention as a method is shown below.

  As a first method, in an information processing method for detecting input coordinates, input information relating to input coordinates is obtained from an information input device, input information is compared with information recorded in a work memory, and input is performed from a temporal change. Judgment is made after calculating the movement direction of the coordinates and the movement speed of the input coordinates, the input information is recorded in the work memory, the movement speed is compared with a preset threshold value, and if the movement speed exceeds the threshold value, the display area starts from the movement direction Selects the appearance direction of the mouse pointer to be displayed at, and outputs the display coordinates of the mouse pointer.

  As a second method, in the first method, while the input information is continuously detected by the information input device, the appearance direction of the mouse pointer is held in a fixed direction.

  As a third method, in the first method, while the input information is continuously detected by the information input device, the appearance direction of the mouse pointer is continuously updated.

  As a fourth method, in the first to third methods, when the input information is not detected by the information input device and the input information is detected again during a predetermined period, the appearance direction of the mouse pointer is the previous time. Keep the direction.

  As a fifth method, in the first to third methods, when the input information is not detected by the information input device and the input information is detected again after a predetermined period, the direction of appearance of the mouse pointer is changed. Update to the initial position.

  As a sixth method, in the first to third methods, the appearance direction of the mouse pointer is the same as the moving direction.

  As a seventh method, in the first to third methods, the appearance direction of the mouse pointer may be any of the four types of positions of the upper right, lower right, upper left, and lower left of the input coordinates based on the direction of the moving direction. Or more selected.

  Moreover, the invention as a means is shown below.

  As a first means, the information processing apparatus calculates input coordinates based on a touch panel, a touch panel control unit, a liquid crystal display unit, a display control unit, an interface for outputting information input from the touch panel, and a signal from the interface. CPU, a program memory for recording a program for calculating input coordinates, a storage unit for recording information when the CPU is operated, and a work memory used as a working area for all control programs such as communication, display, calculation, and input And In the aforementioned program memory, input information related to the input coordinates is obtained from the information input device, the input information is compared with the input information recorded in the work memory, and the movement direction of the input coordinates and the movement of the input coordinates are detected from the temporal change. Determine the speed, record the input information in the work memory, compare the moving speed with a preset threshold, and if the moving speed exceeds the threshold, select the appearance direction of the mouse pointer to display in the display area from the moving direction, and A program for outputting the display coordinates of the pointer is stored.

  As a second means, while the input information continues to be detected on the touch panel in the first means, the appearance direction of the mouse pointer is held in a fixed direction.

  As a third means, while the input information continues to be detected on the touch panel in the first means, the appearance direction of the mouse pointer is continuously updated.

  As a fourth means, in the first to third means, when the input information is not detected on the touch panel and the input information is detected again during a predetermined period, the appearance direction of the mouse pointer is held last time. Direction.

  As a fifth means, in the first to third means, when the input information is not detected on the touch panel and the input information is detected again after a predetermined period of time, the appearance direction of the mouse pointer is set to the initial position. Update to

  As a sixth means, in the first to third means, the appearance direction of the mouse pointer is the same as the direction of the movement direction.

  As a seventh means, in the first to third means, the appearance direction of the mouse point is any of the four types of positions of the upper right, lower right, upper left, and lower left of the input coordinates based on the direction of the moving direction. Choose.

  In the conventional method of determining the appearance position of the mouse pointer with respect to the finger from the position of placing the finger on the touch panel, it is difficult for the operator to know where the mouse pointer appears around the finger. In addition, since the entire area of the display screen cannot be pointed, the operator may be confused by having to switch the appearance of the mouse pointer to the finger with a switch or the like, but the present invention wants to point to the mouse pointer. Since the mouse pointer appears in the intended direction simply by moving the finger faster in the direction, a natural feeling of operation can be obtained.

  In addition, since the function of the present invention can be realized only by being mounted on the touch panel driver side, the function can be used in place of a conventional touch panel without changing the OS and application that control the coordinate of the screen. Therefore, it also has the advantage of realizing this function with a small development investment.

The figure explaining the positional relationship as which the coordinate and the mouse pointer which pressed the touch panel are displayed. The figure explaining the case where a mouse pointer is displayed just under a finger. The figure which shows the state in which the mouse pointer is displayed on the left when putting a finger in the center of the screen. The figure which shows that a mouse pointer is displayed on the left side of a finger when a finger is placed on the right edge of the screen. The figure explaining the range which cannot be pointed with a mouse pointer. The figure explaining the unnaturalness of the operativity of the solution method by patent document 1. FIG. The figure explaining the unnaturalness of the operativity of the solution method by patent document 1. FIG. The figure explaining the flow which determines the display position of a mouse cursor. The figure which shows the appearance coordinate of a mouse pointer in the place away from the finger. The figure explaining that the display position of a mouse pointer changes when a finger | toe moves in the X direction at the speed | rate larger than Vk or the speed smaller than -Vk. The figure explaining that the display position of a mouse pointer changes when a finger | toe moves in the Y direction at the speed | rate larger than Vk or the speed smaller than -Vk. The figure explaining the range of the speed where the appearance position of a mouse pointer changes in an XY coordinate system. The figure explaining that the display position of a mouse pointer changes when a finger | toe moves at the speed larger than Vk in both X and Y directions, and the speed smaller than -Vk. The figure explaining the movement vector of a finger | toe, and the appearance position of a mouse pointer. The figure explaining the Example of a structure of an apparatus. The figure explaining the mouse pointer which appears on a screen. The figure explaining that the shape changed when the mouse pointer selected the item. The figure explaining that it becomes invisible because a mouse pointer is displayed under a finger when an item is selected with a finger. The figure explaining that a mouse pointer is displayed on the upper left of a finger, when an item is selected with a finger. The figure explaining that a mouse pointer changes and is displayed on the upper left of a finger when an item is selected. The figure which shows the range which a mouse pointer can point on the screen. The figure which shows the range which a mouse pointer cannot point on the screen. The figure explaining that the mouse pointer displayed on the upper left of a finger cannot point to the hyperlink on the upper right of the screen. The figure explaining pointing the hyperlink of the screen upper right by switching a mouse pointer from the upper left of a finger to the upper right. The figure explaining a mode that a mouse pointer is operated with a three-dimensional coordinate.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated with reference to drawings.

The present invention provides a new means for pointing with a finger a character smaller than a fingertip displayed on a liquid crystal screen or the like, a hyperlink attached to an image, or the like. In the following example, a character smaller than a fingertip displayed on a transparent touch panel on a display screen, and a hyperlink stretched on an image is directly operated with a finger to describe the means of the present invention as an example. And
In the following explanation, it is explained as an instruction means for pointing an instruction display position to a place where the mouse pointer is intentionally pointed on the screen. However, a pointer indicating a cursor or an area that is the same instruction means, or a color of the indicated area The instruction means may be reversed.

  According to the present invention, a mouse pointer is displayed around the finger instead of displaying the mouse pointer directly under the finger.

  The relationship between the position of the finger and the display position of the mouse pointer is displayed at a position separated by a coordinate distance with a predefined offset value, and the display position of the mouse pointer is displayed from the movement direction and movement speed of the finger. The position shall be determined.

  In describing the operation, the following definition is made for convenience.

  FIG. 8 shows an example of a specific processing flow of the present invention.

  In FIG. 1, the detected coordinates 4 of the touch panel are defined as coordinates (X0, Y0), and the coordinates 6 pointed to by the mouse pointer are defined as coordinates (X1, Y1).

  In the process 81 of FIG. 8, the detection coordinate 4 (X0, Y0) of the touch panel is obtained.

FIG. 2 shows that when the operator looks at the coordinate 6 pointed to by the mouse pointer from above the finger, the detected coordinate 4 (X0, Y0) of the touch panel is the same as the coordinate 6 (X1, Y1) pointed to by the mouse pointer, FIG. 2 explains that it is in a state of being calibrated as shown in equation (2).
(Equation 1) X1 = X0
(Equation 2) Y1 = Y0
This is the same concept as the coordinates operated on a touch panel using a commonly used pen. That is, the detected coordinates 4 (X0, Y0) of the touch panel are converted into logical coordinates on the display, and the application software selects a button or a hyperlink from the coordinates 6 (X1, Y1) indicated by the mouse pointer. You can know whether or not.

  As an example of the present invention, as shown below, logical coordinates on the display are displayed at relative offset positions (Xa, Ya) that are separated from the detected coordinates 4 (X0, Y0) of the touch panel by the size of the fingertip or more. To do.

That is, the following equations 3 and 4 are given.
(Equation 3) X1 = X0 + Xa
(Expression 4) Y1 = Y0 + Ya
The relationship between the detected coordinates 4 (X0, Y0) of the touch panels of Equations 3 and 4 and the coordinates 6 (X1, Y1) indicated by the mouse pointer is illustrated in the positional relationship shown in FIG.

  In the process 82 of FIG. 8, the moving direction and moving speed of the finger are obtained.

  It is assumed that the appearance coordinates (X1, Y1) of the mouse pointer are determined by Equations 3 and 4 according to the moving speed while the touch panel is kept pressed.

  A specific method for setting Xa and Ya will be described.

  Xa and Ya are assumed to have preset initial values Xa0 and Ya0.

That is, the initial values of Xa and Ya are expressed by the following equations 5 and 6.
(Formula 5) Xa = −Xa0
(Equation 6) Ya = −Ya0
FIG. 9 shows the relationship between the coordinates of the finger and the mouse pointer when the initial values set in advance are in the formulas 5 and 6.

  If the initial values Xa0 and Ya0 are both positive constants, the coordinates (X0, Y0) at which the touch panel is pressed are related to the logical coordinates (X1, Y1) on the display as shown in FIG. A mouse pointer is displayed in the upper right.

  The function of the present invention is that Xa and Ya change when the temporal change in coordinates when the touch panel is pressed exceeds a predetermined threshold value. It is assumed that the time-dependent change of the coordinates where the touch panel is pressed, that is, the moving speed is (Vx, Vy). At reference numeral 83 in FIG. 8, a process for determining the moving speed of the finger is performed.

Xa and Ya change under the conditions of the following equations 7 to 10.
(Expression 7) When Vx> Vk, Xa = Xa0
(Formula 8) When Vx <−Vk, Xa = −Xa0
(Equation 9) When Vy> Vk, Ya = Ya0
(Equation 10) When Vy <−Vk, Ya = −Ya0
When Vx exceeds a predetermined constant Vk, Xa becomes a positive number Xa0 as shown in Equation 7.

  10, FIG. 11 and FIG. 13 show the relationship between the coordinates (X0, Y0) touched by the finger and the mouse pointer (X1, Y1) based on the equations (7) to (10). Regarding the movement in the X direction, FIG. 10B shows that the mouse pointer changes the display position in the right direction of the finger when the finger moving speed condition shown in Equation 7 is satisfied. Similarly, FIG. 10A shows that the mouse pointer changes its display position to the left of the finger when the finger movement speed condition shown in Equation 8 is satisfied. As for movement in the Y direction, FIG. 11B shows that the mouse pointer changes the display position in the downward direction of the finger when the finger movement speed condition shown in Equation 9 is satisfied. Similarly, FIG. 11A shows that the mouse pointer changes the display position in the upward direction of the finger under the condition of the finger moving speed expressed by Equation (10).

  FIG. 12 illustrates a range in which the speed in the X and Y directions and the appearance position of the mouse pointer with respect to the finger change in the XY coordinate system. FIG. 12A shows the range of change of the position where the mouse pointer appears in the X direction as hatched areas 124 and 125. In the hatched area 124 smaller than the position of Vx = −Vk indicated by 122, Equation 8 is applied. Is done. In the hatched range 125 larger than the position of Vx = Vk indicated by 123, Expression 7 is applied.

  FIG. 12B shows the range of change of the position where the mouse pointer appears in the Y direction as hatched areas 128 and 129. In the hatched area 128 smaller than the position of Vy = −Vk indicated by 126, Equation 10 is applied. Is done. In the hatched area 129 larger than the position of Vy = Vk indicated by 127, Expression 9 is applied.

  FIG. 13 shows that the positional relationship between the finger and the mouse pointer changes in the X and Y directions when the movement speeds Vx and Vy of the finger in the X and Y directions exceed −Vk or Vk. To explain the relationship with FIG. 12, the mouse pointer is displayed at the position shown in FIG. 13A or 13C in the range 124 in FIG. 12A, and the range 125 shown in FIG. Then, a mouse pointer is displayed at the position shown in FIG. 13B or FIG.

In the range of 128 in FIG. 12B, the mouse pointer is displayed at the position of FIG. 13A or 13B, and in the range of 129 in FIG. 12B, FIG. 13C or FIG. The mouse pointer is displayed at the position).
As a result, the appearance position of the mouse pointer is assumed to appear in any of the four locations at the upper left, the lower left, the upper right, and the lower right at a predetermined distance from just below the finger as shown in FIG.

  The process of selecting the appearance position of the mouse from the finger moving direction is indicated by process 84 in FIG. Further, in the process 85 of FIG. 8, the mouse pointer is displayed at the position indicated by the coordinates (X1, Y1).

  FIG. 14 shows an example in which the number of appearance candidates for the mouse pointer is increased.

  In the above embodiment, the appearance position of the mouse pointer is limited to four places around the finger. However, the claims of the present invention need not limit the appearance position to four places. For example, as shown in FIG. It is obvious that there may be several appearances around the and the appearance position is selected by the moving direction and moving speed of the mouse pointer. That is, when the moving speed vector exceeds a certain value, the position at which the mouse pointer appears may be determined from information on the direction of the moving speed vector.

  Specifically, when the scalar amount of the finger movement vector exceeds Vk, the appearance location of the previous mouse pointer is set as a new display location.

  Next, the finger movement vector is obtained as a scalar quantity in the direction of each movement vector candidate, and the largest one is set as a candidate for the appearance location of the mouse pointer.

  Alternatively, the above-described XY coordinate plane may be reviewed as polar coordinates, and the one with the finger movement vector angle closest to the appearance location of the mouse pointer may be used as the appearance location candidate.

  Specific means will be described with reference to FIG. Reference numeral 140 denotes a finger movement vector, which indicates the movement direction and movement speed of the finger. The mouse pointer A (142) to the mouse pointer H (149) are candidates for appearance of mouse pointers as position candidates 141 having a distance from the finger touch position where the mouse pointer is not hidden by the finger.

  Next, from the candidates of the mouse pointer A (142) to the mouse pointer H (149) in order to newly select the appearance position of the mouse pointer when the scalar amount of the finger movement vector 141 exceeds the set amount Vk. Select one of them.

FIG. 14 shows a method for selecting which of the mouse pointer A (142) and the mouse pointer B (143) is selected for easy understanding. The scalar amounts of the mouse pointer A (142) and the mouse pointer B (143) of the finger movement vector 141 are respectively perpendicular to the finger movement vector 141, and each amount is a scalar amount in the 142 direction of the finger movement vector. (142a) and the scalar movement amount (143a) of the finger movement vector in the direction 143. Expression 11 is shown in FIG.
(Expression 11) 142a <143a
It is determined that the scalar quantity in the vector direction 143 is larger than the scalar quantity in the vector direction 142, and the mouse pointer B (143) is selected. As a result, the mouse pointer B (143) in the direction 143 is selected and displayed. When the scalar amount of the finger movement vector 141 does not exceed the set amount Vk, it is assumed that the relative position between the detected coordinates of the touch panel and the mouse pointer display position does not change.

In the present invention, when the finger is separated for a predetermined time t or longer, the mouse pointer is always displayed at a predetermined position such as the upper left of the finger. In other words, it is only necessary to set as in Expression 12 and Expression 13.
(Formula 12) Xa = −Xa0
(Equation 13) Ya = −Ya0
This function also works to distinguish from operations such as tapping.

  For example, the mouse pointer changes shape when pointing to a hyperlink. Thereafter, the operator taps this. At this time, the finger leaves the touch panel for a short time. In this case, if the appearance position of the mouse pointer changes, the operator is confused. Therefore, the time t may be set longer than the interval time recognized as a tap. By doing so, the appearance position of the mouse pointer is not changed by the tap and remains in the same place, so that the operator can recognize the target selected by the mouse pointer and execute the tap.

  If it is convenient for the display position of the mouse pointer to be fixed at the upper left position of the finger due to the nature of the application software, the function of the present invention may be turned off. For example, in an application where it is difficult to move the appearance position of the mouse pointer by moving the finger, such as drawing or character input, the function of the present invention may be turned off so that the mouse pointer is always displayed on the upper left of the finger.

  The touch panel control unit acquires the coordinates where the finger is pressed. When the finger moves while being pressed, the moving direction and moving speed of the finger can be obtained from temporal changes in coordinates. The appearance position around the finger of the mouse pointer is selected from the moving direction, and the coordinates of the mouse pointer are determined at the selected appearance position spaced from the finger when the moving speed exceeds a certain value. The display control unit displays the mouse pointer at the obtained mouse pointer position. The OS performs control based on the obtained position information of the mouse pointer.

The present invention is suitable for use such as displaying a large amount of character information on a screen such as a portable terminal and selecting it with a mouse pointer.
For example, it is also effective for an application in which there are icons and characters in the operation of a WEB browser of a mobile phone and the map display displayed on the PDA, and there is a hyperlink pointing to it with a mouse pointer.

The liquid crystal display unit 2 in FIG. 1 is a liquid crystal display of any size such as a several inch size used in a mobile phone or a 10 inch size such as a PDA terminal. The display unit may be not only a liquid crystal but also a cathode ray tube monitor such as a CRT.
The touch panel 1 is a touch panel capable of transparent coordinate detection in which a display content can be seen through a liquid crystal display by a capacitance method, a resistance film method, an ultrasonic method, a temperature detection method, an electromagnetic induction method, or the like.

  Moreover, the liquid crystal display body which integrated the liquid crystal display part 2 and the touchscreen 1 may be sufficient.

  FIG. 15 is a configuration example of an embodiment of the present invention, which will be described.

  The touch panel 1 is bonded to the display upper surface of the liquid crystal display unit 2. The liquid crystal display unit 2 receives the display data 158 from the display control unit, drives the liquid crystal and displays it. When the operator touches the touch panel 1 on the touch panel 1, detection information 157 of the finger 3 is passed to the touch panel control unit 155. The detection information 157 of the finger 3 includes an electrical signal that is a source of information such as finger distance information, the strength of pressing the finger, the area, and the number of fingers.

  The touch panel control unit extracts the XY coordinates of the finger, the strength of pressing the finger, the area, the number of fingers, and the like as the coordinate data 159 by calculation processing from the detection information of the finger 3.

  The coordinate data 159 is transmitted to the CPU 150 via the touch panel interface 153. As the touch panel interface 153, USB, PS2 interface, I2C-BUS, synchronous type, asynchronous serial type, and the like are used as general interfaces.

  The CPU 150 stores a program for controlling the operation in the program memory, and reads and executes it. Some apparatuses have an OS loaded as software, and in many cases, an OS program is stored in a storage unit. The storage unit is a semiconductor memory or hard disk similar to the program memory 152 described above. The CPU 150 uses the work memory 154 as a working area for all control programs such as communication, display, calculation, and input. The work memory 154 is a semiconductor memory, and SRAM and DRAM are mainly used today.

  The display control unit 156 outputs a display image as a video to a display device such as an LCD or a monitor that is a connected output device. The display control unit 156 prepares a display memory as an image to be displayed as a screen or a plurality of buffers, and scans and reads this according to a display cycle, and sends it to a display device. The CPU 150 writes the mouse pointer image together with the background image to the display memory. Then, the display control unit 156 sends this to the display device as display data, and displays the mouse pointer and its background image on the display device.

  FIG. 16 is a diagram illustrating an example of an ordering terminal in which a waitress used at a family restaurant, a pub, or the like places an order for cooking.

  FIG. 16 shows a state in which an order menu for three products is displayed on the ordering terminal. The waitress selects any one of the three items using the touch panel, and selects one of the hyperlink steak 161, hamburger 162, and salad bar 163 displayed on the screen. The mouse pointer 164 is in a state in which none of the hyperlinks described above is selected.

  FIG. 17 is a diagram for explaining when the mouse pointer selects an order item in the order order terminal of FIG.

  FIG. 17 shows that the mouse pointer 171 has selected the hamburger 162, and the shape of the mouse pointer 171 has changed from the mouse pointer 164 of FIG. 16 to a finger mark.

  FIG. 18 is a diagram when the order item on the order ordering terminal is pressed and selected with a finger.

  However, the ordering terminal carried by the waitress tends to have a small device shape, a small screen, and a small menu to be displayed with emphasis on portability. Further, when the three menus on the screen with hyperlinks are displayed in a small size, there arises a problem that the mouse pointer is hidden under the finger 3 as shown in FIG. Therefore, even if a hamburger is selected under the finger 3 and the mouse pointer shape is changed as indicated by 171 in FIG. 17, the waitress cannot see it.

  In order to avoid this problem, if the touch panel is operated using a small pen instead of a finger, the changing mouse pointer 171 as shown in FIG. 17 can be confirmed from the gap of the pressed pen. However, it takes time and labor to take out the pen each time an order is placed, and there is a risk of losing the pen, which hinders operation. As a result, the use of the portable terminal using the pen having such an inconvenient specification is limited.

  When a problem when a finger is used to operate a conventional touch panel is described with reference to FIG. 18, it can be understood that the mouse pointer is hidden under the finger 3 and is inconvenient.

  FIG. 19 is a diagram for explaining the case where the mouse pointer is displayed at the upper left of the finger.

  FIG. 20 is a diagram showing that the mouse pointer 211 is selected by moving the finger 3 so that the mouse pointer moves to the order to be ordered.

  FIG. 19 shows a case where the mouse pointer 191 is always displayed on the upper left of the finger 3. The relationship between the finger 3 and the mouse pointer 191 is displayed at a distance away from the finger 3 with a distance that is not hidden by the finger as shown in FIG. By using such means, it becomes possible to confirm that the shape of the mouse pointer changes by moving the finger to a position for selecting a hamburger as shown in FIG.

  It can be seen that the mouse pointer displayed in the upper left of the finger in this way can point to the range of the display screen as shown in FIG. If it is desired to point to the upper left of the screen, the finger is placed at the position 211 and the mouse pointer is displayed at the position 215. When pointing to the lower left of the screen, the finger is placed at 212 and the mouse pointer is displayed at 216. When pointing to the upper right of the screen, the finger is placed at the position 213 and the mouse pointer is displayed at the position 217. When pointing to the lower right of the screen, the finger is placed at the position 214 and the mouse pointer is displayed at the position 218. However, there is a range that cannot be indicated as shown by the hatched range 219 in FIGS.

  FIG. 23 shows an example in which a hyperlink for page movement is provided at the upper end of the screen of the order order terminal. In such an application, since the mouse pointer 231 is displayed at the upper left of the finger 232, the hyperlink 234 marked “next page” at the upper right of the screen cannot be pointed by the mouse pointer 231.

  FIG. 24 shows an example in which the finger is moved on the screen of the order order terminal and the mouse pointer is successfully pointed to the hyperlink at the upper right end. In the present invention, as shown in FIG. 24, the finger is moved from the position 242 to the position 244 in the direction indicated by 243 over the speed determined in the upper right direction. Thereafter, the position of the mouse cursor 245 is displayed on the upper right of the finger. The operator can gradually point while moving to a position pointing to the hyperlink 234 marked “next page”.

  In the description of the present invention, the transparent touch panel has been described as an example of the touch position detection function. However, as a category of the touch position detection function, for example, a finger touch position detection function that can be operated in three dimensions may be defined. For example, the mouse pointer is displayed around the finger so that the operator does not hide the mouse pointer with the finger from the position of the finger detected by the touch position detection function that touches the image that appears floating in three dimensions with the finger. It can also be used in configurations.

  FIG. 25 is not limited to the case where there is a touch panel to be operated on the display screen, but also when the display image and the mouse pointer are operated with three-dimensional coordinates such that they exist in a three-dimensional space having X, Y, and Z axes. It explains what can be used. Reference numerals 256, 257, and 258 are images so that an image that is an object to be pointed exists in three-dimensional coordinates, and a strawberry image, a grape image, and a peach image are displayed, respectively. The operator's finger 251 can be operated in a three-dimensional direction to point to them. However, although the operator's line of sight 254 looks at the finger, even if a mouse pointer exists beyond the line of sight 254, it cannot be confirmed by hiding the finger that the image is the object. If the operator wants to point to the image 256, the finger is quickly moved in the XY coordinate direction in the 256 direction to display the mouse pointer at the position of the mouse pointer 252, and further slowly point to the image 256 of the object. By operating the finger in this manner, the image 256 of the object can be pointed in the direction of the line of sight 253 without hiding the mouse pointer 252 behind the finger 252.

  In addition, the appearance direction of the mouse pointer is not limited to the XY coordinate plane on the three-dimensional coordinate, and the mouse pointer appears in a direction corresponding to the moving direction on the three-dimensional coordinate by adding the Z-axis direction. You can also.

DESCRIPTION OF SYMBOLS 1 Touch panel 2 Liquid crystal display part 3 Finger 4 Touch panel detection coordinate 5 Mouse pointer 6 Mouse pointer coordinate 21 Mouse pointer in the same coordinate as the touch panel detection coordinate 31 Mouse pointer on the left side of the finger 41 Mouse pointer on the left side of the finger 51 Mouse pointer non-selectable range 61 Mouse pointer displayed on the right side of the finger 71 Mouse pointer displayed on the left side of the finger 81 Processing for obtaining the touch coordinates of the finger 82 Processing for obtaining the moving direction and moving speed of the finger 83 Moving speed is threshold Processing 84 for determining whether the value has been exceeded Processing 84 for selecting the mouse pointer appearance position from the moving direction Processing 91 for displaying the mouse pointer Y-axis offset distance Ya
92 X-axis offset distance Xa
101 Mouse pointer displayed at the upper left of the finger 102 Mouse pointer displayed at the upper right of the finger 103 Offset distance in the negative direction of the X axis Xa = −Xa0
104 X-axis positive offset distance Xa = Xa0
111 Mouse pointer displayed on the upper right of the finger 112 Mouse pointer displayed on the lower right of the finger 113 Offset distance in the positive Y-axis direction Ya = Ya0
114 Offset distance in the negative Y-axis direction Ya = -Ya0
121 Position of velocity 0 in both XY directions 122 Vx = −Vk position 123 Vx = Vk position 124 Area of velocity in X direction to which Formula 8 is applied (shaded portion)
125 Area of velocity in X direction to which number 7 is applied (shaded area)
126 Vy = −Vk position 127 Vy = Vk position 128 Region of speed in Y direction to which number 10 is applied (shaded portion)
129 Area of velocity in Y direction to which number 9 is applied (shaded part)
131 Mouse pointer displayed at the upper left of the finger 132 Mouse pointer displayed at the upper right of the finger 133 Mouse pointer displayed at the lower left of the finger 134 Mouse pointer displayed at the lower right of the finger 135 Offset distance in the Y axis positive direction Ya = Ya0
136 Offset distance in the negative X-axis direction Xa = -Xa0
137 Offset distance in X axis positive direction Xa = Xa0
138 Offset distance in the negative Y-axis direction Ya = -Ya0
140 Finger movement vector 141 Mouse pointer position candidate 142 Mouse pointer display position A
142a Scalar amount 143 of finger movement vector in 142 direction Mouse pointer display position B
143a Scalar amount 144 of finger movement vector in 143 direction Mouse pointer display position C
145 Mouse pointer display position D
146 Mouse pointer display position E
147 Mouse pointer display position F
148 Mouse pointer display position G
149 Mouse pointer display position H
150 CPU
151 Storage unit 152 Program memory 153 Touch panel interface 154 Work memory 155 Touch panel control unit 156 Display control unit 157 Finger detection information 158 Display data 159 Touch panel detection coordinate data 161 to 163 Hyperlink character 164 Mouse pointer 171 when hyperlink is not selected Mouse pointer 191 when hyperlink is selected Mouse pointer 201 when hyperlink is not selected Mouse pointer 211 when hyperlink is selected 211 Upper left finger 212 Lower left finger 213 Upper right finger 214 Right lower finger 215, 216, 217 , 218 Mouse pointer 219 displayed at upper left of finger 219 Mouse pointer non-selectable range 231 Mouse pointer 232 when hyperlink is not selected Finger 233, 234 Hyperlink character 41 before moving the mouse pointer 242 moves the mouse pointer 251 finger 252 mouse pointer 253 operator's viewpoint A before the finger 243 after the finger 245 moves after movement vector 244 moves the finger
254 Operator's point of view B
255 Operator's Eye 256 Strawberry Image 257 Grape Image 258 Peach Image

Claims (14)

In an information processing method for detecting input coordinates,
Obtain input information about input coordinates from the information input device,
Compare the input information with the information recorded in the work memory, and determine after calculating the moving direction of the input coordinates and the moving speed of the input coordinates from the change over time,
Recording the input information into the work memory;
Compare the moving speed with a preset threshold value,
If the moving speed exceeds the threshold, select the appearance direction of the mouse pointer to be displayed in the display area from the moving direction,
An information processing method for outputting display coordinates of a mouse pointer.   The information processing method according to claim 1, wherein the appearance direction of the mouse pointer is held in a fixed direction while input information is continuously detected by the information input device.   The information processing method according to claim 1, wherein the appearance direction of the mouse pointer is continuously updated while input information is continuously detected by the information input device.   The input direction of the mouse pointer is the direction held last time when the input information is no longer detected by the information input device and the input information is detected again during a predetermined period. Information processing method according to item.   The input direction of the mouse pointer is updated to the initial position when the input information is no longer detected by the information input device and the input information is detected again after a predetermined period has elapsed. Information processing method according to item.   The information processing method according to claim 1, wherein an appearance direction of the mouse pointer is the same as a direction of the movement direction.   The appearance direction of the mouse pointer is selected from any one of four types of positions of the upper right, lower right, upper left, and lower left of the input coordinates based on the direction of the moving direction. The information processing method according to item 1. A touch panel;
A touch panel control unit;
A liquid crystal display,
A display control unit;
An interface for outputting information input on the touch panel;
A CPU that calculates input coordinates based on signals from the interface;
A program memory for recording a program for calculating the input coordinates;
A storage unit for recording information when operating the CPU;
Work memory used as a working area for all control programs such as communication, display, calculation, and input,
In an information processing apparatus that is
In the program memory,
Obtain input information about input coordinates from the information input device,
Compare the input information with the input information recorded in the work memory, determine the moving direction of the input coordinates and the moving speed of the input coordinates from the change over time,
Recording the input information in the work memory;
Compare the moving speed with a preset threshold value,
If the moving speed exceeds the threshold, select the appearance direction of the mouse pointer to be displayed in the display area from the moving direction,
An information processing apparatus in which a program for outputting display coordinates of a mouse pointer is stored.   The information processing apparatus according to claim 8, wherein the appearance direction of the mouse pointer is held in a fixed direction while input information is continuously detected on the touch panel.   The information processing apparatus according to claim 8, wherein the appearance direction of the mouse pointer is continuously updated while input information is continuously detected on the touch panel.   The input direction of the mouse pointer is the direction held last time when the input information is no longer detected on the touch panel and the input information is detected again during a predetermined period. The information processing apparatus described.   The input direction of the mouse pointer is updated to an initial position when input information is no longer detected on the touch panel and input information is detected again after a predetermined period of time has elapsed. The information processing apparatus described.   The information processing apparatus according to claim 8, wherein an appearance direction of the mouse pointer is the same as a direction of the movement direction.   The appearance direction of the mouse point selects any one of four types of positions of upper right, lower right, upper left, and lower left of the input coordinates based on the direction of the moving direction. The information processing apparatus according to item.

Images