JP5275429B2 - Information processing apparatus, program, and pointing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize an information processor that has a multi-touch function capable of detecting touch or adjacency of an object at plural points and that can execute mouse emulation capable of being intuitively operated by a user. <P>SOLUTION: When a user touches a point displaying a cursor 33 on a touch panel 12 by a finger 52 and touches the touch panel 12 by other finger at the same time with the cursor 33 displayed on an LCD 11 of a PC, the PC performs processing according to a point touched by the other finger. When a finger 51 taps at the left of the touch point of the finger 52 on the display, a left-click event occurs. When the finger 51 taps at the right, a right-click event occurs. The PC causes a predetermined event that can be operated and input by mouse input corresponding to coordinate information of two touch points on the touch panel 12 by the user. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an information processing apparatus having a touch panel, and particularly to a method, a program, and an information processing apparatus for having a touch panel capable of detecting multi-touch and performing a pointing operation on a screen.

  In recent PCs (Personal Computers) and the like, touch panels have been developed that allow various operation inputs to the PC by directly touching the display surface of the display image. In normal touch panel operations, there are many cases of selecting by a tap action that touches the place you want to point on the display surface for a short time (equivalent to a left click when operating with a mouse). The cursor is often not displayed.

  However, there are many functions that the user can use when operating the PC with the mouse, such as a mouse-over function that displays information by placing the cursor on the icon without performing a predetermined selection operation, and a context menu displayed by right-clicking. Existing. Those functions cannot be input when the cursor is not displayed or when the touch panel is tapped, and the user often feels inconvenience.

  Therefore, an input control method that can emulate a mouse by an operation using a touch panel even in a pointing operation using the touch panel has been proposed (see Patent Document 1).

JP 2006-179006 A

  Although the input control method according to the above proposal emulates input by a mouse, the user cannot provide an intuitive operation feeling such as having to release his / her finger once when performing a series of operations.

  The present invention has been made in view of the above problems, and is an information processing apparatus having a multi-touch function capable of detecting contact or proximity of an object at a plurality of points, and is operated intuitively by a user. An object of the present invention is to realize an information processing apparatus, a program, and a pointing method that can execute mouse emulation.

The information processing apparatus according to the present invention detects display means for displaying image information on a display surface, and that an object is in contact with or close to the first and second detection locations on the display surface, and Coordinate information acquisition means for acquiring coordinate information of a second detection location, and when the first detection location is within a predetermined region on the display surface, the size of the contact point of the object on the display surface When the first and second detection locations are on the cursor and at least one of the first and second detection locations has moved, the first and second after the movement are displayed. Based on the second detection location, the cursor display means for enlarging, reducing or rotating the cursor to display on the display surface, and when the first detection location is on the cursor, the second detection location Depending on the coordinate information, When a predetermined event corresponding to the input operation is generated, the first detection location is on the cursor, and the first detection location is continuously moving, the first detection location is When the cursor is displayed at the display location to which it has moved, and the coordinate information of the second detection location indicates the left side when viewed from the side on which it is assumed that the user exists, rather than the coordinate information of the first detection location, When an event corresponding to the left click of the button in the mouse input operation is generated, and the coordinate information of the second detection location indicates the right side of the coordinate information of the first detection location, is characterized by comprising processing means Ru raised the event corresponding to the right-click.

Further, the program according to the present invention is a program executed in the information processing apparatus, displaying image information on the display surface, and having an object touching or approaching the first and second detection locations on the display surface simultaneously. , The coordinate information of the first and second detection locations is acquired, and when the first detection location is within a predetermined region on the display surface, the contact point of the object on the display surface A cursor larger than the size of the first detection point , the first detection point and the second detection point are on the cursor, and when at least one of the first and second detection points is moved, Based on the first and second detection locations, the cursor is enlarged, reduced or rotated to be displayed on the display surface, and when the first detection location is on the cursor, the coordinates of the second detection location Depending on the information, When a predetermined event corresponding to the input operation is generated, the first detection location is on the cursor, and the first detection location is continuously moving, the first detection location is When the cursor is displayed at the display location to which it has moved, and the coordinate information of the second detection location indicates the left side when viewed from the side on which it is assumed that the user exists, rather than the coordinate information of the first detection location, When an event corresponding to the left click of the button in the mouse input operation is generated, and the coordinate information of the second detection location indicates the right side of the coordinate information of the first detection location, It is characterized in that to execute Rukoto caused the event corresponding to the right-click on the information processing apparatus.

The pointing method according to the present invention is a pointing method for an information processing apparatus, displaying image information on a display surface, and an object touching or approaching the first and second detection locations on the display surface simultaneously. And the coordinate information of the first and second detection points is acquired, and when the first detection point is within a predetermined region on the display surface, the object touches the display surface. When a cursor larger than the size of the point is displayed, the first and second detection locations are on the cursor, and at least one of the first and second detection locations has moved, Based on the first and second detection locations, the cursor is enlarged, reduced or rotated and displayed on the display surface. When the first detection location is on the cursor, the second detection location Depending on the coordinate information When a predetermined event corresponding to a mouse input operation is generated, the first detection location is on the cursor, and the first detection location is continuously moving, the first detection location When the cursor is displayed at the display location to which the user has moved, and the coordinate information of the second detection location indicates the left side as viewed from the side on which it is assumed that the user exists, rather than the coordinate information of the first detection location When an event corresponding to the left click of the button in the mouse input operation is generated, and the coordinate information of the second detection location indicates the right side of the coordinate information of the first detection location, the button in the mouse input operation is characterized in Rukoto caused the event to be equivalent to the right click.

Information processing apparatus having a multi-touch function capable of detecting contact or proximity of an object at a plurality of points, and capable of executing mouse emulation that can be operated intuitively by a user, and a program And a pointing method can be realized.

The perspective view which shows an example of the external appearance of PC in one Embodiment of this invention. The block diagram which shows an example of the hardware constitutions of PC in one Embodiment of this invention. The figure which shows an example of the cursor display method in one Embodiment of this invention. The figure which shows an example of the cursor display method in one Embodiment of this invention. The figure which shows an example of the mouse | mouth emulation input method in one Embodiment of this invention. The figure which shows an example of the display of LCD at the time of mouse-over function display in one Embodiment of this invention. The figure which shows an example of the implementation | achievement method of the cursor display change function in one Embodiment of this invention. The functional block diagram which shows an example of the functional block of PC in one Embodiment of this invention. The figure which shows an example of the input discrimination | determination method of the click discrimination | determination part in one Embodiment of this invention, and the process based on each discrimination | determination. The flowchart which shows an example of the processing flow in one Embodiment of this invention. The flowchart which shows an example of the cursor display phase in one Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a perspective view showing an example of an overview of the PC 10 in the present embodiment. FIG. 1 shows a PC 10, LCD 11, touch panel 12, keyboard 13, and touch pad 14.

  The PC 10 is an information processing apparatus that performs various calculation processes in accordance with user instructions. The PC 10 also has a touch panel function that detects contact or proximity of an object to the display surface and performs various processes using the detection result. In this embodiment, a PC is illustrated as an example of an information processing apparatus, but the present invention is not limited to this, and the present invention can be applied to various information processing apparatuses such as a PDA.

  An LCD 11 (Liquid Crystal Display) is a liquid crystal display, and is a display device having a function as display means for displaying image information output from the PC 10 to a user. In the present embodiment, the liquid crystal display is exemplified, but any display device capable of displaying image information may be used, and various other display devices such as a plasma display may be considered.

  The touch panel 12 is provided on the surface of the LCD 11 and has a function of detecting contact or proximity of an object and inputting detection information to the PC 10 as an electrical signal. In addition, the touch panel 12 can detect multi-touch that is contact or proximity of two objects at the same time. Here, contact or proximity of an object to the touch panel 12 at two locations simultaneously is referred to as multi-touch, and contact or proximity of the object to the touch panel 12 at only one location is referred to as single touch. The touch panel 12 is transparent, and the user can view the display on the LCD 11 through the transparent touch panel 12.

  The keyboard 13 has a function of detecting a user's key press and inputting detection information to the PC 10 as an electrical signal.

  The touch pad 14 has a function of detecting movement when the user traces the surface with a finger and inputting detection information to the PC 10 as an electrical signal.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the PC 10 in the present embodiment. FIG. 2 shows the PC 10, CPU 21, ROM 22, RAM 23, HDD 24, display control unit 25, LCD 11, touch panel 12, input control unit 26, keyboard 13, touch pad 14, and bus 27.

  The CPU 21 is a central processing unit, and controls the entire PC 10. It also has a function of executing various programs and executing predetermined processing according to the programs.

  The ROM 22 is constituted by a semiconductor memory and stores a program executed by the CPU 21. The ROM 22 has an OS (Operation System) that is a program for controlling the entire system of the PC 10.

  The RAM 23 is composed of a semiconductor memory, and is used as a program and data storage area when the CPU 21 processes the program.

  The HDD 24 is a magnetic disk device, for example, and is used as a non-volatile area for storing data of the PC 10. The stored program and data can be read out by an instruction from the CPU 21. The HDD 24 also has a touch input application program that is a program for controlling multi-touch input in the present embodiment. This touch input application program will be described later.

  The display control unit 25 is a semiconductor chip for image processing, and has a function of drawing an image in accordance with a drawing instruction from the CPU 21 or the like and outputting it to the LCD 11.

  The input control unit 26 is a controller that controls user input information from the touch panel 12, the keyboard 13, and the touch pad 14 and inputs the information to the PC 10.

  The bus 27 connects the modules in the PC 10 to be connected so that they can communicate with each other.

  FIG. 3 is a diagram showing an example of a cursor display method in the present embodiment. FIG. 3 shows the PC 10, LCD 11, touch panel 12, finger 31, finger 32, cursor 33, and folder 34.

  The finger 31 and the finger 32 are fingers of a user who is operating the PC 10. At this time, the user can give various instructions to the PC 10 by touching the touch panel 12 with both the finger 31 and the finger 32.

  A cursor 33 is a selection display for selecting an object on the display surface of the LCD 11. The user can move the cursor 33 and point to select an object that exists at the tip of the arrow of the cursor 33. The PC 10 in this embodiment displays the size of the cursor 33 larger than the size of the contact point on the touch panel 12 with a finger. When the cursor 33 is displayed in such a large size, when the user is in contact with the cursor 33 with a finger, the cursor 33 is not invisible by the finger, and the user always performs an operation input while visually checking the cursor 33. be able to.

  On the other hand, when the cursor 33 is not displayed, for example, an object smaller than the contact point of the user's finger on the touch panel 12 becomes difficult to see with the finger when selected. For this reason, it is difficult for the user to determine whether or not the target object has been correctly selected. However, if the cursor 33 is configured to be displayed in a large size in this way, the selection is performed by the tip of the arrow of the cursor 33, so that the user can easily see the object to be selected and can easily select even a small object. it can.

  The folder 34 is a display object that indicates a storage destination of data or the like used by the user. By selecting the folder 34, the user can refer to the storage destination of the data he wants to use.

  In the present embodiment, the cursor 33 is not displayed on the LCD 11 in the initial state. In this state, when the user taps an object to be selected with a finger, the tapped object is selected. Here, the tap refers to an operation of touching the touch panel 12 for a relatively short predetermined time. Further, there is an operation input called dragging as an operation different from the tap. The drag is an input operation that keeps touching the touch panel 12 for a longer time than the predetermined time.

  In the present embodiment, when the user taps (multi-touch) simultaneously with two fingers 31 and 32 on the touch panel 21 as shown in FIG. 3, the PC 10 displays the cursor 33 on the LCD 11. Here, the PC 10 displays the cursor 33 at the midpoint between the two touch detection points on the touch panel 12 (the touch points of the finger 31 and the finger 32). As described above, in the present embodiment, the cursor 33 is displayed at the midpoint between the two contact detection points. However, this is merely an example, and the display position is not limited thereto. As an example of a place where the cursor 33 by multi-touch contact is displayed, for example, the cursor 33 may be displayed at a point where one of the fingers touches. For example, the LCD 11 may touch any point on the touch panel 12. An initial value that is displayed at the center (or lower right) may be provided. Further, in the present embodiment, the cursor 33 is not displayed unless it receives an operation input from the user using the cursor 33 for a predetermined time or longer (described later). When the cursor 33 is displayed again after the cursor 33 is hidden, the cursor 33 may be displayed at the place where the cursor 33 is displayed immediately before the cursor 33 is hidden when the cursor 33 is displayed next time. A combination of the above is also conceivable.

  FIG. 4 is a diagram showing an example of a cursor display method in the present embodiment. FIG. 4 shows the PC 10, LCD 11, touch panel 12, folder 34, finger 41, and cursor display area 42. Here, an example of a display method different from the display method of the cursor 33 shown in FIG. 3 will be described.

  The cursor display area 42 is an area in which the PC 10 displays the cursor 33 on the display of the LCD 11 when the user performs an operation input by tapping with the finger 41.

  In FIG. 3, the cursor 33 is displayed from non-display by multi-touch to the user, but the cursor 33 may be displayed by tapping the cursor display area 42 as shown in FIG. This cursor display area 42 is always displayed when the cursor 33 is not displayed. When the cursor 33 is displayed by the user tapping on the cursor display area 42, the cursor display area 42 may be displayed as it is or may be hidden. If the cursor display area 42 is not displayed when the cursor 33 is displayed, unnecessary display is eliminated and the display on the LCD 11 is easy to see. If the cursor display area 42 is displayed when the cursor 33 is displayed, the cursor 33 may be hidden if the user taps the cursor display area 42 when the cursor 33 is displayed. At this time, the cursor display area 42 is a user operation input area for switching the display / non-display of the cursor 33 by a toggle. In the present embodiment, the cursor display area 42 is illustrated as being displayed at the lower right on the screen of the LCD 11, but the display position may be freely designed as appropriate.

  The input for switching the display / non-display of the cursor 33 may be a press of a predetermined key on the keyboard 13 (or simultaneous press of two predetermined keys, etc.), or a dedicated button may be provided on the PC 10. An example of inputting by pressing a dedicated button is also conceivable.

  FIG. 5 is a diagram showing an example of a mouse emulation input method according to this embodiment. FIG. 5 shows the PC 10, LCD 11, touch panel 12, cursor 33, folder 34, finger 51, finger 52, and finger 53.

  When the user moves the finger 52 while dragging on the cursor 33 displayed as shown in FIG. 5 (while continuing to touch the display of the cursor 33 for a predetermined time or longer), the user moves along the movement of the finger 52 ( The cursor 33 moves in the same movement direction as the finger 52.

  Further, when the user taps the left side of the finger 52 (the left side of the broken line on the screen) with another finger, the user can perform a left click for mouse input. When the user taps the right side of the finger 52 (the right side of the broken line on the screen) with another finger, the user can right click on the mouse input. That is, a left click event occurs when the user taps the left side of the finger 52, and a right click event occurs when the user taps the right side.

  With this configuration, the PC 10 can perform mouse emulation that can be intuitively operated by the user. When using the mouse, the user usually performs a click operation with two fingers. In the present embodiment, the touch of two fingers can be detected simultaneously by the multi-touch function, so that unlike a single touch input, the user touches the touch panel with a feeling similar to that of using a normal mouse using a plurality of fingers. The operation input to the PC 10 by 12 can be performed.

  At this time, for example, if the user performs a tap input twice within a predetermined time on the left side of the broken line in FIG.

  In addition, by moving the finger 52 while the finger 51 is in contact with the left side of the touch panel 12 from the finger 52, it is possible to easily set the range of the selected region. At this time, if the finger 51 touches the touch panel 12, the point on the display surface where the tip of the arrow of the cursor 33 exists when the contact with the finger 51 starts, and the point where the tip of the arrow of the cursor 33 exists when the contact ends. A rectangular area having a diagonal point is a selection area. When setting the range of the selected area in the operation input on the touch panel, a procedure for releasing the finger once is necessary in single touch, and it is difficult to set the range of the selected area as an intuitive operation. However, the PC 10 according to the present embodiment can intuitively set the range of the selection region without taking steps such as releasing the finger once.

  FIG. 6 is a diagram showing an example of display on the LCD 11 when the mouse over function is exhibited in the present embodiment. The PC 10, LCD 11, touch panel 12, cursor 33, folder 34, finger 61, and tool tip 62 are shown.

  In this embodiment, when the cursor 33 is displayed, if the cursor 33 is moved by a drag operation of the finger 61 and the cursor 33 is stopped on an object such as the folder 34 for a certain time, the PC 10 exhibits a mouse-over function. Here, the mouse-over function is a function in which the PC 10 performs a predetermined process when the user stops the cursor 33 on an object such as the folder 34 without selecting an object by left clicking or the like. In the present embodiment, when the cursor 33 is kept stationary for a certain period of time without performing a selection process on the folder 34, the PC 10 displays information related to the folder 34 on the tool tip 62 by the mouse over function. The information displayed in FIG. 6 is a folder size and a folder name. In FIG. 6, the folder 34 has a folder name “patent data”, and the size of the folder is 500 KB. In the present embodiment, the mouse-over function will be described by exemplifying the above-described processing (the information of the folder 34 is displayed by the tool tip 62 when the cursor 33 is stopped on the folder 34). However, the present invention is not limited to this. In addition, as the mouse-over function, there are various processes such as emphasizing the outline of the object and making the object currently pointed by the arrow of the cursor 33 stand out.

  At this time, when the cursor 33 is released from the folder 34, the tool tip 62 is not displayed. In addition, the tool tip 62 remains displayed even when the user stops touching the touch panel 12 while the cursor 33 exists on the display of the folder 34 and releases the finger from the touch panel 12. These are examples of processing performed by the PC 10 after the tooltip 62 is displayed, and are not limited thereto.

  Thus, in this embodiment, since the cursor 33 is displayed in the touch panel input method, the mouse-over function can be exhibited.

  FIG. 7 is a diagram showing an example of a method for realizing the cursor display change function in the present embodiment. FIG. 7 shows the PC 10, LCD 11, touch panel 12, cursor 33, folder 34, finger 71, and finger 72.

  In the present embodiment, the cursor display change function refers to a function of the PC 10 that rotates or enlarges / reduces the cursor 33 in accordance with a predetermined operation input from the user.

  In the present embodiment, as shown in FIG. 7A, when the user touches two fingers on the display of the cursor 33 on the touch panel 12 and rotates the finger in the direction of the arrow in the figure, accordingly, FIG. As shown in B), the cursor 33 rotates in the direction of the arrow.

  At this time, the cursor 33 is redrawn after being rotated so as to maintain the positional relationship with the contact points of the finger 71 and the finger 72. That is, the point of the cursor 33 in contact with the finger 71 moves as the finger 71 moves, and the point of the cursor 33 in contact with the finger 72 moves as the finger 72 moves. The cursor 33 is rotated or enlarged / reduced so that the shape (the aspect ratio of the cursor 33 shape, etc.) does not change by the movement of each of the two points, and the shape is maintained. At this time, the PC 10 stores the contact point on the cursor 33 and matches the point on the touch panel 11 moved by the drag operation with the contact point on the previous cursor 33. 33 drawing processes are performed. With such a configuration, the user can rotate and enlarge / reduce the cursor 33 by moving two fingers while bringing the two fingers into contact with the display of the cursor 33. These may be realized in combination with movement.

  Especially when the cursor 33 is displayed in a large size as in the present embodiment, it is difficult to align the tip of the arrow of the cursor 33 with the end of the LCD 11 which is the opposite direction of the direction in which the arrow is pointing. This is because the finger protrudes from the screen of the LCD 11 and cannot be detected by the touch panel 12 when trying to match the tip of the arrow. Therefore, if the cursor 33 can be rotated as in the present embodiment, the direction of the arrow can be changed as appropriate, so the number of cases where it becomes difficult to align the tip of the arrow of the cursor 33 with the corner of the screen is reduced. It will be easy. In the present embodiment, the user can change the size of the cursor 33 as appropriate, and the user can use the cursor 33 having a suitable size according to the usage method of the cursor 33, so that convenience for the user is improved.

  FIG. 8 is a functional block diagram showing an example of functional blocks of the PC 10 in the present embodiment. FIG. 8 shows the LCD 11, the touch panel 12, the input control unit 26, the touch input application 81, the event reception unit 82, the touch detection unit 83, the click determination unit 84, the event transmission unit 85, the cursor display unit 86, and the OS 87. Yes.

  The touch input application 81 is an application for executing a touch input application program stored in the HDD 24. The touch input application 81 is a processing unit having a function of realizing a touch input by controlling a user input by the touch panel 12. The multi-touch application 81 has a function of performing various processes based on the signal input from the input control unit 26 and outputting various signals to the OS 87 and the display control unit 25. The touch input application 81 includes an event reception unit 82, a touch detection unit 83, a click determination unit 84, an event transmission unit 85, and a cursor display unit 86.

  The input control unit 26 detects the contact or proximity of an object such as a finger with the touch panel 12, acquires an electrical signal generated by the detection, and outputs the electrical signal to the touch detection unit 83 of the event reception unit 82. It has a function. As described above, the input control unit 26 can acquire information on contact or proximity (multi-touch) of an object to the touch panel 12 at two locations simultaneously.

  The event receiving unit 82 has a function of acquiring user operation input to the touch panel 12 as various event information and outputting various instructions to the event transmitting unit 85 and the cursor display unit 86.

  The touch detection unit 83 has a function of calculating the contact or proximity of an object as coordinate information on the touch panel 12 based on an electrical signal input from the multi-touch driver 81. The touch detection unit 83 outputs the calculated coordinate information of contact or proximity of the object to the click determination unit 84 as needed. The touch panel 12 and the touch detection unit 83 function as coordinate information acquisition means.

  The click discriminating unit 84 performs various discriminations based on the coordinate information of the contact or proximity of the object on the touch panel 12 acquired from the touch detection unit 83, and the event transmission unit 85, the cursor so as to perform processing based on the discrimination result. It has a function of giving various instructions to the display unit 86. Here, the items to be determined by the click determination unit 84 include whether the touch or proximity of the object to the touch panel 12 is a single touch / multi-touch, whether the input to the touch panel 12 is a tap or a drag input, and the acquired object There are items such as whether or not the coordinate information of the contact or the proximity point is on the display of the cursor 33 on the LCD 11. Whether the input to the touch panel 12 is a tap input or a drag input is determined by the click determination unit 84 using a timer (not shown) that the click determination unit 84 has. Further, the determination as to whether or not the coordinate information of the contact of the object or the proximity location is on the display of the cursor 33 will be described later. The click determination unit 84 has a function of performing the above determination and performing processing based on the combination of the determinations. The combination and processing based on the combination will be described later with reference to FIG.

  The event transmission unit 85 has a function of transmitting the processing instruction received from the click determination unit 84 to the OS 87.

  The cursor display unit 86 has a function of performing predetermined processing based on processing instructions and coordinate information received from the click determination unit 84. From the click discriminating unit 84, the drawing instruction of the cursor 33 and the coordinate information of the contact or proximity of the object on the touch panel 12 are transmitted to the cursor display unit 86. The cursor display unit 86 has a function as a cursor display unit that creates the shape of the cursor 33 in accordance with an instruction from the click determination unit 84 and causes the display control unit 25 to draw the cursor 33. In addition, the cursor display unit 86 transmits the shape and position information of the created cursor 33 to the click determination unit 84. Further, the cursor display unit 86 has a function of transmitting position information on the display surface of the LCD 11 at the tip of the arrow of the cursor 33 to the OS 87.

  The OS 87 is an application that controls the entire PC 10. The OS 87 operates in the same manner as when the user performs an operation input using the mouse or the touch pad 14 except that the cursor is not displayed when the user performs the operation input using the touch panel 12. At this time, the OS 87 receives coordinate information of the tip of the arrow of the cursor 33 from the cursor display unit 86, and when a click operation is performed by the user, the OS 87 clicks using the coordinate information of the tip of the arrow of the cursor 33. Specify where to select. Further, when the coordinate information of the tip of the arrow of the cursor 33 exists on a predetermined object, the OS 87 exhibits a mouse-over function.

  The display control unit 25 has a function of creating an image of the cursor 33 in accordance with a drawing instruction of the cursor 33 received from the cursor display unit 86 and displaying the image on the LCD 11.

  FIG. 9 is a diagram illustrating an example of the input recognition method of the click determination unit 84 and processing recognized by the PC 10 corresponding to each determination result in the present embodiment. FIG. 9 shows a discrimination table 900.

  The click discriminating unit 84 discriminates the user's input operation based on the coordinate information of the contact or proximity of the object on the touch panel 12 acquired from the touch detection unit 83 and the coordinate information of the display position of the cursor 33 acquired from the cursor display unit 86. Then, various processes based on each determination result are performed. In the cursor display phase in the processing flow in the present embodiment to be described later, the click determination unit 84 performs determination of user input operations and various processes with reference to the determination table 900 of FIG.

  The coordinate information of the contact location (the contact or proximity location of the object will be referred to as a contact location for the sake of simplicity) acquired from the touch detection unit 83 in the top row column (column 901, column 902) of the discrimination table 900. Is included in the coordinate information on the cursor 33 that is acquired from the cursor display unit 86. The status (including the cursor / not including the cursor) is shown. If the second column from the top (column 903, column 904) includes the cursor, and if the input is by multi-touch, the coordinate information of the multi-touch contact location is only on the cursor. The status (one / both) is shown as to whether or not both are on the cursor. In the leftmost column (column 905, column 906) of the discrimination table 900, the status (multi-touch / single touch) indicating whether the input is multi-touch or single touch is shown. In the second row from the left (column 907 to column 910), the status (tap / drag) indicating whether each input is a tap input or a drag input is shown.

  The click determination unit 84 determines processing corresponding to the status of these inputs. For example, when the input is a drag input by single touch and the coordinate information of the contact location is included in the coordinate information displaying the cursor 33, a movement process (column 919) of the cursor 33 is performed. The movement of the cursor 33 at this time is the movement along the movement of the contact location (the same movement amount in the same movement direction as the contact location) as described above with reference to FIG.

  Here, each column of the discrimination table 900 will be described.

  A column 911 shows a process in the case of input by multi-touch with a tap, and only one contact point exists on the cursor 33. At this time, it is determined whether a left-click event or a right-click event is generated according to the positional relationship between the contact location on the cursor 33 and the contact location not on the cursor 33 (the contact location not on the cursor 33 is determined from the contact location on the cursor 33). If the left side is the left click, and if the contact point that is not the cursor 33 is the right side of the contact point on the cursor 33, the determined event information is transmitted to the OS 87 via the event transmission unit 85.

  A column 912 shows a process in the case of multi-touch input by tapping and both contact points are present on the cursor 33. At this time, the click determination unit 84 does not perform any particular processing.

  A column 913 shows a process in the case where the input is performed by multi-touch with a tap, and neither of the contact points is included in the cursor 33. At this time, the click determination unit 84 instructs the cursor display unit 86 not to display the cursor 33.

  A column 914 shows a process in the case of multi-touch input by dragging and only one contact point exists on the cursor 33. At this time, each click event is generated in the same manner as the processing in the column 911. At this time, the click is in a state where the click is kept pressed, and the click determination unit 84 transmits information indicating that the click is kept pressed to the OS 87. Accordingly, the selection range can be set in the area described with reference to FIG. Further, for example, when a drawing application is executed, input of a handwritten drawing line of the user may be performed.

  A column 915 shows a process in the case where the input is performed by multi-touch by dragging, and both contact points exist on the cursor 33. At this time, the click determination unit 84 performs the cursor display change process shown in FIG. The click discriminating unit 84 transmits the coordinate information of each touch location of multi-touch to the cursor display unit 86. Using the received coordinate information, the cursor display unit 86 reconstructs the image of the cursor 33 based on the movement of each contact location as described above, thereby displaying the rotation, enlargement / reduction, etc. of the cursor 33. .

  A column 916 shows a process in the case where the input is performed by multi-touch by dragging, and neither of the contact points is included in the cursor 33. At this time, the click determination unit 84 performs no processing.

  A column 917 shows a process in the case where the input is a single touch with a tap and the contact location is on the cursor 33. At this time, the click determination unit 84 has a function of generating a left click event on the assumption that the left click has been performed, and transmitting the generation information of the left click event to the OS 87 via the event transmission unit 85. At this time, the tip of the arrow of the cursor 33 is a selected location.

  A column 918 is an input by a single touch with a tap, and shows processing when the contact point is not on the cursor 33. At this time, the click determination part 84 moves the display position of the cursor 33 to the contact location at that time. The click discriminating unit 84 transmits the coordinate information of the contact location on the touch panel 12 to the cursor display unit 86, and transmits an instruction to the cursor display unit 86 to display the cursor 33 with the coordinates as the tip of the arrow. . The cursor display unit 86 creates the shape of the cursor 33 in accordance with the instruction, draws it on the display control unit 25, and displays the moved cursor 33 on the LCD 11. The cursor display unit 86 transmits the coordinate information at the tip of the arrow of the cursor 33 to the click determination unit 84 at that time. With this configuration, the user can easily display the cursor 33 at a suitable position without having to trace the touch panel 12 with a finger.

  A column 919 shows a process when the input is a single touch by dragging and the contact location is on the cursor 33. At this time, the click determination unit 84 transmits the coordinate information of the contact location to the cursor display unit 86, and the cursor display unit 86 creates and displays an image of the cursor 33 so as to move the cursor 33 along the contact location. .

  A column 920 shows a process when the input is a single touch by dragging and the contact location is not on the cursor 33. At this time, the click determination unit 84 transmits the coordinate information of the contact location to the cursor display unit 86, the cursor display unit 86 displays the cursor 33 at the contact location, and the cursor 33 moves along with the movement of the contact location. To be displayed.

  FIG. 10 is a flowchart showing an example of a processing flow in the present embodiment. The processing flow in FIG. 10 starts from a state where the cursor 33 is not displayed.

  First, the PC 10 determines whether or not the touch or proximity of an object is detected on the touch panel 12 (S101). When the touch or proximity of the object is not detected on the touch panel 12 (No), the processing flow returns to S101 again. When the touch or proximity of the object is detected on the touch panel 12 in S101 (Yes), the input control unit 26 transmits the detection result of the touch panel 12 to the touch detection unit 83 as an electrical signal, and the touch detection unit 83 receives the received signal. The coordinate information of the detected part is acquired from the electrical signal, and the coordinate information is transmitted to the click determination unit 84. The click determination unit 84 that has received the coordinate information of the detection location determines whether the touch or proximity of the object on the touch panel 12 is a single touch of only one location or multi-touch of 2 locations (S102). When it is determined in S102 that there is only one contact or proximity location on the touch panel 12 (No), the click determination unit 84 instructs the event transmission unit 85 to transmit the left click event occurrence information to the OS 87. I do. Upon receiving the instruction, the event transmission unit 85 transmits left click event occurrence information to the OS 87, and the OS 87 that has transmitted the information performs processing when a left click is performed (S103).

  If it is determined in S102 that the click determination unit 84 is multi-touch (Yes), the coordinate information of the contact location and the display instruction of the cursor 33 are given to the cursor display unit 86. The cursor display unit 86 creates the shape of the cursor 33 based on the received instruction and coordinate information. At this time, the position where the cursor 33 is displayed may be anywhere on the LCD 11 as described above, and the cursor display unit 86 transmits the coordinate information of the cursor 33 to be displayed to the click determination unit 84. The cursor display unit 86 that has created the shape of the cursor 33 generates an image of the cursor 33 to the display control unit 25 and instructs the LCD 11 to display the image, and the cursor 33 is displayed on the LCD 11 (S104). . In addition, the cursor display unit 86 transmits coordinate information of the tip of the arrow of the cursor 33 to the OS 87. When S104 ends, the processing flow moves to the cursor display phase (S105). The cursor display phase will be described later with reference to FIG. When the cursor display phase (S105) ends, the click determination unit 84 transmits an instruction to hide the cursor 33 to the cursor display unit 86, and the cursor display unit 86 that has received the instruction interrupts the cursor display processing. The cursor 33 is not displayed (S106). When S103 and S106 are finished, the processing flow is finished.

  Here, as described with reference to FIG. 4, when the cursor 33 is displayed by providing the cursor display area 42, the determination in S102 is "Is the contact detection location within the cursor display area 42?" If it is within the area 42 (Yes), the process flow proceeds to S103 if not proceeding to S104.

  FIG. 11 is a flowchart showing an example of the cursor display phase in the present embodiment. Hereinafter, processing in the cursor display phase of the processing flow shown in FIG. 10 will be described with reference to FIG.

  First, in a state where the cursor 33 is displayed, the click determination unit 84 first determines whether or not an object has touched or approached the touch panel 12 within a predetermined time (S111). Here, the predetermined time in the present embodiment is, for example, 30 seconds, but this is merely an example in the present embodiment. If contact or proximity of an object is detected within 30 seconds in S111 (Yes), the click determination unit 84 uses the coordinate information on the touch panel 12 of the contact location and the coordinate information displayed on the cursor 33 in FIG. The user's input operation is discriminated while referring to the table shown, and the process corresponding to the discrimination result is determined (S112). Depending on the processing determined by the click determination unit 84, the click determination unit 84, the event transmission unit 85, the cursor display unit 86, the OS 87, the display control unit 25, and the like perform the various processes described above with reference to FIG. 9 (S113). . When S113 ends, the process flow returns to S111. When the click determination unit 84 does not detect contact or proximity of an object within 30 seconds in S111 (No), the cursor display phase ends, and the processing flow proceeds to S106 in FIG.

  In the case of the PC 10 in this embodiment, when the contact or proximity of an object at a plurality of locations on the display surface is detected, a mouse operation corresponding to each detected location can be emulated, thereby allowing the user to operate intuitively. It is possible to realize an information processing apparatus that can do this.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

10: PC
11: LCD
12: Touch panel 13: Keyboard 14: Touchpad 21: CPU
22: ROM
23: RAM
24: HDD
25: Display control unit 26: Input control unit 27: Bus 31, 32: Finger 33: Cursor 34: Folder 41: Finger 42: Cursor display area 51-53: Finger 61: Finger 62: Tool tip 71, 72: Finger 81 : Touch input application 82: Event reception unit 83: Touch detection unit 84: Click discrimination unit 85: Event transmission unit 86: Cursor display unit 87: OS
900: Discrimination tables 901-920: Field

Claims (6)

Display means for displaying image information on the display surface;
Coordinate information acquisition means for detecting that an object has simultaneously contacted or approached the first and second detection locations on the display surface, and acquires coordinate information of the first and second detection locations;
When the first detection location is within a predetermined area on the display surface, a cursor larger than the size of the contact point of the object is displayed on the display surface, and the first and second detection locations are When the cursor is on the cursor and at least one of the first and second detection points moves, the cursor is enlarged, reduced or rotated based on the first and second detection points after the movement. Cursor display means for displaying on the display surface ;
When the first detection location is on the cursor, a predetermined event corresponding to a mouse input operation is generated according to the coordinate information of the second detection location, and the first detection location is the cursor. When the first detection location is on the top and is continuously moving, the cursor is displayed at the display location to which the first detection location has moved, and the coordinate information of the second detection location Indicates the left side when viewed from the side on which it is assumed that the user exists rather than the coordinate information of the first detection location, an event corresponding to the left click of the button in the mouse input operation is generated, and the second detection when the coordinate information of the point indicates the right side of the coordinate information of the first detecting portion, and characterized by including a processing means for Ru raised the event corresponding to the right-click button of the input operation of the mouse That the information processing apparatus. The cursor display means detects the first detection point when the first and second detection points are on the cursor and the first and second detection points move to the third and fourth detection points, respectively. A first point on the cursor corresponding to a location matches the third detection location, and a second point on the cursor corresponding to the second detection location matches the fourth detection location The information processing apparatus according to claim 1, wherein the cursor is enlarged, reduced, or rotated for display. The cursor display means further includes a switching means for switching between display and non-display of the cursor on the display surface,
The processing means includes
When the cursor display means does not display the cursor, and when an object is in contact with or close to two places on the display surface at the same time, or within two hours on the display surface within a predetermined time The information processing apparatus according to claim 1, wherein the cursor is displayed when touching or approaching. 2. The information according to claim 1, further comprising mouse over means for displaying predetermined information corresponding to the predetermined position when the cursor is at a predetermined position on the display surface for a predetermined time or more. Processing equipment. A program executed in the information processing apparatus,
Display image information on the display surface,
Detecting that the object is in contact with or close to the first and second detection points on the display surface at the same time, and obtaining coordinate information of the first and second detection points;
When the first detection location is within a predetermined area on the display surface, a cursor larger than the size of the contact point of the object is displayed on the display surface, and the first and second detection locations are When the cursor is on the cursor and at least one of the first and second detection points moves, the cursor is enlarged, reduced or rotated based on the first and second detection points after the movement. Displayed on the display surface,
When the first detection location is on the cursor, a predetermined event corresponding to a mouse input operation is generated according to the coordinate information of the second detection location,
When the first detection location is on the cursor and the first detection location is continuously moving, the cursor is displayed at the display location to which the first detection location has moved , An event corresponding to a left click of a button in a mouse input operation when the coordinate information of the second detection location indicates the left side as viewed from the side on which the user is assumed to be present than the coordinate information of the first detection location to generate, when the coordinate information of the second detection portion is shown to the right than the coordinate information of the first detection portion, wherein the Rukoto caused the event corresponding to the right-click button of the input operation of the mouse A program that is executed by an information processing apparatus. A pointing method for an information processing device, comprising:
Display image information on the display surface,
Detecting that the object is in contact with or close to the first and second detection points on the display surface at the same time, and obtaining coordinate information of the first and second detection points;
When the first detection location is within a predetermined area on the display surface, a cursor larger than the size of the contact point of the object is displayed on the display surface, and the first and second detection locations are When the cursor is on the cursor and at least one of the first and second detection points moves, the cursor is enlarged, reduced or rotated based on the first and second detection points after the movement. Displayed on the display surface,
When the first detection location is on the cursor, a predetermined event corresponding to a mouse input operation is generated according to the coordinate information of the second detection location,
When the first detection location is on the cursor and the first detection location is continuously moving, the cursor is displayed at the display location to which the first detection location has moved , An event corresponding to a left click of a button in a mouse input operation when the coordinate information of the second detection location indicates the left side as viewed from the side on which the user is assumed to be present than the coordinate information of the first detection location to generate, when the coordinate information of the second detection portion is shown to the right than the coordinate information of the first detection portion, wherein Rukoto caused the event corresponding to the right-click button in a mouse of the input operation Pointing method.

Images